CN109741144A

CN109741144A - Commodity checking method, device, host and equipment

Info

Publication number: CN109741144A
Application number: CN201910009098.XA
Authority: CN
Inventors: 魏秀参; 崔权; 杨磊
Original assignee: Xuzhou Kuang Shi Data Technology Co Ltd; Nanjing Kuanyun Technology Co Ltd; Beijing Megvii Technology Co Ltd
Current assignee: Xuzhou Kuang Shi Data Technology Co Ltd; Nanjing Kuanyun Technology Co Ltd; Beijing Megvii Technology Co Ltd
Priority date: 2019-01-04
Filing date: 2019-01-04
Publication date: 2019-05-10
Anticipated expiration: 2039-01-04
Also published as: CN109741144B

Abstract

This application involves commodity retail technical field, a kind of commodity checking method, device, host and equipment are provided.Wherein, commodity checking method includes: to obtain the veritification image comprising commodity to be veritified；The commodity to be veritified in image are veritified using the target detection model inspection of pre-training, and generate the inventory of commodity to be veritified according to testing result；Obtain the inventory of received payment commodity；The inventory of commodity to be veritified and the inventory of received payment commodity are compared, if the type of commodity and the quantity of every kind of commodity are consistent in two inventories, it is determined that commodity to be veritified are consistent with received payment commodity.Above-mentioned commodity checking method is based on algorithm of target detection and executes automatically, is not necessarily to manpower intervention, effectively saves cost of human resources, avoid the mistake in verification procedure, meet the unmanned development trend of retail trade.

Description

Commodity checking method, device, host and equipment

Technical field

This application involves commodity retail technical field, in particular to a kind of commodity checking method, device, host and Equipment.

Background technique

Unmanned is one of the important trend of current commodity retail industry, and some shopping places have begun support and care for Visitor independently pays the bill in shopping process, and the commodity for then veritifying its carrying again when customer leaves shopping place are and received payment quotient Whether product are consistent, and customer is avoided to take away the commodity for forgetting payment.However, in the prior art, this verification procedure is still by people Work is responsible for, and not only increases human cost, also runs in the opposite direction with unmanned retail theory.

Summary of the invention

In view of this, the embodiment of the present application provides a kind of commodity checking method, device, host and equipment, using being based on mesh The method of mark detection carries out commodity veritification automatically, improves the problem of commodity verification procedure needs manpower intervention.

To achieve the above object, the embodiment of the present application provides the following technical solutions:

In a first aspect, a kind of commodity checking method of the embodiment of the present application, comprising:

Obtain the veritification image comprising commodity to be veritified；

Commodity to be veritified described in veritification image described in target detection model inspection using pre-training, and according to detection As a result the inventory of the commodity to be veritified is generated, wherein the target detection model is the nerve net that algorithm of target detection uses Network model, the inventory of the commodity to be veritified include the type of commodity to be veritified and the quantity of every kind of commodity to be veritified；

Obtain received payment commodity inventory, the inventory of the received payment commodity include received payment commodity type and every kind The quantity of received payment commodity；

The inventory of the commodity to be veritified and the inventory of the received payment commodity are compared, if commodity in two inventories Type and every kind of commodity quantity it is consistent, it is determined that the commodity to be veritified are consistent with the received payment commodity.

The above method generates the inventory of commodity to be veritified according to the result of target detection, and verifies the inventory of commodity to be veritified Whether content is consistent with the inventory content of received payment commodity, and then determines whether commodity to be veritified have paid the bill.Entire commodity core It tests process to execute automatically based on algorithm of target detection, is not necessarily to manpower intervention, effectively saves cost of human resources, avoid veritification Mistake in the process meets the unmanned development trend of retail trade.

In some embodiments, described in the veritification image described in the target detection model inspection using pre-training Wait before veritifying commodity, the method also includes:

Using the single-item image and/or the combination image training target detection model of pre-acquired, and utilize the combination Target detection model described in image measurement, wherein the single-item image refers to the image for the sole product only concentrated comprising commodity, The constitutional diagram seems the combined image for referring to a variety of commodity concentrated comprising the commodity, and the commodity collection refers to for training institute State the set of the commodity of target detection model preparation.

Since when actually veritifying progress commodity veritification in environment, commodity to be veritified may include the combination of a variety of commodity, Therefore carrying out the training of target detection model and test with the combination image for the combination for equally including a variety of commodity has preferably Effect.And single-item image has the advantages that acquisition is convenient, especially when commodity are concentrated and increase new commodity, need to only supplement acquisition The single-item image of a small amount of new commodity can carry out the training of new model.Alternatively, can also training when combine combination image with And single-item image.

In some embodiments, the single-item image is to be placed on turntable and rotated to difference in the sole product Position when, it is collected by the image capture device for being set to different shooting angle.

When acquiring single-item image, image capture device is made to collect image of the sole product under each visual angle as far as possible, To simulate the various placement angles of commodity to be veritified under practical veritification environment, be conducive to the training of target detection model.

In some embodiments, the single-item image is in the sole product respectively according to face-up and back side court On mode when being placed on the turntable, it is collected that equipment is acquired by described image.

For some commodity, such as box-like, bag-like commodity, the appearance of front and back there may be larger difference, The face-up and back side can be divided, and two ways carries out the acquisition of single-item image respectively upward, so that single-item image set more adds It is standby.

In some embodiments, each combination image corresponds to one of preset multiple combinations concentration, every kind of group Close one point that concentration corresponds to the total quantity of a distributed area and commodity for the kind number of commodity in the combination image Cloth section.

Actually veritify environment in, the region for putting commodity is usually limited, thus the quantity of commodity to be veritified not Meanwhile being presented as that the density for veritifying commodity to be veritified in image is different, different commodity density are also acquired when image is combined in acquisition Image so that combination image set closer to actual veritification image.

In some embodiments, the type of commodity and the quantity of wherein every kind of commodity are true at random in each combination image It is fixed.

It is practical to veritify the type of commodity to be veritified under environment and wherein the quantity of every kind of commodity is often random, therefore with Machine determines that the content of combination image is conducive to simulate actual verification procedure, improves the quality of target detection model.

In some embodiments, the random of commodity is put in each combination image.

The practical placement position for veritifying commodity to be veritified under environment is often random, therefore after commodity are put at random again Acquisition combination image is conducive to simulate actual verification procedure, improves the quality of target detection model.

In some embodiments, the commodity concentrate the commodity including multiple classifications, a variety of quotient for including in each classification Product have similitude in appearance.

Concentrating in commodity includes the similar commodity of appearance, the fine-grained feature of commodity collection is embodied, so that the quotient will be based on The single-item image of product collection and/or combination image are used for training objective detection model, trained model also can effective district exceptionally See approximate commodity.

In some embodiments, using the single-item image training target detection model, and the constitutional diagram is utilized As testing the target detection model, comprising:

Front and back scape segmentation is carried out to each single-item image, obtains foreground image only comprising the sole product；

Composograph is generated after repeatedly choosing foreground image according to background image and multiple foreground images of selection to obtain Obtain multiple composographs；

Utilize the multiple composograph training target detection model；

Utilize target detection model described in the combination image measurement.

In these embodiments, it is only necessary to limited single-item image can a large amount of composograph of combination producing, avoid Plenty of time is spent in the acquisition of training set, meanwhile, it can be free group between the corresponding foreground image of each single-item image It closes, can also be freely combined between foreground image and background image, so that training set can effectively cover the difference of commodity veritification Scene is conducive to the quality for improving target detection model, and then improves commodity and veritify result.

In some embodiments, the single-item image combines image and acquires in different environments with described.

The acquisition environment of single-item image and the actual acquisition environment for veritifying image are likely to be different, if therefore single-item Image combines image for training pattern for test model, the two can also be allowed to acquire in different environments, with simulation Training environment and the practical difference for veritifying environment.

In some embodiments, using the combination image training target detection model, and the constitutional diagram is utilized As testing the target detection model, comprising:

Utilize a part combination image training target detection model；

Utilize target detection model described in another part combination image measurement.

In these implementations, training set and test set directly are selected from combination image set, implements fairly simple, and And due to including the combination of a variety of commodity in combination image, with the actual veritification content of image mesh that is approximate, therefore training It is also preferable for commodity veritification effect to mark detection model.

In some embodiments, the target detection model is using the training for having supervision, the corresponding prison of image of training Signal is superintended and directed using one of following three kinds:

The inventory of commodity in the image of the training, wherein the inventory of the commodity includes the image of the training The quantity of the type of middle commodity and every kind of commodity；

The center of every commodity and type in the image of the training；

The encirclement frame and type of every commodity in the image of the training.

The supervision effect of three of the above supervisory signals can be selected according to actual needs by weak to strong.

In some embodiments, the training target detection model, comprising:

When the training of every wheel, the image of epicycle training is input to the target detection model, according to the target detection The prediction result of model output calculates the evaluation index for assessing target detection effect, and in the calculating knot of the evaluation index Fruit terminates training when meeting preset requirement, and model when training is terminated is determined as trained target detection model；

Or

When the training of every wheel, the image of epicycle training is input to the target detection model, according to the target detection The prediction result of model output calculates the evaluation index for assessing target detection effect, and in the target detection model training After termination, it is determined as trained mesh for optimal model is showed on the evaluation index in the model saved after every wheel training Mark detection model；

The test target detection model, comprising:

The image of test is input to the trained target detection model, according to the trained target detection The prediction result of model output calculates the evaluation index, and meets the preset requirement in the calculated result of the evaluation index When, the trained target detection model is determined as to the target detection model by test.

It can be by evaluation index come the prediction effect of quantitative evaluation target detection model, and the trained target of test Whether detection model can put into actual use, and the effective guarantee quality of model is avoided because of the bad influence commodity of model performance Veritification result.

In some embodiments, the trained target detection model is determined as examining by the target of test described It surveys after model, the method also includes:

By in the case where actually veritifying environment the combined image comprising a variety of commodity that acquires be input to it is described by testing Target detection model calculates the evaluation index according to the prediction result that the target detection model by test exports, and Determine that the target detection model by test is available when the calculated result of the evaluation index meets the preset requirement The commodity to be veritified described in detecting in the veritification image, wherein the practical environment of veritifying refers to the acquisition veritification figure The environment of picture.

Although target detection model has passed through the test of test set, the acquisition environment of test image and practical veritification ring Border be not necessarily it is identical, therefore can also actually veritify environment under again test model whether can satisfy commodity core The demand for testing task, avoiding performing poor in actually veritification environment because of model influences the veritification result of commodity.

In some embodiments, the evaluation index comprises at least one of the following:

In all images to be assessed, to comprising the quantity of every kind of commodity all predict shared by correctly image to be assessed Ratio；

In all images to be assessed, the mean value of the prediction error of each image to be assessed a, wherein image to be assessed Prediction error refer to the sum of prediction error of quantity that the image to be assessed includes, every kind of commodity；

In the commodity of all categories, a kind of mean value of the prediction error accounting of every kind of commodity, wherein pre- sniffing of commodity Accidentally accounting refer to the sum of the prediction error of the quantity of this kind of commodity in each image to be assessed, with the quantity of this kind of commodity every The ratio of the sum of true value in a image to be assessed；

In the commodity of all categories, the mean value of the consistency of every kind of commodity, wherein a kind of consistency of commodity refers to this The quantity of kind of commodity the sum of smaller value in each image to be assessed, with the quantity of this kind of commodity in each image to be assessed The sum of the larger value ratio, smaller value of the quantity of this kind of commodity in an image to be assessed refer to the quantity of this kind of commodity Smaller in the predicted value and true value in the image to be assessed, the quantity of this kind of commodity is in an image to be assessed The larger value refers to the greater of the quantity of this kind of commodity in the predicted value and true value in the image to be assessed.

Second aspect, the embodiment of the present application provide a kind of commodity veritification device, comprising:

Image collection module, for obtaining the veritification image comprising commodity to be veritified；

First inventory obtain module, for using described in target detection model inspection veritification image in described in quotient to be veritified Product, and the inventory of the commodity to be veritified is generated according to testing result, wherein the target detection model is algorithm of target detection The neural network model used, the inventory of the commodity to be veritified include the commodity to be veritified type and every kind wait veritify The quantity of commodity；

Second inventory obtains module, and for obtaining the inventory of received payment commodity, the inventory of the received payment commodity includes The quantity of the type of pay for goods and every kind of received payment commodity；

Module is veritified, for the inventory of the commodity to be veritified and the inventory of the received payment commodity to be compared, if The type of commodity and the quantity of every kind of commodity are consistent in two inventories, it is determined that the commodity to be veritified and the received payment Commodity are consistent.

The third aspect, the embodiment of the present application provide a kind of commodity veritification host, comprising: memory and processor, it is described It is stored with computer program instructions in memory, when the computer program instructions are read and run by the processor, executes The step of method that the possible embodiment of any one of first aspect or first aspect provides.

Fourth aspect, the embodiment of the present application provide a kind of commodity apparatus for checking, comprising: the commodity that the third aspect provides are veritified Host and image capture device, described image acquire equipment and connect with commodity veritification host, veritify image for acquiring And the veritification image is sent to the commodity and veritifies host.

In some embodiments, the equipment further include: output equipment is veritified host with the commodity and connect, the quotient Product veritify host be also used to obtain indicate commodity to be veritified inventory and received payment commodity inventory in commodity type and After the whether consistent veritification result of the quantity of every kind of commodity, controls the output equipment and export the veritification result.

Output equipment can be exported in a manner of image, voice etc. to customer or other related personnel veritify as a result, so that These people, which are known, to be veritified result and takes corresponding measure.

In some embodiments, the commodity apparatus for checking further include: channel barrier equipment veritifies host with the commodity Connection, the commodity veritify the kind that host is also used to the commodity in the inventory of the inventory and received payment commodity that determine commodity to be veritified When class and the consistent quantity of every kind of commodity, the open channel for entering and leaving shopping place of the channel barrier equipment is controlled.

Channel barrier equipment can be the equipment such as gate, gate, the passage when not yet carrying out commodity veritification, so that caring for Objective impassable, the open channel after veritifying successfully, can be with if veritification is unsuccessful so that customer may exit off shopping place Continue passage, so that the customer for carrying arrearage commodity can not leave shopping place.

5th aspect, the embodiment of the present application provide a kind of computer readable storage medium, the computer-readable storage medium It is stored with computer program instructions in matter, when the computer program instructions are read out by the processor and run, executes first aspect Or any one possible embodiment of first aspect provide method the step of.

To enable the above-mentioned purpose of the application, technical scheme and beneficial effects to be clearer and more comprehensible, special embodiment below, and Cooperate appended attached drawing, is described in detail below.

Detailed description of the invention

Technical solution in ord to more clearly illustrate embodiments of the present application, below will be to needed in the embodiment attached Figure is briefly described, it should be understood that the following drawings illustrates only some embodiments of the application, therefore is not construed as pair The restriction of range for those of ordinary skill in the art without creative efforts, can also be according to this A little attached drawings obtain other relevant attached drawings.

Fig. 1 shows the structural schematic diagram that a kind of commodity provided by the embodiments of the present application veritify host；

Fig. 2 shows a kind of structural schematic diagrams of commodity apparatus for checking provided by the embodiments of the present application

Fig. 3 shows a kind of flow chart of commodity checking method provided by the embodiments of the present application；

Fig. 4 shows a kind of schematic diagram of acquisition mode of single-item image provided by the embodiments of the present application；

Fig. 5 shows a kind of schematic diagram of acquisition mode of combination image provided by the embodiments of the present application；

Fig. 6 shows the functional block diagram that a kind of commodity provided by the embodiments of the present application veritify device.

Specific embodiment

Below in conjunction with attached drawing in the embodiment of the present application, technical solutions in the embodiments of the present application carries out clear, complete Ground description, it is clear that described embodiments are only a part of embodiments of the present application, instead of all the embodiments.Usually exist The component of the embodiment of the present application described and illustrated in attached drawing can be arranged and be designed with a variety of different configurations herein.Cause This, is not intended to limit claimed the application's to the detailed description of the embodiments herein provided in the accompanying drawings below Range, but it is merely representative of the selected embodiment of the application.Based on embodiments herein, those skilled in the art are not being done Every other embodiment obtained under the premise of creative work out, shall fall in the protection scope of this application.

It should also be noted that similar label and letter indicate similar terms in following attached drawing, therefore, once a certain Xiang Yi It is defined in a attached drawing, does not then need that it is further defined and explained in subsequent attached drawing.Meanwhile the application's In description, term " first ", " second " etc. are only used for distinguishing one entity or operation from another entity or operation, It is not understood to indicate or imply relative importance, can not be understood as require that or imply and be deposited between these entities or operation In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or its any other variant are intended to Non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or equipment Intrinsic element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that There is also other identical elements in process, method, article or equipment including the element.

Fig. 1 shows the structural schematic diagram that a kind of commodity provided by the embodiments of the present application veritify host.Referring to Fig.1, commodity Veritifying host 100 includes that one or more processors 102 and one or more storage devices 104, these components pass through bus The interconnection of bindiny mechanism's (not shown) of system 106 and/or other forms.

Processor 102 can be central processing unit (CPU) or have data-handling capacity and/or instruction execution capability Other forms processing unit, and can control commodity veritify host 100 in other assemblies to execute desired function.

Storage device 104 can with various forms of computer readable storage mediums, such as volatile memory and/or it is non-easily The property lost memory.Volatile memory for example may include random access memory (RAM) and/or cache memory (cache) etc..Nonvolatile memory for example may include read-only memory (ROM), hard disk, flash memory etc..Computer-readable Can store one or more computer program instructions on storage medium, processor 102 can run computer program instructions, with The step of realizing commodity checking method provided by the embodiments of the present application and/or other desired functions.Computer-readable Various application programs and various data, such as the various numbers that application program is used and/or generated can also be stored in storage medium According to etc..

It is appreciated that structure shown in FIG. 1 is only to illustrate, commodity veritification host 100 may also include more than shown in Fig. 1 Perhaps less component or with the configuration different from shown in Fig. 1.Each component shown in Fig. 1 can use hardware, software Or combinations thereof realize.In the embodiment of the present application, commodity, which veritify host 100, may be, but not limited to, desktop computer, notebook electricity The entity devices such as brain, smart phone, intelligent wearable device, mobile unit.Commodity, which veritify host 100, can also be the void such as virtual machine It proposes standby.

The embodiment of the present application also provides a kind of commodity apparatus for checking 200, as shown in Figure 2.Referring to Fig. 2, commodity apparatus for checking 200 include that commodity veritify host 220 and image capture device 210, and the two is connected with each other and can carry out data communication, connects Mode can be it is wired be also possible to it is wireless.

Wherein, commodity are veritified host 220 and can be used, but are not limited to commodity shown in fig. 1 and veritify host 100.Image is adopted Collection equipment 210 may be, but not limited to, the equipment such as camera and be sent to after image capture device 210 collects veritification image Host 220 is veritified to commodity, commodity veritify host 220 and receive veritification image, and then execute commodity provided by the embodiments of the present application Checking method.

Fig. 3 shows a kind of flow chart of commodity checking method provided by the embodiments of the present application.When illustrating, in conjunction with Fig. 2 The commodity apparatus for checking 200 shown is illustrated, it is not intended that the commodity checking method only can be in commodity apparatus for checking 200 Middle application.It applies for example, the commodity checking method can also be veritified only in host 100 in commodity, does not pass through wherein veritifying image Image capture device 210 obtains.It is convenient to illustrate, the application scenarios of commodity apparatus for checking 200 are further limited in shopping field The commodity of its carrying are veritified in the exit of institute (such as supermarket) for customer when leaving shopping place, it is apparent that, this is not answered When being considered as the limitation to the application protection scope.It is somebody's turn to do for example, commodity apparatus for checking 200 can also show that purpose is executed for function Commodity checking method.

Referring to Fig. 3, commodity checking method includes:

Step S30: commodity veritify host 220 and obtain the veritification image comprising commodity to be veritified.

Commodity to be veritified refer to the commodity veritified, for example, purchase is left in attempt after customer does shopping in shopping place The commodity carried when object field institute.Veritifying image can be acquired by image capture device 210, for example, image capture device 210 can be with It is set to above an articles holding table, visual field covers articles holding table, and commodity to be veritified can be actively placed on articles holding table by customer, so Image is veritified in the acquisition of triggering image capture device 210 afterwards, and acquired image is sent to commodity and veritified by image capture device 210 Host 220, so that veritifying image is that commodity are veritified acquired in host 220.Wherein, the acquisition row of image capture device 210 is triggered For mode be not construed as limiting, such as customer can be veritified by clicking button, clicking touch screen, the modes such as voice control to commodity Host 220 assigns instruction, so that commodity is veritified host 220 and controls the acquisition that image capture device 210 veritify image.

Step S31: commodity are veritified host 220 and are veritified in image using target detection (objectdection) model inspection Commodity to be veritified, and generate the inventory of commodity to be veritified according to testing result.

Wherein, target detection model is the neural network model that algorithm of target detection uses, and algorithm of target detection refers to benefit The algorithm of specific objective is detected from image with the target detection model of pre-training, it may for example comprise but it is not limited to Faster- RCNN, Mask-RCNN, FPN scheduling algorithm, here, the specific objective to be detected just refer to commodity to be veritified.Target detection model To veritify image as input, what output can be that model prediction goes out veritify in the image position of every commodity to be veritified and Type is trained target detection model with the image comprising commodity to be veritified before step S31 execution, can make this Model has the ability for detecting commodity to be veritified.Wherein, the position of commodity to be veritified, which can be, veritifies commodity to be veritified in image Encirclement frame (bounding box), the type of commodity to be veritified can be the title and/or specification (such as weight, body of the commodity Product, price etc.), it can also be ID coding of the commodity etc..

Statistic of classification is carried out according to the type for veritifying every commodity to be veritified in image, so that it may be obtained to veritify and be wrapped in image The quantity which plants commodity to be veritified and every kind of commodity to be veritified is included, that is, obtains the inventory of commodity to be veritified.For example, to The inventory for veritifying commodity can be following form:

Type of merchandize	Quantity
		The beer of brand A	3
The milk of brand B	2
		The pencil of brand C	20

It should be understood that the inventory of commodity to be veritified can also include other information, such as the price etc. of every kind of commodity.

Step S32: commodity veritify the inventory that host 220 obtains received payment commodity.

Received payment commodity refer to that the commodity that clearing have been carried out before starting to veritify, the inventory of received payment commodity include The quantity of the type of received payment commodity and every kind of received payment commodity.In some implementations, commodity veritify host 220 can be with The inventory of received payment commodity is obtained from server, a kind of possible generating mode of inventory is as follows: the end that customer uses at it The application program that network operator's exploitation of shopping place is installed in end equipment (such as mobile phone), applies journey using this in shopping process The barcode scanning functionality scan bar code of commodity that sequence provides, and before veritification using terminal equipment by the inventories of scanned commodity It is committed to server, after server returns to corresponding payment link, customer completes the payment to these commodity on the terminal device, After server confirms the payment behavior of customer, the inventory submitted before terminal device is labeled as received payment.

It should be understood that the inventory of received payment commodity is also possible to generate by other means, the above generating mode is only A kind of example.For example, it is also possible to be generated and sent by terminal device to server.In addition, commodity veritify host 220 also not necessarily The inventory of received payment commodity is obtained from independent server, for example, commodity veritify the inherently server of host 220, directly It is interacted with terminal device, the inventory of received payment commodity is locally generated at it.

Step S33: commodity veritify host 220 and compare the inventory of commodity to be veritified and the inventory of received payment commodity.

If the type of commodity and the quantity of every kind of commodity are consistent in two inventories, commodity are veritified host 220 and are determined Commodity to be veritified are consistent with received payment commodity, that is, veritify successfully, and otherwise commodity veritify host 220 and determine veritification commodity and received payment Commodity are inconsistent, that is, veritify failure.Veritification can be generated after the completion of veritifying as a result, veritifying may include in two inventories in result The type of commodity and the whether consistent information of the quantity of every kind of commodity.

In some implementations, commodity apparatus for checking 200 further includes veritifying the output equipment that host 220 is connect with commodity 230.Commodity veritify host 220 after obtaining veritification result, can control the output of output equipment 230 and veritify result.Wherein, it exports Equipment 230 may be, but not limited to, the equipment such as display, loudspeaker, and according to the difference of equipment, the mode of output information can be with It is the modes such as image, voice.After veritifying result output, relevant to commodity veritification personnel, such as customer etc. can know veritification As a result, and taking appropriate measures according to result is veritified.For example, when veritifying the result shows that there are unpaid-for in veritification commodity When commodity, customer can pay the bill to these commodity, then re-start veritification.

In some implementations, commodity apparatus for checking 200 further includes veritifying the channel barrier that host 220 is connect with commodity Equipment 240.Channel barrier equipment 240 blocks the channel for entering and leaving shopping place when customer not yet carries out commodity veritification, so that caring for Visitor is impassable, and commodity veritify the kind of the commodity in the inventory of the inventory and received payment commodity that determine commodity to be veritified of host 220 When class and the consistent quantity of every kind of commodity, control channel stops 240 open channel of equipment, so that customer may exit off shopping field Institute, and when two inventories are inconsistent, channel barrier equipment 240 can be allowed to continue passage, so that customer is at unfinished pair Shopping place can not be left when storewide payment.Wherein, channel barrier equipment 240 may be, but not limited to, gate, gate Etc. equipment the clearance to customer is combined with the result that commodity are veritified in these embodiments, be advantageously implemented shopping field Unmanned management.

It should be understood that commodity apparatus for checking 200 can also carry out other to result is veritified after step S33 has been executed Processing, such as storage, analysis etc., will not enumerate.

In conclusion commodity checking method provided by the embodiments of the present application is generated according to the result of algorithm of target detection to core The inventory of commodity is tested, and whether the inventory content for verifying commodity to be veritified is consistent with the inventory content of received payment commodity, and then really Whether fixed commodity to be veritified have paid the bill.Entire commodity verification procedure is not necessarily to manpower intervention, effectively saves cost of human resources, The mistake in verification procedure is avoided, the unmanned development trend of retail trade is met.Meanwhile with deep learning field The continuous progress of research, it is anticipated that the precision of algorithm of target detection will also be continuously improved, thus the accuracy that commodity are veritified It will be continuously improved, i.e., this method has good application prospect.

In some implementations, before executing step S31, this method further include:

Using the single-item image and/or combination image training objective detection model of pre-acquired, and utilize combination image measurement Target detection model.

Wherein, single-item image refers to that the image for the sole product only concentrated comprising commodity, constitutional diagram seem to refer to comprising commodity The combined image for a variety of commodity concentrated, commodity collection refer to the set of the commodity prepared by training objective detection model, quotient The commodity that product are concentrated cover possible veritification commodity, certainly can also include more commodity.

Further, commodity concentrate the commodity including multiple classifications, the multiple types for including in the commodity of each classification (also often referred to as a variety of herein) commodity, the quantity of every kind of commodity is multiple.For example, concentrated in a possible commodity, it can To include dilated food, kernel preserved fruit, dried food and nuts, reconstitute, instant noodles, sweets, beverage, wine, can, chocolate, chewing gum, sugar The commodity of fruit, seasoning, personal nursing, paper handkerchief and stationery totally 17 classifications may include again product under this classification of chocolate The dark chocolate bar of board A, the white chocolate of brand A, brand B milk chocolate totally 3 kinds of chocolates, every kind of chocolate can prepare 5.It should be understood that above-mentioned merchandise classification, type, quantity are illustratively, not constitute the limit to the application protection scope System.

Optionally, have in appearance some or all of in a variety of commodity for including in each classification of commodity collection similar Property, for example, brand A dark chocolate bar and brand A white chocolate its packaging be all strip plastics package, surface of package Text is also similar, only the chocolate pattern of surface of package the former be black, the latter be white.Concentrating in commodity includes appearance phase As commodity, embody the fine-grained feature of commodity collection, i.e. the single-item image based on the commodity collection and/or combination image is for instructing Practice target detection model, trained model also can effective district exceptionally see approximate commodity, because of the approximate quotient of these appearances Product are already contained in the image of training sample.

Based on certain part commodity that commodity are concentrated, image only comprising the commodity, i.e. the single-item image of the commodity can be acquired, The single-item image of one commodity can acquire one or more.The acquisition mode of single-item image is not construed as limiting, such as can be used Mode shown in Fig. 4.Referring to Fig. 4, a turntable is set on a level table, the sole product that acquire image is placed in On turntable, multiple images are set near turntable and acquire equipment, each image capture device is in different bats from turntable Take the photograph angle, wherein the angle of horizontal plane where shooting angle can refer to the main shaft and turntable of image capture device, such as scheming In 4,4 image capture devices are shared, shooting angle (image capture device position at 90 ° that is respectively 0 °, 30 °, 45 ° and 90 ° Right above the turntable), it should be pointed out that, image capture device herein, which refers to, to be used when acquisition single-item image, does not refer to use The image capture device 210 of image is veritified in acquisition.In collection process, turntable be it is rotatable, when commodity are rotated to not When with position, each image capture device will acquire corresponding single-item image respectively.For example, turntable 9 ° of pauses one of every rotation Under, a single-item image is respectively acquired by 4 image capture devices, 4 single-item images is acquired altogether, is then acquired altogether after rotating a circle 360 °/9 ° * 4=160 are directed to the single-item image of the commodity.The every kind of commodity concentrated for commodity all acquire its corresponding list Product image constitutes single-item image set, for example for commodity collection exemplified above, single-item image set may include 160 brand A The single-item image of dark chocolate bar, single-item image of the white chocolate of 160 brand A, etc..

For part commodity, there are the concepts of front and back, such as packaging or box-packed commodity for appearance.This kind of commodity When face-up placement, the back side will be blocked, and when the back side is placed upward, front will be blocked, and its front and back is outer There may be certain differences for sight, when it being caused to place in different ways, it appears that seem two different commodity.To this kind of It can be placed in turntable in the way of upward by the face-up and back side respectively when acquiring single-item image by commodity, And be acquired in the way of being set forth above respectively, the single-item picture being achieved in that could be than more fully showing this kind of commodity Appearance.It should be understood that if certain commodity is not present front and back or front and back appearance and does not have significant difference, Or only can fix a certain face-up placement under normal circumstances, then adopting for single-item image only can also be carried out to the single side of the commodity Collection.

Since single-item image can be used for training objective detection model, when acquiring single-item image, by presented hereinbefore Method so that image capture device is collected image of the sole product under each visual angle as far as possible, to simulate practical veritification environment Under commodity to be veritified various placement angles, be conducive to the training of target detection model, and the raising of target detection model quality, The accuracy of commodity veritification will directly be improved.

Based on a variety of commodity that above-mentioned commodity are concentrated, the combined image comprising a variety of commodity can also be acquired, that is, is combined Image.The combination of a variety of commodity designated herein refers to that each combination image may be comprising different types of commodity, every kind of commodity It may include one or more, while even if the types and quantity of the commodity in two combination images are identical, but it is put Position, the mode put it is different, also belong to different combination images.Obviously, it is difficult to all combination images of exhaustion, Ke Yigen Several combination images are acquired as combination image set according to actual demand.

The acquisition mode of combination image is not construed as limiting, such as can be in such a way that Fig. 5 is shown.Referring to Fig. 5, in a level One is arranged on table top for placing the placement region of commodity, for example, it may be the cardboard of a rectangle, will acquire the more of image Kind commodity are placed in the region.One image capture device is set above placement region, and the visual field of the image capture device is covered Lid placement region, so that the commodity being located in the region can be collected.It is actual in order to simulate that placement region, which is arranged, Environment being veritified, because placing the articles holding table limited area of commodity to be veritified in practice, it is also not possible to the customer that leaves places commodity, when It so can also be not provided with placement region in some implementations, being equivalent to default placement region at this time is exactly image capture device Field coverage.It may be noted that image capture device herein uses when referring to acquisition combination image, does not refer to and be used for The image capture device 210 of image is veritified in acquisition.

In some embodiments, each combination image can correspond to one of preset multiple combinations concentration, often In kind of the corresponding combination image of combination concentration a distributed area of the kind number (that is, quantity of the type of commodity) of commodity and One distributed area of the total quantity of commodity.For example, combination concentration be divided into it is three kinds basic, normal, high:

Combine concentration	Commodity kind number	Commodity total quantity
			It is low	3~5	3~10
In	5~8	10~15
			It is high	8~10	15~20

For when combining concentration and being low, it is meant that include 3 to 5 kinds of commodity in combination image, while combining image The total quantity of middle commodity is 3 to 10, as the quantity of wherein every kind of commodity, does not then limit its method of salary distribution.Assuming that acquire 30000 combination images then can acquire 10000 according to basic, normal, high three kinds of combinations concentration respectively.

Before it has been noted that the region for putting commodity is limited, therefore in commodity to be veritified actually veritifying in environment Quantity difference when, be presented as that the density for veritifying commodity to be veritified in image is different.Since combination image can be used for training objective Detection model, therefore when image is combined in acquisition, different combination concentrations is distinguished, so that commodity are presented in placement region Different density improves the effect of model training so as to simulate actual veritification image.

In some embodiments, it acquires the type of commodity in each combination image and wherein the quantity of every kind of commodity can be with It is random to determine.For example, the inventory for the commodity that acquisition combination image uses every time is first generated with random algorithm on computers, then It concentrates the corresponding commodity of acquisition to be put into placement region from commodity to be acquired.It is when with combination above concentration being low Example, a kind of possible way are: random algorithm determines a value, such as 3 at random from type of merchandize 3 to 5 every time, then again from 3 kinds of commodity are determined in all commodity of commodity collection at random, then determine a value at random from the total quantity 3-10 of commodity again, Such as 10, it then determines the quantity of every kind of commodity, such as 3,3,4 at random again, needs to guarantee every kind of commodity when determining certainly Quantity is greater than 0 and is no more than the total quantity that commodity concentrate such commodity.It should be understood that above-mentioned only a kind of illustrative random Algorithm, random algorithm can also use other implementations.

Optionally, barcode scanning can also be carried out to commodity when placing, generate the inventory for placing commodity, and raw with random algorithm At inventory compare on computers, confirm that the commodity of placement and the specified commodity of algorithm are consistent, avoid wrong placement Commodity destroy the randomness of combination image.

Due to it is practical veritify under environment the type of commodity to be veritified and wherein the quantity of every kind of commodity be often it is random, because This random content for determining combination image is conducive to simulate actual verification procedure, improves the quality of target detection model.

In some embodiments, the random of commodity is put in each combination image.It may be noted that may due to putting commodity It can be related to manual operation, therefore designated herein putting at random is not putting in absolute sense at random, and refer in constitutional diagram picture What is presented puts effect, i.e., commodity put be not feel the pulse with three fingers simulataneously according to certain specific rule put (for example, always same Kind commodity are put together), but more free disposing way can be taken.For same commodity, after putting at random, Multiple different combination images can be collected, the training value of these commodity can be sufficiently excavated.

Since the practical placement position for veritifying commodity to be veritified under environment is likely to random (true according to the wish of customer It is fixed), therefore combination image is acquired again after commodity are put at random, the placement shape of commodity complexity when being conducive to simulate actual veritification Condition improves the quality of target detection model.

Above-mentioned determining combination concentration, puts commodity conduct at the random type and quantity for determining commodity at random Optional mode, can independently implement, can also be combined with each other implementation when acquisition combination image.

The training and test of target detection model is described below.Target detection model can based on combination image training and Test.

For example, in some implementations, can use a part of image in combination image set as training set training Target detection model, using another part image in combination image set as test set test target detection model.Training set It can be entire combination image set with the union of test set, only can also entirely combine a part of image set.Directly from group It closes and selects training set and test set in image set, implement fairly simple, and due to carrying out quotient in actually veritification environment When product are veritified, commodity to be veritified may include the combination of a variety of commodity, thus with equally comprising a variety of commodity combination (that is, with Veritify picture material it is similar) combination image carry out target detection model training and test have preferable effect.

Image is veritified for preferable simulation, in the alternative, the acquisition environment for combining image can simulate practical core Test environment, for example, by using with it is practical veritify environmental classes as illumination condition, surrounding enviroment, image capture device and commodity it is opposite Positional relationship etc. can be simulated respectively if practical veritify there are many environment.

Target detection model can also be based on the training of single-item image and based on combination image measurement.

Single-item image is used directly for training, but also mentions before, when actually veritifying progress commodity veritification in environment, It veritifies and may include a variety of combinations for veritifying commodity in image, veritifying image for preferably simulation in some implementations can To generate composograph based on single-item image, and utilize composograph training objective detection model.Specific step is as follows:

Divide firstly, carrying out front and back scape to each single-item image in single-item image set, obtaining only includes sole product Foreground image, after whole single-item image procossings, acquisition foreground picture image set.For example, if single-item image is relatively fixed In the environment of acquire, can by directly by background subtraction in a manner of extract foreground image.In another example can also be by part single-item figure As training a convolutional neural networks to extract foreground image after carrying out front and back scape mark.Certainly other modes can also be used, No longer illustrate one by one.

Secondly, generating composograph according to background image and multiple foreground images of selection.Background image can be list The acquisition background of product image can be from preset background image library, which may include practical veritification ring Possible background image in border.The mode that multiple foreground images are chosen from prospect image set is not construed as limiting, such as can be referred to Each foreground image is considered as a kind of commodity and randomly selected by the collection process for combining image.So-called composograph, refer to by The multiple foreground images and background image selected are superimposed the image of generation, wherein multiple foreground images are as the conjunction At the prospect of image, i.e., the commodity for including in composograph.The position of each foreground image in the composite image is not construed as limiting, example Such as each foreground image can be considered as a kind of commodity and put at random in the composite image with the collection process of reference portfolios image. Different foreground images is chosen repeatedly and generates composograph, obtains multiple composographs altogether；

Finally, using multiple composographs as training set training objective detection model, and using multiple combination images as survey Examination collects test target detection model, certainly in some implementations, can also be only by a part combined in image as survey Examination collection.

In these implementations, it is only necessary to limited single-item image can a large amount of composograph of combination producing, keep away Exempt to spend in the plenty of time in the acquisition of training set, meanwhile, it can be free between the corresponding foreground image of each single-item image Combination, can also be freely combined between foreground image and background image, veritify not so that training set can effectively cover commodity Same scene is conducive to the quality for improving target detection model, and then improves commodity and veritify result.

In addition, single-item Image Acquisition is more convenient, without largely putting commodity, is especially concentrated in commodity and increase new quotient When product, the single-item image that need to only supplement a small amount of new commodity of acquisition can carry out the training of new model, i.e. single-item image set has Good scalability.Compared to using combine image and be trained, it is not necessary that new commodity and commodity are concentrated existing commodity difference It is combined and acquires, workload is smaller.Although combination image has the advantages that close to image is veritified, according to single-item image The composograph of generation is combined with a variety of single-item images due to that can be considered as, and is also the good simulation to image is veritified, Training effect is good.

Further, if carrying out model training using single-item image, model measurement is carried out using combination image, then single-item figure As with combine image and can acquire in different environments.The reason is that the acquisition environment of single-item image and practical veritification ring Border is likely to be different, therefore combination image is allowed to acquire in other environments, can be with simulated training environment (i.e. single-item image Acquisition environment) and the practical difference for veritifying environment, to test out the trained model under a certain environment and change an environment and be It is no still available, it is more steady to be conducive to the model for making to train.

It should be understood that in some implementations, can also both use single-item image in training objective detection model (including directly using single-item image or using composograph) uses unit assembly drawing picture again, and in test target detection model Using combination image, to obtain the test result for being comparatively close to practical veritification image.

In some implementations, target detection model is using the training for having supervision, the corresponding supervision of image of training Signal (that is, label of training sample) is using one of following three kinds:

First, using the inventory of commodity in the image of training, wherein the inventory of commodity includes quotient in the image of training The image of the quantity of the type of product and every kind of commodity, training designated herein refers to the image in training set, does not distinguish Bottom is using single-item image (or composograph) or combines image as training set.

Second, using the center of every commodity and the type of commodity in the image of training, wherein every commodity The positions of the center whole pixels that can include according to commodity calculated.

Third, using the type for surrounding frame and commodity of every commodity in the image of training, wherein encirclement frame can be with It is the rectangle frame just including commodity.

For the supervision effect of three of the above supervisory signals by weak to strong, stronger supervisory signals mean this prediction essence to model Degree requirement is higher, can be selected according to actual needs.It is corresponding for single-item image about the mark of supervisory signals Type of merchandize be it is known, the position of commodity in the picture then can by front and back scape divide know automatically, certainly using manually It marks also possible；For composograph, since the position of each foreground image is known, and foreground picture in synthesis The type of picture has marked, so the supervisory signals of composograph can automatically generate；For combining image, can take artificial The mode of mark.

Since the accuracy that commodity are veritified directly is linked up with economic factor, and the accuracy that commodity are veritified depends primarily on mesh Mark the detection accuracy of detection model, therefore in some implementations, can design some evaluation indexes in the training of model and The links such as test assess the detection effect of model, avoid model of low quality from being applied to actual commodity and veritify, Cause the loss of shopping both sides.

For example, have a image to be assessed of N (N >=1), while commodity concentrate shared K (K >=1) to plant commodity, it will be N number of to be assessed Image is input to target detection model, model can output test result, the prediction including position and type to commodity, after statistics The predicted value of the type of commodity and the quantity of every kind of commodity can be obtained, meanwhile, image to be assessed, may in advance by mark Notation methods above illustrated, the type and every kind of commodity of image commodity to be assessed can be obtained according to annotation results The true value of quantity, wherein image to be assessed is specifically which kind of image is not construed as limiting, for example, it may be training set or test set In image.

To be defined as follows: P convenient for illustrating_i,kIndicate the prediction of the quantity of kth kind commodity in i-th of image to be assessed Value, if model prediction is not the result shows that include kth kind commodity in image to be assessed, it may be considered that predicted value is 0, convenient for meter It calculates.GT_i,kIndicate the true value of the quantity of kth kind commodity in i-th of image to be assessed, CD_i,kIt indicates in i-th of image to be assessed The true value of the quantity of kth kind commodity and the absolute value of the difference of predicted value.

Evaluation index may include, but be not limited to following index:

(1) in all images to be assessed, to comprising the quantity of every kind of commodity all predict correctly image institute to be assessed The ratio accounted for.

The evaluation index is denoted as A1, A1 can be formulated are as follows:

Wherein, return to 1 when the value of function δ () expression formula in bracket is 0, otherwise return to 0, even model to it is a certain to The quantity for every kind of commodity that assessment image includes all is predicted correctly, thenValue be 1, be otherwise 0.Index A1 is assessed according to image, A1 bigger (closer to 1), and showing model prediction, correctly image to be assessed is to be assessed in whole Accounting is bigger in image, and model performance is better.

(2) in all images to be assessed, the mean value of the prediction error of each image to be assessed, wherein one is to be assessed The prediction error of image refers to the sum of prediction error of quantity that the image to be assessed includes, every kind of commodity.

The evaluation index is denoted as A2, A2 can be formulated are as follows:

Wherein, the prediction error of quantity of the kth kind commodity in i-th of image to be assessed is exactly CD_i,k, i-th to be assessed The prediction error of image is exactlyIndex A2 is assessed according to image, index A2 smaller (closer to 0), Show on average, prediction error of the model on each image to be assessed is smaller, and model performance is better.

(3) in the commodity of all categories, the mean value of the prediction error accounting of every kind of commodity, wherein a kind of commodity it is pre- Sniffing miss accounting refer to the sum of the prediction error of quantity of this kind of commodity in each image to be assessed, with this kind of commodity each The ratio of the sum of the true value of quantity in image to be assessed.

The evaluation index is denoted as A3, A3 can be formulated are as follows:

Wherein, the prediction error of quantity of the kth kind commodity in i-th of image to be assessed is exactly CD_i,k, kth kind commodity exist The sum of the prediction error of quantity in each image to be assessed is exactlyKth kind commodity are in i-th of figure to be assessed The true value of quantity as in is exactly GT_i,k, the true value of quantity the sum of of the kth kind commodity in each image to be assessed be exactlyThe prediction error accounting of kth kind commodity is exactlyIndex A3 is assessed according to type of merchandize, Index A3 smaller (closer to 0) shows on average, prediction error of the model in each type of merchandize is smaller, model Performance is better.

(4) in the commodity of all categories, the mean value of the consistency of every kind of commodity, wherein a kind of consistency of commodity is Refer to the sum of the smaller value of the quantity of this kind of commodity in each image to be assessed, with the quantity of this kind of commodity in each figure to be assessed The ratio of the sum of the larger value as in, smaller value of the quantity of this kind of commodity in an image to be assessed refer to this kind of commodity Smaller of the quantity in the predicted value and true value in the image to be assessed, the quantity of this kind of commodity is in an image to be assessed In the larger value refer to the greater of the quantity of this kind of commodity in the predicted value and true value in the image to be assessed.

The evaluation index is denoted as A4, A4 can be formulated are as follows:

Wherein, smaller value of the kth kind commodity in i-th of image to be assessed is exactly min (GT_i,k,P_i,k), kth kind commodity The sum of smaller value in each image to be assessed is exactlyKth kind commodity are to be evaluated at i-th Estimating the larger value in image is exactly max (GT_i,k,P_i,k), smaller value the sum of of the kth kind commodity in each image to be assessed is exactlyIndex A4 bigger (closer to 1), shows on average, model is in each type of merchandize On predicted value and true value be all closer to, model performance is better.

In the training process of target detection model, a kind of possible usage of above-mentioned evaluation index are as follows:

When the training of every wheel, the image of epicycle training is input to target detection model, is exported according to target detection model Prediction result calculate evaluation index, and when the calculated result of evaluation index meets preset requirement terminate training, will training eventually Model when only is determined as trained target detection model.

The training of neural network model is divided into more wheels, every wheel training can all choose a collection of image (corresponding to it is above it is N number of to Assess image), the index of calculating can choose one or more of 4 indexs above, naturally it is also possible to select other energy The index of enough assessment models prediction effects.Above-mentioned preset requirement can be some threshold value for referring to and being directed to target setting, with index A1 For, a threshold value 0.99 can be set, only the value of calculated index A1 is more than 0.99 after certain wheel training, just calculates and meets Preset requirement, since index A1 is better closer to 1 model performance, the value of present index A1 alreadys exceed 0.99, it is believed that Model has trained, therefore can be with deconditioning.If using multiple evaluation indexes simultaneously, can be set for each evaluation index Threshold value is set, when all evaluation indexes all meet the condition of threshold value restriction after certain wheel training, ability deconditioning.Furthermore, it is necessary to It points out, the prediction result of these evaluation indexes and target detection model is closely related, therefore when evaluation index performance is preferable, mould Type has necessarily restrained, and is not in the situation that training terminates and model is not converged.

If evaluation index is not still able to satisfy preset requirement, can after having trained scheduled time or scheduled wheel number To take appropriate measures, such as the image of the training of supplement newly re-starts training.

In the training process of target detection model, the alternatively possible usage of above-mentioned evaluation index are as follows:

When the training of every wheel, the image of epicycle training is input to target detection model, is exported according to target detection model Prediction result calculate evaluation index and save epicycle training after model, target detection model training termination after, taken turns every Optimal model is showed in the model of preservation on evaluation index and is determined as trained target detection model.

Wherein, the condition that training terminates is not construed as limiting, such as be can be model convergence, had trained scheduled wheel number, training The combination of one or more conditions in the scheduled time.When exercise wheel number is excessive, performance can not necessarily rise model, Performance may be caused to decline because of reasons such as over-fittings instead, be presented as that the calculated result of evaluation index is deteriorated.It therefore can be from every Select to show optimal model on evaluation index as trained target detection model in the model cached after wheel training, and It is not the model directly selected when training terminates.

If showing on evaluation index, optimal model performance is still bad, for example, the numerical value of its corresponding evaluation index It is unsatisfactory, it can take appropriate measures, such as the image of the training of supplement newly re-starts training.

In the test process of target detection model, the alternatively possible usage of above-mentioned evaluation index are as follows:

The image of test is input to trained target detection model, is exported according to trained target detection model Prediction result calculate evaluation index, and when the calculated result of evaluation index meets preset requirement, trained target is examined It surveys model and is determined as the target detection model by test.The image of test designated herein can refer to entire test set In image, may also mean that a part of image in test set, the latter can repeatedly be tested.

If being unsatisfactory for preset requirement according to the calculated evaluation index of the image of test, can take appropriate measures, Such as the image of the training of supplement newly re-starts training, then retests again.

In some implementations, use can be direct plungeed by the target detection model of test, that is, is used as practical quotient Product model to be used when veritifying, in other implementations, it is contemplated that the acquisition environment and practical core of the image of test Test environment be not necessarily it is identical, therefore can also actually veritify environment under again test model whether can satisfy quotient Product veritify the demand of task, and avoiding performing poor in actually veritification environment because of model influences the veritification result of commodity.Specifically may be used To use following way:

The combined image comprising a variety of commodity acquired under actually veritification environment is input to the target by test Detection model calculates evaluation index according to the prediction result of the target detection model output by test, and in evaluation index When calculated result meets preset requirement determine by test target detection model can be used for detecting veritification image in wait veritify Commodity.

Evaluation index is introduced to model in links such as the training and test of model, quantitative evaluation target detection model is come with this Prediction effect, and whether the trained target detection model of test can put into actual use, effective guarantee model Quality is conducive to improve the accuracy that commodity veritify result.

In addition, target detection model is redesigned according to the calculated result of some evaluation indexes, by taking index A3 as an example, according to Every kind of commodity are calculated in K kind commodityValue can also judge whether certain commodity is difficult to detect (value compared with Greatly), for being difficult to the commodity detected, a kind of possible way is the sample re -training of the supplement commodity, alternatively possible Way is to be individually for one new detection model of this commodity design, uses original target detection model simultaneously when commodity are veritified It being used simultaneously with new model, model originally is only used to detect the other kinds of commodity in addition to difficult detection commodity, newly Model is only used to detect the commodity of difficult detection, and the inventory of commodity is generated in conjunction with the testing result of the two.

The embodiment of the present application also provides a kind of commodity veritification device 300, as shown in Figure 6.Referring to Fig. 6, which includes:

Image collection module 610, for obtaining the veritification image comprising commodity to be veritified；

First inventory obtains module 620, in veritification image described in the target detection model inspection using pre-training Commodity to be veritified, and generate the inventory of the commodity to be veritified according to testing result, wherein the target detection model is The neural network model that algorithm of target detection uses, the inventory of the commodity to be veritified include the type of the commodity to be veritified with And the quantity of every kind of commodity to be veritified；

Second inventory obtains module 630, and for obtaining the inventory of received payment commodity, the inventory of the received payment commodity includes The quantity of the type of received payment commodity and every kind of received payment commodity；

Module 640 is veritified, for the inventory of the commodity to be veritified and the inventory of the received payment commodity to be compared, If the type of commodity and the quantity of every kind of commodity are consistent in two inventories, it is determined that the commodity to be veritified with it is described paid Money commodity are consistent.

The technical effect of commodity veritification device 600 provided by the embodiments of the present application, realization principle and generation is in aforementioned side By the agency of in method embodiment, to briefly describe, Installation practice part does not refer to that place, the method for can refer to are applied in corresponding in example Hold.

It should be noted that all the embodiments in this specification are described in a progressive manner, each embodiment weight Point explanation is the difference from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For device class embodiment, since it is basically similar to the method embodiment, so being described relatively simple, related place ginseng See the part explanation of embodiment of the method.

In several embodiments provided herein, it should be understood that disclosed device and method can also pass through it His mode is realized.The apparatus embodiments described above are merely exemplary, for example, the flow chart and block diagram in attached drawing are aobvious The device of multiple embodiments according to the application, architectural framework in the cards, the function of method and computer program product are shown It can and operate.In this regard, each box in flowchart or block diagram can represent one of a module, section or code Point, a part of the module, section or code includes one or more for implementing the specified logical function executable Instruction.It should also be noted that function marked in the box can also be attached to be different from some implementations as replacement The sequence marked in figure occurs.For example, two continuous boxes can actually be basically executed in parallel, they sometimes may be used To execute in the opposite order, this depends on the function involved.It is also noted that each of block diagram and or flow chart The combination of box in box and block diagram and or flow chart can be based on the defined function of execution or the dedicated of movement The system of hardware is realized, or can be realized using a combination of dedicated hardware and computer instructions.

In addition, each functional module in each embodiment of the application can integrate one independent portion of formation together Point, it is also possible to modules individualism, an independent part can also be integrated to form with two or more modules.

It, can be with if the function is realized and when sold or used as an independent product in the form of software function module It is stored in computer-readable storage medium.Based on this understanding, the technical solution of the application is substantially in other words to existing Having the part for the part or the technical solution that technology contributes can be embodied in the form of software products, the computer Software product is stored in a storage medium, including some instructions are with so that computer equipment executes each embodiment institute of the application State all or part of the steps of method.Computer equipment above-mentioned includes: personal computer, server, mobile device, intelligently wears The various equipment with execution program code ability such as equipment, the network equipment, virtual unit are worn, storage medium above-mentioned includes: U Disk, mobile hard disk, read-only memory, random access memory, magnetic disk, tape or CD etc. are various to can store program code Medium.

The above, the only specific embodiment of the application, but the protection scope of the application is not limited thereto, it is any Those familiar with the art within the technical scope of the present application, can easily think of the change or the replacement, and should all contain Lid is within the scope of protection of this application.Therefore, the protection scope of the application shall be subject to the protection scope of the claim.

Claims

1. a kind of commodity checking method characterized by comprising

Obtain the veritification image comprising commodity to be veritified；

Commodity to be veritified described in veritification image described in target detection model inspection using pre-training, and according to testing result Generate the inventory of the commodity to be veritified, wherein the target detection model is the neural network mould that algorithm of target detection uses Type, the inventory of the commodity to be veritified include the type of commodity to be veritified and the quantity of every kind of commodity to be veritified；

Obtain received payment commodity inventory, the inventory of the received payment commodity include received payment commodity type and every kind it is paid The quantity of money commodity；

The inventory of the commodity to be veritified and the inventory of the received payment commodity are compared, if in two inventories commodity kind Class and the quantity of every kind of commodity are consistent, it is determined that the commodity to be veritified are consistent with the received payment commodity.

2. commodity checking method according to claim 1, which is characterized in that in the target detection mould using pre-training Type detect it is described wait before veritifying commodity in the veritification image, the method also includes:

Using the single-item image and/or the combination image training target detection model of pre-acquired, and utilize the combination image Test the target detection model, wherein the single-item image refers to the image for the sole product only concentrated comprising commodity, described Constitutional diagram seems the combined image for referring to a variety of commodity concentrated comprising the commodity, and the commodity, which integrate, to be referred to as the training mesh The set for the commodity that mark detection model is prepared.

3. commodity checking method according to claim 2, which is characterized in that the single-item image is in the sole product When being placed on turntable and rotating to different positions, the image capture device by being set to different shooting angle is acquired It arrives.

4. commodity checking method according to claim 3, which is characterized in that the single-item image is in the sole product Respectively when the face-up and back side is placed on the turntable in the way of upward, equipment is acquired by described image and is adopted Collect.

5. commodity checking method according to claim 2, which is characterized in that each combination image corresponds to preset a variety of groups One of concentration is closed, every kind of combination concentration corresponds to a distributed area of the kind number of commodity in the combination image An and distributed area of the total quantity of commodity.

6. commodity checking method according to claim 2, which is characterized in that it is each combination image in commodity type and Wherein the quantity of every kind of commodity determines at random.

7. commodity checking method according to claim 2, which is characterized in that the commodity in each combination image are put at random It puts.

8. the commodity checking method according to any one of claim 2-7, which is characterized in that the commodity, which are concentrated, includes The commodity of multiple classifications, a variety of commodity for including in each classification have similitude in appearance.

9. commodity checking method according to claim 2, which is characterized in that utilize the single-item image training target Detection model, and utilize target detection model described in the combination image measurement, comprising:

Composograph is generated according to background image and multiple foreground images of selection, after repeatedly choosing foreground image, is obtained more A composograph；

Utilize the multiple composograph training target detection model；

Utilize target detection model described in the combination image measurement.

10. commodity checking method according to claim 9, which is characterized in that the single-item image and the combination image It acquires in different environments.

11. commodity checking method according to claim 2, which is characterized in that utilize the combination image training mesh Detection model is marked, and utilizes target detection model described in the combination image measurement, comprising:

Utilize a part combination image training target detection model；

12. the commodity checking method according to any one of claim 9-11, which is characterized in that the target detection mould Type uses one of following three kinds using the training for having supervision, the corresponding supervisory signals of image of training:

The inventory of commodity in the image of the training, wherein the inventory of the commodity includes quotient in the image of the training The quantity of the type of product and every kind of commodity；

The center of every commodity and type in the image of the training；

13. the commodity checking method according to any one of claim 9-11, which is characterized in that the training mesh Mark detection model, comprising:

When the training of every wheel, the image of epicycle training is input to the target detection model, according to the target detection model The prediction result of output calculates the evaluation index for assessing target detection effect, and full in the calculated result of the evaluation index Training is terminated when sufficient preset requirement, model when training is terminated is determined as trained target detection model；

Or

When the training of every wheel, the image of epicycle training is input to the target detection model, according to the target detection model The prediction result of output calculates the evaluation index for assessing target detection effect, and terminates in the target detection model training Afterwards, optimal model will be showed on the evaluation index in the model that saves after every wheel training be determined as trained target inspection Survey model；

The test target detection model, comprising:

The image of test is input to the trained target detection model, according to the trained target detection model The prediction result of output calculates the evaluation index, and when the calculated result of the evaluation index meets the preset requirement, The trained target detection model is determined as to the target detection model by test.

14. commodity checking method according to claim 13, which is characterized in that examine the trained target described Model is surveyed to be determined as after the target detection model by test, the method also includes:

The combined image comprising a variety of commodity acquired under actually veritification environment is input to the target by test Detection model calculates the evaluation index according to the prediction result that the target detection model by test exports, and in institute It states and determines that the target detection model by test can be used for examining when the calculated result of evaluation index meets the preset requirement Survey the commodity to be veritified in the veritification image, wherein the practical environment of veritifying refers to the acquisition veritification image Environment.

15. commodity checking method according to claim 13, which is characterized in that the evaluation index includes following at least one Kind:

In all images to be assessed, to comprising the quantity of every kind of commodity all predict ratio shared by correctly image to be assessed Example；

In all images to be assessed, the mean value of the prediction error of each image to be assessed, wherein image to be assessed it is pre- It surveys error and refers to the sum of prediction error of quantity that the image to be assessed includes, every kind of commodity；

In the commodity of all categories, the mean value of the prediction error accounting of every kind of commodity, wherein a kind of prediction error of commodity accounts for Than refer to the sum of the prediction error of the quantity of this kind of commodity in each image to be assessed, with the quantity of this kind of commodity it is each to Assess the ratio of the sum of true value in image；

In the commodity of all categories, the mean value of the consistency of every kind of commodity, wherein a kind of consistency of commodity refers to this kind of quotient The quantity of product the sum of smaller value in each image to be assessed, with the quantity of this kind of commodity in each image to be assessed compared with The ratio of the sum of big value, smaller value of the quantity of this kind of commodity in an image to be assessed refer to the quantity of this kind of commodity at this The quantity of the smaller in predicted value and true value in image to be assessed, this kind of commodity are larger in an image to be assessed Value refers to the greater of the quantity of this kind of commodity in the predicted value and true value in the image to be assessed.

16. a kind of commodity veritify device characterized by comprising

First inventory obtain module, for using pre-training target detection model inspection described in veritification image in described in core Commodity are tested, and generate the inventory of the commodity to be veritified according to testing result, wherein the target detection model is target detection The neural network model that algorithm uses, the inventory of the commodity to be veritified include the commodity to be veritified type and every kind to Veritify the quantity of commodity；

Second inventory obtains module, and for obtaining the inventory of received payment commodity, the inventory of the received payment commodity includes received payment The quantity of the type of commodity and every kind of received payment commodity；

Module is veritified, for the inventory of the commodity to be veritified and the inventory of the received payment commodity to be compared, if two The type of commodity and the quantity of every kind of commodity are consistent in inventory, it is determined that the commodity to be veritified and the received payment commodity Unanimously.

17. a kind of commodity veritify host characterized by comprising memory and processor are stored with meter in the memory Calculation machine program instruction, when the computer program instructions are read and run by the processor, perform claim requires to appoint in 1-14 The step of method described in one.

18. a kind of commodity apparatus for checking characterized by comprising commodity described in claim 17 veritify host and image Equipment is acquired, described image acquisition equipment and the commodity are veritified host and connects, for acquisition veritification image and by the veritification Image is sent to the commodity and veritifies host.

19. commodity apparatus for checking according to claim 18, which is characterized in that the equipment further include: output equipment, with The commodity veritify host connection, and the commodity veritify host and are also used to obtaining inventory and the received payment for indicating commodity to be veritified In the inventory of commodity after the type of commodity and the whether consistent veritification result of the quantity of every kind of commodity, the output equipment is controlled Export the veritification result.

20. commodity apparatus for checking described in 8 or 19 according to claim 1, which is characterized in that the commodity apparatus for checking also wraps Include: channel barrier equipment is veritified host with the commodity and is connect, and the commodity veritify host and are also used to determining commodity to be veritified Inventory it is consistent with the quantity of the type of commodity in the inventory of received payment commodity and every kind of commodity when, control the channel barrier The open channel for entering and leaving shopping place of equipment.

21. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium Program instruction, when the computer program instructions are read out by the processor and run, perform claim is required described in any one of 1-15 Method the step of.