CN109359553A - Commodity detection method, device, computer equipment and the storage medium of fish eye images - Google Patents
Commodity detection method, device, computer equipment and the storage medium of fish eye images Download PDFInfo
- Publication number
- CN109359553A CN109359553A CN201811106155.8A CN201811106155A CN109359553A CN 109359553 A CN109359553 A CN 109359553A CN 201811106155 A CN201811106155 A CN 201811106155A CN 109359553 A CN109359553 A CN 109359553A
- Authority
- CN
- China
- Prior art keywords
- commodity
- detection
- fish eye
- eye images
- image data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/35—Categorising the entire scene, e.g. birthday party or wedding scene
- G06V20/36—Indoor scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
- G07G1/00—Cash registers
- G07G1/0036—Checkout procedures
- G07G1/0045—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The present invention is suitable for detection technique field, provide the commodity detection methods of fish eye images a kind of, device, computer equipment and can storage medium, the described method includes: acquisition commodity image data, and the commodity image data are screened, establish commodity training image collection；Data extending processing is carried out to the commodity training image collection；In detection network, Yolov2/v3 detection layers target cost item is removed；According to the Yolov2/v3 detection layers feature, each grid position is arranged different position and type convolution coefficient, classified part replaces normalization exponential function using S type function, and detection network does inhibition processing etc. to the non-maximal term of S type function output and detects to commodity.The present invention is by the adjustment to commodity detection framework, so that commodity detection performance and training effectiveness have different degrees of promotion, can be widely applied to the intelligence retail cabinet that machine vision Automatic-settlement is carried out using fish eye lens.
Description
Technical field
The present invention relates to detection technique field, more particularly to a kind of commodity detection method based on fish eye images, device,
Computer equipment and storage medium.
Background technique
In recent years with the burning hot development of depth learning technology, commodity detection algorithm is also calculated from the tradition based on manual feature
Method has turned to the detection technique based on deep neural network.In practical application in industry, due to the limitation of computation complexity, more
Using single step (one-stage) detection algorithm.
However, current commodity detection algorithm is usually the natural image for being used to handle analytic routines camera, for spy
Fixed fish-eye detection and analysis are seldom.Meanwhile the particular community of the commodity test problems of fixed scene is not also considered.
Due to fish-eye special nature, in different imaging positions, different spies can be presented in different perspectives in identical commodity
Point.Due to model size, the limitation of calculation amount, using general single step (one-stage) algorithm of target detection (such as SSD,
Yolov2/v3) there can be the problems such as computation complexity is high, detection accuracy is low.
Summary of the invention
The embodiment of the present invention provides a kind of commodity detection method of fish eye images, it is intended to solve problem above.
The embodiments of the present invention are implemented as follows, a kind of commodity detection method of fish eye images, which comprises
Commodity image data are acquired, and the commodity image data are screened, establish commodity training image collection；
Data extending processing is carried out to the commodity training image collection；
In detection network, Yolov2/v3 detection layers target cost item is removed；
According to the Yolov2/v3 detection layers feature, each grid position is arranged different position and type convolution coefficient, point
Class part replaces normalization exponential function using S type function, and detection network exports non-maximal term to S type function and does inhibition processing；
Based on the different location of current fish eye images, different candidate frame subsets are selected, wherein only one quotient of same candidate frame
Product；
Based on abnormality cost function, in the fish eye images output state exception, position and kind category zero setting optimize different
Normal status items, and complete commodity detection.
The embodiment of the invention also provides a kind of commodity detection device of fish eye images, described device includes:
Training set establishes unit, screens for acquiring commodity image data, and to the commodity image data, establishes commodity
Training image collection；
Data extending unit, for carrying out data extending processing to the commodity training image collection；
First processing units, for removing Yolov2/v3 detection layers target cost item in detection network；
The second processing unit, for each grid position being arranged different positions according to the Yolov2/v3 detection layers feature
With type convolution coefficient, classified part replaces normalization exponential function using S type function, and detection network exports S type function non-
Maximal term does inhibition processing；
Selecting unit selects different candidate frame subsets, wherein same candidate for the different location based on current fish eye images
Only one commodity of frame；And
Optimize unit, for being based on abnormality cost function, in the fish eye images output state exception, position and type
Item zero setting, optimizes abnormality item, and completes commodity detection.
The embodiment of the invention also provides a kind of computer equipment, the computer equipment includes memory and processor,
Computer program is stored in the memory, when the computer program is executed by the processor, so that the processor
The step of executing the commodity detection method of above-mentioned fish eye images.
The embodiment of the invention also provides a kind of computer readable storage medium, deposited on the computer readable storage medium
Computer program is contained, when the computer program is executed by processor, so that the processor executes above-mentioned fish eye images
The step of commodity detection method.
In the embodiment of the present invention, by acquiring commodity image data, and the commodity image data are screened, is established
Commodity training image collection, and data extending processing is carried out to the commodity training image collection；In detection network, removal
Yolov2/v3 detection layers target cost item；And according to the Yolov2/v3 detection layers feature, not to the setting of each grid position
Same position and type convolution coefficient, classified part replace normalization exponential function using S type function, detect network to S type letter
Number exports non-maximal term and does inhibition processing；Based on abnormality cost function, in the fish eye images output state exception, position
Category zero setting is set and planted, optimizes abnormality item, and then detect to commodity；The present invention passes through the tune to commodity detection framework
It is whole, so that commodity detection performance and training effectiveness have different degrees of promotion, meanwhile, algorithm used calculates simple, detection essence
Degree is high, can be widely applied to the intelligence retail cabinet that machine vision Automatic-settlement is carried out using fish eye lens.
Detailed description of the invention
Fig. 1 is the schematic diagram of the commodity whole detection frame system of fish eye images provided in an embodiment of the present invention；
Fig. 2 is a kind of implementation flow chart of the commodity detection method of fish eye images provided in an embodiment of the present invention；
Fig. 3 is the implementation flow chart of the commodity detection method of another fish eye images provided in an embodiment of the present invention；
Fig. 4 is the implementation flow chart of the commodity detection method of another fish eye images provided in an embodiment of the present invention；
Fig. 5 is a kind of structural schematic diagram of the commodity detection device of fish eye images provided in an embodiment of the present invention.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to the accompanying drawings and embodiments, right
The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and
It is not used in the restriction present invention.
The term used in embodiments of the present invention is only to be not intended to be limiting merely for for the purpose of describing particular embodiments
The present invention.Packet is also intended in the "an" and "the" of the embodiment of the present invention and singular used in the attached claims
Most forms are included, unless the context clearly indicates other meaning.It is also understood that term "and/or" used herein
Refer to and includes that one or more associated any or all of project listed may combine.
It will be appreciated that though various information may be described in embodiments of the present invention using term first, second etc., but
These information should not necessarily be limited by these terms.These terms are only used to for same type of information being distinguished from each other out.
The technical means and efficacy taken in order to which the present invention is further explained for the predetermined goal of the invention of realization, according to as follows
The commodity detection method of fish eye images provided in an embodiment of the present invention is described in detail in embodiment.
The commodity detection method of fish eye images provided by the invention, by being carried out at data extending to commodity training image collection
Reason；And in detection network, Yolov2/v3 detection layers target cost item is removed, and special according to the Yolov2/v3 detection layers
Point, each grid position is arranged different position and type convolution coefficient, and classified part is referred to using S type function instead of normalization
Number function, detection network export non-maximal term to S type function and do inhibition processing etc. to commodity；The present invention passes through to commodity detection block
The adjustment of frame can be widely applied to so that commodity detection performance and training effectiveness have different degrees of promotion using flake mirror
Head carries out the intelligence retail cabinet of machine vision Automatic-settlement.
Fig. 1 shows the commodity whole detection frame system structure of fish eye images of the invention, by scheming given detection
As being used as multiple scale detecting by the way that several detection layers are added, parallel is added two not after the layer of pond by trunk structure
Same linear layer is used as abnormality judgement and judgement of hazing；So commodity detection and shape can be completed at the same time by single network
State judges two functions.
Fig. 2 shows a kind of implementation processes of the commodity detection method of fish eye images provided in an embodiment of the present invention, are described in detail
It is as follows:
In step s 201, commodity image data are acquired, and the commodity image data are screened, establish commodity training figure
Image set.
In embodiments of the present invention, as shown in figure 3, the step S201, specifically includes:
In step S301, using the image comprising items list and without similar mutual exclusion commodity as positive sample data, and it will not wrap
The image of similar mutual exclusion commodity is included as negative sample data, obtains the first commodity image data.
In embodiments of the present invention, positive sample is containing the real scene shooting fish eye images that need to detect commodity, and negative sample is the flake
Other commodity images not similar with current commodity collection of shooting environmental shooting；Positive negative sample be related to allow neural network more
The good more accurate actual characteristic description for learning currently detect commodity out.
In step s 302,3D commodity are modeled, simulation shooting environmental rendering generates virtual image, to obtain second
Commodity image data.
In embodiments of the present invention, described that 3D commodity are modeled, it specifically includes: to commodity in 3D data collecting instrument
It scans, reconstructs true form, then by surface textures, construct complete commodity 3D model.There are the 3D model and flake
Lens distortion parameter then can virtually generate the commodity image that fish eye images are shot in cabinet.It is different in commodity posture, it claps
Take the photograph number it is relatively limited when, can by these 3D model virtuals generate data as commodity detection supplementary data set use, provide
Detection accuracy.
In step S303, typical target is detected into training set and part generates the screening image of erroneous detection, as third
Commodity image data.
In embodiments of the present invention, typical target detection training set (COCO, OpenImage, WiderFace etc.) is to use
Make the general public data collection of conventional general target detection algorithm assessment, this is no longer going to repeat them.
In step s 304, to the first commodity image data, the second commodity image data, third commodity image data
It is screened, establishes commodity training image collection.
In practical applications, when data loading is done in hands-on, above-mentioned three kinds of data are selected by different probability.It is loaded into
Refer to that, in training more new model, batch is loaded into (load) data and updates as neural network parameter.These image datas are positive
By neural network, all neural network parameter values then can be updated by this mechanism by reversed feedback；However, one
As training image number can be formed according to 2:1:1 ratio, when training image be much smaller than negative sample natural image when, can to training scheme
As being repeated as many times to meet ratio data.
In step S202, data extending processing is carried out to the commodity training image collection.In embodiments of the present invention, such as
Shown in Fig. 4, the step S202 is specifically included:
In step S401, the commodity training image is translated rotate or color disturbance expansion processing.
In step S402, the fuzzy noise processing of light weight is done the part commodity training image.
In step S403, haze simulation process the diffusion that the part commodity training image does light weight.
In step s 404, when external environment is excessively bright, the high optical analog of light weight is done the part commodity training image
Processing.
In step S405, high color is added the part commodity training image, is obscured, or processing of hazing, and is added
Abnormal label.
In step S203, in detection network, Yolov2/v3 detection layers target cost item is removed.
In embodiments of the present invention, remove Yolov2/v3 [2] detection layers target (objectness) cost item, due to
This judges the objective attribute target attribute of all commodity, difficult in lightweight trunk (backbone) structure.Because calculation amount
Limitation, it is generally not high that lightweight trunk structure exports dimension.It is all hundreds of for evaluating a region in this dimension
One of commodity to be detected are generally difficult simply to provide.In contrast, only judgement is that one of which can simply very
It is more.Lightweight trunk structure has Mobilenetv1/v2, Shufflenet etc., and general calculation amount is smaller.
In embodiments of the present invention, in backbone structure, image can not stop it is down-sampled, for example, 14x14 size 512
Dimension, 1024 dimension of 7x7 size.The simple layer that detection layers (Detection Layer) can be yolov2, i.e., in backbone
Finally, it is also possible to and the multiple dimensioned extraction layer 2-3 of yolov3 (such as the position 7x7 connects a detection layers, 7x7,14x14 feature merges
It is followed by a detection layers).Setting multilayer is typically due to target sizes and differs greatly, in network with some addition detection layers in front
Part Small object can be detected.
In embodiments of the present invention, Yolov2 detection layers can define cost function, such as the loss of frame positioning accuracy
(loss_xy), which is that the loss (loss_objectness) of target and target classification accurately lose (loss
Class), can refer to SSD scheme, eliminate target loss that；Meanwhile Classification Loss is replaced with sigmoid
softmax。
In step S204, according to the Yolov2/v3 detection layers feature, each grid position is arranged different positions
With type convolution coefficient, classified part replaces normalization exponential function using S type function, and detection network exports S type function non-
Maximal term does inhibition processing.
In embodiments of the present invention, different to the position each grid (grid) using the detection layers feature of Yolov2/v3
Position (Loc.) and type (Class) convolution coefficient, classified part using S type (Sigmoid) function replace normalization index
Function (Softmax), as it is assumed that same candidate frame (anchor) only one commodity, detection network is defeated to Sigmoid function
Non- maximal term, which is done, out inhibits.
In practical applications, it is assumed that trunk layer exports 7x7x512, it is assumed that anchor_size=K, 20 kinds of article, detection layers
Output is 7x7xKx(4+20), wherein 4 be location parameter, 20 be species parameter.General detection layers convolution kernel is 1x1, so want
Learn the deconvolution parameter of the matrix of one (512x (Kx(4+20)).Since fish eye images center and surrounding are widely different, so,
Several regions, each a different set of parameter of optimization of region can be divided into according to region from picture centre difference.
In practical applications, Softmax to all kinds of probability and is the number that the output of 1, S type is [- 1,1].Such as above-mentioned example,
Shared 49xK alternative frames have each commodity S type response output (20 dimensional vector) to each frame, can when assuming that commodity mutual exclusion
It is exported using the highest merchandise classification of Response to selection value as the alternative frame, the subsequent alternative frame root different to the spatial position 49xK again
It does and inhibits according to spatial position.
Further, it is based on focal loss cost function, processing is balanced to negative sample data.Use focal loss
(Focal Loss) cost function solves the problems, such as negative sample data nonbalance, is a general positive and negative sample imbalance solution party
Case reduces a simple easily point loss and realizes that for details, reference can be made to the prior arts by modifying Classification Loss function.
In step S205, based on the different location of current fish eye images, different candidate frame subsets are selected, wherein same
Only one commodity of candidate frame.
In embodiments of the present invention, selected based on the anchor of position and commodity: to image different location, selection is different
Anchor subset presets it and corresponds to optional anchor list to the different shape feature of different commodity；For example it defines
16x16,32x32,64x64 different candidate frames, can be according to commodity size property, selected section candidate frame；Meanwhile in image
Middle section selects bigger some candidate frame subsets (such as 32x32 and 64x64), in marginal portion, selects smaller one
Divide candidate frame subset (such as 16x16 and 32x32).
In step S206, be based on abnormality cost function, in the fish eye images output state exception, position and
Kind category zero setting, optimizes abnormality item, and completes commodity detection.
In embodiments of the present invention, abnormality cost function is added, in image output state exception (such as camera
It blocks, seriously hazes, automatic white balance, auto-focusing, automatic exposure (3A) deposits when abnormal, and Loc./Class zero setting are excellent
Change abnormality item；For example Binary Loss function (such as binary cross entropy) is defined, so that estimated state
(0,1) and virtual condition (0 or 1) solve.
Further, learnt using monocycle learning rate (One Cycle Learning Rate, CLR) optimization training process
Rate selects, newer general optimum selection strategy, and renewal learning rate and momentum(momentum are adjusted in adjusting training) value.
The commodity detection method of fish eye images provided in an embodiment of the present invention, by acquiring commodity image data, and to institute
It states commodity image data to be screened, establishes commodity training image collection, and data extending is carried out to the commodity training image collection
Processing；In detection network, Yolov2/v3 detection layers target cost item is removed；And it is special according to the Yolov2/v3 detection layers
Point, each grid position is arranged different position and type convolution coefficient, and classified part is referred to using S type function instead of normalization
Number function, detection network export non-maximal term to S type function and do inhibition processing；Based on abnormality cost function, in the fish
When eye image output state exception, position and kind category zero setting optimize abnormality item, and then detect to commodity；This hair
The bright adjustment by commodity detection framework, so that commodity detection performance and training effectiveness have different degrees of promotion, it can be wide
The general intelligence retail cabinet for being applied to carry out machine vision Automatic-settlement using fish eye lens.
Above-mentioned method is described further with concrete application example below.
Specific invention embodiment: the intelligence of the machine vision monitored using fish-eye camera is sold unmanned refrigerator-freezer.
1. every layer of commodity price of pair refrigerator-freezer monitors type of merchandize and quantity on every layer of shelf using fish-eye camera；
2. after subscriber authentication, starting shopping of opening the door.After user closes the door, intelligent cabinet locking, camera starts to shoot picture；
3. the picture that each camera is shot carries out intelligent Target detection and can be used as the nerve net inferred after being sent into training
In network model, testing result is obtained；
4. will test result to be sent into whole detection frame system of the present invention, all commodity are detected, detection block is marked, then counts
Number is cumulative to obtain surplus commodities sum.
5. the type and quantity of the last transaction surplus commodities of comparison, obtain this trade user shopping items and quantity.
The present invention can promote commodity in the relatively low situation of calculation amount after to the modification of existing commodity detection framework
Detection accuracy.
Fig. 5 shows a kind of commodity detection device 500 of fish eye images provided in an embodiment of the present invention, and details are as follows:
The commodity detection device 500 of fish eye images includes: that training set establishes unit 501, the processing of data extending unit 502, first
Unit 503, the second processing unit 504, selecting unit 505 and optimization unit 506.
Training set establishes unit 501, screens for acquiring commodity image data, and to the commodity image data,
Establish commodity training image collection.
In embodiments of the present invention, training set establishes unit 501 for acquiring commodity image data, and to the commodity figure
As data are screened, commodity training image collection is established；Wherein, the acquisition commodity image data, and to the commodity image
Data are screened, and are established commodity training image collection, are specifically included: by the image comprising items list and without similar mutual exclusion commodity
It as positive sample data, and will not include the image of similar mutual exclusion commodity as negative sample data, the first commodity image number of acquisition
According to；3D commodity are modeled, simulation shooting environmental rendering generates virtual image, to obtain the second commodity image data；By allusion quotation
Type target detection training set and part generate the screening image of erroneous detection, as third commodity image data；To first quotient
Product image data, the second commodity image data, third commodity image data are screened, and commodity training image collection is established.
Data extending unit 502, for carrying out data extending processing to the commodity training image collection.
In embodiments of the present invention, data extending unit 502 is used to carry out data extending to the commodity training image collection
Processing；Wherein, data extending processing is carried out to the commodity training image collection, specifically includes: the commodity training image is done
Translation or rotation or color disturbance expansion processing；The fuzzy noise processing of light weight is done the part commodity training image；It is right
The diffusion that the part commodity training image does light weight is hazed simulation process；When external environment is excessively bright, the commodity described part
Training image does the bloom simulation process of light weight；The part commodity training image is added high color, obscures, or processing of hazing,
And abnormal label is added.
First processing units 503, for removing Yolov2/v3 detection layers target cost item in detection network.
In embodiments of the present invention, first processing units 503 are used in detection network, remove Yolov2/v3 detection layers
Target cost item；Wherein, Yolov2 detection layers can define cost function, such as the loss (loss_xy) of frame positioning accuracy, should
Position is that the loss (loss_objectness) of target and target classification accurately lose (loss class), can be referred to
SSD scheme, eliminate target loss that；Meanwhile Classification Loss replaces softmax with sigmoid.
The second processing unit 504, for according to the Yolov2/v3 detection layers feature, each grid position to be arranged not
Same position and type convolution coefficient, classified part replace normalization exponential function using S type function, detect network to S type letter
Number exports non-maximal term and does inhibition processing.
In embodiments of the present invention, the second processing unit 504 is used for according to the Yolov2/v3 detection layers feature, to every
Different position and type convolution coefficient is arranged in a grid position, and classified part replaces normalization exponential function using S type function,
Detection network exports non-maximal term to S type function and does inhibition processing；Using the detection layers feature of Yolov2/v3, to each grid
(grid) there are different position (Loc.) and type (Class) convolution coefficient in position, and classified part uses S type (Sigmoid) letter
Number replaces normalization exponential function (Softmax), as it is assumed that same candidate frame (anchor) only one commodity, detect network
Is exported by non-maximal term and is done for Sigmoid function and is inhibited.
Selecting unit 505 selects different candidate frame subsets for the different location based on current fish eye images, wherein same
Only one commodity of one candidate frame.
In embodiments of the present invention, selecting unit 505 is used for the different location based on current fish eye images, selects different times
Select frame collection, wherein only one commodity of same candidate frame；It is selected based on the anchor of position and commodity: to image difference position
It sets, selects different anchor subsets, to the different shape feature of different commodity, preset it and correspond to optional anchor list；Than
16x16 is such as defined, 32x32,64x64 different candidate frames can be according to commodity size property, selected section candidate frame；Together
When, region is entreated in the picture, selects bigger some candidate frame subsets (such as 32x32 and 64x64), in marginal portion, selection
Smaller a part of candidate frame subset (such as 16x16 and 32x32).
Optimize unit 506, for being based on abnormality cost function, in the fish eye images output state exception, position
Category zero setting is set and planted, abnormality item is optimized, and completes commodity detection.
In embodiments of the present invention, optimization unit 506 is used to be based on abnormality cost function, defeated in the fish eye images
When doing well abnormal, position and kind category zero setting optimize abnormality item, and complete commodity detection；Abnormality cost is added
Function, and in image output state exception (such as camera blocks, and seriously hazes, automatic white balance, auto-focusing, it is automatic to expose
Light (3A) is deposited when abnormal, Loc./Class zero setting, optimizes abnormality item；Such as define Binary Loss function (such as
Binary cross entropy) so that estimated state (0,1) and virtual condition (0 or 1) solve.
The commodity detection device of fish eye images provided in an embodiment of the present invention, by acquiring commodity image data, and to institute
It states commodity image data to be screened, establishes commodity training image collection, and data extending is carried out to the commodity training image collection
Processing；In detection network, Yolov2/v3 detection layers target cost item is removed；And it is special according to the Yolov2/v3 detection layers
Point, each grid position is arranged different position and type convolution coefficient, and classified part is referred to using S type function instead of normalization
Number function, detection network export non-maximal term to S type function and do inhibition processing；Based on abnormality cost function, in the fish
When eye image output state exception, position and kind category zero setting optimize abnormality item, and then detect to commodity；This hair
The bright adjustment by commodity detection framework, so that commodity detection performance and training effectiveness have different degrees of promotion, it can be wide
The general intelligence retail cabinet for being applied to carry out machine vision Automatic-settlement using fish eye lens.
The embodiment of the invention also provides a kind of computer equipment, which includes processor, and processor is used for
The commodity detection for the fish eye images that above-mentioned each embodiment of the method provides is realized when executing the computer program stored in memory
The step of method.
The embodiments of the present invention also provide a kind of computer readable storage medium, it is stored thereon with computer program/refer to
It enables, which realizes the fish eye images that above-mentioned each embodiment of the method provides when being executed by above-mentioned processor
The step of commodity detection method.
Illustratively, computer program can be divided into one or more modules, one or more module is stored
In memory, and by processor it executes, to complete the present invention.One or more modules, which can be, can complete specific function
Series of computation machine program instruction section, the instruction segment is for describing implementation procedure of the computer program in computer installation.Example
Such as, the computer program can be divided into the commodity detection method for the fish eye images that above-mentioned each embodiment of the method provides
Step.
It will be understood by those skilled in the art that the description of above-mentioned computer installation is only example, do not constitute to calculating
The restriction of machine device may include component more more or fewer than foregoing description, perhaps combine certain components or different portions
Part, such as may include input-output equipment, network access equipment, bus etc..
Alleged processor can be central processing unit (Central Processing Unit, CPU), can also be it
His general processor, digital signal processor (Digital Signal Processor, DSP), specific integrated circuit
(Application Specific Integrated Circuit, ASIC), ready-made programmable gate array (Field-
Programmable Gate Array, FPGA) either other programmable logic device, discrete gate or transistor logic,
Discrete hardware components etc..General processor can be microprocessor or the processor is also possible to any conventional processor
Deng the processor is the control centre of the computer installation, utilizes various interfaces and the entire user terminal of connection
Various pieces.
The memory can be used for storing the computer program and/or module, and the processor is by operation or executes
Computer program in the memory and/or module are stored, and calls the data being stored in memory, described in realization
The various functions of computer installation.The memory can mainly include storing program area and storage data area, wherein storage program
It area can application program (such as sound-playing function, image player function etc.) needed for storage program area, at least one function
Deng；Storage data area, which can be stored, uses created data (such as audio data, phone directory etc.) etc. according to mobile phone.In addition,
Memory may include high-speed random access memory, can also include nonvolatile memory, such as hard disk, memory, grafting
Formula hard disk, intelligent memory card (Smart Media Card, SMC), secure digital (Secure Digital, SD) card, flash memory
Block (Flash Card), at least one disk memory, flush memory device or other volatile solid-state parts.
If the integrated module/unit of the computer equipment is realized in the form of SFU software functional unit and as independent
Product when selling or using, can store in a computer readable storage medium.Based on this understanding, the present invention is real
All or part of the process in existing above-described embodiment method, can also instruct relevant hardware come complete by computer program
At the computer program can be stored in a computer readable storage medium, which is being executed by processor
When, it can be achieved that the step of above-mentioned each embodiment of the method.Wherein, the computer program includes computer program code, described
Computer program code can be source code form, object identification code form, executable file or certain intermediate forms etc..The meter
Calculation machine readable medium may include: can carry the computer program code any entity or device, recording medium, USB flash disk,
Mobile hard disk, magnetic disk, CD, computer storage, read-only memory (ROM, Read-Only Memory), random access memory
Device (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium etc..
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all in essence of the invention
Made any modifications, equivalent replacements, and improvements etc., should all be included in the protection scope of the present invention within mind and principle.
Claims (8)
1. a kind of commodity detection method of fish eye images, which is characterized in that the described method includes:
Commodity image data are acquired, and the commodity image data are screened, establish commodity training image collection；
Data extending processing is carried out to the commodity training image collection；
In detection network, Yolov2/v3 detection layers target cost item is removed；
According to the Yolov2/v3 detection layers feature, each grid position is arranged different position and type convolution coefficient, point
Class part replaces normalization exponential function using S type function, and detection network exports non-maximal term to S type function and does inhibition processing；
Based on the different location of current fish eye images, different candidate frame subsets are selected, wherein only one quotient of same candidate frame
Product；
Based on abnormality cost function, in the fish eye images output state exception, position and kind category zero setting optimize different
Normal status items, and complete commodity detection.
2. the commodity detection method of fish eye images according to claim 1, which is characterized in that the acquisition commodity image number
According to, and the commodity image data are screened, commodity training image collection is established, is specifically included:
It using the image comprising items list and without similar mutual exclusion commodity as positive sample data, and will not include similar mutual exclusion commodity
Image as negative sample data, obtain the first commodity image data；
3D commodity are modeled, simulation shooting environmental rendering generates virtual image, to obtain the second commodity image data；
Typical target is detected into training set and part generates the screening image of erroneous detection, as third commodity image data；
The first commodity image data, the second commodity image data, third commodity image data are screened, commodity are established
Training image collection.
3. the commodity detection method of fish eye images according to claim 2, which is characterized in that the method also includes:
Based on focal loss cost function, processing is balanced to negative sample data.
4. the commodity detection method of fish eye images according to claim 1, which is characterized in that described to the commodity training
Image set carries out data extending processing, specifically includes:
The commodity training image is translated or rotate or color disturbance expansion processing；
The fuzzy noise processing of light weight is done the part commodity training image；
It hazes simulation process the diffusion that the part commodity training image does light weight；
When external environment is excessively bright, the bloom simulation process of light weight is done the part commodity training image；
High color is added the part commodity training image, is obscured, or processing of hazing, and abnormal label is added.
5. the commodity detection method of fish eye images according to claim 1, which is characterized in that the fish eye images export shape
State abnormal phenomenon, including camera is blocked, is seriously hazed, automatic white balance, auto-focusing and automatic exposure are in the presence of abnormal existing
As.
6. a kind of commodity detection device of fish eye images, which is characterized in that described device includes:
Training set establishes unit, screens for acquiring commodity image data, and to the commodity image data, establishes commodity
Training image collection；
Data extending unit, for carrying out data extending processing to the commodity training image collection；
First processing units, for removing Yolov2/v3 detection layers target cost item in detection network；
The second processing unit, for each grid position being arranged different positions according to the Yolov2/v3 detection layers feature
With type convolution coefficient, classified part replaces normalization exponential function using S type function, and detection network exports S type function non-
Maximal term does inhibition processing；
Selecting unit selects different candidate frame subsets, wherein same candidate for the different location based on current fish eye images
Only one commodity of frame；And optimization unit, it is different in the fish eye images output state for being based on abnormality cost function
Chang Shi, position and kind category zero setting, optimize abnormality item, and complete commodity detection.
7. a kind of computer equipment, which is characterized in that the computer equipment includes memory and processor, in the memory
It is stored with computer program, when the computer program is executed by the processor, so that the processor perform claim requires 1
To fish eye images described in any one of 4 claims commodity detection method the step of.
8. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium
Program, when the computer program is executed by processor, so that the processor perform claim requires any one of 1 to 4 right
It is required that the step of commodity detection method of the fish eye images.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811106155.8A CN109359553A (en) | 2018-09-21 | 2018-09-21 | Commodity detection method, device, computer equipment and the storage medium of fish eye images |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811106155.8A CN109359553A (en) | 2018-09-21 | 2018-09-21 | Commodity detection method, device, computer equipment and the storage medium of fish eye images |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109359553A true CN109359553A (en) | 2019-02-19 |
Family
ID=65351186
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811106155.8A Pending CN109359553A (en) | 2018-09-21 | 2018-09-21 | Commodity detection method, device, computer equipment and the storage medium of fish eye images |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109359553A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109828578A (en) * | 2019-02-22 | 2019-05-31 | 南京天创电子技术有限公司 | A kind of instrument crusing robot optimal route planing method based on YOLOv3 |
| CN110889829A (en) * | 2019-11-09 | 2020-03-17 | 东华大学 | Monocular distance measurement method based on fisheye lens |
| CN111144417A (en) * | 2019-12-27 | 2020-05-12 | 创新奇智(重庆)科技有限公司 | Intelligent container small target detection method and detection system based on teacher student network |
| CN111353526A (en) * | 2020-02-19 | 2020-06-30 | 上海小萌科技有限公司 | Image matching method and device and related equipment |
| CN116385723A (en) * | 2023-04-11 | 2023-07-04 | 特斯联科技集团有限公司 | Intelligent retail system and method based on indoor positioning |
| CN117422937A (en) * | 2023-12-18 | 2024-01-19 | 成都阿加犀智能科技有限公司 | Intelligent shopping cart state identification method, device, equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108320510A (en) * | 2018-04-03 | 2018-07-24 | 深圳市智绘科技有限公司 | One kind being based on unmanned plane video traffic information statistical method and system |
| CN108416283A (en) * | 2018-02-28 | 2018-08-17 | 华南理工大学 | A kind of pavement marking recognition methods based on SSD |
| CN108520594A (en) * | 2018-07-10 | 2018-09-11 | 合肥美的智能科技有限公司 | Vending machine and control method thereof |
-
2018
- 2018-09-21 CN CN201811106155.8A patent/CN109359553A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108416283A (en) * | 2018-02-28 | 2018-08-17 | 华南理工大学 | A kind of pavement marking recognition methods based on SSD |
| CN108320510A (en) * | 2018-04-03 | 2018-07-24 | 深圳市智绘科技有限公司 | One kind being based on unmanned plane video traffic information statistical method and system |
| CN108520594A (en) * | 2018-07-10 | 2018-09-11 | 合肥美的智能科技有限公司 | Vending machine and control method thereof |
Non-Patent Citations (4)
| Title |
|---|
| JOSEPH REDMON E.T.: "YOLOv3: An Incremental Improvement", 《ARXIV:1804.02767》 * |
| LIZHENG LIU E.T.: "A Smart Unstaffed Retail Shop Based on Artificial Intelligence and IoT", 《2018 IEEE 23RD INTERNATIONAL WORKSHOP ON COMPUTER AIDED MODELING AND DESIGN OF COMMUNICATION LINKS AND NETWORKS(CAMAD)》 * |
| TSUNG-YI LIN E.T.: "Focal Loss for Dense Object Detection", 《IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE》 * |
| 魏源璋: "基于深度协同神经网络的无人机避障系统设计与实现", 《中国优秀硕士学位论文全文数据库 工程科技II辑》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109828578A (en) * | 2019-02-22 | 2019-05-31 | 南京天创电子技术有限公司 | A kind of instrument crusing robot optimal route planing method based on YOLOv3 |
| CN109828578B (en) * | 2019-02-22 | 2020-06-16 | 南京天创电子技术有限公司 | Instrument inspection robot optimal route planning method based on YOLOv3 |
| CN110889829A (en) * | 2019-11-09 | 2020-03-17 | 东华大学 | Monocular distance measurement method based on fisheye lens |
| CN110889829B (en) * | 2019-11-09 | 2023-11-03 | 东华大学 | Monocular distance measurement method based on fish eye lens |
| CN111144417A (en) * | 2019-12-27 | 2020-05-12 | 创新奇智(重庆)科技有限公司 | Intelligent container small target detection method and detection system based on teacher student network |
| CN111353526A (en) * | 2020-02-19 | 2020-06-30 | 上海小萌科技有限公司 | Image matching method and device and related equipment |
| CN116385723A (en) * | 2023-04-11 | 2023-07-04 | 特斯联科技集团有限公司 | Intelligent retail system and method based on indoor positioning |
| CN116385723B (en) * | 2023-04-11 | 2023-09-15 | 特斯联科技集团有限公司 | Intelligent retail system and method based on indoor positioning |
| CN117422937A (en) * | 2023-12-18 | 2024-01-19 | 成都阿加犀智能科技有限公司 | Intelligent shopping cart state identification method, device, equipment and storage medium |
| CN117422937B (en) * | 2023-12-18 | 2024-03-15 | 成都阿加犀智能科技有限公司 | Intelligent shopping cart state identification method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109359553A (en) | Commodity detection method, device, computer equipment and the storage medium of fish eye images | |
| CN104115161B (en) | Method and system for movement images | |
| Hussin et al. | Digital image processing techniques for object detection from complex background image | |
| Tursun et al. | An objective deghosting quality metric for HDR images | |
| CN108108754A (en) | The training of identification network, again recognition methods, device and system again | |
| CN110210560A (en) | Increment training method, classification method and the device of sorter network, equipment and medium | |
| US11416718B2 (en) | Item identification method, device and system based on vision and gravity sensing | |
| Pound et al. | A patch-based approach to 3D plant shoot phenotyping | |
| CN109726756A (en) | Image processing method, device, electronic equipment and storage medium | |
| CN111696080A (en) | Face fraud detection method, system and storage medium based on static texture | |
| CN111126264A (en) | Image processing method, device, equipment and storage medium | |
| CN113160313A (en) | Transparent object grabbing control method and device, terminal and storage medium | |
| Kirkegaard et al. | Bin-picking based on harmonic shape contexts and graph-based matching | |
| CN111506755A (en) | Picture set classification method and device | |
| CN109741380A (en) | Textile picture fast matching method and device | |
| CN109784379A (en) | The update method and device in textile picture feature library | |
| CN108932703A (en) | Image processing method, picture processing unit and terminal device | |
| CN114830145A (en) | Object analysis model learning device and method based on data enhancement | |
| CN109657083A (en) | The method for building up and device in textile picture feature library | |
| CN112488985A (en) | Image quality determination method, device and equipment | |
| US20220358752A1 (en) | Apparatus and method for developing space analysis model based on data augmentation | |
| CN116434303A (en) | Facial expression capturing method, device and medium based on multi-scale feature fusion | |
| CN115630660A (en) | Barcode positioning method and device based on convolutional neural network | |
| CN108182406A (en) | The article display recognition methods of retail terminal and system | |
| Mian et al. | A novel algorithm for automatic 3D model-based free-form object recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190219 |