CN108597582A - A kind of method and apparatus for executing Faster R-CNN neural network computings - Google Patents
A kind of method and apparatus for executing Faster R-CNN neural network computings Download PDFInfo
- Publication number
- CN108597582A CN108597582A CN201810352111.7A CN201810352111A CN108597582A CN 108597582 A CN108597582 A CN 108597582A CN 201810352111 A CN201810352111 A CN 201810352111A CN 108597582 A CN108597582 A CN 108597582A
- Authority
- CN
- China
- Prior art keywords
- food
- cnn
- volume
- volume ratio
- food item
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
- G16H20/60—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to nutrition control, e.g. diets
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Nutrition Science (AREA)
- Epidemiology (AREA)
- Medical Informatics (AREA)
- Primary Health Care (AREA)
- Public Health (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Image Analysis (AREA)
Abstract
A kind of method and apparatus for executing Faster R CNN neural network computings, the method includes：Obtain the multiple images of the different angle with serving；The recommendation region of pattern detection is determined using RPN；Recommend the classification and frame of food item in region using Fast R CNN predictions；According to the frame of the food of prediction, the volume ratio shared by each food item is predicted using Volume R CNN；According to food item classification and food item volume ratio, the volume ratio of different classes of food is calculated；The volume ratio of calculated various kinds of foods is multiplied respectively with the density of various kinds of foods, obtains the mass ratio of various kinds of foods；The mass ratio of various kinds of foods is multiplied with the gross mass of food, acquires the quality of various kinds of foods；The quality of various kinds of foods is multiplied with corresponding nutrient content, obtains food nutrition constituent content.The present invention can measure complicated, multiple types foods, by using artificial neural network technology and chip, more accurately and quickly to the identification of food.
Description
Technical field
The present invention relates to technical field of image processing, and in particular to one kind is for executing Faster R-CNN neural networks fortune
The method and apparatus of calculation.
Background technology
As the quickening of modern society's life rhythm and people's living standard improve, requirement of the people for diet is also got over
Come higher.Can people are no longer concerned about have enough, but whether focus on diet healthy.However to lack enough diet strong by many people
Kang Zhishi, so just needing a kind of can intelligently measure the equipment of food energy and nutritional ingredient to help people more reasonably to drink
Food.
One of prior art is to calculate food energy by weight.Metering device is mainly made of following sections：Pallet,
Weight-measuring device and display screen.Weight-measuring device passes to food weight information micro- for measuring food weight
Computer processor；Processor calculates after food energy that the results are shown on liquid crystal display.
The problem of above-mentioned technology, is that processing capacity is weak, is only suitable for measuring the energy of single diet type.For comprising more
The food of kind food qualification category measures just inaccuracy, can not also calculate corresponding nutritive element content.Do not have expansion, for not
The food information of typing can not then be handled.
Another prior art is that the vertical view and side view that measure food are shot by mobile phone, passes through artificial neural network
It identifies food species, the volume of each food is calculated further according to formula；Its nutritive element content is calculated according to food volume.
The problem of above-mentioned technology, is complicated for operation, requires input photo high；Side view is easy blocking there are food
Problem.The parameters such as food length, width and height are predicted using focal length, and for different mobile phones, there may be certain deviations, use equation meter
Food volume is calculated, is not suitable for food in irregular shape, error calculated is larger.
Invention content
In order to solve the problems in the existing technology, on the one hand, the present invention provides one kind for executing Faster R-
The method of CNN neural network computings, including：
Obtain the multiple images of the different angle with serving；
The recommendation region of pattern detection is determined using RPN；
Recommend the classification and frame of food item in region using Fast R-CNN predictions；
According to the frame of the food of prediction, the volume ratio shared by each food item is predicted using Volume R-CNN；
According to the food item classification of Fast R-CNN predictions and the food item volume ratio of Volume R-CNN predictions
Example, calculates the volume ratio of different classes of food；
The volume ratio of calculated various kinds of foods is multiplied respectively with the density of various kinds of foods, obtains the matter of various kinds of foods
Amount ratio；
The mass ratio of various kinds of foods is multiplied with the gross mass of food, acquires the quality of various kinds of foods；
The quality of various kinds of foods is multiplied with corresponding nutrient content, obtains food nutrition constituent content；
Wherein, the RPN, Fast R-CNN and Volume R-CNN share convolutional layer.
Preferably, determining that the RPN carries out multilayer convolution operation, extraction to the picture of input first when recommending region
The Feature Mapping for going out picture, reuse sliding window to Feature Mapping carry out convolution operation, then use Classification Loss function and
Frame returns loss function Liang Ge branches and comes zoning classification and region recurrence, obtains and recommends region.
Preferably, the Fast R-CNN will recommend area maps to the Feature Mapping to obtain RoIs, then to each RoI
Pondization operation is carried out, the characteristic pattern of same size is converted into, the RoIs after then being operated to pondization carries out two full connections respectively
Network operations calculate the food item classification in each recommendation region and are accurately predicted frame.
Preferably, the frame parameter of prediction is mapped in the Feature Mapping by the Volume R-CNN, then to corresponding
Mapping area carry out pondization operation, become the sample areas of same size, then to each sample areas progress multilayer connect entirely
Network operations are connect, the volume intermediate variable v of each food item in figure is calculatedi, viFor positive number；It then again will be in the volume
Between variables transformations be corresponding volume ratio fi, calculation formula is：Wherein i=1,2 ... n, n are to be eaten in image
The number of object object.
Preferably, it is by the method that the frame parameter of prediction is mapped on the characteristic pattern：Each coordinate data is multiplied by
The ratio between the size of characteristic pattern and original image.
Preferably, the form of loss function Volume loss is in the Volume R-CNNWherein fiFor the volume ratio of each food item of prediction, fi *For actual value, instruction
The flag data inputted when practicing.
Preferably, the output of neural network includes during prediction：By Volume R-CNN, calculated one indicates figure
The n-dimensional vector of the shared volume ratio of each food item as in, each element be located at section (0,1] interior, and the sum of each element
It is 1；By calculated one n*m matrix for indicating each food item generic in image of Fast R-CNN, m is recognizable
The class number of food item, it is 1 that matrix, which only has an element per a line, remaining m-1 element is 0, and the corresponding row of element 1 are
For food item generic；And the two-dimensional array of a n*4 for indicating each food item frame.
Preferably, the algorithm further includes that will indicate that the n-dimensional vector of each food item volume ratio is multiplied by indicate each
The n*m two-dimensional arrays of food item generic obtain the volume ratio vector of various kinds of foods, are m dimensional vectors, the m dimensional vectors
Per one-dimensional correspondence one group food, indicate the volume ratio shared by corresponding classification food per the numerical value on one-dimensional.
Preferably, the method further includes calculating a m for indicating food volume ratio of all categories to each image
Then all m dimensional vectors are added again divided by the number of image by dimensional vector again, find out average vector as food of all categories finally
The volume ratio vector of object.
Preferably, the method further includes adaptivity training step, including：
Step 1, RPN netinit network parameters calculate each detection according to the image information propagated forward of input
The class label and region adjusting parameter in region；Come using stochastic gradient descent algorithm or Adam algorithms further according to backpropagation
The relevant parameter for updating RPN includes the parameter of the unique portion parameter of RPN and shared conventional part, and training is until convergence；
Step 2, Fast R-CNN utilize the shared convolutional layer parameter initialization convolution layer parameter of step 1 training, and will
The recommendation region that step 1 obtains is trained as the recommendation region during neural computing, and updates network parameter,
Including shared convolutional network, until network convergence；
Step 3, the shared convolutional network that RPN networks are obtained using step 2 continue to train and update the only of RPN networks
There are partial parameters, does not include the convolution layer parameter shared；
Step 4, the recommendation region that Fast R-CNN Web vector graphic step 3 obtains is trained, and only updates Fast
The exclusive part of R-CNN networks, shared convolutional layer parameter constant；
The food item frame that step 4 obtains is mapped to shared convolutional network by step 5, Volume R-CNN networks
Last layer of Feature Mapping, and the exclusive partial parameters of update are trained, until network convergence；
The training operation of each step is that input data is obtained the loss function of each part by network forward calculation,
Backpropagation again updates network parameter using stochastic gradient descent or Adam algorithms；
The training process of wherein above-mentioned five steps can recycle execution.
On the other hand, the present invention provides a kind of devices for executing Faster R-CNN neural network computings, including
Information input unit obtains different in the multiple images, the gross mass of food, food of the different angle with serving
The density and nutritive element content of classification food；
Information treatment part, for being handled described image and being calculated；
Wherein described information processing unit includes：
Storage unit, for storing described image, gross mass, density and nutritive element content；
Recommend region generation unit, the recommendation region of pattern detection is determined using RPN；
Classification and frame predicting unit recommend the classification and frame of food item in region using Fast R-CNN predictions；
Food item volume ratio predicting unit, it is pre- using Volume R-CNN according to the frame of the food item of prediction
Volume ratio in altimetric image shared by each food item；
Food qualification category volume ratio predicting unit, according to the food item classification and Volume of Fast R-CNN predictions
The food item volume ratio of R-CNN predictions, calculates the volume ratio shared by each classification food, and to the meter of different images
Results are averaged for calculation；
Mass ratio predicting unit, by the density of the volume ratio and different classes of food of calculated different classes of food
It is multiplied respectively, obtains the mass ratio of different classes of food；
The mass ratio of different classes of food is multiplied with the gross mass of food, acquires difference by quality of food predicting unit
The quality of classification food；And
The quality of various kinds of foods is multiplied with corresponding nutrient content, obtains food nutrition by nutrient content predicting unit
Constituent content；
Wherein, the RPN, Fast R-CNN and Volume R-CNN share convolutional layer.
Preferably, described information input unit includes image-input device and mass input device.
Preferably, described information processing unit further includes Date Conversion Unit, and the q dimension nutrition for exporting processing unit contains
Amount vector is converted into corresponding output.
Preferably, described device further includes information output part, for receiving output information from described information processing unit, and will
Presentation of information comes out.
Preferably, described device further includes networked components, for measurement data to be uploaded to database in real time, while also may be used
Most recent parameters model is updated from high in the clouds.
Preferably, described information processing unit is neural network chip.
Compared with prior art, the invention has the advantages that：
1) compared to existing invention, increasingly complex, a greater variety of foods can be measured.
2) artificial neural network technology and chip are used, it is more accurate to the identification of food, quickly.
3) oblique upper is used to overlook picture, it is possible to prevente effectively from blocking between different foods, while having to practical judgment
More comprehensive understanding.
4) artificial neural network technology and chip is used to calculate food volume, result of calculation is more accurate and with training
Data are continuously increased, and precision of prediction can also improve.
5) artificial neural network chip computing capability is powerful, off-line operation neural network is supported, in no cloud server
Assist the work that user terminal/front end can be realized food nourishment composition detection and accordingly controls offline in the case of calculating.When
Chip is networked, and when obtaining cloud server assistance calculating, chip computing capability is more powerful.
6) device is easy to operate, more intelligent, meets people's daily life demand.
7) suggestion more rationalized can be provided with the diet of people, improves people's quality of life.
Description of the drawings
Fig. 1 is the structure chart of neural network of the embodiment of the present invention；
Fig. 2 is the prognostic chart of food item classification and frame in the embodiment of the present invention；
Fig. 3 is the network structure of Volume R-CNN in the embodiment of the present invention；
Fig. 4 is the structural schematic diagram of device in the embodiment of the present invention.
Specific implementation mode
To make the objectives, technical solutions, and advantages of the present invention clearer, below in conjunction with specific embodiment, and reference
Attached drawing, the present invention is described in further detail.
A kind of method for executing Faster R-CNN neural network computings of the present invention mainly includes the following steps that：It is right
The key feature of image is extracted and is handled, and identifies the type of food and various kinds of foods volume proportion in image；Again
The weight ratio of various kinds of foods is calculated according to the density of various kinds of foods；Last processing unit is according to the weight ratio of various kinds of foods
The actual mass that tested various kinds of foods is calculated with total weight, food can be found out in conjunction with the constituent content table of various foods
Energy and nutritional ingredient.
Input picture includes the top view photograph to multiple different angles of same serving.
To the processing stage of single image, processor Faster Region improved to the imagery exploitation of input
Convolutional Neural Network (faster R-CNN) network is calculated, and each food in image is marked
The prediction volume ratio (volume) of not (class), each food frame (bounding box), and each food.Wherein
Volume is expressed as the two-decimal of 0-1.
Neural network structure in the present invention is improved on the basis of Faster R-CNN networks, and prediction is added to
The part of food volume ratio.Network structure is as shown in Figure 1：
The neural network can be divided into three parts：Region Proposal Networks for predicting recommendation region
(RPN) network；For the classification of object in prognostic chart picture and the Fast R-CNN networks of accurate adjustment frame；For each in prognostic chart picture
The Volume RCNN networks of volume ratio shared by a food item.Three network share convolutional layers, constitute one it is unified whole
Volume grid.
The frame of food be refer to include some food of image minimum rectangle frame.It is specifically as shown in Fig. 2, oval in figure
Food of different shapes is indicated with irregular figure, and dotted border is food frame.
Fig. 3 is Volume R-CNN network structures, and wherein convolutional layer (CNN) is shared part.Needed for this partial arithmetic
Frame is obtained by second part (Fast R-CNN).The form of loss function Volume loss isWherein fiFor the volume ratio of each object of prediction, fi *For actual value, when training
The flag data of input.
This nerve Web vector graphic Region Proposal Networks (RPN) come determine target detection recommend region.RPN
Method first carries out multilayer convolution operation to the image of input, extracts the Feature Mapping (feature maps) of image, reuses
The sliding window of 3*3 carries out convolution operation to Feature Mapping, carrys out zoning classification using Liang Ge branches later and region is returned
Return, obtains and recommend region.Here territorial classification judges that estimation range belongs to the probability of foreground and background；Here recommendation
The parameter in region is the parameter relative to original input picture.
In order to predict it is each recommend region food qualification category and by food frame accurate adjustment, so will recommend area maps extremely
Above-mentioned feature maps obtain RoIs (Region of Interest), then carry out pondization operation to each RoI, are converted into same
Etc. sizes characteristic pattern.It can carry out two fully-connected network operations respectively to the RoIs after pond later, calculate every
The food qualification category in a region and frame is accurately predicted.
Finally, then by the frame parameter of frame branch prediction it is mapped on feature maps, to corresponding map section
Domain carries out pondization operation, becomes the region of same size.The full attended operation of multilayer is carried out to each target area again, is calculated every
The volume intermediate variable of a food, intermediate variable is positive number, it is not intended that food volume.Include one in each target area
A food, a corresponding volume intermediate variable vi, then converting volume intermediate variable to corresponding ratio fi；Specific formula
For：Wherein i=1,2 ... n, n are the number of food item in image.fiNumber be food in image
Number n, it is possible to which it is a vector for including n element to export food volume ratio.Food qualification category prediction output be
The two-dimensional matrix of one n*m, wherein m are the quantity of comestible classification, one and only one element of each row vector of matrix is 1,
Remaining element is 0；1 place columns of element corresponds to the classification of food.The output of food frame predicted branches is the two dimension of a n*4
Matrix, the element per a line correspond to the centre coordinate (x, y) of frame, high width (h, w) respectively.Frame is mapped to from original image
Operation on characteristic pattern is：Each coordinate data is multiplied by the ratio between the size of characteristic pattern and original image.
So the output of neural network is during prediction：One n-dimensional vector for indicating each food volume ratio, often
A element is located in section [0,1], and the sum of each element is 1；One n*m two-dimensional array for indicating each food generic；
The two-dimensional array of one n*4 for indicating each food frame.Then, then it will indicate that the n-dimensional vector of each food volume ratio multiplies
The upper two-dimensional array for indicating each food generic obtains the volume ratio vector of various kinds of foods, is m dimensional vectors.The m tie up to
One group food of every one-dimensional correspondence of amount indicates the volume ratio shared by corresponding classification food per the numerical value on one-dimensional.
The method of the present invention further includes to each figure in one group of image (to the photo of the different angle of same disk food)
All m dimensional vectors, are then added again divided by every group by the m dimensional vectors as all calculating an expression various kinds of foods volume ratio again
Image number finds out average vector as final classification volume ratio in kind vector.
The method of the present invention further include by calculated food qualification category volume ratio vector and food qualification category intensity vector by
Position is multiplied, and obtains food qualification category mass ratio vector, then be multiplied with the food gross mass of input, acquire food qualification category quality to
Amount, each indicates the quality of the food of respective classes.
The method of the present invention further includes that m is tieed up food qualification category quality vector and corresponding food qualification category nutrient content matrix
It is multiplied, obtains food nutrition constituent content vector, each indicates certain nutritive element content in food.Wherein food qualification category is sought
It supports containing the two-dimensional matrix that moment matrix is a m*q, wherein q is the detectable nutrient type number of system.Food qualification category is sought
It supports every a line containing moment matrix and corresponds to a group food, a kind of corresponding nutrient of each row is indicated per in group food unit mass
The nutritive element content contained.Finally obtained food nutrition constituent content vector is a q dimensional vector.
The method of the present invention further includes the method for carrying out adaptivity training to information processing unit.
Input data is the image with flag data, and the corresponding flag data of every image is the class of each food in figure
Not (n-dimensional vector), each food frame information (two-dimensional matrix of n*4), each food shared volume ratio (n tie up to
Amount)；Wherein n is the sum of food item in image.Processing unit first pre-processes the data information of input, if for example,
Food qualification category information is word, then is translated into the corresponding number of classification.
Training process is divided into five steps, RPN networks, the Fast R-CNN nets returned to food qualification category detection and frame
Network predicts the network cross-training of food volume ratio.
Step 1, RPN netinit network parameters calculate each detection according to the image information propagated forward of input
The class label and region adjusting parameter in region；Come using stochastic gradient descent algorithm or Adam algorithms further according to backpropagation
The relevant parameter for updating RPN, includes the parameter of the unique portion parameter of RPN and shared conventional part.Training is until convergence.
Step 2, Fast R-CNN utilize the shared convolutional layer parameter initialization convolution layer parameter of step 1 training, and will
The recommendation region that step 1 obtains is trained as the recommendation region during network calculations, and updates network parameter, including
Shared convolutional network.Until network convergence.
Step 3, the shared convolutional network that RPN networks are obtained using step 2 continue to train and update the only of RPN networks
There are partial parameters, does not include the convolution layer parameter shared.
Step 4, the recommendation region that Fast R-CNN Web vector graphic step 3 obtains is trained, and only updates Fast
The exclusive part of R-CNN networks, shared convolutional layer parameter constant.
The food frame that step 4 obtains is mapped to the last of shared convolutional network by step 5, Volume R-CNN networks
One layer of Feature Mapping, and the exclusive partial parameters of update are trained, until network convergence.
The training operation of each step is that input data is obtained the loss function of each part by network forward calculation,
Backpropagation again updates network parameter using stochastic gradient descent or Adam algorithms.
The training process of above-mentioned five steps can recycle execution.
The present invention also provides a kind of devices for executing Faster R-CNN neural network computings, including information input
Component, information processing apparatus, information output part, as shown in Figure 4.
Wherein, information input part includes one or more cameras, and the food for inputting one group of different angle is overlooked
Image；One apparatus for measuring quality, for measuring quality of food and being transferred to processing component.
Information processing apparatus includes storage unit and data processing unit, and storage unit is for receiving and storing input number
According to, instruction and output data, wherein input data include one group of image and a positive number (quality of food)；Data processing unit is first
The key feature for including in input data is extracted and handled using neural network, generating one to every image indicates food
Nutritive element content vector in object finds out all images and corresponds to the average value of vector as test food for same group of image
Final nutrient content vector.
Information processing apparatus further includes a data conversion module, the q dimension nutrient contents for export processing unit to
Amount is converted into corresponding output, and output can be table, the form of pie chart.
Information output part includes a liquid crystal display, and display receives output information from information processing apparatus, and will
Presentation of information comes out.
Output knot of the information processing apparatus according to food nutrition content vectorial (q dimensional vectors) control predicted on the screen
Fruit.Data translation processor converts q dimensional vectors to corresponding storage information, and format is：Nutrient name, content.Nutrition member
Plain name can be corresponded to by each position index subscript of q dimensional vectors and be obtained, 0 element in vector is ignored.In addition, the device can be with
Storage or networking obtain the nutrient intake that the people of each age group recommends daily, and assess test food, instant
Various nutritive element contents in object are often eaten required too high levels or too low compared to human body, provide rational dietary recommendation.
Nutritive element content and the dietary recommendation in food are finally exported on a display screen.The output form of nutritive element content can be
Table and pie chart.
The device can also include networked components, it may be connected to which measurement data is uploaded database by internet in real time, is expanded
Big data quantity, while also most recent parameters model can be updated from high in the clouds, improve operation efficiency and precision.
Data processing unit uses neural network chip, is suitable for neural computing, computing capability is powerful.
Particular embodiments described above has carried out further in detail the purpose of the present invention, technical solution and advantageous effect
Describe in detail bright, it should be understood that the above is only a specific embodiment of the present invention, is not intended to restrict the invention, it is all
Within the spirit and principles in the present invention, any modification, equivalent substitution, improvement and etc. done should be included in the protection of the present invention
Within the scope of.
Claims (16)
1. a kind of method for executing Faster R-CNN neural network computings, including：
Obtain the multiple images of the different angle with serving；
The recommendation region of pattern detection is determined using RPN；
Recommend the classification and frame of food item in region using Fast R-CNN predictions；
According to the frame of the food of prediction, the volume ratio shared by each food item is predicted using Volume R-CNN；
According to the food item classification of Fast R-CNN predictions and the food item volume ratio of Volume R-CNN predictions, meter
Calculate the volume ratio of different classes of food；
The volume ratio of calculated various kinds of foods is multiplied respectively with the density of various kinds of foods, obtains the mass ratio of various kinds of foods
Example；
The mass ratio of various kinds of foods is multiplied with the gross mass of food, acquires the quality of various kinds of foods；
The quality of various kinds of foods is multiplied with corresponding nutrient content, obtains food nutrition constituent content；
Wherein, the RPN, Fast R-CNN and Volume R-CNN share convolutional layer.
2. according to the method described in claim 1, wherein, determine recommend region when, the RPN first to the picture of input into
Row multilayer convolution operation, extracts the Feature Mapping of picture, reuses sliding window and carries out convolution operation to Feature Mapping, then
Loss function Liang Ge branches are returned using Classification Loss function and frame and come zoning classification and region recurrence, obtain recommended area
Domain.
3. according to the method described in claim 1, wherein, the Fast R-CNN will recommend area maps to the Feature Mapping
RoIs is obtained, then pondization operation is carried out to each RoI, the characteristic pattern of same size is converted into, after then being operated to pondization
RoIs carries out two fully-connected network operations respectively, calculate it is each recommendation region in food item classification and to frame into
The accurate prediction of row.
4. according to the method described in claim 1, wherein, the frame parameter of prediction is mapped to described by the Volume R-CNN
In Feature Mapping, then pondization operation is carried out to corresponding mapping area, becomes the sample areas of same size, then to each sample
One's respective area carries out multilayer fully-connected network operation, calculates the volume intermediate variable v of each food item in figurei, viFor positive number；
Then the volume intermediate variable is converted to corresponding volume ratio f againi, calculation formula is：Wherein i=1,
2 ... n, n are the number of food item in image.
5. according to the method described in claim 1, wherein, the method frame parameter of prediction being mapped on the characteristic pattern
For：Each coordinate data is multiplied by the ratio between the size of characteristic pattern and original image.
6. according to the method described in claim 1, wherein, the shape of loss function Volume loss in the Volume R-CNN
Formula isWherein fiFor the volume ratio of each food item of prediction, fi *For reality
Value, the flag data inputted when training.
7. according to the method described in claim 1, wherein, during prediction the output of neural network include：By Volume R-
Calculated one of CNN indicates that the n-dimensional vector of the shared volume ratio of each food item in image, each element are located at section
(0,1] in, and the sum of each element is 1；By Fast R-CNN, calculated one indicates each affiliated class of food item in image
Other n*m matrixes, m are the class number of recognizable food item, and it is 1 that matrix, which only has an element per a line, remaining m-1 member
Element is 0, and the corresponding row of element 1 are food item generic；And the two of a n*4 for indicating each food item frame
Dimension group.
8. according to the method described in claim 1, wherein, the algorithm further includes that will indicate each food item volume ratio
N-dimensional vector is multiplied by the n*m two-dimensional arrays for indicating each food item generic, obtains the volume ratio vector of various kinds of foods,
For m dimensional vectors, one group food of every one-dimensional correspondence of the m dimensional vectors is indicated per the numerical value on one-dimensional shared by corresponding classification food
Volume ratio.
9. according to the method described in claim 1, wherein, the method further includes calculating one to each image to indicate each
Then all m dimensional vectors are added again divided by the number of image by the m dimensional vectors of classification food volume ratio again, find out it is average to
Measure the volume ratio vector as final food of all categories.
10. according to the method described in claim 1, wherein, the method further includes adaptivity training step, including：
Step 1, RPN netinit network parameters calculate each detection zone according to the image information propagated forward of input
Class label and region adjusting parameter；It is updated using stochastic gradient descent algorithm or Adam algorithms further according to backpropagation
The relevant parameter of RPN includes the parameter of the unique portion parameter of RPN and shared conventional part, and training is until convergence；
Step 2, Fast R-CNN utilize the shared convolutional layer parameter initialization convolution layer parameter of step 1 training, and by step
One obtained recommendation region is trained as the recommendation region during neural computing, and updates network parameter, including
Shared convolutional network, until network convergence；
Step 3, the shared convolutional network that RPN networks are obtained using step 2 continue the exclusive portion for training and updating RPN networks
Divide parameter, does not include the convolution layer parameter shared；
Step 4, the recommendation region that Fast R~CNN Web vector graphic step 3 obtains are trained, and only update Fast R-
The exclusive part of CNN networks, shared convolutional layer parameter constant；
The food item frame that step 4 obtains is mapped to the last of shared convolutional network by step 5, Volume R-CNN networks
One layer of Feature Mapping, and the exclusive partial parameters of update are trained, until network convergence；
The training of each step operates input data obtaining the loss function of each part by network forward calculation, then instead
To propagation, network parameter is updated using stochastic gradient descent or Adam algorithms；
The training process of wherein above-mentioned five steps can recycle execution.
11. a kind of device for executing Faster R-CNN neural network computings, including
Information input unit obtains different classes of in the multiple images, the gross mass of food, food of the different angle with serving
The density and nutritive element content of food；
Information treatment part, for being handled described image and being calculated；
Wherein described information processing unit includes：
Storage unit, for storing described image, gross mass, density and nutritive element content；
Recommend region generation unit, the recommendation region of pattern detection is determined using RPN；
Classification and frame predicting unit recommend the classification and frame of food item in region using Fast R-CNN predictions；
Food item volume ratio predicting unit utilizes Volume R-CNN prognostic charts according to the frame of the food item of prediction
Volume ratio as in shared by each food item；
Food qualification category volume ratio predicting unit, according to the food item classification and Volume R-CNN of Fast R-CNN predictions
The food item volume ratio of prediction calculates the volume ratio shared by each classification food, and to the calculating knot of different images
Fruit is averaged；
Mass ratio predicting unit distinguishes the density of the volume ratio of calculated different classes of food and different classes of food
It is multiplied, obtains the mass ratio of different classes of food；
The mass ratio of different classes of food is multiplied with the gross mass of food, acquires different classes of by quality of food predicting unit
The quality of food；And
The quality of various kinds of foods is multiplied with corresponding nutrient content, obtains food nutrition element by nutrient content predicting unit
Content；
Wherein, the RPN, Fast R-CNN and Volume R-CNN share convolutional layer.
12. according to the devices described in claim 11, wherein described information input unit includes image-input device and mass input
Device.
13. according to the devices described in claim 11, wherein described information processing unit further includes Date Conversion Unit, and being used for will
The q dimension nutrient content vectors of processing unit output are converted into corresponding output.
14. according to the devices described in claim 11, wherein described device further includes information output part, is used for from described information
Processing unit receives output information, and presentation of information is come out.
15. according to the devices described in claim 11, wherein described device further includes networked components, for measurement data is real
When upload to database, while also can update most recent parameters model from high in the clouds.
16. according to the devices described in claim 11, wherein described information processing unit is neural network chip.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810352111.7A CN108597582B (en) | 2018-04-18 | 2018-04-18 | Method and device for executing fast R-CNN neural network operation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810352111.7A CN108597582B (en) | 2018-04-18 | 2018-04-18 | Method and device for executing fast R-CNN neural network operation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108597582A true CN108597582A (en) | 2018-09-28 |
| CN108597582B CN108597582B (en) | 2021-02-12 |
Family
ID=63613739
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810352111.7A Active CN108597582B (en) | 2018-04-18 | 2018-04-18 | Method and device for executing fast R-CNN neural network operation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108597582B (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109846303A (en) * | 2018-11-30 | 2019-06-07 | 广州富港万嘉智能科技有限公司 | Service plate surplus automatic testing method, system, electronic equipment and storage medium |
| CN110174399A (en) * | 2019-04-10 | 2019-08-27 | 晋江双龙制罐有限公司 | Solid content qualification detection method and its detection system in a kind of transparent can |
| CN110569759A (en) * | 2019-08-26 | 2019-12-13 | 王睿琪 | Method, system, server and front end for acquiring individual eating data |
| CN111008626A (en) * | 2018-10-04 | 2020-04-14 | 斯特拉德视觉公司 | Method and device for detecting object based on R-CNN |
| CN111564200A (en) * | 2020-05-08 | 2020-08-21 | 深圳市万佳安人工智能数据技术有限公司 | Old people diet feature extraction device and method based on rapid random gradient descent |
| CN111696151A (en) * | 2019-03-15 | 2020-09-22 | 青岛海尔智能技术研发有限公司 | Method and device for identifying volume of food material in oven and computer readable storage medium |
| CN112257761A (en) * | 2020-10-10 | 2021-01-22 | 天津大学 | Method for analyzing food nutrient components in image based on machine learning |
| CN113111925A (en) * | 2021-03-29 | 2021-07-13 | 宁夏新大众机械有限公司 | Feed qualification classification method based on deep learning |
| CN113539427A (en) * | 2020-04-22 | 2021-10-22 | 深圳市前海高新国际医疗管理有限公司 | Convolutional neural network-based nutrition intervention analysis system and analysis method |
| WO2022052021A1 (en) * | 2020-09-11 | 2022-03-17 | 京东方科技集团股份有限公司 | Joint model training method, object information processing method, apparatus, and system |
| US20220399098A1 (en) * | 2020-12-25 | 2022-12-15 | Boe Technology Group Co., Ltd. | Diet recommendation method, device, storage medium and electronic device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103162627A (en) * | 2013-03-28 | 2013-06-19 | 广西工学院鹿山学院 | Method for estimating fruit size by citrus fruit peel mirror reflection |
| CN106709525A (en) * | 2017-01-05 | 2017-05-24 | 北京大学 | Method for measuring food nutritional component by means of camera |
-
2018
- 2018-04-18 CN CN201810352111.7A patent/CN108597582B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103162627A (en) * | 2013-03-28 | 2013-06-19 | 广西工学院鹿山学院 | Method for estimating fruit size by citrus fruit peel mirror reflection |
| CN106709525A (en) * | 2017-01-05 | 2017-05-24 | 北京大学 | Method for measuring food nutritional component by means of camera |
Non-Patent Citations (4)
| Title |
|---|
| K A FORBES等: "Estimating Fruit Volume from Digital Images", 《1999 IEEE AFRICON. 5TH AFRICON CONFERENCE IN AFRICA (CAT. NO.99CH36342)》 * |
| SHAO QING REN等: "Faster R-CNN：Towards Real-Time Object Detection with Region Proposal Networks", 《ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 28 (NIPS 2015)》 * |
| TAKUMI EGE等: "Estimating Food Calories for Multiple-dish Food Photos", 《2017 4TH IAPR ASIAN CONFERENCE ON PATTERN RECOGNITION (ACPR)》 * |
| YANCHAO LIANG等: "COMPUTER VISION-BASED FOOD CALORIE ESTIMATION：DATASET,METHOD,AND EXPERIMENT", 《COMPUTER VISION AND PATTERN RECOGNITION》 * |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111008626B (en) * | 2018-10-04 | 2023-10-13 | 斯特拉德视觉公司 | Method and device for detecting object based on R-CNN |
| CN111008626A (en) * | 2018-10-04 | 2020-04-14 | 斯特拉德视觉公司 | Method and device for detecting object based on R-CNN |
| CN109846303A (en) * | 2018-11-30 | 2019-06-07 | 广州富港万嘉智能科技有限公司 | Service plate surplus automatic testing method, system, electronic equipment and storage medium |
| CN111696151A (en) * | 2019-03-15 | 2020-09-22 | 青岛海尔智能技术研发有限公司 | Method and device for identifying volume of food material in oven and computer readable storage medium |
| CN110174399A (en) * | 2019-04-10 | 2019-08-27 | 晋江双龙制罐有限公司 | Solid content qualification detection method and its detection system in a kind of transparent can |
| CN110569759B (en) * | 2019-08-26 | 2020-11-03 | 王睿琪 | Method, system, server and front end for acquiring individual eating data |
| CN110569759A (en) * | 2019-08-26 | 2019-12-13 | 王睿琪 | Method, system, server and front end for acquiring individual eating data |
| CN113539427A (en) * | 2020-04-22 | 2021-10-22 | 深圳市前海高新国际医疗管理有限公司 | Convolutional neural network-based nutrition intervention analysis system and analysis method |
| CN111564200A (en) * | 2020-05-08 | 2020-08-21 | 深圳市万佳安人工智能数据技术有限公司 | Old people diet feature extraction device and method based on rapid random gradient descent |
| WO2022052021A1 (en) * | 2020-09-11 | 2022-03-17 | 京东方科技集团股份有限公司 | Joint model training method, object information processing method, apparatus, and system |
| US20220327361A1 (en) * | 2020-09-11 | 2022-10-13 | Boe Technology Group Co., Ltd. | Method for Training Joint Model, Object Information Processing Method, Apparatus, and System |
| CN112257761A (en) * | 2020-10-10 | 2021-01-22 | 天津大学 | Method for analyzing food nutrient components in image based on machine learning |
| US20220399098A1 (en) * | 2020-12-25 | 2022-12-15 | Boe Technology Group Co., Ltd. | Diet recommendation method, device, storage medium and electronic device |
| CN113111925A (en) * | 2021-03-29 | 2021-07-13 | 宁夏新大众机械有限公司 | Feed qualification classification method based on deep learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108597582B (en) | 2021-02-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108597582A (en) | A kind of method and apparatus for executing Faster R-CNN neural network computings | |
| CN107092870B (en) | A kind of high resolution image Semantic features extraction method | |
| CN109685152A (en) | A kind of image object detection method based on DC-SPP-YOLO | |
| CN109670452A (en) | Method for detecting human face, device, electronic equipment and Face datection model | |
| CN106340137A (en) | Automatic weighing machine based on deep learning and program control method thereof | |
| JP6884116B2 (en) | Information processing equipment, information processing methods, and programs | |
| CN109711401A (en) | A kind of Method for text detection in natural scene image based on Faster Rcnn | |
| CN110070107A (en) | Object identification method and device | |
| CN108304820A (en) | A kind of method for detecting human face, device and terminal device | |
| CN108537329A (en) | A kind of method and apparatus carrying out operation using Volume R-CNN neural networks | |
| CN108898065A (en) | Candidate regions quickly screen and the depth network Ship Target Detection method of dimension self-adaption | |
| CN108256527A (en) | A kind of cutaneous lesions multiclass semantic segmentation method based on end-to-end full convolutional network | |
| CN106340000A (en) | Bone age assessment method | |
| CN113435282A (en) | Unmanned aerial vehicle image ear recognition method based on deep learning | |
| CN107423537A (en) | A kind of method of the surface temperature NO emissions reduction based on adaptive threshold | |
| CN114331985A (en) | Electronic component scratch defect detection method and device and computer equipment | |
| CN112614133A (en) | Three-dimensional pulmonary nodule detection model training method and device without anchor point frame | |
| CN110119479A (en) | A kind of restaurant recommendation method, apparatus, equipment and readable storage medium storing program for executing | |
| CN113887410A (en) | Deep learning-based multi-category food material identification system and method | |
| CN111563458A (en) | Target detection and positioning method based on YOLOv3 and OpenCV | |
| Wang et al. | Features Extraction of Flotation Froth Images and BP Neural Network Soft‐Sensor Model of Concentrate Grade Optimized by Shuffled Cuckoo Searching Algorithm | |
| CN115131783A (en) | User diet nutrient component information autonomous perception method based on machine vision | |
| CN111814804B (en) | Human body three-dimensional size information prediction method and device based on GA-BP-MC neural network | |
| CN116662593B (en) | FPGA-based full-pipeline medical hyperspectral image neural network classification method | |
| CN108765384B (en) | Significance detection method for joint manifold sequencing and improved convex hull |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |