CN108224894B

CN108224894B - Food material freshness identification method and device based on deep learning, refrigerator and medium

Info

Publication number: CN108224894B
Application number: CN201810015399.9A
Authority: CN
Inventors: 唐红强; 戴江
Original assignee: Hefei Midea Intelligent Technologies Co Ltd
Current assignee: Hefei Midea Intelligent Technologies Co Ltd
Priority date: 2018-01-08
Filing date: 2018-01-08
Publication date: 2020-04-28
Anticipated expiration: 2038-01-08
Also published as: CN108224894A

Abstract

The invention provides a food material freshness identification method and device based on deep learning, a refrigerator and a medium, wherein the method comprises the following steps: collecting food material image information; determining food material freshness identification result information according to the food material image information and a pre-trained food material freshness identification model; based on deep learning, the food freshness identification model is trained and updated by taking food material image information collected in the food material freshness identification process as a sample according to a preset period. According to the technical scheme, the identification accuracy is high, real-time identification can be achieved, optimization and improvement are performed in the application of the food freshness identification model, the application range of food freshness identification is widened, and the method and the device can be applied to complex scenes.

Description

Food material freshness identification method and device based on deep learning, refrigerator and medium

Technical Field

The invention relates to the technical field of intelligent refrigerators, in particular to a food freshness identification method based on deep learning, a food freshness identification device based on deep learning, a refrigerator and a computer-readable storage medium.

Background

With the development of the information technology, the refrigerator not only bears the function of storing food materials, but also gradually develops towards smart home, so that more intelligent services are provided for users, and food freshness identification is an important information providing way for the intelligent services.

In the related art, the food freshness identification is generally performed by image matching identification, and the following technical defects exist:

(1) the identification rate is low, and the food freshness identification under a complex scene is difficult to deal with.

(2) The calculation complexity is high, and the real-time requirement of food freshness identification is difficult to meet.

(3) The food freshness identification result is not effectively supervised, and the existing model is difficult to optimize and update.

Disclosure of Invention

The present invention is directed to solving at least one of the problems of the prior art or the related art.

Therefore, the invention aims to provide a food freshness identification method based on deep learning.

The invention further aims to provide a food material freshness identification device based on deep learning.

It is still another object of the present invention to provide a refrigerator.

It is yet another object of the present invention to provide a computer-readable storage medium.

In order to achieve the above object, a first aspect of the present invention provides a food material freshness identification method based on deep learning, including: collecting food material image information; determining food material freshness identification result information according to the food material image information and a pre-trained food material freshness identification model; based on deep learning, the food freshness identification model is trained and updated by taking food material image information collected in the food material freshness identification process as a sample according to a preset period.

In this technical scheme, through gathering edible material image information, according to edible material image information and the good edible material new freshness degree identification model of training in advance, confirm edible material new freshness degree identification result information, the real-time identification of edible material new degree has been realized, and the discernment rate of accuracy is higher, be applicable to complicated scene, through being based on degree of depth study, use the edible material image information of gathering in the edible material new freshness degree identification process as the sample according to preset cycle, the new degree identification model of edible material is eaten in the training update, the training update to edible material new degree identification model has been realized, optimize in the application of edible material new degree identification model and promote, be favorable to promoting the application scope of edible material new degree identification, the rate of accuracy of edible material new degree identification.

It should be noted that the deep learning may be convolutional neural network deep learning, recursive neural network deep learning, or cyclic neural network deep learning.

In above-mentioned technical scheme, preferably, food material freshness degree recognition result information includes discernment freshness degree and discernment result confidence, based on degree of depth study, uses the food material image information of gathering in the food material freshness degree recognition process as the sample according to preset cycle, trains to update the food material freshness degree recognition model, includes: judging whether the confidence coefficient of the recognition result is smaller than a first preset threshold value or not; when the confidence coefficient of the recognition result is judged to be smaller than a first preset threshold value, recording food material image information corresponding to the confidence coefficient of the recognition result; marking the actual freshness of the food material image information recorded in a preset period; judging whether the actual freshness is the same as the corresponding identified freshness; when the actual freshness and the identified freshness are different, determining the corresponding food material image information as a sample; and training and updating the food freshness identification model according to all samples determined in a preset period based on deep learning.

In the technical scheme, whether the confidence of the identification result is smaller than a first preset threshold value or not is judged, food material image information corresponding to the confidence of the identification result is recorded when the confidence of the identification result is smaller than the first preset threshold value, marking confirmation of food material image information with lower confidence of the identification result is facilitated, whether the actual freshness is the same as the corresponding identification freshness is judged by marking the actual freshness of the food material image information recorded in a preset period, screening of food material image information with identification errors is facilitated, supervised learning optimization of the food material freshness identification model is realized by determining the corresponding food material image information as a sample when the actual freshness is judged to be different from the identification freshness, training and updating the food material freshness identification model according to all samples determined in the preset period based on deep learning, and training and updating the food material freshness identification model by using the food material image information with identification errors as a sample, the method has the advantages that the recognition accuracy of the food freshness recognition model is improved, and meanwhile, the training amount is reduced.

Note that the first preset threshold is 80%.

In any one of the above technical solutions, preferably, the food freshness identification result information includes identification failure information, and based on deep learning, the food freshness identification model is trained and updated by using food material image information acquired in the food freshness identification process as a sample according to a preset period, including: recording food material image information corresponding to the identification failure information; determining the food material type of the food material image information recorded in a preset period, and correspondingly marking the actual freshness; adding other food material image information with different actual freshness corresponding to the food material types, and correspondingly marking the actual freshness; based on deep learning, the food material freshness identification model is trained and updated by taking food material image information recorded in a preset period and other added food material image information as samples.

In the technical scheme, a data basis is provided for training and updating of the food freshness identification model by recording food material image information corresponding to identification failure information, the food material type of the food material image information recorded in a preset period is determined, the actual freshness is correspondingly marked, the new food material type which is not contained in the food material freshness identification model is favorably determined in the application process of the food material freshness identification model, other food material image information with different actual freshness of the new food material type is favorably added, the food material freshness identification model is trained and updated by adding other food material image information with different actual freshness corresponding to the food material type and correspondingly marking the actual freshness, and then the food material freshness identification model is trained and updated by taking the food material image information recorded in the preset period and the added other food material image information as samples based on deep learning, the method is favorable for improving the capability of the food freshness identification model to adapt to new food material types, and further improves the application range of the food freshness identification model.

In any one of the above technical solutions, preferably, the method further includes: acquiring and constructing a food material image information set in advance; correspondingly marking the food material type and the actual freshness of each food material image information in the food material image information set; and training and constructing a food freshness identification model by taking each food material image information in the food material image information set as an input sample based on deep learning.

In the technical scheme, the food material image information sets are constructed through pre-acquisition, the food material types and the actual freshness of each piece of food material image information in the food material image information sets are correspondingly marked, a data basis is provided for training and constructing the food material freshness identification model, the food material freshness identification model is trained and constructed by taking each piece of food material image information in the food material image information sets as an input sample based on deep learning, the pre-training construction of the food material freshness identification model is realized, the food material freshness identification model is favorably realized, and the food material freshness identification result information is determined according to the food material freshness identification model.

It should be noted that, the training and construction of the food freshness identification model are performed in advance, then the trained food freshness identification model is stored, real-time food freshness identification is performed, the food freshness identification model can be stored in a server, the food image information is collected by a terminal and then uploaded to the server for food freshness identification, the food freshness identification model can also be stored in the terminal, and the food freshness identification is directly performed after the food image information is collected by the terminal.

In any one of the above technical solutions, preferably, based on deep learning, training and constructing a food freshness identification model by using each food material image information in a food material image information set as an input sample includes: based on deep learning, taking each piece of food material image information in the food material image information set as an input sample, sequentially performing convolution layer operation, pooling layer operation, normalization operation and full-connection layer operation, and outputting result information; calculating a loss function value according to the result information and the marking information of the input sample; according to the loss function value, updating corresponding parameters of the convolution layer operation, the pooling layer operation, the normalization operation and the full-connection layer operation so as to train and construct a food freshness identification model; judging whether the absolute value of the loss function value is smaller than a second preset threshold value or not; if the absolute value of the loss function value is judged to be larger than or equal to a second preset threshold value, training and constructing a food freshness identification model by continuously taking each food material image information in the food material image information set as an input sample; and if the loss function value is smaller than a second preset threshold value, storing the trained food freshness identification model.

In the technical scheme, based on deep learning, each food material image information in a food material image information set is taken as an input sample, the food material image information is sequentially subjected to a convolutional layer operation, a pooling layer operation, a normalization operation and a full connection layer operation, result information is output, then a loss function value is calculated according to the result information and label information of the input sample, a data basis is provided for continuously training and constructing a food material freshness identification model and updating parameters in the food material freshness identification model, the food material freshness identification model is trained and constructed by updating the corresponding parameters of the convolutional layer operation, the pooling layer operation, the normalization operation and the full connection layer operation according to the loss function value, the gradual training and construction of the food material freshness identification model are perfected, the trained food material freshness identification model is more practical and has higher identification accuracy, whether the absolute value of the loss function value is smaller than a second preset threshold value or not is judged, when the absolute value of the loss function value is judged to be larger than or equal to the second preset threshold value, the food material image information is continuously concentrated by food material image information and is taken as an input sample, a food material freshness identification model is trained and constructed, when the loss function value is judged to be smaller than the second preset threshold value, the trained food material freshness identification model is stored, on one hand, the training effectiveness of the food material freshness identification model is favorably improved, on the other hand, the trained food material freshness identification model is more practical in fitting, and the identification accuracy is higher.

The convolution layer operation refers to performing convolution operation on food material image information by adopting a specified convolution core, increasing bias, performing feature extraction to obtain feature image information, and achieving dimension reduction of the food material image information; the pooling layer operation is to utilize spatial correlation, pool down-sample aggregation and reduce the redundancy of characteristic image information; normalization operation, namely, input parameters are normalized to normal distribution with the mean value of 0 and the variance of 1 again, so that generalization capability can be improved, and training speed is improved; the full-connection layer operation refers to synthesizing the characteristic image information to obtain a characteristic vector.

It should be noted that the second preset threshold is a value close to 0, such as 0.001.

The technical scheme of the second aspect of the invention provides a food material freshness identification device based on deep learning, which comprises the following steps: the acquisition unit is used for acquiring food material image information; the determining unit is used for determining the information of the food material freshness identification result according to the food material image information and a pre-trained food material freshness identification model; and the training updating unit is used for training and updating the food freshness identification model by taking the food material image information acquired in the food material freshness identification process as a sample according to a preset period based on deep learning.

In the above technical solution, preferably, the food freshness identification result information includes identification freshness and an identification result confidence, and the food freshness identification apparatus further includes: the judging unit is used for judging whether the confidence coefficient of the recognition result is smaller than a first preset threshold value or not; the first recording unit is used for recording the food material image information corresponding to the confidence coefficient of the recognition result when the confidence coefficient of the recognition result is judged to be smaller than a first preset threshold value; the marking unit is used for marking the actual freshness of the food material image information recorded in the preset period; the judging unit is further configured to: judging whether the actual freshness is the same as the corresponding identified freshness; the determination unit is further configured to: when the actual freshness and the identified freshness are different, determining the corresponding food material image information as a sample; the training update unit is further configured to: and training and updating the food freshness identification model according to all samples determined in a preset period based on deep learning.

Note that the first preset threshold is 80%.

In any one of the above technical solutions, preferably, the food freshness identification result information includes identification failure information, and the food freshness identification apparatus further includes: the second recording unit is used for recording the food material image information corresponding to the identification failure information; the determination unit is further configured to: determining the food material type of the food material image information recorded in a preset period, and correspondingly marking the actual freshness; food freshness identification device still includes: the increasing unit is used for increasing other food material image information with different actual freshness corresponding to the food material types and correspondingly marking the actual freshness; the training update unit is further configured to: based on deep learning, the food material freshness identification model is trained and updated by taking food material image information recorded in a preset period and other added food material image information as samples.

In any one of the above technical solutions, preferably, the method further includes: the acquisition and construction unit is used for acquiring and constructing a food material image information set in advance; the marking unit is further configured to: correspondingly marking the food material type and the actual freshness of each food material image information in the food material image information set; food freshness identification device still includes: and the training construction unit is used for training and constructing a food freshness identification model by taking each piece of food material image information in the food material image information set as an input sample based on deep learning.

In any one of the above technical solutions, preferably, the method further includes: the output unit is used for outputting result information by taking each piece of food material image information in the food material image information set as an input sample through convolutional layer operation, pooling layer operation, normalization operation and full-link layer operation in sequence based on deep learning; a calculation unit for calculating a loss function value based on the result information and the label information of the input sample; the training construction unit is further configured to: according to the loss function value, updating corresponding parameters of the convolution layer operation, the pooling layer operation, the normalization operation and the full-connection layer operation so as to train and construct a food freshness identification model; the judging unit is further configured to: judging whether the absolute value of the loss function value is smaller than a second preset threshold value or not; the training construction unit is further configured to: if the absolute value of the loss function value is judged to be larger than or equal to a second preset threshold value, training and constructing a food freshness identification model by continuously taking each food material image information in the food material image information set as an input sample; food freshness identification device still includes: and the storage unit is used for storing the trained food freshness identification model if the loss function value is judged to be smaller than a second preset threshold value.

A third aspect of the present invention proposes a refrigerator, which includes any one of the deep learning-based food material freshness identification devices as set forth in the second aspect of the present invention; and the image collector is arranged in the refrigerator, is connected with the food freshness identification device and is used for collecting food material image information under the control of the food material freshness identification device.

In this technical scheme, the refrigerator includes any food material freshness identification device based on deep learning provided by the technical scheme of the second aspect of the present invention, so that all the beneficial effects of any food material freshness identification device based on deep learning provided by the technical scheme of the second aspect of the present invention are achieved, and no description is repeated herein, and through the image collector which is arranged in the refrigerator and connected with the food material freshness identification device, the food material image information can be collected under the control of the food material freshness identification device, which is beneficial to collecting more accurate food material image information, and is further beneficial to improving the accuracy of food material freshness identification.

An aspect of the fourth aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the deep learning-based food material freshness identification method according to any one of the aspects of the first aspect of the present invention.

In this technical solution, a computer-readable storage medium has a computer program stored thereon, and the computer program when executed by a processor implements the steps of any one of the deep learning-based food material freshness identification methods proposed by the technical solutions of the first aspect of the present invention, so that all the beneficial effects of any one of the deep learning-based food material freshness identification methods proposed by the technical solutions of the first aspect of the present invention are achieved, and are not described herein again.

Through above technical scheme, training is found and is eaten material new freshness degree recognition model based on degree of depth study, and through eating material new freshness degree recognition model discernment material new freshness, the discernment rate of accuracy is higher, can realize real-time identification, optimizes the promotion in eating material new freshness degree recognition model's application moreover, has promoted the application scope of eating material new freshness degree discernment, can be applicable to complicated scene.

Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.

Drawings

The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

fig. 1 shows a schematic flow chart of a food material freshness identification method based on deep learning according to an embodiment of the present invention;

fig. 2 shows a schematic block diagram of a deep learning based food freshness identification apparatus according to an embodiment of the present invention;

fig. 3 illustrates a schematic block diagram of a refrigerator according to an embodiment of the present invention;

fig. 4 shows a schematic flow chart of a food material freshness identification method based on deep learning according to another embodiment of the present invention.

Detailed Description

In order that the above objects, features and advantages of the present invention can be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described herein, and therefore the scope of the present invention is not limited by the specific embodiments disclosed below.

Fig. 1 shows a schematic flow chart of a food material freshness identification method based on deep learning according to an embodiment of the present invention.

As shown in fig. 1, a food material freshness identification method based on deep learning according to an embodiment of the present invention includes:

s102, collecting food material image information;

s104, determining food material freshness identification result information according to food material image information and a pre-trained food material freshness identification model;

and S106, training and updating the food freshness identification model by taking food material image information acquired in the food material freshness identification process as a sample according to a preset period based on deep learning.

In this embodiment, through gathering edible material image information, according to edible material image information and the good edible material new freshness degree identification model of training in advance, it is new degree identification result information to confirm edible material, the real-time identification of edible material new degree has been realized, and the discernment rate of accuracy is higher, and be applicable to complicated scene, through learning based on the degree of depth, use the edible material image information of gathering in the new degree identification process of edible material as the sample according to preset cycle, the new degree identification model of edible material is updated in the training, the training to edible material new degree identification model is updated, optimize in the application of edible material new degree identification model and promote, be favorable to promoting the application scope of edible material new degree identification, the rate of accuracy of edible material new degree.

In the foregoing embodiment, preferably, the food freshness identification result information includes identification freshness and an identification result confidence, and based on deep learning, training and updating a food freshness identification model by using food material image information acquired in a food freshness identification process as a sample according to a preset period includes: judging whether the confidence coefficient of the recognition result is smaller than a first preset threshold value or not; when the confidence coefficient of the recognition result is judged to be smaller than a first preset threshold value, recording food material image information corresponding to the confidence coefficient of the recognition result; marking the actual freshness of the food material image information recorded in a preset period; judging whether the actual freshness is the same as the corresponding identified freshness; when the actual freshness and the identified freshness are different, determining the corresponding food material image information as a sample; and training and updating the food freshness identification model according to all samples determined in a preset period based on deep learning.

In this embodiment, by determining whether the confidence of the recognition result is smaller than a first preset threshold, when the confidence of the recognition result is smaller than the first preset threshold, the food material image information corresponding to the confidence of the recognition result is recorded, which is beneficial to realize the labeling confirmation of the food material image information with lower confidence of the recognition result, by labeling the actual freshness of the food material image information recorded in a preset period, determining whether the actual freshness is the same as the corresponding recognition freshness, which is beneficial to screening out the food material image information with a recognition error, by determining the corresponding food material image information as a sample when the actual freshness is determined to be different from the recognition freshness, based on deep learning, training and updating the food material freshness recognition model according to all samples determined in the preset period, the supervised learning optimization of the food material freshness recognition model is realized, and the food material freshness recognition model is trained and updated by using the food material image information with a recognition error, the method has the advantages that the recognition accuracy of the food freshness recognition model is improved, and meanwhile, the training amount is reduced.

Note that the first preset threshold is 80%.

In any one of the above embodiments, preferably, the food freshness identification result information includes identification failure information, and based on deep learning, training and updating the food freshness identification model by taking the food material image information collected in the food freshness identification process as a sample according to a preset period includes: recording food material image information corresponding to the identification failure information; determining the food material type of the food material image information recorded in a preset period, and correspondingly marking the actual freshness; adding other food material image information with different actual freshness corresponding to the food material types, and correspondingly marking the actual freshness; based on deep learning, the food material freshness identification model is trained and updated by taking food material image information recorded in a preset period and other added food material image information as samples.

In this embodiment, a data basis is provided for training and updating of the food freshness identification model by recording food material image information corresponding to the identification failure information, the food freshness identification model is trained and updated by determining the food material type of the food material image information recorded in the preset period and correspondingly marking the actual freshness, so as to be beneficial for determining the new food material type not included in the food material freshness identification model during the application process of the food freshness identification model and to be beneficial for adding other food material image information with different actual freshness of the new food material type, the other food material image information with different actual freshness corresponding to the food material type is added and correspondingly marking the actual freshness, and then the food freshness identification model is trained and updated based on the deep learning by taking the food material image information recorded in the preset period and the added other food material image information as samples, the method is favorable for improving the capability of the food freshness identification model to adapt to new food material types, and further improves the application range of the food freshness identification model.

In any one of the above embodiments, preferably, the method further includes: acquiring and constructing a food material image information set in advance; correspondingly marking the food material type and the actual freshness of each food material image information in the food material image information set; and training and constructing a food freshness identification model by taking each food material image information in the food material image information set as an input sample based on deep learning.

In the embodiment, the food material image information sets are constructed through pre-acquisition, and then the food material type and the actual freshness of each piece of food material image information in the food material image information sets are correspondingly marked, so that a data basis is provided for training and constructing the food material freshness identification model, and the food material freshness identification model is trained and constructed by taking each piece of food material image information in the food material image information sets as an input sample based on deep learning, so that the pre-training construction of the food material freshness identification model is realized, and the determination of the food material freshness identification result information according to the food material freshness identification model is favorably realized.

In any one of the above embodiments, preferably, training and constructing the food freshness identification model by using each food material image information in the food material image information set as an input sample based on deep learning includes: based on deep learning, taking each piece of food material image information in the food material image information set as an input sample, sequentially performing convolution layer operation, pooling layer operation, normalization operation and full-connection layer operation, and outputting result information; calculating a loss function value according to the result information and the marking information of the input sample; according to the loss function value, updating corresponding parameters of the convolution layer operation, the pooling layer operation, the normalization operation and the full-connection layer operation so as to train and construct a food freshness identification model; judging whether the absolute value of the loss function value is smaller than a second preset threshold value or not; if the absolute value of the loss function value is judged to be larger than or equal to a second preset threshold value, training and constructing a food freshness identification model by continuously taking each food material image information in the food material image information set as an input sample; and if the loss function value is smaller than a second preset threshold value, storing the trained food freshness identification model.

In the embodiment, based on deep learning, each food material image information in the food material image information set is taken as an input sample, sequentially subjected to a convolutional layer operation, a pooling layer operation, a normalization operation and a full-link layer operation, result information is output, then a loss function value is calculated according to the result information and label information of the input sample, a data basis is provided for continuous training and construction of parameters in a food material freshness identification model and a food material freshness identification model, corresponding parameters of the convolutional layer operation, the pooling layer operation, the normalization operation and the full-link layer operation are updated according to the loss function value to train and construct the food material freshness identification model, gradual training and construction of the food material freshness identification model are perfected, the trained food material freshness identification model is more fit to the reality, the identification accuracy is higher, and whether the absolute value of the loss function value is smaller than a second preset threshold value or not is judged, and when judging that the absolute value of the loss function value is greater than or equal to a second preset threshold value, continuing to use each food material image information in the food material image information set as an input sample, training and constructing a food material freshness identification model, and when judging that the loss function value is less than the second preset threshold value, storing the trained food material freshness identification model.

Fig. 2 shows a schematic block diagram of an apparatus 200 for recognizing food material freshness based on deep learning according to an embodiment of the present invention.

As shown in fig. 2, the food freshness identification apparatus 200 based on deep learning according to the embodiment of the present invention includes: the acquisition unit 202 is used for acquiring food material image information; a determining unit 204, configured to determine information of a food freshness identification result according to the food material image information and a pre-trained food material freshness identification model; and the training updating unit 206 is configured to train and update the food freshness identification model by taking the food material image information acquired in the food material freshness identification process as a sample according to a preset period based on deep learning.

In the above embodiment, preferably, the food freshness identification result information includes identification freshness and identification result confidence, and the food freshness identification apparatus 200 further includes: a judging unit 208, configured to judge whether the recognition result confidence is smaller than a first preset threshold; the first recording unit 210 is configured to record food material image information corresponding to the recognition result confidence level when it is determined that the recognition result confidence level is smaller than a first preset threshold; a marking unit 212 for marking the actual freshness of the food material image information recorded within a preset period; the determining unit 208 is further configured to: judging whether the actual freshness is the same as the corresponding identified freshness; the determining unit 204 is further configured to: when the actual freshness and the identified freshness are different, determining the corresponding food material image information as a sample; the training update unit 206 is further configured to: and training and updating the food freshness identification model according to all samples determined in a preset period based on deep learning.

Note that the first preset threshold is 80%.

In any one of the above embodiments, preferably, the food freshness identification result information includes identification failure information, and the food freshness identification apparatus 200 further includes: a second recording unit 214, configured to record food material image information corresponding to the identification failure information; the determining unit 204 is further configured to: determining the food material type of the food material image information recorded in a preset period, and correspondingly marking the actual freshness; the food freshness identification apparatus 200 further includes: an increasing unit 216, configured to increase image information of other food materials with different actual freshness corresponding to the food material types, and mark the actual freshness correspondingly; the training update unit 206 is further configured to: based on deep learning, the food material freshness identification model is trained and updated by taking food material image information recorded in a preset period and other added food material image information as samples.

In any one of the above embodiments, preferably, the method further includes: an acquisition and construction unit 218, configured to acquire and construct a food material image information set in advance; the marking unit 212 is further configured to: correspondingly marking the food material type and the actual freshness of each food material image information in the food material image information set; the food freshness identification apparatus 200 further includes: the training and constructing unit 220 is configured to train and construct a food freshness identification model by taking each piece of food material image information in the food material image information set as an input sample based on deep learning.

In any one of the above embodiments, preferably, the method further includes: an output unit 222, configured to output result information by taking each piece of food material image information in the food material image information set as an input sample and sequentially performing convolutional layer operation, pooling layer operation, normalization operation, and full-link layer operation based on the deep learning; a calculation unit 224 for calculating a loss function value based on the result information and the label information of the input sample; training building unit 220 is further configured to: according to the loss function value, updating corresponding parameters of the convolution layer operation, the pooling layer operation, the normalization operation and the full-connection layer operation so as to train and construct a food freshness identification model; the determining unit 208 is further configured to: judging whether the absolute value of the loss function value is smaller than a second preset threshold value or not; training building unit 220 is further configured to: if the absolute value of the loss function value is judged to be larger than or equal to a second preset threshold value, training and constructing a food freshness identification model by continuously taking each food material image information in the food material image information set as an input sample; the food freshness identification apparatus 200 further includes: the storage unit 226 is configured to store the trained food freshness identification model if it is determined that the loss function value is smaller than the second preset threshold.

Fig. 3 shows a schematic block diagram of a refrigerator 300 according to one embodiment of the present invention.

As shown in fig. 3, according to a refrigerator 300 of an embodiment of the present invention, the refrigerator 300 includes the food material freshness identification apparatus 200 based on deep learning according to any one of the embodiments proposed by the above-mentioned embodiment of the present invention; and an image collector 302, disposed in the refrigerator 300, connected to the food freshness identification device 200, and configured to collect food material image information under the control of the food freshness identification device 200.

In this embodiment, the refrigerator 300 includes any food material freshness identification device 200 based on deep learning proposed by the above embodiment of the present invention, so that all the beneficial effects of any food material freshness identification device 200 based on deep learning proposed by the above embodiment of the present invention are achieved, and no further description is given here, and through the image collector 302 disposed in the refrigerator 300 and connected to the food material freshness identification device 200, the food material image information can be collected under the control of the food material freshness identification device 200, which is beneficial to collecting more accurate food material image information, and is further beneficial to improving the accuracy of food material freshness identification.

The computer-readable storage medium according to an embodiment of the present invention has a computer program stored thereon, and the computer program, when executed by a processor, implements the steps of the deep learning-based food material freshness identification method according to any one of the embodiments of the present invention set forth above.

In this embodiment, a computer-readable storage medium, on which a computer program is stored, is executed by a processor to implement the steps of any one of the deep learning-based food material freshness identification methods proposed in the embodiments of the present invention, so that all the beneficial effects of any one of the deep learning-based food material freshness identification methods proposed in the embodiments of the present invention are achieved, and details are not repeated herein.

As shown in fig. 4, a food material freshness identification method based on deep learning according to an embodiment of the present invention includes:

s402, collecting food material image information;

s404, determining food material freshness identification result information according to food material image information and a pre-trained food material freshness identification model;

s406, determine whether the food freshness identification result information is identification freshness and identification result confidence or identification failure information? (ii) a

If the food material freshness identification result information is identification freshness and an identification result confidence level, executing S408, and judging whether the identification result confidence level is smaller than a first preset threshold value;

if the determination of the step S408 is yes, executing a step S410, recording the food material image information corresponding to the confidence of the recognition result, and if the determination of the step S408 is no, ending the process;

s412, marking the actual freshness of the food material image information recorded in the preset period;

s414, judging whether the actual freshness is the same as the corresponding identified freshness;

if the determination in S414 is yes, ending the process, and if the determination in S414 is no, executing S416 to determine that the corresponding food material image information is a sample;

s418, training and updating the food freshness identification model according to all samples determined in a preset period based on deep learning;

if the food freshness identification result information in the S406 is identification failure information, executing S420, and recording food material image information corresponding to the identification failure information;

s422, determining the food material types of the food material graphic information recorded in the preset period, and correspondingly marking the actual freshness;

s424, adding other food material image information with different actual freshness corresponding to the food material types, and correspondingly marking the actual freshness;

and S426, training and updating the food freshness identification model by taking the food material image information recorded in the preset period and the added other food material image information as samples based on deep learning.

The technical scheme of the invention is described in detail with reference to the drawings, and the invention provides a food freshness identification method based on deep learning, a food freshness identification device based on deep learning, a refrigerator and a computer-readable storage medium.

The steps in the method of the invention can be sequentially adjusted, combined and deleted according to actual needs.

The units in the device of the invention can be merged, divided and deleted according to actual needs.

It will be understood by those skilled in the art that all or part of the steps in the methods of the embodiments described above may be implemented by instructions associated with a program, which may be stored in a computer-readable storage medium, where the storage medium includes Read-Only Memory (ROM), Random Access Memory (RAM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), One-time Programmable Read-Only Memory (OTPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), compact disc-Read-Only Memory (CD-ROM), or other Memory, magnetic disk, magnetic tape, or magnetic tape, Or any other medium which can be used to carry or store data and which can be read by a computer.

The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A food material freshness identification method based on deep learning is characterized by comprising the following steps:

collecting food material image information;

determining food material freshness identification result information according to the food material image information and a pre-trained food material freshness identification model;

based on deep learning, training and updating the food freshness identification model by taking the food material image information collected in the food material freshness identification process as a sample according to a preset period;

the food freshness identification result information comprises identification freshness and an identification result confidence coefficient, and based on deep learning, the food freshness identification model is trained and updated by taking the food image information collected in the food freshness identification process as a sample according to a preset period, and the method comprises the following steps of:

judging whether the confidence of the recognition result is smaller than a first preset threshold value or not;

when the confidence coefficient of the recognition result is judged to be smaller than the first preset threshold value, recording the food material image information corresponding to the confidence coefficient of the recognition result;

marking the actual freshness of the food material image information recorded in the preset period;

judging whether the actual freshness is the same as the corresponding identified freshness;

when the actual freshness and the identified freshness are judged to be different, determining the corresponding food material image information as the sample;

and training and updating the food freshness identification model according to all the samples determined in the preset period based on deep learning.

2. The food material freshness identification method based on deep learning of claim 1, wherein the food material freshness identification result information comprises identification failure information, and the deep learning based food material freshness identification model is trained and updated by taking the food material image information collected in the food material freshness identification process as a sample according to a preset period, and the method comprises:

recording the food material image information corresponding to the identification failure information;

determining the food material type of the food material image information recorded in the preset period, and correspondingly marking the actual freshness;

adding other food material image information corresponding to the food material types and different in actual freshness, and correspondingly marking the actual freshness;

and training and updating the food freshness identification model by taking the food image information recorded in the preset period and the added other food image information as samples based on deep learning.

3. The food material freshness identification method based on deep learning of claim 2, further comprising:

acquiring and constructing a food material image information set in advance;

correspondingly marking the food material type and the actual freshness of each food material image information in the food material image information set;

and training and constructing the food freshness identification model by taking each food material image information in the food material image information set as an input sample based on deep learning.

4. The method for recognizing food material freshness based on deep learning of claim 3, wherein the training and constructing the food material freshness recognition model by using each food material image information in the food material image information set as an input sample based on deep learning comprises:

based on deep learning, taking each piece of food material image information in the food material image information set as the input sample, sequentially performing convolution layer operation, pooling layer operation, normalization operation and full-connection layer operation, and outputting result information;

calculating a loss function value according to the result information and the marking information of the input sample;

according to the loss function value, updating corresponding parameters of the convolution layer operation, the pooling layer operation, the normalization operation and the full-connection layer operation so as to train and construct the food freshness identification model;

judging whether the absolute value of the loss function value is smaller than a second preset threshold value or not;

if the absolute value of the loss function value is judged to be larger than or equal to the second preset threshold value, continuing to train and construct the food freshness identification model by taking each piece of food material image information in the food material image information set as the input sample;

and if the loss function value is smaller than the second preset threshold value, storing the trained food freshness identification model.

5. The utility model provides an edible material freshness degree recognition device based on degree of deep learning which characterized in that includes:

the acquisition unit is used for acquiring food material image information;

the determining unit is used for determining the information of the food material freshness identification result according to the food material image information and a pre-trained food material freshness identification model;

the training and updating unit is used for training and updating the food freshness identification model by taking the food material image information acquired in the food material freshness identification process as a sample according to a preset period based on deep learning;

the food freshness identification result information includes identification freshness and an identification result confidence, and the food freshness identification apparatus further includes:

the judging unit is used for judging whether the confidence coefficient of the recognition result is smaller than a first preset threshold value or not;

the first recording unit is used for recording the food material image information corresponding to the recognition result confidence coefficient when the recognition result confidence coefficient is judged to be smaller than the first preset threshold value;

the marking unit is used for marking the actual freshness of the food material image information recorded in the preset period;

the judging unit is further configured to: judging whether the actual freshness is the same as the corresponding identified freshness;

the determination unit is further configured to: when the actual freshness and the identified freshness are judged to be different, determining the corresponding food material image information as the sample;

the training update unit is further configured to: and training and updating the food freshness identification model according to all the samples determined in the preset period based on deep learning.

6. The deep learning based food material freshness identification apparatus according to claim 5, wherein the food material freshness identification result information includes identification failure information, the food material freshness identification apparatus further comprising:

the second recording unit is used for recording the food material image information corresponding to the identification failure information;

the determination unit is further configured to: determining the food material type of the food material image information recorded in the preset period, and correspondingly marking the actual freshness;

the food freshness identification device further comprises:

the increasing unit is used for increasing the image information of other food materials with different actual freshness corresponding to the food material types and correspondingly marking the actual freshness;

the training update unit is further configured to: and training and updating the food freshness identification model by taking the food image information recorded in the preset period and the added other food image information as samples based on deep learning.

7. The deep learning based food material freshness identification apparatus according to claim 6, further comprising:

the acquisition and construction unit is used for acquiring and constructing a food material image information set in advance;

the marking unit is further configured to: correspondingly marking the food material type and the actual freshness of each food material image information in the food material image information set;

the food freshness identification device further comprises:

and the training construction unit is used for training and constructing the food freshness identification model by taking each piece of food material image information in the food material image information set as an input sample based on deep learning.

8. The deep learning based food material freshness identification apparatus according to claim 7, further comprising:

the output unit is used for outputting result information by taking each piece of food material image information in the food material image information set as the input sample through convolutional layer operation, pooling layer operation, normalization operation and full-link layer operation in sequence based on deep learning;

a calculation unit for calculating a loss function value based on the result information and the label information of the input sample;

the training construction unit is further configured to: according to the loss function value, updating corresponding parameters of the convolution layer operation, the pooling layer operation, the normalization operation and the full-connection layer operation so as to train and construct the food freshness identification model;

the judging unit is further configured to: judging whether the absolute value of the loss function value is smaller than a second preset threshold value or not;

the training construction unit is further configured to: if the absolute value of the loss function value is judged to be larger than or equal to the second preset threshold value, continuing to train and construct the food freshness identification model by taking each piece of food material image information in the food material image information set as the input sample;

the food freshness identification device further comprises:

and the storage unit is used for storing the trained food freshness identification model if the loss function value is judged to be smaller than the second preset threshold value.

9. A refrigerator, characterized by comprising:

the deep learning based food material freshness identification apparatus according to any one of claims 5 to 8;

and the image collector is arranged in the refrigerator, is connected with the food freshness identification device and is used for collecting food material image information under the control of the food freshness identification device.

10. A computer readable storage medium, on which a computer program is stored, wherein the computer program, when being executed by a processor, implements the steps of the deep learning based food material freshness identification method according to any one of claims 1 to 4.