CN110966833A - Method for detecting food material information in refrigerator and refrigerator - Google Patents
Method for detecting food material information in refrigerator and refrigerator Download PDFInfo
- Publication number
- CN110966833A CN110966833A CN201811161687.1A CN201811161687A CN110966833A CN 110966833 A CN110966833 A CN 110966833A CN 201811161687 A CN201811161687 A CN 201811161687A CN 110966833 A CN110966833 A CN 110966833A
- Authority
- CN
- China
- Prior art keywords
- detection model
- refrigerator
- data
- freshness
- hyperspectral
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
- F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
- F25D23/00—General constructional features
- F25D23/12—Arrangements of compartments additional to cooling compartments; Combinations of refrigerators with other equipment, e.g. stove
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
- F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
- F25D29/00—Arrangement or mounting of control or safety devices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
- F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
- F25D2500/00—Problems to be solved
- F25D2500/06—Stock management
Abstract
The invention provides a method for detecting food material information in a refrigerator and the refrigerator. Wherein the storage compartment of the refrigerator is internally provided with a hyperspectral imaging device. The method for detecting the food material information in the refrigerator comprises the following steps: acquiring hyperspectral data of the food material shot by a hyperspectral imaging device; respectively extracting image data and spectrum data from the hyperspectral data; detecting the type of food materials according to the extracted image data, acquiring a freshness detection model and a nutrient component detection model corresponding to the type of the food materials, and inputting the extracted spectral data into the freshness detection model and the nutrient component detection model to obtain the information of the freshness and the nutrient component of the food materials in the refrigerator. The food material management is convenient for the user.
Description
Technical Field
The invention relates to the technical field of storage, in particular to a method for detecting food material information in a refrigerator and the refrigerator.
Background
With the progress of society and the improvement of living standard of people, consumers pay attention to the nutritional value and safety of food materials when purchasing the food materials, and also consider factors such as price, taste, appearance, freshness and the like, and the role of the refrigerator is gradually changed from simple storage and preservation to a food material management center and a family nutrition center, which also provides a new challenge for the refrigerator, and meanwhile, provides a chance for various intelligent detection technologies to be applied to the refrigerator. The mode of storing food material types in the refrigerator is solved, and the mode of actually checking by opening the refrigerator door is changed into intelligent detection. By utilizing an automatic detection technology, the function of detecting the types of food materials on a household refrigerator is realized, and the development trend of an intelligent refrigerator is reached.
The automatic detection technology is a technology which applies a specific detection device, enables a detected article to approach the detection device, automatically obtains relevant information of the detected article, and provides the relevant information to a computer processing system to complete relevant subsequent processing. The automatic detection technology applied to the refrigerator at present comprises radio frequency detection, image detection and the like, wherein radio frequency detection is realized by pasting radio frequency detection codes on food materials put in the refrigerator and detecting the food materials by using a radio frequency detection device installed on the refrigerator. The image detection technology is also applied to refrigerators, but the correct detection rate is low, and the technology mainly depends on detection of different colors of food material images or different shapes and textures of food materials, so that the food materials with similar colors and shapes are difficult to detect correctly, and the detection of the freshness of the food materials cannot be realized.
The existing food freshness detection is generally realized by adopting a gas sensor array. The gas sensor array is arranged in a refrigerator chamber, various gases are continuously released along with the prolonging of the storage time of food materials stored in the refrigerator, and at the moment, each gas sensor respectively responds to certain specific gases, so that the freshness and the change condition of the food materials are judged, and the freshness of the food materials is comprehensively judged.
When the gas sensor array detects the freshness of food materials, a large misjudgment risk exists. First, a typical gas sensor may be sensitive to a class of chemical substances, and many gases may contain the substance, so that the sensor may not be able to truly detect which gas caused a response, thereby causing a false determination. Secondly, when a plurality of food materials are mixed and placed, the emitted gas is mixed together, so that the sensor responds, but can not detect which food material is released, and misjudgment is caused.
For the existing detection of the nutrient components of the food materials, the existing method is to use different instruments such as a glucometer and a hardness meter to respectively detect each physicochemical index so as to obtain the total nutrient component information of the food materials.
The fast detection means, namely the spectrum method, which is gradually developed provides a fast detection means, and the spectrum information is related to the chemical composition and the molecular structure of the object to be detected, so that the nutrient component information of food can be accurately carried out. When the food to be detected is identified, firstly, the spectrum information of the food to be detected is obtained, then the characteristic spectrum information which can represent the food is extracted, a series of foods and the respective characteristic spectrum information are modeled, and then the nutrient content of the food to be detected can be detected. However, the method generally performs single-point detection, and the user experience is not good; meanwhile, the area of the detected area is small, and the result is often inconsistent when a certain food material is subjected to multi-point detection, so that the accuracy of the technology needs to be further improved.
Disclosure of Invention
The invention aims to provide a method for automatically detecting food material information.
A further object of the present invention is to improve the accuracy of detecting the food material information.
The invention firstly provides a method for detecting information of food materials in a refrigerator, which is suitable for a hyperspectral imaging device for shooting food materials in a storage chamber of the refrigerator. The method for detecting the food material information in the refrigerator comprises the following steps: acquiring hyperspectral data shot by a hyperspectral imaging device; preprocessing the hyperspectral data, and respectively extracting image data and spectral data; detecting the type of the food material according to the image data; acquiring a freshness detection model and a nutrient component detection model corresponding to the types of food materials, wherein the freshness detection model and the nutrient component detection model are obtained by training according to spectral data of the food materials with different qualities in advance respectively; respectively carrying out classification analysis calculation on the spectral data by using a freshness detection model and a nutrient component detection model so as to determine the freshness and nutrient component information of the food material; and outputting the information of the type, freshness and nutritional ingredients of the food materials in the refrigerator through a display screen of the refrigerator.
Optionally, the step of detecting the type of the food material according to the image data includes: acquiring image data; acquiring a food material type detection model, wherein the food material type detection model is obtained by training according to hyperspectral data of different types of food materials in advance; inputting the image data into a food material type detection model; and carrying out mode detection by the food material type detection model to obtain the type information of the food materials.
Optionally, the step of classifying the spectral data captured by the hyperspectral imaging device by using the nutrient content detection model comprises: extracting image data required by the nutrient component detection model from hyperspectral data shot by the hyperspectral imaging device; inputting image data required by the nutrient component detection model into the nutrient component detection model; and carrying out mode detection by using the nutrient component detection model to obtain the nutrient component information of the food material.
Optionally, the step of classifying the spectral data captured by the hyperspectral imaging device by the freshness detection model includes: extracting image data required by a freshness detection model from hyperspectral data shot by a hyperspectral imaging device; inputting image data required by the freshness detection model into the freshness detection model; and carrying out mode detection by the freshness detection model to obtain the freshness information of the food materials.
Optionally, the hyperspectral data includes a set number of ternary data sets, each ternary data set includes two image pixel elements and one spectral wavelength element of one pixel point, each pixel point has a plurality of sets of ternary data sets, the image data is obtained by analyzing and extracting data in the image pixel elements, and the spectral data is obtained by analyzing and extracting data in the spectral wavelength elements.
Optionally, the resolution of the spectral wavelength of each pixel in the hyperspectral data is less than or equal to 2 nm.
Optionally, in the process of starting the hyperspectral imaging device, a light source system matched with the hyperspectral imaging device is also started simultaneously to provide light required for shooting by the hyperspectral imaging device, wherein the spectral range of the light source system is 400nm to 1100 nm.
Optionally, the step of determining the freshness and the nutritional ingredient information of the food material further includes: and outputting the information of the type, freshness and nutritional ingredients of food materials in the refrigerator through a mobile terminal bound with the refrigerator.
According to another aspect of the invention, a refrigerator is also provided. The refrigerator includes: the refrigerator comprises a refrigerator body, a storage compartment and a storage box, wherein the refrigerator body is internally limited with the storage compartment; the hyperspectral imaging device is arranged in the storage room and is configured to shoot food materials in the storage room; and the controller comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the controller is used for realizing the method for detecting the food material information in the refrigerator.
Optionally, the refrigerator further comprises: and an information output interface configured to provide information of the food material to a display screen of the refrigerator or a mobile terminal bound with the refrigerator to output to a user.
According to the method for detecting the information of the food materials in the refrigerator and the refrigerator, the hyperspectral imaging device is arranged in the refrigerator, the hyperspectral data of the food materials are obtained by shooting, the hyperspectral data are used for detecting the type, freshness and nutritional ingredients of the food materials, the detection accuracy is high, and the requirements for quickly and nondestructively obtaining the type, freshness and nutritional ingredient information of the food materials are met.
Furthermore, according to the method for detecting the food material information in the refrigerator and the refrigerator, disclosed by the invention, by utilizing the characteristics of spectral information of food materials and the degree of the types, freshness and nutritional ingredients of the food materials, a mode detection technology is adopted, the freshness detection of the food materials is carried out by virtue of the freshness detection model, and the nutritional ingredients of the food materials are detected by virtue of the nutritional ingredients detection model, so that the accuracy of detecting the food material information is obviously improved, and the food material management is convenient for a user.
The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments thereof, taken in conjunction with the accompanying drawings.
Drawings
Some specific embodiments of the invention will be described in detail hereinafter, by way of illustration and not limitation, with reference to the accompanying drawings. The same reference numbers in the drawings identify the same or similar elements or components. Those skilled in the art will appreciate that the drawings are not necessarily drawn to scale. In the drawings:
fig. 1 is a schematic view of a refrigerator according to one embodiment of the present invention;
FIG. 2 is a functional schematic block diagram of a refrigerator according to one embodiment of the present invention;
fig. 3 is a schematic view of a refrigerator according to another embodiment of the present invention;
FIG. 4 is a schematic diagram of a method for detecting the type of food material in a refrigerator according to one embodiment of the invention; and
fig. 5 is a schematic view illustrating a method for detecting freshness and nutritional ingredients of food materials in a refrigerator according to an embodiment of the present invention.
Detailed Description
Fig. 1 is a schematic diagram of a refrigerator 10 according to one embodiment of the present invention. The refrigerator 10 of the present embodiment may generally include: the device comprises a box body 110, a door body 120 and a hyperspectral imaging device 210.
The cabinet 110 defines at least one storage compartment 130, typically a plurality of compartments, such as a refrigerating compartment, a freezing compartment, a temperature-changing compartment, and the like, having an open front side therein. The number and the functions of the storage compartments 130 can be configured according to the preset requirements, and in some embodiments, the preservation temperature of the refrigerating chamber can be 2-9 ℃, or 4-7 ℃; the preservation temperature of the freezing chamber can be-22 to-14 ℃, or can be-20 to 16 ℃. The freezing chamber is arranged below the refrigerating chamber, and the temperature-changing chamber is arranged between the freezing chamber and the refrigerating chamber. The temperature in the freezer compartment is typically in the range of-14 ℃ to-22 ℃. The temperature-changing chamber can be adjusted according to the requirements to store suitable food materials or be used as a fresh-keeping storage chamber.
And a door 120 provided at the front side of the cabinet 110 to open and close the storage compartment 130. For example, the door bodies 120 may be hingedly disposed at one side of the front portion of the cabinet 110, and the storage compartments 130 may be opened and closed by pivoting, and the number of the door bodies 120 may be matched with the number of the storage compartments 130, so that the storage compartments 130 may be individually opened one by one. For example, a refrigerating chamber door body, a freezing chamber door body and a temperature changing chamber door body can be respectively arranged for the refrigerating chamber, the freezing chamber and the temperature changing chamber. In some alternative embodiments, the door 120 may also be in the form of a side hung door, a side sliding door, a sliding door, or the like.
The storage chamber 130 is provided with cold energy by a refrigerating system so as to realize storage environments of refrigeration, freezing and temperature changing. The refrigeration system may be a refrigeration cycle system constituted by a compressor, a condenser, a throttle device, an evaporator, and the like. The evaporator is configured to provide cooling directly or indirectly into the storage compartment 130. For example, in a compression type direct cooling refrigerator, the evaporator can be arranged on the outer side or the inner side of the rear wall surface of the inner container of the refrigerator. In the compression type air-cooled refrigerator, the cabinet 110 further has an evaporator chamber therein, the evaporator chamber is communicated with the storage compartment 130 through an air path system, an evaporator is provided in the evaporator chamber, and a fan is provided at an outlet thereof to perform circulating refrigeration to the storage compartment 130. Since the box 110, the door 120, and the refrigeration system themselves are well known and easy to implement by those skilled in the art, the details of the box 110, the door 120, and the refrigeration system themselves are not described herein after in order to not obscure and obscure the invention of the present application.
The hyperspectral imaging device 210 is disposed in the storage compartment 130 of the refrigerator 10, and is configured to capture food materials inside the storage compartment 130 and output hyperspectral data.
The hyperspectral data can be a series of ternary data sets, each ternary data set comprises two image pixel elements and a spectral wavelength element of a pixel point, and each pixel point is provided with a plurality of sets of ternary data sets. Therefore, the hyperspectral data simultaneously obtain the continuous spectrum data of each pixel point and the continuous image data of each spectrum wave band. And the spectral data is obtained by analyzing and extracting data in the pixel elements of the image, and the spectral data is obtained by analyzing and extracting data in the wavelength elements of the spectrum. The hyperspectral image is an optical image with continuous wavelength, the spectral range can be set to be 200nm to 2500nm, the hyperspectral image has higher spectral resolution, and the resolution can reach 2-3 nm. The hyperspectral data may be represented by a three-dimensional data block, where two dimensions are image pixel information (x, y) and the third dimension is wavelength information (λ). The data cube obtained at n wavelengths by an image detector array with a resolution of x y pixels is a three-dimensional array of x y x λ.
In the present embodiment, it is preferable to use spectral data in a spectral range of 400nm to 1100nm, since detection of the information on the kind, freshness and nutritional components of the food material 300 is facilitated by a large amount of study of the spectral data in the above spectral range. The resolution requirement of the spectral wavelength of each pixel point in the hyperspectral data shot by the hyperspectral imaging device 210 is less than or equal to 2nm, so that the requirement is met.
The species detection model can be obtained by training hyperspectral data of a large number of food materials, and the training algorithm which can be adopted by the species detection model can comprise a BP neural network, a Support Vector Machine (SVM) and Adaboost. A plurality of different species detection models can be trained in advance according to the species of the food material, for example, corresponding species detection models can be trained for various meats, various fruits, and various vegetables.
The freshness detection model can be obtained by training hyperspectral data of a large number of food materials with different freshness, and the training algorithm which can be adopted by the freshness detection model can comprise a BP neural network, a Support Vector Machine (SVM) and Adaboost. Multiple different freshness detection models can be trained in advance according to the types of food materials, for example, corresponding freshness detection models can be trained for various meats, various fruits and various vegetables.
The nutrient component detection model can be obtained by training hyperspectral data of a large number of food materials with different qualities, and the training algorithm which can be adopted by the nutrient component detection model can comprise a BP neural network, a Support Vector Machine (SVM) and Adaboost. Various different nutrient component detection models can be trained in advance according to the types of food materials, for example, corresponding nutrient component detection models can be trained for various meats, various fruits and various vegetables.
When performing species detection, the following steps may be performed: image data required by the species detection model is extracted from the hyperspectral data captured by the hyperspectral imaging device 210, the image data required by the species detection model is input into the species detection model, and the species detection model performs mode detection to obtain the species of the food material.
When detecting freshness and nutrient components, the following steps can be executed: the method comprises the steps of obtaining a freshness detection model and a nutrient component detection model corresponding to the type of food materials, extracting spectral data required by the freshness detection model and the nutrient component detection model from hyperspectral data shot by a hyperspectral imaging device 210, inputting the spectral data into the freshness detection model and the nutrient component detection model respectively, and performing mode detection by the freshness detection model and the nutrient component detection model respectively to obtain freshness and nutrient components of the food materials.
Freshness and nutrient content detection can also be realized by means of a cloud technology, for example, after hyperspectral data shot by the hyperspectral imaging device 210 is acquired, a data processing device of the refrigerator 10 is subjected to primary processing, the hyperspectral data subjected to the primary processing is uploaded to a cloud end, a mode detection step of a freshness detection model and a nutrient content detection model is completed by the cloud end, and then freshness and nutrient content of the food material 300 are provided for the refrigerator 10 or a mobile terminal bound with the refrigerator 10 so as to be provided for a user. The freshness detection model and the nutrient component detection model are stored in the cloud, so that the data processing pressure of the refrigerator 10 is reduced.
The freshness can reflect the rancidity degree, mildew degree, dehydration degree and the like of the food materials. After the freshness exceeds the set degree, the user can be reminded in time. The nutritional ingredients can reflect the information of moisture, sugar degree, soluble solid, etc. of the food materials. The interaction with the user may be implemented by a human-computer interaction system of the refrigerator 10, for example, outputting information of the kind, freshness and nutritional composition of the above-mentioned food materials on a display screen of the refrigerator 10. In another embodiment, a message including information of the above-described food material's kind, freshness and nutritional components may be transmitted to a mobile terminal bound to the refrigerator 10, and a message fed back by a user through the mobile terminal may be received.
Fig. 2 is a functional schematic block diagram of a refrigerator 10 according to another embodiment of the present invention. The following components can be flexibly selected and added to the refrigerator 10 of this embodiment: controller 270, light source system 230, information output interface 250.
A controller 270, disposed in the refrigerator 10, for extracting image data and spectrum data from the hyperspectral data captured by the hyperspectral imaging apparatus 210, respectively, detecting the type of the food material 300 according to the extracted image data, and obtaining a freshness detection model and a nutritional component detection model corresponding to the type of the food material, where the freshness detection model and the nutritional component detection model are obtained by training according to the spectrum data of the food materials with different qualities in advance, respectively; classifying, analyzing and calculating the spectral data by using a freshness detection model so as to determine the freshness of the food materials; classifying, analyzing and calculating the spectral data by using a nutritional component detection model so as to determine the nutritional components of the food material; the controller 270 includes a memory 271 and a processor 272, a computer program is stored in the memory 271, and the processor 272 is configured to execute the computer program in the memory 271, where the computer program is used to implement the method for detecting information of food materials in a refrigerator in this embodiment.
The light source system 230 is disposed in the storage compartment 130 and configured to provide light required for photographing to the hyperspectral imaging device 210, wherein a spectral range of the light source system 230 is set to 400-1100 nm. The light source system 230 may be disposed at the rear portion of the top wall of the storage compartment 130 to provide a photographing light in an oblique downward direction. The light source system 230 may be activated simultaneously with the hyperspectral imaging apparatus 210 to provide light to the enclosed storage compartment 130.
The information output interface 250 may be configured to provide the kind, freshness and nutritional component information of the food material 300 to a display screen of the refrigerator or a mobile terminal bound to the refrigerator to be output to a user.
In order to ensure that the hyperspectral imaging device 210 can photograph the full view of the food material 300 placed in the storage compartment 130. The hyperspectral imaging device 210 preferably uses a wide-angle lens or a fisheye lens, and is disposed directly above the storage compartment 130.
Fig. 3 is a schematic view of a refrigerator 10 according to another embodiment of the present invention. In the refrigerator 10 of this example, aiming at the problem that the internal space of the refrigerator 10 is narrow and small, the hyperspectral imaging device 210 is difficult to shoot the overall appearance of the storage room 130, and by arranging the reflector 260, hyperspectral data reflecting the overall appearance of the storage room 130 is obtained by means of shooting a reflection image.
The reflector 260 and the hyperspectral imaging device 210 are oppositely arranged inside the storage compartment 130. The hyperspectral imaging apparatus 210 may be configured to photograph the mirror 260 to obtain hyperspectral data of an image reflected by the mirror 260. Since the space inside the refrigerator 10 is narrow and the storage compartment 130 is generally a flat layered structure for facilitating storage, in such a flat region where the space is narrow and narrow, the conventional hyperspectral imaging device 210 is difficult to photograph the entire appearance of the storage compartment 130, and therefore in this embodiment, the problem can be effectively solved by photographing the reflected image of the reflector 260. In some alternative embodiments, the reflector 260 may optionally be a convex mirror to reflect the entire storage compartment 130.
The mirror 260 is disposed on, for example, a top wall of the storage compartment, and the hyperspectral imaging apparatus 210 is disposed in, for example, a bottom wall of the storage compartment. The region where the hyperspectral imaging device 210 is can be set as a blank region, so that the user is prevented from placing the food material 300 to be detected above the hyperspectral imaging device 210 and shielding the lens.
Whether the hyperspectral imaging device 210 adopts an angle lens or a fisheye lens or a reflection mode of the reflector 260, the hyperspectral imaging device can obtain hyperspectral data reflecting the overall appearance of the storage compartment 130, so that the requirement of shooting the food material 300 is met.
When using the information detection function of the refrigerator 10, a specific example is: after a user places an apple in the storage compartment 130, the user issues a detection instruction through a button or a mobile terminal on the refrigerator 10. The hyperspectral imaging device 210 shoots the storage room 130 to obtain hyperspectral data including apples. From the hyperspectral data, image data and spectral data of the food material 300 (apple) can be extracted, the obtained image data determines the type of the food material 300 by utilizing algorithms such as a neural network and/or a chemometric method such as a PLS/SVM and the like, and a freshness detection model and a nutrient component detection model corresponding to the type of the food material are obtained, wherein the freshness detection model and the nutrient component detection model are obtained by respectively training according to the spectral data of the food materials with different qualities in advance; and respectively carrying out classification analysis calculation on the spectral data by using a freshness detection model and a nutrient component detection model so as to determine the freshness and the nutrient component of the food material.
In addition, the type detection function of the refrigerator 10 may be automatically started at regular time intervals to periodically detect the type, freshness and nutritional component information of the food in the storage compartment 130.
The detection result can be used for further establishing food material type, freshness and nutrient component information storage files in the refrigerator 10, recording food material type, freshness and nutrient component storage information and providing a data base for intelligent management of food materials.
The present embodiment further provides a method for detecting a type of food material in the refrigerator 10, which can be used in the refrigerator 10 of any of the above embodiments to detect the type information of the storage compartment 130 inside the refrigerator 10.
Fig. 4 is a schematic diagram of a method for detecting the type of food material in the refrigerator 10 according to an embodiment of the invention. The method for detecting the type of food material in the refrigerator 10 generally may include:
step S402, acquiring hyperspectral data shot by the hyperspectral imaging device 210;
step S404, preprocessing the hyperspectral data and extracting image data;
step S406, obtaining a food material type detection model;
step S408, inputting the image data into a food material type detection model;
step S410, performing mode detection by the food material type detection model to obtain the type information of the food material.
The species detection model can be obtained by training hyperspectral data of a large number of food materials, and the training algorithm which can be adopted by the species detection model can comprise a BP neural network, a Support Vector Machine (SVM) and Adaboost. A plurality of different species detection models can be trained in advance according to the species of the food material, for example, corresponding species detection models can be trained for various meats, various fruits, and various vegetables.
When performing species detection, the following steps may be performed: image data required by the species detection model is extracted from the hyperspectral data captured by the hyperspectral imaging device 210, the image data required by the species detection model is input into the species detection model, and the species detection model performs mode detection to obtain the species of the food material.
In step S402, the hyperspectral data may include a set number of ternary data sets, each ternary data set includes two image pixel elements and one spectral wavelength element of one pixel, each pixel has multiple sets of ternary data sets, and the resolution of the spectral wavelength of each pixel in the hyperspectral data is less than or equal to 2 nm. The light source system 230 is also required to provide a light source when the hyperspectral imaging device 210 is started for shooting, and in order to ensure that the spectral range of the spectral data can be within the range of 400nm to 1100nm which meets the detection requirement of the food material types, the spectral range of the light source system 230 needs to be within the range of 400nm to 1100 nm.
The present embodiment further provides a method for detecting freshness and nutritional ingredients of food in the refrigerator 10, which can be applied to the refrigerator 10 of any of the above embodiments to detect information of freshness and nutritional ingredients of the food 300 in the refrigerator 10.
Fig. 5 is a schematic view illustrating a method for detecting freshness and nutritional ingredients of food materials in a refrigerator according to an embodiment of the present invention. The method for detecting the freshness and the nutrient content of the food in the refrigerator generally comprises the following steps:
step S502, acquiring hyperspectral data of the food material 300 photographed by the hyperspectral imaging apparatus 210;
step S504, respectively extracting image data and spectrum data from the hyperspectral data;
step S506, detecting the type of food material according to the image data;
step S508, a freshness detection model and a nutrient component detection model corresponding to the type of the food material are obtained;
in step S510, the spectral data are classified by using a freshness detection model and a nutritional component model, so as to determine the freshness and the nutritional component of the food material 300.
The freshness detection model can be obtained by training hyperspectral data of a large number of food materials with different qualities, and the training algorithm which can be adopted by the freshness detection model can comprise a BP neural network, a Support Vector Machine (SVM) and Adaboost. Multiple different freshness detection models can be trained in advance according to the types of food materials, for example, corresponding freshness detection models are trained for various meats, various fruits and various vegetables.
The nutrient component detection model can be obtained by training hyperspectral data of a large number of food materials with different qualities, and the training algorithm which can be adopted by the nutrient component detection model can comprise a BP neural network, a Support Vector Machine (SVM) and Adaboost. A plurality of different nutrient component detection models can be trained in advance according to the types of food materials, for example, corresponding nutrient component detection models can be trained for various meats, various fruits and various vegetables.
One specific implementation of step S510 may include: extracting spectral information required by a freshness detection model and a nutrient component detection model from hyperspectral data shot by the hyperspectral imaging device 210; respectively inputting the spectral information required by the freshness detection model and the nutrient component detection model into the freshness detection model and the nutrient component detection model; the freshness and the nutrient content of the food material 300 are obtained by performing pattern recognition on the freshness detection model and the nutrient content model, respectively.
The hyperspectral data can comprise a set number of ternary data sets, each ternary data set comprises two image pixel elements and a spectral wavelength element of one pixel point, each pixel point is provided with a plurality of sets of ternary data sets, the spectral data are obtained by analyzing and extracting data in the image pixel elements, and the spectral data are obtained by analyzing and extracting data in the spectral wavelength elements. The resolution of the spectral wavelength of each pixel point in the hyperspectral data is less than or equal to 2 nm. In order to ensure that the spectral range of the spectral data can be in the range of 400nm to 1100nm that satisfies the detection requirement, the spectral range of the light source system 230 needs to be in the range of 400nm to 1100 nm.
The method for detecting information of food materials in a refrigerator according to the embodiment extracts image data and spectrum data from hyperspectral data, detects the type of the food material 300 in the refrigerator 10 by using the image data, selects a corresponding freshness detection model and a corresponding nutritional component model according to the type of the food material 300, inputs the spectrum data into the freshness detection model and the nutritional component model, and detects to obtain the freshness and nutritional component information of the food material 300. The hyperspectral imaging technology combines the spectrum detection technology with the image recognition technology, and can simultaneously acquire the spatial image data of food materials and the spectrum data of each point. The variety information of the food materials can be rapidly and nondestructively acquired by utilizing the image data; and then, by combining with the spectral data comprehensive analysis, the freshness and nutrient component information of a plurality of pixel points on the image can be obtained simultaneously, and the freshness and nutrient component information of a plurality of points is comprehensively calculated to finally obtain the food freshness and nutrient component information with high accuracy.
Thus, it should be appreciated by those skilled in the art that while a number of exemplary embodiments of the invention have been illustrated and described in detail herein, many other variations or modifications consistent with the principles of the invention may be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Accordingly, the scope of the invention should be understood and interpreted to cover all such other variations or modifications.
Claims (10)
1. A method for detecting information of food materials in a refrigerator is provided, a hyperspectral imaging device for shooting food materials in a storage chamber of the refrigerator is arranged in the storage chamber, and the method comprises the following steps:
acquiring hyperspectral data shot by the hyperspectral imaging device;
preprocessing the hyperspectral data, and respectively extracting image data and spectral data;
detecting the type of the food material according to the image data;
acquiring a freshness detection model and a nutrient component detection model corresponding to the types of food materials, wherein the freshness detection model and the nutrient component detection model are obtained by training according to hyperspectral data of the food materials with different qualities in advance respectively;
respectively carrying out classification analysis calculation on the spectral data by using the freshness detection model and the nutrient component detection model so as to determine the freshness and nutrient component information of the food material; and
and outputting the information of the type, freshness and nutritional ingredients of food materials in the refrigerator through a display screen of the refrigerator.
2. The method of claim 1, wherein the step of detecting the type of the food material from the image data comprises:
acquiring the image data;
acquiring a food material type detection model, wherein the food material type detection model is obtained by training according to hyperspectral data of different types of food materials in advance;
inputting the image data into the food material type detection model;
and carrying out mode detection by the food material type detection model to obtain the type information of the food materials.
3. The method of claim 1, wherein the step of classifying the spectral data captured by the hyperspectral imaging device using the nutrient detection model comprises:
extracting spectral data required by the nutrient component detection model from hyperspectral data shot by the hyperspectral imaging device;
inputting the spectral data required by the nutrient component detection model into the nutrient component detection model;
and carrying out mode detection by the nutritional component detection model to obtain the nutritional component information of the food material.
4. The method of claim 1, wherein the step of classifying spectral data captured by the hyperspectral imaging apparatus using the freshness detection model comprises:
extracting spectral data required by the freshness detection model from hyperspectral data shot by the hyperspectral imaging device;
inputting the spectral data required by the freshness detection model into the freshness detection model;
and carrying out mode detection by the freshness detection model to obtain the freshness information of the food materials.
5. The method of claim 1, wherein,
the high spectrum data comprises a set number of ternary data sets, each ternary data set comprises two image pixel elements and a spectrum wavelength element of a pixel point, each pixel point is provided with a plurality of groups of the ternary data sets, and
the image data is obtained by analyzing and extracting data in the image pixel elements, and the spectrum data is obtained by analyzing and extracting data in the spectrum wavelength elements.
6. The method of claim 5, wherein,
and the resolution ratio of the spectral wavelength of each pixel point in the hyperspectral data is less than or equal to 2 nm.
7. The method of claim 1, wherein,
in the process of starting the hyperspectral imaging device, a light source system matched with the hyperspectral imaging device is started simultaneously to provide light required by the hyperspectral imaging device for shooting, wherein the spectral range of the light source system is 400nm to 1100 nm.
8. The method of claim 1, wherein the step of determining the freshness and nutritional composition information of the food material further comprises:
and outputting the information of the type, freshness and nutritional ingredients of food materials in the refrigerator through a mobile terminal bound with the refrigerator.
9. A refrigerator, comprising:
the refrigerator comprises a refrigerator body, a storage compartment and a storage box, wherein the refrigerator body is internally limited with the storage compartment;
the hyperspectral imaging device is arranged in the storage room and is configured to shoot food materials in the storage room;
a controller comprising a memory and a processor, the memory having stored therein a computer program, and the computer program, when executed by the processor, for implementing the method of any one of claims 1-7.
10. The refrigerator of claim 9, further comprising:
an information output interface configured to provide information of the food material to a display screen of the refrigerator or a mobile terminal bound with the refrigerator for output to a user.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811161687.1A CN110966833B (en) | 2018-09-30 | 2018-09-30 | Method for detecting food material information in refrigerator and refrigerator |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811161687.1A CN110966833B (en) | 2018-09-30 | 2018-09-30 | Method for detecting food material information in refrigerator and refrigerator |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110966833A true CN110966833A (en) | 2020-04-07 |
| CN110966833B CN110966833B (en) | 2023-06-02 |
Family
ID=70029171
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811161687.1A Active CN110966833B (en) | 2018-09-30 | 2018-09-30 | Method for detecting food material information in refrigerator and refrigerator |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110966833B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113409915A (en) * | 2021-05-10 | 2021-09-17 | 王蕴秋 | Safety recommendation system capable of preventing food from deteriorating and working method thereof |
| CN113932553A (en) * | 2020-07-14 | 2022-01-14 | 青岛海尔电冰箱有限公司 | Refrigerator and control method thereof |
| EP3971502A1 (en) * | 2020-09-21 | 2022-03-23 | Samsung Electronics Co., Ltd. | Apparatus for inspecting meat, system for inspecting meat including the same, refrigerator including the same, and method of inspecting meat |
| CN114322438A (en) * | 2020-09-30 | 2022-04-12 | 青岛海尔智能技术研发有限公司 | Method and device for managing refrigerator food materials and refrigerator |
| WO2022242491A1 (en) * | 2021-05-18 | 2022-11-24 | 青岛海尔电冰箱有限公司 | Detection method for usage state of storage compartment of refrigerator, and refrigerator |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040211190A1 (en) * | 2002-09-17 | 2004-10-28 | The Furukawa Electric Co., Ltd. | Temperature adjustment device and laser module |
| CN102889727A (en) * | 2012-10-31 | 2013-01-23 | 昆山黄昆纺织机械有限公司 | Solar energy refrigerator |
| WO2016186301A1 (en) * | 2015-05-15 | 2016-11-24 | 피도연 | Food condition measuring device and food condition measuring system using same |
| CN106871568A (en) * | 2016-12-30 | 2017-06-20 | 青岛海尔智能技术研发有限公司 | The category identification method and refrigerator of refrigerator inside food |
| WO2018165605A1 (en) * | 2017-03-09 | 2018-09-13 | Northwestern University | Hyperspectral imaging sensor |
-
2018
- 2018-09-30 CN CN201811161687.1A patent/CN110966833B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040211190A1 (en) * | 2002-09-17 | 2004-10-28 | The Furukawa Electric Co., Ltd. | Temperature adjustment device and laser module |
| CN102889727A (en) * | 2012-10-31 | 2013-01-23 | 昆山黄昆纺织机械有限公司 | Solar energy refrigerator |
| WO2016186301A1 (en) * | 2015-05-15 | 2016-11-24 | 피도연 | Food condition measuring device and food condition measuring system using same |
| CN106871568A (en) * | 2016-12-30 | 2017-06-20 | 青岛海尔智能技术研发有限公司 | The category identification method and refrigerator of refrigerator inside food |
| WO2018165605A1 (en) * | 2017-03-09 | 2018-09-13 | Northwestern University | Hyperspectral imaging sensor |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113932553A (en) * | 2020-07-14 | 2022-01-14 | 青岛海尔电冰箱有限公司 | Refrigerator and control method thereof |
| EP3971502A1 (en) * | 2020-09-21 | 2022-03-23 | Samsung Electronics Co., Ltd. | Apparatus for inspecting meat, system for inspecting meat including the same, refrigerator including the same, and method of inspecting meat |
| US11852619B2 (en) | 2020-09-21 | 2023-12-26 | Samsung Electronics Co., Ltd. | Apparatus for inspecting meat, system for inspecting meat including the same, refrigerator including the same, and method of inspecting meat |
| CN114322438A (en) * | 2020-09-30 | 2022-04-12 | 青岛海尔智能技术研发有限公司 | Method and device for managing refrigerator food materials and refrigerator |
| CN113409915A (en) * | 2021-05-10 | 2021-09-17 | 王蕴秋 | Safety recommendation system capable of preventing food from deteriorating and working method thereof |
| WO2022242491A1 (en) * | 2021-05-18 | 2022-11-24 | 青岛海尔电冰箱有限公司 | Detection method for usage state of storage compartment of refrigerator, and refrigerator |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110966833B (en) | 2023-06-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110966833B (en) | Method for detecting food material information in refrigerator and refrigerator | |
| CN108663331A (en) | Detect the method and refrigerator of food freshness in refrigerator | |
| CN107873101B (en) | Imaging system for object recognition and evaluation | |
| CN108662842A (en) | The detecting system and refrigerator of food in refrigerator | |
| US11767984B2 (en) | Connected food preparation system and method of use | |
| CN110966834A (en) | Method for identifying food material types in refrigerator and refrigerator | |
| CN106871568A (en) | The category identification method and refrigerator of refrigerator inside food | |
| Cubero et al. | Advances in machine vision applications for automatic inspection and quality evaluation of fruits and vegetables | |
| Wang et al. | Application of hyperspectral imaging to discriminate the variety of maize seeds | |
| US20190234617A1 (en) | Connected food preparation system and method of use | |
| CN109655414B (en) | Electronic equipment, information pushing method and related product | |
| Goel et al. | HyperCam: hyperspectral imaging for ubiquitous computing applications | |
| Gowen et al. | Identification of mushrooms subjected to freeze damage using hyperspectral imaging | |
| CN108662843A (en) | The detecting system and refrigerator of food in refrigerator | |
| CN106871569A (en) | The category identification method and refrigerator of refrigerator inside food | |
| CN110966835B (en) | Method for detecting nutrient components of food materials in refrigerator and refrigerator | |
| CN110017659A (en) | Domestic refrigerator equipment | |
| CN115131783B (en) | Autonomous sensing method of user diet nutrition ingredient information based on machine vision | |
| CN109869966A (en) | Refrigerator | |
| CN110366731B (en) | Systems, methods, and storage media for image capture for instruction meal | |
| CN108663333A (en) | High light spectrum image-forming component and refrigerator for detecting food in refrigerator | |
| CN108139317A (en) | The Object identifying of storage organization | |
| US11928769B2 (en) | Employing controlled illumination for hyperspectral or multispectral imaging of food in an appliance | |
| Kwon et al. | Machine learning-enabled hyperspectral approaches for structural characterization of precooked noodles during refrigerated storage | |
| ARGUN et al. | Non-Destructive Prediction of Bread Staling Using Artificial Intelligence Methods |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20230518 Address after: 266101 Haier Industrial Park, 1 Haier Road, Laoshan District, Shandong, Qingdao Applicant after: QINGDAO HAIER SMART TECHNOLOGY R&D Co.,Ltd. Applicant after: Qingdao guochuang intelligent home appliance Research Institute Co.,Ltd. Address before: 266101 Haier Road, Laoshan District, Qingdao, Qingdao, Shandong Province, No. 1 Applicant before: QINGDAO HAIER SMART TECHNOLOGY R&D Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |