CN114758338A - Method and device for determining food material information, storage medium and electronic device - Google Patents
Method and device for determining food material information, storage medium and electronic device Download PDFInfo
- Publication number
- CN114758338A CN114758338A CN202210289652.6A CN202210289652A CN114758338A CN 114758338 A CN114758338 A CN 114758338A CN 202210289652 A CN202210289652 A CN 202210289652A CN 114758338 A CN114758338 A CN 114758338A
- Authority
- CN
- China
- Prior art keywords
- food material
- information
- depth
- determining
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C3/00—Measuring distances in line of sight; Optical rangefinders
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2379—Updates performed during online database operations; commit processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
- G16H20/60—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to nutrition control, e.g. diets
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Epidemiology (AREA)
- Primary Health Care (AREA)
- Electromagnetism (AREA)
- General Health & Medical Sciences (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Public Health (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Nutrition Science (AREA)
- Geometry (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Cold Air Circulating Systems And Constructional Details In Refrigerators (AREA)
Abstract
The application discloses a method and a device for determining food material information, a storage medium and an electronic device, and relates to the technical field of smart families, wherein the method comprises the following steps: under the condition that the 3D single-point camera detects that the refrigerator drawer is opened, the depth camera is started to take pictures of food materials in the refrigerator drawer through the depth camera; acquiring a first image of the food material acquired by a depth camera and first depth information corresponding to the food material; determining the access state of the food material and the position information of the food material according to the first depth information, and identifying the first image to determine the food material type of the food material in the first image; under the condition that the food material is confirmed to be located in the refrigerator drawer again according to the position information, the food material information is adjusted according to the access state and the food material type of the food material, wherein the food material information is the information of the food material stored in the refrigerator drawer.
Description
Technical Field
The invention relates to the field of communication, in particular to a method and a device for determining food material information, a storage medium and an electronic device.
Background
With the progress of scientific technology and the development of artificial intelligence, an intelligent algorithm is also applied to daily life more and more, particularly for a refrigerator, the intelligent development of the intelligent algorithm is crucial as one of daily household appliances with high use frequency, and the most critical problem of intelligence is to provide convenience for daily life. At present, a camera is arranged on a door body or a refrigerator body of an intelligent refrigerator to shoot food materials in the refrigerator, and the type information of the food materials is obtained through an image recognition algorithm.
However, the existing refrigerator has an oxygen-control fresh-keeping MSA special area in a refrigerating space, the special area is a drawer-shaped space and is used for storing the food materials, so that the food materials can be stored for a longer time, the fresh-keeping effect is better, but the special area is of a drawer-type structure, and the existing camera cannot identify the information of the food materials in the special area.
Aiming at the problem that food material identification cannot be carried out on a refrigerator drawer in the related art, an effective solution is not provided yet.
Disclosure of Invention
The embodiment of the invention provides a method and a device for determining food material information, a storage medium and an electronic device, and at least solves the problem that food material identification cannot be performed on a refrigerator drawer in the related art.
According to an embodiment of the present invention, there is provided a method for determining food material information, including: under the condition that the 3D single-point camera detects that the refrigerator drawer is opened, starting the depth camera to photograph food materials in the refrigerator drawer through the depth camera; acquiring a first image of the food material acquired by a depth camera and first depth information corresponding to the food material; determining the access state of the food material and the position information of the food material according to the first depth information, and identifying the first image to determine the food material type of the food material in the first image; and under the condition that the food material is confirmed to be positioned in the refrigerator drawer again according to the position information, adjusting the food material information according to the access state and the food material type of the food material, wherein the food material information is the information of the food material stored in the refrigerator drawer.
In one exemplary embodiment, activating a depth camera includes: acquiring a first distance between the refrigerator drawer and the inner wall of the refrigerator through the 3D single-point camera; determining a first size relation between the first distance and a first preset distance; and sending a first shooting instruction to the depth camera to start the depth camera under the condition that the first size relation indicates that the first distance is greater than the first preset distance.
In an exemplary embodiment, after adjusting the food material information according to the access status of the food material and the food material category, the method further comprises: acquiring a second distance of the refrigerator drawer relative to the inner wall of the refrigerator through the 3D single-point camera; determining a second size relation between the second distance and a second preset distance; sending a second shooting instruction to the depth camera under the condition that the second size relation indicates that the second distance is equal to the second preset distance, so that the depth camera continuously shoots a plurality of second images; determining a third image in the plurality of second images, wherein the third image is an image which is shot by the depth camera and only contains food materials in a refrigerator drawer, and the difference between the timestamp of the third image and the current time is the minimum; identifying the third image to determine food material information of food material stored in the refrigerator drawer.
In an exemplary embodiment, determining the access status of the food material according to the first depth information comprises: acquiring second depth information corresponding to the food material, wherein the second depth information is depth information acquired by the depth camera before or after the first depth information is acquired; determining that the food material access state is a storage state under the condition that the second depth information is depth information acquired by the depth camera before the first depth information is acquired; and determining that the access state of the food material is a taking-out state when the second depth information is the depth information acquired by the depth camera after the first depth information is acquired.
In an exemplary embodiment, adjusting the food material information according to the access status of the food material and the food material category comprises: deleting the food material information corresponding to the food material when the access state is the taking-out state; and increasing the food material information corresponding to the food material under the condition that the access state is the storage state.
In an exemplary embodiment, the method further comprises, in determining the access status of the food material and the position information of the food material according to the first depth information, and in identifying the first image: acquiring point cloud information of food materials acquired by a depth camera, and identifying the point cloud information to obtain a first volume of the food materials; determining the corresponding relation between the volume and the heat of the food material according to the type of the food material; and determining the heat quantity of the food material according to the corresponding relation between the volume and the heat quantity of the food material.
In an exemplary embodiment, after adjusting the food material information according to the access status of the food material and the food material category, the method further comprises: under the condition that the access state is the taking-out state, acquiring the heat of the food material and acquiring the health condition of a target object, wherein the target object is the object for acquiring the food material; determining a target diet recipe according to the calorie, the food material types and the basic information of the target object; and sending the target diet recipe to a terminal bound by the target object so as to enable the terminal to display the target diet recipe.
In an exemplary embodiment, after determining the access status of the food material and the position information of the food material according to the first depth information, and identifying the first image to determine the food material category of the food material in the first image, the method further comprises: and under the condition that the position information indicates that the food material is located in other storage areas of the refrigerator, adjusting the food material information corresponding to the other storage areas according to the access state of the food material and the food material type.
According to another embodiment of the present invention, there is also provided a food material information determining apparatus, including: the starting module is used for starting the depth camera to shoot food materials in the refrigerator drawer through the depth camera under the condition that the 3D single-point camera detects that the refrigerator drawer is opened; the acquisition module is used for acquiring a first image of the food material acquired by the depth camera and first depth information corresponding to the food material; a determining module, configured to determine, according to the first depth information, an access state of the food material and position information of the food material, and identify the first image to determine a food material type of the food material in the first image; and the adjusting module is used for adjusting the food material information according to the access state of the food material and the food material type under the condition that the food material is confirmed to be positioned in the refrigerator drawer again according to the position information, wherein the food material information is the information of the food material stored in the refrigerator drawer.
According to still another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium having a computer program stored therein, wherein the computer program is configured to execute the method for determining food material information when running.
According to another aspect of the embodiment of the present invention, there is provided an electronic device, including a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the method for determining the food material information through the computer program.
In the embodiment of the invention, under the condition that the 3D single-point camera detects that the refrigerator drawer is opened, the depth camera is started to photograph food materials in the refrigerator drawer through the depth camera; acquiring a first image of the food material acquired by a depth camera and first depth information corresponding to the food material; determining the access state of the food material and the position information of the food material according to the first depth information, and identifying the first image to determine the food material type of the food material in the first image; under the condition that the food material is confirmed to be located in the refrigerator drawer again according to the position information, adjusting the food material information according to the access state and the food material type of the food material, wherein the food material information is information of the food material stored in the refrigerator drawer; by adopting the technical scheme, the problem that the food material identification cannot be carried out on the refrigerator drawer is solved, and further the embodiment of the invention adopts the 3D single-point camera to measure the opening and closing distance of the drawer in the special refrigerating area of the refrigerator, adopts the depth camera to carry out the food material taking in and out and the food material information identification in the special refrigerating area of the refrigerator, and solves the problem that the food material identification cannot be carried out on the refrigerator drawer.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and, together with the description, serve to explain the principles of the application.
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings without inventive labor.
Fig. 1 is a schematic hardware environment diagram of a method for determining food material information according to an embodiment of the present application;
fig. 2 is a flowchart of a method for determining food material information according to an embodiment of the invention;
fig. 3 is a schematic diagram of a determination method of food material information according to an embodiment of the invention;
fig. 4 is a block diagram (one) of the structure of an apparatus for determining food material information according to an embodiment of the present invention;
fig. 5 is a block diagram of a structure of a food material information determining apparatus according to an embodiment of the present invention (ii).
Detailed Description
In order to make the technical solutions of the present application better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
According to an aspect of the embodiment of the application, a method for determining food material information is provided. The method for determining the food material information is widely applied to full-House intelligent digital control application scenes such as Smart Home (Smart Home), Smart Home equipment ecology, Smart Home (Intelligent House) ecology and the like. Alternatively, in the present embodiment, the method for determining food material information may be applied to a hardware environment formed by the terminal device 102 and the server 104 as shown in fig. 1. As shown in fig. 1, the server 104 is connected to the terminal device 102 through a network, and may be configured to provide a service (e.g., an application service) for the terminal or a client installed on the terminal, provide a database on or independent of the server for providing a data storage service for the server 104, and configure a cloud computing and/or edge computing service on or independent of the server for providing a data operation service for the server 104.
The network may include, but is not limited to, at least one of: wired networks, wireless networks. The wired network may include, but is not limited to, at least one of: wide area networks, metropolitan area networks, local area networks, which may include, but are not limited to, at least one of the following: WIFI (Wireless Fidelity ), bluetooth. Terminal equipment 102 can be but not limited to be PC, the cell-phone, the panel computer, intelligent air conditioner, intelligent cigarette machine, intelligent refrigerator, intelligent oven, intelligent kitchen range, intelligent washing machine, intelligent water heater, intelligent washing equipment, intelligent dish washer, intelligent projection equipment, intelligent TV, intelligent clothes hanger, intelligent (window) curtain, intelligence audio-visual, smart jack, intelligent stereo set, intelligent audio amplifier, intelligent new trend equipment, intelligent kitchen guarding equipment, intelligent bathroom equipment, intelligence robot of sweeping the floor, intelligence robot of wiping the window, intelligence robot of mopping the ground, intelligent air purification equipment, intelligent steam ager, intelligent microwave oven, intelligent kitchen is precious, intelligent clarifier, intelligent water dispenser, intelligent lock etc..
In the present embodiment, a method for determining food material information is provided, and fig. 2 is a flowchart of the method for determining food material information according to the embodiment of the present invention, where the flowchart includes the following steps:
step S202, under the condition that the 3D single-point camera detects that the refrigerator drawer is opened, starting a depth camera to shoot food materials in the refrigerator drawer through the depth camera;
step S204, acquiring a first image of the food material acquired by a depth camera and first depth information corresponding to the food material;
step S206, determining the access state of the food material and the position information of the food material according to the first depth information, and identifying the first image to determine the food material type of the food material in the first image;
step S208, under the condition that the food material is confirmed to be located in the refrigerator drawer again according to the position information, adjusting food material information according to the access state of the food material and the food material type, wherein the food material information is information of the food material stored in the refrigerator drawer.
Through the steps, under the condition that the 3D single-point camera detects that the refrigerator drawer is opened, the depth camera is started to photograph the food materials in the refrigerator drawer through the depth camera; acquiring a first image of the food material acquired by a depth camera and first depth information corresponding to the food material; determining the access state of the food material and the position information of the food material according to the first depth information, and identifying the first image to determine the food material type of the food material in the first image; under the condition that the food material is confirmed to be located in the refrigerator drawer again according to the position information, food material information is adjusted according to the access state of the food material and the type of the food material, wherein the food material information is information of the food material stored in the refrigerator drawer, and the problem that the food material identification cannot be carried out on the refrigerator drawer in the related technology is solved.
In one exemplary embodiment, activating a depth camera includes: acquiring a first distance between the refrigerator drawer and the inner wall of the refrigerator through the 3D single-point camera; determining a first size relation between the first distance and a first preset distance; and sending a first shooting instruction to the depth camera to start the depth camera under the condition that the first size relation indicates that the first distance is greater than the first preset distance.
That is, it is possible to acquire a first distance between the inner side of a drawer and the inner side of the refrigerator apparatus according to a 3D single point camera, and determine whether the detected first distance between the inner side of the drawer and the inner side of the refrigerator apparatus is greater than a first preset distance (e.g., 0), determine whether the refrigerator drawer is opened, and activate the depth camera in case that it is determined that the refrigerator drawer is opened.
Optionally, the starting of the depth camera includes at least one of: acquiring a first distance of the refrigerator drawer relative to the inner wall of the refrigerator through a distance sensor; determining a first size relation between the first distance and a first preset distance; sending a first shooting instruction to the depth camera to start the depth camera under the condition that the first size relation indicates that the first distance is greater than the first preset distance; determining whether a refrigerator door corresponding to the refrigerator drawer is opened; and starting the depth camera under the condition that the refrigerator door corresponding to the refrigerator drawer is determined to be opened.
In an exemplary embodiment, after food material information is adjusted according to the access state of the food material and the food material category, a second distance of the refrigerator drawer relative to the inner wall of the refrigerator is obtained through the 3D single-point camera; determining a second size relation between the second distance and a second preset distance; sending a second shooting instruction to the depth camera under the condition that the second size relation indicates that the second distance is equal to the second preset distance, so that the depth camera continuously shoots a plurality of second images; determining a third image in the plurality of second images, wherein the third image is an image which is shot by the depth camera and only contains food materials in a refrigerator drawer, and the difference between the timestamp of the third image and the current time is the minimum; identifying the third image to determine food material information of food material stored in the refrigerator drawer.
That is, a second distance between the inner side of the drawer and the inner side of the refrigerator device is obtained according to the 3D single-point camera, and it is determined whether the detected second distance between the inner side of the drawer and the inner side of the refrigerator device is greater than a second preset distance (for example, a maximum distance between the inner side of the drawer and the inner side of the refrigerator device), the depth camera is controlled to continuously shoot a plurality of second images, and a third image only containing food material images is determined among the plurality of second images, and the third image is recognized to determine food material information in the refrigerator drawer.
When the third images are a plurality of images, time stamps corresponding to the plurality of third images are determined, the time stamp with the minimum difference value with the current time is determined according to the time stamps, the fourth image corresponding to the time stamp with the minimum difference value with the current time is identified, and the food material information in the refrigerator drawer is determined, so that the food material information in the refrigerator drawer can be acquired more accurately.
In an exemplary embodiment, determining the access status of the food material according to the first depth information comprises: acquiring second depth information corresponding to the food material, wherein the second depth information is depth information acquired by the depth camera before or after the first depth information is acquired; determining that the access state of the food material is a storage state under the condition that the second depth information is depth information acquired by the depth camera before the first depth information is acquired; and determining that the access state of the food material is a taking-out state when the second depth information is the depth information acquired by the depth camera after the first depth information is acquired.
It should be noted that, since the first depth information is used to indicate depth information corresponding to an instant at which the target object puts the food material into the refrigerator drawer or takes the food material out of the refrigerator drawer, in a case where the first depth information is used to indicate depth information corresponding to an instant at which the target object puts the food material into the refrigerator drawer, the depth camera inevitably acquires second depth information corresponding to the food material before acquiring the first depth information; and under the condition that the first depth information is used for indicating the depth information corresponding to the moment that the target object takes the food out of the refrigerator drawer, the depth camera acquires the first depth information and then necessarily acquires the second depth information corresponding to the food.
In an exemplary embodiment, adjusting the food material information according to the access status of the food material and the food material category comprises: deleting the food material information corresponding to the food material when the access state is the taking-out state; and increasing the food material information corresponding to the food materials under the condition that the access state is the storage state.
In other words, the taking out or putting in of the target object determines the access state of the food material, and the food material information corresponding to the food material in the refrigerator drawer is deleted under the condition that the target object takes out the food material; and under the condition that the food material is put into the target object, adding food material information corresponding to the food material in the refrigerator drawer.
In an exemplary embodiment, the access state of the food material and the position information of the food material are determined according to the first depth information, and in the process of identifying the first image, point cloud information of the food material acquired by a depth camera is acquired, and the point cloud information is identified to obtain a first volume of the food material; determining the corresponding relation between the volume and the heat of the food material according to the type of the food material; and determining the heat quantity of the food material according to the corresponding relation between the volume and the heat quantity of the food material.
Optionally, in the embodiment of the present invention, the heat corresponding to the food material is determined according to the first volume of the food material and the type of the food material, it should be noted that the point cloud information may be understood as point cloud data, and since the point cloud data has a spatial coordinate, the volume size of the food material may be determined according to the spatial coordinate data.
Optionally, the heat may also be determined by one of the following: under the condition that the heat identification device is a voice interaction device, acquiring first voice information through the voice interaction device, and identifying the heat of the food material from the first voice information; under the condition that the heat identification device is a tag identification device, tag information of the food material is acquired through the tag identification device, and the heat of the food material is determined according to the tag information; and under the condition that the storage position identification device is an information interaction device, acquiring the heat of the input food material through the information interaction device.
In an exemplary embodiment, after food material information is adjusted according to the access state of the food material and the food material category, in the case that the access state is used for indicating a target object to take out the food material, acquiring the heat of the food material and acquiring the health condition of the target object; determining a target diet recipe according to the calorie, the food material types and the basic information of the target object; and sending the target diet recipe to a terminal bound by the target object so as to enable the terminal to display the target diet recipe.
According to the embodiment of the invention, a target diet recipe suitable for a target object is determined according to the health condition of the target object, the heat quantity of the food material and the food material type, and the target diet recipe is sent to a terminal bound by the target object, so that the target can cook the food material according to the target diet recipe, wherein the target diet recipe at least comprises the following components: the amount of the edible material suitable for the target subject.
Optionally, the health condition of the target subject is obtained at least by one of the following means: 1) determining the health condition of the target object through the weight information and the body fat information uploaded by the intelligent body fat scale; 2) determining the health condition of a target object through input information sent by the target object through a target terminal; 3) acquiring heart rate information and motion information of the target object through wearable equipment, and determining the health condition of the target object according to the heart rate information and the motion information.
In an exemplary embodiment, after determining the access state of the food material and the position information of the food material according to the first depth information, and identifying the first image to determine the food material type of the food material in the first image, if the position information indicates that the food material is located in another storage area of the refrigerator, the food material information corresponding to the other storage area is adjusted according to the access state of the food material and the food material type.
In order to better understand the process of the method for determining food material information, an implementation method flow of the determination of the food material information is described below with reference to an optional embodiment, but the technical solution of the embodiment of the present invention is not limited thereto.
In the present embodiment, a method for determining food material information is provided, and fig. 3 is a schematic diagram of the method for determining food material information according to the embodiment of the present invention, as shown in fig. 3, the method includes the following specific steps:
step S301: opening a 3D single-point camera;
step S302: opening and closing detection is carried out on the refrigerator drawer through a 3D single-point camera on the inner wall of the refrigerator;
step S303: detecting whether the opening distance of the drawer reaches an identification distance (corresponding to a second preset distance in the above embodiment), and executing step S304 when the opening distance of the drawer reaches the identification distance is detected; otherwise, go to step S302;
step S304: starting an RGBD camera (equivalent to the depth camera in the above embodiment) at the top of the refrigerator;
step S305: recognizing an RGB image (equivalent to a first image in the embodiment) acquired by an RGBD camera by using a deep learning target detection and recognition algorithm to obtain a food material type, recognizing taking out or putting in of a food material by using 3D information, and recognizing the volume of the food material by using the 3D information of the food material, so as to calculate the calorie content of the food material;
step S306: the recognition result (the kind of the food material, putting in or taking out, calorie information) is output to a terminal device, for example, a refrigerator, a mobile phone, or the like.
Through the embodiment, under the condition that the 3D single-point camera detects that the refrigerator drawer is opened, the depth camera is started to photograph food materials in the refrigerator drawer through the depth camera; acquiring a first image of the food material acquired by a depth camera and first depth information corresponding to the food material; determining the access state of the food material and the position information of the food material according to the first depth information, and identifying the first image to determine the food material type of the food material in the first image; under the condition that the food material is confirmed to be located in the refrigerator drawer again according to the position information, food material information is adjusted according to the access state of the food material and the type of the food material, wherein the food material information is information of the food material stored in the refrigerator drawer, and the problem that the food material identification cannot be carried out on the refrigerator drawer in the related art is solved.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
In this embodiment, a food material information determining device is further provided, and the food material information determining device is used for implementing the above embodiments and preferred embodiments, which have already been described and are not described again. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
Fig. 4 is a block diagram (one) of the structure of a food material information determining apparatus according to an embodiment of the present invention; as shown in fig. 4, includes:
the starting module 42 is configured to start the depth camera to photograph food materials in the refrigerator drawer through the depth camera when the 3D single-point camera detects that the refrigerator drawer is opened;
the acquiring module 44 is configured to acquire a first image of the food material acquired by the depth camera and first depth information corresponding to the food material;
a determining module 46, configured to determine an access status of the food material and position information of the food material according to the first depth information, and identify the first image to determine a food material type of the food material in the first image;
and an adjusting module 48, configured to adjust food material information according to the access state of the food material and the food material type when it is determined that the food material is located in the refrigerator drawer again according to the position information, where the food material information is information of a food material stored in the refrigerator drawer.
By the device, under the condition that the 3D single-point camera detects that the refrigerator drawer is opened, the depth camera is started to photograph food materials in the refrigerator drawer through the depth camera; acquiring a first image of the food material acquired by a depth camera and first depth information corresponding to the food material; determining the access state of the food material and the position information of the food material according to the first depth information, and identifying the first image to determine the food material type of the food material in the first image; under the condition that the food material is confirmed to be located in the refrigerator drawer again according to the position information, food material information is adjusted according to the access state of the food material and the type of the food material, wherein the food material information is information of the food material stored in the refrigerator drawer, and the problem that the food material identification cannot be carried out on the refrigerator drawer in the related art is solved.
In an exemplary embodiment, the starting module 42 is configured to obtain a first distance of the refrigerator drawer relative to an inner wall of the refrigerator through the 3D single point camera; determining a first size relation between the first distance and a first preset distance; and sending a first shooting instruction to the depth camera to start the depth camera under the condition that the first size relation indicates that the first distance is greater than the first preset distance.
In an exemplary embodiment, the determining module 46 is configured to obtain a second distance of the refrigerator drawer relative to the inner wall of the refrigerator through the 3D single point camera; determining a second size relation between the second distance and a second preset distance; sending a second shooting instruction to the depth camera under the condition that the second size relation indicates that the second distance is equal to the second preset distance, so that the depth camera continuously shoots a plurality of second images; determining a third image in the plurality of second images, wherein the third image is an image which is shot by the depth camera and only contains food materials in a refrigerator drawer, and the difference between the timestamp of the third image and the current time is the minimum; identifying the third image to determine food material information of food material stored in the refrigerator drawer.
In an exemplary embodiment, the determining module 46 is further configured to obtain second depth information corresponding to the food material, where the second depth information is depth information obtained by the depth camera before or after obtaining the first depth information; determining that the food material access state is a storage state under the condition that the second depth information is depth information acquired by the depth camera before the first depth information is acquired; and determining that the access state of the food material is a taking-out state when the second depth information is the depth information acquired by the depth camera after the first depth information is acquired.
In an exemplary embodiment, fig. 5 is a block diagram (ii) of a structure of an apparatus for determining food material information according to an embodiment of the present invention; as shown in fig. 5, the adjusting module further includes: a deleting module 52, configured to delete the food material information corresponding to the food material when the access state is the taking-out state; an adding module 54, configured to add food material information corresponding to the food material when the access state is the storage state.
In an exemplary embodiment, the determining module is further configured to acquire point cloud information of the food material acquired by the depth camera, and identify the point cloud information to obtain a first volume of the food material; determining the corresponding relation between the volume and the heat of the food material according to the type of the food material; and determining the heat quantity of the food material according to the corresponding relation between the volume and the heat quantity of the food material.
In an exemplary embodiment, after determining the food material information according to the access status of the food material and the food material category, the method further comprises: under the condition that the access state is the taking-out state, acquiring the heat of the food material and acquiring the health condition of the target object; determining a target diet recipe according to the calorie, the food material types and the basic information of the target object; and sending the target diet recipe to a terminal bound by the target object so as to enable the terminal to display the target diet recipe.
In an exemplary embodiment, the adjusting module is configured to, in a case that the location information indicates another storage area of the refrigerator, adjust food material information corresponding to the another storage area according to the access state of the food material and the food material type.
An embodiment of the present invention further provides a storage medium including a stored program, wherein the program executes any one of the methods described above.
Alternatively, in the present embodiment, the storage medium may be configured to store program codes for performing the following steps:
s1, starting a depth camera to shoot food materials in the refrigerator drawer through the depth camera under the condition that the 3D single-point camera detects that the refrigerator drawer is opened;
s2, acquiring a first image of the food material acquired by the depth camera and first depth information corresponding to the food material;
s3, determining the access state of the food material and the position information of the food material according to the first depth information, and identifying the first image to determine the food material type of the food material in the first image;
and S4, adjusting food material information according to the access state of the food material and the food material type under the condition that the food material is confirmed to be located in the refrigerator drawer again according to the position information, wherein the food material information is the information of the food material stored in the refrigerator drawer.
Embodiments of the present invention also provide an electronic device comprising a memory having a computer program stored therein and a processor arranged to run the computer program to perform the steps of any of the above method embodiments.
Optionally, the electronic apparatus may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.
Optionally, in this embodiment, the processor may be configured to execute the following steps by a computer program:
s1, starting a depth camera to shoot food materials in the refrigerator drawer through the depth camera under the condition that the 3D single-point camera detects that the refrigerator drawer is opened;
s2, acquiring a first image of the food material acquired by the depth camera and first depth information corresponding to the food material;
s3, determining the access state of the food material and the position information of the food material according to the first depth information, and identifying the first image to determine the food material type of the food material in the first image;
and S4, adjusting food material information according to the access state of the food material and the food material type under the condition that the food material is confirmed to be located in the refrigerator drawer again according to the position information, wherein the food material information is the information of the food material stored in the refrigerator drawer.
Optionally, in this embodiment, the storage medium may include, but is not limited to: various media capable of storing program codes, such as a usb disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk.
Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and this embodiment is not described herein again.
It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and alternatively, they may be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
The foregoing is only a preferred embodiment of the present application and it should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.
Claims (10)
1. A method for determining food material information is characterized by comprising the following steps:
under the condition that the 3D single-point camera detects that the refrigerator drawer is opened, starting a depth camera to photograph food materials in the refrigerator drawer through the depth camera;
acquiring a first image of the food material acquired by a depth camera and first depth information corresponding to the food material;
determining the access state of the food material and the position information of the food material according to the first depth information, and identifying the first image to determine the food material type of the food material in the first image;
and under the condition that the food material is confirmed to be located in the refrigerator drawer again according to the position information, adjusting food material information according to the access state of the food material and the food material type, wherein the food material information is the information of the food material stored in the refrigerator drawer.
2. The method of claim 1, wherein activating a depth camera comprises:
acquiring a first distance between the refrigerator drawer and the inner wall of the refrigerator through the 3D single-point camera;
determining a first size relation between the first distance and a first preset distance;
and sending a first shooting instruction to the depth camera to start the depth camera under the condition that the first size relation indicates that the first distance is greater than the first preset distance.
3. The method of claim 2, wherein after adjusting the food material information according to the access status of the food material and the food material category, the method further comprises:
acquiring a second distance of the refrigerator drawer relative to the inner wall of the refrigerator through the 3D single-point camera;
determining a second size relation between the second distance and a second preset distance;
sending a second shooting instruction to the depth camera under the condition that the second size relation indicates that the second distance is equal to the second preset distance, so that the depth camera continuously shoots a plurality of second images;
determining a third image in the plurality of second images, wherein the third image is an image which is shot by the depth camera and only contains food materials in a refrigerator drawer, and the difference between the timestamp of the third image and the current time is the minimum;
identifying the third image to determine food material information of food material stored in the refrigerator drawer.
4. The method of claim 1, wherein determining the access status of the food material according to the first depth information comprises:
acquiring second depth information corresponding to the food material, wherein the second depth information is depth information acquired by the depth camera before or after the first depth information is acquired;
determining that the food material access state is a storage state under the condition that the second depth information is depth information acquired by the depth camera before the first depth information is acquired;
and determining that the access state of the food material is a taking-out state when the second depth information is the depth information acquired by the depth camera after the first depth information is acquired.
5. The method of claim 1, wherein adjusting food material information based on the access status of the food material and the food material type comprises:
deleting the food material information corresponding to the food material when the access state is the taking-out state;
and increasing the food material information corresponding to the food materials under the condition that the access state is the storage state.
6. The method of claim 1, wherein determining the access status of the food material and the location information of the food material according to the first depth information, and identifying the first image further comprises:
acquiring point cloud information of food materials acquired by a depth camera, and identifying the point cloud information to obtain a first volume of the food materials;
determining the corresponding relation between the volume and the heat of the food material according to the type of the food material;
and determining the heat of the food material according to the corresponding relation between the volume and the heat of the food material.
7. The method of claim 6, wherein after adjusting the food material information according to the access status of the food material and the food material category, the method further comprises:
under the condition that the access state is the taking-out state, acquiring the heat of the food material and acquiring the health condition of a target object;
determining a target diet recipe according to the calorie, the food material types and the basic information of the target object;
and sending the target diet recipe to a terminal bound by the target object so as to enable the terminal to display the target diet recipe.
8. An apparatus for determining food material information, comprising:
the starting module is used for starting the depth camera to shoot food materials in the refrigerator drawer through the depth camera under the condition that the 3D single-point camera detects that the refrigerator drawer is opened;
the acquisition module is used for acquiring a first image of the food material acquired by the depth camera and first depth information corresponding to the food material;
the determining module is used for determining the access state of the food material and the position information of the food material according to the first depth information, and identifying the first image so as to determine the food material type of the food material in the first image;
and the adjusting module is used for adjusting the food material information according to the access state of the food material and the food material type under the condition that the food material is confirmed to be positioned in the refrigerator drawer again according to the position information, wherein the food material information is the information of the food material stored in the refrigerator drawer.
9. A computer-readable storage medium, comprising a stored program, wherein the program when executed performs the method of any of claims 1 to 7.
10. An electronic device comprising a memory and a processor, characterized in that the memory has stored therein a computer program, the processor being arranged to execute the method of any of claims 1 to 7 by means of the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210289652.6A CN114758338A (en) | 2022-03-23 | 2022-03-23 | Method and device for determining food material information, storage medium and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210289652.6A CN114758338A (en) | 2022-03-23 | 2022-03-23 | Method and device for determining food material information, storage medium and electronic device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114758338A true CN114758338A (en) | 2022-07-15 |
Family
ID=82327949
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210289652.6A Pending CN114758338A (en) | 2022-03-23 | 2022-03-23 | Method and device for determining food material information, storage medium and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114758338A (en) |
-
2022
- 2022-03-23 CN CN202210289652.6A patent/CN114758338A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110426964A (en) | Domestic electric appliances controller, smart machine, household electrical appliances, appliance control system and method | |
| CN108181819B (en) | Linkage control method, device and system for household electrical appliance and household electrical appliance | |
| CN111643017B (en) | Cleaning robot control method and device based on schedule information and cleaning robot | |
| CN109358546B (en) | Control method, device and system of household appliance | |
| CN114821236A (en) | Smart home environment sensing method, system, storage medium and electronic device | |
| CN114855416B (en) | Method and device for recommending washing program, storage medium and electronic device | |
| CN113269828B (en) | Method and device for searching articles, air conditioning equipment and storage medium | |
| CN114266959A (en) | Food cooking method and device, storage medium and electronic device | |
| CN114758338A (en) | Method and device for determining food material information, storage medium and electronic device | |
| CN115345225A (en) | Method and device for determining recommended scene, storage medium and electronic device | |
| CN115171153A (en) | Object type determination method and device, storage medium and electronic device | |
| CN110762943B (en) | Article display method and device and household appliance | |
| CN114691731A (en) | Usage preference determination method and apparatus, storage medium, and electronic apparatus | |
| CN117726908B (en) | Training method and device for picture generation model, storage medium and electronic device | |
| CN110457004A (en) | The playback method and device of prompt information | |
| CN114691730A (en) | Storage position prompting method and device, storage medium and electronic device | |
| CN113467299B (en) | Equipment door control method and device, storage medium and electronic device | |
| CN113286312B (en) | Method and device for providing solution according to network distribution condition | |
| CN110749137A (en) | Refrigerator and control method for refrigerator | |
| CN112702931A (en) | Flexible clothing, goods shelf and display control method and system | |
| CN116452881B (en) | Food nutritive value detection method, device, equipment and storage medium | |
| CN114839910A (en) | Method and device for sending control command, storage medium and electronic device | |
| CN115542760A (en) | Menu management method and device, storage medium and electronic device | |
| CN114743083A (en) | Maturity state determining method and apparatus, storage medium and electronic apparatus | |
| CN115032907A (en) | Method and device for updating perception parameters, storage medium and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |