WO2022262234A1 - 一种冰箱及其食材管理方法及装置 - Google Patents

一种冰箱及其食材管理方法及装置 Download PDF

Info

Publication number
WO2022262234A1
WO2022262234A1 PCT/CN2021/139239 CN2021139239W WO2022262234A1 WO 2022262234 A1 WO2022262234 A1 WO 2022262234A1 CN 2021139239 W CN2021139239 W CN 2021139239W WO 2022262234 A1 WO2022262234 A1 WO 2022262234A1
Authority
WO
WIPO (PCT)
Prior art keywords
refrigerator
sensor
food
gravity sensor
storage room
Prior art date
Application number
PCT/CN2021/139239
Other languages
English (en)
French (fr)
Inventor
谢飞学
赵启东
董秀莲
田羽慧
李正义
曲磊
任贝贝
高语函
孙菁
马文华
张璧程
陆书轩
王艺深
朴艺兰
Original Assignee
海信集团控股股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202110676948.9A external-priority patent/CN115493343B/zh
Priority claimed from CN202110788223.9A external-priority patent/CN115615093A/zh
Priority claimed from CN202111037567.2A external-priority patent/CN115773608A/zh
Application filed by 海信集团控股股份有限公司 filed Critical 海信集团控股股份有限公司
Publication of WO2022262234A1 publication Critical patent/WO2022262234A1/zh

Links

Images

Classifications

    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
    • F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
    • F25D29/00—Arrangement or mounting of control or safety devices

Definitions

  • the present application relates to the technical field of refrigerators, in particular to a refrigerator and a food material management method and device thereof.
  • refrigerators have become an indispensable and important storage device in daily life.
  • the food location management of the refrigerator is one of the key points of food management, and it is also one of the difficulties.
  • a depth camera is installed on the top of the refrigerator. Through the depth camera, the image of the hand during the food storage process is obtained, and the storage location of the food is determined according to the depth information of the image.
  • the depth camera has a field of view (Field of Vision, FOV), and the depth camera cannot obtain images of areas outside the FOV (called blind spots).
  • FOV Field of Vision
  • the depth camera cannot obtain the first layer of the refrigerator freezer. The leftmost and rightmost regions.
  • the accuracy of determining the access location is low.
  • the present application provides a refrigerator and its food material management method and device, which are used to improve the accuracy of determining the access location and improve the management efficiency of the food material.
  • an embodiment of the present application provides a refrigerator, including:
  • the cabinet including a storage room, the storage room has an opening;
  • a refrigerator door movably connected with the casing, and used to cover the opening;
  • An image acquisition device arranged on the top of the casing, is used to acquire images of the movable area of the refrigerator door;
  • an auxiliary sensor located in the storage room, for collecting measurement information of the storage area in the storage room;
  • Controller configured as:
  • an embodiment of the present application provides a refrigerator, including:
  • the casing includes a storage room, the storage room has an opening, the storage room contains multiple storage spaces, each storage space is provided with at least one auxiliary sensor, and each auxiliary sensor is used to collect and measure information, and sending said measurement information to a controller;
  • a refrigerator door movably connected with the casing, and used to cover the opening;
  • Controller configured as:
  • an embodiment of the present application provides a refrigerator, including:
  • the cabinet including a storage room, the storage room has an opening;
  • a refrigerator door movably connected with the casing, and used to cover the opening;
  • An image acquisition device arranged on the top of the casing, is used to acquire images of the movable area of the refrigerator door;
  • the auxiliary sensor is installed in the storage room and is used to collect measurement information of the storage area in the storage room.
  • the measurement information is used when the refrigerator door is opened and there is no food in the image collected by the image collection device. During the access operation, it is judged whether there is an access operation of the food material.
  • an embodiment of the present application provides a refrigerator, including:
  • the cabinet including a storage room, the storage room has an opening;
  • a refrigerator door movably connected with the casing, and used to cover the opening;
  • the auxiliary sensor is arranged in the storage room, and is used to collect measurement information of the storage area in the storage room, and the measurement information is used to determine whether there is an access operation of food when the refrigerator door is opened.
  • an embodiment of the present application provides a refrigerator, including:
  • An auxiliary sensor includes a code scanner and a ranging sensor;
  • Controller configured as:
  • the access action includes: storing the food in the refrigerator, and taking the food from the refrigerator take out.
  • the controller can accurately determine the user's access action according to the first time and the second time, that is: determine whether the user stores the food in the refrigerator, or the user takes the food out of the refrigerator; In this way, the influence of information such as the surface smoothness and color of the food is eliminated, the recognition accuracy and recognition efficiency of the access action are effectively improved, and an effective data reference is provided for the management of the food in the refrigerator.
  • an embodiment of the present application provides a refrigerator, including:
  • the box body has a storage room inside, and the storage room includes at least one partition for storing ingredients, and the ingredients stored in the partition are identified by radio frequency identification (RFID) tags; the partition includes at least one gravity sensor , used to obtain the quality of the ingredients stored on the partition;
  • RFID radio frequency identification
  • the refrigerator door is arranged at the opening of the storage room, and the side of the refrigerator door close to the storage room includes at least one shelf for storing ingredients, and the ingredients stored in the shelf are identified by RFID tags;
  • the storage rack includes at least one gravity sensor, which is used to obtain the quality of the ingredients stored in the storage rack;
  • the side of the refrigerator door close to the storage room includes an RFID antenna, which is used to scan the RFID tags of the ingredients ;
  • Controller configured to execute:
  • the first gravity sensor is any gravity sensor in the at least one partition, or is any gravity sensor in the at least one shelf.
  • the user's action of storing/retrieving food is determined according to the change of the mass transmitted by the gravity sensor, and the user is determined according to the position of the gravity sensor whose mass is changed.
  • the location of the stored/retrieved ingredients is determined according to the food material information according to the RFID tag of the food material detected by the RFID antenna. It is no longer necessary to manually input the location and type of ingredients, and realizes a senseless and efficient ingredient management.
  • the embodiment of the present application provides a food management method, including:
  • the image acquisition device Responding to the opening of the refrigerator door, acquiring an image of the active area of the refrigerator door captured by the image acquisition device, and acquiring measurement information of the storage area in the storage room collected by the auxiliary sensor, wherein the image acquisition device is arranged on the top of the casing, and the auxiliary sensor is arranged in a storage room;
  • the auxiliary sensor is a TOF sensor, and the distance between the auxiliary sensor and the barrier included in the measurement information is less than or equal to the width of the storage room, then it is determined that there is an access operation for the food material; or, if The auxiliary sensor is an infrared through-beam photoelectric sensor, and the light intensity contained in the measurement information is less than the preset light intensity threshold value, then it is determined that there is a food access operation; or, if the auxiliary sensor is an infrared temperature sensor , and according to the temperature information contained in the measurement information, it is determined that there is an access operation of the food material.
  • an embodiment of the present application provides a food management method, including:
  • the embodiment of the present application provides a food material management method, which is applied to a refrigerator, and the storage room of the refrigerator includes at least one partition for storing food, and the partition includes at least one gravity sensor; the refrigerator The refrigerator door includes at least one shelf, the shelf includes at least one gravity sensor; the method includes:
  • the first gravity sensor is any gravity sensor in the at least one partition, or is any gravity sensor in the at least one shelf.
  • a computer-readable storage medium when the instructions in the computer-readable storage medium are executed by a controller, the controller is able to execute the ingredient management method as described in any one of the above.
  • the image of the active area of the refrigerator door is collected by the image acquisition device installed on the refrigerator, and the measurement information in the storage room is collected by the auxiliary sensor installed in the storage room.
  • the image collected by the image acquisition device cannot determine the existence of food access operations, it can be judged based on the measurement information collected by the auxiliary sensor, thereby improving the judgment accuracy of food material access operations, thereby improving the efficiency and accuracy of food management. Rate.
  • FIG. 1 is an application scene diagram of a refrigerator provided in an embodiment of the present application
  • Figure 2a is a schematic structural diagram of a refrigerator provided in the embodiment of the present application.
  • Fig. 2b is a schematic diagram of a compartment structure of a refrigerator provided in the embodiment of the present application.
  • FIG. 3a is a schematic diagram of the TOF sensor provided in the embodiment of the present application.
  • Figure 3b is a schematic diagram of the infrared through-beam photoelectric sensor provided in the embodiment of the present application.
  • Fig. 3c is another schematic diagram of the infrared through-beam photoelectric sensor provided in the embodiment of the present application.
  • Figure 3d is a schematic diagram of the infrared temperature sensor provided in the embodiment of the present application.
  • FIG. 4a is a schematic structural diagram of another refrigerator provided in the embodiment of the present application.
  • Fig. 4b is a schematic diagram of a compartment structure of another refrigerator provided in the embodiment of the present application.
  • Fig. 4c is a schematic diagram of a compartment structure of another refrigerator provided in the embodiment of the present application.
  • Fig. 5 is a schematic diagram of the first distribution method provided in the embodiment of the present application.
  • Fig. 6 is a schematic diagram of the second distribution method provided in the embodiment of the present application.
  • Fig. 7a is a schematic diagram of the third distribution method provided in the embodiment of the present application.
  • Fig. 7b is a schematic diagram of the storage space on the first floor in the third distribution mode provided in the embodiment of the present application.
  • Fig. 8a is a schematic diagram of FOV in the first distribution mode provided in the embodiment of the present application.
  • Fig. 8b is a schematic diagram of FOV in the second distribution mode provided in the embodiment of the present application.
  • Fig. 8c is a schematic diagram of FOV in the first distribution mode provided in the embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of a controller in a refrigerator provided in an embodiment of the present application.
  • Fig. 10 is a schematic structural diagram of another controller in a refrigerator provided in the embodiment of the present application.
  • Fig. 11 is a schematic flow chart of a food management method provided in the embodiment of the present application.
  • Fig. 12 is a schematic diagram of an array image provided in the embodiment of the present application.
  • Fig. 13 is a schematic flow chart of another food material management method provided in the embodiment of the present application.
  • Fig. 14 is a schematic structural diagram of a refrigerator provided in an embodiment of the present application.
  • Fig. 15 is a schematic structural diagram of another refrigerator provided in the embodiment of the present application.
  • Fig. 16 is a schematic structural diagram of another refrigerator provided in the embodiment of the present application.
  • FIG. 17 is a schematic diagram of the specific structure of the code scanner and distance measuring sensor provided in the embodiment of the present application.
  • FIG. 18 is a schematic diagram of a hardware structure of a refrigerator provided in an embodiment of the present application.
  • Fig. 19 is a flow chart of a specific embodiment provided in the embodiment of this application.
  • Fig. 20A is a perspective view of another refrigerator provided by the embodiment of the present application.
  • Fig. 20B is a top view of a storage rack provided in an embodiment of the present application.
  • Fig. 20C is a top view of a separator provided in the embodiment of the present application.
  • Fig. 21 is a schematic diagram of the appearance of a refrigerator provided in the embodiment of the present application.
  • Fig. 22 is a schematic flow diagram of a gravity sensor transmission mass provided by the embodiment of the present application.
  • Fig. 23 is a comparison diagram of quality curves before and after filtering provided by the embodiment of the present application.
  • Fig. 24 is a schematic flow diagram of a controller obtaining food information according to an embodiment of the present application.
  • Fig. 25 is a schematic diagram of an ingredient management interface provided by the embodiment of the present application.
  • Fig. 26 is a schematic diagram of a process of updating the food material management interface provided by the embodiment of the present application.
  • Fig. 27 is a schematic flow chart of another food material management method provided by the embodiment of the present application.
  • FIG. 28 is a schematic structural diagram of a device for managing refrigerator ingredients provided by an embodiment of the present application.
  • first and second are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features. Thus, a feature defined as “first” and “second” may explicitly or implicitly include one or more of these features. In the description of the present application, unless otherwise specified, "plurality” means two or more.
  • connection should be understood in a broad sense, for example, it can be a fixed connection or a detachable connection. Connected, or integrally connected; it can be mechanically connected or electrically connected; it can be directly connected or indirectly connected through an intermediary, and it can be the internal communication of two components. Those of ordinary skill in the art can understand the specific meanings of the above terms in this application in specific situations.
  • refrigerators have become one of the requisite household appliances in people's lives. People's requirements for refrigerators are no longer simply storing ingredients, and the demand for intelligent refrigerators is becoming increasingly urgent.
  • the core of intelligent refrigerators is the management of ingredients, which can include three core elements: the type of ingredients, the storage location of ingredients, and the user's access operations.
  • a common method is to install a depth camera on the top of the refrigerator to identify whether there is an access operation for ingredients.
  • FIG. 1 it is an application scene diagram of a refrigerator provided by an embodiment of the present application.
  • the refrigerator 100 may provide functions including but not limited to ingredients management.
  • the specific type, size and power of the refrigerator are not limited.
  • the refrigerator 100 also performs data communication with the server 200 through various communication methods.
  • the refrigerator 100 may be allowed to communicate via a local area network (Local Area Network, LAN), a wireless local area network (Wireless Local Area Network, WLAN) and other networks.
  • the server 200 may provide various contents and interactions to the refrigerator 100 .
  • the refrigerator 100 can send and receive information, for example: receiving images, accessing a digital media library stored remotely, sending image data to be detected, and so on.
  • the servers 200 may be in one group or in multiple groups, and may be one or more types of servers.
  • the server 200 can be deployed locally or in the cloud, and functions such as image recognition can be realized through the server 200 .
  • Fig. 2a and Fig. 2b exemplarily show the structure of the first type of refrigerator provided by the embodiment of the present application.
  • the refrigerator 100 includes a casing 110, a refrigeration unit (not shown in the figure) and other accessories (for example, a lighting lamp, a thermometer, etc. can be arranged in the box, not shown in the figure).
  • the main components of the refrigeration system are compressors, condensers, evaporators and capillary restrictors, forming a closed cycle system.
  • the evaporator can be installed above the interior of the refrigerator, and other components can be installed on the back of the refrigerator.
  • the cabinet 110 is installed with a refrigerator door 120 .
  • a display screen (not shown in the figure) may be further provided on the refrigerator door 120, and the display screen is coupled to the controller (for example, connected through a circuit).
  • the cabinet 110 may include a storage compartment.
  • the appearance of the refrigerator 100 may be determined by defining a storage space of a storage room and a location of the refrigerator door 120 .
  • the storage compartment is a box with an opening, which is formed by a storage compartment liner, a storage compartment outer shell, and a foam layer between the two.
  • the refrigerator door 120 is used to block the opening of the storage room.
  • the storage room is vertically divided into a freezer room located below and a refrigerator room located above. Wherein, the freezing room and the refrigerating room can respectively have independent storage spaces.
  • the refrigerator door 120 may include a freezer door and a refrigerator door.
  • the freezing chamber is defined at the lower side of the storage chamber and the opening of the freezing chamber can be selectively covered by the freezing chamber door.
  • the refrigerating compartment is defined at an upper side of the storage compartment, ie, the refrigerating compartment is disposed above the freezing compartment.
  • the opening of the refrigerator compartment is selectively covered by the door body of the refrigerator compartment.
  • the door body of the refrigerating room is pivotably installed on the refrigerating room, so that the opening of the refrigerating room can be selectively opened or closed through the door body of the refrigerating room.
  • the storage room of the refrigerator in the implementation manner of the present application may include a storage drawer and a first shelf and a second shelf located on the storage drawer.
  • the first shelf and the second shelf of the storage drawer can be respectively used to store food materials (for example, fruits, vegetables, etc.).
  • An image acquisition device 130 may also be provided on the casing 110 , and the image acquisition device 130 is used to acquire images of the active area of the refrigerator door 120 .
  • the front side area of the casing 110 at least includes an area extending a certain distance outside the refrigerator based on the first plane, and the image acquisition device 130 can acquire images of this area, that is, The image of the user's hand movements during the process of accessing the ingredients after opening the refrigerator door 120 and the images of the ingredients being accessed are captured.
  • the image acquisition device 130 may adopt, but is not limited to, a depth camera.
  • the cabinet 110 of the refrigerator may include a plurality of compartments (such as compartments 50a to 50d in the figure), and the plurality of compartments divide the storage room into multi-layer storage spaces to facilitate users to store different storage spaces. Foodstuffs are sorted and stored.
  • the image capture device 130 may be disposed on the top of the casing 110 .
  • the image capturing device 130 may be disposed on the upper part of the cabinet 110 close to the refrigerator door 120 so as to be able to capture images in the active area of the refrigerator door 120 .
  • the storage room includes multiple storage spaces, and the auxiliary sensor 140 is disposed on a side of the top storage space close to the refrigerator door 120 .
  • the refrigerator 110 further includes an auxiliary sensor 140 .
  • the auxiliary sensor 140 is disposed in the storage room, and is used for collecting measurement information of a storage area in the storage room.
  • the auxiliary sensor 140 may be, but not limited to, an infrared through-beam photoelectric sensor, a single-point TOF sensor, an infrared temperature sensor, an ultrasonic radar, a millimeter wave radar, and the like.
  • the sensor 140 as an example of an infrared through-beam photoelectric sensor, a single-point TOF sensor, and an infrared temperature sensor.
  • the sensor 140 may be a single-point TOF sensor.
  • the single-point TOF sensor is installed on the side of the refrigerator door 120 in the storage space. If there is no food access operation, the value of the distance L1 measured by the single-point TOF sensor is the width L2 of the refrigerator. If there is a food access operation, the distance L2 measured by the single-point TOF sensor is smaller than the width L2 of the refrigerator due to the occlusion of the hand or the food. Based on the above principle, according to the distance measured by the single-point TOF sensor, it can be determined whether there is an access operation for food. Specifically, if the distance L2 measured by the single-point TOF sensor is less than the width L2 of the refrigerator, it is determined that there is food Access operations.
  • the senor can be an infrared through-beam photoelectric sensor.
  • the infrared through-beam photoelectric sensor includes a transmitting part and a receiving part.
  • the receiving part and the transmitting part are arranged at different ends.
  • the transmitting part emits an infrared emission beam. fetch operation.
  • the infrared emission beam emitted by the emitting part is a very thin laser beam, then in the actual application process, the accuracy of the installation position of the infrared through-radiation photoelectric sensor is required to be high. Therefore, in order to reduce the sensor installation requirements, refer to Figure 3c.
  • the infrared emission beam can have a certain FOV.
  • the range outside the FOV of the auxiliary sensor may be referred to as a blind area of the auxiliary sensor.
  • the auxiliary sensor can also use an infrared temperature sensor.
  • the infrared temperature sensor can detect temperature changes, and then determine whether there is a food access operation according to the temperature changes. For example, when the infrared temperature sensor located in the top-floor storage space detects a sudden temperature rise, it is determined that there is an access operation for ingredients in the top-floor storage space.
  • Fig. 4a and Fig. 4b exemplarily show the structure of the second refrigerator provided by the embodiment of the present application.
  • the refrigerator 200 includes a casing 210, a refrigeration unit (not shown in the figure) and other accessories (such as lighting lamps, thermometers, etc., which are not shown in the figure, can be arranged in the box).
  • the cabinet 210 is installed with a refrigerator door 220 .
  • the casing 210 may include a storage room, which is a box with an opening.
  • the refrigerator door 220 is used to block the opening of the storage room.
  • the structure of the refrigerator 200 please refer to the refrigerator 100 for details, which will not be repeated here.
  • the cabinet 210 of the refrigerator 200 may include multiple compartments, and the multiple compartments divide the storage room into multi-layer storage spaces to facilitate users to classify and store different ingredients.
  • the refrigerator 200 includes a plurality of auxiliary sensors 230 .
  • one auxiliary sensor 230 is disposed in each storage space.
  • the auxiliary sensor 230 may be, but not limited to, any one of a single-point TOF sensor, an infrared temperature sensor, and an infrared through-beam photoelectric sensor.
  • the transmitting part and the receiving part of the infrared through-ray photoelectric sensor are not at the same end, as shown in FIG.
  • the component and the receiving component are respectively disposed on two sides close to the refrigerator door 220 in the storage space.
  • each storage space of the refrigerator 200 is provided with multiple Infrared through-radiation photoelectric sensors, in this way, are densely distributed (that is, multiple infrared through-radiation photoelectric sensors are arranged in each storage space) to reduce or even eliminate blind spots.
  • the first distribution method on one side of the storage room, the transmitting part of the infrared through-beam photoelectric sensor is arranged in each storage space, and correspondingly, the corresponding receiving part is arranged on the other side of the storage room.
  • the transmitting part 1 and the transmitting part 2 are arranged on the left side of the storage room, and the receiving part 1 and the receiving part 2 are arranged on the right side of the storage room, wherein the receiving part 1 and the transmitting part Part 1 belongs to the infrared through-beam photoelectric sensor 1 , and the receiving part 2 and the transmitting part 2 belong to the infrared through-beam photoelectric sensor 3 .
  • the transmitting part 3 and the transmitting part 4 are arranged on the left side of the storage room, and the receiving part 3 and the receiving part 4 are arranged on the right side of the storage room, wherein the receiving part 3 and the emitting part 3 belong to infrared radiation
  • the photoelectric sensor 3 , the receiving part 4 and the emitting part 4 belong to the infrared through-beam photoelectric sensor 4 .
  • the dense distribution of multiple infrared through-radiation photoelectric sensors may cause interference between receiving components, thereby resulting in inability to accurately detect access actions.
  • the receiving part 2 When the occluder (foodstuff or hand) is at the position as shown in Figure 5, for the receiving part 2, although the object blocks the infrared laser beam emitted by the emitting part 2, the receiving part 2 cannot receive the infrared laser emitted by the emitting part 2 beam, but the receiving part 2 can receive the infrared laser beam emitted by the emitting part 3.
  • the second distribution method on one side of the storage room, a plurality of transmitting parts and receiving parts of auxiliary sensors are arranged alternately in each storage space, and on the other side of the storage room, corresponding receiving parts and emitting parts are arranged.
  • the transmitting part 1 and the receiving part 2 are arranged in sequence on the left side of the storage room, and the receiving part 1 and the transmitting part 2 are arranged on the right side of the storage room, wherein the receiving part 1 And the emitting part 1 belongs to the infrared through-beam photoelectric sensor 1, and the receiving part 2 and the emitting part 2 belong to the infrared through-beam photoelectric sensor 3.
  • the transmitting part 3 and the receiving part 4 are arranged on the left side of the storage room, and the receiving part 3 and the transmitting part 4 are correspondingly arranged on the right side of the storage room, wherein the receiving part 3 and the emitting part 3 belong to the infrared pair
  • the photoelectric sensor 3 , the receiving part 4 and the emitting part 4 belong to the infrared photoelectric sensor 4 .
  • adopting the second method can eliminate the interference among multiple infrared photoelectric sensors to a certain extent, and compared with the first method, without reducing the FOV of the emitting part, the densely distributed emitters The interval can be reduced by 1/2.
  • the third distribution method on one side of the storage room, multiple transmitting parts of auxiliary sensors are provided in the storage space of the first floor, and the receiving parts of auxiliary sensors are provided in the storage space of the adjacent layer; on the other side of the storage room, There are corresponding receiving parts and transmitting parts.
  • the transmitting part 1 and the transmitting part 2 are sequentially arranged on the left side of the storage room, and the receiving part 1 and the receiving part 2 are correspondingly arranged on the right side of the storage room, wherein the receiving part 1 And the emitting part 1 belongs to the infrared through-beam photoelectric sensor 1, and the receiving part 2 and the emitting part 2 belong to the infrared through-beam photoelectric sensor 3.
  • a receiving part 3 and a receiving part 4 are provided on the left side of the storage room, and a transmitting part 3 and a transmitting part 4 are correspondingly provided on the right side of the storage room, wherein the receiving part 3 and the transmitting part 3 belong to the infrared pair
  • the photoelectric sensor 3 , the receiving part 4 and the emitting part 4 belong to the infrared photoelectric sensor 4 .
  • the transmitting part 5 and the transmitting part 6 are sequentially arranged on the left side of the storage room, and the receiving part 5 and the receiving part 6 are correspondingly arranged on the right side of the storage room, wherein the receiving part 5 and the transmitting part 5 belong to
  • the infrared through-beam photoelectric sensor 5 , the receiving part 6 and the emitting part 6 belong to the infrared through-beam photoelectric sensor 6 . In this way, the interference between the infrared through-ray photoelectric sensors in the storage spaces of different layers is reduced.
  • the receiving part 1 can receive the infrared laser beam emitted by the emitting part 1 at the same time and the infrared laser beam emitted by the emitting part 2, the receiving part 1 can also receive the infrared laser beam emitted by the emitting part 1 and the infrared laser beam emitted by the emitting part 2 at the same time.
  • the receiving part 1 cannot receive the infrared laser beam emitted by the emitting part 2, but can receive the infrared laser beam emitted by the emitting part 2, and the receiving part 2 cannot receive the infrared laser beam emitted by the emitting part 2.
  • the infrared laser beam emitted by the emitting part 1 can receive the infrared laser beam emitted by the emitting part 1 .
  • the preset light intensity threshold is the light intensity received by the receiving component when there is no occlusion.
  • the target FOV without interference between multiple infrared through-beam photoelectric sensors will be described in sequence when the above three distribution methods are adopted, which may also be called the maximum FOV.
  • the second distribution method and the third distribution method both reduce the requirements for the FOV of the infrared photoelectric sensor, reduce the difficulty of equipment installation, and reduce the number of infrared photoelectric sensors. Interference between sensors.
  • the third distribution method greatly reduces the requirements for the FOV of the infrared photoelectric sensor, and at the same time, reduces the mutual interference between adjacent or interval refrigerator compartments, greatly The detection stability is improved, and the refrigerator management process is simplified, thereby facilitating the user to manage the refrigerator.
  • Table 1 The requirements of the three distribution methods for the FOV of the infrared photoelectric sensor
  • a radio frequency identification (Radio Frequency Identification, RFID) reader is provided in the refrigerator, which can be used to read the information on the food packaging.
  • RFID tags to obtain information such as the type and quantity of ingredients.
  • the refrigerator also has a voice function, which can recognize the input voice, so as to obtain information such as the type and quantity of the ingredients input by the user through voice.
  • FIG. 9 exemplarily shows a schematic structural diagram of a controller in a refrigerator.
  • the controller 900 is connected with an image acquisition device and an auxiliary sensor.
  • the controller 900 may include: an acquisition unit 901 and a processing unit 902 .
  • the acquisition unit 901 is configured to, in response to the opening of the refrigerator door, acquire the image of the active area of the refrigerator door collected by the image acquisition device, and obtain the measurement information of the storage area in the storage room collected by the auxiliary sensor, wherein the image acquisition device is set to On the top of the casing, the auxiliary sensor is set in the storage room;
  • the processing unit 902 is configured to, if it is determined that there is no food item access operation in the image, based on the measurement information, determine whether there is a food item access operation.
  • processing unit 902 is specifically configured to:
  • the auxiliary sensor is a single-point time-of-flight TOF sensor, and the distance between the auxiliary sensor and the occluder contained in the measurement information is less than or equal to the width of the storage room, then it is determined that there is an access operation for food; or, if The auxiliary sensor is an infrared through-beam photoelectric sensor, and the light intensity contained in the measurement information is less than the preset light intensity threshold value, then it is determined that there is a food access operation; or, if the auxiliary sensor is an infrared temperature sensor , then according to the temperature information contained in the measurement information, it is determined that there is an access operation of the food material.
  • Fig. 10 exemplarily shows a structural diagram of another controller in a refrigerator, as shown in the figure, the controller 1000 is connected with an auxiliary sensor.
  • the controller 1000 may include: an acquiring unit 1001 and a judging unit 1002 .
  • the acquiring unit 1001 is configured to acquire the measurement information collected by each auxiliary sensor set in the multi-storey storage space of the storage room in response to the opening of the refrigerator door, wherein at least one auxiliary sensor is set in each storey space;
  • the judging unit 1002 is configured to judge whether there is an access operation of the ingredients based on the measurement information.
  • the judging unit 1002 is specifically used for:
  • the auxiliary sensor is a TOF sensor, then based on the distance between the auxiliary sensor and the occluder contained in the measurement information, it is judged whether there is a food access operation; or, if the auxiliary sensor is an infrared through-beam photoelectric sensor, based on the light intensity contained in the measurement information, it is judged whether there is a material access operation; or, if the auxiliary sensor is an infrared temperature sensor, it is judged whether there is a food material according to the temperature information contained in the measurement information Access operations.
  • the refrigerator when the image acquisition device detects the hand image in front of the refrigerator, the refrigerator sends a shooting instruction to the image acquisition device to receive the image sent by the image acquisition device.
  • the image to be sent may be a depth image or a color image, and its image format may be: JPEG, TIFF, RAW, BMP, GIF, PNG, etc.
  • the measurement information may include the distance between the auxiliary sensor and the occluder; if the auxiliary sensor uses an infrared through-beam photoelectric sensor, the measurement information may include light intensity; if the auxiliary sensor uses If an infrared temperature sensor is used, the measurement information may include temperature information.
  • neural network algorithms can also be used, wherein the neural network can be but not limited to (deep) neural network, convolutional neural network, deep belief network, deep stack neural network and other network structures species or its derived models, which are not limited and will not be repeated here.
  • Case 1 If the auxiliary sensor is a TOF sensor, and the distance between the auxiliary sensor and the occluder contained in the measurement information is less than or equal to the width of the storage room, it is determined that there is an operation of food material access.
  • the occluder may refer to the food material, or may refer to the user's hand.
  • the distance between the auxiliary sensor included in the measurement information and the obstruction is 60 cm, at this time, the distance between the auxiliary sensor included in the measurement information and the obstruction is smaller than the storage room width , to determine the existence of ingredients access operations.
  • the food storage location can also be determined according to the distance between the auxiliary sensor contained in the measurement information and the obstruction.
  • auxiliary sensor is an infrared through-beam photoelectric sensor, and the light intensity contained in the measurement information is less than the preset light intensity threshold, it is determined that there is an access operation for food.
  • the preset light intensity threshold value may be the light intensity received by the receiving part of the infrared through-beam photoelectric sensor when there is no shield.
  • the preset light intensity threshold value is 10 candela (candela, cd)
  • the light intensity included in the measurement information is 5cd
  • the light intensity included in the measurement information is less than the preset light intensity threshold value , to determine the existence of ingredients access operations.
  • auxiliary sensor is an infrared temperature sensor, and according to the temperature information contained in the measurement information, it is determined that there is an operation of food material access.
  • the temperature information includes the pixel value sets corresponding to each collection time. If the number of target pixel values in a pixel value set exceeds the set number threshold, it is determined that there is a food material access operation, wherein the target pixel value is used for A pixel value that represents the difference between the temperature value set and the set pixel value and reaches the preset difference threshold.
  • the infrared temperature sensor is an infrared temperature sensor with 8 ⁇ 8 pixels, that is to say, in the temperature information, a temperature value set contains 8 ⁇ 8 pixel values, and the pixel value is also Yes, 8 ⁇ 8 temperature values look like this:
  • the processed array image is obtained, wherein (a) is the array image without occlusion, and (b) is the array image with Array image when occluded.
  • FIG 13 it is another controller provided in the embodiment of the present application, which is applied to the refrigerators shown in Figure 4a, Figure 4b, and Figure 4c above, and can be configured to perform the following program steps:
  • the measurement information can include the distance between the auxiliary sensor and the occluder; if the auxiliary sensor uses an infrared photoelectric sensor, the measurement information can include light intensity; if the auxiliary sensor uses infrared temperature sensor, temperature information can be included in the measurement information.
  • auxiliary sensors such as a code scanner and a distance measuring sensor can also be provided, and the controller can determine the time at which the code scanner scans the identification code on the food material.
  • the first time is to determine the second time when the distance measuring sensor measures that the user passes the measurement area of the distance measuring sensor, and according to the first time and the second time, the user's access action is determined; the access action includes: storing the ingredients into and out of the refrigerator.
  • the auxiliary sensor may also include at least one gravity sensor disposed in a partition in the refrigerator storage room, at least one gravity sensor in a shelf on the refrigerator door, and an RFID antenna positioned on the refrigerator door , the controller can obtain the mass transmitted by the gravity sensor in the partition and the shelf; when it is determined that the mass transmitted by the first gravity sensor increases, according to the RFID tag of the first food material detected by the RFID antenna on the refrigerator door, determine the first The food is stored at the position of the first gravity sensor; or, when it is determined that the mass transmitted by the first gravity sensor is reduced, according to the RFID tag of the second food detected by the RFID antenna on the refrigerator door, it is determined that the second food is removed from the The position of the first gravity sensor is taken out; wherein, the first gravity sensor is any gravity sensor in at least one partition, or is any gravity sensor in at least one shelf.
  • the refrigerator may include: a code scanner 10 , a distance measuring sensor 20 and a controller 30 .
  • the code scanner 10 and the ranging sensor 20 may be collectively referred to as an auxiliary sensor.
  • the controller 30 is configured to: determine the first time when the code scanner 10 scans the identification code on the food material; determine the second time when the distance measuring sensor 20 detects that the user passes through the measurement area of the distance measuring sensor; according to the first time and At the second time, the user's access action is determined; the access action includes: storing the ingredients into the refrigerator and taking the ingredients out of the refrigerator.
  • the controller can accurately determine the user's access action according to the first time and the second time, that is: determine whether the user stores the food in the refrigerator or the user takes the food out of the refrigerator; thus, It eliminates the influence of information such as the surface smoothness and color of the ingredients, effectively improves the recognition accuracy and recognition efficiency of access actions, and provides an effective data reference for the management of ingredients in the refrigerator.
  • the following methods may be included for the setting methods of the code scanner and the distance measuring sensor:
  • the refrigerator includes: a refrigerator door 40 and a box body 50;
  • Both the code scanner 10 and the distance measuring sensor 20 are located on the same layer (not shown) in the compartment included in the refrigerator door, and/or the code scanner 10 and the distance measuring sensor 20 are both located on the compartment included in the box body 50
  • the same layer in a as shown in Figure 15, and the code scanner 10 and distance measuring sensor 20 are not limited to be located on the left side of the compartment a, but can also be located on the right side of the compartment a);
  • the distance measuring sensor 20 is located on the side of the code scanner 10 close to the inside of the box body 50 (as shown in the partially enlarged schematic diagram in FIG. 15 , from which the code scanner 10 blocks the distance measuring sensor 20 , to indicate that the distance measuring sensor 20 is closer to the inside of the box body 50).
  • the thinking asset management can only access the ingredients, which provides convenience for users, and can also realize the identification of access actions.
  • the distance between the code scanner and the ranging sensor is smaller than a preset value.
  • the preset value can be set according to actual needs, which is not limited here.
  • the code scanner and the distance measuring sensor are set close to each other, so that when the user takes out ingredients from the refrigerator and wants to scan the code on the code scanner, he must pass through the measurement area of the distance measuring sensor, or the user wants to store After the ingredients are scanned on the scanner with the ingredients, they will inevitably pass through the measurement area of the distance sensor.
  • the process of scanning the code and taking out the ingredients is required. Assume that the moment of scanning the code (that is, the moment when the scanner scans the identification code) is t. Before that (such as t- ⁇ t) If it is detected that the food or the user's hand has passed the measurement area of the ranging sensor, according to this logic, it can be determined that the food is taken out;
  • ingredients When the user deposits ingredients, assuming that the moment of scanning the code is t, no ingredients are detected before (such as t- ⁇ t) or the user’s hand has passed the measurement area of the distance sensor, and after that (such as t+ ⁇ t) t) If it is detected that the ingredients or the user's hand has passed the measurement area of the ranging sensor, it can be determined that the ingredients are stored;
  • the time interval (that is, ⁇ t) between the time of scanning the code and the time when the food material or the user's hand passes through the measurement area of the ranging sensor is very short, that is, ⁇ t can be relatively small.
  • the scanner and the measuring The distance between the sensors is d, assuming that the time for the scanner to obtain the identification code and the response time of the ranging sensor are ignored, then:
  • ⁇ t may be approximately 50 ms.
  • the senseless recognition of the access action can be realized, that is, when the user accesses the ingredients, he only needs to complete the code scanning operation at the corresponding location, and the refrigerator can complete the management of the access action.
  • the controller is configured to:
  • the time difference between the first time and the second time is less than a preset time.
  • the preset time mentioned here can be understood as ⁇ t mentioned above.
  • the distance measuring sensor since the distance measuring sensor is located on the side of the code scanner close to the inside of the box, if the code scanner scans the identification code first, and then the distance measuring sensor detects that the user passes the measurement area, then it can be determined that the user is placing the ingredients Stored in the refrigerator; if the ranging sensor first detects that the user passes the measurement area, and then the code scanner scans the identification code first, then it can be determined that the user took the food out of the refrigerator.
  • the user's access action can be determined according to the first time and the second time, providing effective data reference for the management of ingredients.
  • the controller can also be configured to:
  • the user's access action can be determined based on the installation positions of the code scanner and the distance measuring sensor, combined with the order of the first time and the second time.
  • the refrigerator includes: a refrigerator door and a cabinet, and the refrigerator door and/or cabinet includes a plurality of compartments;
  • a distance measuring sensor is arranged in each compartment, which is not shown in the figure.
  • Each compartment is provided with a code scanner and a distance measuring sensor.
  • any one of the above-mentioned cases 1, 2 and 3 can be used, and can be selected according to actual needs. limited.
  • the code scanner 10 and the distance measuring sensor 20 are located in different compartments, as shown in Figure 16, the code scanner 10 is located in the compartment, and the distance measuring sensor 20 is located in the compartment b .
  • the installation positions of the code scanner 10 and the distance measuring sensor 20 are not limited to those shown in FIG. 16 , and only the one shown in FIG. 16 is taken as an example for illustration.
  • the positions of the code scanner and the distance measuring sensor can be set according to actual needs, so as to improve the flexibility of design and meet the needs of different application scenarios.
  • the user's access action can be determined; for specific implementation, please refer to the above-mentioned The embodiment in mode 1 will not be described in detail here.
  • this method 2 when setting the number of code scanners and distance measuring sensors, you can set them arbitrarily as needed, as long as you ensure that the code scanners and distance measuring sensors are in different compartments.
  • the refrigerator mentioned in the embodiment of this application may include other functions besides determining the user’s access actions, as follows:
  • the refrigerator includes a plurality of compartments, at least some of which are provided with code scanners 10, as shown in FIG. 15 ;
  • the controller is configured to: according to the identification of the code scanner that scans the identification code, determine that the scanned identification code corresponds to the corresponding compartment where the ingredients are stored. For example, taking the example shown in Figure 15, when each compartment is provided with a code scanner 10, assuming that the user wants to store an apple in the compartment marked a, then: the user can hold the apple to the compartment Scan the code scanner (denoted as code scanner a) in layer a, so that the code scanner a can scan the identification code on the apple; when the code scanner a scans the identification code, it can send a The message of the identification code and self-identification; when the controller receives the message, it can determine that the code scanner a has scanned the identification code, and then determine that the compartment for storing apples corresponding to the scanned identification code is compartment a.
  • the storage location of the ingredients can be realized by distinguishing the identification of the code scanner.
  • Each compartment that the user needs to manage is equipped with a code scanner.
  • the scanner scans the code, and puts the scanned ingredients into the compartment corresponding to the scanner; the controller can distinguish the access location of the ingredients by analyzing the identification of the scanner and the corresponding table of the compartment .
  • the storage location of the ingredients can be determined only based on the identification of the barcode scanner, thereby providing an effective data reference for the management of the ingredients.
  • the controller is configured to:
  • the identification code scanned by the scanner and the corresponding relationship between the preset identification code and the type of food material (as shown in Table 2 below, the identification code in Table 2 is shown as an example, and only shows The corresponding relationship between some barcodes and ingredients), and determine the type of ingredients corresponding to the identification code.
  • the types of ingredients can be determined quickly, accurately, and effectively, avoiding misjudgment, and while improving the recognition accuracy, it can also help improve the management efficiency of the refrigerator.
  • the identification code may include: a barcode and/or a two-dimensional code.
  • the identification code may also include other forms that can be used to distinguish types, which is not limited here.
  • the type of ingredients is determined by the identification code pasted on the ingredients, taking the identification code as a barcode as an example:
  • the identification code can be a standard barcode or a custom barcode
  • the controller can determine the type of ingredients through the barcode stored locally and the comparison table of the types of ingredients;
  • the controller can obtain the types of ingredients by accessing the cloud database interface.
  • the ingredients management function may include: remaining shelf life query, quantity query, location query, nutrition encyclopedia query, and meal recommendation, etc.;
  • the principle of querying the remaining shelf life can be as follows: take the time when the ingredients are stored in the refrigerator for the first time as the starting point, and the shelf life minus the storage time of the ingredients in the refrigerator is the remaining shelf life; specifically, the controller can record the time when the ingredients are first stored , by querying the local database or the cloud database, the shelf life of the ingredients can be determined, and then the remaining shelf life can be determined;
  • the principle of quantity query can be as follows: according to the determined user's access action, record the ingredients stored in the refrigerator and the ingredients taken out each time, based on this, the quantity and type of the remaining ingredients in the refrigerator can be determined within a period of time ;
  • the principle of location query can be as follows: based on the determination method of the above-mentioned types of ingredients and storage locations, the types and storage locations of ingredients stored in the refrigerator can be determined each time, and query data can be provided for users when they need to query later;
  • Nutrition Encyclopedia query can be pre-stored in the local database by the controller or obtained from the cloud database;
  • the principle of meal recommendation can be as follows: For example, during a specific period of time, monitor the ingredients taken out by the user from the refrigerator, and analyze whether the user has scientifically taken in the nutrients needed by the body according to the ingredients taken out.
  • it may further include: a display and/or a player; the display is configured to: display the information determined by the controller; the player is configured to: play the information determined by the controller .
  • the determined information can be displayed and played through the display and the player, so as to be displayed to the user.
  • it may further include: a voice collector; the voice collector is configured to: collect voice information input by the user.
  • the user can implement functions such as shelf life query, quantity query, location query, and category query through display input or voice input; among them, for the meal recommendation function, it can be recommended through the pop-up window of the display.
  • Recommendations can be made through different methods; shelf life query, quantity query, location query, nutrition encyclopedia query, etc. can be queried by voice input or by display input.
  • the refrigerator can also realize other functions realized by the code scanner, distance measuring sensor and controller in addition to all the above functions, which will not be detailed here. stated.
  • the ranging sensor may include a single-point time-of-flight sensor.
  • the code scanner 10 may include three parts: a supplementary light 11 , a positioning light bar 12 , and a photosensitive element 13; wherein the supplementary light 11 is used to: increase the brightness of the photosensitive element 13 when scanning codes
  • the positioning light bar 12 is used to: facilitate the user to align the barcode scanning; the photosensitive element 13 is used to shoot: take the barcode;
  • the single-point time-of-flight sensor 20 can include: the transmitting end 21 and the receiving end 22; the transmitting end can emit such as but not limited to Infrared rays are reflected when the infrared rays are incident on the user's hand or the surface of the food, and the reflected infrared rays can enter the receiving end.
  • the receiving end When the receiving end receives the reflected infrared rays, it can be determined that the user has passed the measurement area; or, When the reflected infrared rays are received at the receiving end, it can also be understood as: the user approaches the distance measuring sensor, or the user blocks the distance measuring sensor.
  • the refrigerator provided by the embodiment of the present application may include three parts: a perception layer, a decision-making layer, and an interaction layer; where the perception layer is used for: responsible for data Perceptual acquisition, including: the acquisition of identification codes, the acquisition of scanner logos, and the identification of distance measuring sensors for users passing through the measurement area; /Transmitter, Universal Asynchronous Receiver Transmitter) interface to transmit information, between the ranging sensor and the controller can but is not limited to transmit information through the I2C interface; the decision-making layer is used to: analyze the acquired and identified information to determine the output The type of ingredients, the storage location of ingredients, and the user's access actions; the interaction layer is used to present the content perceived by the perception layer and the data analyzed by the decision-making layer to the user through the display and player in an intuitive way, which is convenient The user's use and query; and, the controller and the display can transmit information through the UART interface, but the controller and the player can transmit information through the
  • the user's actions are as follows: scan the code on the code scanner in the first compartment; put Tsingtao beer in the first compartment.
  • the code scanner 1 when the user scans the code on the code scanner (referred to as the code scanner 1) in the first compartment, the code scanner 1 can obtain the identification code of Tsingtao Beer, and The obtained identification code and the identification of the code scanner 1 are transmitted to the decision-making layer; the controller in the decision-making layer can determine that the type of food material is Tsingtao beer by analyzing the identification code, and then determine that the code scanner 1 is located at the first One compartment; the user scans the code first, and then deposits, and the ranging sensor is located on the side of the scanner close to the inside of the box.
  • the decision-making layer can judge The user's action just now is to deposit Tsingtao beer into the first compartment; the interactive layer can pop up a picture of Tsingtao beer stored in the first compartment on the display, and/or voice broadcast "Deposit in the first compartment" Tsingtao Brewery” intuitively presents information on accessing ingredients to users.
  • a refrigerator generally includes: a refrigerator and a freezer.
  • the refrigerator can be used to keep food fresh and store, so the freshness of the food is required to be high, and according to the characteristics of the food itself, even if it is stored in the refrigerator, the food will rot.
  • time i.e., green vegetables are stored for a short time and are prone to rot, while fruits such as apples are stored for a relatively long time
  • the cold room is very necessary for the management of ingredients; it should be pointed out that in cold storage
  • drawers may also be included in the compartment, so a code scanner and a distance sensor can also be set in each drawer.
  • the freezer it is used to freeze and store food materials, even if they are stored for a long time, there will be no rot, so the necessity of the freezer for food management is obviously less than that of the refrigerator.
  • the determination of the access action mentioned in the embodiment of the present application can only be applied to the cold room (as shown in Figure 15). , or suitable for refrigerators and freezers.
  • the system initialization may include: initialization of the processing program, initialization of the scanner, and initialization of the ranging sensor; wherein, the initialization of the scanner may include: scanning mode, scanning interval, positioning light bar brightness, supplementary light
  • the setting of parameters such as light brightness; the initialization of the ranging sensor may include: the setting of parameters such as working mode and transmission power.
  • S1905. Determine whether the first time sent by the scanner is received within the preset time after the second time; if yes, execute S1906; if not, continue to execute S1901;
  • the preset time in S1902 and S1905 can be set to be the same or different, and can be set according to actual needs, so as to improve the flexibility of design and meet the needs of different application scenarios.
  • the ranging sensor After the ranging sensor detects being blocked at time t, it can judge whether there is a code scanning result within [t- ⁇ t,t] time, if there is a code scanning result, it can be considered as [t- ⁇ t,t] time
  • the ingredients scanned in the code are the ingredients currently stored in the refrigerator by the user; since ⁇ t can be set small, it can filter out many interference signals well, and the time boundary of scanning the code to store the ingredients is also easy to distinguish, which can eliminate large Most of the misjudgments improve the accuracy of access action recognition; the ranging sensor detects that it is blocked, and there is no scanning result within [t- ⁇ t,t], then you can wait for ⁇ t ⁇ n (where n can be But not limited to 1), if there is a code scanning result within [t, t+ ⁇ t ⁇ n] time, it can be considered that the ingredients scanned within [t, t+ ⁇ t ⁇ n] time are taken out of the refrigerator by the user If there is no scanning result within [t, t
  • the refrigerator may also include other structures that can be used to realize the functions of the refrigerator, which will not be described in detail here.
  • an embodiment of the present application provides a refrigerator control method.
  • the control method may include: determining the first time when the identification code on the food material is obtained; The second time of the measurement area of the ranging sensor; according to the first time and the second time, the user's access action is determined; the access action includes: storing the food in the refrigerator and taking the food out of the refrigerator.
  • the refrigerator includes: a refrigerator door and a cabinet; both the code scanner and the distance measuring sensor are located on the same layer in the compartment included in the refrigerator door, and/or the code scanner and the distance measuring sensor They are all located on the same layer of the compartments included in the box; the distance measuring sensor is located on the side of the code scanner close to the inside of the box; at this time, according to the first time and the second time, the user's access action is determined, specifically including : When the first time is before the second time, it is determined that the user has stored the ingredients in the refrigerator; when the first time is later than the second time, it is determined that the user has taken the ingredients out of the refrigerator; where the first time and the second time The time difference between them is less than the preset time.
  • the refrigerator when the refrigerator includes a plurality of compartments, and at least some of the compartments are provided with code scanners, it also includes: determining the scanned code according to the identifier of the code scanner that has scanned the identification code.
  • the identification code corresponds to the corresponding compartment where the ingredients are stored.
  • FIG. 20A is a perspective view of a refrigerator provided in the embodiment of the present application.
  • the refrigerator provided in the embodiment of the present application is approximately rectangular in shape and consists of a storage room 301 providing storage space and at least one refrigerator door 320 arranged at the opening of the storage room 301 .
  • the storage room 301 is a box with an opening.
  • the refrigerator compartment 301B also includes a first partition 311, a second partition 312...the Nth partition 31N for storing foodstuffs. In FIG. 20A, only two partitions are shown as examples.
  • Each partition includes a shelf for storing ingredients and a support beam for supporting the shelf.
  • the refrigerating compartment 301B is selectively opened or closed by a refrigerator door 320 pivotably installed on the refrigerating compartment 301B.
  • the refrigerator door 320 also includes a first shelf 321 , a second shelf 322 . . . an Nth shelf 32N for storing food.
  • FIG. 20A only two storage racks are exemplarily shown. It should be known that the present application does not specifically limit the number of storage racks.
  • the refrigerator involved in the present application also includes a controller and a memory, which are not shown in FIG. 20A , and the present application does not specifically limit the positions of the controller and the memory.
  • Multiple gravity sensors are also deployed in the refrigerator provided in the embodiment of the present application, and the multiple gravity sensors are deployed in the above-mentioned storage rack.
  • the gravity sensor can also be deployed in the partition, for example, it can be deployed in a storage board included in the partition or in a support beam for supporting the storage board.
  • the gravity sensor can be used as an auxiliary sensor.
  • the gravity sensor is located in the storage rack and the support beam as an example. What needs to be known is that multiple gravity sensors in the refrigerator are connected to the controller, and the gravity sensors can transmit the acquired mass to the controller.
  • FIG. 20B it shows a top view of a storage rack, which includes a plurality of gravity sensors deployed in the storage rack.
  • the plurality of gravity sensors deployed in the rack may be evenly distributed, in order to make the rack evenly stressed, so that the data measured by the gravity sensors at different positions on a rack are basically consistent.
  • the gravity sensors can be deployed in other ways besides uniform distribution, such as deploying the gravity sensors according to a set sorting rule.
  • FIG. 20B is only an example, and the present application does not specifically limit the location and deployment method of the gravity sensor.
  • FIG. 20C it shows a top view of a partition provided by an embodiment of the present application, including a storage board, a support beam, and multiple gravity sensors disposed in the support beam. It should be noted that FIG. 20C is only an example, and the application does not specifically limit the deployment position and manner of the gravity sensor.
  • FIG. 21 is a schematic diagram of the appearance of a refrigerator provided in the embodiment of the present application.
  • a display screen is provided on the refrigerator door 320 of the refrigerating room, and the display screen can be embedded in the refrigerator door.
  • the plane where the screen of the display screen is located is flush with the outer surface of the box door.
  • the refrigerator further includes a controller, the controller is connected to the display screen, and is used to interact with the server to obtain information and data, generate a material management interface, and output display information to the display screen, and the display screen receives information output by the controller. Display the signal and display the corresponding image information, such as the food management interface, etc.
  • radio frequency identification (Radio Frequency Identification, RFID) tags are used to identify the ingredients in the storage room.
  • the identification information of the RFID tags can be obtained in advance, such as in the RFID tag
  • the written tag code establishes and stores the corresponding relationship between the identification information of the RFID tag and the ingredient information, and realizes the association between the RFID tag and the ingredient information.
  • the ingredient information may include information such as the type and name of the ingredient, or the pattern of the ingredient.
  • the refrigerator door 320 of the refrigerator can also be provided with an RFID antenna.
  • the RFID antenna emits electromagnetic wave signals and receives the return of the RFID tag Activate the signal and transmit the activation signal to the controller.
  • the RFID antenna can also be used as an auxiliary sensor.
  • the activation signal carries the identification information of the RFID tag.
  • the controller can determine the information of the food material corresponding to the RFID tag through the unique correspondence between the identification information of the RFID tag and the food material information established in advance, and display the food material information on the display screen. , so that the user can know the ingredients in the refrigerator through the display screen.
  • the refrigerator door 320 may also include an RFID tag box for storing unused RFID tags, and the RFID tag box may also have a built-in shielding layer for shielding the RFID antenna. That is, when the RFID tag is in the RFID tag box, the RFID antenna cannot detect the RFID tag.
  • the method adopted in the management of refrigerator ingredients is: manually enter the name and location of the ingredients each time the ingredients are stored or taken out. This method not only wastes time, but also has cumbersome operation steps and poor user experience.
  • This application proposes a refrigerator and a food management method for the refrigerator.
  • Multiple gravity sensors are deployed in the refrigerator to obtain the quality of the food.
  • the change of the condition determines whether to store or take out the ingredients.
  • the RFID antenna is deployed on the door of the refrigerator, and the information of the stored or taken-out ingredients is determined by scanning the RFID tags on the ingredients. It is no longer necessary to manually enter food information, and realize non-sensing and efficient refrigerator food management.
  • the refrigerator proposed by the present application includes a plurality of gravity sensors disposed in support beams and shelves, and an RFID antenna disposed on the door for scanning RFID tags of food items.
  • the controller of the refrigerator determines that the mass transmitted by the first gravity sensor increases, it determines the location where the first food product is stored in the first gravity sensor according to the label of the first food product detected by the RFID antenna.
  • the controller of the refrigerator determines that the mass transmitted by the first gravity sensor decreases, it determines that the second food is taken out from the position of the first gravity sensor according to the label of the second food detected by the RFID antenna.
  • the first gravity sensor is any gravity sensor in at least one partition, or is any gravity sensor in at least one shelf.
  • the controller determines that the mass transmitted by the first gravity sensor increases, it can determine that the user has stored food at the location where the first gravity sensor is located. Then the position of the stored ingredients can be further determined according to the position where the first gravity sensor is deployed. For example, if the first gravity sensor is located in the first rack shown in FIG. 20A, it can be determined that the user has stored the ingredients in the first rack middle. Still further, the controller may determine that the food item stored by the user is the first food item according to the RFID antenna detecting the RFID tag of the food item stored by the user. For example, if the first ingredient is tomatoes, the controller may determine that the user places the tomatoes in the first shelf.
  • the controller when the controller determines that the mass transmitted by the first gravity sensor decreases, it can determine that the user takes out the food from the position where the first gravity sensor is located. Then the controller can determine the position of the food taken out according to the position where the first gravity sensor is deployed, and further, according to the RFID antenna detecting the RFID tag of the food material taken out by the user, it can be determined that the food material taken out by the user from the position where the first gravity sensor is located is Second ingredient.
  • the memory in the refrigerator may store the mass transmitted by each gravity sensor deployed in the refrigerator.
  • the gravity sensor in the refrigerator can transmit the mass to the controller every set interval. For example, each gravity sensor in a refrigerator transmits mass to the controller every 50ms.
  • the controller may determine that the mass transmitted by the first gravity sensor has increased according to that the mass transmitted by the first gravity sensor received this time is greater than the mass transmitted by the first gravity sensor last time stored in the memory.
  • the controller may determine that the mass transmitted by the first gravity sensor has decreased according to that the mass transmitted by the first gravity sensor received this time is less than the mass transmitted by the first gravity sensor last time stored in the memory.
  • the gravity sensor involved in the present application may use pressure-sensitive materials whose resistance changes with changes in pressure. For example, when a voltage of 12V is applied to both ends of the gravity sensor and the pressure of an object of 10-20kg is applied, the current passing through the gravity sensor is about 10-20mA.
  • the current signal output by the gravity sensor is a small signal
  • the current signal output by the gravity sensor can be processed by the signal amplification circuit and passed through the digital-to-analog conversion circuit (Analog-to-Digital Converter, ADC) Then convert the analog signal to digital signal. Further, the controller may determine the quality according to the received current and the stored correspondence between current and quality.
  • the remote terminal unit in the modbus communication protocol can be used to transmit the mass.
  • the controller can perform data transmission according to the modbus protocol. Parsing gets the quality.
  • the above-mentioned process of the gravity sensor transmitting the mass to the controller can also refer to the flow chart shown in FIG. 22 .
  • the controller when storing or taking out ingredients in the refrigerator, there may be some uncontrollable factors that affect the quality of the gravity sensor acquisition, for example, factors such as the user pressing the partition (or shelf) with his hand when placing the ingredients. Due to the influence of these factors, the quality collected by the gravity sensor is inaccurate, and the controller also needs to wait for the quality transmitted by the gravity sensor to be stable before it can determine the access action and the position of the food according to the quality, so that the food access action can be determined And the process of food material location takes a long time and is inefficient.
  • the controller provided in the embodiment of the present application can also filter the received quality after receiving the quality transmitted by the gravity sensor, so as to reduce the noise of the gravity sensor itself, eliminate interference, and improve the real-time identification of the food storage location. sex and accuracy.
  • a Kalman filter algorithm may be used to process the received quality.
  • other filtering algorithms can also be used.
  • the filtered quality can be obtained through the filtering equation shown in the following formula (1):
  • x k is the mass after filtering
  • x k-1 is the mass after last filtering
  • z k is the mass transmitted by the gravity sensor
  • z k-1 is the mass transmitted by the gravity sensor last time.
  • the filtered mass may be used for determination. After the filtered mass is determined, the mass of the gravity sensor stored in the memory may be updated to the filtered mass of the gravity sensor. The controller may determine that the mass of the first gravity sensor has increased according to that the mass of the first gravity sensor determined this time is greater than the mass of the first gravity sensor stored in the memory.
  • the controller may determine that the mass of the first gravity sensor has decreased according to that the mass of the first gravity sensor determined this time is smaller than the mass of the first gravity sensor stored in the memory.
  • FIG. 23 shows a comparison chart of quality curves before filtering and after filtering provided by the embodiment of the present application. It can be seen that the Kalman filter algorithm effectively filters out the error amount in the process of storing and taking out ingredients, and can judge the access action more accurately and quickly.
  • the controller determines that the mass transmitted by the gravity sensor has increased or decreased, it can also determine the information of the ingredients stored (or taken out) by the user according to the RFID tags of the ingredients detected by the RFID antenna.
  • the controller determines that the information of the ingredients stored (or taken out) by the user can be determined in the following three ways:
  • the controller can control the RFID antenna on the door to emit electromagnetic wave signals when the door of the refrigerator is opened, for example, the controller can detect the opening of the door through the sensor on the door , closed state, and when the door is opened, the RFID antenna is controlled to emit electromagnetic wave signals outward, and in other cases, the RFID antenna does not emit electromagnetic wave antenna outward. Further, the controller determines that the label of the food stored (or taken out) by the user is detected according to the activation signal returned by the RFID tag of the food received through the RFID antenna, and determines that the detected Information about the ingredients stored (or taken out) by the user.
  • the RFID antenna can transmit electromagnetic wave signals outward in real time, and can only receive activation signals returned by tags within the set range of the antenna.
  • the set distance can be 2cm, that is to say, only when the distance between the RFID tag of the food material and the RFID antenna is less than 2cm, the RFID antenna can detect the RFID tag.
  • the controller may determine that the user is stored at the first gravity sensor through the label of the food material detected by the RFID antenna on the door within the first preset time period before determining that the mass transmitted by the first gravity sensor increases. Information about ingredients. Similarly, the controller can determine the information of the ingredients taken out by the user from the position of the first gravity sensor through the tags of the ingredients detected by the RFID antenna within the second preset time period after determining that the mass transmitted by the first gravity sensor has decreased.
  • the process for the controller to determine the information of storing/retrieving ingredients can also refer to the flow chart shown in FIG. 24 , which specifically includes: the controller obtains the activation signal returned by the RFID tag of the ingredients by controlling the RFID antenna.
  • the controller has the function of interpreting the activation signal received by the RFID antenna returned by the RFID tag.
  • the function of parsing the activation signal returned by the RFID tag can be implemented by an RFID read-write module (or RFID reader-writer) configured and inside the controller.
  • the RFID read-write module analyzes the activation signal to obtain the identification information of the corresponding tag.
  • the controller may determine the food material information corresponding to the identified identification information obtained through analysis according to the stored correspondence between the RFID tag and the food material information.
  • the controller may further determine the position of the food material stored or taken out according to the position of the first gravity sensor whose quality of transmission changes.
  • the controller may number each gravity sensor according to the position of each gravity sensor in the refrigerator, and receive the mass transmitted by each gravity sensor in the sequence of numbers.
  • the controller can number the gravity sensor in the first support beam as one, the gravity sensor in the second support beam number as two, the gravity sensor in the Nth support beam number as N, and the gravity sensor in the first rack number as Sensors are N+1...and so on.
  • the number of the first gravity sensor in the first support beam is one
  • the number of the second gravity sensor in the first support beam is two, and so on. This application does not specifically limit the way of numbering.
  • the transmission can be carried out in the order of numbers, for example, the gravity sensor in the first support beam transmits the first mass, and the gravity sensor in the second support beam transmits the second mass...
  • the controller determines the positions of the gravity sensors whose transmitted masses change according to the received sequence of the gravity sensors whose transmitted masses change. For example, if the controller determines that the mass transmitted by the second received gravity sensor decreases, it may be determined that the user has taken food from the position of the second support beam.
  • each gravity sensor can correspond to a transmission channel or each support beam (or shelf The gravity sensor in ) corresponds to a channel.
  • the controller can determine the position of its corresponding gravity sensor according to the quality received by different channels.
  • the first channel corresponds to the gravity sensor in the first support beam, and when the controller determines that the mass received through the first channel decreases, it can determine that the user has taken food from the first partition.
  • a mark can also be added to the data, which is used to characterize the location of the gravity sensor. It should be noted that the logos added when each gravity sensor transmits mass are different.
  • the controller can determine the position of the gravity sensor according to the identification in the received data, and then determine the position of the ingredients.
  • the refrigerator involved in the present application may also include a display screen.
  • the controller may also adjust the ingredients management interface according to these information, and The adjusted ingredients management interface is displayed on the display screen.
  • the controller may also display the adjusted ingredients management interface on the display screen in response to the user's touch or remote control operation.
  • the display interface may include a refresh control 701 for updating the interface, and the controller renders the adjusted ingredient management interface to the display screen in response to the user's operation on the control 701, see FIG. (b) in 26. It should be noted that this application does not specifically limit the position of the control 701 in the ingredient management interface, and FIG. 26 is only an example.
  • FIG. 27 shows a flow chart of the refrigerator material management method proposed by the present application, which specifically includes:
  • the controller determines that the refrigerator door is opened, the controller controls the RFID antenna to emit electromagnetic wave signals.
  • the controller judges whether the RFID antenna receives the activation signal returned by the RFID tag of the food item. If yes, continue to step 2703. If not, continue to step 2705.
  • the controller judges whether the mass transmitted by the gravity sensor increases. If it increases, the controller can determine that the user has stored ingredients in the refrigerator, and proceed to step 2704 . If not, return to step 2702.
  • the controller determines the location and information of the stored ingredients.
  • the controller determines the location of the stored ingredients according to the position of the gravity sensor with the increased mass transmitted, and determines the information of the stored ingredients according to the activation signal returned by the RFID tag received through the RFID antenna.
  • the controller may also update the content of the ingredient management interface according to the determined ingredient location and ingredient information, and render the updated ingredient management interface to the display screen.
  • the controller judges whether the mass transmitted by the gravity sensor is reduced. If it is reduced, the controller can determine that the user has taken out the ingredients from the refrigerator, and proceed to step 2706 . If not, the controller can remind the user to scan the RFID tags of the ingredients.
  • step 2706. Determine whether the controller detects the RFID tag through the RFID antenna within a preset time period. If yes, continue to step 2707. If not, the controller can remind the user to scan the RFID tags of the ingredients.
  • the controller determines the location and information of the ingredients to be taken out.
  • the controller determines the position of the taken-out food material according to the position of the gravity sensor whose mass is reduced by transmission, and determines the information of the taken-out food material according to the RFID tag received through the RFID antenna.
  • the controller may also update the content of the ingredient management interface according to the determined ingredient location and ingredient information, and render the updated ingredient management interface to the display screen.
  • an embodiment of the present application provides an apparatus 900 for implementing a food material management method.
  • the device 900 may perform any step in the above method, and to avoid repetition, details are not described here again.
  • the device 900 includes: an acquisition unit 901 , a processing unit 902 , a storage unit 903 and a display unit 904 .
  • the acquiring unit 901 is configured to acquire the mass transmitted by the gravity sensor in the partition and the shelf; the processing unit 902 is configured to detect the mass transmitted by the first gravity sensor according to the RFID antenna located on the box door when determining that the mass transmitted by the first gravity sensor increases. If the RFID tag of the first food item is received, it is determined that the first food item is stored at the position of the first gravity sensor; The detected RFID tag of the second food material determines that the second food material is taken out from the position of the first gravity sensor; wherein the first gravity sensor is any gravity sensor in the at least one partition , or any gravity sensor in the at least one shelf.
  • the device further includes a storage unit 903, configured to store the mass transmitted by the first gravity sensor
  • the processing unit 902 is specifically configured to: store the first gravity sensor in the storage unit 903 When the mass transmitted by a gravity sensor last time is less than the mass transmitted by the first gravity sensor received this time, it is determined that the mass transmitted by the first gravity sensor has increased; or, the stored in the storage unit 903 When the mass transmitted by the first gravity sensor last time is greater than the mass transmitted by the first gravity sensor received this time, it is determined that the mass transmitted by the first gravity sensor has decreased.
  • the processing unit 902 is specifically configured to: when the box door is opened, control the RFID antenna to emit electromagnetic wave signals outward, and according to the RFID antenna of the first food item received by the RFID antenna, According to the activation signal returned by the tag, it is determined that the tag of the first food is detected, or according to the activation signal returned by the RFID tag of the second food received through the RFID antenna, it is determined that the tag of the second food is detected.
  • the processing unit 902 is specifically configured to: within a first preset time period before determining that the mass transmitted by the first gravity sensor increases, the RFID tag of the first food material detected by the RFID antenna, It is determined that the first food item is stored at the position of the first gravity sensor; or, within a second preset time period after it is determined that the mass transmitted by the first gravity sensor has decreased, the weight of the second food material detected by the RFID RFID tag, then it is determined that the second food item is taken out from the position of the first gravity sensor.
  • the processing unit 902 is specifically configured to determine the position of the first gravity sensor according to the receiving sequence of the mass transmitted by the first gravity sensor; wherein the gravity of different positions in the refrigerator The quality transmitted by the sensor is received in a different order.
  • the device further includes: a display unit 904, configured to update the display when the ingredients are stored on the partition or the shelf or when the ingredients are taken out from the partition or the shelf.
  • the subsequent food material management interface is used to display the food material information and the position of the food material corresponding to the RFID tags of the food material stored on the partition and the shelf.
  • an embodiment of the present application provides a storage medium, and when instructions in the storage medium are executed by a processor, the processor can execute any method implemented by the refrigerator in the above process.

Landscapes

  • Engineering & Computer Science (AREA)
  • Chemical & Material Sciences (AREA)
  • Combustion & Propulsion (AREA)
  • Physics & Mathematics (AREA)
  • Mechanical Engineering (AREA)
  • Thermal Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Cold Air Circulating Systems And Constructional Details In Refrigerators (AREA)

Abstract

一种冰箱及其食材管理方法及装置，用于提高食材的管理效率，该方法为：通过设置在冰箱（100）上的图像采集装置（130），采集冰箱门活动区域的图像，通过设置在储藏室内的辅助传感器（140），采集储藏室内的测量信息，在确定图像中不存在食材存取操作时，则基于测量信息，判断是否存在食材存取操作。这样，提高了食材存取操作的判断准确率，从而提高了食材管理效率和准确率。

Description

一种冰箱及其食材管理方法及装置
相关申请的交叉引用
本申请要求在2021年06月18日提交中国专利局、申请号为202110676948.9、申请名称为“一种冰箱及其食材管理方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中；本申请要求在2021年07月13日提交中国专利局、申请号为202110788223.9、申请名称为“一种冰箱及冰箱的食材管理方法”的中国专利申请的优先权，其全部内容通过引用结合在本申请中；本申请要求在2021年09月06日提交中国专利局、申请号为202111037567.2、申请名称为“一种智能冰箱”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及冰箱技术领域，特别涉及一种冰箱及其食材管理方法及装置。
背景技术
随着科技的逐步发展，冰箱成为日常生活中不可或缺的重要存储装置。冰箱的食材位置管理是食材管理的重点之一，同时也是难点之一。
现有的食材位置管理方案中，冰箱顶部安装有深度相机，通过深度相机，获取食材存取过程中手部的图像，并根据图像的深度信息，确定食材的存取位置。
然而，深度相机存在视场角(Field of Vision，FOV)，深度相机无法获取FOV以外的区域(称为盲区)的图像，特别地，在冰箱场景下，深度相机无法获取冰箱冷藏室第一层最左边以及最右边的区域。此外，由于受手部追踪算法、深度信息之间的误差的影响，存取位置的确定准确率低。
发明内容
本申请提供了一种冰箱及其食材管理方法及装置，用以提高存取位置的确定准确率，提高食材的管理效率。
第一方面，本申请实施例中提供一种冰箱，包括：
机壳，所述机壳包括储藏室，所述储藏室具有开口；
冰箱门，与所述机壳活动连接，用于遮挡所述开口；
图像采集装置，设于所述机壳顶部，用于采集所述冰箱门的活动区域的图像；
辅助传感器，设于所述储藏室内，用于采集所述储藏室内的存储区域的测量信息；
控制器，被配置为：
响应于所述冰箱门开启，获取所述图像采集装置采集的图像，以及获取所述辅助传感器采集的测量信息；
若确定所述图像中不存在食材存取操作，则基于所述测量信息，判断是否存在食材存取操作。
第二方面，本申请实施例中提供一种冰箱，包括：
机壳，所述机壳包括储藏室，所述储藏室具有开口，所述储藏室内包含多层储藏空间，每层储藏空间内均设有至少一个辅助传感器，每个辅助传感器均用于采集测量信息，并将所述测量信息发送到控制器；
冰箱门，与所述机壳活动连接，用于遮挡所述开口；
控制器，被配置为：
响应于所述冰箱门开启，获取各个辅助传感器采集的测量信息；
基于所述测量信息，判断是否存在食材存取操作。
第三方面，本申请实施例中提供一种冰箱，包括：
机壳，所述机壳包括储藏室，所述储藏室具有开口；
冰箱门，与所述机壳活动连接，用于遮挡所述开口；
图像采集装置，设于所述机壳顶部，用于采集所述冰箱门的活动区域的图像；
辅助传感器，设于所述储藏室内，用于采集所述储藏室内的存储区域的测量信息，所述测量信息用于在冰箱门开启，且所述图像采集装置采集的所述图像中不存在食材存取操作时，判断是否存在食材存取操作。
第四方面，本申请实施例中提供一种冰箱，包括：
机壳，所述机壳包括储藏室，所述储藏室具有开口；
冰箱门，与所述机壳活动连接，用于遮挡所述开口；
辅助传感器，设于所述储藏室内，用于采集所述储藏室内的存储区域的测量信息，所述测量信息用于在冰箱门开启时，判断是否存在食材存取操作。
第五方面，本申请实施例中提供一种冰箱，包括：
辅助传感器，所述辅助传感器包括扫码器和测距传感器；
控制器，被配置为：
确定所述扫码器扫描食材上的识别码的第一时间；
确定所述测距传感器测量到用户经过所述测距传感器的测量区域时的第二时间；
根据所述第一时间和所述第二时间，确定所述用户的存取动作；所述存取动作包括：将所述食材存入至所述冰箱、以及将所述食材从所述冰箱中取出。
本申请实施例通过在冰箱中安装扫码器和测距传感器，可以确定出扫码器扫描到食材上的识别码的第一时间，以及测距传感器测量到用户经过测量区域时的第二时间，此时，控制器可以根据第一时间和第二时间，准确地确定出用户的存取动作，也即：确定出用户是将食材存入至冰箱中，还是用户将食材从冰箱中取出；如此，消除了食材的表面光滑度和颜色等信息的影响，有效提高了存取动作的识别准确度，以及识别效率，为冰箱中食材的管理提供了有效的数据参考。
第六方面，本申请实施例中提供一种冰箱，包括：
箱体，内部设有储藏室，所述储藏室中包括至少一个用于存放食材的隔板，所述隔板存放的食材通过射频识别RFID标签进行标识；所述隔板中包括至少一个重力传感器，用于获取存放在所述隔板上的食材的质量；
冰箱门，设于所述储藏室的开口处，所述冰箱门靠近所述储藏室的一侧上包括至少一个用于存放食材的置物架，所述置物架存放的食材通过RFID标签进行标识；所述置物架中包括至少一个重力传感器，用于获取存放在所述置物架中的食材的质量；所述冰箱门靠近所述储藏室的一侧上包括RFID天线，用于扫描食材的RFID标签；
控制器，被配置为执行：
确定第一重力传感器传输的质量增加时，根据所述RFID天线检测到的第一食材的RFID标签，确定所述第一食材存放于所述第一重力传感器的位置处；或者，
确定第一重力传感器传输的质量减少时，根据所述RFID天线检测到的第二食材的RFID标签，确定所述第二食材被从所述第一重力传感器的位置处取出；
其中，所述第一重力传感器为所述至少一个隔板中的任一重力传感器，或者为所述至少一个置物架中的任一重力传感器。
本申请实施例通过在冰箱的隔板和置物架中部署重力传感器，根据重力传感器传输的质量的变化确定用户的存/取食材的动作，并且根据传输的质量发生变化的重力传感器的位置确定用户存/取的食材的位置。并且进一步根据RFID天线检测到食材的RFID标签确定食材信息。不再需要人工输入食材的位置和种类，实现了无感、高效的食材管理。
第七方面，本申请实施例中提供一种食材管理方法，包括：
响应于冰箱门开启，获取图像采集装置采集的冰箱门活动区域的图像，以及获取辅助 传感器采集的储藏室内的存储区域的测量信息，其中，所述图像采集装置设置于机壳顶部，辅助传感器设置于储藏室内；
若确定所述图像中不存在食材存取操作，则基于所述测量信息，判断是否存在食材存取操作。
可选的，若所述辅助传感器为TOF传感器，且所述测量信息中包含的所述辅助传感器与遮挡物之间的距离小于或等于储藏室宽度，则确定存在食材存取操作；或者，若所述辅助传感器为红外对射光电传感器，且所述测量信息中包含的光线强度小于预设的光强门限值，则确定存在食材存取操作；或者，若所述辅助传感器为红外温度传感器，且根据所述测量信息中包含的温度信息，则确定存在食材存取操作。
第八方面，本申请实施例中提供一种食材管理方法，包括：
响应于冰箱门开启，获取储藏室的多层储藏空间内设置的各个辅助传感器采集的测量信息，其中，每层储藏空间内均设有至少一个辅助传感器；
基于所述测量信息，判断是否存在食材存取操作。
第九方面，本申请实施例提供一种食材管理方法，应用于冰箱，所述冰箱的储藏室包括至少一个用于存放食材的隔板，所述隔板中包括至少一个重力传感器；所述冰箱的冰箱门包括至少一个置物架，所述置物架包括至少一个重力传感器；所述方法包括：
获取所述隔板和所述置物架中的重力传感器传输的质量；
确定第一重力传感器传输的质量增加时，根据位于所述冰箱门上的RFID天线检测到的第一食材的RFID标签，确定所述第一食材存放于所述第一重力传感器的位置处；或者，确定第一重力传感器传输的质量减少时，根据位于所述冰箱门上的RFID天线检测到的第二食材的RFID标签，确定所述第二食材被从所述第一重力传感器的位置处取出；其中，所述第一重力传感器为所述至少一个隔板中的任一重力传感器，或者为所述至少一个置物架中的任一重力传感器。
第十方面，一种计算机可读存储介质，当所述计算机可读存储介质中的指令由控制器执行时，使得所述控制器能够执行如上述任一项所述的食材管理方法。
综上所述，本申请实施例中，通过设置在冰箱上的图像采集装置，采集冰箱门活动区域的图像，通过设置在储藏室内的辅助传感器，采集储藏室内的测量信息，在确定图像中不存在食材存取操作时，则基于测量信息，判断是否存在食材存取操作。这样，在根据图像采集装置采集的图像无法确定存在食材存取操作时，可以根据辅助传感器采集的测量信息进行判断，从而提高了食材存取操作的判断准确率，从而提高了食材管理效率和准确率。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本申请实施例中提供的一种冰箱的应用场景图；
图2a为本申请实施例中提供的一种冰箱的结构示意图；
图2b为本申请实施例中提供的一种冰箱的隔层结构示意图；
图3a为本申请实施例中提供的TOF传感器的示意图；
图3b为本申请实施例中提供的红外对射光电传感器的一种示意图；
图3c为本申请实施例中提供的红外对射光电传感器的另一种示意图；
图3d为本申请实施例中提供的红外温度传感器的示意图；
图4a为本申请实施例中提供的另一种冰箱的结构示意图；
图4b为本申请实施例中提供的另一种冰箱的隔层结构示意图；
图4c为本申请实施例中提供的另一种冰箱的隔层结构示意图；
图5为本申请实施例中提供的第一种分布方式的示意图；
图6为本申请实施例中提供的第二种分布方式的示意图；
图7a为本申请实施例中提供的第三种分布方式的示意图；
图7b为本申请实施例中提供的第三种分布方式中的第一层储藏空间的示意图；
图8a为本申请实施例中提供的第一种分布方式中FOV的示意图；
图8b为本申请实施例中提供的第二种分布方式中FOV的示意图；
图8c为本申请实施例中提供的第一种分布方式中FOV的示意图；
图9为本申请实施例中提供的一种冰箱中控制器的结构示意图；
图10为本申请实施例中提供的另一种冰箱中控制器的结构示意图；
图11为本申请实施例中提供的一种食材管理方法的流程示意图；
图12为本申请实施例中提供的一种阵列图像的示意图；
图13为本申请实施例中提供的另一种食材管理方法的流程示意图；
图14为本申请实施例中提供的一种冰箱的结构示意图；
图15为本申请实施例中提供的另一种冰箱的结构示意图；
图16为本申请实施例中提供的另一种冰箱的结构示意图；
图17为本申请实施例中提供的扫码器和测距传感器的具体结构示意图；
图18为本申请实施例中提供的一种冰箱的硬件结构示意图；
图19为本申请实施例中提供的一种具体实施例的流程图；
图20A为本申请实施例提供的另一种冰箱的立体图；
图20B为本申请实施例提供的一种置物架的俯视图；
图20C为本申请实施例提供的一种隔板的俯视图；
图21为本申请实施例提供的一种冰箱的外观示意图；
图22为本申请实施例提供的一种重力传感器传输质量的流程示意图；
图23为本申请实施例提供的一种滤波前后的质量曲线对比图；
图24为本申请实施例提供的一种控制器获取食材信息的流程示意图；
图25为本申请实施例提供的一种食材管理界面的示意图；
图26为本申请实施例提供的一种食材管理界面更新的过程的示意图；
图27为本申请实施例提供的另一种食材管理方法的流程示意图；
图28为本申请实施例提供的一种用于实现冰箱食材管理的装置结构示意图。
具体实施方式
为了使本技术领域的人员更好地理解本申请中的技术方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都应当属于本申请保护的范围。
在本申请的描述中，需要理解的是，术语“中心”、“上”、“下”、“前”、“后”、“左”、“右”、“竖直”、“水平”、“顶”、“底”、“内”、“外”等指示的方位或位置关系为基于附图所示的方位或位置关系，仅是为了便于描述本申请和简化描述，而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作，因此不能理解为对本申请的限制。
术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个该特征。在本申请的描述中，除非另有说明，“多个”的含义是两个或两个以上。
在本申请的描述中，需要说明的是，除非另有明确的规定和限定，术语“安装”、“相连”、“连接”应做广义理解，例如，可以是固定连接，也可以是可拆卸连接，或一体地连接；可 以是机械连接，也可以是电连接；可以是直接相连，也可以通过中间媒介间接相连，可以是两个元件内部的连通。对于本领域的普通技术人员而言，可以具体情况理解上述术语在本申请中的具体含义。
随着经济的发展与人民生活水平的提高，冰箱已经成为人们生活中必不可少的家电之一。人们对于冰箱的要求也已经不再是简单的存储食材，对于冰箱智能化的需求日益迫切。冰箱智能化的核心为食材的管理，其中可以包括：食材的种类、食材的存放位置、以及用户的存取操作三大核心要素。对于存取操作的识别，通常的方法为：在冰箱的顶部安装深度相机，以识别是否存在食材存取操作。
针对现有技术中深度相机无法获取FOV以外的区域的图像，从而无法对FOV以外的区域的食材进行管理，且受手部追踪算法、深度信息之间的误差影响，得到的食材存取位置的准确率低。本申请实施例中，通过辅助传感器采集的储藏室内的存储区域的测量信息，判断是否存在食材存取操作，从而实现对冰箱中位于图像采集装置的盲区的食材进行管理，提高了存取操作的识别准确率，进而提高了食材管理效率。
为了使本领域普通人员更好地理解本申请的技术方案，下面将结合附图，对本申请实施例中的技术方案进行清楚、完整地描述。
参阅图1所示，为本申请实施例提供的一种冰箱的应用场景图。
冰箱100可提供包括但不限于食材管理功能。具体冰箱类型、尺寸大小和功率等不作限定。
冰箱100还与服务器200通过多种通信方式进行数据通信。这里可允许冰箱100通过局域网(Local Area Network，LAN)、无线局域网(Wireless Local Area Network，WLAN)和其他网络进行通信连接。服务器200可以向冰箱100提供各种内容和互动。示例性的，冰箱100可以发送和接收信息，例如：接收图像、访问远程储存的数字媒体库、发送待检测图像数据等。服务器200可以一组，也可以多组，可以一类或多类服务器。服务器200可以部署于本地，也可以部署于云端，通过服务器200实现图像识别等功能。
图2a和图2b中示例性示出了本申请实施例提供的第一种冰箱的结构。
参阅图2a所示，冰箱100包括机壳110、制冷部(未在图中示出)以及其他附件(比如箱体内可设置照明灯、温度计等，未在图中示出)。制冷系统中主要组成有压缩机、冷凝器、蒸发器和毛细管节流器等部件，自成一个封闭的循环系统。其中，蒸发器可安装在冰箱内部的上方，其他部件安装在冰箱的背面。
机壳110安装有冰箱门120。在一些实施例中，冰箱门120上可进一步设置有显示屏(未在图中示出)，显示屏与控制器耦接(如通过电路连接)。
机壳110可以包括储藏室。冰箱100的外观可以通过限定储藏室的储藏空间和冰箱门120的位置进行确定。在一些实施例中，储藏室为具有开口的箱体，其由储藏室内胆、储藏室外壳以及位于两者之间的发泡层形成。冰箱门120用于遮挡储藏室的开口。储藏室被竖直分隔成位于下方的冷冻室以及位于上方的冷藏室。其中，冷冻室和冷藏室可分别具有独立的储藏空间。
在一些实施例中，冰箱门120可以包括冷冻室门体和冷藏室门体。示例性的，冷冻室限定在储藏室的下侧处并且可通过冷冻室门体将冷冻室的开口进行选择性地覆盖。
在一些实施例中，冷藏室限定在储藏室的上侧处，即冷藏室设置在冷冻室上方。并且，通过冷藏室门体将冷藏室的开口进行选择性地覆盖。在实际应用中，冷藏室门体可枢转地安装在冷藏室上，这样可以使冷藏室的开口通过冷藏室门体选择性地打开或关闭。
在一些实施例中，在本申请实施方式中的冰箱的储藏室内，可以包括储物抽屉以及位于储物抽屉上的第一层架和第二层架。其中，储物抽屉第一层架和第二层架可以分别用来放置存储食材(例如，水果、蔬菜等)。
在机壳110上还可设置图像采集装置130，该图像采集装置130用于采集冰箱门120的活动区域的图像。其中，以冰箱门所在平面为第一平面，机壳110前侧区域至少包括以 第一平面为基准，向冰箱外方向扩展一定距离的区域，图像采集装置130可采集该区域的图像，即可以拍摄到用户打开冰箱门120后进行食材存取过程中的手部动作图像以及所存取的食材的图像。示例性的，图像采集装置130可以采用但不限于深度相机。
参阅图2b所示，冰箱的机壳110中可包括多个隔层(如图中的隔层50a至隔层50d)，多个隔层将储藏室分割为多层储藏空间以方便用户对不同食材进行分类存储。
在一些实施例中，图像采集装置130可设置在机壳110的顶部。示例性的，像采集装置130可设置在机壳110的上部靠近冰箱门120的位置，以便能够拍摄冰箱门120的活动区域内的图像。
在一些实施例中，储藏室内包含多层储藏空间，辅助传感器140设置于顶层储藏空间中靠近冰箱门120的一侧。
在一些实施例中，冰箱110中还包含辅助传感器140。辅助传感器140设置于储藏室内，用于采集储藏室内的存储区域的测量信息。示例性的，辅助传感器140可以采用但不限于红外对射光电传感器、单点TOF传感器、红外温度传感器、超声波雷达、毫米波雷达等。
下面，分别以传感器140为红外对射光电传感器、单点TOF传感器、红外温度传感器为例进行说明。
参阅图3a所示，传感器140可以采用单点TOF传感器。单点TOF传感器安装在储藏空间中靠近冰箱门120的一侧，若不存在食材存取操作，单点TOF传感器测量得到的距离L1的取值为冰箱的宽度L2。若存在食材存取操作，由于手部或者食材的遮挡，单点TOF传感器测量得到的距离L2小于冰箱的宽度L2。基于上述原理，根据单点TOF传感器测得的距离，可以确定是否存在食材存取操作，具体的，若单点TOF传感器单点TOF传感器测量得到的距离L2小于冰箱的宽度L2，则确定存在食材存取操作。
参阅图3b，传感器可以采用红外对射光电传感器。红外对射光电传感器包含发射部件和接收部件，接收部件和发射部件设置于不同的两端，发射部件发射红外发射光束，接收部件根据接收到红外发射光束，判断是否存在遮挡，即是否存在食材存取操作。
若发射部件发射的红外发射光束是一束很细的激光束，那么在实际应用过程中，对红外对射光电传感器的安装位置的精度要求高，因此，为降低传感器安装要求，参阅图3c所示，红外发射光束可以有一定FOV。本申请实施例中，可以将辅助传感器的FOV之外的范围称为辅助传感器的盲区。
参阅图3d，辅助传感器还可以采用红外温度传感器。红外温度传感器可以检测温度变化，进而根据温度变化确定是否存在食材存取操作。例如，当位于顶层储藏空间的红外温度传感器检测到温度骤升时，确定顶层储藏空间中存在食材存取操作。
图4a和图4b中示例性示出了本申请实施例提供的第二种冰箱的结构。
参阅图4a所示，冰箱200包括机壳210、制冷部(未在图中示出)以及其他附件(比如箱体内可设置照明灯、温度计等，未在图中示出)。机壳210安装有冰箱门220。机壳210可以包括储藏室，储藏室为具有开口的箱体。冰箱门220用于遮挡储藏室的开口。冰箱200的结构，具体可参阅冰箱100，在此不再赘述。
参阅图3b所示，冰箱200的机壳210中可包括多个隔层，多个隔层将储藏室分割为多层储藏空间以方便用户对不同食材进行分类存储。冰箱200中包含多个辅助传感器230。
在一些实施例中，每层储藏空间中均设置一个辅助传感器230。辅助传感器230可以采用但不限于单点TOF传感器、红外温度传感器、红外对射光电传感器中的任一项。
需要说明的是，由于红外对射光电传感器的发射部件和接收部件不在同一端，因此，参阅图4c所示，若辅助传感器230采用红外对射光电传感器，则可将红外对射光电传感器的发射部件和接收部件分别设置于储藏空间中靠近冰箱门220的两侧。
在一些实施例中，由于红外对射光电传感器的发射部件存在盲区，当对体积较小的食材存取时，可能出现漏检的情况，因此，冰箱200的每层储藏空间中均设置多个红外对射 光电传感器，这样，通过密布(即每层储藏空间中设置多个红外对射光电传感器)的方式，以实现减小甚至消除盲区。
下文中，仅以每层储藏空间中设置两个红外对射光电传感器为例进行说明。
第一种分布方式：在储藏室的一侧，每层储藏空间中设有红外对射光电传感器的发射部件，相应的，在储藏室的另一侧，设有相应的接收部件。
参阅图5所示，第一层储藏空间中，储藏室的左侧设置有发射部件1和发射部件2，储藏室的右侧设置有接收部件1和接收部件2，其中，接收部件1和发射部件1属于红外对射光电传感器1，接收部件2和发射部件2属于红外对射光电传感器3。第二层储藏空间中，储藏室的左侧设置有发射部件3和发射部件4，储藏室的右侧设置有接收部件3和接收部件4，其中，接收部件3和发射部件3属于红外对射光电传感器3，接收部件4和发射部件4属于红外对射光电传感器4。
然而，采用第一种分布方式，密布多个红外对射光电传感器可能会造成接收部件之间的干扰，进而导致不能准确地检测出存取动作。例如，各个红外对射光电传感器的FOV的取值为：FOV＝2×arctan(height/width)，其中，height表示每层储藏空间的层高，width用于表示每层储藏空间的宽度。
当遮挡物(食材或者手部)位于如图5所示的位置时，对于接收部件2，虽然物体挡住了发射部件2发射的红外激光光束，接收部件2接收不到发射部件2发射的红外激光光束，但是接收部件2可以接收到发射部件3发射的红外激光光束。
为了减少多个红外对射光电传感器之间的干扰，可以采用但不限于以下方式：
第二种分布方式：在储藏室的一侧，每层储藏空间中交替设有多个辅助传感器的发射部件和接收部件，在储藏室的另一侧，设有相应的接收部件和发射部件。
参阅图6所示，第一层储藏空间中，储藏室的左侧依次设置有发射部件1和接收部件2，储藏室的右侧对应设置有接收部件1和发射部件2，其中，接收部件1和发射部件1属于红外对射光电传感器1，接收部件2和发射部件2属于红外对射光电传感器3。第二层储藏空间中，储藏室的左侧设置有发射部件3和接收部件4，储藏室的右侧对应设置有接收部件3和发射部件4，其中，接收部件3和发射部件3属于红外对射光电传感器3，接收部件4和发射部件4属于红外对射光电传感器4。
显然，采用第二种方式可以一定程度上消除这种多个红外对射光电传感器之间的干扰，且相较于第一种方式，在不减少发射部件的FOV的条件下，发射器密布的间隔可以减小1/2。
然而，对于FOV的取值大于2×arctan(height/width)的红外对射光电传感器，采用第一种方式和第二种方式均会产生一定程度的干扰，为了进一步的减少多个红外对射光电传感器之间的干扰，本申请实施例中提供了第三种可能的实现方式。
第三种分布方式：在储藏室的一侧，一层储藏空间设有多个辅助传感器的发射部件，且相邻层储藏空间内设有辅助传感器的接收部件，在储藏室的另一侧，设有相应的接收部件和发射部件。
参阅图7a所示，第一层储藏空间中，储藏室的左侧依次设置有发射部件1和发射部件2，储藏室的右侧对应设置有接收部件1和接收部件2，其中，接收部件1和发射部件1属于红外对射光电传感器1，接收部件2和发射部件2属于红外对射光电传感器3。第二层储藏空间中，储藏室的左侧设置有接收部件3和接收部件4，储藏室的右侧对应设置有发射部件3和发射部件4，其中，接收部件3和发射部件3属于红外对射光电传感器3，接收部件4和发射部件4属于红外对射光电传感器4。在第三层储藏空间中，储藏室的左侧依次设置有发射部件5和发射部件6，储藏室的右侧对应设置有接收部件5和接收部件6，其中，接收部件5和发射部件5属于红外对射光电传感器5，接收部件6和发射部件6属于红外对射光电传感器6。这样，减少了不同层储藏空间的红外对射光电传感器之间的干扰。
需要说明的是，参阅图7b所示，由于红外对射光电传感器的FOV的取值大于2×arctan(height/width)，因此，接收部件1可以同时接收到发射部件1发射出的红外激光光束和发射部件2发射出的红外激光光束，接收部件1也可以同时接收到发射部件1发射出的红外激光光束和发射部件2发射出的红外激光光束。若用户在第一层储藏空间中进行食材存取，接收部件1接收不到发射部件2发射出的红外激光光束，但可以接收到发射部件2发射出的红外激光光束，接收部件2接收不到发射部件1发射出的红外激光光束，但可以接收到发射部件1发射出的红外激光光束。为了提高检测效率，可以在接收部件接收到的光线强度小于预设光强阈值时，确定存在食材存取操作。其中，预设光强阈值为无遮挡时接收部件接收到的光线强度。
下面，针对上述三种分布方式，依次说明采用上述三种分布方式时，多个红外对射光电传感器之间不产生干扰的目标FOV，也可以称为最大FOV。
参阅图8a所示，当红外对射光电传感器的FOV小于2×arctan((2b/3)/a)时，相邻层储藏空间的红外接收传感器之间不会产生干扰，其中，b表示每层储藏空间的层高，a用于表示每层储藏空间的宽度。例如，当b＝20cm，a＝75cm，最大FOV的取值为20.16°。
参阅图8b所示，当红外对射光电传感器的FOV小于2×arctan(b/a)时，相邻层储藏空间的红外接收传感器之间不会产生干扰。例如，当b＝20cm，a＝75cm，最大FOV的取值为29.86°。
参阅图8c所示，当红外对射光电传感器的FOV小于2×arctan((5b/3)/a)时，相邻层储藏空间的红外接收传感器之间不会产生干扰。例如，当b＝20cm，a＝75cm，最大FOV的取值为47.92°。
参阅表1所示，其为上述三种分布方式对红外对射光电传感器的FOV的要求，即最大FOV。可见，相对于第一种分布方式，第二种分布方式和第三种分布方式均减少了对红外对射光电传感器的FOV的要求，降低了设备安装难度，且减少了多个红外对射光电传感器之间的干扰。相较于第一种分布方式和第二种分布方式，第三种分布方式极大地减少对红外对射光电传感器的FOV的要求，同时，减少相邻或间隔冰箱隔层间的相互干扰，大大提高了检测的稳定性，简化了冰箱管理流程，从而方便了用户对冰箱的管理。
表1 三种分布方式对红外对射光电传感器的FOV的要求
Figure PCTCN2021139239-appb-000001
需要说明的是，图2a、图2b、图4a、图4b、图4c所示的冰箱的结构仅为一种实施例，本申请实施例对冰箱的大小、以及对冰箱门的数量(比如可以是单个冰箱门，也可以是多个冰箱门)等不做限制，比如，在一些实施例中，冰箱中设置有射频识别(Radio Frequency Identification，RFID)读写器，可用来读取食材包装上的RFID标签，以获取食材的种类和数量等信息。在另一些实施例中，冰箱还具有语音功能，能够识别输入的语音，以获取用户通过语音方式输入的食材的种类和数量等信息。
图9示例性示出了一种冰箱中的控制器的结构示意图，如图所示，控制器900与图像采集装置、辅助传感器相连接。控制器900可包括：获取单元901和处理单元902。
其中，获取单元901，用于响应于冰箱门开启，获取图像采集装置采集的冰箱门活动区域的图像，以及获取辅助传感器采集的储藏室内的存储区域的测量信息，其中，所述图像采集装置设置于机壳顶部，辅助传感器设置于储藏室内；
处理单元902，用于若确定所述图像中不存在食材存取操作，则基于所述测量信息，判断是否存在食材存取操作。
可选的，处理单元902具体用于：
若所述辅助传感器为单点飞行时间TOF传感器，且所述测量信息中包含的所述辅助传感器与遮挡物之间的距离小于或等于储藏室宽度，则确定存在食材存取操作；或者，若所述辅助传感器为红外对射光电传感器，且所述测量信息中包含的光线强度小于预设的光强门限值，则确定存在食材存取操作；或者，若所述辅助传感器为红外温度传感器，则根据所述测量信息中包含的温度信息，确定存在食材存取操作。
图10示例性示出了另一种冰箱中的控制器的结构示意图，如图所示，控制器1000与辅助传感器相连接。控制器1000可包括：获取单元1001和判断单元1002。
其中，获取单元1001，用于响应于冰箱门开启，获取储藏室的多层储藏空间内设置的各个辅助传感器采集的测量信息，其中，每层储藏空间内均设有至少一个辅助传感器；
判断单元1002，用于基于所述测量信息，判断是否存在食材存取操作。
可选的，判断单元1002具体用于：
若所述辅助传感器为TOF传感器，则基于所述测量信息中包含的所述辅助传感器与遮挡物之间的距离，判断是否存在食材存取操作；或者，若所述辅助传感器为红外对射光电传感器，则基于所述测量信息中包含的光线强度，判断是否存在食材存取操作；或者，若所述辅助传感器为红外温度传感器，则根据所述测量信息中包含的温度信息，判断是否存在食材存取操作。
下面对本申请实施中提供的食材管理方法进行说明。
参阅图11所示，其为本申请实施例中提供的一种控制器，应用于上述图2a、图2b所示的冰箱中，可以被配置为执行如下程序步骤：
S1101、响应于冰箱门开启，获取图像采集装置采集的冰箱门活动区域的图像，以及获取辅助传感器采集的储藏室内存储区域的测量信息，其中，图像采集装置设置于机壳顶部，辅助传感器设置于储藏室内。
其中，在图像采集装置在冰箱前方检测到手部图像时，冰箱向图像采集装置发送拍摄指令，以接收图像采集装置发送的图像。在一些实施例中，发送的图像可为深度图像或彩色图像，其图像格式可为：JPEG、TIFF、RAW、BMP、GIF、PNG等。
示例性的，若辅助传感器采用TOF传感器，则测量信息中可以包含辅助传感器与遮挡物之间的距离，若辅助传感器采用红外对射光电传感器，则测量信息中可以包含光线强度，若辅助传感器采用红外温度传感器，则测量信息中可以包含温度信息。
S1102、若确定图像中不存在食材存取操作，则基于测量信息，判断是否存在食材存取操作。
本申请实施例中，可以采用但不限于特征提取的方式确定，如提取Haar特征(Haar-Like Features，哈尔特征)、LBP特征(Local Binary Pattern，局部二值特征)、HOG特征(Histogram Of Oriented Gradients，方向梯度直方图)等，也可以使用神经网络的算法，其中神经网络可以是但不限于(深度)神经网络、卷积神经网络、深度置信网络、深度堆栈神经网络等网络结构的一种或其衍生模型，对此不做限定，在此不再赘述。
本申请实施例中，存在但不限于以下三种情况：
情况一：若辅助传感器为TOF传感器，且测量信息中包含的辅助传感器与遮挡物之间的距离小于或等于储藏室宽度，则确定存在食材存取操作。
其中，遮挡物可以是指食材，也可以是指用户的手部。
例如，假设，储藏室宽度为75厘米，测量信息中包含的辅助传感器与遮挡物之间的 距离为60厘米，此时，测量信息中包含的辅助传感器与遮挡物之间的距离小于储藏室宽度，确定存在食材存取操作。
由于测量信息中包含的辅助传感器与遮挡物之间的距离，在一些实施例中，也可以根据测量信息中包含的辅助传感器与遮挡物之间的距离，确定食材存取位置。
情况二：若辅助传感器为红外对射光电传感器，且测量信息中包含的光线强度小于预设的光强门限值，则确定存在食材存取操作。
其中，预设的光强门限值可以是无遮挡时红外对射光电传感器的接收部件接收到的光线强度。
例如，假设，预设的光强门限值为10坎德拉(candela，cd)，测量信息中包含的光线强度为5cd，此时，测量信息中包含的光线强度小于预设的光强门限值，确定存在食材存取操作。
情况三：若辅助传感器为红外温度传感器，且根据测量信息中包含的温度信息，则确定存在食材存取操作。
具体的，温度信息中包含各个采集时刻各自对应的像素值集合，若一个像素值集合中，目标像素值的数目超过设定数目阈值时，确定存在食材存取操作，其中，目标像素值用于表征温度值集合中与设定像素值的差值，达到预设的差值阈值的像素值。
以红外温度传感器采用阵列红外温度传感器为例，假设，红外温度传感器为像素8×8的红外温度传感器，也就是说，温度信息中，一个温度值集合包含8×8个像素值，像素值也可以，8×8个温度值如下所示：
[‘19.0’，‘18.0’，‘20.5’，‘20.8’，‘21.1’，‘22.0’，‘22.0’，‘22.0’]
[‘16.5’，‘16.8’，‘19.8’，‘20.5’，‘20.8’，‘21.0’，‘21.8’，‘22.2’]
[‘17.8’，‘17.5’，‘19.0’，‘20.2’，‘20.5’，‘21.8’，‘22.0’，‘22.8’]
[‘19.2’，‘18.8’，‘19.5’，‘20.2’，‘21.0’，‘21.5’，‘22.5’，‘22.5’]
[‘24.5’，‘25.8’，‘26.0’，‘22.5’，‘22.8’，‘22.8’，‘22.8’，‘22.5’]
[‘27.0’，‘28.8’，‘28.8’，‘27.0’，‘24.0’，‘23.2’，‘22.8’，‘23.2’]
[‘27.8’，‘29.0’，‘29.2’，‘29.0’，‘26.0’，‘23.8’，‘22.8’，‘23.5’]
[‘27.2’，‘29.0’，‘30.2’，‘29.5’，‘25.2’，‘23.2’，‘22.8’，‘23.2’]
参阅图12所示，对红外温度传感器采集的测量信息进行图像金字塔上采样、伪彩色变换后，得到处理后的阵列图像，其中，(a)为无遮挡时的阵列图像，(b)为有遮挡时的阵列图像。
参阅图13所示，其为本申请实施例中提供的另一种控制器，应用于上述图4a、图4b、图4c所示的冰箱中，可以被配置为执行如下程序步骤：
S1301、响应于冰箱门开启，获取储藏室的多层储藏空间内设置的各个辅助传感器采集的测量信息，其中，每层储藏空间内均设有至少一个辅助传感器。
其中，若辅助传感器采用TOF传感器，则测量信息中可以包含辅助传感器与遮挡物之间的距离，若辅助传感器采用红外对射光电传感器，则测量信息中可以包含光线强度，若辅助传感器采用红外温度传感器，则测量信息中可以包含温度信息。
S1302、基于测量信息，判断是否存在食材存取操作。
由于S1302，与S1102中判断是否存在食材存取操作的方式相同，在此不再赘述。
在一种可选的实施例中，为了能够更准确地识别用户的存取动作，还可以设置扫码器和测距传感器等辅助传感器，控制器可以确定扫码器扫描食材上的识别码的第一时间，确定测距传感器测量到用户经过测距传感器的测量区域时的第二时间，根据第一时间和所述第二时间，确定用户的存取动作；存取动作包括：将食材存入至冰箱、以及将食材从冰箱中取出。
在一种可选的实施例中，辅助传感器还可以包括设置在冰箱储藏室内的隔板中的至少一个重力传感器、冰箱门上的置物架中的至少一个重力传感器和位于冰箱门上的RFID天 线，控制器可以获取隔板和置物架中的重力传感器传输的质量；确定第一重力传感器传输的质量增加时，根据位于冰箱门上的RFID天线检测到的第一食材的RFID标签，确定第一食材存放于第一重力传感器的位置处；或者，确定第一重力传感器传输的质量减少时，根据位于冰箱门上的RFID天线检测到的第二食材的RFID标签，确定第二食材被从所述第一重力传感器的位置处取出；其中，第一重力传感器为至少一个隔板中的任一重力传感器，或者为至少一个置物架中的任一重力传感器。
在另一些实施例中，为了避免出现由于食材的表面光滑度、颜色等信息的影响，导致深度相机测距不准，最终导致不能正确识别用户的存取动作的现象。本申请实施例提供一种冰箱，如图14所示，该冰箱可以包括：扫码器10，测距传感器20和控制器30。其中，扫码器10和测距传感器20可以统称为辅助传感器。控制器30，被配置为：确定扫码器10扫描食材上的识别码的第一时间；确定测距传感器20测量到用户经过测距传感器的测量区域时的第二时间；根据第一时间和第二时间，确定用户的存取动作；存取动作包括：将食材存入至冰箱、以及将食材从冰箱中取出。
如此，通过在冰箱中安装扫码器和测距传感器，可以确定出扫码器扫描到食材上的识别码的第一时间，以及测距传感器测量到用户经过测量区域时的第二时间，此时，控制器可以根据第一时间和第二时间，准确地确定出用户的存取动作，也即：确定出用户是将食材存入至冰箱中，还是用户将食材从冰箱中取出；如此，消除了食材的表面光滑度和颜色等信息的影响，有效提高了存取动作的识别准确度，以及识别效率，为冰箱中食材的管理提供了有效的数据参考。
在具体实施时，在本申请实施例中，对于扫码器和测距传感器的设置方式，可以包括以下方式：
方式1：
可选地，在本申请实施例中，如图15所示，冰箱包括：冰箱门40和箱体50；
扫码器10和测距传感器20均位于冰箱门包括的隔层中的同一层(未给出图示)，和/或扫码器10和测距传感器20均位于箱体50包括的隔层a中的同一层(如图15所示，且扫码器10和测距传感器20并不限于位于隔层a的左侧，还可以位于隔层a的右侧)；
测距传感器20位于扫码器10靠近箱体50的内部的一侧(如图15所示中的局部放大示意图，从局部放大示意图中来看扫码器10对测距传感器20造成了遮挡，以表示测距传感器20更靠近箱体50的内部)。
如此，使得在确定用户的存取动作时，可以符合用户存取的逻辑，符合用户的思维习惯，进而使得用户在使用该冰箱时并不需要按照某个特殊要求存取食材，而是根据平常的思维资管存取食材即可，为用户的使用提供了便利，同时还可以实现存取动作的识别。
具体地，在本申请实施例中，扫码器与测距传感器之间的距离小于预设值。
其中，预设值可以根据实际需要进行设置，在此并不限定。
也即：扫码器和测距传感器设置的较近，使得用户在从冰箱中拿出食材想在扫码器上进行扫码时，必然会经过测距传感器的测量区域，或者，用户想存放食材并拿着食材在扫码器上进行扫码之后，必然也会经过测距传感器的测量区域。
说明一点，当用户取出食材时，需要扫码且取出食材这一过程，假设扫码的时刻(也即扫码器扫描到识别码的时刻)为t，在这之前(如t-△t)若检测到食材或用户的手经过了测距传感器的测量区域，根据这一逻辑，可以确定出食材为取出；
在用户存入食材时，假设扫码的时刻为t，在这之前(如t-△t)未检测到食材或用户的手经过了测距传感器的测量区域，且在这之后(如t+△t)检测到食材或用户的手经过了测距传感器的测量区域，则可以确定食材为存入；
其中，扫码的时刻与食材或用户的手经过了测距传感器的测量区域的时刻之间的时间间隔(也即△t)很短，也即△t可以较小。
例如，若扫码器扫描到识别码的时刻为t1，测距传感器检测到用户经过测量区域的时 刻为t2，用户的手在存取食材的过程中的移动速度为v，扫码器和测距传感器之间的距离为d，假设忽略扫码器获取识别码的时间、以及测距传感器的响应时间，那么：
t1-t2＝△t＝d/v；
其中，v可以约为1m/s，d可以约为5cm时，△t约为50ms。
如此，可以实现存取动作的无感识别，即用户在存取食材时，仅需要在相应的位置完成扫码操作，冰箱即可完成存取动作的管理。
并且，无需对用户的存取动作进行特殊的规定，仅凭用户的思维习惯即可确定存取动作，简化了存取动作识别的难度，提高了存取动作识别的效率。
此外，可以有利于提高用户的存取动作的识别准确率，避免因扫码器和测距传感器之间相隔的太远而导致第一时间和第二时间之间的时长较大，进而避免出现误判。
具体地，在本申请实施例中，控制器被配置为：
在第一时间先于第二时间时，确定用户将食材存入至冰箱；
在第一时间晚于第二时间时，确定用户将食材从冰箱中取出；
其中，第一时间与第二时间之间的时间差小于预设时间。
此处提及的预设时间可以理解为上述内容提及的△t。
也就是说，由于测距传感器位于扫码器靠近箱体的内部的一侧，所以若扫码器先扫描到识别码，之后测距传感器检测到用户经过测量区域，那么可以确定用户是将食材存入至冰箱中；若测距传感器先检测到用户经过测量区域，之后扫码器先扫描到识别码，那么可以确定用户是将食材从冰箱中取出。
如此，可以根据第一时间和第二时间确定出用户的存取动作，为食材的管理提供有效的数据参考。
当然，具体地，在测距传感器位于扫码器远离箱体的内部的一侧时，控制器还可以被配置为：
在第一时间晚于第二时间时，确定用户将食材存入至冰箱；
在第一时间先于第二时间时，确定用户将食材从冰箱中取出。
也就是说，可以基于扫码器和测距传感器的设置位置，再结合第一时间和第二时间的先后顺序，确定用户的存取动作。
具体地，在对扫码器和测距传感器的设置数量进行设置时，可以包括以下几种情况：
情况1：
可选地，在本申请实施例中，扫码器和测距传感器均设置有一个，未给出图示。
也即：仅在某一隔层中设置有扫码器和测距传感器。
如此，可以有效减少扫码器和测距传感器的设置数量，进而降低冰箱的制作成本，同时还可以实现用户的存取动作的识别。
情况2：
可选地，在本申请实施例中，扫码器设置有一个，测距传感器设置有多个；
冰箱包括：冰箱门和箱体，且冰箱门和/或箱体包括多个隔层；
每个隔层中均设置有一个测距传感器，未给出图示。
其中，扫码器可以设置任一隔层中。
如此，可以有效减少扫码器的设置数量，进而在一定程度上降低冰箱的制作成本，同时还可以实现用户的存取动作的识别。
情况3：
可选地，在本申请实施例中，扫码器和测距传感器均设置有多个；
每个隔层中均设置有扫码器和测距传感器。
如此，可以对每个隔层中的食材的存取进行监控和分析，实现对冰箱内各位置的有效监控。
总之，在具体实施时，在对扫码器和测距传感器的数量进行设置时，可以采用上述情 况1、情况2和情况3中的任一种，可以根据实际需要进行选择，在此并不限定。
方式2：
可选地，在本申请实施例中，扫码器10和测距传感器20位于不同隔层，如图16所示，扫码器10位于隔层此种，测距传感器20位于隔层b中。
说明一点，在此方式2中，扫码器10和测距传感器20的设置位置并不限于图16所示，此处只是以图16所示为例进行说明而已。
如此，可以根据实际需要对扫码器和测距传感器的位置进行设置，以提高设计的灵活性，满足不同应用场景的需要。
具体地，在此方式2中，同样地，可以基于扫码器和测距传感器的设置，再结合第一时间和第二时间的先后顺序，确定用户的存取动作；具体实施方式可以参见上述方式1中的实施例，在此不再详述。
具体地，在此方式2中，在对扫码器和测距传感器的数量进行设置时，可以根据需要进行随意设置，只要保证扫码器和测距传感器在不同的隔层即可。
总之，在对扫码器和测距传感器的位置进行设置时，可以参见上述方式1和方式2中任一种，具体可以根据实际需要进行选择，在此并不限定。
在具体实施时，本申请实施例中提及的冰箱除了可以确定用户的存取动作之外，还可以包括其他功能，具体如下：
1、确定食材的存放位置。
可选地，在本申请实施例中，冰箱包括多个隔层，至少部分隔层内设置有扫码器10，如图15所示；
控制器被配置为：根据扫描到识别码的扫码器的标识，确定扫描到的识别码对应食材存放的对应隔层。例如，以图15所示为例，在每个隔层中均设置有扫码器10时，假设用户想要将苹果存入标记为a的隔层中，那么：用户可以拿着苹果到隔层a中的扫码器(记为扫码器a)上扫一下，使得扫码器a可以扫描到苹果上的识别码；扫码器a扫描到识别码时，可以向控制器发送携带有识别码和自身标识的消息；控制器接收到消息时，即可确定出是扫码器a扫描到了识别码，进而确定扫描到的识别码对应的苹果存放的隔层为隔层a。
也就是说，食材的存放位置可以通过区分扫码器的标识实现，用户需要管理的每个隔层均安装了扫码器，用户在某个隔层存取食材时，需要对着相应隔层的扫码器进行扫码，并把扫码后的食材放入与扫码器对应的隔层中；控制器可以通过解析扫码器的标识和隔层的对应表，区分食材的存取位置。如此，仅依据扫码器的标识即可确定出食材的存放位置，进而为食材的管理提供有效的数据参考。
2、确定食材的种类。
可选地，在本申请实施例中，控制器被配置为：
根据扫码器扫描到的识别码、以及预设的识别码与食材种类的对应关系(如下面表2所示，其中表2中是以识别码为条码为例示出的，且仅示出了部分条码和食材的对应关系)，确定识别码对应食材的种类。
表2
条码 食材
6932571080478 巧克力牛奶饮品
6902131110549 梨
6902265908011 苹果
6946681900052 青岛啤酒
如此，可以快速、准确、有效地确定出食材的种类，避免出现误判，在提高识别准确率的同时，还可以有利于提高冰箱的管理效率。
具体地，在本申请实施例中，识别码可以包括：条形码和/或二维码。
当然，识别码除了可以包括条形码和二维码之外，还可以包括其他可以用于区分种类的形式，在此并不限定。
具体地，食材的种类是通过粘贴在食材上面的识别码确定的，以识别码为条形码为例：
识别码可为标准的条形码，也可为自定义的条形码；
对于自定义的条形码，控制器可通过本地存储的条形码和食材的种类的对照表，确定食材的种类；
对于标准的条形码，由于食材的种类成千上万，且会更新，控制器可以通过访问云端数据库接口的方式获取食材的种类。
3、食材管理功能。
具体地，在本申请实施例中，食材管理功能可以包括：剩余保质期查询、数量查询、位置查询、营养百科查询、以及膳食推荐等；
其中，剩余保质期查询的原理可以为：以食材首次存入冰箱的时间为时间起点，保质期减去食材在冰箱中的存放时间即为剩余保质期；具体地，控制器可以记录食材首次存入的时间，通过查询本地数据库或者云端数据库即可确定出食材的保质期，进而确定出剩余保质期；
数量查询的原理可以为：根据确定出的用户的存取动作，记录每次向冰箱中存入的食材和取出的食材，基于此，可以确定出一段时间内冰箱中剩余的食材的数量和种类；
位置查询的原理可以为：基于上述食材的种类和存放位置的确定方法，可以确定出每次向冰箱存入的食材的种类以及存放位置，为用户在后期需要查询时提供查询数据；
营养百科查询可以为控制器预先存储在本地数据库或者从云端数据库获取到的；
膳食推荐的原理可以为：如特定一段时间，监测用户从冰箱中取出的食材，根据取出的食材分析用户近期是否科学地摄入了身体所需营养元素。
4、辅助功能。
可选地，在本申请实施例中，还可以包括：显示器和/或播放器；显示器，被配置为：显示控制器确定出的信息；播放器，被配置为：播放控制器确定出的信息。如此，可以通过显示器和播放器将确定出的信息显示和播放出来，以便于展示给用户。
可选地，在本申请实施例中，还可以包括：语音采集器；语音采集器被配置为：采集用户输入的语音信息。
具体地，在包括显示器和语音采集器时，用户可以通过显示器输入或语音输入的方式实现保质期查询、数量查询、位置查询、种类查询等功能；其中，对于膳食推荐功能，可以通过显示器弹窗推荐的方式来推荐；保质期查询、数量查询、位置查询，营养百科查询等，既可以通过语音输入的方式查询，也可以通过显示器输入的方式查询。
总之，上述给出了4种其他功能，但在实际情况中，冰箱还可以实现除上述全部功能之外，且通过扫码器、测距传感器和控制器实现的其他功能，在此不再详述。
可选地，在本申请实施例中，测距传感器可以包括单点飞行时间传感器。
具体地，如图17所示，扫码器10可以包括补光灯11、定位灯条12、以及感光元件13三部分；其中，补光灯11用于：增加感光元件13扫码时的亮度；定位灯条12用于：方便用户对准条码扫描；感光元件13用于拍：摄条形码；单点飞行时间传感器20可以包括：发射端21和接收端22；发射端可以发射诸如但不限于红外射线，在红外射线入射到用户的手或食材表面时发生反射，反射的红外射线可以进入至接收端，在接收端接收到反射的红外射线时，即可确定用户经过了测量区域；或者，在接收端接收到反射的红外射线时，还可以理解为：用户靠近测距传感器，或者用户遮挡测距传感器。
基于上述对冰箱的介绍，可以确定的是：如图18所示，本申请实施例提供的冰箱可以包括：感知层、决策层、以及交互层三部分；其中，感知层用于：负责数据的感知获取，其中包括：识别码的获取、扫码器标识的获取、以及测距传感器对用户经过测量区域的识 别；并且，扫码器与控制器之间可以但不限于通过UART(Universal Asynchronous Receiver/Transmitter，通用异步收发传输器)接口传输信息，测距传感器与控制器之间可以但不限于通过为I2C接口传输信息；决策层用于：对获取和识别到的信息进行解析，以确定出食材的种类、食材的存放位置、以及用户的存取动作；交互层用于：采用直观的方式，将感知层感知到的内容和决策层解析出的数据通过显示器和播放器呈现给用户，方便用户的使用、查询；并且，控制器与显示器之间可以但不限于通过UART接口传输信息，控制器与播放器之间可以但不限于通过I2S接口传输信息。
下面结合图15和图18所示，举例说明。若用户想要在冰箱的第一隔层中存入青岛啤酒，那么用户的动作依次为：在第一隔层的扫码器上进行扫码；将青岛啤酒放至第一隔层。
其中，在上述动作的执行过程中，当用户在第一隔层中的扫码器(记为扫码器1)上进行扫码时，扫码器1可以获取到青岛啤酒的识别码，并将获取到的识别码和扫码器1的标识传输至决策层；决策层中的控制器可以通过解析识别码，确定出食材的种类为青岛啤酒，再根据标识确定出扫码器1位于第一隔层；用户为先扫码，再存入，且测距传感器位于扫码器靠近箱体内部的一侧，假设忽略扫码器获取识别码的时间、以和测距传感器的响应时间，那么扫码器获得识别码的时刻先于测距传感器被挡住(也即测距传感器检测到用户经过测量区域，或者可以理解为用户的手是否靠近测距传感器)的时刻，至此决策层可以判断出，用户刚才的动作为向第一隔层中存入青岛啤酒；交互层可以在显示器上弹出第一隔层中存入青岛啤酒的图片，和/或语音播报“第一隔层中存入青岛啤酒”，直观地给用户呈现存取食材的信息。
说明一点，冰箱一般可以包括：冷藏室和冷冻室，冷藏室可以用于对食材进行保鲜储藏，所以对食材的保鲜程度要求高，且根据食材自身特点，即使保存在冷藏室，出现食材腐烂的时间存在不同(也即：如绿色蔬菜保存时间较短，容易出现腐烂，而水果如苹果的保存时间相对长一点)，所以冷藏室对于食材的管理就显得非常必要；需要指出的是，在冷藏室中，除了可以包括隔层之外，可能还包括抽屉，所以在每个抽屉中同样可以设置扫码器和测距传感器。对于冷冻室而言，用于对食材进行冷冻储藏，即使保存较长时间也可以不会出现腐烂的现象，所以冷冻室对于食材管理的必要性要明显逊于冷藏室。
基于此，本申请实施例中提及的存取动作的确定、食材的种类的确定、以及食材的存放位置的确定等对于食材的管理，可以仅适用于冷藏室(如图15中所示)，或适用于冷藏室和冷冻室。
当然，在适用于冷冻室时，需要考虑扫码器和测距传感器在低温环境下的工作情况，在低温环境下无法正常工作时，可以考虑对扫码器和测距传感器进行改进，以使得扫码器和测距传感器可以适应低温环境。
下面结合具体实施例，对本申请实施例提供的上述冰箱的工作过程进行描述。
结合图15所述的结构，以及图19所述的流程图；并且以下过程中的执行主体为控制器。
S1901、在系统初始化完成后，判断是否接收到测距传感器发送的第二时间；若是，则执行S1902；若否，则继续执行本步骤；
具体地，系统初始化可以包括：处理程序的初始化、扫码器的初始化、以及测距传感器的初始化；其中，扫码器的初始化可以包括：扫描模式、扫码间隔、定位灯条亮度、补光灯亮度等参数的设置；测距传感器的初始化可以包括：工作模式、发射功率等参数的设置。
S1902、判断在第二时间之前的预设时间内是否接收到扫码器发送的第一时间；若是，则执行S1903；若否，则执行S1905；
S1903、确定用户的存取动作为向冰箱中存入食材；
S1904、根据扫码器发送的识别码和标识，确定食材的种类和存放位置；结束流程；
S1905、判断在第二时间之后的预设时间内是否接收到扫码器发送的第一时间；若是， 则执行S1906；若否，则继续执行S1901；
其中，S1902和S1905中的预设时间可以设置为相同，也可以设置为不同，可以根据实际需要进行设置，以提高设计的灵活性，满足不同应用场景的需要。
S1906、确定用户的存取动作为从冰箱中取出食材；执行S1904。
需要强调的是，在用户扫码的过程中，无论是扫码存入还是扫码取出，用户的手均会靠近扫码器，不同的是靠近扫码器和挡住测距传感器的先后次序；
测距传感器在t时刻检测到被挡住后，可以判断在[t-△t,t]时间内，是否有扫码结果，如果有扫码结果，则可以认为[t-△t,t]时间内扫码的食材为用户当前存入冰箱中的食材；由于△t可以设置的较小，因此可以很好地过滤掉很多干扰信号，扫码存入食材的时间边界也容易区分，可以消除大多数的误判，提高存取动作识别的准确率；测距传感器检测到被遮挡，且[t-△t,t]时间内没有扫码结果，此时可以等待△t×n(其中n可以但不限于为1)，如果在[t,t+△t×n]时间内有扫码结果，则可以认为在[t,t+△t×n]时间内扫码的食材为用户从冰箱中取出的食材，如果在[t,t+△t×n]时间内无扫码结果，则可以忽略本次测距传感器被挡住的触发。
可选地，在本申请实施例中，冰箱除了可以包括上述内容中提及的结构之外，还可以包括其他可以用于实现冰箱功能的结构，在此不再详述。
基于同一发明构思，本申请实施例提供了一种冰箱的控制方法，在冰箱包括测距传感器时，该控制方法可以包括：确定获取到食材上的识别码的第一时间；确定获取到用户经过测距传感器的测量区域的第二时间；根据第一时间和第二时间，确定用户的存取动作；存取动作包括：将食材存入至冰箱、以及将食材从冰箱中取出。
可选地，在本申请实施例中，冰箱包括：冰箱门和箱体；扫码器和测距传感器均位于冰箱门包括的隔层中的同一层，和/或扫码器和测距传感器均位于箱体包括的隔层中的同一层；测距传感器位于扫码器靠近箱体的内部的一侧；此时，根据第一时间和第二时间，确定用户的存取动作，具体包括：在第一时间先于第二时间时，确定用户将食材存入至冰箱；在第一时间晚于第二时间时，确定用户将食材从冰箱中取出；其中，第一时间与第二时间之间的时间差小于预设时间。
可选地，在本申请实施例中，在冰箱包括多个隔层，至少部分隔层内设置有扫码器时，还包括：根据扫描到识别码的扫码器的标识，确定扫描到的识别码对应食材存放的对应隔层。
可选地，在本申请实施例中，还包括：根据扫码器扫描到的识别码、以及预设的识别码与食材种类的对应关系，确定识别码对应食材的种类。
在另一些实施例中，为了克服目前冰箱在确定食材的位置和名称时，通过人工录入的方式进行，效率较低的问题，本申请实施例提供一种冰箱。图20A为本申请实施例提供的一种冰箱的立体图，本申请实施例提供的冰箱的形状近似长方体，由提供存储空间的储藏室301和设置在储藏室301开口处的至少一个冰箱门320组成。储藏室301为具有开口的箱体，在图20A所示的示例中，储藏室301包括位于下方的冷冻室301A以及位于上方的冷藏室301B，冷冻室301A和冷藏室301B分别具有一个或者多个独立的存储空间。冷藏室301B中还包括用于存放食材的第一隔板311、第二隔板312……第N隔板31N，在图20A中，仅示例性的展示了两个隔板，需要知道的是，本申请对于隔板的数量不作具体限定。每一个隔板都包括用于存放食材的置物板和用于支撑该置物板的支撑梁。在一些实施例中，冷藏室301B通过可枢转地安装在冷藏室301B上的冰箱门320选择性地打开或者关闭。冰箱门320上还包括用于存放食物的第一置物架321、第二置物架322……第N置物架32N。在图20A中，仅示例性的展示了两个置物架，需要知道的是，本申请对于置物架的数量不作具体限定。
需要说明的是，本申请涉及的冰箱中还包括控制器和存储器，控制器和存储器并未在图20A中示出，本申请对于控制器和存储器的位置不作具体限定。本申请实施例提供的冰 箱中还部署有多个重力传感器，多个重力传感器部署在上述的置物架中。当然，重力传感器也可以部署在隔板中，例如，可以部署于隔板包括的置物板中或者部署在用于支撑置物板的支撑梁中。重力传感器可以作为一种辅助传感器。后续以重力传感器位于置物架和支撑梁中为例进行介绍。需要知道的是，冰箱中的多个重力传感器与控制器连接，重力传感器可以将获取的质量传输到控制器中。
参见图20B，示出了一种置物架的俯视图，其中包括部署于置物架中的多个重力传感器。可选地，部署于置物架中的多个重力传感器可以是均匀分布的，目的是使置物架受力均匀，使得一个置物架上的不同位置的重力传感器测量得到的数据基本一致。当然，重力传感器的部署方式除了均匀分布之外还可以采用其他的部署方式，例如按照设定的排序规则来部署重力传感器。需要说明的是，图20B仅作为一种示例，本申请对于重力传感器部署的位置和部署方式不作具体限定。
参见图20C示出了本申请实施例提供的一种隔板的俯视图，包括置物板、支撑梁和部署在支撑梁中的多个重力传感器。需要说明的是，图20C仅作为一种示例，本申请对于重力传感器的部署位置和部署方式不作具体限定。
图21为本申请实施例提供的一种冰箱的外观示意图，如图21所示，在冷藏室的冰箱门320上设有显示屏，显示屏可以嵌入到冰箱门中。在一些实施例中，显示屏的屏幕所在的平面与箱门外表面平齐。
在一些实施例中，冰箱还包括控制器，控制器与显示屏相连接，用于与服务器进行交互获取信息和数据，生成食材管理界面，输出显示信息给显示屏，显示屏接收控制器输出的显示信号，显示相应的图像信息，如食材管理界面等。
在一些实施例中，为了实现对食材的智能管理，使用射频识别(Radio Frequency Identification，RFID)标签对储藏室中的食材进行标识，具体地，可以预先获取RFID标签的标识信息，如RFID标签中写入的标签编码，建立RFID标签的标识信息与食材信息的对应关系并存储，实现RFID标签与食材信息的关联。其中，食材信息可以包括食材的种类、名称或者食材的图样等信息。在食材上或者食材外包装上贴RFID标签或者夹上RFID标签后将食材放入冰箱中，冰箱的冰箱门320中还可以设置有RFID天线，RFID天线向外发射电磁波信号并接收RFID标签返回的激活信号，并将激活信号传输到控制器中。RFID天线也可以作为一种辅助传感器。激活信号中携带着RFID标签的标识信息，控制器通过预先建立的RFID标签的标识信息与食材信息的唯一对应关系，即可确定RFID标签对应的食材信息，并通过将食材信息显示在显示屏中，使得用户通过显示屏获知冰箱中的食材。
可选地，冰箱门320上还可以包括用于存放未使用的RFID标签的RFID标签盒，RFID标签盒中还可以有内置的屏蔽层，用于屏蔽RFID天线。即当RFID标签在RFID标签盒中时，RFID天线无法检测到RFID标签。
现有技术中，在冰箱食材管理方面所采用的方法是：在每次存放或者取出食材时，人工录入食材名称和位置。这种方法不仅浪费时间，且操作步骤繁琐，用户体验差。
为了实现更加智能的食材管理，提升用户的体验。本申请提出了一种冰箱及冰箱的食材管理方法，在冰箱中部署多个重力传感器，用于获取食材的质量，通过获取的质量发生变化的重力传感器的位置确定冰箱中食材的位置以及根据质量的变化情况确定是存放食材还是取出食材。并且在冰箱的箱门上部署RFID天线，通过扫描食材上的RFID标签确定存放或者取出的食材的信息。不再需要人工录入食材信息，实现无感、高效率的冰箱食材管理。
本申请提出的冰箱包括多个部署于支撑梁和置物架中的重力传感器，以及部署于箱门上的用于扫描食材的RFID标签的RFID天线。
第一种场景中，冰箱的控制器在确定第一重力传感器传输的质量增加时，根据RFID天线检测到的第一食材的标签，确定第一食材存放于第一重力传感器的位置。
第二种场景中，冰箱的控制器在确定第一重力传感器传输的质量减少时，根据RFID 天线检测到的第二食材的标签，确定第二食材被从第一重力传感器的位置取出。
其中，第一重力传感器为至少一个隔板中的任一重力传感器，或者为至少一个置物架中的任一重力传感器。
在上述第一种场景中，控制器在确定第一重力传感器传输的质量增加时，就可以确定用户在第一重力传感器所在的位置存放了食材。然后可以进一步根据第一重力传感器部署的位置确定存放的食材的位置，比如，第一重力传感器位于上述图20A示出的第一置物架中，则可以确定用户将食材存放在了第一置物架中。再进一步地，控制器可以根据RFID天线检测到用户存放食材的RFID标签确定用户存放的食材为第一食材。例如第一食材为西红柿，则控制器可以确定用户的将西红柿放置于第一置物架中。
在上述第二种场景中，控制器在确定第一重力传感器传输的质量减少时，就可以确定用户从第一重力传感器所在的位置取出食材。然后控制器可以根据第一重力传感器部署的位置确定取出的食材所在的位置，进一步地，根据RFID天线检测到用户取出的食材的RFID标签确定用户从第一重力传感器所处的位置取出的食材为第二食材。
在一些实施例中，冰箱中的存储器可以存储有部署于冰箱中的各个重力传感器传输的质量。作为一种可选的方式，冰箱中的重力传感器可以每间隔设定的时长向控制器传输一次质量。例如，每间隔50ms，冰箱中的各个重力传感器向控制器传输质量。控制器可以根据本次接收到的第一重力传感器传输的质量大于存储器中存储的第一重力传感器上一次传输的质量，确定第一重力传感器的传输的质量增加。
同理，控制器可以根据本次接收到第一重力传感器传输的质量小于存储器中存储的第一重力传感器上一次传输的质量，确定第一重力传感器传输的质量减少。
在一些实施例中，本申请涉及的重力传感器可以采用压敏材料，其电阻会随着压力的变化而变化。例如，在重力传感器两端加12V的电压，承受10-20kg物体的压力时，通过重力传感器的电流约为10-20mA。在这种情况下，由于重力传感器输出的电流信号为小信号，所以可以对重力传感器输出的电流信号进行信号放大电路的处理，并在经过数模转换电路(Analog-to-Digital Converter，ADC)后将模拟信号转换为数字信号。进一步地，控制器可以根据接收到的电流以及根据存储的电流与质量的对应关系确定质量。可选地，重力传感器在进行传输质量时，可以采用modbus通信协议中的远程终端单元(Remote Terminal Unit，RTU)进行传输质量，在将质量传输到控制器之后，控制器可以根据modbus协议进行数据解析获取质量。作为一种举例，上述重力传感器将质量传输到控制器的过程也可以参见图22所示的流程图。
可选地，在冰箱中存放或者取出食材时，可能会存在一些不可控因素影响重力传感器采集的质量，例如，用户在放置食材时手部按压隔板(或者置物架)等因素。由于这些因素的影响，重力传感器采集的质量不准确，并且控制器也需要等待重力传感器传输的质量稳定之后，才可以根据该质量确定食材的存取动作以及食材的位置，使得确定食材存取动作以及食材位置的过程耗时较长，效率低下。基于此，本申请实施例提供的控制器在接收到重力传感器传输的质量之后还可以对接收到的质量进行滤波处理，以降低重力传感器自身的噪声，排除干扰，提升食材存取位置识别的实时性和准确性。例如，可以采用卡尔曼滤波算法对接收到的质量进行处理。当然，也可以采用其他的滤波算法，作为一种举例，可以通过如下公式(1)所示的滤波方程来获得滤波后的质量：
Figure PCTCN2021139239-appb-000002
其中，x k为滤波之后的质量，x k-1为上一次滤波后的质量，z k为重力传感器传输的质量，z k-1为上一次重力传感器传输的质量。
进一步地，在确定重力传感器的质量增加或者减少时，可以采用滤波后的质量来确定。在确定滤波后的质量后，可以将存储器所存储的该重力传感器的质量更新为该重力传感器的滤波后的质量。控制器可以根据本次确定的第一重力传感器的质量大于存储器中存储的第一重力传感器的质量，确定第一重力传感器的质量增加。
同理，控制器可以根据本次确定的第一重力传感器的质量小于存储器中存储的第一重力传感器的质量，确定第一重力传感器的质量减少。
作为一种举例，可以参见图23，示出了本申请实施例提供的一种滤波前和滤波后的质量曲线对比图。可以看出，通过卡尔曼滤波算法后有效地过滤掉了存放和取出食材过程中的误差量，可以更加准确、快速地判断存取动作。
进一步地，控制器在确定重力传感器传输的质量增加或者减少之后，还可以根据RFID天线检测到的食材的RFID标签确定用户存放(或者取出)的食材信息。可选地，控制器确定用户存放(或者取出)的食材信息可以根据如下三种方式确定：
在一种可能的方式中，控制器可以在冰箱的箱门开启时，控制位于箱门上的RFID天线向外发射电磁波信号，例如，控制器可以通过位于箱门上的传感器检测箱门的开启、关闭状态，并在箱门开启的情况下，控制RFID天线向外发射电磁波信号，在其他的情况下，RFID天线并不向外发射电磁波天线。进一步地，控制器根据通过RFID天线接收到的食材的RFID标签返回的激活信号确定检测到用户存放(或取出)的食材的标签，并根据存储的RFID标签与食材信息的对应关系，确定检测到的用户存放(或者取出)的食材的信息。
在另一种可能的方式中，RFID天线可以实时向外发射电磁波信号，并且只能接收距离该天线设定范围内的标签返回的激活信号。例如，设定距离可以为2cm，也就是说，只有食材的RFID标签与RFID天线的距离小于2cm时，RFID天线才能够检测到该RFID标签。
可选地，控制器可以在确定第一重力传感器传输的质量增加之前的第一预设时长内，通过位于箱门上的RFID天线检测到的食材的标签确定用户存放于第一重力传感器位置处食材的信息。同理，控制器可以在确定第一重力传感器传输的质量减少之后的第二预设时长内，通过RFID天线检测到的食材的标签确定用户从第一重力传感器位置处取出的食材的信息。
作为一种举例，控制器确定存/取食材的信息的过程还可以参见图24所示的流程图，具体包括：控制器通过控制RFID天线获取食材的RFID标签返回的激活信号。控制器具备用于解析RFID标签返回的通过RFID天线接收的激活信号的功能。示例性地，执行该解析RFID标签返回的激活信号功能可以由配置与控制器内部的RFID读写模块(或者称为RFID读写器)实现。RFID读写模块解析激活信号后得到对应的标签的标识信息。进一步地，控制器可以根据存储的RFID标签与食材信息的对应关系确定解析得到的标识信息对应的食材信息。
再进一步地，控制器在确定用户在冰箱中存放或者取出的食材的信息之后，可以进一步根据传输的质量发生变化的第一重力传感器的位置确定该存放或者取出的食材的位置。
在一种可能的方式中，控制器可以根据冰箱中各个重力传感器的位置对各个重力传感器进行编号，并且按照编号的顺序接收各个重力传感器传输的质量。例如，控制器可以将第一支撑梁中的重力传感器编号为一、第二支撑梁编号中的重力传感器为二、第N支撑梁编号中的重力传感器为N、第一置物架编号中的重力传感器为N+1……以此类推。当然还可以采用其他的编号方式，例如第一支撑梁中的第一个重力传感器编号为一、第一支撑梁中的第二个重力传感器编号为二……以此类推。本申请对于编号的方式不作具体限定。进一步地，在进行质量的传输时，可以按照编号的顺序进行传输，例如第一支撑梁中的重力传感器第一个传输质量、第二支撑梁中的重力传感器第二个传输质量……以此类推。控制器根据接收到的传输的质量发生变化的重力传感器传输质量的次序，确定传输的质量发生变化的重力传感器的位置。例如，控制器确定第二个接收到的重力传感器传输的质量减少，则可以确定用户从第二支撑梁的位置处取走了食材。
在另一种可能的方式中，多个重力传感器在向控制器传输质量时，可以通过不同的通道进行传输，例如，可以是每一个重力传感器对应一个传输通道或者每一个支撑梁(或者置物架)中的重力传感器对应一个通道。这样，控制器可以根据不同的通道接收到的质量 确定其对应的重力传感器的位置。例如，第一通道对应第一支撑梁中的重力传感器，控制器在确定通过第一通道接收到的质量减少时，可以确定用户从第一隔板上取走了食材。
在另一种可能的方式中，每个重力传感器在传输质量时，还可以在数据中添加标识，用于表征重力传感器所在的位置。需要说明的是，每一个重力传感器传输质量时添加的标识都是不相同的。控制器可以根据接收到的数据中的标识确定重力传感器的位置，进而确定食材的位置。
在一些实施例中，本申请涉及的冰箱还可以包含显示屏，控制器在确定了用户存放(或者取出)的食材的信息和食材的位置之后，还可以根据这些信息进行调整食材管理界面，并在显示屏中显示调整后的食材管理界面。例如，可以参见图25所示的显示界面。可选地，控制器还可以响应于用户的触控或者遥控的操作，在显示屏中显示调整后的食材管理界面。例如，参见图26中的(a)，显示界面中可以包括用于更新界面的刷新控件701，控制器响应于用户对于控件701的操作将调整后的食材管理界面渲染至显示屏中，参见图26中的(b)。需要说明的是，本申请对于控件701在食材管理界面中的位置不作具体限定，图26仅作为一种示例。
下面，为了进一步理解本申请提出的方案，结合具体的实施例进行介绍。参见图27，示出了本申请提出的冰箱的食材管理方法的流程图，具体包括：
2701，控制器在确定冰箱门开启时，控制RFID天线向外发射电磁波信号。
2702，控制器判断RFID天线是否接收到食材的RFID标签返回的激活信号。若是，则继续步骤2703。若不是，则继续步骤2705。
2703，控制器判断是否有重力传感器传输的质量增加。若增加，则控制器可以确定用户向冰箱中存入食材，继续步骤2704。若未增加，则返回步骤2702。
2704，控制器确定存入的食材的位置和食材信息。
具体地，控制器根据传输的质量增加的重力传感器的位置确定存入的食材的位置，并根据通过RFID天线接收到的RFID标签返回的激活信号确定存入的食材的信息。
可选地，控制器还可以根据确定的食材的位置和食材信息更新食材管理界面的内容，并将更新后的食材管理界面渲染至显示屏中。
2705，控制器判断是否有重力传感器传输的质量是否减少。若减少，则控制器可以确定用户从冰箱中取出食材，继续步骤2706。若未减少，控制器可以提醒用户进行扫描食材的RFID标签。
2706，判断控制器在预设时长内是否通过RFID天线检测到RFID标签。若是，则继续步骤2707。若不是，控制器可以提醒用户进行扫描食材的RFID标签。
2707，控制器确定取出的食材的位置和食材信息。
具体地，控制器根据传输的质量减少的重力传感器的位置确定取出的食材的位置，并根据通过RFID天线接收到的RFID标签确定取出的食材的信息。
可选地，控制器还可以根据确定的食材的位置和食材信息更新食材管理界面的内容，并将更新后的食材管理界面渲染至显示屏中。
基于与上述方法的同一构思，参见图28，本申请实施例提供了一种用于实现食材管理方法的装置900。装置900可以执行上述方法中的任一步骤，为了避免重复，此处不再详述。装置900包括：获取单元901、处理单元902、存储单元903和显示单元904。
获取单元901，用于获取所述隔板和所述置物架中的重力传感器传输的质量；处理单元902，确定第一重力传感器传输的质量增加时，根据位于所述箱门上的RFID天线检测到的第一食材的RFID标签，确定所述第一食材存放于所述第一重力传感器的位置处；或者，确定第一重力传感器传输的质量减少时，根据位于所述箱门上的RFID天线检测到的第二食材的RFID标签，确定所述第二食材被从所述第一重力传感器的位置处取出；其中，所述第一重力传感器为所述至少一个隔板中的任一重力传感器，或者为所述至少一个置物架中的任一重力传感器。
在一些实施例中，所述装置还包括存储单元903，用于存储所述第一重力传感器传输的质量，所述处理单元902，具体用于：在所述存储单元903中存储的所述第一重力传感器上一次传输的质量小于本次接收到的所述第一重力传感器传输的质量时，确定所述第一重力传感器传输的质量增加；或者，在所述存储单元903中存储的所述第一重力传感器上一次传输的质量大于本次接收到的所述第一重力传感器传输的质量时，确定所述第一重力传感器传输的质量减少。
在一些实施例中，所述处理单元902，具体用于：在所述箱门开启时，控制所述RFID天线向外发射电磁波信号，并根据所述RFID天线接收的所述第一食材的RFID标签返回的激活信号，确定检测到所述第一食材的标签，或者根据通过所述RFID天线接收的所述第二食材的RFID标签返回的激活信号，确定检测到所述第二食材的标签。
在一些实施例中，所述处理单元902，具体用于：在确定第一重力传感器传输的质量增加之前的第一预设时长内，通过所述RFID天线检测到的第一食材的RFID标签，确定所述第一食材存放于所述第一重力传感器的位置处；或者，在确定第一重力传感器传输的质量减少之后的第二预设时长内，通过所述RFID检测到的第二食材的RFID标签，则确定所述第二食材被从所述第一重力传感器的位置处取出。
在一些实施例中，所述处理单元902，具体用于根据所述第一重力传感器传输的质量的接收顺序，确定所述第一重力传感器的位置；其中，所述冰箱中不同的位置的重力传感器传输的质量的接收顺序不同。
在一些实施例中，所述装置还包括：显示单元904，用于在所述隔板或者所述置物架上存放的食材或者从所述隔板或者所述置物架上取出食材时，显示更新后的食材管理界面，所述食材管理界面用于展示所述隔板和所述置物架上存放的食材的RFID标签对应的食材信息和食材的位置。
基于同一发明构思，本申请实施例提供一种存储介质，当所述存储介质中的指令由处理器执行时，使得所述处理器能够执行上述流程中冰箱实现的任一项方法。

Claims (29)

  1. 一种冰箱，其特征在于，包括：
    机壳，所述机壳包括储藏室，所述储藏室具有开口；
    冰箱门，与所述机壳活动连接，用于遮挡所述开口；
    图像采集装置，设于所述机壳顶部，用于采集所述冰箱门的活动区域的图像；
    辅助传感器，设于所述储藏室内，用于采集所述储藏室内的存储区域的测量信息；
    控制器，被配置为：
    响应于所述冰箱门开启，获取所述图像采集装置采集的图像，以及获取所述辅助传感器采集的测量信息；
    若确定所述图像中不存在食材存取操作，则基于所述测量信息，判断是否存在食材存取操作。
  2. 如权利要求1所述的冰箱，其特征在于，所述储藏室内包含多层储藏空间，所述辅助传感器设置于顶层储藏空间中靠近所述冰箱门的一侧。
  3. 如权利要求1或2所述的方法，其特征在于，所述控制器具体用于：
    若所述辅助传感器为单点飞行时间TOF传感器，且所述测量信息中包含的所述辅助传感器与遮挡物之间的距离小于或等于储藏室宽度，则确定存在食材存取操作；或者，
    若所述辅助传感器为红外对射光电传感器，且所述测量信息中包含的光线强度小于预设的光强门限值，则确定存在食材存取操作；或者，
    若所述辅助传感器为红外温度传感器，则根据所述测量信息中包含的温度信息，确定存在食材存取操作。
  4. 一种冰箱，其特征在于，包括：
    机壳，所述机壳包括储藏室，所述储藏室具有开口，所述储藏室内包含多层储藏空间，每层储藏空间内均设有至少一个辅助传感器，每个辅助传感器均用于采集测量信息，并将所述测量信息发送到控制器；
    冰箱门，与所述机壳活动连接，用于遮挡所述开口；
    控制器，被配置为：
    响应于所述冰箱门开启，获取各个辅助传感器采集的测量信息；
    基于所述测量信息，判断是否存在食材存取操作。
  5. 如权利要求4所述的冰箱，其特征在于，每层储藏空间内均设有一个辅助传感器，所述辅助传感器为单点TOF传感器、红外温度传感器、红外对射光电传感器中的任一项。
  6. 如权利要求4所述的冰箱，其特征在于，每层储藏空间内均设有多个辅助传感器，所述多个辅助传感器为红外对射光电传感器，所述红外对射光电传感器包含发射部件和相应的接收部件。
  7. 如权利要求6所述的冰箱，其特征在于：
    在所述储藏室的一侧，每层储藏空间中设有所述多个辅助传感器的发射部件，在所述储藏室的另一侧，设有相应的接收部件；或者，
    在所述储藏室的一侧，每层储藏空间中交替设有所述多个辅助传感器的发射部件和接收部件，在所述储藏室的另一侧，设有相应的接收部件和发射部件；或者，
    在所述储藏室的一侧，一层储藏空间设有所述多个辅助传感器的发射部件，且相邻层储藏空间内设有辅助传感器的接收部件，在所述储藏室的另一侧，设有相应的接收部件和发射部件。
  8. 如权利要求4所述的冰箱，其特征在于，所述辅助传感器包括扫码器和测距传感器；所述控制器具体用于：
    确定所述扫码器扫描食材上的识别码的第一时间，确定所述测距传感器测量到用户经过所述测距传感器的测量区域时的第二时间，根据所述第一时间和所述第二时间，确定所 述用户的存取动作；所述存取动作包括：将所述食材存入至所述冰箱、以及将所述食材从所述冰箱中取出。
  9. 如权利要求4所述的冰箱，其特征在于，所述辅助传感器包括设置在冰箱储藏室内的隔板中的至少一个重力传感器、冰箱门上的置物架中的至少一个重力传感器和位于冰箱门上的RFID天线；所述控制器具体用于：
    获取所述隔板和所述置物架中的重力传感器传输的质量；确定第一重力传感器传输的质量增加时，根据位于所述冰箱门上的RFID天线检测到的第一食材的RFID标签，确定所述第一食材存放于所述第一重力传感器的位置处；或者，确定第一重力传感器传输的质量减少时，根据位于所述冰箱门上的RFID天线检测到的第二食材的RFID标签，确定所述第二食材被从所述第一重力传感器的位置处取出；其中，所述第一重力传感器为所述至少一个隔板中的任一重力传感器，或者为所述至少一个置物架中的任一重力传感器。
  10. 一种冰箱，其特征在于，包括：
    机壳，所述机壳包括储藏室，所述储藏室具有开口；
    冰箱门，与所述机壳活动连接，用于遮挡所述开口；
    图像采集装置，设于所述机壳顶部，用于采集所述冰箱门的活动区域的图像；
    辅助传感器，设于所述储藏室内，用于采集所述储藏室内的存储区域的测量信息，所述测量信息用于在冰箱门开启，且所述图像采集装置采集的所述图像中不存在食材存取操作时，判断是否存在食材存取操作。
  11. 一种冰箱，其特征在于，包括：
    机壳，所述机壳包括储藏室，所述储藏室具有开口；
    冰箱门，与所述机壳活动连接，用于遮挡所述开口；
    辅助传感器，设于所述储藏室内，用于采集所述储藏室内的存储区域的测量信息，所述测量信息用于在冰箱门开启时，判断是否存在食材存取操作。
  12. 一种冰箱，其特征在于，包括：
    辅助传感器，所述辅助传感器包括扫码器和测距传感器；
    控制器，被配置为：
    确定所述扫码器扫描食材上的识别码的第一时间；
    确定所述测距传感器测量到用户经过所述测距传感器的测量区域时的第二时间；
    根据所述第一时间和所述第二时间，确定所述用户的存取动作；所述存取动作包括：将所述食材存入至所述冰箱、以及将所述食材从所述冰箱中取出。
  13. 如权利要求12所述的冰箱，其特征在于，所述冰箱包括：冰箱门和箱体；
    所述扫码器和所述测距传感器均位于所述冰箱门包括的隔层中的同一层，和/或所述扫码器和所述测距传感器均位于所述箱体包括的隔层中的同一层；
    所述测距传感器位于所述扫码器靠近所述箱体的内部的一侧。
  14. 如权利要求13所述的冰箱，其特征在于，所述扫码器与所述测距传感器之间的距离小于预设值。
  15. 如权利要求13所述的冰箱，其特征在于，所述控制器被配置为：
    在所述第一时间先于所述第二时间时，确定所述用户将所述食材存入至所述冰箱；
    在所述第一时间晚于所述第二时间时，确定所述用户将所述食材从所述冰箱中取出；
    其中，所述第一时间与所述第二时间之间的时间差小于预设时间。
  16. 如权利要求12所述的冰箱，其特征在于，所述冰箱包括多个隔层，至少部分所述隔层内设置有所述扫码器；
    所述控制器被配置为：
    根据扫描到识别码的所述扫码器的标识，确定扫描到的识别码对应食材存放的对应所述隔层。
  17. 如权利要求13-16任一项所述的冰箱，其特征在于，每个所述隔层中均设置有所 述扫码器和所述测距传感器。
  18. 如权利要求12所述的冰箱，其特征在于，所述控制器被配置为：
    根据所述扫码器扫描到的识别码、以及预设的识别码与食材种类的对应关系，确定所述识别码对应食材的种类。
  19. 如权利要求12所述的冰箱，其特征在于，所述测距传感器包括单点飞行时间传感器。
  20. 如权利要求12所述的冰箱，其特征在于，还包括：显示器和/或播放器；
    所述显示器，被配置为：显示所述控制器确定出的信息；
    所述播放器，被配置为：播放所述控制器确定出的信息。
  21. 如权利要求12所述的冰箱，其特征在于，所述识别码包括：条形码和/或二维码。
  22. 一种冰箱，其特征在于，包括：
    箱体，内部设有储藏室，所述储藏室中包括至少一个用于存放食材的隔板，所述隔板存放的食材通过射频识别RFID标签进行标识；所述隔板中包括至少一个重力传感器，用于获取存放在所述隔板上的食材的质量；
    冰箱门，设于所述储藏室的开口处，所述冰箱门靠近所述储藏室的一侧上包括至少一个用于存放食材的置物架，所述置物架存放的食材通过RFID标签进行标识；所述置物架中包括至少一个重力传感器，用于获取存放在所述置物架中的食材的质量；所述冰箱门靠近所述储藏室的一侧上包括RFID天线，用于扫描食材的RFID标签；
    控制器，被配置为执行：
    确定第一重力传感器传输的质量增加时，根据所述RFID天线检测到的第一食材的RFID标签，确定所述第一食材存放于所述第一重力传感器的位置处；或者，
    确定第一重力传感器传输的质量减少时，根据所述RFID天线检测到的第二食材的RFID标签，确定所述第二食材被从所述第一重力传感器的位置处取出；
    其中，所述第一重力传感器为所述至少一个隔板中的任一重力传感器，或者为所述至少一个置物架中的任一重力传感器。
  23. 如权利要求22所述的冰箱，其特征在于，所述冰箱还包括存储器，用于存储所述第一重力传感器传输的质量，所述控制器，具体用于：
    在所述存储器中存储的所述第一重力传感器上一次传输的质量小于本次接收到的所述第一重力传感器传输的质量时，确定所述第一重力传感器传输的质量增加；或者，
    在所述存储器中存储的所述第一重力传感器上一次传输的质量大于本次接收到的所述第一重力传感器传输的质量时，确定所述第一重力传感器传输的质量减少。
  24. 如权利要求22或23所述的冰箱，其特征在于，所述控制器，具体用于：
    在所述冰箱门开启时，控制所述RFID天线向外发射电磁波信号，并根据所述RFID天线接收的所述第一食材的RFID标签返回的激活信号，确定检测到所述第一食材的标签，或者根据通过所述RFID天线接收的所述第二食材的RFID标签返回的激活信号，确定检测到所述第二食材的标签。
  25. 如权利要求22或23所述的冰箱，其特征在于，所述控制器，具体用于：
    在确定第一重力传感器传输的质量增加之前的第一预设时长内，通过所述RFID天线检测到的第一食材的RFID标签，确定所述第一食材存放于所述第一重力传感器的位置处；或者，
    在确定第一重力传感器传输的质量减少之后的第二预设时长内，通过所述RFID检测到的第二食材的RFID标签，则确定所述第二食材被从所述第一重力传感器的位置处取出。
  26. 如权利要求22或23所述的冰箱，其特征在于，所述控制器，具体用于根据所述第一重力传感器传输的质量的接收顺序，确定所述第一重力传感器的位置；
    其中，所述冰箱中不同的位置的重力传感器传输的质量的接收顺序不同。
  27. 如权利要求22所述的冰箱，其特征在于，所述冰箱还包括：
    显示屏，设于所述冰箱门背离所述储藏室的一侧上，用于在所述隔板或者所述置物架上存放或者取出食材时，显示更新后的食材管理界面，所述食材管理界面用于展示所述隔板和所述置物架上存放的食材的RFID标签对应的食材信息和食材的位置。
  28. 一种食材管理方法，其特征在于，包括：
    响应于冰箱门开启，获取图像采集装置采集的冰箱门活动区域的图像，以及获取辅助传感器采集的储藏室内的存储区域的测量信息，其中，所述图像采集装置设置于机壳顶部，辅助传感器设置于储藏室内；
    若确定所述图像中不存在食材存取操作，则基于所述测量信息，判断是否存在食材存取操作。
  29. 一种食材管理方法，其特征在于，应用于冰箱，所述冰箱的储藏室包括至少一个用于存放食材的隔板，所述隔板中包括至少一个重力传感器；所述冰箱的冰箱门包括至少一个置物架，所述置物架包括至少一个重力传感器；
    所述方法包括：
    获取所述隔板和所述置物架中的重力传感器传输的质量；
    确定第一重力传感器传输的质量增加时，根据位于所述冰箱门上的RFID天线检测到的第一食材的RFID标签，确定所述第一食材存放于所述第一重力传感器的位置处；或者，
    确定第一重力传感器传输的质量减少时，根据位于所述冰箱门上的RFID天线检测到的第二食材的RFID标签，确定所述第二食材被从所述第一重力传感器的位置处取出；
    其中，所述第一重力传感器为所述至少一个隔板中的任一重力传感器，或者为所述至少一个置物架中的任一重力传感器。
PCT/CN2021/139239 2021-06-18 2021-12-17 一种冰箱及其食材管理方法及装置 WO2022262234A1 (zh)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
CN202110676948.9A CN115493343B (zh) 2021-06-18 2021-06-18 一种冰箱及其食材管理方法及装置
CN202110676948.9 2021-06-18
CN202110788223.9A CN115615093A (zh) 2021-07-13 2021-07-13 一种冰箱及冰箱的食材管理方法
CN202110788223.9 2021-07-13
CN202111037567.2 2021-09-06
CN202111037567.2A CN115773608A (zh) 2021-09-06 2021-09-06 一种智能冰箱

Publications (1)

Publication Number Publication Date
WO2022262234A1 true WO2022262234A1 (zh) 2022-12-22

Family

ID=84526806

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/139239 WO2022262234A1 (zh) 2021-06-18 2021-12-17 一种冰箱及其食材管理方法及装置

Country Status (1)

Country Link
WO (1) WO2022262234A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20240035737A1 (en) * 2022-07-28 2024-02-01 Haier Us Appliance Solutions, Inc. Smart adjustable shelves for refrigerator appliances

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150294451A1 (en) * 2012-01-13 2015-10-15 Lg Electronics Inc. Method for controlling operation of refrigerator by using speech recognition, and refrigerator employing same
CN105444515A (zh) * 2015-12-11 2016-03-30 四川长虹电器股份有限公司 一种冰箱
CN106288591A (zh) * 2016-11-08 2017-01-04 朱恺 一种具有食材管理功能的冰箱及食材管理方法
CN107763955A (zh) * 2016-08-23 2018-03-06 海信（山东）冰箱有限公司 一种冰箱及其控制方法
CN109114877A (zh) * 2018-08-13 2019-01-01 长虹美菱股份有限公司 一种基于冰箱动态识别的食品管理系统
CN111023690A (zh) * 2019-11-20 2020-04-17 芜湖乐哈哈信息科技有限公司 一种基于智能冰箱的食物识别系统
CN111442604A (zh) * 2020-04-10 2020-07-24 海信集团有限公司 一种冰箱及其控制方法
CN111462190A (zh) * 2020-04-20 2020-07-28 海信集团有限公司 一种智能冰箱及食材录入方法

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150294451A1 (en) * 2012-01-13 2015-10-15 Lg Electronics Inc. Method for controlling operation of refrigerator by using speech recognition, and refrigerator employing same
CN105444515A (zh) * 2015-12-11 2016-03-30 四川长虹电器股份有限公司 一种冰箱
CN107763955A (zh) * 2016-08-23 2018-03-06 海信（山东）冰箱有限公司 一种冰箱及其控制方法
CN106288591A (zh) * 2016-11-08 2017-01-04 朱恺 一种具有食材管理功能的冰箱及食材管理方法
CN109114877A (zh) * 2018-08-13 2019-01-01 长虹美菱股份有限公司 一种基于冰箱动态识别的食品管理系统
CN111023690A (zh) * 2019-11-20 2020-04-17 芜湖乐哈哈信息科技有限公司 一种基于智能冰箱的食物识别系统
CN111442604A (zh) * 2020-04-10 2020-07-24 海信集团有限公司 一种冰箱及其控制方法
CN111462190A (zh) * 2020-04-20 2020-07-28 海信集团有限公司 一种智能冰箱及食材录入方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20240035737A1 (en) * 2022-07-28 2024-02-01 Haier Us Appliance Solutions, Inc. Smart adjustable shelves for refrigerator appliances

Similar Documents

Publication Publication Date Title
CN111442604B (zh) 一种冰箱及其控制方法
EP3008414B1 (de) Kältegerät mit einem kameramodul
JP6938116B2 (ja) 在庫管理装置および在庫管理方法
CN107144086B (zh) 基于食材的冰箱温度控制方法与计算机存储介质
CN106979659B (zh) 基于食材的冰箱温度控制方法与计算机存储介质
CN106958975B (zh) 基于食材的冰箱温度控制方法与计算机存储介质
CN105841436B (zh) 智能冰箱的食品管理方法及食品管理系统
CN106546063A (zh) 冷藏冷冻装置的食物信息管理方法及系统
WO2022262234A1 (zh) 一种冰箱及其食材管理方法及装置
CN111780477A (zh) 冰箱及食材管理方法
CN106642969A (zh) 冰箱的食物管理方法及冰箱
CN107192216A (zh) 基于食材的冰箱温度控制方法与计算机存储介质
CN106979647B (zh) 基于食材的冰箱温度控制方法与计算机存储介质
CN111811182B (zh) 一种冰箱及食谱推荐方法
CN110017659A (zh) 家用制冷器具设备
KR102473583B1 (ko) 냉장고 및 그 제어 방법
KR102017980B1 (ko) 인공지능을 이용하여 제품을 식별하고 구분된 이미지를 출력하는 냉장고 및 방법
CN112417282B (zh) 信息推荐方法、食材储藏设备及服务器
WO2014198632A1 (de) Kältegerät mit einem kameramodul
CN112199406A (zh) 信息推荐方法、食材存储设备及服务器
CN113915887A (zh) 一种冰箱及其食材定位方法
JP7113281B2 (ja) 冷蔵庫
CN115493343B (zh) 一种冰箱及其食材管理方法及装置
CN113932530B (zh) 一种冰箱及冰箱的控制方法
CN113465266B (zh) 一种食材存储管理方法、智能冰箱及服务器

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21945810

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21945810

Country of ref document: EP

Kind code of ref document: A1