WO2023131202A1 - 带有智能抽屉的制冷电器 - Google Patents

带有智能抽屉的制冷电器 Download PDF

Info

Publication number
WO2023131202A1
WO2023131202A1 PCT/CN2023/070499 CN2023070499W WO2023131202A1 WO 2023131202 A1 WO2023131202 A1 WO 2023131202A1 CN 2023070499 W CN2023070499 W CN 2023070499W WO 2023131202 A1 WO2023131202 A1 WO 2023131202A1
Authority
WO
WIPO (PCT)
Prior art keywords
drawer
food
food product
storage compartment
food storage
Prior art date
Application number
PCT/CN2023/070499
Other languages
English (en)
French (fr)
Inventor
罗伯特 伊登凯尔
斯科特 约翰逊埃里克
古德曼 施罗德迈克尔
Original Assignee
海尔智家股份有限公司
青岛海尔电冰箱有限公司
海尔美国电器解决方案有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 海尔智家股份有限公司, 青岛海尔电冰箱有限公司, 海尔美国电器解决方案有限公司 filed Critical 海尔智家股份有限公司
Publication of WO2023131202A1 publication Critical patent/WO2023131202A1/zh

Links

Images

Classifications

    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
    • F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
    • F25D17/00—Arrangements for circulating cooling fluids; Arrangements for circulating gas, e.g. air, within refrigerated spaces
    • F25D17/04—Arrangements for circulating cooling fluids; Arrangements for circulating gas, e.g. air, within refrigerated spaces for circulating air, e.g. by convection
    • F25D17/042—Air treating means within refrigerated spaces
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
    • F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
    • F25D29/00—Arrangement or mounting of control or safety devices
    • F25D29/005—Mounting of control devices
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
    • F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
    • F25D17/00—Arrangements for circulating cooling fluids; Arrangements for circulating gas, e.g. air, within refrigerated spaces
    • F25D17/04—Arrangements for circulating cooling fluids; Arrangements for circulating gas, e.g. air, within refrigerated spaces for circulating air, e.g. by convection
    • F25D17/042—Air treating means within refrigerated spaces
    • F25D17/045—Air flow control arrangements
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
    • F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
    • F25D25/00—Charging, supporting, and discharging the articles to be cooled
    • F25D25/02—Charging, supporting, and discharging the articles to be cooled by shelves
    • F25D25/024—Slidable shelves
    • F25D25/025—Drawers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/68—Food, e.g. fruit or vegetables
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/18—Status alarms
    • G08B21/182—Level alarms, e.g. alarms responsive to variables exceeding a threshold
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
    • F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
    • F25D2700/00—Means for sensing or measuring; Sensors therefor
    • F25D2700/06—Sensors detecting the presence of a product

Definitions

  • the present invention relates generally to refrigeration appliances, and more particularly to systems and methods for managing the status of items, such as produce, stored in such refrigeration appliances.
  • Refrigerated appliances typically include a cabinet with a freezer compartment. A wide variety of foods can be stored in the freezer. The lower temperature of the freezer relative to the ambient atmosphere can extend the shelf life of foods stored in the freezer.
  • Agricultural products eg, fruits and vegetables stored in refrigeration appliances undergo various physical and chemical changes over time, such as ripening.
  • Various agricultural products may be incompatible with each other, such as may have different storage requirements.
  • the optimal temperature, humidity and/or atmospheric composition may be different for one agricultural product than for another.
  • Different produce may also be incompatible when stored together because during storage, each produce changes differently over time, such as one produce may produce certain atmospheric chemicals as it matures that are harmful to the other produce.
  • a cooling appliance with a revamped inventory management system should be useful. More specifically, a refrigeration appliance including a produce inventory management system capable of monitoring produce inventory and tracking the status of such items during storage would be useful.
  • the method also includes detecting, using the sensor, an atmospheric condition within the food storage compartment above a predetermined threshold, and based on the analysis of the images, identifying one of the first food product and the second food product as causing the atmospheric condition to be above the predetermined threshold. source.
  • a method of operating a refrigeration appliance includes a cabinet defining a food storage compartment with a drawer slidably mounted within the food storage compartment. The drawer slides between a closed position and an open position. The drawer includes a plurality of walls defining food storage compartments.
  • the refrigeration appliance also includes a sensor operable to detect atmospheric conditions within the food storage compartment of the drawer and a camera assembly positioned and configured to monitor the drawer.
  • the method includes acquiring images using a camera assembly and analyzing the images to identify a first food product and a second food product in a food storage compartment of a drawer.
  • the method also includes setting a first threshold for atmospheric conditions based on identifying the first food product and setting a second threshold for atmospheric conditions based on identifying the second food product.
  • the method also includes monitoring, by the sensor, atmospheric conditions.
  • the method also includes issuing a first user notification when the atmospheric condition reaches a first threshold and issuing a second user notification when the atmospheric condition reaches a second threshold.
  • a method of operating a refrigeration appliance includes a cabinet defining a food storage compartment with a drawer slidably mounted within the food storage compartment. The drawer slides between a closed position and an open position. The drawer includes a plurality of walls defining food storage compartments.
  • the refrigeration appliance also includes a sensor operable to detect atmospheric conditions within the food storage compartment of the drawer and a camera assembly configured to monitor the drawer.
  • the method includes acquiring images using a camera assembly and analyzing the images to identify a first food product and a second food product in a food storage compartment of a drawer. The method also includes determining that the first food product and the second food product are incompatible for co-storage and issuing a user notification including a suggestion to relocate one of the first food product and the second food product.
  • FIG. 1 provides a front view of a refrigeration appliance according to an exemplary embodiment of the present invention.
  • FIG. 2 provides a perspective view of the refrigeration appliance of FIG. 1 .
  • FIG. 3 provides a front view of the refrigeration appliance of FIG. 1 with the door of the refrigeration appliance in an open position.
  • Figure 4 provides a front view of a portion of a refrigeration appliance according to one or more exemplary embodiments of the present invention.
  • FIG. 5 provides a cross-sectional view of a portion of the refrigeration appliance of FIG. 4 .
  • FIG. 6 illustrates an exemplary image of a storage drawer and the contents of the drawer in an open position of a refrigeration appliance, such as the exemplary refrigeration appliance of FIG. 1 , that may be captured by a camera assembly in the refrigeration appliance.
  • Figure 7 provides a perspective view of a drawer that may be incorporated into a refrigeration appliance in one or more exemplary embodiments of the present invention.
  • FIG. 8 provides an enlarged view of a portion of the drawer of FIG. 7 .
  • FIG. 9 provides a flowchart of an example method for operating a refrigeration appliance according to one or more example embodiments of the present disclosure.
  • FIG. 10 provides a flowchart of another exemplary method for operating a refrigeration appliance according to one or more additional exemplary embodiments of the present invention.
  • FIG. 11 provides a flowchart of yet another exemplary method for operating a refrigeration appliance according to one or more additional exemplary embodiments of the present invention.
  • FIG. 1 is a front view of an exemplary embodiment of a refrigeration appliance 100 .
  • FIG. 2 is a perspective view of the refrigeration appliance 100 .
  • FIG. 3 is a front view of the refrigeration appliance 100 with the food preservation door 128 in an open position.
  • the refrigeration appliance 100 extends along a vertical direction V between a top 101 and a bottom 102 .
  • the cooling appliance 100 also extends in a lateral direction L between a first side 105 and a second side 106 .
  • the lateral direction T may be perpendicular to the vertical direction V and the lateral direction L.
  • the refrigeration appliance 100 extends in a transverse direction T between a front portion 108 and a rear portion 110 .
  • the refrigeration appliance 100 includes a cabinet or housing 120 having an upper fresh food compartment 122 ( FIG. 3 ) and a lower freezer compartment or frozen food storage compartment 124 arranged in a vertical direction V below the fresh food compartment 122 .
  • an auxiliary food storage compartment (not shown) may be disposed (eg, along the vertical direction V) between the fresh food storage compartment 122 and the frozen food storage compartment 124 . Because the frozen food storage compartment 124 is disposed below the fresh food storage compartment 122, the refrigeration appliance 100 is often referred to as a bottom-mounted refrigerator.
  • housing 120 also has a mechanical chamber (not shown) for housing a sealed cooling system (not shown).
  • the refrigerator doors 128 are each rotatably hinged to the edge of the casing 120 for opening and closing the fresh food compartment 122 . It should be noted that while two doors 128 in a "French door" configuration are illustrated, any suitable door arrangement utilizing one, two or more doors is within the scope and spirit of the invention.
  • the freezer door 130 is disposed under the refrigerator door 128 for opening and closing the freezer compartment 124 .
  • freezer door 130 is coupled to a freezer drawer (not shown) slidably mounted within freezer compartment 124 .
  • the auxiliary door 127 may be coupled to an auxiliary drawer (not shown) slidably mounted within an auxiliary compartment (not shown).
  • Operation of the refrigeration appliance 100 may be regulated by a controller 134 operatively coupled to a user interface panel 136 .
  • the user interface panel 136 provides options, such as temperature options, for the user to manipulate the operation of the refrigeration appliance 100 to modify the environmental conditions therein.
  • the user interface panel 136 can be adjacent to the dispenser assembly.
  • Panel 136 provides options for the user to manipulate the operation of refrigeration appliance 100, such as temperature options, options for automatic or manual override humidity control (as described in more detail below), and the like.
  • the controller 134 controls the operation of various components of the refrigeration appliance 100.
  • Operation of the cooling appliance 100 may be regulated by the controller 134 , which may adjust the operation of various components of the cooling appliance 100 in response to programming and/or user manipulation of the user interface panel 136 , for example.
  • Controller 134 may include memory and one or more microprocessors, CPUs, etc., such as general purpose or special purpose microprocessors operable to execute programmed instructions or micro control codes associated with operating refrigeration appliance 100 .
  • the memory may represent random access memory such as DRAM or read only memory such as ROM or FLASH.
  • a processor executes programmed instructions stored in memory.
  • the memory can be a separate component from the processor, or it can be on-board within the processor. It should be noted that the controller 134 as disclosed herein is capable and operable to perform any method and associated method steps as disclosed herein.
  • the controller 134 can be positioned at various positions throughout the refrigeration appliance 100 .
  • the controller 134 may be located within the door 128 .
  • input/output (“I/O”) signals may be transmitted between the controller and the various operating components of refrigeration appliance 100 .
  • the user interface panel 136 may represent a general purpose I/O ("GPIO") device or functional block.
  • the user interface 136 may include input components such as one or more of various electrical, mechanical, or electromechanical input devices including rotary dials, buttons, and touch pads.
  • User interface 136 may include display components such as digital or analog display devices designed to provide operational feedback to the user.
  • user interface 136 may include a touch screen that provides both input and display functions.
  • the user interface 136 may communicate with the controller via one or more signal lines or a shared communication bus.
  • a plurality of food storage elements are disposed within the food preservation storage compartment 122 .
  • the drawer 140 may be configured for storing produce such as fruits and vegetables, and in particular the refrigeration appliance may be operable to improve and be configured for improving the shelf life of the produce stored therein.
  • the drawer 140 may also be referred to as a produce drawer 140 or a vegetable drawer 140 and a fruit drawer 140 .
  • a refrigeration appliance may include two drawers 140, eg, as shown in FIG. 3 .
  • the refrigeration appliance 100 may also include an inventory management system generally configured to monitor one or more compartments of the refrigeration appliance 100 to monitor the status of the inventory stored therein. More specifically, as detailed below, an inventory management system may include one or more sniffers or sensors 200 (see, e.g., FIGS. 4 and 5 ), cameras 192 (see, e.g., FIGS. Other detection devices for monitoring the fresh food compartment 122 and, in particular, the drawers 140 to detect and monitor objects in or out of the drawers 140 (eg, generally identified by reference numeral 182 in FIG. 6 ).
  • sniffers or sensors 200 see, e.g., FIGS. 4 and 5
  • cameras 192 see, e.g., FIGS.
  • Other detection devices for monitoring the fresh food compartment 122 and, in particular, the drawers 140 to detect and monitor objects in or out of the drawers 140 eg, generally identified by reference numeral 182 in FIG. 6 ).
  • the inventory management system may use data from each of these devices to obtain the identity, location, and/or other qualitative characteristics of objects 182 (e.g., produce such as fruits and/or vegetables) within drawers 140. Representation or knowledge of a characteristic or quantitative property. While the inventory management system is described herein as monitoring drawers 140 for detecting objects 182, it should be understood that aspects of the invention may be used to monitor objects or items in any other suitable appliance, compartment, or the like.
  • objects 182 e.g., produce such as fruits and/or vegetables
  • the inventory management system may include a camera assembly 190 generally positioned and configured to capture images of refrigeration appliance 100 during operation.
  • camera assembly 190 includes one or more cameras 192 mounted to cabinet 120 , door 128 or otherwise positioned in view of food preservation compartment 122 .
  • camera 192 of camera assembly 190 is mounted to cabinet 120 at the front opening of food preservation compartment 122 and is oriented to have a field of view 194 that covers the front opening and/or enters the food preservation compartment.
  • Chamber 122 and particularly into drawer 140 extend forward, such as generally in transverse direction T such that one or both of drawers 140 extend beyond between camera assembly 190 and drawer 140 .
  • the front edge (front) of the shelf 142 or shelves 142 such as above the drawer 140 .
  • each camera 192 is shown in FIG. specific monitoring area or range.
  • the field of view 194 of each camera 192 may be limited or focused on a particular area within the food preservation compartment 122 , such as one camera 192 per drawer 140 .
  • each camera 192 may be positioned adjacent the front opening of the fresh food compartment 122 and to orient each camera 190 such that the field of view 194 is directed into the fresh food compartment 112 .
  • camera assembly 190 may be used to improve the inventory management process of refrigeration appliance 100 .
  • each camera 192 may be positioned at an opening into the food preservation compartment 122 to monitor food (generally identified as object 182) being added to or removed from the food preservation compartment 122, particularly being placed in or removed from a drawer. 140 for food.
  • the camera assembly 190 may include any suitable number, type, size, and configuration of cameras 192 for capturing images of any suitable area in or around the cooling appliance 100 according to alternative embodiments. Additionally, it should be understood that each camera 192 may include features for adjusting the field of view and/or orientation.
  • the images captured by the camera assembly 190 may vary in number, frequency, angle, resolution, detail, etc., in order to enhance the clarity of certain areas around or within the refrigeration appliance 100 .
  • the controller 134 may be configured to illuminate the freezing compartment with one or more light sources prior to acquiring an image.
  • controller 134 (or any other suitable dedicated controller) of refrigeration appliance 100 is communicatively coupled to camera assembly 190 and may be programmed or configured to analyze images acquired by camera assembly 190, for example, In order to identify items being added to or removed from refrigeration appliance 100, as described in more detail below.
  • controller 134 may be operable to couple to camera assembly 190 for analyzing one or more images acquired by camera assembly 190 to extract useful information about objects 182 located within drawer 140 .
  • images captured by camera assembly 190 may be used to extract barcodes, identify products, monitor the movement of products, or obtain other product information related to object 182 .
  • the analysis may be performed locally (e.g., on the controller 134) or may be sent to a remote server for analysis (e.g., in the "cloud," as those of ordinary skill in the art will recognize to refer to remote server or database in a distributed computing environment including at least one remote server and local controller 134).
  • Such analysis is intended to facilitate inventory management, for example, by identifying food products that are being added to or removed from the fresh food compartment 122 .
  • camera 192 (or cameras 192 collectively in camera assembly 190 ) is oriented downward from the top center of cabinet 120 and has a width (e.g., two The overall width of the drawer 140 ) of the field of view 194 (eg, as schematically shown in FIG. 3 and corresponding to the exemplary image of FIG. 6 ).
  • the field of view 194 of the camera 192 and the resulting images acquired may capture any motion or movement of objects entering and/or exiting the drawer 140 .
  • Images captured by camera assembly 190 may include one or more still images, one or more video clips, or any other suitable type and quantity suitable for food product identification (e.g., generally identified by reference numeral 182) or inventory analysis. Image.
  • camera assembly 190 may acquire images in response to any suitable trigger, such as a time-based imaging schedule in which camera assembly 190 periodically images and monitors drawer 140 .
  • the camera assembly 190 may periodically take low-resolution images until motion (such as opening, for example, sliding one or both drawers 140 forward) is detected (e.g., by image differentiation of the low-resolution images). ), at which point one or more high-resolution images can be acquired.
  • the cooling appliance 100 may include one or more motion sensors (eg, optical, acoustic, electromagnetic, etc.) that are triggered when an object 182 is being added to or removed from the drawer 140 motion sensors, and camera assembly 190 may be operable to couple to such motion sensors to acquire images of object 182 during such movement.
  • motion sensors eg, optical, acoustic, electromagnetic, etc.
  • the refrigeration appliance 100 may include a door switch that detects when the refrigerator door 128 is opened, at which point the camera assembly 190 may begin to acquire one or more images.
  • image 300 may be acquired continuously or periodically when refrigerator door 128 is in an open position and/or when one or both drawers 140 are in an open position.
  • acquiring an image 300 may include determining that a door and/or drawer of the refrigeration appliance is open, and capturing images at a set frame rate while the door and/or when the drawer is open.
  • the motion of food items between image frames may be used to determine whether food items 182 are being removed from or added to fresh food compartment 122 .
  • the images captured by camera assembly 190 may vary in number, frequency, angle, resolution, detail, etc., in order to improve the clarity of food product 182 .
  • controller 134 may be configured to illuminate a refrigerator light source (not shown) while acquiring image 300 .
  • Other suitable imaging triggers may also be employed and are within the scope of the present invention.
  • the refrigeration appliance 100 may include an atmospheric condition sensor or sniffer 200 in fluid communication with the fresh food compartment 122 .
  • sensor 200 may be disposed within housing 120, such as within fresh food compartment 122 disposed therein, such as within drawer 140 in fresh food compartment 122, such that a fluid (e.g., gas, such as within fresh food compartment 122 (particularly within drawer 140, such as within food storage compartment 144 defined therein) or other atmospheric gas) flows to and around and/or through sensor 200, whereby sensor 200 can detect or monitor atmospheric conditions, such as food Atmospheric composition, temperature, humidity, and other similar atmospheric conditions within fresh-keeping compartment 122 and drawer 140.
  • a fluid e.g., gas, such as within fresh food compartment 122 (particularly within drawer 140, such as within food storage compartment 144 defined therein) or other atmospheric gas
  • multiple sensors 200 may be provided.
  • a sensor 200 when a sensor 200 is provided in one of the drawers 140 , another sensor 200 may be provided in the other drawer 140 .
  • a plurality of sensors 200 may be provided, each sensor operable and configured to measure a different atmospheric condition, such as a temperature sensor and a chemical sensor, for example, a chemical sensor may be a sensor that detects or measures the concentration of a particular chemical or chemical A sniffer for the type of substance (such as ethylene).
  • an example drawer 140 may include a food storage compartment 144 .
  • the food storage compartment 144 may be defined by a plurality of walls of the drawer 140 .
  • the plurality of walls may include a front wall 146 , a rear wall 148 , a left wall 150 , and a right wall 152 .
  • Directional terms such as “left” and “right” are used herein with reference to the perspective of a user standing in front of the refrigeration appliance 100 approaching items stored therein.
  • One of the walls eg, front wall 146 in the illustrative example embodiment of FIG. 7
  • drawer 140 may include a humidity control knob, slider or lever to adjust the opening of vent 154 , such as slider 156 , for example, as shown in FIG. 8 .
  • Slider 156 is movable to selectively vary the degree of opening and closing of one or more vents 154, such as in an open position where one or more vents are unobstructed to enhance air circulation into and through food storage compartment 144 versus in an open position.
  • One or more vent holes are closed by slider 156 to restrict air flow between the closed positions of food storage compartment 144 .
  • slider 156 as shown in FIG. 8 may move in two generally opposite directions within track 158 along a single path (eg, in direction 1000 as shown in FIG. 8 ).
  • the slider 156 is movable through a number of intermediate positions between the open position and the closed position, such as the exemplary intermediate position shown in FIG.
  • One or more vent holes are partially blocked by slider 156 .
  • slider 156 is movable within track 158 in two opposite directions (eg, moving forward and backward) in direction 1000 between the open position and the closed position and through a plurality of intermediate positions therebetween.
  • the exemplary embodiment shown in FIG. 8 includes a slider 156 and a plurality of vent holes 154 at an exemplary intermediate position, wherein one vent hole 154 is completely unobstructed or fully open and the other is adjacent.
  • the ventilation holes 154 are partially open, eg, partially blocked.
  • Changing the position of the slider 156 as described achieves different humidity levels within the food storage compartment 144 of the drawer 140 .
  • the humidity e.g., the moisture content is relatively low compared to the ambient air outside the refrigeration appliance and/or the air inside the rest of the food preservation compartment 122 outside the drawer 140. Higher air
  • the humidity level in the drawer 140 will reach equilibrium with the ambient humidity level, for example, in the outside of the drawer 140 Inside the remainder of the food preservation compartment 122.
  • the various intermediate positions provide different rates at which the humidity within the food storage compartment 144 of the drawer 140 is equilibrated with the humidity in the remainder of the fresh food compartment 122 outside of the drawer 140, such as when the slider 156 is in the middle
  • equilibrium is reached faster (and causes the humidity in the food storage compartment 144 of the drawer 140 to be lower)
  • neutral position of the slider 156 is closer to the closed position
  • equilibrium is reached more slowly (and causes the drawer 140
  • the humidity in the food storage compartment 144 is higher).
  • the intermediate position shown in FIG. 8 (in which one vent 154 is open and the other vent is partially open and partially closed) allows some moisture to escape from the atmosphere within the food storage compartment 144, while also allowing, for example, Produce that prefers moderate storage humidity maintains some humidity.
  • controller 134 may be configured to implement one or more of the following exemplary methods. It should be understood, however, that the exemplary methods discussed herein are for the purpose of describing exemplary aspects of the invention and are not intended to be limiting.
  • a refrigeration appliance may include a controller and a cabinet having a food storage compartment with a drawer slidably mounted within the food storage compartment. The drawer slides between a closed position and an open position.
  • the drawer may include a plurality of walls defining food storage compartments.
  • a refrigerator may also include sensors for detecting atmospheric conditions within the food storage compartment of the drawer and a camera assembly positioned and configured for monitoring the drawer, as described above.
  • method 400 includes (at step 410 ) acquiring an image of a cooling chamber of a refrigeration appliance using a camera assembly.
  • camera assembly 190 of refrigeration appliance 100 may acquire image 300 (e.g., as shown in FIG. Multiple objects 182 .
  • camera assembly 190 of refrigeration appliance 100 may capture one or more images (eg, such as image 300 ) of fresh food compartment 122, freezer compartment 124, or any other zone or area in or around refrigeration appliance 100. .
  • the method may also include and/or the refrigeration appliance may be further configured to identify one or more food products, such as identifying a first food product and a second food product based on one or more images.
  • the identification of the food product may be accomplished using the camera assembly 190 .
  • the refrigeration appliance may include a camera, and the step of identifying the food may include identifying the food based on an image captured by the camera.
  • operation of the camera may be associated with door opening, for example, the camera may be operable and configured to capture images whenever the door is opened and/or whenever the door is closed after detection of door opening.
  • controller 134 of refrigeration appliance 100 may be configured to perform image-based processing, for example, to identify food products based on images of food products (e.g., photographs of food products taken using camera 192 of camera assembly 190). .
  • the controller 134 may be configured to identify the food product by comparing the image to stored images of known or previously identified food products.
  • method 400 may include a step 420 of analyzing the image acquired at step 410 to identify a first food product and a second food product in a food storage compartment of a drawer.
  • controller 134 or any other suitable dedicated controller of refrigeration appliance 100 is communicatively coupled to camera assembly 190 and may be programmed or configured to analyze images acquired by camera assembly 190, e.g., In order to identify items stored in the refrigeration appliance 100, as detailed above.
  • Step 420 includes analyzing the image to identify objects disposed in the food storage compartment 144 of the drawer 140, such as at least a first food product and a second food product. It should be understood that the analysis may utilize any suitable image analysis technique, image decomposition, image segmentation, image processing, and the like. This analysis may be performed entirely by controller 134, may be transferred to a remote server for analysis, may be performed with user assistance (eg, via user interface panel 136), or may be performed in any other suitable manner. According to an exemplary embodiment of the invention, the analysis performed at step 420 may include a machine learning image recognition process.
  • this image analysis may use any suitable image processing technique, image recognition process, or the like.
  • image analysis and the like may generally be used to refer to any suitable method of observing, analyzing, image decomposition, feature extraction, image classification, etc., of one or more images, videos, or other visual representations of objects.
  • this image analysis may include implementing image processing techniques, image recognition techniques, or any suitable combination thereof.
  • the image analysis may use any suitable image analysis software or algorithm to continuously or periodically monitor objects within the fresh food compartment 122, such as within the drawers 140 therein. It should be appreciated that this image analysis or processing may be performed locally (eg, by controller 134 ) or remotely (eg, by transferring the image data to a remote server or network, eg, in the cloud).
  • analyzing the one or more images may include implementing an image processing algorithm.
  • image processing and the like are generally intended to refer to any suitable method or algorithm for analyzing images that does not rely on artificial intelligence or machine learning techniques (e.g., as compared to the machine learning image recognition process described below ).
  • image processing algorithms may rely on image differentiation, eg, a pixel-by-pixel comparison of two sequential images. This comparison can help identify substantial differences between sequentially acquired images, for example, to identify movement, the presence of a particular object, the presence of a particular condition, and the like.
  • one or more reference images may be acquired when a particular condition exists, and these reference images may be stored for future comparison with images acquired during operation of the appliance. The similarity and/or difference between the reference image and the acquired image can be used to extract useful information for improving the performance of the electrical appliance.
  • image differentiation can be used to determine when a pixel-level motion metric exceeds a predetermined motion threshold.
  • Processing algorithms may also include measures for isolating or removing noise in image alignments, eg, due to image resolution, data transmission errors, lighting inconsistencies, or other imaging errors. By removing this noise, image processing algorithms improve accurate object detection, avoid false object detections, and isolate important objects, regions, or patterns within an image. Additionally, or alternatively, the image processing algorithm may use other suitable techniques for discerning or identifying particular items or objects, such as edge matching, divide and conquer search, grayscale matching, receptive field histograms, or another suitable routine (eg, at controller 134 based on one or more captured images from one or more cameras). Other image processing techniques may also be employed and are within the scope of the present invention.
  • image analysis may include the use of artificial intelligence (“AI”), such as machine learning image recognition processes, neural network classification modules, any other suitable artificial intelligence (AI) techniques, and/or Any other suitable image analysis technique, examples of which are described in more detail below.
  • AI artificial intelligence
  • each of the exemplary image analysis or evaluation procedures described below may be used individually, together, or interchangeably to extract detailed information about the image being analyzed to facilitate one or more of the methods described herein. method, or otherwise improve the operation of the appliance.
  • any suitable number and combination of image processing, image recognition, or other image analysis techniques may be used to accurately analyze the acquired images.
  • the image recognition process may use any suitable artificial intelligence technique, eg, any suitable machine learning technique, or eg, any suitable deep learning technique.
  • the image recognition process may include implementing a form of image recognition known as a region-based convolutional neural network ("R-CNN").
  • R-CNN may involve taking an input image and extracting candidate regions that include potential objects or regions of the image.
  • a "candidate region” may be one or more regions in an image that may belong to a particular object, and may also include adjacent regions that share common pixel characteristics. Then, features of the candidate regions are computed using a convolutional neural network, and then, the extracted features are used to determine the classification of each specific region.
  • the image segmentation process may be used in conjunction with R-CNN image recognition.
  • image segmentation creates pixel-based masks for each object in the image and enables a more detailed or finer understanding of various objects in a given image.
  • image segmentation may involve segmenting an image into segments (e.g., into groups of pixels containing similar attributes) that may be analyzed individually or in parallel to obtain better information about one or more objects in the image.
  • Detailed representation rather than processing the entire image (eg, a large number of pixels, many of which may not contain useful information). This may be referred to in this paper as "Mask R-CNN" etc., rather than the regular R-CNN architecture.
  • mask R-CNN can be based on fast R-CNN which is slightly different from R-CNN.
  • R-CNN first applies a Convolutional Neural Network (“CNN”), which is then assigned to proposal regions on the covn5 attribute map, rather than to the initially split proposal regions.
  • CNN Convolutional Neural Network
  • a standard CNN may be used to acquire, identify, or detect any other qualitative or quantitative data related to one or more objects or regions within one or more images.
  • a K-means algorithm may be used.
  • the image recognition process may use any other suitable neural network process while remaining within the scope of the present invention.
  • the step of analyzing the one or more images may include using a deep belief network (“DBN”) image recognition process.
  • the DBN image recognition process can typically involve stacking many separate unsupervised networks, using the hidden layers of each network as input to the next layer.
  • the step of analyzing the one or more images may include implementing a deep neural network (“DNN”) image recognition process, which typically involves using a Neural Networks (computing systems inspired by biological neural networks).
  • DNN deep neural network
  • Other suitable image recognition processes, neural network processes, artificial intelligence analysis techniques, and combinations of the methods described above or other known methods may be used while remaining within the scope of the present invention.
  • the neural network architecture can be pre-trained with public datasets (such as VGG16/VGG19/ResNet50), and then the last layer can be retrained with appliance-specific datasets.
  • the image recognition process may include detecting certain conditions based on comparing initial conditions, which may rely on image subtraction techniques, image stacking techniques, image concatenation, and the like. For example, subtracting images can be used to train a multi-class neural network for future comparison and image classification.
  • the machine learning image recognition model may be actively trained by the appliance using new images, may be provided with training data by the manufacturer or another remote source, or may be trained by any other suitable means.
  • the image recognition process relies at least in part on a neural network that is trained using multiple images of appliances in different configurations, undergoes different conditions, or interacts in different ways.
  • This training data can be stored locally or remotely and can be passed to a remote server for use in training other appliances and models.
  • image processing and machine learning image recognition processes may be used in conjunction to facilitate improved image analysis, object detection, or other useful qualitative or quantitative data or information extraction from one or more images, which data or information may be used in Improve the operation or performance of the appliance.
  • the methods described herein may use any or all of these techniques interchangeably to improve the image analysis process and facilitate improved appliance performance and consumer satisfaction.
  • the image processing algorithms and machine learning image recognition processes described in the present invention are exemplary only and are not intended to limit the scope of the present invention in any way.
  • the exemplary method 400 may also include a step 430 of detecting atmospheric conditions within the food storage compartment (eg, food storage compartment 144 ) above a predetermined threshold.
  • the predetermined threshold may be a default value stored in a memory of the controller. Atmospheric conditions may be monitored and/or detected with one or more sniffers or sensors 200, as described above.
  • the predetermined threshold may be, for example, an ethylene level. Additional exemplary atmospheric conditions and their corresponding predetermined thresholds include temperature, humidity levels, and/or levels or concentrations of any other chemicals or constituents in the atmosphere within the drawer (eg, drawer 140 ).
  • Method 400 may also include step 440 of identifying one of the first food product and the second food product as a source causing the atmospheric condition to be above a predetermined threshold based on the analysis of the images.
  • steps 420 and 440 may use multiple images or the same image from the same set of images, where the set of images includes multiple images of the same field or location taken over time.
  • identifying one of a first food product and a second food product as a source of atmospheric conditions may include image analysis whereby a color change in a food product, such as a fruit, vegetable, or other similar object, is identified based on a time series of images of the same object in a drawer. Produce darkens or browns, etc.
  • the steps are not necessarily performed in the order given, for example, the detecting step 430 may occur prior to identifying the first food product and the second food product, such as may occur in response to detecting atmospheric conditions above a predetermined threshold. identify.
  • the atmospheric conditions may be ethylene levels
  • the predetermined threshold may be excessive ethylene levels, for example, since ethylene levels may be detrimental to storage of at least one agricultural product, such ethylene levels may be Excessive, wherein the method may thus include detecting excess ethylene levels and, in response to detecting ethylene levels, acquiring and analyzing images to locate the source of the ethylene levels.
  • method 400 may also include issuing a user notification.
  • the user notification may include an indication or identification that one of the first food product and the second food product has been identified as a source causing the atmospheric condition above a predetermined threshold.
  • sensors may be used to detect atmospheric conditions within the food storage compartment of the drawer when the drawer is in the closed position, such as step 430 may be performed when the drawer is in the closed position.
  • the controller may send or query a sensor when the drawer is in a closed position, wherein the closed position may be detected by the controller based on a position switch or a position sensor (e.g., a Hall effect sensor) and/or based on images from a camera assembly, Among other things, the controller can analyze such images to identify and detect when the drawer is in the closed position.
  • measuring or detecting atmospheric conditions provides a more accurate reading of the atmosphere within the drawer itself (such as in a food storage compartment therein) rather than ambient conditions external to the drawer, e.g. In the rest of the food preservation compartment and/or on the outside of refrigeration appliances.
  • the camera assembly can be positioned and configured to monitor the food storage compartment of the drawer when the drawer is in the open position.
  • a clearer e.g., less occluded
  • the camera assembly can be positioned and configured to monitor the food storage compartment of the drawer when the drawer is in the open position.
  • the drawer may also include a ventilation hole disposed in and through one of the walls.
  • the camera assembly may be configured to monitor a drawer vent, for example, the vent may be positioned within the camera's field of view.
  • Such embodiments may also include determining an optimal humidity level for at least one of the first food product and the second food product and determining an optimal position of the slider at the vent hole corresponding to the determined optimal humidity.
  • Exemplary embodiments where the camera assembly is positioned and configured to monitor a drawer's vent may also include analyzing the image to determine if the slider at the vent is in an optimal position, and when the slider at the vent is not in the optimal position.
  • the user notification can be, for example, an audible user notification and/or a visible user notification as described in detail below, and can also be locally and locally as described in detail below. /or provided remotely.
  • embodiments of the present disclosure may include a method 500 of operating a refrigeration appliance, such as the exemplary refrigeration appliance 100 described above.
  • a refrigeration appliance may include a controller and multiple food storage drawers, etc., as described above.
  • the method 500 also includes an image acquisition step 510 and an analysis and identification step 520 similar to steps 410 and 420 described above, and for the sake of brevity, such descriptions are not repeated.
  • Method 500 may include defining new or additional thresholds for one or more atmospheric conditions in the refrigeration appliance, such as in drawer 140 , compared to the predetermined thresholds described above with respect to example method 400 .
  • one or more thresholds may be based on and/or responsive to an identified food product, such as an expected or expected ethylene level for a particular type of produce, where the expected or expected ethylene level corresponds to the identified food product becoming ripe (or mature, etc.).
  • exemplary method 500 may include the step 530 of setting a first threshold of atmospheric conditions based on identification of a first food product and the step of setting a second threshold of atmospheric conditions based on identification of a second food product 540.
  • the first food product may be different from the second food product, and thus the first threshold value may also be different from the second threshold value, although different food products may not necessarily have different threshold values.
  • method 500 may further include a step 550 of monitoring atmospheric conditions, wherein at steps 530 and 540 a first threshold and a second threshold of atmospheric conditions are set.
  • monitoring may be performed at least in part by sensors, such as by a controller of a refrigeration appliance operable to communicate with the sensors, for example, exemplary methods may include monitoring atmospheric conditions using and/or by sensors.
  • method 500 may also include the step of issuing one or more user notifications.
  • Such notifications may be sent locally (e.g., on the user interface panel 136 of the refrigeration appliance 100) and/or remotely (such as on a remote device not directly physically attached or connected to the refrigeration appliance, e.g., a smartphone, smart home system or other similar device).
  • User notifications may include one or more of visual notifications (eg, illuminating a light or providing a text notification) and/or audible notifications (such as a bell or alarm sound, etc.).
  • method 500 may include the step 560 of issuing a first user notification when a first atmospheric condition threshold based on identifying a first food product is reached, and issuing a second user notification when a second atmospheric condition threshold based on identifying a second food product is reached. Step 570.
  • customized and responsive monitoring and inventory management may be provided in method 500, wherein each food product is individually and specifically tracked based on atmospheric conditions that are more pronounced or sensitive to a particular identified food product.
  • the first atmospheric condition threshold of step 530 may comprise a first ethylene level and the second atmospheric condition threshold of step 540 may comprise a second ethylene level.
  • the senor may be operable to detect atmospheric conditions within the food storage compartment of the drawer when the drawer is in the closed position.
  • the controller may send or query a sensor when the drawer is in a closed position, wherein the closed position may be detected by the controller based on a position switch or a position sensor (e.g., a Hall effect sensor) and/or based on images from a camera assembly, Among other things, the controller can analyze such images to identify and detect when the drawer is in the closed position.
  • a position switch or a position sensor e.g., a Hall effect sensor
  • the controller can analyze such images to identify and detect when the drawer is in the closed position.
  • measuring or detecting atmospheric conditions provides a more accurate reading of the atmosphere within the drawer itself (such as in a food storage compartment therein) rather than ambient conditions external to the drawer, e.g. In the rest of the food preservation compartment and/or on the outside of refrigeration appliances.
  • the camera assembly may be configured to monitor the food storage compartment of the drawer when the drawer is in the open position.
  • a clearer e.g., less occluded
  • the camera assembly may be configured to monitor the food storage compartment of the drawer when the drawer is in the open position.
  • the drawer may also include a ventilation hole through one of the walls.
  • the camera assembly may be configured to monitor a drawer vent, for example, the vent may be positioned within the camera's field of view.
  • Such embodiments may also include determining an optimal humidity level for at least one of the first food product and the second food product and determining an optimal position of the slider at the vent hole corresponding to the determined optimal humidity.
  • such embodiments may also or instead include a humidity level as a first threshold and a second threshold for atmospheric conditions, for example, the atmospheric condition may be humidity and the first and first thresholds may each be a humidity level.
  • Exemplary embodiments where the camera assembly is positioned and configured to monitor a drawer's vent may also include analyzing the image to determine if the slider at the vent is in an optimal position, and when the slider at the vent is not in the optimal position.
  • the user notification may be, for example, an audible user notification and/or a visible user notification as described above, and may also be locally and/or as described above Offered remotely.
  • embodiments of the invention may also include a method 600 of operating a refrigeration appliance, such as the exemplary refrigeration appliance 100 described above.
  • a refrigeration appliance may include a controller and multiple food storage drawers, etc., as described above.
  • Method 600 also includes an image acquisition step 610 and an analysis and identification step 620 similar to steps 410/510 and 420/520 described above, and for the sake of brevity, such descriptions are not repeated.
  • Method 600 may also include a step 630 of determining that the first food product and the second food product are incompatible for co-storage.
  • incompatibilities may include different optimal humidity levels and/or temperatures.
  • incompatibility may also or instead include that one of the first food and the second food produces ethylene, e.g., when the food ripens or ages, the food produces or releases a significant amount of ethylene (with the other agricultural product), and the first food and the other of the first food are sensitive to ethylene, for example, where exposure to levels of ethylene released by one food may accelerate the rate of aging of the other food.
  • the determination of incompatibility can be based at least in part on the rate at which one of the first food product and the second food product produces ethylene.
  • method 600 may then include a step 640 of providing a user notification including relocating the first food product and the second food product.
  • the suggestion may include a suggestion to move a food item to another drawer.
  • a recommendation may include a recommendation to move a food product to another part of the food preservation compartment, for example, the outside of one or more drawers, or may include a recommendation to store a food product at room temperature, for example, in a refrigerated appliance. external.
  • a user notification may just provide a suggestion to remove or relocate an item of food from a drawer without specifying where that item of food should be moved.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Chemical & Material Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Thermal Sciences (AREA)
  • Mechanical Engineering (AREA)
  • Combustion & Propulsion (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Business, Economics & Management (AREA)
  • Emergency Management (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Cold Air Circulating Systems And Constructional Details In Refrigerators (AREA)

Abstract

提供了操作制冷电器的方法。制冷电器包括具有食品存储隔室的机柜，该食品存储隔室带有可滑动地安装在食品存储隔室内的抽屉。制冷电器也包括用于检测抽屉的食品存储隔间内的大气状况的传感器以及被配置成用于监测抽屉的摄像机组件。该方法通常包括使用摄像机组件获取图像并且分析该图像以识别抽屉的食品存储隔间中的第一食品和第二食品。

Description

带有智能抽屉的制冷电器 技术领域
本发明整体涉及制冷电器，更具体地，涉及用于管理存储在这此类制冷电器中的物品(诸如农产品)的状态的系统和方法。
背景技术
制冷电器通常包括具有冷冻室的机柜。种类繁多的食品可存储在冷冻室内。冷冻室相对于环境大气的较低温度可以延长存储在冷冻室内的食品的保质期。
存储在制冷电器中的农产品(例如，水果和蔬菜)随着时间的推移会经历各种物理变化和化学变化，例如成熟。各种不同的农产品可能彼此不相容，诸如可能具有不同的存储要求。例如，一种农产品的最佳温度、湿度和/或大气成分可能不同于另一种农产品。不同的农产品也可能一起存储时不相容，因为在存储期间，每种农产品随着时间的推移发生不同的变化，诸如一种农产品可能在成熟时产生某些对其他农产品有害的大气化学物质。
因此，带有改良后的库存管理系统的制冷电器应该很有用。更具体地，包括能够监测农产品库存并且追踪此类物品在存储期间的状态的农产品库存管理系统的制冷电器应该很有用。
发明内容
本发明的各个方面和优点将在以下描述中进行部分阐述，或者可根据描述而变得显而易见，或者可通过实践本发明习得。
在示例性实施方式中，提供了一种运行制冷电器的方法。制冷电器包括具有食品存储室的机柜，该食品存储室设置有可滑动地安装在食品存储室内的抽屉。抽屉可在关闭位置和打开位置之间滑动。抽屉包括多个限定食品存储隔间的壁。制冷电器也包括用于检测抽屉的食品存储隔间内的大气状况的传感器以及被配置成用于监测抽屉的摄像机组件。该方法包括使用摄像机组件获取图像并且分析图像以识别抽屉的食品存储隔间中的第一食品和第二食品。该方法也包括使用传感器检测到食品存储隔间内的大气状况高于预定阈值，并且基于对图像的分析，将第一食品和第二食品中的一者识别为造成大气状况高于预定阈值的来源。
在另一个示例性实施方案中，提供了一种运行制冷电器的方法。制冷电器包括限定食品存储隔室的机柜，该食品存储隔室带有可滑动地安装在食品存储隔室内的抽屉。抽屉可在关闭位置和打开位置之间滑动。抽屉包括多个限定食品存储隔间的壁。制冷电器也包括能够操作为检测抽屉的食品存储隔间内的大气状况的传感器以及被定位和配置成用于监测抽屉的摄像机组件。该方法包括使用摄像机组件获取图像并且分析图像以识别抽屉的食品存储隔间中的第一食品和第二食品。该方法也包括基于识别第一食品来设定大气状况的第一阈值以及基于识别第二食品来设定大气状况的第二阈值。该方法还包括由传感器监测大气状况。该方法也包括当大气状况达到第一阈值时发出第一用户通知并且当大气状况达到第二阈值时发出第二用户通知。
在又另一个示例性实施方式中，提供了一种运行制冷电器的方法。制冷电器包括限定食品存储隔室的机柜，该食品存储隔室带有可滑动地安装在食品存储隔室内的抽屉。抽屉可在关闭位置和打开位置之间滑动。抽屉包括多个限定食品存储隔间的壁。制冷电器也包括能够操作为检测抽屉的食品存储隔间内的大气状况的传感器以及被配置成用于监测抽屉的摄像机组件。该方法包括使用摄像机组件获取图像并且分析图像以识别抽屉的食品存储隔间中的第一食品和第二食品。该方法也包括确定第一食品和第二食品共同存储不相容并且发出用户通知，该用户通知包括重新放置第一食品和第二食品中的一者的建议。
参考以下描述和所附权利要求书，将更好地理解本发明的这些特征和其它特征、各方面和优点。并入本说明书中并构成本说明书的一部分的附图示出了本发明的实施方案，并且与描述一起用于解释本发明的原理。
附图说明
参考附图的说明书中阐述了针对本领域的普通技术人员的本发明的全面且可行的公开，包括其最佳实施方式。
图1提供了根据本发明的示例性实施方式的制冷电器的前视图。
图2提供了图1的制冷电器的透视图。
图3提供了图1的制冷电器的前视图，其中，该制冷电器的门处于打开位置。
图4提供了根据本发明的一个或多个示例性实施方式的制冷电器的一部分的前视图。
图5提供了图4的制冷电器的一部分的剖视图。
图6示出了制冷电器(诸如图1的示例性制冷电器)的处于打开位置的存储抽屉和抽屉的容纳物的示例性图像，该图像可由制冷电器中的摄像机组件捕获。
图7提供了可结合到本发明的一个或多个示例性实施方式中的制冷电器中的抽屉的透视图。
图8提供了图7的抽屉的一部分的放大视图。
图9提供了根据本方式的一个或多个示例性实施方式的用于运行制冷电器的示例性方法的流程图。
图10提供了根据本发明的一个或多个附加的示例性实施方式的用于运行制冷电器的另一种示例性方法的流程图。
图11提供了根据本发明的一个或多个附加的示例性实施方式的用于运行制冷电器的又一种示例性方法的流程图。
具体实施方式
现在将详细参考本发明的实施方式，其中，这些实施方式的一个或多个示例在附图中示出。通过对本发明进行解释而不是对本发明进行限制的方式提供每个示例。事实上，对于本领域的技术人员而言将显而易见，可在没有脱离本发明的范围或精神的情况下在本发明中进行各种修改和变动。例如，被示出或描述作为一个实施方式的一部分的特征可与另一个实施方式结合使用以提供另外的实施方式。因此，本发明旨在涵盖落入所附权利要求书以及其等同权利要求的范围内的此类修改和变动。
图1是制冷电器100的示例性实施方式的前视图。图2是制冷电器100的透视图。图3是制冷电器100的前视图，其中，该制冷电器的食品保鲜门128处于打开位置。制冷电器100沿垂直方向V在顶部101和底部102之间延伸。制冷电器100也沿侧向方向L在第一侧面105和第二侧面106之间延伸。如图2中所示，横向方向T可为垂直于垂直方向V和侧向方向L。制冷电器100沿横向方向T在前面部分108和后面部分110之间延伸。
制冷电器100包括具有上部食品保鲜室122(图3)以及沿垂直方向V布置在食品保鲜室122下方的下部冷冻室或食品冷冻存储室124的机柜或壳体120。在一些实施方案中，辅助食品存储室(未示出)可(例如，沿垂直方向V)设置在食品保鲜存储室122和食品冷冻存储室124之间。因为食品冷冻存储室124设置在食品保鲜存储室122下方，所以制冷电器100通常被称为底部安装冰箱。在示例性实施方式中，壳体120还具有用于容纳密封冷却系统(未示出)的机械式室(未示)。使用本发明所公开的教导内容，本领域的技术人员将会理解，本发明也可与其他类型的冰箱(例如，对门式)一起使用。因此，本发明阐述的描述仅用于说明目的，并且不旨在在任何方面限制本发明。
冰箱门128各自可旋转地铰接到壳体120的边缘，用于开闭食品保鲜室122。需要说明的是，虽然例示了“法式门”构型的两个门128，但是利用一个、两个或更多个门的任何合适的门排布结构均在本发明的范围和精神内。冷冻门130布置在冰箱门128下方，用于开闭冷冻室124。在示例性实施方式中，冷冻门130联接到可滑动地安装在冷冻室124内的冷冻抽屉(未示出)。辅助门127可联接到可滑动地安装在辅助室(未示)内的辅助抽屉(未示出)。
制冷电器100的运行可由能够操作为联接到用户界面面板136的控制器134进行调节。用户 界面面板136提供用于用户操纵制冷电器100的运行以修改其中的环境状况的选项，诸如温度选项等。在一些实施方式中，用户界面面板136可邻近分配器组件。面板136提供用于用户操纵制冷电器100的运行的选项，例如，温度选项、自动式或手动式超控湿度控制的选项(如下文更详细所述)等。响应于用户操纵用户界面面板136，控制器134控制制冷电器100的各种部件运行。制冷电器100的运行可由控制器134调节，例如，控制器134可响应于编程和/或用户操纵用户界面面板136而调节制冷电器100的各种部件的运行。
控制器134可包括存储器和一个或多个微处理器、CPU等，诸如能够操作为执行与运行制冷电器100相关联的编程指令或微控制代码的通用微处理器或专用微处理器。存储器可表示随机存取存储器(诸如DRAM)或只读存储器(诸如ROM或FLASH)。在一个实施方案中，处理器执行存储在存储器中的编程指令。存储器可以是与处理器分离的部件，或者可板载在处理器内。需要说明的是，如本文所公开的控制器134能够并且能够操作为执行如本文所公开的任何方法和相关联的方法步骤。
控制器134可定位在整个制冷电器100的各个位置。在例示的实施方案中，控制器134可位在门128内。在此类实施方案中，可在控制器和制冷电器100的各种运行部件之间传输输入/输出(“I/O”)信号。在一个实施方式中，用户界面面板136可表示通用I/O(“GPIO”)设备或功能块。在一个实施方式中，用户界面136可包括输入部件，诸如包括旋转仪表盘、按钮和触摸板的各种电气输入设备、机械输入设备或机电输入设备中的一种或多种设备。用户界面136可包括显示部件，诸如被设计用于向用户提供运行反馈的数字显示设备或模拟显示设备。例如，用户界面136可包括提供输入和显示两种功能的触摸屏。用户界面136可经由一条或多条信号线或共享通信总线与控制器通信。
可如图3中所示，多个食品存储元件(诸如盒体138、搁架142和抽屉140)设置在食品保鲜存储室122内。如下文将所详述，抽屉140可被配置成用于存储农产品(诸如水果和蔬菜)，并且具体地，制冷电器可能够被操作为改善并且配置成用于改善存储在其中的农产品的保质期。因此，抽屉140也可称为农产品抽屉140或蔬菜抽屉140和水果抽屉140。例如，在一些实施方式中，制冷电器可包括两个抽屉140，例如，如图3所示。
现在大体上参考图3至图6，制冷电器100还可包括库存管理系统，该库存管理系统通常被配置成监测制冷电器100的一个或多个室，以监测存储在其中的库存的状态。更具体地，如下文所详述，库存管理系统可包括一个或多个嗅探器或传感器200(参见例如，图4和图5)、摄像机192(参见例如，图3和图5)或用于监测食品保鲜室122以及尤其抽屉140的其他检测设备，以检测和监测位于或移出抽屉140的物体(例如，通常由图6中的参考标号182识别)。就这一点而言，库存管理系统可使用来自这些设备中的每个设备的数据来获取抽屉140内的物体182(例如，诸如水果和/或蔬菜等农产品)的身份、位置和/或其他定性特性或定量特性的表示或知识。虽然在本文中将库存管理系统描述为用于检测物体182的监测抽屉140，但是应当理解，本发明的各方面可用于监测在任何其他合适的电器、室等中的物体或物品。
如图3中示意性所示，库存管理系统可包括摄像机组件190，该摄像机组件通常被定位和配置成用于在运行期间获取制冷电器100的图像。具体地，根据例示实施方式，摄像机组件190包括一个或多个摄像机192，该一个或多个摄像机安装到机柜120、门128或以其他方式设置在食品保鲜隔室122的视野中。如图3所示，摄像机组件190的摄像机192在食品保鲜隔室122的前开口处安装到机柜120，并且被定向为具有视场194，该视场194覆盖前开口和/或进入食品保鲜隔室122并且尤其是进入抽屉140内，诸如当抽屉140处于打开位置时，诸如大体上沿横向方向T向前延伸使得抽屉140中的一个或两个抽屉延伸超过在摄像机组件190和抽屉140之间的一个搁架142或多个搁架142的前边缘(前方)，诸如在抽屉140上方。
虽然图3中示出了单个摄像机192，但是应当理解，摄像机组件190可包括多个设置在机柜120内的摄像机192，其中，多个摄像机192中的每个摄像机均具有在食品保鲜室122周围的特定监测区或范围。就这一点而言，例如，每个摄像机192的视场194可被限制或聚焦在食品保鲜隔室122内的特定区域，诸如每个抽屉140一个摄像机192。
然而，尤其，可能需要将每个摄像机192设置成邻近食品保鲜室122的前开口并且将每个摄像机190定向成使得视场194被引导到食品保鲜隔室112中。这样，可减轻或完全避免与获取电器100的用户的图像相关的隐私忧虑。根据示例性实施方案，摄像机组件190可用于改善制冷电器100的库存管理过程。因此，每个摄像机192可设置在通向食品保鲜隔室122的开口处，以监测正在添加到或移出食品保鲜隔室122的食品(通常标识为物体182)，尤其是正在放置在或移出抽屉140的食品。
应当理解，根据替代的实施方式，摄像机组件190可包括任何合适数量、类型、尺寸和构型的摄像机192，以用于获取任何合适的制冷电器100内或周围的区域的图像。此外，应当理解，每个摄像机192可包括用于调整视场和/或定向的特征部。
应当理解，由摄像机组件190获取的图像可在数量、频率、角度、分辨率、细节等方面有所变化，以便提高在制冷电器100周围或其内的特定区域的清晰度。此外，根据示例性实施方式，控制器134可被配置成用于在获取图像之前使用一个或多个光源照亮冷冻隔室。尤其，制冷电器100的控制器134(或任何其他合适的专用控制器)以能够通信的方式联接到摄像机组件190，并且可被编程或配置成用于分析由摄像机组件190获取的图像，例如，以便识别正在添加到或移出制冷电器100的物品，如下文所详述。
一般而言，控制器134可能够操作为联接到摄像机组件190，以用于分析由摄像机组件190获取的一张或多张图像，以提取有关位于抽屉140内的物体182的有用信息。就这一点而言，例如，由摄像机组件190获取的图像可用于提取条形码、识别产品、监测产品的运动或获取与物体182相关的其他产品信息。尤其，该分析可在本地(例如，在控制器134上)执行或可发送到用于进行分析的远程服务器(例如，在“云端”，如本领域的普通技术人员应认识到其指的是包括至少一个远程服务器和本地控制器134的在分布式计算环境中的远程服务器或数据库)。此类分析旨在促进库存管理，例如，通过识别正在添加到或移出食品保鲜隔室122的食品。
具体地，根据示例性实施方式，摄像机192(或摄像机组件190中的多个摄像机192整体)被定向为从机柜120的顶部中心朝下，并且具有覆盖食品保鲜室122的宽度(例如，两个抽屉140的总宽度)的视场194(例如，如图3中示意性地示出并且对应于图6的示例性图像)。这样，摄像机192的视场194以及获取的结果图像可捕获物体进入和/或离开抽屉140的任何运动或移动。由摄像机组件190获取的图像可包括一个或多个静态图像、一个或多个视频片段、或适于进行食品标识(例如，通常由参考标号182标识)或库存分析的任何其他合适的类型和数量的图像。
尤其，摄像机组件190可应任何合适的触发获取图像，诸如基于时间的摄像机组件190周期性地对抽屉140进行成像和监测的成像时间表。根据其它的实施方式，摄像机组件190可周期性地拍摄低分辨率图像，直到(例如，通过低分辨率图像的图像微分)检测到运动(诸如打开，例如，向前滑动一个或两个抽屉140)，此时可获取一个或多个高分辨率图像。根据其它的实施方式，制冷电器100可包括一个或多个运动传感器(例如，光学式、声学式、电磁式等)，当正在将物体182添加到或移出抽屉140时，触发该一个或多个运动传感器，并且摄像机组件190可能够操作为联接到此类运动传感器，以在此类移动期间获取物体182的图像。
根据其它的实施方式，制冷电器100可包括门开关，该门开关检测冰箱门128何时被打开，此时，摄像机组件190可开始获取一张或多张图像。根据示例性实施方式，当冰箱门128处于打开状态和/或当一个或两个抽屉140处于打开位置时，则可连续地或周期性地获取图像300。就这一点而言，获取图像300可包括确定制冷电器的门和/或抽屉处于打开状态，并且在门和/或者当抽屉处于打开状态时以设定的帧速率捕获图像。
尤其，食品在图像帧之间的运动可用于确定食品182是否正在被移出或添加到食品保鲜室122。应当理解，由摄像机组件190获取的图像可在数量、频率、角度、分辨率、细节等方面有所变化，以便提高食品182的清晰度。此外，根据示例性实施方案，控制器134可被配置成用于照亮冰箱光源(未示出)，同时获取图像300。其他合适的成像触发也可以采用并且在本发明的范围内。
如图4和图5所示，在各种实施方式中，制冷电器100可包括与食品保鲜室122流体连通的 大气状况传感器或嗅探器200。例如，传感器200可被配置在壳体120内，诸如设置在其中的食品保鲜室122内，诸如在食品保鲜室122中的抽屉140内，使得流体(例如，气体，诸如食品保鲜隔室122内(尤其是抽屉140内，诸如限定在其中的食品存储间144内)的空气或其他大气气体)流到并且围绕和/或流过传感器200，由此传感器200可检测或监测大气状况，诸如食品保鲜室122和抽屉140内的大气成分、温度、湿度和其他类似的大气状况。
在一些实施方式中，可提供多个传感器200。例如，当传感器200设置在抽屉140中的一个抽屉时，另一个传感器200可设置在其他抽屉140中。又如，可提供多个传感器200，每个传感器能够操作为并且配置成测量不同的大气状况，诸如温度传感器和化学物质传感器，例如，化学物质传感器可以是检测或测量特定化学物质的浓度或化学物质的类型(诸如乙烯)的嗅探器。
现在参考图7和图8，示例性抽屉140可包括食品存储隔间144。可由抽屉140的多个壁限定食品存储隔间144。例如，多个壁可包括前壁146、后壁148、左壁150和右壁152。本文中参考站在制冷电器100的前面接近存储在其中的物品的用户的视角，使用了方向性术语，诸如“左”和“右”。其中一个壁(例如，如图7的例示性示例实施方案中的前壁146)可包括设置在壁(例如，前壁146)中并且穿过该壁的一个或多个通风孔154。在此类实施方式中，抽屉140可包括湿度控制旋钮、调整通风孔154的开放程度的滑块或操纵杆，诸如滑块156，例如，如图8中所示。滑块156可移动以选择性地改变一个或多个通风孔154的开闭程度，诸如在一个或多个通风孔畅通无阻以增强空气循环进入和通过食物存储隔间144的打开位置与在一个或多个通风孔被滑块156关闭以限制空气流入食品存储隔间144的关闭位置之间。
例如，如图8中所示滑块156可在轨道158内沿单条路线(例如，沿如图8中示出的方向1000)在两个大致上相对的方向上移动。另外，本领域的普通技术人员应认识到，滑块156可移动通过在打开位置和关闭位置之间的多个中间位置，诸如图8中示出的示例性中间位置，在该中间位置处，一个或多个通风孔被滑块156部分阻塞。因此，滑块156可在轨道158内沿在打开位置和关闭位置之间的方向1000在两个相对的方向上(例如，前后移动)移动并且通过在它们之间的多个中间位置。具体地，图8中示出的示例性实施方式包括在示例性中间位置处的滑块156和多个通风孔154，其中，一个通风孔154完全不受阻塞或完全打开，而另一个相邻的通风孔154部分打开，例如，部分被阻塞。
如所述的改变滑块156的位置实现抽屉140的食品存储隔间144内的不同湿度水平。例如，当滑块处于关闭位置或中间位置时，湿度(例如，与制冷电器的外部的环境空气和/或抽屉140的外部的食品保鲜隔室122的其余部分内的空气相比，水分含量相对较高的空气)可能聚积在抽屉140的食品存储隔间144内，而当滑块156处于打开位置时，抽屉140内的湿度水平将与环境湿度水平到达平衡，例如，在抽屉140的外部的食品保鲜室122的其余部分内。此外，各种中间位置提供了不同的使抽屉140的食品存储隔间144内的湿度与抽屉140的外部的食品保鲜室122的其余部分中的湿度达到平衡的速率，诸如当滑块156的中间位置靠近打开位置时，较快达到平衡(并且导致抽屉140的食品存储隔间144内的湿度较低)，而当滑块156的中间位置靠近关闭位置时，较慢达到平衡(并且导致抽屉140的食品存储隔间144内的湿度较高)。例如，图8中示出的中间位置(其中，一个通风孔154打开，而另一个通风孔部分打开和部分关闭)允许一些水分从食品存储隔间144内的大气中逸出，同时也为例如偏好适中的存储湿度的农产品保持一部分湿度。
通过使用本文所公开的教导内容，本领域的技术人员将会理解，本发明可与其他类型的冰箱结合使用，诸如冰箱/冷藏柜组合、对门式制冷电器、底部安装式制冷电器、紧凑型制冷电器和任何其他类型或样式的制冷电器。因此，可提供制冷电器100的其他构型，应当理解，附图中示出的构型和本文中阐述的描述仅作为用于说明目的的示例。
现在，已经呈现了根据本发明的示例性实施方案的摄像机组件190和制冷电器100的构造和配置，提供了用于运行制冷电器(诸如制冷电器100)的示例性方法。此类方法也可用于运行摄像机组件，例如摄像机组件190，或用于监测电器运行或库存的任何其他合适的摄像机组件。就这一点而言，例如，控制器134可被配置成实现一种或多种以下示例性方法。然而，应当理解， 本文所论述的示例性方法仅为了描述本发明的示例性方面，而不是旨在进行限制。
现在转到图9，本发明的实施方式可包括运行制冷电器(诸如上文所述的示例性制冷电器100)的方法400。例如，制冷电器可包括控制器以及具有食品存储室的机柜，该食品存储室带有可滑动地安装在食品存储隔室内的抽屉。抽屉可在关闭位置和打开位置之间滑动。抽屉可包括限定食品存储隔间的多个壁。同样作为示例，冰箱还可包括用于检测抽屉的食品存储隔间内的大气状况的传感器以及被定位并且配置成用于监测抽屉的摄像机组件，如上文所述。
如图9中所示，方法400包括(在步骤410处)使用摄像机组件获取制冷电器的冷却室的图像。例如，制冷电器100的摄像机组件190可获取食品保鲜室122和/或抽屉140的食品存储间144内的图像300(例如，如图6中所示)，抽屉140可包括在其视场中的多个物体182。就这一点而言，制冷电器100的摄像机组件190可获取食品保鲜室122、冷冻室124或制冷电器100内或周围的任何其他区或区域的一张或多张图像(例如，诸如图像300)。
在一些实施方式中，该方法也可包括和/或制冷电器还可被配置成用于识别一个或多个食品，诸如基于一张或更多张图像识别第一食品和第二食品。在一些实施方式中，可使用摄像机组件190完成食品的识别。例如，制冷电器可包括摄像机，并且识别食品的步骤可包括基于由摄像机捕获的图像来识别食品。在一些实施方式中，摄像机的运行可与门打开关联，例如，摄像机可能够操作成并且配置成每当门被打开时和/或每当在检测到门打开之后，门被关闭时捕获图像。本领域的普通技术人员理解摄像机的结构和运行，并且因此，为了简洁明了，在本文中未进一步详细地示出或描述摄像机。在此类实施方式中，制冷电器100的控制器134可被配置成用于进行基于图像的处理，例如，基于食品的图像(例如，使用摄像机组件190的摄像机192拍摄的食品的照片)识别食品。例如，控制器134可被配置成通过将图像与已知或先前识别的食品的已存储的图像进行比较来识别食品。
在图9中示出的示例性实施方式中，方法400可包括分析在步骤410处获取的图像以识别在抽屉的食品存储隔室中的第一食品和第二食品的步骤420。例如，制冷电器100的控制器134(或任何其他合适的专用控制器)以能够通信的方式联接到摄像机组件190，并且可被编程或配置成用于分析由摄像机组件190获取的图像，例如，以便识别存储在制冷电器100中的物品，如上方所详述。
步骤420包括分析图像，以识别设置在抽屉140的食品存储隔间144中的物体，例如至少第一食品和第二食品。应当理解，该分析可利用任何合适的图像分析技术、图像分解、图像分割、图像处理等。该分析可完全由控制器134执行，可转移到远程服务器进行分析，可在用户协助下(例如，通过用户界面面板136)进行分析，或可以任何其他合适的方式进行分析。根据本发明的示例性实施方式，在步骤420处执行的分析可包括机器学习图像识别过程。
根据示例性实施方式，该图像分析可使用任何合适的图像处理技术、图像识别过程等。如本文中所用，术语“图像分析”等通常可用于指代任何合适的针对一张或多张图像、视频或物体的其他视觉表示进行观察、分析、图像分解、特征提取、图像分类等方法。如下文更详细地解释，该图像分析可包括实现图像处理技术、图像识别技术或它们的任何合适的组合。就这一点而言，图像分析可使用任何合适的图像分析软件或算法以不断地或周期性地监测食品保鲜室122内的物体，诸如在其中的抽屉140内。应当理解，该图像分析或处理可以在本地(例如，由控制器134)执行或远程(例如，通过将图像数据转移到远程服务器或网络，例如，在云端)执行。
具体地，分析一张或多张图像可包括实施图像处理算法。如本文所用，术语“图像处理”等通常旨在指代任何合适的未依靠人工智能或机器学习技术的用于分析图像的方法或算法(例如，与下文所述的机器学习图像识别过程相比)。例如，图像处理算法可依靠图像微分，例如，对两个按顺序的图像进行逐个像素比较。该比较可有助于识别按顺序获取的图像之间的实质性差异，例如，以识别移动、特定物体的出现、特定状况的存在等。例如，当存在特定状况时，可获取一张或多张参考图像，并且可存储这些参考图像以备将来与在电器运行期间获取的图像进行比较。参考图像和所获取的图像之间的相似度和/或差异可用于提取用于提高电器性能的有用信息。例如，图像微分可用于确定像素级运动度量何时超过预定运动阈值。
处理算法还可包括用于分离或消除图像比对中的噪声的措施，例如，由于图像分辨率、数据传输误差、照明不一致或其他成像误差造成。通过消除此类噪声，图像处理算法可改善精确的物体检测，避免错误的物体检测，并且分离图像内的重要物体、区域或图案。此外，或另选地，图像处理算法可使用其他合适的用于辨别或识别特定物品或物体的技术，诸如边缘匹配、分治搜索、灰度匹配、感受域直方图，或另外合适的例程(例如，基于来自一个或多个摄像机的一张或多张捕获的图像在控制器134处执行)。其他图像处理技术也可以采用并且在本发明的范围内。
除了上文所述的图像处理技术之外，图像分析可包括利用人工智能(“AI”)，诸如机器学习图像识别过程、神经网络分类模块、任何其他合适的人工智能(AI)技术和/或任何其他合适的图像分析技术，其示例将在下文更详细地描述。此外，下文所述的示例性图像分析或评估过程各自均可以单独使用、共同使用或可互换使用，以提取关于被分析的图像的详细信息，以促进本发明所述的一种或多种方法的性能，或以其他方式改善电器运行。根据示例性实施方式，可使用任何合适数量的图像处理、图像识别，或其他图像分析技术和它们的组合，以对所获取的图像进行精确分析。
就这一点而言，图像识别过程可使用任何合适的人工智能技术，例如，任何合适的机器学习技术，或例如，任何合适的深度学习技术。根据示例性实施方式，图像识别过程可包括实现一种被称为基于区域的卷积神经网络(“R-CNN”)的图像识别的形式。一般而言，R-CNN可包括获得输入图像并且提取包括图像的潜在物体或区域的候选区域。就这一点而言，“候选区域”可以是可能属于特定物体的图像中的一个或多个区域，也可包括共享共同的像素特性的相邻区域。然后，使用卷积神经网络计算候选区域的特征，然后，使用提取到的特征确定每个特定区域的分类。
根据其它的实施方式，图像分割过程可以与R-CNN图像识别结合使用。一般而言，图像分割为图像中的每个物体创建基于像素的掩模并且实现更详细或更精细地理解给定的图像中的各种物体。就这一点而言，图像分割可涉及将图像分割成片段(例如，分割成包含类似属性的像素组)，这些片段可以单独地或并行地进行分析以获取图像中的一个或多个物体的更详细的表示，而不是处理整张图像(例如，大量的像素，其中的很多像素可能并不包含有用信息)。这在本文中可称为“掩模R-CNN”等，而不是常规的R-CNN架构。例如，掩模R-CNN可基于与R-CNN略有不同的fast R-CNN。例如，R-CNN首先应用卷积神经网络(“CNN”)，然后将其分配给covn5属性映射上的推荐区，而不是分配给最初拆分的推荐区。此外，根据示例性实施方案，标准CNN可用于获取、识别，或检测与一张或多张图像内的一个或多个物体或区域相关的任何其他定性数据或定量数据。此外，可使用K均值算法。
根据其它的实施方式，图像识别过程可使用任何其他合适的神经网络过程，同时仍然在本发明的范围内。例如，分析一张或多张图像的步骤可包括使用深度置信网络(“DBN”)图像识别过程。DBN图像识别过程通常可包括堆叠很多单独的无监督网络，无监督网络使用每个网络的隐藏层作为下一层的输入。根据又其它的实施方案，分析一张或多张图像的步骤可包括实现深度神经网络(“DNN”)图像识别过程，深度神经网络图像识别过程通常包括使用在输入和输出之间具有多层的神经网络(由生物神经网络启发的计算系统)。可使用其他合适的图像识别过程、神经网络过程、人工智能分析技术以及上文所述方法或其他已知的方法的组合，同时仍然在本发明的范围内。
此外，应当理解，可使用各种转移技术，但不需要使用此类技术。如果使用转移技术学习，可以用公共数据集(诸如VGG16/VGG19/ResNet50)预先训练神经网络架构，然后可以用电器特定数据集重新训练最后一层。此外，或另选地，图像识别过程可包括基于对初始条件进行比较来检测特定条件，可依靠图像相减技术、图像堆叠技术、图像串联等。例如，相减图像可用于训练多类别神经网络，以备将来进行比较和图像分类之用。
应当理解，机器学习图像识别模型可由电器使用新的图像主动训练，可由制造商或另一个远程来源提供训练数据，或可通过任何其他合适的方式进行训练。例如，根据示例性实施方式，该图像识别过程至少部分依靠使用不同配置的电器的多张图像进行训练、经历不同状况，或以不同方式交互的神经网络。该训练数据可本地存储或远程存储并且可传递到远程服务器以用于训练其 他电器和模型。
应当理解，图像处理和机器学习图像识别过程可以结合使用，以促进改善图像分析、物体检测，或从一张或多张图像中提取其他有用的定性或定量数据或信息，该数据或信息可用于提高电器的运行或性能。实际上，本发明所述的方法可以互换使用这些技术中的任一种技术或所有技术来改善图像分析过程并且促进改善电器性能和消费者满意度。本发明所述的图像处理算法和机器学习图像识别过程仅是示例性的，并且不旨在以任何方式限制本发明的范围。
再次具体地参考图9，示例性方法400还可包括检测食品存储室(例如，食品存储隔间144)内的大气状况高于预定阈值的步骤430。例如，预定阈值可以是存储在控制器的存储器中的默认值。可用一个或多个嗅探器或传感器200监测和/或检测大气状况，如上文所述。预定阈值可以是例如乙烯水平。附加的示例性大气状况以及其对应的预定阈值包括温度、湿度水平和/或抽屉(例如，抽屉140)内的大气中的任何其他化学物质或成分的水平或浓度。
方法400也可包括步骤440，基于对图像的分析将第一食品和第二食品中的一者识别为造成大气状况高于预定阈值的来源。例如，步骤420和步骤440可使用来自同一组图像的多张图像或同一张图像，其中，该组图像包括随时间的推移拍摄到的同一域或位置的多张图像。例如，将第一食品和第二食品中的一者识别为大气状况的来源可包括图像分析，由此根据抽屉中的相同物体的时间序列图像识别食品的颜色变化，诸如水果、蔬菜或其他类似农产品变暗或变褐色等。
此外，应当理解，这些步骤不一定按照给定的顺序执行，例如，检测步骤430可发生在识别第一食品和第二食品之前，诸如可响应于检测到大气状况高于预定阈值而发生食品的识别。作为很多可能的示例中的一个示例，大气状况可以是乙烯水平，并且预定阈值可以是过量的乙烯水平，例如，因为乙烯水平可能对至少一种农产品的存储有害，所以，这种乙烯水平可能是过量的，其中，该方法因此可包括检测过量的乙烯水平并且响应于检测乙烯水平，获取和分析图像以定位乙烯水平的来源。
在一些实施方案中，方法400还可包括发出用户通知。用户通知可包括第一食品和第二食品中的一者已经被识别为造成大气状况高于预定阈值的来源的指示或识别。
在一些实施方案中，传感器可用于当抽屉处于关闭位置时检测抽屉的食品存储隔间内的大气状况，诸如当抽屉处于关闭位置时可执行步骤430。例如，当抽屉处于关闭位置时，控制器可发送或查询传感器，其中，关闭位置可由控制器基于位置开关或位置传感器(例如，霍尔效应传感器)和/或基于来自摄像机组件的图像进行检测，其中，控制器可分析此类图像以识别和检测抽屉何时处于关闭位置。有利的是，当抽屉处于关闭位置时，测量或检测大气状况可更精确地读取抽屉本身内(诸如在其中的食品存储隔间中)的大气，而不是抽屉的外部的环境状况，例如，在食品保鲜隔间的其余部分中和/或制冷电器的外部。
在一些实施方案中，摄像机组件可被定位和配置成当抽屉处于打开位置时监测抽屉的食品存储隔间。例如，有利的是，当抽屉处于打开位置的同时获取图像时，抽屉内的容纳物的图像可能更清晰(例如，较少被遮挡)，诸如，当抽屉处于打开位置时，可将抽屉向外延伸远离冰箱中的其他容纳物和结构(例如，搁架)，从而允许当抽屉处于打开位置时可以更清晰且更完整地观察抽屉的内部以及其中的容纳物。
在一些实施方式中，抽屉也可包括设置在并且穿过多个壁中的一个壁的通风孔。在此类实施方式中，摄像机组件可被配置成用于监测抽屉的通风孔，例如，通风孔可被设置在摄像机的视场内。此类实施方式也可包括确定第一食品和第二食品中的至少一者的最佳湿度水平并且确定在对应于所确定的最佳湿度的通风孔处的滑块的最佳位置。摄像机组件被定位和配置成用于监测抽屉的通风孔的示例性实施方式还可包括分析图像以确定在通风孔处的滑块是否处于最佳位置，以及当在通风孔处的滑块未处于最佳位置时发出用户通知，其中，用户通知可以是例如，如下文所详述的可听到的用户通知和/或可看到的用户通知，并且也可以如下文所详述的在本地和/或远程提供。
现在转到图10，本公开的实施方式可包括运行制冷电器(诸如上文所述的示例性制冷电器100)的方法500。例如，制冷电器可包括控制器和多个食品存储抽屉等，如上文所述。
方法500也包括类似于上文所述的步骤410和步骤420的图像获取步骤510以及分析和识别步骤520，并且为了简洁起见，不再重复此类描述。
与相对于示例性方法400的上文所述的预定阈值相比，方法500可包括为制冷电器中(诸如抽屉140中)的一种或多种大气状况定义新的阈值或附加的阈值。例如，一个或多个阈值可基于和/或响应于所识别的食品，诸如特定类型的农产品的预料或预期的乙烯水平，其中，预料或预期的乙烯水平对应于所识别的食品变成熟(或过成熟等)。因此，在一些实施方案中，示例性方法500可包括基于第一食品的识别来设定大气状况的第一阈值的步骤530以及基于第二食品的识别来设定大气状况的第二阈值的步骤540。在至少一些实施方式中，第一食品可以不同于第二食品，并且因此第一阈值也可以不同于第二阈值，虽然不同的食品可能不一定具有不同的阈值。
如图10所示，方法500还可包括监测大气状况的步骤550，其中，在步骤530和步骤540处设定大气状况的第一阈值和第二阈值。此类监测可至少部分由传感器执行，诸如由能够操作为与传感器通信的制冷电器的控制器执行，例如，示例性方法可包括使用传感器和/或由传感器监测大气状况。
仍然参考图10,方法500也可包括发出一个或多个用户通知的步骤。此类通知可在本地(例如，在制冷电器100的用户界面面板136上)发出和/或远程(诸如在未直接以物理方式附接或连接到制冷电器的远程设备上，例如，智能电话、智能家居系统或其他类似设备上)提供。用户通知可包括视觉通知(例如，照亮指示灯或提供文本通知)和/或可听到的通知(诸如铃声或警报音等)中的一种或多种通知。例如，方法500可包括当达到基于识别第一食品的第一大气状况阈值时发出第一用户通知的步骤560，以及当达到基于识别第二食品的第二大气状况阈值时发出第二用户通知的步骤570。因此，例如，可在方法500中提供定制化和响应式监测和库存管理，其中，基于对特定所识别的食品更明显或敏感的大气状况对每种食品进行单独且具体的跟踪。
在一些实施方式中，步骤530的第一大气状况阈值可包括第一乙烯水平并且步骤540的第二大气状况阈值可包括第二乙烯水平。
在一些实施方式中，传感器可能够操作为当抽屉处于关闭位置时检测抽屉的食品存储隔间内的大气状况。例如，当抽屉处于关闭位置时，控制器可发送或查询传感器，其中，关闭位置可由控制器基于位置开关或位置传感器(例如，霍尔效应传感器)和/或基于来自摄像机组件的图像进行检测，其中，控制器可分析此类图像以识别和检测抽屉何时处于关闭位置。有利的是，当抽屉处于关闭位置时，测量或检测大气状况可更精确地读取抽屉本身内(诸如在其中的食品存储隔间中)的大气，而不是抽屉的外部的环境状况，例如，在食品保鲜隔间的其余部分中和/或制冷电器的外部。
在一些实施方案中，摄像机组件可被配置成当抽屉处于打开位置时监测抽屉的食品存储隔间。例如，有利的是，当抽屉处于打开位置的同时获取图像时，抽屉内的容纳物的图像可能更清晰(例如，较少被遮挡)，诸如，当抽屉处于打开位置时，可将抽屉向外延伸远离冰箱中的其他容纳物和结构(例如，搁架)，从而允许当抽屉处于打开位置时可以更清晰且更完整地观察抽屉的内部。
在一些实施方式中，抽屉也可包括穿过多个壁中的一个壁的通风孔。在此类实施方式中，摄像机组件可被配置成用于监测抽屉的通风孔，例如，通风孔可被设置在摄像机的视场内。此类实施方式也可包括确定第一食品和第二食品中的至少一者的最佳湿度水平并且确定在对应于所确定的最佳湿度的通风孔处的滑块的最佳位置。此外，此类实施方式也可包括或代替地包括作为大气状况的第一阈值和第二阈值的湿度水平，例如，大气状况可以是湿度，并且第一阈值和第一阈值各自可以是湿度水平。摄像机组件被定位和配置成用于监测抽屉的通风孔的示例性实施方式还可包括分析图像以确定在通风孔处的滑块是否处于最佳位置，以及当在通风孔处的滑块未处于最佳位置时发出用户通知，其中，用户通知可以是例如，如上文所述的可听到的用户通知和/或可看到的用户通知，并且也可以如上文所述的在本地和/或远程提供。
现在转到图11，本发明的实施方式也可包括运行制冷电器(诸如上文所述的示例性制冷电器100)的方法600。例如，制冷电器可包括控制器和多个食品存储抽屉等，如上文所述。
方法600也包括类似于上文所述的步骤410/510和步骤420/520的图像获取步骤610以及分析和识别步骤620，并且为了简洁起见，不再重复此类描述。
方法600也可包括确定第一食品和第二食品共同存储不相容的步骤630。例如，此类不相容性可包括不同的最佳湿度水平和/或温度。又如，此类不相容性也可包括或代替地包括第一食品和第二食品中的一者产生乙烯，例如，当该食品成熟或老化时，该食品生成或释放大量乙烯(与其他农产品相比)，并且第一食品和第一食品中的另一者对乙烯敏感，例如，在暴露于由一种食品释放的乙烯水平可能加速另一种食品的老化速率的情况下。在此类实施方式中，不相容性的确定可至少部分基于第一食品和第二食品中的一者产生乙烯的速率。
在确定了第一食品和第二食品共同存储不相容之后，并且响应于此类确定，然后，方法600可包括提供用户通知的步骤640，该用户通知包括重新放置第一食品与第二食品中的一者的建议。例如，当制冷电器中包括一个以上的抽屉时，则建议可包括建议将一种食品移动到另一个抽屉。又如，建议可包括建议将一种食品移动到食品保鲜隔室的另一个部分，例如，一个或多个抽屉的外部，或可包括建议在室温下存储一种食品，例如，在制冷电器的外部。此外，不需要此类示例，例如，用户通知可仅提供将一种食品移出抽屉或进行重新定位的建议，而无需指定该一种食品应该移动到何处。
本书面描述使用示例来公开本发明，包括最佳实施方式，并且还使本领域的技术人员能够实施本发明，包括制造和使用任何设备或系统以及执行任何并入的方法。本发明的可授予专利的范围由权利要求书限定，并且可包括本领域的技术人员想到的其它示例。如果此类其它示例包括与权利要求书的字面语言没有区别的结构元素，或者如果此类其它示例包括与权利要求书的字面语言存在微小差别的等效结构元素，则此类其它示例旨在落入权利要求书的范围内。

Claims (18)

  1. 一种操作制冷电器的方法，其特征在于，所述制冷电器包括具有食品存储室的机柜，所述食品存储室设置有可滑动地安装在所述食品存储室内的抽屉，所述抽屉可在关闭位置和打开位置之间滑动，所述抽屉包括多个限定食品存储隔间的壁、用于检测所述抽屉的所述食品存储隔间内的大气状况的传感器，以及被设置成用于监测所述抽屉的摄像机组件，所述方法包括：
    使用所述摄像机组件获取图像；
    分析所述图像以识别所述抽屉的所述食品存储隔间中的第一食品和第二食品；
    使用所述传感器检测到所述食品存储隔间内的大气状况高于预定阈值；以及
    基于对所述图像的分析，将所述第一食品和所述第二食品中的一者识别为造成所述大气状况高于所述预定阈值的来源。
  2. 根据权利要求1所述的方法，其特征在于，所述大气状况包括乙烯水平。
  3. 根据权利要求1所述的方法，其特征在于，所述传感器能够操作为当所述抽屉处于所述关闭位置时检测所述抽屉的所述食品存储隔间内的所述大气状况。
  4. 根据权利要求1所述的方法，其特征在于，所述摄像机组件被设置成当所述抽屉处于所述打开位置时监测所述抽屉的所述食品存储隔间。
  5. 根据权利要求1所述的方法，其特征在于，所述抽屉还包括穿过所述多个壁中的一个壁的通风孔，所述摄像机组件被配置成用于监测所述抽屉的所述通风孔。
  6. 根据权利要求5所述的方法，其特征在于，所述方法还包括确定所述第一食品和所述第二食品中的至少一者的最佳湿度水平，确定在对应于所述所确定的最佳湿度，所述通风孔处的滑块的最佳位置，分析所述图像以判断在所述通风孔处的所述滑块是否处于所述最佳位置，以及当在所述通风孔处的所述滑块未处于所述最佳位置时发出用户通知。
  7. 一种操作制冷电器的方法，其特征在于，所述制冷电器包括具有食品存储隔室的机柜，所述食品存储隔室带有可滑动地安装在所述食品存储隔室内的抽屉，所述抽屉可在关闭位置和打开位置之间滑动，所述抽屉包括多个限定食品存储隔间的壁、用于检测所述抽屉的所述食品存储隔间内的大气状况的传感器，以及被设置成用于监测所述抽屉的摄像机组件，所述方法包括：
    使用所述摄像机组件获取图像；
    分析所述图像以识别所述抽屉的所述食品存储隔间中的第一食品和第二食品；
    基于识别所述第一食品，设定大气状况的第一阈值；
    基于识别所述第二食品，设定所述大气状况的第二阈值；
    由所述传感器监测所述大气状况；
    当所述大气状况达到所述第一阈值时，发出第一用户通知；以及
    当所述大气状况达到所述第二阈值时，发出第二用户通知。
  8. 根据权利要求7所述的方法，其特征在于，所述第一阈值包括第一乙烯水平并且所述第二阈值包括第二乙烯水平。
  9. 根据权利要求7所述的方法，其特征在于，所述传感器被配置为：当所述抽屉处于所述关闭位置时，检测所述抽屉的所述食品存储隔间内的所述大气状况。
  10. 根据权利要求7所述的方法，其特征在于，所述摄像机组件被配置成当所述抽屉处于所述打开位置时，监测所述抽屉的所述食品存储隔间。
  11. 根据权利要求7所述的方法，其特征在于，所述抽屉还包括穿过所述多个壁中的一个壁的通风孔，并且其中，所述摄像机组件被配置成用于监测所述抽屉的所述通风孔。
  12. 根据权利要求11所述的方法，其特征在于，所述方法还包括：确定所述第一食品和所述第二食品中的至少一者的最佳湿度水平，确定在对应于所述所确定的最佳湿度的所述通风孔处的滑块的最佳位置，分析所述图像以判断在所述通风孔处的所述滑块是否处于所述最佳位置，以及当在所述通风孔处的所述滑块未处于所述最佳位置时发出用户通知。
  13. 一种运行制冷电器的方法，其特征在于，所述制冷电器包括具有食品存储隔室的机柜，所述食品存储隔室带有可滑动地安装在所述食品存储隔室内的抽屉，由此所述抽屉可在关闭位置和打开位置之间滑动，所述抽屉包括多个限定食品存储隔间的壁、用于检测所述抽屉的所述食品存储隔间内的大气状况的传感器，以及被配置成用于监测所述抽屉的摄像机组件，所述方法包括：
    使用所述摄像机组件获取图像；
    分析所述图像以识别所述抽屉的所述食品存储隔间中的第一食品和第二食品；
    确定所述第一食品和所述第二食品共同存储不相容；以及
    发出包括重新放置所述第一食品和所述第二食品中的一者的建议的用户通知。
  14. 根据权利要求13所述的方法，其特征在于，“确定所述第一食品和所述第二食品共同存储不相容”至少部分基于所述第一食品和所述第二食品中的一者产生乙烯的速率。
  15. 根据权利要求13所述的方法，其特征在于，所述抽屉还包括穿过所述多个壁中的一个壁的通风孔，所述摄像机组件被配置成用于监测所述抽屉的所述通风孔。
  16. 根据权利要求15所述的方法，其特征在于，所述方法还包括确定所述第一食品和所述第二食品中的另一者的最佳湿度水平，确定在对应于所述所确定的最佳湿度的所述通风孔处的滑块的最佳位置，分析所述图像以判断在所述通风孔处的所述滑块是否处于所述最佳位置，以及当在所述通风孔处的所述滑块未处于所述最佳位置时发出用户通知。
  17. 根据权利要求13所述的方法，其特征在于，所述传感器被配置为当所述抽屉处于所述关闭位置时检测所述抽屉的所述食品存储隔间内的所述大气状况。
  18. 根据权利要求13所述的方法，其特征在于，所述摄像机组件被配置成当所述抽屉处于所述打开位置时监测所述抽屉的所述食品存储隔间。
PCT/CN2023/070499 2022-01-06 2023-01-04 带有智能抽屉的制冷电器 WO2023131202A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US17/569,646 US11965691B2 (en) 2022-01-06 2022-01-06 Refrigerator appliance with smart drawers
US17/569,646 2022-01-06

Publications (1)

Publication Number Publication Date
WO2023131202A1 true WO2023131202A1 (zh) 2023-07-13

Family

ID=87073210

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/070499 WO2023131202A1 (zh) 2022-01-06 2023-01-04 带有智能抽屉的制冷电器

Country Status (2)

Country Link
US (1) US11965691B2 (zh)
WO (1) WO2023131202A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106123470A (zh) * 2016-06-20 2016-11-16 青岛海尔股份有限公司 检测冰箱储物间室内食物新鲜度的方法
CN108061415A (zh) * 2017-11-29 2018-05-22 深圳市赛亿科技开发有限公司 一种智能冰箱及基于智能冰箱食品管理的方法
CN108151408A (zh) * 2016-12-05 2018-06-12 彭州市运达知识产权服务有限公司 一种智能冰箱
CN110017659A (zh) * 2017-12-28 2019-07-16 Bsh家用电器有限公司 家用制冷器具设备
WO2021176445A1 (en) * 2020-03-02 2021-09-10 Nuversys Ltd. A stable food-grade microcapsule for the delivery of unstable and food-incompatible active ingredients to food products

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7296422B2 (en) * 2004-03-30 2007-11-20 Whirlpool Corporation Produce preservation system
US7685934B2 (en) * 2005-10-25 2010-03-30 Lg Electronics Inc. Refrigerator and method for keeping food using the same
US8820864B2 (en) * 2008-11-14 2014-09-02 Electrolux Home Products, Inc. Refrigerator drawers with trim
US20140139088A1 (en) * 2012-11-21 2014-05-22 Whirlpool Corporation Transparent touch displays for refrigerator drawers
US20140284239A1 (en) * 2013-03-19 2014-09-25 Jeffrey S. Melcher Method for perishable food or item in a container with a container storage technology
JP2016148503A (ja) 2015-02-05 2016-08-18 日本電産コパル株式会社 冷蔵庫、冷蔵庫管理方法及びプログラム
US10373472B2 (en) * 2016-03-14 2019-08-06 Amazon Technologies, Inc. Scent-based spoilage sensing refrigerator
US10281200B2 (en) * 2016-03-14 2019-05-07 Amazon Technologies, Inc. Image-based spoilage sensing refrigerator
KR102327848B1 (ko) * 2017-05-18 2021-11-18 삼성전자주식회사 냉장고 및 냉장고의 음식 관리방법
DE102017210789A1 (de) 2017-06-27 2018-12-27 BSH Hausgeräte GmbH Verfahren zum Einstellen einer Molekülkonzentration in einem Lagerbereich für Lebensmittel eines Haushaltskältegeräts, sowie Haushaltskältegerät
US10655907B2 (en) * 2017-12-15 2020-05-19 International Business Machines Corporation Content and context aware microscopic cooling optimization for refrigerators
EP3973233A1 (en) 2019-05-20 2022-03-30 Arçelik Anonim Sirketi A cooling appliance having an ethylene absorption system
KR102234691B1 (ko) * 2019-08-09 2021-04-02 엘지전자 주식회사 인공 지능을 이용하여, 물품을 관리하는 냉장고 및 그의 동작 방법
US11340011B2 (en) * 2019-11-05 2022-05-24 Electrolux Home Products, Inc. Refrigerator drawer with cassette filter

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106123470A (zh) * 2016-06-20 2016-11-16 青岛海尔股份有限公司 检测冰箱储物间室内食物新鲜度的方法
CN108151408A (zh) * 2016-12-05 2018-06-12 彭州市运达知识产权服务有限公司 一种智能冰箱
CN108061415A (zh) * 2017-11-29 2018-05-22 深圳市赛亿科技开发有限公司 一种智能冰箱及基于智能冰箱食品管理的方法
CN110017659A (zh) * 2017-12-28 2019-07-16 Bsh家用电器有限公司 家用制冷器具设备
WO2021176445A1 (en) * 2020-03-02 2021-09-10 Nuversys Ltd. A stable food-grade microcapsule for the delivery of unstable and food-incompatible active ingredients to food products

Also Published As

Publication number Publication date
US11965691B2 (en) 2024-04-23
US20230228481A1 (en) 2023-07-20

Similar Documents

Publication Publication Date Title
KR20200034903A (ko) 냉장고 내 객체의 상태와 관련된 정보를 제공하는 방법 및 시스템
US7190809B2 (en) Enhanced background model employing object classification for improved background-foreground segmentation
US20200234079A1 (en) Methods and systems for processing image data
US11335010B2 (en) Methods for viewing and tracking stored items
WO2021057769A1 (zh) 用于查看并跟踪所储存的物品的方法
US20220325946A1 (en) Selective image capture using a plurality of cameras in a refrigerator appliance
JP2021165604A (ja) 保管容器、冷蔵庫及び熟成度推定装置
US11692769B2 (en) Inventory management system for a refrigerator appliance
CN109631484A (zh) 冷藏库
CN107527363B (zh) 一种冷藏装置存储物管理系统和冷藏装置
WO2023131202A1 (zh) 带有智能抽屉的制冷电器
CN113124635B (zh) 冰箱
CN107526991B (zh) 一种冷藏装置存储物管理系统和冷藏装置
JP7113281B2 (ja) 冷蔵庫
US20230076984A1 (en) Inventory management system in a refrigerator appliance
US20220414391A1 (en) Inventory management system in a refrigerator appliance
WO2023151694A1 (zh) 具有智能门体警报的制冷电器
CN111488831B (zh) 一种食材联想识别方法及冰箱
US20240068731A1 (en) Refrigerator appliance with image-assisted odor control
US20230308611A1 (en) Multi-camera vision system in a refrigerator appliance
US20230375258A1 (en) Refrigerator appliances and image-based methods of detecting a door position
US20240068742A1 (en) Gasket leak detection in a refrigerator appliance
US11796250B1 (en) Multi-camera vision system facilitating detection of door position using audio data
US20230097905A1 (en) Inventory management system in a refrigerator appliance
KR20220011450A (ko) 물품을 식별하는 장치, 서버 및 식별하는 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23737065

Country of ref document: EP

Kind code of ref document: A1