WO2023185834A1 - 冰箱相机模块和用于解决相机透镜上的持续状况的方法 - Google Patents

冰箱相机模块和用于解决相机透镜上的持续状况的方法 Download PDF

Info

Publication number
WO2023185834A1
WO2023185834A1 PCT/CN2023/084378 CN2023084378W WO2023185834A1 WO 2023185834 A1 WO2023185834 A1 WO 2023185834A1 CN 2023084378 W CN2023084378 W CN 2023084378W WO 2023185834 A1 WO2023185834 A1 WO 2023185834A1
Authority
WO
WIPO (PCT)
Prior art keywords
camera module
refrigeration appliance
camera
refrigeration
fog
Prior art date
Application number
PCT/CN2023/084378
Other languages
English (en)
French (fr)
Inventor
古德曼 施罗德迈克尔
凯里亚库斯蒂芬诺斯
Original Assignee
海尔智家股份有限公司
青岛海尔电冰箱有限公司
海尔美国电器解决方案有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 海尔智家股份有限公司, 青岛海尔电冰箱有限公司, 海尔美国电器解决方案有限公司 filed Critical 海尔智家股份有限公司
Publication of WO2023185834A1 publication Critical patent/WO2023185834A1/zh

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N17/00—Diagnosis, testing or measuring for television systems or their details
    • H04N17/002—Diagnosis, testing or measuring for television systems or their details for television cameras
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
    • F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
    • F25D11/00—Self-contained movable devices, e.g. domestic refrigerators
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
    • F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
    • F25D29/00—Arrangement or mounting of control or safety devices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/183—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/188—Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
    • F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
    • F25D2400/00—General features of, or devices for refrigerators, cold rooms, ice-boxes, or for cooling or freezing apparatus not covered by any other subclass
    • F25D2400/36—Visual displays
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
    • F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
    • F25D2700/00—Means for sensing or measuring; Sensors therefor
    • F25D2700/02—Sensors detecting door opening
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30168—Image quality inspection

Definitions

  • the present invention generally relates to systems and methods for preventing persistent conditions from obscuring the view of cameras, particularly cameras in refrigeration appliances.
  • Storage enclosures such as refrigeration appliances and food storage cabinets often provide enclosed chambers for receiving multiple items or objects.
  • refrigeration appliances often include a box defining a refrigeration compartment. Users can place food or objects in the refrigerated chamber to prevent the food from spoiling. Thus, the useful life of the perishable item or object can be increased.
  • a refrigeration appliance may include a box, a door, a camera module and a controller.
  • the box can define a refrigeration compartment.
  • the door may be rotatably hinged to the cabinet to provide selective access to the refrigeration compartment.
  • the camera module can be mounted to the cabinet inside the refrigeration room.
  • the controller can be operably coupled to the camera module.
  • the controller can be configured to initiate operating routines.
  • the operating routine may include receiving one or more images for each of the plurality of capture sequences, detecting a prevailing blur condition at the camera module based on the received one or more images, and responding to detecting the prevailing blur condition And guide the response action on the refrigeration appliance.
  • a method of operating a refrigeration appliance may include receiving one or more images for each of the plurality of capture sequences. The method may also include detecting a prevalent blur condition at the camera module based on the received one or more images. The method may further include directing a responsive action on the refrigeration appliance in response to detecting the prevailing blur condition.
  • the refrigeration appliance may include a box, a door, a camera module and a controller.
  • the box can define a refrigeration compartment.
  • the door may be rotatably hinged to the cabinet to provide selective access to the refrigeration compartment.
  • the camera module can be mounted to the cabinet inside the refrigeration room.
  • the controller can be operably coupled to the camera module.
  • the controller can be configured to initiate operating routines.
  • the operating routine may include receiving a first image from the camera module, detecting a blur condition at the camera module based on the received first image, initiating an anti-fog capture sequence at the camera module, detecting a door opening event, receiving from the camera during the door opening event a second image, re-evaluating the blur condition based on the received second image, and directing a response action based on the re-evaluation.
  • Figure 1 provides a front elevation view of a refrigeration appliance according to an exemplary embodiment of the invention.
  • Figure 2 provides a front elevation view of a refrigeration appliance according to an exemplary embodiment of the present invention, wherein the refrigeration appliance The door is shown in the open position.
  • Figure 3 provides a schematic diagram of a refrigeration appliance according to an exemplary embodiment of the invention.
  • Figure 4 provides a schematic cross-sectional view of a camera module of a refrigeration appliance according to an exemplary embodiment of the present invention.
  • Figure 5 provides a schematic cross-sectional view of a camera module of a refrigeration appliance according to other exemplary embodiments of the present invention.
  • Figure 6 provides a flowchart illustrating a method of operating a refrigeration appliance according to an exemplary embodiment of the present invention.
  • Figure 7 provides a flowchart illustrating a method of operating a refrigeration appliance according to an exemplary embodiment of the present invention.
  • the term “or” is generally intended to be inclusive (ie, “A or B” is intended to mean “A or B or both”).
  • the terms “first,” “second,” and “third” may be used interchangeably to distinguish one component from another component, and these terms are not intended to denote the position or importance of the various components.
  • approximate terms such as “approximately,” “approximately” or “approximately” mean within a ten percent margin of error.
  • the present invention provides systems and methods for detecting and resolving when a camera module's lens is covered in a low temperature environment, such as a refrigerator compartment. For example, accumulated condensation, dust, or dirt may blur the camera.
  • Systems and methods automatically detect these conditions without any direct user input or knowledge.
  • systems and methods may be used to mitigate such conditions (eg, without direct user input) or to instruct users how such conditions may be resolved.
  • FIG. 1 provides a front elevation view of a refrigeration appliance 100 according to an exemplary embodiment of the present invention, with the refrigeration door 128 of the refrigeration appliance 100 shown in a closed position.
  • FIG. 2 provides a front elevation view of the refrigeration appliance 100 in which the refrigeration door 128 is shown in an open position to expose the food preservation compartment 122 of the refrigeration appliance 100 .
  • the refrigeration appliance 100 includes a housing or box 120 extending in a vertical direction V between a top 101 and a bottom 102 .
  • Box 120 defines a refrigerated compartment for receiving food for storage.
  • the box 120 defines a food preservation chamber 122 arranged at or adjacent to the top 101 of the box 120 and a freezing chamber 124 arranged at or adjacent to the bottom 102 of the box 120 .
  • the refrigeration appliance 100 is generally called a bottom-mounted refrigerator.
  • the benefits of the present invention are applicable to other types and styles of storage enclosures, such as overhead refrigeration appliances, side-by-side refrigeration appliances, and the like. Accordingly, the description set forth herein is for illustrative purposes only and is not intended to be limiting in any respect to any particular storage enclosure or refrigeration chamber configuration.
  • the refrigeration door 128 is rotatably hinged to the edge of the box 120 to selectively enter the food preservation compartment 122 .
  • a freezing door 130 is arranged below the refrigeration door 128 to selectively enter the freezing chamber 124 .
  • Freezer door 130 is coupled to a freezer drawer 142 (not shown) slidably mounted within freezer compartment 124 .
  • the refrigeration door 128 and the freezer door 130 are shown in a closed position in FIG. 1
  • the refrigeration door 128 is shown in an open position in FIG. 2 .
  • various storage components are installed within the food preservation compartment 122 to facilitate storage of food items therein.
  • the storage components include boxes 140, drawers 142, and shelves 144 installed in the food preservation compartment 122. Boxes 140, drawers 142, and shelves 144 are configured to receive stored items (eg, beverages or solid food items) and can help organize such food items.
  • stored items eg, beverages or solid food items
  • drawer 142 may receive fresh food (eg, vegetables, fruit, or cheese) and increase the useful life of such fresh food.
  • the refrigeration appliance 100 also includes features for assisting the user in identifying food products located within the food preservation compartment 122 or the freezer compartment 124 .
  • a user may utilize these features, for example, to view food items (i.e., stored items) stored in the food freshness compartment 122 or the freezer compartment 124, or to create an inventory of such stored items.
  • FIG. 3 provides a schematic diagram of a refrigeration appliance 100 .
  • the refrigeration appliance 100 includes a controller 150 operably coupled or in communication with components of a refrigeration system of the refrigeration appliance 100 that is configured to cool the food freshness compartment 122 or the freezer compartment 124 .
  • Components include compressor 170, evaporator fan 172, and condenser fan 174.
  • the controller 150 can selectively operate such components to cool the food freshness compartment 122 or the freezer compartment 124 .
  • Controller 150 also communicates with a thermostat (eg, a thermocouple or thermistor).
  • the thermostat may be disposed in the food freshness compartment 122 or the freezer compartment 124 .
  • the controller 150 may receive a signal corresponding to the temperature of the food freshness compartment 122 or the freezer compartment 124 from the thermostat.
  • Controller 150 may also include an internal timer for counting elapsed time periods.
  • Controller 150 may include memory and one or more microprocessors, CPUs, etc., such as general purpose or special purpose microprocessors.
  • a processor configured to execute programming instructions or microcontrol code associated with the operation of the refrigeration appliance 100 .
  • Memory may represent random access memory such as DRAM or read-only memory such as ROM or FLASH.
  • the processor executes persistent programming instructions stored in memory.
  • the instructions include a software package configured to operate the appliance 100, or to perform an operating routine (eg, example method 600 or 700 described below with reference to Figure 6 or Figure 7).
  • the memory may be a separate component from the processor or may be included on a board within the processor.
  • the controller 150 may be constructed without the use of a microprocessor (e.g., using a combination of independent analog or digital logic circuits; such as switches, amplifiers, integrators, comparators, flip-flops, AND gates, etc.) To perform control functions rather than relying on software.
  • a microprocessor e.g., using a combination of independent analog or digital logic circuits; such as switches, amplifiers, integrators, comparators, flip-flops, AND gates, etc.
  • the controller 150 may be disposed at various locations throughout the refrigeration appliance 100 .
  • Input/output (“I/O”) signals may be routed between controller 150 and various operating components of refrigeration appliance 100 .
  • One or more components of refrigeration appliance 100 may communicate (eg, electrically) with controller 150 via one or more conductive signal lines or a shared communications bus. Additionally or alternatively, one or more components of refrigeration appliance 100 may communicate (eg, wirelessly) with controller 150 via one or more wireless signal bands.
  • refrigeration appliance 100 also includes a camera or camera module 160 .
  • Camera 160 may be any type of device suitable for capturing two-dimensional photographs or images.
  • camera 160 may be a video camera or a digital camera with an electronic image sensor, such as a charge coupled device (CCD) or CMOS sensor.
  • CCD charge coupled device
  • CMOS complementary metal-oxide-semiconductor
  • controller 150 When assembled, camera 160 communicates (eg, electrically or wirelessly) with controller 150 such that controller 150 can receive signals from camera 160 corresponding to images captured by camera 160 .
  • the camera 160 is provided on the refrigeration appliance 100 .
  • camera 160 is mounted within food preservation compartment 122 at its top (eg, near top 101 and away from bottom 102 ).
  • camera 160 may be secured to or directed through the top wall of the liner defining food preservation compartment 122 .
  • camera 160 may be pointed downward.
  • camera 160 is directed toward one or more refrigeration compartments (eg, food preservation compartment 122).
  • camera 160 may be pointed at at least a portion of any particular one or combination of drawers 142 and shelves 144 .
  • the camera 160 may capture images of one of the drawers 142, all of the drawers 142, one of the shelves 144, all of the shelves 144, or any suitable combination thereof.
  • a camera 160 including a circuit board 162 and its main lens 164 is housed in a camera housing 210 .
  • camera housing 210 extends around at least a portion of camera 160 (eg, over liner 123).
  • housing sidewall 212 may extend from liner 123 and around camera 160 (eg, such that the camera is radially surrounded by housing sidewall 212).
  • housing sidewall 212 may extend from top end 214 to bottom end 216 . If installed on top of liner 123, bottom end 216 may be disposed at Below the lowermost surface of camera 160 (eg, as defined at main lens 164).
  • Front wall 218 extends radially inwardly from housing sidewall 212 (eg, at bottom end 216).
  • the front wall 218 may define an opening or aperture 220 along or aligned with the line of sight LS of the camera 160 such that the view of the camera 160 is generally unobstructed and the camera 160 is able to collect images of the food preservation chamber 122 through the aperture 220 .
  • FIG. 4 illustrates aperture 220 as an open void
  • FIG. 5 illustrates that aperture 220 may be covered by transparent secondary lens 224 spaced apart from primary lens 164 to close or seal camera housing 210 to prevent or restrict camera 160 Fluid communication between the refrigeration compartment (eg, food preservation compartment 122) in which it is installed.
  • the refrigeration compartment eg, food preservation compartment 122
  • front wall 218 is spaced (eg, vertically) from camera 160.
  • camera housing 210 When assembled, camera housing 210 generally defines an open volume 222 surrounding camera 160 .
  • Camera housing 210 may be formed from a solid, impermeable material, such as a suitable polymer or metal.
  • heat e.g., due in part to circuit resistance as current is conducted to/from camera 160
  • this heat may be held or trapped within open volume 222 , thereby advantageously preventing Condensation accumulates on the primary lens 164 or the auxiliary lens 224.
  • refrigeration appliance 100 includes an integrated display 180 .
  • the integrated display 180 may be mounted on the refrigeration door 128 or any other suitable location on the refrigeration appliance 100 .
  • the integrated display 180 communicates with the controller 150 such that the integrated display 180 can receive signals corresponding to images captured by the camera 160 from the controller 150 .
  • Integrated display 180 may receive such signals from controller 150 and visually present images to the user.
  • Integrated display 180 may include, for example, a liquid crystal display panel (LCD), a plasma display panel (PDP), or any other suitable mechanism for displaying images (eg, a projector).
  • LCD liquid crystal display panel
  • PDP plasma display panel
  • the refrigeration appliance 100 includes a network interface (not shown) that couples the refrigeration appliance 100 (eg, controller 150 ) to the network 190 such that the refrigeration appliance 100 can transmit over the network 190 and receive information.
  • Network 190 may be any wired or wireless network, such as a WAN, LAN, or HAN.
  • refrigeration appliance 100 communicates with mobile display 182 via network 190.
  • Mobile display 182 may be any device configured to communicate over network 190 and display images received therefrom.
  • mobile display 182 may be a computer, smartphone, or tablet.
  • Mobile display 182 communicates with controller 150 such that mobile display 182 can receive signals from controller 150 (via network 190 ) corresponding to the user interface or images captured by camera 160 .
  • Mobile display 182 may receive such signals from controller 150 and visually present one or more images to the user.
  • the mobile display 182 may include, for example, a liquid crystal display panel (LCD), a plasma display panel (PDP), or a display panel for displaying images. Any other suitable mechanism (e.g. projector).
  • Mobile display 182 may also include an interface (eg, tactile input, such as buttons, or a graphical user interface) that allows mobile display 182 to initiate communication with refrigeration appliance 100 over network 190 .
  • an interface eg, tactile input, such as buttons, or a graphical user interface
  • refrigeration appliance 100 includes a detection component 186 (eg, to detect the presence of a user).
  • detection component 186 may include one or more sensors configured to directly detect the presence of a user. (e.g., by detecting biometric or personalized markers corresponding to a specific individual user) or indirectly (e.g., by detecting movement of door 128 or 130).
  • detection component 186 may include a forward-facing camera configured to identify a user's face based on a captured two-dimensional image.
  • detection component 186 may include a fingerprint imaging sensor configured to visually detect a user's fingerprint.
  • the detection component 186 may include a signal detection sensor configured to detect in a wireless communication band (eg, a BLE band using short wavelength UHF radio waves in the ISM band of 2.4 to 2.485 GHz).
  • Device address may be, for example, a programmed Bluetooth address of the mobile display 182 .
  • the detection component 186 can determine if and when the mobile display 182 is in close proximity to the refrigeration appliance 100 .
  • the detection component 186 may include a door switch (e.g., a reed switch, a push rod switch, a Hall effect sensor, etc.) that selectively engages the refrigerated door 128 to detect whether the door 128 /when in the open position.
  • a door switch e.g., a reed switch, a push rod switch, a Hall effect sensor, etc.
  • Such a switch is generally understood and may, for example, simultaneously control the activation of a light for illuminating the food preservation compartment 122 .
  • opening the refrigeration door 128 may activate the light and transmit a signal indicative of the user's detection.
  • camera 160 may capture one or more two-dimensional images (e.g., as a video feed or a series of images) that may be transmitted to controller 150 (e.g., as a data signal). Sequential static images), as commonly understood. For each image capture, bursts or pulses of heat may be generated by the camera 160 from resistors that carry current or voltage through various electrical components or circuits of the camera. Thus, bursts or pulses of heat may generally correspond to samples or frame rates of a particular image capture sequence. The specific image capture sequence or further analysis of the received image signals at controller 150 may change or alternate (eg, based on one or more conditions of refrigeration appliance 100).
  • a continuous anti-fog capture sequence may direct camera 160 to collect and transmit images (e.g., sequentially at a set sampling rate, such as in frames defined per second).
  • images will be static (e.g., within the field of view of camera 160 The object will not be moving, or the light within the food compartment 122 will not be sufficient to collect any image other than a blank image).
  • the blank or static image may be discarded (eg, at controller 150).
  • a door opening capture sequence may direct the camera 160 to collect and transmit images (e.g., sequentially at a set sampling rate that is different from the sampling rate of the continuous anti-fog capture sequence) of one or more doors simultaneously. 128 open or the user otherwise exists. Typically, such an image will be occupied (eg, have information or data values for objects or lights within the field of view of camera 160).
  • the monitor sequence may direct the camera 160 to collect and transmit images even when the door 128 is closed or no user is otherwise present (e.g., in less time than a continuous anti-fog capture sequence or a door opening sequence)
  • the sampling rate is set (sampling rate sequentially).
  • images will be static (eg, objects within the field of view of camera 160 will not be moving, or the light within food compartment 122 will not be sufficient to collect any image other than a blank image).
  • the blank or static image may be discarded (eg, at controller 150).
  • a particular image (eg, an occupied image) may be recorded (eg, at least temporarily at the controller 150 ) so that the occupied image may be transmitted for presentation (eg, on an integrated display 180 or remote display 182).
  • items within the field of view (eg, set field of view) of camera 160 may be automatically identified by the controller 150 (eg, to catalog items within the food preservation compartment 122).
  • identifying such items may be through edge matching, divide and conquer search, grayscale matching, histograms of receptive field responses, or another suitable routine (e.g., based on one or more images captured from camera 160 in (executed at the controller 150).
  • the controller 150 may identify one or more blurred portions (eg, portions of the image that are blurred, confused, or blocked out of focus). This blurred portion may indicate condensation, dust, or dirt that has accumulated on the lens of camera 160 . Identifying such items may be through edge matching, divide and conquer search, grayscale matching, histograms of receptive field responses, or another suitable routine (e.g., performed at controller 150 based on one or more images captured from camera 160 ) to execute.
  • This blurred portion may indicate condensation, dust, or dirt that has accumulated on the lens of camera 160 . Identifying such items may be through edge matching, divide and conquer search, grayscale matching, histograms of receptive field responses, or another suitable routine (e.g., performed at controller 150 based on one or more images captured from camera 160 ) to execute.
  • one or more artificial intelligence or machine learning models may be applied to the blurred portion such that the controller 150 can distinguish, for example, fog conditions (e.g., corresponding to accumulation on the lens of the camera 160 and temporary condensation) and static debris conditions (e.g., corresponding to accumulated dust, dirt, or other generally static foreign matter that may accumulate on the lens of camera 160 ).
  • the controller 150 may use images from multiple independent events (e.g., door opening events) to determine conditions or identify blurred portions (e.g., based on specific blurring events). any changes over time).
  • changes in the shape or color of the blurred portion over time may indicate fog conditions as opposed to static debris conditions, which may correspond to substantially unchanged blurred portions.
  • the blurry portion may cause a general blurry condition that remains constant over time (eg, across multiple independent door opening events).
  • camera 160 may be directed to one or more motion responses based on detection of partial blur or general blur conditions.
  • the camera 160 may be selectively directed to a continuous anti-fog capture sequence and a door-opening capture sequence such that the camera 160 remains operational to capture images at regular intervals (e.g., based on any one of the image capture sequences being performed at a given moment in time). sampling rate) generates heat.
  • this heat generation may prevent condensation or fog from accumulating within the line of sight LS of the camera 160 (eg, at the lens or auxiliary lens 224) and blurring the images captured by the camera 160.
  • the anti-fog capture sequence may be initiated based on detection of a partially blurred or pervasive blur condition that is determined to be or potentially be (eg, within a set probability percentage) a fog condition.
  • the door opening capture sequence may be initiated (e.g., thereby stopping or interrupting a continuous anti-fog capture sequence) by a predetermined engagement action, such as opening door 128, detecting a refrigeration compartment (e.g., For example, movement within the food preservation compartment 122), user engagement or movement within the range of the detection component 186, providing user input at the user interface or integrated display 180, etc.
  • the anti-fog capture sequence (eg, a second anti-fog capture sequence) may be initiated by a separate predetermined action or in the absence of an engaged action that initiates the door-opening capture sequence (eg, thereby stopping or interrupting the door-opening capture sequence) .
  • the anti-fog capture sequence (e.g., the first or second anti-fog capture sequence) may be stopped or paused in response to a blur condition that is determined to be or potentially is (e.g., in a set Within the probability percentage) static debris conditions.
  • camera 160 and controller 150 are configured to capture multiple (eg, a pair) sequential two-dimensional images (eg, at a set sampling rate) as an anti-fog capture sequence or as a separate monitor part of the sequence.
  • each two-dimensional image includes a plurality of pixels (eg, arranged in a predefined grid).
  • Sequential images e.g., a previously captured image and a most recently captured image
  • the same field of view or line of sight e.g., the same area of the food preservation room 122
  • sequential images may be compared to each other or to a baseline value/set of values (eg, of pixel brightness or color). Changes in sequential images or changes from a baseline value/set of values can be detected to prompt a new sequence of image captures. Specifically, a detected change, such as an elevated image value, may prompt the controller 150 to alternate from a continuous anti-fog sequence to an open door capture sequence. Elevated image values may be detected by any suitable comparison or pixel characteristic, such as a brightness value or color value that may indicate a change in brightness value or detection of movement (eg, from corresponding pixels in a sequential two-dimensional image).
  • the brightness range may be, for example, an RGB brightness range between 0 and 255 (eg, where "0" is the minimum RGB pixel brightness value and "255" is the maximum RGB pixel brightness value).
  • the brightness values of multiple pixels may be detected in order to measure brightness. For example, the average brightness (Bm) value of the pixel brightness values can be calculated for the corresponding two-dimensional image (or a sub-region thereof). Additionally or alternatively, deviation values for a plurality of pixels may be calculated (eg, as brightness values) compared to the equilibrium value.
  • the brightness value (Vbr) of the captured image may be calculated as the absolute value of the difference between the average brightness value minus the balance value (eg, 125) divided by the balance value.
  • the brightness value can be expressed as:
  • Vbr (
  • an elevated brightness value may prompt a door opening capture sequence.
  • an increased brightness value may be an increase in brightness value between one or more pixels in the first captured image and one or more corresponding pixels in the second captured image.
  • the corresponding captured image has a lower brightness value.
  • the light from inside the food preservation chamber 122 can make the brightness value of the corresponding captured image higher.
  • a detected increase in brightness value eg, an increase in a set amount, relative percentage, etc.
  • methods 600 and 700 provide a method of operating refrigeration appliance 100 (FIG. 1), which includes camera 160, as described above.
  • Method 600 or 700 may be performed, for example, by controller 150 (FIG. 3).
  • controller 150 may communicate with camera 160 or detection component 186 (Fig. 3).
  • controller 150 may send and receive signals to and from camera 160 or detection component 186 .
  • controller 150 may also communicate with other suitable components of appliance 100 to facilitate operation of appliance 100 .
  • the method according to the invention can effectively prevent condensation within a portion of the refrigeration compartment (eg without the need or use of dedicated heaters or heating elements).
  • the disclosed methods may allow the camera's own heat generation to constantly prevent the accumulation of condensation (eg, on the lens of the camera) while the cooling appliance is operating.
  • the present method may facilitate efficient and effective management of captured images (eg, so that the controller is not overwhelmed by stored or received data).
  • FIGS. 6 and 7 depict steps performed in a specific order for purposes of illustration and discussion. Using the disclosure provided herein, one of ordinary skill in the art will understand that the methods are not mutually exclusive and that the steps of any method disclosed herein may be modified, adapted, rearranged, omitted, or expanded in various ways without departing from scope of the invention (unless otherwise described or required).
  • method 600 includes receiving one or more images for each of a plurality of capture sequences.
  • at least one image may be received from multiple independent capture sequences.
  • Individual capture sequences can be performed at different points in time.
  • each image capture sequence may correspond to a unique door opening event or other sequence in which the captured image is occupied (ie, not blank or completely black).
  • method 600 includes initiating each of a plurality of capture sequences in response to independent door opening events.
  • one or more (eg, each or some) of the plurality of capture sequences may be a gated capture sequence.
  • independent door opening events may be detected, for example, based on image values or signals from individual sensors or switches.
  • independent door opening events for each of the plurality of capture sequences may be detected based on elevated image values in the received still image signal. Additionally or alternatively, independent door opening events for each of the plurality of capture sequences may be detected based on receiving an opening signal from the door body switch.
  • a door-opening capture sequence can direct images to be captured sequentially at the camera module.
  • the two-dimensional image may be captured at a set sampling rate greater than that of the anti-fog capture sequence.
  • the sampling rate of the door opening capture sequence can thus be higher, such as between 25 frames per second and 120 frames per second. Additionally or alternatively, the sampling rate may be greater than or equal to 30 frames per second. Further additionally or alternatively, the sampling rate may be approximately 60 frames per second.
  • one or more two-dimensional images may be captured at a set sampling rate (eg, depending on the type of capture sequence performed).
  • such two-dimensional images can be recorded (eg, for analysis or display).
  • method 600 includes detecting a prevalent blur condition at the camera module based on the received one or more images.
  • one or more blurred portions may be identified or identified in one or more images.
  • one or more artificial intelligence or machine learning models eg, stored locally on controller 150
  • ambiguous parts can be identified or marked.
  • Blurry portions may optionally be tracked or identified across multiple images (eg, over time) from separate or independent capture sequences.
  • one or more models may identify blurred portions as corresponding to prevailing blur conditions (eg, fog conditions or static debris conditions).
  • one or more images from one or more models may be assigned a prediction score to predict the likelihood that a particular image indicates a prevalent blur condition, such as a fog condition or a static debris condition (e.g., based on The shape, color, or variation of the identified blurry portion within one or more images).
  • a prediction score for a corresponding prevalent blur condition that is higher than a set threshold may lead to or confirm the determination of a prevalent blur condition.
  • method 600 includes directing a responsive action on the refrigeration appliance in response to detecting a general ambiguity condition.
  • the corresponding response action may be automatically initiated (e.g., without direct user input or approved).
  • the prevailing blur condition is a fog condition (eg, corresponding to persistent condensation accumulating on the lens of the camera module, such as may occur when a dedicated heater is absent or malfunctioning).
  • 630 may include initiating a continuous anti-fog capture sequence at the camera module.
  • a continuous anti-fog capture sequence can direct images to be captured sequentially at the camera module.
  • a 2D image can be captured at a set sampling rate.
  • the anti-fog capture sequence may have a lower sampling rate, such as between 1 frame per second and 25 frames per second. Additionally or alternatively, the sampling rate of the anti-fog capture sequence may be approximately 10 frames per second.
  • an initiated anti-fog capture sequence may result in receiving a still image signal from the camera module during a continuous anti-fog capture sequence.
  • images captured at the camera may be transmitted to (and received by) the controller of the refrigeration appliance (eg, when such images are captured).
  • the still image signal may thus correspond to an image captured at the camera module as part of a discarding of the received still image signal of a continuous anti-fog capture sequence.
  • a continuous anti-fog capture sequence captures images of a refrigerated compartment when little or no information is expected to be discernible.
  • the still image signal may be discarded immediately after being received at the controller.
  • the static image signal may be discarded after a brief evaluation (eg, in response to a brief evaluation) in order to detect elevated pixel values or to compare a pair of sequential image signals.
  • determining the fog condition may include projecting 630 at a user interface or display of the refrigeration appliance (eg, a display on the case or a mobile display of the user device) corresponding to Alert message for fog conditions.
  • the alert message may include text or images indicating the presence of fog conditions.
  • still images or video clips may be provided to illustrate how fog conditions may be resolved (eg, by cleaning or repairing the camera module).
  • the alarm signal may be transmitted to a third party remote server (eg, to notify maintenance personnel of fog conditions) as or as part of a response action.
  • the prevailing blur condition is a static debris condition (eg, corresponding to accumulated dust, dirt, or other generally static foreign matter that accumulates on the lens of the camera module).
  • 630 may include projecting an alert message corresponding to the static debris condition at a user interface or display of the refrigeration appliance (eg, a display on the box or a mobile display of the user device).
  • the alert message may include text or images indicating the presence of a static debris condition.
  • a still image or video clip may be provided to illustrate how a static debris condition may be resolved (eg, by cleaning or repairing the camera module).
  • the alarm signal may be transmitted to a third party remote server (eg, to notify maintenance personnel of a static debris condition) as or as part of a response action.
  • method 700 includes receiving a first image from a camera module. For example, as a catch One or more images are captured sequentially at a camera module as part of a sequence.
  • 2D images can be captured at a set sampling rate.
  • such an image may be an occupied image captured as part of a door opening capture sequence.
  • the first image is captured in response to a detected door opening event.
  • independent door opening events may be detected, for example, based on image values or signals from individual sensors or switches.
  • independent door opening events for each of the plurality of capture sequences may be detected based on elevated image values in the received still image signal.
  • independent door opening events for each of the plurality of capture sequences may be detected based on receiving an opening signal from the door body switch.
  • method 700 includes detecting a blur condition at the camera module based on the received first image.
  • the blurred portion may be identified or identified within the first image.
  • method 700 includes initiating an anti-fog capture sequence at the camera module. Specifically, 730 may respond to 720. Optionally, 730 may require the door to be in a closed position (eg, as indicated by the end of the door opening event).
  • a continuous anti-fog capture sequence can direct images to be captured sequentially at the camera module. For example, a 2D image can be captured at a set sampling rate.
  • the anti-fog capture sequence may have a lower sampling rate, such as between 1 frame per second and 25 frames per second. Additionally or alternatively, the sampling rate of the anti-fog capture sequence may be approximately 10 frames per second.
  • an initiated anti-fog capture sequence may result in receiving a still image signal from the camera module during a continuous anti-fog capture sequence.
  • images captured at the camera may be transmitted to (and received by) the controller of the refrigeration appliance (eg, when such images are captured).
  • the still image signal may thus correspond to an image captured at the camera module as part of a discarding of the received still image signal of a continuous anti-fog capture sequence.
  • a continuous anti-fog capture sequence captures images of a refrigerated compartment when little or no information is expected to be discernible.
  • the still image signal may be discarded immediately after being received at the controller.
  • the static image signal may be discarded after a brief evaluation (eg, in response to a brief evaluation) in order to detect elevated pixel values or to compare a pair of sequential image signals.
  • method 700 includes detecting a door opening event after 730 .
  • independent door opening events may be detected, for example, based on image values or signals from individual sensors or switches.
  • independent door opening events for each of the plurality of capture sequences may be detected based on elevated image values in the received still image signal.
  • independent door opening events for each of the plurality of capture sequences may be detected based on receiving an opening signal from the door body switch.
  • method 700 includes receiving a second image from the camera during the door opening event.
  • one or more images will be captured sequentially at the camera module as part of a capture sequence (eg, a new or corresponding door opening sequence).
  • 2D images can be captured at a set sampling rate.
  • such an image could be So the occupied image is captured as part of the door opening capture sequence.
  • method 700 includes re-evaluating the blur condition based on the received second image.
  • the second image can be analyzed (e.g., using the included model) to detect blurred portions (e.g., to determine whether the same blur condition from the first image is present, whether/how it changes, etc.).
  • a change in color or shape or the appearance of a new blurred portion may indicate that the blurred portion is associated with a fog condition.
  • small changes eg, within a set percentage or range
  • the reassessment can determine that a general ambiguity exists.
  • method 700 includes directing a response action based on the re-evaluation.
  • 770 may include initiating at the camera module Continuous anti-fog capture sequence.
  • a continuous anti-fog capture sequence can direct images to be captured sequentially at the camera module.
  • a 2D image can be captured at a set sampling rate.
  • the anti-fog capture sequence may have a lower sampling rate, such as between 1 frame per second and 25 frames per second.
  • the sampling rate of the anti-fog capture sequence may be approximately 10 frames per second.
  • an initiated anti-fog capture sequence may result in receiving a still image signal from the camera module during a continuous anti-fog capture sequence.
  • images captured at the camera may be transmitted to (and received by) the controller of the refrigeration appliance (eg, when such images are captured).
  • the still image signal may thus correspond to an image captured at the camera module as part of a discarding of the received still image signal of a continuous anti-fog capture sequence.
  • a continuous anti-fog capture sequence captures images of a refrigerated compartment when little or no information is expected to be discernible.
  • such images may be deleted or discarded from the controller.
  • the still image signal may be discarded immediately after being received at the controller.
  • the static image signal may be discarded after a brief evaluation (eg, in response to a brief evaluation) in order to detect elevated pixel values or to compare a pair of sequential image signals.
  • 770 may include An alert message corresponding to the fog condition is projected at a user interface or display of the refrigeration appliance (eg, a display on a box or a mobile display of a user device).
  • the alert message may include text or images indicating the presence of fog conditions.
  • still images or video clips may be provided to illustrate how fog conditions may be resolved (eg, by cleaning or repairing the camera module).
  • the alarm signal may be transmitted to a third party remote server (eg, to notify maintenance personnel of fog conditions) as or as part of a response action.
  • a re-evaluation of the determination of accumulated dust, dirt, or other generally static foreign matter on the lens of the module 770 may be included in the user interface or display of the refrigeration appliance (e.g., a display on the case or a mobile display of the user device) Project an alert message corresponding to a static debris condition.
  • the alert message may include text or images indicating the presence of a static debris condition.
  • a still image or video clip may be provided to illustrate how a static debris condition may be resolved (eg, by cleaning or repairing the camera module).
  • the alarm signal may be transmitted to a third party remote server (eg, to notify maintenance personnel of a static debris condition) as or as part of a response action.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Combustion & Propulsion (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Thermal Sciences (AREA)
  • Mechanical Engineering (AREA)
  • Chemical & Material Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Quality & Reliability (AREA)
  • Cold Air Circulating Systems And Constructional Details In Refrigerators (AREA)

Abstract

一种制冷电器可以包括箱体、门体、相机模块以及控制器。箱体可以限定制冷间室。门体可以可旋转地铰接到箱体，以提供选择性地到达制冷间室的途径。相机模块可以在制冷间室内安装到箱体。控制器可以可操作地联接到相机模块。该控制器可以被配置为启动操作例程。操作例程可以包括接收用于多个捕获序列中的每一个的一个或多个图像，基于所接收的一个或多个图像检测在相机模块处的普遍模糊状况，以及响应于检测到普遍模糊状况而在制冷电器上引导响应动作。

Description

冰箱相机模块和用于解决相机透镜上的持续状况的方法 技术领域
本发明总体涉及用于防止持续状况使相机、特别是制冷电器中的相机的视线模糊的系统和方法。
背景技术
诸如制冷电器和食品储藏柜的储存外壳通常提供用于接收多个物品或物体的封闭腔室。例如，制冷电器通常包括限定制冷间室的箱体。用户可以将食品或物体放置在制冷间室内，以便阻止这种食品的腐烂。从而，可以增加易腐物品或物体的可用寿命。
随着时间的推移，大量的储存物品(例如，食品)可能累积在冰箱的制冷间室内。随着所储存的物品的累积，制冷电器的用户可能难以识别位于制冷电器内的物品。另外，用户可能难以确定制冷电器内的某些物品的数量。尤其是当多个用户在不与其他用户通信的情况下向公共制冷电器添加/从其取出物品。因此，用户可能意外地购买了过多或不需要的物品。例如，某些食品在制冷间室内不容易腐烂，并且这种食品可能不经常消耗。由此，这种食品可以在制冷间室内保持较长的时间段。尽管已经具有可接受的物品，用户可能忘记这种食品并购买替代品。这样，可能给用户造成不便或者不必要地花钱。另外或可选地，一些用户可能不知道某些物品已经被取出或消耗。由此，用户可能无法更换或补充这种物品。
现有系统已经尝试通过在制冷电器的制冷间室内设置相机(例如，以查看或跟踪制冷间室的内容物)来解决这些问题。然而，在制冷间室内使用相机产生了额外的问题。具体地，空气温度或水分含量的快速变化(例如，由门体的打开/关闭引起)可能导致凝结物(即，雾)累积在制冷间室的表面上。例如，水分可能凝结在相机模块的透镜上，从而阻挡或以其他方式模糊相机的视线并且通常劣化性能。另外或可选地，灰尘或污垢可能累积在透镜上，这也可能使相机的视线模糊并且劣化性能。然而，用户可能难以知道何时或如何解决这种持续模糊状况，尤其是在它们随时间而持续时。
因此，具有用于识别和解决沿着相机的视线的持续模糊的特征的方法或制冷电器将是有用的。
发明内容
本发明的各个方面以及优点将会在下文的描述中进行阐述，或者是通过描述可以显而易见的，或者是可以通过实施本发明而学到。
在本发明的一个示例性方面，提供了一种制冷电器。该制冷电器可以包括箱体、门体、相机模块以及控制器。箱体可以限定制冷间室。门体可以可旋转地铰接到箱体，以提供选择性地到达制冷间室的途径。相机模块可以在制冷间室内安装到箱体。控制器可以可操作地联接到相机模块。该控制器可以被配置为启动操作例程。操作例程可以包括接收用于多个捕获序列中的每一个的一个或多个图像，基于所接收的一个或多个图像检测在相机模块处的普遍模糊状况，以及响应于检测到普遍模糊状况而在制冷电器上引导响应动作。
在本发明的另一个示例性方面，提供了一种操作制冷电器的方法。方法可包括：接收用于多个捕获序列中的每一个的一个或多个图像。方法还可以包括：基于所接收的一个或多个图像检测在相机模块处的普遍模糊状况。方法还可以包括：响应于检测到普遍模糊状况而在制冷电器上引导响应动作。
在本发明的又一示例性方面，提供了一种操作制冷电器的方法。该制冷电器可以包括箱体、门体、相机模块以及控制器。箱体可以限定制冷间室。门体可以可旋转地铰接到箱体，以提供选择性地到达制冷间室的途径。相机模块可以在制冷间室内安装到箱体。控制器可以可操作地联接到相机模块。该控制器可以被配置为启动操作例程。操作例程可以包括从相机模块接收第一图像，基于所接收的第一图像检测在相机模块处的模糊状况，在相机模块处启动防雾捕获序列，检测开门事件，在开门事件期间从相机接收第二图像，基于所接收的第二图像重新评估模糊状况，以及基于重新评估来引导响应动作。
参照下文的描述以及所附权利要求，本发明的这些和其它的特征、方面以及优点将变得更容易理解。结合在本说明书中并且构成本说明书一部分的附图显示了本发明的实施方式并且与描述一起用于对本发明的原理进行解释。
附图说明
参照附图，说明书中阐述了面向本领域普通技术人员的本发明的完整公开，这种公开使得本领域普通技术人员能够实现本发明，包括本发明的最佳实施例。
图1提供了根据本发明的示例性实施方式的制冷电器的前立面图。
图2提供了根据本发明的示例性实施方式的制冷电器的前立面图，其中，冷藏 门体被示出为处于打开位置。
图3提供了根据本发明的示例性实施方式的制冷电器的示意图。
图4提供了根据本发明的示例性实施方式的制冷电器的相机模块的示意剖视图。
图5提供了根据本发明的其他示例性实施方式的制冷电器的相机模块的示意剖视图。
图6提供了示例了根据本发明的示例性实施方式的操作制冷电器的方法的流程图。
图7提供了示例了根据本发明的示例性实施方式的操作制冷电器的方法的流程图。
具体实施方式
现在将详细地参照本发明的实施方式，其中的一个或多个示例示于附图中。每个示例都以对发明进行解释的方式给出，并不对本发明构成限制。实际上，对于本领域技术人员而言显而易见的是，能够在不偏离本发明的范围的前提下对本发明进行多种改型和变型。例如，作为一个实施方式的一部分示出或者进行描述的特征能够用于另一个实施方式，从而产生又一个实施方式。因此，期望的是，本发明覆盖落入所附权利要求及其等同形式的范围内的这些改型以及变型。
如本文所用的，术语“或”通常旨在是包括的(即，“A或B”旨在意指“A或B或两者”)。术语“第一”、“第二”和“第三”可以互换使用以将一个部件与另一个部件区分开，并且这些术语并不旨在表示各个部件的位置或重要性。此外，如本文所用的，近似的用语，诸如“近似”、“大致”或“大约”是指在百分之十的误差裕度内。
通常，本发明提供了检测和解决相机模块的透镜何时在低温环境(诸如冰箱制冷间室)中被覆盖的系统和方法。例如，累积的凝结物、灰尘或污垢可能使相机模糊。系统和方法自动检测这些状况，而无需任何直接的用户输入或知识。而且，系统和方法可以用于减轻这样的状况(例如，没有直接的用户输入)或指导用户可以如何解决这些状况。
现在转向附图，图1提供了根据本发明的示例性实施方式的制冷电器100的前立面图，其中制冷电器100的冷藏门体128被示出为处于关闭位置。图2提供了制冷电器100的前立面图，其中冷藏门体128被示出为处于打开位置，以露出制冷电器100的食物保鲜室122。
制冷电器100包括壳体或箱体120，该壳体或箱体120沿着竖向V在顶部101与底部102之间延伸。箱体120限定用于接收食品以便储存的制冷间室。特别地，箱体120限定设置在箱体120的顶部101处或与其相邻设置的食物保鲜室122和布置在箱体120的底部102处或与其相邻布置的冷冻室124。由此可见，制冷电器100通常被称为底置式冰箱。然而，认识到，本发明的益处适用于其他类型和样式的储存外壳，诸如顶置式制冷电器、对开门式制冷电器等。因此，本文阐述的描述仅出于说明性目的，而无意于在任何方面限制任何特定的储存外壳或制冷室构造。
冷藏门体128可旋转地铰接到箱体120的边缘，以便选择性地进入食物保鲜室122。另外，在冷藏门体128的下方布置冷冻门体130，以便选择性地进入冷冻室124。冷冻门体130联接至可滑动地安装在冷冻室124内的冷冻抽屉142(未示出)。如上所述，冷藏门体128和冷冻门体130在图1中被示出为处于关闭位置，并且冷藏门体128在图2中被示出为处于打开位置。
在一些实施方式中，如本领域技术人员将理解的，各种储存部件被安装在食物保鲜室122内，以促进食品在其中的储存。特别地，储存部件包括安装在食物保鲜室122内的盒140、抽屉142以及层架144。盒140、抽屉142以及层架144被构造为接收所储存物品(例如，饮料或固体食品)，并且可以帮助组织这种食品。作为示例，抽屉142可以接收新鲜食品(例如，蔬菜、水果或奶酪)，并且增加这种新鲜食品的使用寿命。
在可选实施方式中，制冷电器100还包括用于辅助用户识别位于食物保鲜室122或冷冻室124内的食品的特征。用户可以利用这些特征，例如，来查看储存在食物保鲜室122或冷冻室124内的食品(即，所储存物品)，或创建这种所储存物品的库存。
一般参见图1至图3，图3提供了制冷电器100的示意图。制冷电器100包括控制器150，该控制器150与制冷电器100的制冷系统的部件可操作地联接或通信，该制冷系统被配置为冷却食物保鲜室122或冷冻室124。部件包括压缩机170、蒸发器风扇172以及冷凝器风扇174。控制器150可以选择性地操作这种部件，以便冷却食物保鲜室122或冷冻室124。控制器150还与温控器(例如，热电偶或热敏电阻)通信。温控器可以设置在食物保鲜室122或冷冻室124中。控制器150可以从温控器接收与食物保鲜室122或冷冻室124的温度对应的信号。控制器150还可以包括用于计算经过的时间段的内部计时器。
控制器150可以包括存储器和一个或多个微处理器、CPU等，诸如通用或专用微 处理器，该微处理器用于执行与制冷电器100的操作关联的编程指令或微控制代码。存储器可以表示诸如DRAM的随机存取存储器或诸如ROM或FLASH的只读存储器。在一些实施方式中，处理器执行存储在存储器中的永久编程指令。对于某些实施方式，指令包括软件包，该软件包被配置为操作电器100，或者执行操作例程(例如，下面参见图6或图7描述的示例性方法600或700)。存储器可以是与处理器分开的部件，或者可以包含在处理器内的板上。可选地，控制器150可以在不使用微处理器的情况下(例如，使用独立的模拟或数字逻辑电路的组合；诸如开关、放大器、积分器、比较器、触发器、与门等)构建为执行控制功能，而不是依靠软件。
控制器150可以设置在整个制冷电器100中的各种位置。输入/输出(“I/O”)信号可以在控制器150与制冷电器100的各种操作部件之间路由。制冷电器100的一个或多个部件可以经由一条或多条传导信号线或共享的通信总线与控制器150通信(例如，电气通信)。另外或可选地，制冷电器100的一个或多个部件可以经由一个或多个无线信号带与控制器150通信(例如，无线通信)。
在一些实施方式中，制冷电器100还包括相机或相机模块160。相机160可以是适于捕获二维照片或图像的任意类型的装置。作为示例，相机160可以是具有电子图像传感器【例如，电荷耦合器件(CCD)或CMOS传感器】的摄像机或数字相机。当组装时，相机160与控制器150通信(例如，电通信或无线通信)，使得控制器150可以从相机160接收与由相机160捕获的图像对应的信号。
通常，相机160设置在制冷电器100上。在一些实施方式中，相机160在其顶部(例如，接近顶部101且远离底部102)处安装在食物保鲜室122内。例如，相机160可以固定到或被引导穿过限定食物保鲜室122的内衬的顶壁。在这种实施方式中，如图2示例，相机160可以指向下方。
在某些实施方式中，相机160指向一个或多个制冷间室(例如，食物保鲜室122)。例如，相机160可以指向抽屉142和层架144中的任何特定一个或组合的至少一部分。由此，相机160可以捕获抽屉142中的一个、所有抽屉142、层架144中的一个、所有层架144或其任何合适组合的图像。
暂时转向图4和图5，在一些实施方式中，包括电路板162和其主透镜164的相机160容纳在相机外壳210中。如图所示，相机外壳210围绕相机160的至少一部分(例如，在内衬123上)延伸。例如，外壳侧壁212可以从内衬123并且围绕相机160延伸(例如，使得相机被外壳侧壁212径向围绕)。具体地，外壳侧壁212可以从顶端214延伸到底端216。如果安装在内衬123的顶部，则底端216可以布置在 相机160的最下表面(例如，如在主透镜164处限定的)下方。前壁218从外壳侧壁212(例如，在底端216处)径向向内延伸。前壁218可以限定沿着相机160的视线LS或与其对齐的开口或孔口220，使得相机160的视角通常不被阻挡，并且相机160能够通过孔口220收集食物保鲜室122的图像。尽管图4将孔口220示例为开放的空隙，但是图5示例了孔口220可以被与主透镜164隔开的透明辅助透镜224覆盖，以封闭或密封相机外壳210，从而防止或限制相机160与其所安装的制冷间室(例如，食物保鲜室122)之间的流体连通。
如图4和图5中一般示出的，前壁218与相机160隔开(例如，竖直地)。当组装时，相机外壳210通常限定围绕相机160的开放体积222。相机外壳210可由固体的不可渗透材料形成，诸如合适的聚合物或金属。当相机160生成热量(例如，部分由于电流被传导到相机160/从相机传导电流时的电路电阻)来捕获图像时，这种热量可以被保持或捕获在开放体积222内，从而有利地防止在主透镜164或辅助透镜224上累积凝结物。
一般回到图1至图3，在某些实施方式中，制冷电器100包括集成显示器180。集成显示器180可以安装在冷藏门体128上或制冷电器100上的任何其它合适位置处。集成显示器180与控制器150通信，使得集成显示器180可以从控制器150接收与由相机160捕获的图像对应的信号。集成显示器180可以从控制器150接收这种信号，并且向用户可视地呈现图像。集成显示器180可以包括例如液晶显示面板(LCD)、等离子显示面板(PDP)或用于显示图像的任何其他合适的机构(例如投影仪)。
在另外或可选的实施方式中，制冷电器100包括网络接口(未示出)，该网络接口将制冷电器100(例如，控制器150)联接到网络190，使得制冷电器100可以通过网络190传输和接收信息。网络190可以是任何有线或无线网络，诸如WAN、LAN或HAN。
在一些这种实施方式中，制冷电器100(例如，控制器150)经由网络190与移动显示器182通信。移动显示器182可以是被配置为通过网络190通信并显示从其接收的图像的任何装置。例如，移动显示器182可以是电脑、智能电话或平板电脑。移动显示器182与控制器150通信，使得移动显示器182可以从控制器150(经由网络190)接收与用户界面或由相机160捕获的图像对应的信号。移动显示器182可以从控制器150接收这种信号，并且向用户可视地呈现一个或多个图像。移动显示器182可以包括例如液晶显示面板(LCD)、等离子显示面板(PDP)或用于显示图像的 任何其他合适的机构(例如投影仪)。移动显示器182还可以包括允许移动显示器182通过网络190启动与制冷电器100的通信的界面(例如，触觉输入，诸如按钮，或图形用户界面)。
在某些实施方式中，制冷电器100包括检测组件186((例如，以检测用户的存在)。例如，检测组件186可以包括一个或多个传感器，该一个或多个传感器被配置为直接检测用户的存在(例如，通过检测与特定个体用户相对应的生物特征或个性化标记)或间接检测用户的存在(例如，通过检测门体128或130的移动)。
作为示例，检测组件186可以包括前向相机，该前向相机被配置为基于捕获的二维图像来识别用户的面部。
作为另一示例，检测组件186可以包括被配置为视觉地检测用户指纹的指纹成像传感器。
作为又一示例，检测组件186可以包括信号检测传感器，该信号检测传感器被配置为检测在无线通信频带(例如，使用2.4至2.485GHz的ISM频带中的短波长UHF无线电波的BLE频带)中的装置地址。装置地址可以是例如移动显示器182的编程的蓝牙地址。由此，检测组件186可以确定移动显示器182是否以及何时很靠近制冷电器100。
作为又一示例，检测组件186可包括门体开关(例如，簧片开关、推杆开关、霍尔效应传感器等)，该门体开关选择性地与冷藏门体128接合以检测门体128是否/何时处于打开位置。这种开关通常是理解的，并且例如可以同时控制用于照亮食物保鲜室122的灯的启动。由此，打开冷藏门体128可以启动灯并传输指示用户的检测的信号。
在使用期间，诸如在一个或多个图像捕获序列期间，相机160可以捕获可被传输到控制器150(例如，作为数据信号)的一个或多个二维图像(例如，作为视频馈送或一系列顺序静态图像)，如通常理解的。对于各个图像捕获，热量的脉冲串或脉冲可以由相机160从通过相机的各个电气部件或电路传输电流或电压的电阻生成。由此，热量的脉冲串或脉冲通常可对应于特定图像捕获序列的样本或帧率。控制器150处的所接收图像信号的特定图像捕获序列或进一步分析可以改变或交替(例如，基于制冷电器100的一个或多个状况)。
作为使用的示例，即使当门体128关闭或没有用户以其他方式存在时，连续的防雾捕获序列也可引导相机160收集和传输图像(例如，以设定的采样率顺序地，如以帧每秒定义的)。通常，这样的图像将是静态的(例如，在相机160的视场内的 物体将是不移动的，或者在食物保鲜室122内的光将不足以收集除空白图像之外的任何图像)。一旦被捕获，空白或静态图像就可以被丢弃(例如，在控制器150处)。
作为额外或可选的使用示例，开门捕获序列可引导相机160收集和传输图像(例如，以不同于连续防雾捕获序列的采样率的设定采样率顺序地)，同时一个或多个门体128打开或用户以其他方式存在。通常，这样的图像将被占用(例如，具有用于相机160的视场内的物体或光的信息或数据值)。
作为进一步额外或可选的使用示例，即使在门体128关闭或没有用户以其他方式存在时，监视器序列也可引导相机160收集和传输图像(例如，以小于连续防雾捕获序列或开门序列的采样率的设定采样率顺序地)。通常，这样的图像将是静态的(例如，在相机160的视场内的物体将是不移动的，或者在食物保鲜室122内的光将不足以收集除空白图像之外的任何图像)。一旦被捕获，空白或静态图像就可以被丢弃(例如，在控制器150处)。
一旦被捕获，特定图像(例如，被占用的图像)就可以被记录(例如，至少临时地记录在控制器150处)，使得被占用的图像可以被传输以用于呈现(例如，在集成显示器180或远程显示器182处)。
另外或可选地，根据所记录的占用图像，在相机160的视场(例如，设定视场)内的物品(例如，诸如食物的所储存物品、或诸如用户附肢、层架、可移动抽屉等的非储存物品)可以由控制器150自动识别(例如，以对食物保鲜室122内的物品编目录)。如所理解的，识别这种物品可以通过边缘匹配、分治搜索、灰度匹配、感受野响应的直方图或另一适当的例程(例如，基于从相机160捕获的一个或多个图像在控制器150处执行)来执行。
进一步另外或可选地，根据所记录的占用图像，控制器150可以识别一个或多个模糊部分(例如，图像的被模糊、混淆或阻挡而不聚焦的部分)。这种模糊部分可以指示已经累积在相机160的透镜上的凝结物、灰尘或污垢。识别这种物品可以通过边缘匹配、分治搜索、灰度匹配、感受野响应的直方图或另一适当的例程(例如，基于从相机160捕获的一个或多个图像在控制器150处执行)来执行。而且，一个或多个人工智能或机器学习模型(例如，本地存储在控制器150上)可以应用于模糊部分，使得控制器150能够区分例如雾状况(例如，对应于相机160的透镜上的累积和暂时凝结)和静态碎屑状况(例如，对应于可能累积在相机160的透镜上的累积灰尘、污垢或其他大体静态的异物)。可选地，控制器150可使用来自多个独立事件(例如，开门事件)的图像来确定状况或识别模糊部分(例如，基于特定模糊 部分随时间的任何变化)。作为示例，模糊部分的形状或颜色随时间(例如，在若干独立图像或事件期间)的变化可以指示与静态碎屑状况相反的雾状况，静态碎屑状况可以对应于大体未变化的模糊部分。在任一情况下，模糊部分可以引起随时间(例如，跨多个独立的开门事件)保持不变的普遍模糊状况。
在一些实施方式中，基于对模糊部分或普遍模糊状况的检测，相机160可以被引导到一个或多个动作响应。作为示例，相机160可以被选择性地引导到连续的防雾捕获序列和开门捕获序列，使得相机160保持工作，以便以规则的间隔(例如，根据在给定时刻执行的任何一个图像捕获序列的采样率)生成热量。有利地，这种热量生成可以防止凝结物或雾累积在相机160的视线LS内(例如，在透镜或辅助透镜224处)并模糊由相机160捕获的图像。可选地，防雾捕获序列可以基于模糊部分或普遍模糊状况的检测来启动，普遍模糊状况被确定为或潜在地为(例如，在设定的可能性百分比内)雾状况。另外或可选地，开门捕获序列可以通过预定的接合动作来启动(例如，从而停止或中断连续的防雾捕获序列)，预定的接合动作诸如打开门体128、检测到的在制冷间室(例如，食物保鲜室122)内的移动、用户在检测组件186的范围内接合或移动、在用户界面或集成显示器180处提供用户输入等。进一步另外或可选地，可以通过单独的预定动作或不存在启动开门捕获序列的接合动作来启动防雾捕获序列(例如，第二防雾捕获序列)(例如，从而停止或中断开门捕获序列)。进一步另外或可选地，防雾捕获序列(例如，第一或第二防雾捕获序列)可响应于模糊状况而停止或暂停，该模糊状况被确定为或潜在地为(例如，在设定的可能性百分比内)静态碎屑状况。
在可选实施方式中，相机160和控制器150被配置为捕获多个(例如，一对)顺序的二维图像(例如，以设定的采样率)作为防雾捕获序列或单独的监视器序列的一部分。通常，如所理解的，各个二维图像包括多个像素(例如，以预定义的网格布置)。用于防雾捕获序列的相同视场或视线(例如，食物保鲜室122的相同区域)的顺序图像(例如，先前捕获的图像和最近捕获的图像)在被丢弃之前可以在控制器150处进行比较。例如，顺序图像可以彼此进行比较或与(例如，像素亮度或颜色的)基线值/值集合比较。可以检测顺序图像的变化或从基线值/值集合的变化以提示新的图像捕获序列。具体地，检测到的变化(诸如升高的图像值)可以提示控制器150从连续防雾序列交替到开门捕获序列。升高的图像值可以通过任何适当的比较或像素特性来检测，诸如可以指示亮度值的变化或(例如，来自顺序二维图像中的对应像素的)移动检测的亮度值或颜色值。
如将理解的，二维图像或设定视场内的各个像素具有亮度范围。可选地，亮度范围可以是例如0至255之间的RGB亮度范围(例如，其中“0”是最小RGB像素亮度值，而“255”是最大RGB像素亮度值)。在某些实施方式中，可以检测多个像素的亮度值，以便测量亮度。例如，可以针对对应的二维图像(或其子区域)计算像素亮度值的平均亮度(Bm)值。另外或可选地，与平衡值相比，可以计算多个像素的偏差值(例如，作为亮度值)。例如，所捕获图像的亮度值(Vbr)可以被计算为平均亮度值减去平衡值(例如，125)的差的绝对值除以平衡值。换言之，在一些实施方式中，亮度值可表达为：
Vbr＝(|Bm–125|)/125。
在一些实施方式中，在防雾捕获序列或监视器序列期间，升高的亮度值可提示开门捕获序列。例如，升高的亮度值可以是第一捕获图像中的一个或多个像素与第二捕获图像中的一个或多个对应像素之间的亮度值的增加。当门体128关闭时，对应的捕获图像的亮度值较低。相反，当门体128打开时，来自食物保鲜室122内部的光可以使对应的捕获图像的亮度值较高。由此，检测到的亮度值的增加(例如，设定量、相对百分比等的增加)可以指示门体128打开。
现在转向图6和图7，提供了根据本发明的示例性实施方式的方法(例如600和700)的流程图。通常，方法600和700提供了一种操作制冷电器100(图1)的方法，该制冷电器包括相机160，如上所述。方法600或700可例如由控制器150(图3)执行。例如，如所讨论的，控制器150可以与相机160或检测组件186(图3)通信。在操作期间，控制器150可以向相机160或检测组件186发送信号和从其接收信号。通常，控制器150还可以与电器100的其他合适部件通信，以促进电器100的操作。
有利地，根据本发明的方法可以高效地防止在制冷间室的一部分内的凝结(例如，不需要或使用专用加热器或加热元件)。例如，所公开的方法可以允许相机自身的热量生成以在制冷电器操作时恒定地防止(例如，在相机的透镜上的)凝结物的累积。另外或可选地，本方法可促进对所捕获图像的高效且有效的管理(例如，使得控制器不被所存储或所接收数据淹没)。
图6和图7描述了为了说明和讨论的目的而以特定顺序执行的步骤。使用本文所提供的公开内容，本领域普通技术人员应该理解，方法不是相互排斥的，并且本文所公开的任何方法的步骤可以以各种方式修改、改编、重新排列、省略或扩展，而不脱离本发明的范围(除了以其他方式描述或需要)。
在610，方法600包括接收用于多个捕获序列中的每一个的一个或多个图像。换言之，可从多个独立捕获序列接收至少一个图像。各个捕获序列可以在不同的时间点执行。例如，各个图像捕获序列可以对应于唯一的开门事件或捕获图像被占用的其他序列(即，不是空白或全黑)。在示例性实施方式中，方法600包括响应于独立开门事件而启动多个捕获序列中的每一个。例如，多个捕获序列中的一个或多个(例如，各个或一些)可以是开门捕获序列。如上所述，独立开门事件可以例如基于来自单独的传感器或开关的图像值或信号来检测。进而，可以基于所接收的静态图像信号中的升高的图像值来检测多个捕获序列中的每一个的独立开门事件。另外或可选地，可基于从门体开关接收到打开信号来检测多个捕获序列中的每一个的独立开门事件。
通常，开门捕获序列可以引导图像在相机模块处顺序地捕获。例如，可以以大于防雾捕获序列的采样率的设定采样率捕获二维图像。开门捕获序列的采样率由此可以较高，诸如在每秒25帧至每秒120帧之间。另外或可选地，采样率可大于或等于每秒30帧。进一步另外或可选地，采样率可以是大约每秒60帧。
如根据本发明将理解的，对于各个捕获序列，可以以设定的采样率(例如，根据执行的捕获序列的类型)捕获一个或多个二维图像。进而，可以记录这样的二维图像(例如，用于分析或显示)。
在620，方法600包括基于所接收的一个或多个图像检测在相机模块处的普遍模糊状况。具体地，可以在一个或多个图像中识别或标识一个或多个模糊部分。另外或可选地，可将一个或多个人工智能或机器学习模型(例如，本地存储在控制器150上)应用于二维图像。根据这种应用，可以识别或标识模糊部分。可以可选地跨来自单独或独立捕获序列的多个图像(例如，随着时间)跟踪或识别模糊部分。另外或可选地，一个或多个模型可以将模糊部分识别为对应于普遍模糊状况(例如，雾状况或静态碎屑状况)。在一些这样的实施方式中，可以向来自一个或多个模型的一个或多个图像分配预测分数，以预测特定图像指示普遍模糊状况(诸如雾状况或静态碎片状况)的可能性(例如，基于一个或多个图像内的所识别的模糊部分的形状、颜色或变化)。本领域普通技术人员根据本发明通常可以理解或组装这样的模型。可选地，对于对应的普遍模糊状况高于设定阈值的预测分数可以导致或确认普遍模糊状况的确定。
在630，方法600包括响应于检测到普遍模糊状况而在制冷电器上引导响应动作。换言之，响应于620，可以自动地启动对应的响应动作(例如，没有直接的用户输入 或批准)。
在一些实施方式中，普遍模糊状况是雾状况(例如，对应于在相机模块的透镜上累积的持续凝结物，诸如在专用加热器不存在或发生故障时可能出现)。在这样的情况下，630可以包括在相机模块处启动连续的防雾捕获序列。通常，连续防雾捕获序列可以引导图像在相机模块处顺序地捕获。例如，可以以设定的采样率捕获二维图像。防雾捕获序列的采样率可以较低，诸如在每秒1帧至每秒25帧之间。另外或可选地，防雾捕获序列的采样率可以是大约每秒10帧。进而，启动的防雾捕获序列可导致在连续防雾捕获序列期间从相机模块接收静态图像信号。如上所述，在相机处捕获的图像可以被传输到制冷电器的控制器(并且由其接收)(例如，当这样的图像被捕获时)。静态图像信号由此可以对应于作为连续防雾捕获序列的丢弃所接收的静态图像信号的一部分在相机模块处捕获的图像。通常，当预期很少或没有信息可辨别时，连续防雾捕获序列可捕获制冷间室的图像。由此，可以从控制器删除或丢弃这样的图像(如与图像信号一起提供的)。可选地，静态图像信号可以在控制器处接收后立即丢弃。可选地，可在短暂评估之后(例如，响应于短暂评估)丢弃静态图像信号，以便检测升高的像素值或比较一对顺序图像信号。
与防雾捕获序列分开或除了防雾捕获序列之外，确定雾状况可使得630包括在制冷电器的用户界面或显示器(例如，在箱体上的显示器或用户装置的移动显示器)处投影对应于雾状况的警报消息。通常，警报消息可以包括指示雾状况的存在的文本或图像。可选地，可以提供静态图像或视频剪辑以示例可以如何解决雾状况(例如，通过清洁或修理相机模块)。另外或可选地，如将理解的，警报信号可被传输到第三方远程服务器(例如，以便向维修人员通知雾状况)，作为响应动作或作为响应动作的一部分。
在一些实施方式中，普遍模糊状况是静态碎片状况(例如，对应于累积在相机模块的透镜上的累积的灰尘、污垢、或其他通常静态异物)。在这种情况下，630可以包括在制冷电器的用户界面或显示器(例如，在箱体上的显示器或用户装置的移动显示器)处投影对应于静态碎屑状况的警报消息。通常，警报消息可以包括指示静态碎屑状况的存在的文本或图像。可选地，可以提供静态图像或视频剪辑以示例可以如何解决静态碎屑状况(例如，通过清洁或修理相机模块)。另外或可选地，如将理解的，警报信号可被传输到第三方远程服务器(例如，以便向维修人员通知静态碎屑状况)，作为响应动作或作为响应动作的一部分。
现在转向图7，在710，方法700包括从相机模块接收第一图像。例如，作为捕 获序列的一部分，在相机模块处顺序地捕获一个或多个图像。可选地，可以以设定的采样率捕获二维图像。作为示例，这样的图像可以是作为开门捕获序列的一部分而捕获的被占用图像。
在一些这样的实施方式中，响应于检测到的开门事件捕获第一图像。如上所述，独立开门事件可以例如基于来自单独的传感器或开关的图像值或信号来检测。进而，可以基于所接收的静态图像信号中的升高的图像值来检测多个捕获序列中的每一个的独立开门事件。另外或可选地，可基于从门体开关接收到打开信号来检测多个捕获序列中的每一个的独立开门事件。
在720，方法700包括基于所接收的第一图像检测在相机模块处的模糊状况。具体地，如上所述，可以在第一图像内识别或标识模糊部分。
在730，方法700包括在相机模块处启动防雾捕获序列。具体地，730可以响应720。可选地，730可以要求门体处于关闭位置(例如，如开门事件的结束所指示的)。通常，连续防雾捕获序列可以引导图像在相机模块处顺序地捕获。例如，可以以设定的采样率捕获二维图像。防雾捕获序列的采样率可以较低，诸如在每秒1帧至每秒25帧之间。另外或可选地，防雾捕获序列的采样率可以是大约每秒10帧。进而，启动的防雾捕获序列可导致在连续防雾捕获序列期间从相机模块接收静态图像信号。如上所述，在相机处捕获的图像可以被传输到制冷电器的控制器(并且由其接收)(例如，当这样的图像被捕获时)。静态图像信号由此可以对应于作为连续防雾捕获序列的丢弃所接收的静态图像信号的一部分在相机模块处捕获的图像。通常，当预期很少或没有信息可辨别时，连续防雾捕获序列可捕获制冷间室的图像。由此，可以从控制器删除或丢弃这样的图像(如与图像信号一起提供的)。可选地，静态图像信号可以在控制器处接收后立即丢弃。可选地，可在短暂评估之后(例如，响应于短暂评估)丢弃静态图像信号，以便检测升高的像素值或比较一对顺序图像信号。
在740，方法700包括在730之后检测开门事件。如上所述，独立开门事件可以例如基于来自单独的传感器或开关的图像值或信号来检测。进而，可以基于所接收的静态图像信号中的升高的图像值来检测多个捕获序列中的每一个的独立开门事件。另外或可选地，可基于从门体开关接收到打开信号来检测多个捕获序列中的每一个的独立开门事件。
在750，方法700包括在开门事件期间从相机接收第二图像。例如，作为捕获序列(例如，新的或对应的开门序列)的一部分，将在相机模块处顺序地捕获一个或多个图像。可选地，可以以设定的采样率捕获二维图像。作为示例，这样的图像可 以是作为开门捕获序列的一部分而捕获的被占用图像。
在760，方法700包括基于所接收的第二图像来重新评估模糊状况。具体地，可以分析第二图像(例如，使用所包括的模型)以检测模糊部分(例如，以确定是否存在来自第一图像的相同模糊状况，它是否/如何变化等)。可选地，颜色或形状的变化或新的模糊部分的出现可以指示模糊部分与雾状况相关联。另外或可选地，对第一图像与第二图像之间的模糊部分的微量改变(例如，在设定百分比或范围内)可指示静态碎屑状况。由此，重新评估可以确定存在普遍模糊状况。
在770，方法700包括基于重新评估来引导响应动作。作为示例，基于雾状况(例如，对应于累积在相机模块的透镜上的持续凝结物，诸如在专用加热器不存在或发生故障时可能出现)的重新评估确定，770可以包括在相机模块处启动连续的防雾捕获序列。通常，连续防雾捕获序列可以引导图像在相机模块处顺序地捕获。例如，可以以设定的采样率捕获二维图像。防雾捕获序列的采样率可以较低，诸如在每秒1帧至每秒25帧之间。另外或可选地，防雾捕获序列的采样率可以是大约每秒10帧。进而，启动的防雾捕获序列可导致在连续防雾捕获序列期间从相机模块接收静态图像信号。如上所述，在相机处捕获的图像可以被传输到制冷电器的控制器(并且由其接收)(例如，当这样的图像被捕获时)。静态图像信号由此可以对应于作为连续防雾捕获序列的丢弃所接收的静态图像信号的一部分在相机模块处捕获的图像。通常，当预期很少或没有信息可辨别时，连续防雾捕获序列可捕获制冷间室的图像。由此，可以从控制器删除或丢弃这样的图像(如与图像信号一起提供的)。可选地，静态图像信号可以在控制器处接收后立即丢弃。可选地，可在短暂评估之后(例如，响应于短暂评估)丢弃静态图像信号，以便检测升高的像素值或比较一对顺序图像信号。
作为额外或可选的示例，基于雾状况(例如，对应于累积在相机模块的透镜上的持续凝结物，诸如在专用加热器不存在或发生故障时可能出现)的重新评估确定，770可以包括在制冷电器的用户界面或显示器(例如，在箱体上的显示器或用户装置的移动显示器)处投影对应于雾状况的警报消息。通常，警报消息可以包括指示雾状况的存在的文本或图像。可选地，可以提供静态图像或视频剪辑以示例可以如何解决雾状况(例如，通过清洁或修理相机模块)。另外或可选地，如将理解的，警报信号可被传输到第三方远程服务器(例如，以便向维修人员通知雾状况)，作为响应动作或作为响应动作的一部分。
作为进一步额外或可选的示例，基于静态碎屑状况(例如，对应于累积在相机 模块的透镜上的累积的灰尘、污垢、或其他通常静态的异物)的重新评估确定，770可以包括在制冷电器的用户界面或显示器(例如，在箱体上的显示器或用户装置的移动显示器)处投影对应于静态碎屑状况的警报消息。通常，警报消息可以包括指示静态碎屑状况的存在的文本或图像。可选地，可以提供静态图像或视频剪辑以示例可以如何解决静态碎屑状况(例如，通过清洁或修理相机模块)。另外或可选地，如将理解的，警报信号可被传输到第三方远程服务器(例如，以便向维修人员通知静态碎屑状况)，作为响应动作或作为响应动作的一部分。
本书面描述使用示例对本发明进行了公开(其中包括最佳实施例)，并且还使本领域技术人员能够实施本发明(其中包括制造和使用任意装置或系统并且执行所包含的任意方法)。本发明的可专利范围通过权利要求进行限定，并且可以包括本领域技术人员能够想到的其它的示例。如果这种其它的示例包括与权利要求的字面语言没有区别的结构元件，或者如果这种其它的示例包括与权利要求的字面语言没有实质区别的等同结构元件，则期望这种其它的示例落入权利要求的范围中。

Claims (19)

  1. 一种制冷电器，其特征在于，包括：
    箱体，所述箱体限定制冷间室；
    门体，所述门体可旋转地铰接到所述箱体，以提供选择性地到达所述制冷间室的途径；
    相机模块，所述相机模块在所述制冷间室内安装到所述箱体；以及
    控制器，所述控制器可操作地联接到所述相机模块，所述控制器被配置为发起操作例程，所述操作例程包括：
    接收用于多个捕获序列中的每一个的一个或多个图像，
    基于所接收的一个或多个图像检测在所述相机模块处的普遍模糊状况，以及
    响应于检测到所述普遍模糊状况而在所述制冷电器上引导响应动作。
  2. 根据权利要求1所述的制冷电器，其特征在于，所述普遍模糊状况是雾状况，并且
    其中，引导所述响应动作包括：
    在所述相机模块处启动连续的防雾捕获序列，
    在所述连续防雾捕获序列期间从所述相机模块接收静态图像信号，以及
    丢弃所述所接收的静态图像信号。
  3. 根据权利要求2所述的制冷电器，其特征在于，所述防雾捕获序列具有每秒1帧至每秒25帧之间的采样率。
  4. 根据权利要求1所述的制冷电器，其特征在于，所述普遍模糊状况是静态碎屑状况，并且
    其中，引导所述响应动作包括：
    在所述制冷电器的用户界面处投影对应于所述静态碎屑状况的警报消息。
  5. 根据权利要求1所述的制冷电器，其特征在于，所述操作例程还包括：
    响应于独立开门事件而启动所述多个捕获序列中的每一个。
  6. 根据权利要求5所述的制冷电器，其特征在于，基于所接收的静态图像信号中的升高的图像值来检测所述多个捕获序列中的每一个的所述独立开门事件。
  7. 根据权利要求5所述的制冷电器，其特征在于，还包括门体开关，所述门体开关与所述门体选择性地通信，其中，基于从所述门体开关接收到打开信号来检测 所述多个捕获序列中的每一个的所述独立开门事件。
  8. 根据权利要求1所述的制冷电器，其特征在于，所述相机模块容纳在相机外壳内，所述相机外壳包括前壁，所述前壁限定沿着所述相机模块的视线的孔口，并且其中，所述前壁与所述相机模块隔开。
  9. 一种操作制冷电器的方法，其特征在于，所述制冷电器包括安装在制冷间室内的相机模块，所述方法包括：
    接收用于多个捕获序列中的每一个的一个或多个图像；
    基于所接收的一个或多个图像检测在所述相机模块处的普遍模糊状况；以及
    响应于检测到所述普遍模糊状况而在所述制冷电器上引导响应动作。
  10. 根据权利要求9所述的方法，其特征在于，所述普遍模糊状况是雾状况，并且
    其中，引导所述响应动作包括：
    在所述相机模块处启动连续的防雾捕获序列，
    在所述连续防雾捕获序列期间从所述相机模块接收静态图像信号，以及
    丢弃所述所接收的静态图像信号。
  11. 根据权利要求10所述的方法，其特征在于，所述防雾捕获序列具有每秒1帧至每秒25帧之间的采样率。
  12. 根据权利要求9所述的方法，其特征在于，所述普遍模糊状况是静态碎屑状况，并且
    其中，引导所述响应动作包括：
    在所述制冷电器的用户界面处投影对应于所述静态碎屑状况的警报消息。
  13. 根据权利要求9所述的方法，其特征在于，还包括：
    响应于独立开门事件而启动所述多个捕获序列中的每一个。
  14. 根据权利要求13所述的方法，其特征在于，基于所接收的静态图像信号中的升高的图像值来检测所述多个捕获序列中的每一个的所述独立开门事件。
  15. 根据权利要求13所述的方法，其特征在于，基于从门体开关接收到打开信号来检测所述多个捕获序列中的每一个的所述独立开门事件。
  16. 根据权利要求9所述的方法，其特征在于，所述相机模块容纳在相机外壳内，所述相机外壳包括前壁，所述前壁限定沿着所述相机模块的视线的孔口，并且其中，所述前壁与所述相机模块隔开。
  17. 一种制冷电器，其特征在于，包括：
    箱体，所述箱体限定制冷间室；
    门体，所述门体可旋转地铰接到所述箱体，以提供选择性地到达所述制冷间室的途径；
    相机模块，所述相机模块在所述制冷间室内安装到所述箱体；以及
    控制器，所述控制器可操作地联接到所述相机模块，所述控制器被配置为发起操作例程，所述操作例程包括：
    从所述相机模块接收第一图像，
    基于所接收的第一图像检测在所述相机模块处的模糊状况，
    在所述相机模块处启动防雾捕获序列，
    检测开门事件，
    在所述开门事件期间从所述相机接收第二图像，
    基于所述所接收的第二图像重新评估所述模糊状况，以及
    基于所述重新评估来引导响应动作。
  18. 根据权利要求17所述的制冷电器，其特征在于，引导所述响应动作包括：
    在所述相机模块处启动连续的防雾捕获序列，
    在所述连续防雾捕获序列期间从所述相机模块接收静态图像信号，以及
    丢弃所述所接收的静态图像信号。
  19. 根据权利要求17所述的制冷电器，其特征在于，引导所述响应动作包括：
    在所述制冷电器的用户界面处投影对应于所述模糊状况的警报消息。
PCT/CN2023/084378 2022-03-28 2023-03-28 冰箱相机模块和用于解决相机透镜上的持续状况的方法 WO2023185834A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US17/705,642 US20230308632A1 (en) 2022-03-28 2022-03-28 Refrigerator camera modules and methods for addressing persistent conditions on the camera lens
US17/705,642 2022-03-28

Publications (1)

Publication Number Publication Date
WO2023185834A1 true WO2023185834A1 (zh) 2023-10-05

Family

ID=88096753

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/084378 WO2023185834A1 (zh) 2022-03-28 2023-03-28 冰箱相机模块和用于解决相机透镜上的持续状况的方法

Country Status (2)

Country Link
US (1) US20230308632A1 (zh)
WO (1) WO2023185834A1 (zh)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20150068824A (ko) * 2013-12-12 2015-06-22 엘지전자 주식회사 로봇 청소기 및 이의 제어 방법
JP2017040425A (ja) * 2015-08-19 2017-02-23 三菱電機株式会社 冷蔵庫およびプログラム
KR101778561B1 (ko) * 2016-04-20 2017-09-14 주식회사 아이엠에스일렉트로닉스 카메라의 오염 검출 및 안내장치, 그리고 그의 제어방법
CN111476194A (zh) * 2020-04-20 2020-07-31 海信集团有限公司 一种感知模组工作状态检测方法及冰箱
CN111928571A (zh) * 2019-05-13 2020-11-13 青岛海尔智能技术研发有限公司 冷藏设备中图像采集的系统、方法、装置及存储介质
US11122203B1 (en) * 2020-05-26 2021-09-14 Haier Us Appliance Solutions, Inc. Refrigerator camera modules and methods for preventing lens fog

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20150068824A (ko) * 2013-12-12 2015-06-22 엘지전자 주식회사 로봇 청소기 및 이의 제어 방법
JP2017040425A (ja) * 2015-08-19 2017-02-23 三菱電機株式会社 冷蔵庫およびプログラム
KR101778561B1 (ko) * 2016-04-20 2017-09-14 주식회사 아이엠에스일렉트로닉스 카메라의 오염 검출 및 안내장치, 그리고 그의 제어방법
CN111928571A (zh) * 2019-05-13 2020-11-13 青岛海尔智能技术研发有限公司 冷藏设备中图像采集的系统、方法、装置及存储介质
CN111476194A (zh) * 2020-04-20 2020-07-31 海信集团有限公司 一种感知模组工作状态检测方法及冰箱
US11122203B1 (en) * 2020-05-26 2021-09-14 Haier Us Appliance Solutions, Inc. Refrigerator camera modules and methods for preventing lens fog

Also Published As

Publication number Publication date
US20230308632A1 (en) 2023-09-28

Similar Documents

Publication Publication Date Title
US11122203B1 (en) Refrigerator camera modules and methods for preventing lens fog
CN112984913B (zh) 物品显示系统
US11335010B2 (en) Methods for viewing and tracking stored items
WO2021057769A1 (zh) 用于查看并跟踪所储存的物品的方法
US20160138859A1 (en) Refrigeration appliance system
JP2018146177A (ja) 物品貯蔵庫の在庫管理装置
US20140168396A1 (en) Method for viewing contents of a refrigerator appliance
JP7466117B2 (ja) 冷蔵庫、管理システム、及びプログラム
JP7012261B2 (ja) 冷蔵庫
JP6655908B2 (ja) 冷蔵庫およびプログラム
JP2022033235A (ja) 冷蔵庫
US11288506B2 (en) Refrigerator appliances and methods for tracking stored items
WO2023185834A1 (zh) 冰箱相机模块和用于解决相机透镜上的持续状况的方法
JP7113281B2 (ja) 冷蔵庫
US11512896B2 (en) Refrigerator appliance having a movable camera
US11692767B2 (en) Refrigerator appliance and methods of user identification
US20230316755A1 (en) Refrigerator appliance and methods for tracking stored items in a freezer chamber
US20230375258A1 (en) Refrigerator appliances and image-based methods of detecting a door position
WO2023131202A1 (zh) 带有智能抽屉的制冷电器
JP7289084B2 (ja) 冷蔵庫
US11768031B1 (en) Refrigerator appliance and methods for responding to ambient humidity levels
JP7289083B2 (ja) 冷蔵庫
WO2023193756A1 (zh) 制冷设备及其控制方法
CN213248024U (zh) 货柜
CN117560568A (zh) 设备内物品图像采集方法、设备和计算机存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23778199

Country of ref document: EP

Kind code of ref document: A1