US20220146190A1

US20220146190A1 - Refrigerator appliances and methods for tracking stored items

Info

Publication number: US20220146190A1
Application number: US17/092,645
Authority: US
Inventors: Choon Jae Ryu; David C. McCalpin
Original assignee: Haier US Appliance Solutions Inc
Current assignee: Haier US Appliance Solutions Inc
Priority date: 2020-11-09
Filing date: 2020-11-09
Publication date: 2022-05-12
Also published as: WO2022095995A1

Abstract

A refrigerator appliance may include a cabinet, a door, a scanner module, and a controller operably coupled to the scanner module. The door may be rotatably hinged to the cabinet to provide selective access to the chilled chamber. The scanner module may be mounted to the cabinet. The controller may be configured to initiate an operation routine. The operation routine may include initiating a scan sequence at the scanner module, identifying a barcode capture for a product barcode of a stored item based on the scan sequence, processing barcode data for the barcode capture according to one or more predetermined user preferences, establishing an item identifier based on the processed barcode data, and directing an update to a user list with the established item identifier.

Description

FIELD OF THE INVENTION

The present subject matter relates generally to tracking items within a storage enclosure, such as a refrigerator appliance.

BACKGROUND OF THE INVENTION

Storage enclosures, such as refrigerator appliances and pantries, generally provide an enclosed chamber for receiving multiple items or objects. For example, refrigerator appliances generally include a cabinet that defines a chilled chamber. A user can place food items or objects within the chilled chamber in order to hinder perishing of such food items. Thereby, a useable life of perishable items or objects can be increased.
Over time, a large volume or number of stored items (e.g., food items) can accumulate within the refrigerator's chilled chamber. As stored items accumulate, users of the refrigerator appliance can have difficulty identifying the items located within the refrigerator appliance. Additionally, users can have difficulty determining a quantity of certain items within the refrigerator appliance. This is especially true when multiple users add/remove items from a common refrigerator appliance without communicating with other users. Consequently, the users may accidentally purchase excessive or undesired items. For example, certain food items do not readily perish within the chilled chamber, and such food items may be consumed infrequently. Thus, such food items can remain within the chilled chamber for extended periods of time. The users can forget about such food items and purchase replacements despite already having acceptable items. In such a manner, the users can be inconvenienced or expend money needlessly. Additionally or alternatively, some users may be unaware that certain items have been removed or consumed. Thus, users may fail to replace or replenish such items.
Some existing appliances have attempted to address these issues by requiring a user to manually input each item being stored to create a list of stored items. Other appliances have used various methods, such as scales, to estimate or guess the quantity or identification of items being stored. Nonetheless, such attempts have been insufficiently cumbersome and prone to inaccuracies. Moreover, it can be difficult for a user to know when a specific item is removed or expired, and thus may need to be replaced.
Accordingly, a refrigerator appliance with features for assisting a user with tracking contents of a chilled chamber of the refrigerator appliance would be useful. In particular, a refrigerator appliance with features for assisting a user with accurately tracking the contents of a chilled chamber of the refrigerator appliance in order to establish an inventory of stored items positioned within the chilled chamber or a “shopping list” of needed items would be useful.

BRIEF DESCRIPTION OF THE INVENTION

Aspects and advantages of the invention will be set forth in part in the following description, or may be obvious from the description, or may be learned through practice of the invention.
In one exemplary aspect of the present disclosure, a refrigerator appliance is provided. The refrigerator appliance may include a cabinet, a door, a scanner module, and a controller operably coupled to the scanner module. The door may be rotatably hinged to the cabinet to provide selective access to the chilled chamber. The scanner module may be mounted to the cabinet. The controller may be configured to initiate an operation routine. The operation routine may include initiating a scan sequence at the scanner module, identifying a barcode capture for a product barcode of a stored item based on the scan sequence, processing barcode data for the barcode capture according to one or more predetermined user preferences, establishing an item identifier based on the processed barcode data, and directing an update to a user list with the established item identifier.
In another exemplary aspect of the present disclosure, a method of operating a refrigerator appliance is provided. The method may include initiating a scan sequence at the scanner module and identifying a barcode capture for a product barcode of a stored item based on the scan sequence. The method may further include processing barcode data for the barcode capture according to one or more predetermined user preferences. The method may still further include establishing an item identifier based on the processed barcode data and directing an update to a user list with the established item identifier.
These and other features, aspects and advantages of the present invention will become better understood with reference to the following description and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

A full and enabling disclosure of the present invention, including the best mode thereof, directed to one of ordinary skill in the art, is set forth in the specification, which makes reference to the appended figures.

FIG. 1 provides a front elevation view of a refrigerator appliance according to exemplary embodiments of the present disclosure.

FIG. 2 provides a front elevation view of a refrigerator appliance according to exemplary embodiments of the present disclosure, wherein refrigerator doors are shown in an open position.

FIG. 3 provides a schematic view of a refrigerator appliance according to exemplary embodiments of the present disclosure.

FIG. 4 provides a flow chart illustrating a method of operating a refrigerator appliance according to exemplary embodiments of the present disclosure.

FIG. 5 provides a flow chart illustrating a method of operating a refrigerator appliance according to exemplary embodiments of the present disclosure.

DETAILED DESCRIPTION

Reference now will be made in detail to embodiments of the invention, one or more examples of which are illustrated in the drawings. Each example is provided by way of explanation of the invention, not limitation of the invention. In fact, it will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the scope of the invention. For instance, features illustrated or described as part of one embodiment can be used with another embodiment to yield a still further embodiment. Thus, it is intended that the present invention covers such modifications and variations as come within the scope of the appended claims and their equivalents.
As used herein, the term “or” is generally intended to be inclusive (i.e., “A or B” is intended to mean “A or B or both”). The terms “first,” “second,” and “third” may be used interchangeably to distinguish one component from another and are not intended to signify location or importance of the individual components.
Generally, the present disclosure provides methods to aid in managing an inventory for food items within a storage enclosure, such as a refrigerator appliance or pantry. The methods may include one or more steps for determining what items (i.e., stored items) are added or removed from the storage enclosure. The determination may be made using information gathered using a barcode that is printed or embedded on one or more food items within the refrigerator.
Turning now to the figures, FIG. 1 provides a front elevation view of a refrigerator appliance 100 according to exemplary embodiments of the present disclosure with refrigerator doors 128 of the refrigerator appliance 100 shown in a closed position. FIG. 2 provides a front view elevation of refrigerator appliance 100 with refrigerator doors 128 shown in an open position to reveal a fresh food chamber 122 of refrigerator appliance 100.
Refrigerator appliance 100 includes a housing or cabinet 120 that extends between a top 101 and a bottom 102 along a vertical direction V. Cabinet 120 defines chilled chambers for receipt of food items for storage. In particular, cabinet 120 defines fresh food chamber 122 positioned at or adjacent top 101 of cabinet 120 and a freezer chamber 124 arranged at or adjacent bottom 102 of cabinet 120. As such, refrigerator appliance 100 is generally referred to as a bottom mount refrigerator. It is recognized, however, that the benefits of the present disclosure apply to other types and styles of storage enclosure, such as a top mount refrigerator appliance, a side-by-side style refrigerator appliance, or an unrefrigerated pantry enclosure. Consequently, the description set forth herein is for illustrative purposes only and is not intended to be limiting in any aspect to any particular storage enclosure or refrigerator chamber configuration.
Refrigerator doors 128 are rotatably hinged to an edge of cabinet 120 for selectively accessing fresh food chamber 122. In addition, a freezer door 130 is arranged below refrigerator doors 128 for selectively accessing freezer chamber 124. Freezer door 130 is coupled to a freezer drawer 142 (not shown) slidably mounted within freezer chamber 124. As discussed above, refrigerator doors 128 and freezer door 130 are shown in the closed configuration in FIG. 1, and refrigerator doors 128 are shown in the open position in FIG. 2.
Turning now to FIG. 2, various storage components are mounted within fresh food chamber 122 to facilitate storage of food items therein as will be understood by those skilled in the art. In particular, the storage components include bins 140, drawers 142, and shelves 144 that are mounted within fresh food chamber 122. Bins 140, drawers 142, and shelves 144 are configured for receipt of stored items (e.g., beverages or solid food items) and may assist with organizing such food items. As an example, drawers 142 can receive fresh food items (e.g., vegetables, fruits, or cheeses) and increase the useful life of such fresh food items.
Refrigerator appliance 100 also includes features for assisting a user with identifying food items positioned within fresh food chamber 122 or freezer chamber 124. The user can use such features, for example, to view food items stored (i.e., stored items) within fresh food chamber 122 or freezer chamber 124 or create an inventory or list of such stored items. Such an inventory or list may be formed, at least in part, using one or more product barcode (e.g., printed on the packaging on or within which a stored item is held). Such features are discussed in greater detail below.
FIG. 3 provides a schematic view of refrigerator appliance 100. Refrigerator appliance 100 includes a controller 150 that is operatively coupled or in communication with components of a refrigeration system of refrigerator appliance 100 configured for cooling fresh food chamber 122 or freezer chamber 124 (FIG. 2). The components include a compressor 170, an evaporator fan 172, and a condenser fan 174. Controller 150 can selectively operate such components in order to cool fresh food chamber 122 or freezer chamber 124. Controller 150 is also in communication with a thermostat (e.g., a thermocouple or thermistor). The thermostat may be positioned in fresh food compartment 122 or freezer compartment 124. Controller 150 may receive a signal from the thermostat that corresponds to a temperature of fresh food compartment 122 or freezer compartment 124. Controller 150 may also include an internal timer for calculating elapsed time periods.
Controller 150 may include a memory and one or more microprocessors, CPUs or the like, such as general or special purpose microprocessors operable to execute programming instructions or micro-control code associated with operation of refrigerator appliance 100. The memory may represent random access memory such as DRAM, or read only memory such as ROM or FLASH. In some embodiments, the processor executes non-transitory programming instructions stored in memory. For certain embodiments, the instructions include a software package configured to operate appliance 100 or execute an operation routine (e.g., one or more of the exemplary methods 400 and 500 described below with reference to FIGS. 4 and 5). The memory may be a separate component from the processor or may be included onboard within the processor. Alternatively, controller 150 may be constructed without using a microprocessor (e.g., using a combination of discrete analog or digital logic circuitry; such as switches, amplifiers, integrators, comparators, flip-flops, AND gates, and the like) to perform control functionality instead of relying upon software.
Controller 150 may be positioned in a variety of locations throughout refrigerator appliance 100. Input/output (“I/O”) signals may be routed between controller 150 and various operational components of refrigerator appliance 100. One or more components of refrigerator appliance 100 may be in communication (e.g., electric communication) with controller 150 via one or more conductive signal lines or shared communication busses. Additionally or alternatively, one or more components of refrigerator appliance 100 may be in communication (e.g., wireless communication) with controller 150 via one or more wireless signal bands.
In some embodiments, refrigerator appliance 100 also includes a camera or scanner module 160. Scanner module 160 may be any type of device suitable for detecting or capturing a printed product barcode (e.g., as reflected optical impulses or a two-dimensional image). As an example, scanner module 160 may include a pen-type reader or laser scanner having a beam emitter and reflector reader to capture reflected optical impulses (e.g., of light projected from the beam emitter). Once received, such optical impulses may be identified as being reflected from a product barcode (e.g., prior to the barcode being decoded), as would be understood. As an additional or alternative example, scanner module 160 may include a video camera or a digital camera with an electronic image sensor [e.g., a charge coupled device (CCD) or a CMOS sensor]. From one or more images, a product barcode may be detected or identified, such as by edge matching, divide-and-conquer search, greyscale matching, histograms of receptive field responses, or another suitable routine (e.g., executed at the controller 150 based on one or more captured images from scanner module 160). When assembled, scanner module 160 is in communication (e.g., electric or wireless communication) with controller 150 such that controller 150 may receive a signal from scanner module 160 corresponding to the reflected optical impulses or image captured by scanner module 160.
Generally, scanner module 160 is positioned on refrigerator appliance 100. In some embodiments, scanner module 160 is mounted within fresh food chamber 122 at a top portion thereof (e.g., adjacent top 101). For instance, scanner module 160 (e.g., beam emitter or camera) may be fixed to or directed through a wall of an internal liner defining fresh food chamber 122. In such embodiments, scanner module 160 may be directed downward, as illustrated in FIG. 2.
In certain embodiments, is directed toward one or more chilled chamber (e.g., fresh food chamber 122—FIG. 2). For instance, scanner module 160 may be directed towards at least a portion of any particular one of or combination of drawers 142 and shelves 144 (FIG. 2). Thus, scanner module 160 can capture reflected optical impulses from or images of one of drawers 142, all of drawers 142, one of shelves 144, all of shelves 144, or any suitable combination thereof.
Although scanner module 160 is illustrated as being mounted within fresh food chamber 122, it is understood that additional or alternative embodiments include a scanner assembly (e.g., pen-type reader, laser scanner, or digital camera with an electronic image sensor) mounted at another suitable portion of refrigerator appliance, such as a sidewall of the internal liner defining fresh food chamber 122 or door 128. Optionally, a scanner assembly mounted at door 128 may be directed outward or in front of refrigerator appliance 100 and may thus capture images of a user or area positioned forward from refrigerator appliance 100 (e.g., when the corresponding door is in a closed position).
During use, such as during a scan sequence, scanner module 160 may capture one or more reflected optical impulses or two-dimensional images. As an example, the scan sequence may provide for emitting or projecting light from light emitter, reflections of which can be received at the scanner module 160. Once received, the optical impulses may be transmitted to the controller 150 and decoded (e.g., by measuring intensity thereof), as would be understood. As an additional or alternative example, the scan sequence may provide for capturing one or more two-dimensional images (e.g., as a video feed or series of sequential static images) that may be transmitted to the controller 150 (e.g., as a data signal), as is generally understood.
In certain embodiments, refrigerator appliance 100 includes an integrated display 180. Integrated display 180 may be mounted on refrigerator door 128 (FIG. 1) or at any other suitable location on refrigerator appliance 100. Integrated display 180 is in communication with controller 150 such that integrated display 180 may receive a signal from controller 150 corresponding to an optical impulse or image captured by scanner module 160. Integrated display 180 can receive such signal from controller 150 and present the image to a user visually. Integrated display 180 may include, for example, a liquid crystal display panel (LCD), a plasma display panel (PDP), or any other suitable mechanism for displaying an image (e.g., a projector).
Separate from or in addition to integrated display 180, refrigerator 100 may include a communications assembly 184 in communication with controller 150. Generally, communications assembly 184 may be mounted on any suitable portion of refrigerator 100, such as within cabinet 120 or a door 128. In certain embodiments, the communications assembly includes a loudspeaker (e.g., dynamic loudspeaker, electrostatic loudspeaker, planar magnetic loudspeaker, piezoelectric loudspeaker etc.). As is understood, the loudspeaker may be configured to generate soundwaves from one or more electrical signals (e.g., digital sound signals received from controller 150). The loudspeaker may thus audibly communicate information to a user. In additional or alternative embodiments, the communications assembly includes a microphone (e.g., dynamic microphone, ribbon microphone, fiber-optic microphone, piezoelectric microphone, etc.). As is understood, the microphone may generate one or more electrical signals (e.g., to be received by controller 150) from one or more received soundwaves (e.g., from a user). The microphone may thus receive audible prompts or commands from a user that can be directed to the controller 150.
In exemplary embodiments, refrigerator appliance 100 includes a network interface (not shown) that couples refrigerator appliance 100 (e.g., controller 150) to a network 190 such that refrigerator appliance 100 can transmit and receive information over network 190. Network 190 can be any wired or wireless network such as a WAN, LAN, or HAN.
In optional embodiments, refrigerator appliance 100 includes a user-detection assembly 186. Generally, user-detection assembly 186 includes one or more sensors configured to detect a biometric or personalized marker corresponding to a specific individual user. As an example, user-detection assembly 186 may include a forward-facing camera configured to recognize or identify a user's face based on a captured two-dimensional image. As another example, user-detection assembly 186 may include a fingerprint imaging sensor configured to visually detect a user's fingerprint. As yet another example, user-detection assembly 186 may include a signal-detection sensor configured to detect a device address over a wireless communications band (e.g., a BLE band using short-wavelength UHF radio waves in the ISM band from 2.4 to 2.485 GHz). The device address may be a programmed Bluetooth address of, for instance, mobile display 182 (e.g., provided on or with a personal computer, cell phone, tablet, stationary kitchen hub, etc.). The user-detection assembly 186 may thus determine if and when a mobile display 182 is within close proximity to refrigerator appliance 100.
In certain embodiments, scanner module 160 is included as part of user-detection assembly 186. As an example, user-detection assembly 186 may be configured to recognize or identify a user from a two-dimensional image captured at scanner module 160. In some such embodiments, controller 150 is further configured to recognize one or more defining features below a user's elbow, such as skin tone, arm/hand size, jewelry, typical clothing, etc. As is understood, recognizing such defining features may be performed by edge matching, divide-and-conquer search, greyscale matching, histograms of receptive field responses, or another suitable routine (e.g., executed at the controller 150 based on one or more captured images from scanner module 160).
Optionally, a scan sequence may be initiated by a predetermined user action, such as opening a door 128, detected movement within a chilled chamber (e.g., fresh food chamber 122), engaging or moving within range of user-detection assembly 186, proving a user input at communications assembly 184 or integrated display 180, etc.
In some embodiments, one or more remote servers 192, such as a web server, is in operable communication with controller 150. The remote server 192 can be used to host a retailer's point of sale system. In other words, remote server 192 may be or include a retailer point of sale server that tracks, for example, an identifier and quantity of purchased items, a time or date stamp of purchased items, pricing of purchased items, a customer identifier (i.e., an identifier of the purchasing customer), etc. Additionally or alternatively, the remote server 192 can be used to host a retailer's stock management system. In other words, remote server 192 may include, or be provided as, a retailer stock management server that tracks, for example, stocking data relating to items offered for sale by a retailer (e.g., expiration data, location data, cost data, etc.). Also additionally or alternatively, remote server 192 may include, or be provided as, a user database. In other words, remote server 192 may include a database that includes data regarding a particular user (or group of users) (i.e., user profile), such as an item history of stored items, a “shopping list” of stored items to be bought or replaced, identification data, one or more images, etc. Further additionally or alternatively, the remote server 192 can be used to host one or more other information databases (e.g., including product data relating to commercially-available items, such as data related to the barcode, identification, quantity, price, or expiration of an item).
The remote server 192 can be implemented using any suitable computing device(s). The remote server 192 may include one or more processors and one or more memory devices (i.e., memory). The one or more processors can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, a FPGA, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. The memory device can include one or more non-transitory computer-readable storage mediums, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory devices can store data and instructions which are executed by the processor to cause remote server 192 to perform operations. For example, instructions could be instructions for receiving/transmitting user profile data signals, receiving/transmitting inventory management data signals, receiving/transmitting data signals relating to a stored item (i.e., product data signals), etc.
The memory devices may also include data, such as user profile data, customer identification data, inventory data, expiration data, product data, etc., that can be retrieved, manipulated, created, or stored by processor. The data can be stored in one or more databases. The one or more databases can be connected to remote server 192 by a high bandwidth LAN or WAN, or can also be connected to remote server 192 through network 502. The one or more databases can be split up so that they are located in multiple locales.
Remote server 192 includes a network interface such that remote server 192 can connect to and communicate over one or more networks (e.g., network 190) with one or more network nodes. In turn, remote server 192 can exchange data with one or more nodes over the network 190. In particular, remote server 192 can exchange data with controller 150. Although not pictured, it is understood that remote server 192 may further exchange data with any number of client devices over the network 190 (e.g., mobile display 182).
In certain embodiments, scanner module 160 and controller 150 are configured to capture a barcode via one or more reflected optical impulses or two-dimensional images. Optionally, multiple, sequential scans may be obtained (e.g., at a predetermined rate or pattern) as part of a scan sequence. Sequential scans (e.g., a previously-captured impulse or image and a more-recently-captured impulse or image) may be recorded (e.g., temporarily) and compared at the controller 150. From the comparison, changes or differences between sequential images may be detected. Optionally, the scan sequence may subsequently be stopped or halted according to one or more end conditions, such as expiration of a predetermined time period, failure to detect further changes in sequential scans, or closing of the door 128.
From one or more barcode scans, controller 150 may automatically attempt to recognize items (e.g., stored items, such as food) within the field of view of the scanner module 160. As is understood, recognizing such items, may be performed by looking up or comparing the captured barcode (e.g., an image or decoded barcode data) to a preestablished database of product data, including identification (e.g., what type of food object is being identified or enclosed within the packaging on which the barcode is printed), the quantity (e.g., number of discrete units, volume, or mass) of the stored item, the price of the stored item, or the expected expiration of the stored item (e.g., estimated expiration date based on a known shelf life of the stored item).
Turning now to FIGS. 4 and 5, flow charts are provided of methods 400 and 500 according to exemplary embodiments of the present disclosure. Generally, FIGS. 4 and 5 illustrate methods of operating a refrigerator appliance 100 (FIG. 1) that includes a scanner module 160, as described above. The method 400 or 500 can be performed, for instance, by the controller 150 (FIG. 3). For example, controller 150 may, as discussed, be in communication with scanner module 160, integrated display 180 (FIG. 3), mobile display 182 (FIG. 3), communications assembly 184, user-detection assembly 186, or remote server(s) 192. During operations, controller 150 may send signals to and receive signals from scanner module 160, integrated display 180, mobile display 182, communications assembly 184, user-detection assembly 186, or remote server(s) 192. Controller 150 may further be operatively coupled to other suitable components of the appliance 100 to facilitate operation of the appliance 100 generally.
Advantageously, methods in accordance with the present disclosure may permit improved detection of one or more items within, for example, a refrigerator appliance. Additionally or alternatively, detection may be made while reducing the processing demands on one or more controllers, processors, or processing devices of the refrigerator appliance.
FIGS. 4 through 5 depict steps performed in a particular order for purpose of illustration and discussion. Those of ordinary skill in the art, using the disclosures provided herein, will understand that the steps of any of the methods disclosed herein can be modified, adapted, rearranged, omitted, or expanded in various ways without deviating from the scope of the present disclosure (except as otherwise described). Thus, the steps of the methods 400 and 500 are non-exclusive and one or more steps may be interchanged with or added to the other method 500 or 400, as would be understood.
Turning especially to FIG. 4, at 410, the method 400 includes initiating a scan sequence at the scanner module. The scan sequence may include capturing one or more reflected impulses or two-dimensional images, as described above. Specifically, one or more scans may be made of one or more particular stored items (e.g., within chilled chamber). Optionally, a plurality of scans may be obtained as part of the scan sequence. For instance, the scan sequence may include capturing a plurality of two-dimensional images (e.g., a first two-dimensional image, subsequent second two-dimensional image, etc.), such as within a video feed or a series of sequential static images (e.g., taken or captured according to a predetermined rate or condition). Upon being captured at the scanner module, the impulses or images (i.e., scans) may be transmitted to the controller (e.g., as a data signal). The capture(s) may then be recorded (e.g., temporarily) for further processing or evaluation.
In certain embodiments, 410 is contingent on identifying a specific user. For instance, the scan sequence, or any further recording or evaluating of impulses or images from the scanner assembly, may be prevented until the specific user is identified (e.g., automatically or, alternatively, via a manual selection). The identification of a particular user may include receiving a user-detection signal indicating a biometric or personalized marker corresponding to the specific individual user, as described above. As an example, the user-detection signal may include a captured two-dimensional image of the specific user's face. From the captured two-dimensional image, the user may be recognized. As another example, the user-detection signal may include a fingerprint image from which the specific user's fingerprint may be recognized. As yet another example, the user-detection signal may include a detected device address of the specific user's mobile display (e.g., detected over a wireless communications band, such as, a BLE band using short-wavelength UHF radio waves in the ISM band from 2.4 to 2.485 GHz).
In additional or alternative embodiments, the scan sequence is initiated in response to detecting movement within a chilled chamber (e.g., fresh food chamber) of the refrigerator appliance. Thus, recording or evaluating scans from the scanner module may be prevented until movement is detected (e.g., at the scanner module or a separate sensor). As an example, movement may be detected by reception of an optical impulse of light reflected from a beam emitter of the scanner module. As another example, changes in light or pixels captured by the scanner module may be detected (e.g., between multiple images captured over time) indicating the movement of one or more objects within the field of view of the scanner module, as is generally understood. As yet another example, movement may be detected in response to receiving a signal from a separate sensor, such as a switch selectively engaged with the door. Such switches are generally understood and may, for instance, simultaneously control activation of a light for illuminating the chilled chamber. Opening the refrigerator door may thus activate the light and transmit a signal indicating motion within the chilled chamber.
Optionally, the scan sequence may continue until one or more end conditions are met. As an example, an end condition may include expiration of a predetermined time period (e.g., span of time) following the start of a corresponding detection of movement. As another example, an end condition may include failure to detect further changes in sequential scans of the scan sequence. In other words, the scan sequence may end after sequential impulses or images stop changing or detecting further movement. The end condition may specifically require a preset number or time period of unchanged sequential scans. As yet another example, an end condition may include detecting closure of the door to the chilled chamber. In other words, the scan sequence may end in response to the door being moved to the closed position.
At 420, the method 400 includes identifying a barcode capture for a product barcode of a stored item on the scan sequence. Specifically, 420 may provide for identifying that a product barcode is covered or within the field of view of the scanner assembly (i.e., identifying the presence of the product barcode) at 410. As an example, reflected optical impulses may be detected as a barcode reflection, as would be understood. As an additional or alternative example, a product barcode may be identified within a two-dimensional image captured at a camera of the scanner assembly, as would further be understood.
At 430, the method 400 includes processing barcode data for the barcode capture according to one or more predetermined user preferences. For instance, the barcode data for the product barcode that has been captured and identified may be decoded to obtain a string of text or characters. Such decoding is generally understood. If an attempt at decoding is unsuccessful, the method 400 may return to 410 (e.g., to obtain a new capture of the product barcode).
In some embodiments, the decoded barcode data is transmitted to a remote server. Specifically, the decoded barcode data may be transmitted from the controller of the refrigerator appliance via a connected network. At the remote server, the barcode data may be associated with the “personal data” (e.g., inventory or history of stored items) of a user profile. Additionally or alternatively, the barcode data may be matched (or attempted to be matched) to a corresponding product identifier (e.g., name of the product or stored item on which the captured and decoded product barcode is held). Matching the barcode data may include comparing the decoded barcode data to a database of programmed barcodes (e.g., stored within the remote server, or another network-connected server). Once matched, the method 400 may proceed to 440. Nonetheless, if matching the barcode is not possible (e.g., because no corresponding programmed barcode is found), the method 400 may still proceed to 440.
At 440, the method 400 includes establishing an item identifier for the stored item (e.g., captured within the scan of 410). Specifically, the item identifier may be based on the processed barcode data of 430. In other words, what the item identifier is (or can be) is determined, at least in part, by the results of the processing of the barcode data.
As an example, if the barcode data is matched to a programmed barcode, 440 may include receiving product data (e.g., the identification or name of the product, quantity or size of the product, pricing of the product, expected expiration of the product, etc.) included with or corresponding to the programmed barcode data. Thus, the received product data may relate to the stored item, which matches the corresponding programmed barcode.
As an additional or alternative example, if matching the barcode data fails, 440 may include initiating secondary identification steps. For instance, if permitted by a user's programmed preferences, a two-dimensional image of the stored item may be recorded. The two-dimensional image may be captured, for instance, at the scanner assembly (e.g., during 410 or subsequently in response to 430). If capturing or storing a two-dimensional image is not possible (e.g., because of user preferences, physical limitations, inadequate image conditions, etc.), a user may be prompted to manually input an item identifier (e.g., at the refrigerator appliance using a physical keyboard, voice dictation, or any other suitable input system, as would be understood).
At 450, the method 400 includes directing an update to a user list with the established item identifier. The user list may generally be provided as an inventory or shopping list of products that are either currently within chilled chamber (i.e., stored items) or have been determined to be needed (e.g., because a stored item has been removed from the chilled chamber, a stored item has expired, has otherwise been identified as being desirable for a user). The user list may be included with the “personal data” stored on the remote server or locally on the refrigerator appliance. Optionally, the user list may be accessible on a user's mobile display (e.g., provided on or with a personal computer, cell phone, tablet, stationary kitchen hub, etc.).
In some embodiments, the established item identifier may include one or more portion of the received product data. Thus, such portions of the product data may be stored on the user list so that the product may be readily and easily understood by a user. In additional or alternative embodiments, the established item identifier may include a two-dimensional image of 430. Thus, the capture two-dimensional image may be stored on the user list so that the product may be readily and easily understood by a user.
Turning now to FIG. 5, at 510, the method 500 includes initiating a scan sequence at the scanner module. The scan sequence may include capturing one or more reflected impulses or two-dimensional images, as described above. Specifically, one or more scans may be made of one or more particular stored items (e.g., within chilled chamber). Optionally, a plurality of scans may be obtained as part of the scan sequence. For instance, the scan sequence may include capturing a plurality of two-dimensional images (e.g., a first two-dimensional image, subsequent second two-dimensional image, etc.), such as within a video feed or a series of sequential static images (e.g., taken or captured according to a predetermined rate or condition). Upon being captured at the scanner module, the impulses or images (i.e., scans) may be transmitted to the controller (e.g., as a data signal). The capture(s) may then be recorded (e.g., temporarily) for further processing or evaluation.
In certain embodiments, 510 is contingent on identifying a specific user. For instance, the scan sequence, or any further recording or evaluating of impulses or images from the scanner assembly, may be prevented until the specific user is identified (e.g., automatically or, alternatively, via a manual selection). The identification of a particular user may include receiving a user-detection signal indicating a biometric or personalized marker corresponding to the specific individual user, as described above. As an example, the user-detection signal may include a captured two-dimensional image of the specific user's face. From the captured two-dimensional image, the user may be recognized. As another example, the user-detection signal may include a fingerprint image from which the specific user's fingerprint may be recognized. As yet another example, the user-detection signal may include a detected device address of the specific user's mobile display (e.g., detected over a wireless communications band, such as, a BLE band using short-wavelength UHF radio waves in the ISM band from 2.4 to 2.485 GHz).
In additional or alternative embodiments, the scan sequence is initiated in response to detecting movement within a chilled chamber (e.g., fresh food chamber) of the refrigerator appliance. Thus, recording or evaluating scans from the scanner module may be prevented until movement is detected (e.g., at the scanner module or a separate sensor). As an example, movement may be detected by reception of an optical impulse of light reflected from a beam emitter of the scanner module. As another example, changes in light or pixels captured by the scanner module may be detected (e.g., between multiple images captured over time) indicating the movement of one or more objects within the field of view of the scanner module, as is generally understood. As yet another example, movement may be detected in response to receiving a signal from a separate sensor, such as a switch selectively engaged with the door. Such switches are generally understood and may, for instance, simultaneously control activation of a light for illuminating the chilled chamber. Opening the refrigerator door may thus activate the light and transmit a signal indicating motion within the chilled chamber.
Optionally, the scan sequence may continue until one or more end conditions are met. As an example, an end condition may include expiration of a predetermined time period (e.g., span of time) following the start of a corresponding detection of movement. As another example, an end condition may include failure to detect further changes in sequential scans of the scan sequence. In other words, the scan sequence may end after sequential impulses or images stop changing or detecting further movement. The end condition may specifically require a preset number or time period of unchanged sequential scans. As yet another example, an end condition may include detecting closure of the door to the chilled chamber. In other words, the scan sequence may end in response to the door being moved to the closed position.
At 520, the method 500 includes identifying a barcode capture for a product barcode of a stored item on the scan sequence. Specifically, 520 may provide for identifying that a product barcode is covered or within the field of view of the scanner assembly (i.e., identifying the presence of the product barcode) at 510. As an example, reflected optical impulses may be detected as a barcode reflection, as would be understood. As an additional or alternative example, a product barcode may be identified within a two-dimensional image captured at a camera of the scanner assembly, as would further be understood.
At 530, the method 400 includes processing barcode data for the barcode capture according to one or more predetermined user preferences. Specifically, an attempt is made to decode the barcode data for the product barcode that has been captured and identified. Such decoding is generally understood to provide a string of text or characters. If the attempt at decoding is unsuccessful, the method 500 may return to 510 (e.g., to obtain a new capture of the product barcode).
Once the decoding is successful, 530 includes transmitting the decoded barcode data from the controller of the refrigerator appliance via a connected network. At the remote server, the barcode data is associated with the “personal data” (e.g., inventory or history of stored items) of a user profile. Moreover, the remote server may attempt to match the barcode data to a particular corresponding product identifier (e.g., name of the product or stored item on which the captured and decoded product barcode is held) with a set of programmed database of “product data.” Matching the barcode data may include comparing the decoded barcode data to a database of programmed barcodes (e.g., stored within the remote server, or another network-connected server). The programmed barcodes may include, for instance, the corresponding identification or name of the product, quantity or size of the product, pricing of the product, expected expiration of the product, etc.)
The method 500 may proceed according to the results of the matching. If data is available (i.e., the attempt to match the barcode data to a particular programmed barcode is successful), the method 500 may proceed with 542.
Furthermore, if no image is included, the method may proceed with 544.
If data is not available (i.e., the attempt to match the barcode data to a particular programmed barcode is unsuccessful), the method may proceed to 546 according to one or more user preferences (e.g., set by a user at the refrigerator appliance). Specifically, a user may indicate whether it is acceptable to update the user list without an image of a product and whether it is acceptable to identify products exclusively by an image. Thus, if a user has selected image-required updates or image-exclusive identification as being acceptable, the method 500 may proceed with 544. If a user has selected image-exclusive identification as being unacceptable, the method 500 may proceed with 546.
At 542, the method 500 includes transmitting product data (e.g., the identification or name of the product, quantity or size of the product, pricing of the product, expected expiration of the product, etc.) included with or corresponding to the programmed barcode data. For instance, the remote server may transmit the product data to the “personal data” associated with the user and viewable at the refrigerator appliance or a mobile display. As described above, the received product data may relate to the stored item, which matches the corresponding programmed barcode. Moreover, the product data may be established as or as part of an item identifier.
At 544, the method includes requesting a two-dimensional image corresponding to the product (i.e., stored item). The two-dimensional image may be captured by the refrigerator appliance, such as during 510 or, alternatively, in response to 542. Alternatively, the two-dimensional image may be requested by the remote server (e.g., using a search provider or separate connected remote server). Once requested and received, the two-dimensional image may be transmitted to the “personal data” associated with the user and viewable at the refrigerator appliance or remote or a mobile display. Moreover, the two-dimensional image may be established as or as part of an item identifier.
At 546, the method 500 includes prompting a manual input of an item identifier. Specifically, manual input may be requested (e.g., by a message, notification, or display at the refrigerator appliance). Once manual input is requested, such an input may be received from a user (e.g., at the refrigerator appliance using a physical keyboard, voice dictation, or any other suitable input system, as would be understood). The received manual input may be compared to a list of previous updates (e.g., identified items or products). For instance, if it is determined that the manual input matches (or is within an acceptable margin of error) a previous update, the manual input may be established as or as part of an item identifier.
Following 542, 544, or 546, the method 500 may proceed to 550.
At 550, the method 500 includes directing an update to a user list with the established item identifier. The user list may generally be provided as an inventory or shopping list of products that are either currently within chilled chamber (i.e., stored items) or have been determined to be needed (e.g., because a stored item has been removed from the chilled chamber, a stored item has expired, has otherwise been identified as being desirable for a user). The user list may be included with the “personal data” stored on the remote server or locally on the refrigerator appliance. Optionally, the user list may be accessible on a user's mobile display (e.g., provided on or with a personal computer, cell phone, tablet, stationary kitchen hub, etc.).
After updating the user list, a notification signal, such as a signal to prompt an audio or visual output, may be transmitted to the refrigerator appliance to inform the user that the user list has been updated.
This written description uses examples to disclose the invention, including the best mode, and also to enable any person skilled in the art to practice the invention, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they include structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal languages of the claims.

Claims

What is claimed is:

1. A refrigerator appliance comprising:

a cabinet defining a chilled chamber;

a door being rotatably hinged to the cabinet to provide selective access to the chilled chamber;

a scanner module mounted to the cabinet; and

a controller operably coupled to the scanner module, the controller being configured to initiate an operation routine comprising

initiating a scan sequence at the scanner module,

identifying a barcode capture for a product barcode of a stored item based on the scan sequence,

processing barcode data for the barcode capture according to one or more predetermined user preferences,

establishing an item identifier based on the processed barcode data, and

directing an update to a user list with the established item identifier.

2. The refrigerator appliance of claim 1, wherein the scanner module comprises a beam emitter or a camera directed to the chilled chamber.

3. The refrigerator appliance of claim 1, wherein processing barcode data comprises decoding the barcode capture.

4. The refrigerator appliance of claim 3, wherein processing barcode data comprises transmitting the decoded barcode data to a remote server.

5. The refrigerator appliance of claim 1, wherein establishing an item identifier comprises recording a two-dimensional image of the stored item.

6. The refrigerator appliance of claim 5, wherein directing the update to the user list comprises initiating storage of the two-dimensional image on the user list.

7. The refrigerator appliance of claim 5, wherein the operation routine further comprises capturing the two-dimensional image during the scan sequence.

8. The refrigerator appliance of claim 1, wherein establishing an item identifier comprises receiving product data relating to the stored item.

9. The refrigerator appliance of claim 8, wherein directing the update to the user list comprises initiating storage of one or more portion of the received product data on the user list.

10. A method of operating a refrigerator appliance, the method comprising:

initiating a scan sequence at the scanner module;

identifying a barcode capture for a product barcode of a stored item based on the scan sequence;

processing barcode data for the barcode capture according to one or more predetermined user preferences;

establishing an item identifier based on the processed barcode data; and

directing an update to a user list with the established item identifier.

11. The method of claim 10, wherein the scanner module comprises a beam emitter or a camera directed to the chilled chamber.

12. The method of claim 10, wherein processing barcode data comprises decoding the barcode capture.

13. The method of claim 12, wherein processing barcode data comprises transmitting the decoded barcode data to a remote server.

14. The method of claim 10, wherein establishing an item identifier comprises recording a two-dimensional image of the stored item.

15. The method of claim 14, wherein directing the update to the user list comprises initiating storage of the two-dimensional image on the user list.

16. The method of claim 14, further comprising capturing the two-dimensional image during the scan sequence.

17. The method of claim 10, wherein establishing an item identifier comprises receiving product data relating to the stored item.

18. The method of claim 17, wherein directing the update to the user list comprises initiating storage of one or more portion of the received product data on the user list.