JP7012261B2 - refrigerator - Google Patents

Description

The present invention relates to a refrigerator.

Conventionally, an internal camera was installed in the center of the food storage room, and when the door of the storage room was opened and closed, the storage room was imaged by the internal camera, and when the food was stored from the time-series captured images. A refrigerator has been proposed in which the image of the above and the current image are output side by side (see, for example, Patent Document 1). In the refrigerator described in Patent Document 1, depending on the arrangement of foods in the containment chamber, the foods may overlap in the image pickup direction of the camera in the refrigerator, and some foods may not be recognized from the captured image.
Therefore, we provided a switch to detect the opening and closing of the door of the containment chamber and a camera whose imaging range is the opening of the containment chamber. Alternatively, a refrigerator has been proposed in which the food to be delivered is imaged (see, for example, Patent Document 2).

Japanese Unexamined Patent Publication No. 2016-57022 Japanese Unexamined Patent Publication No. 2015-81762

In the refrigerator described in Patent Document 2, when the opening of the door of the containment chamber is detected, the food passing through the opening of the containment chamber is imaged. Therefore, depending on the timing from when the user opens the door to when the food is stored, the food may not be imaged. In addition, a part of the door boxet or the like is imaged instead of the food, and there is a possibility that something other than the food is erroneously recognized as the food.
The present invention has been made in view of such a background, and an object of the present invention is to provide a refrigerator capable of more reliably obtaining information on foods stored in a containment chamber.

In order to solve the conventional problem, the first aspect of the refrigerator of the present invention includes a storage chamber having an opening, a passing object detection unit for detecting an object passing through the opening, and a range including the opening. When an object passing through the opening is detected by the passing object detection unit, the camera captures the image, and the image obtained by the imaging is used to obtain the food to be stored in the storage chamber. The food information acquisition unit includes a food information acquisition unit for acquiring information and a storage capacity recognition unit for recognizing the amount of food contained in the storage chamber, and the food information acquisition unit passes through the opening by a passing object detection unit. When the object to be detected is detected and the amount of food in the containment chamber recognized by the accommodation capacity recognition unit does not change, the image captured by the camera is invalidated in response to the detection of the object. ..
A second aspect of the refrigerator of the present invention includes a storage chamber having an opening, a passing object detection unit that detects an object passing through the opening, a camera that captures an image of a range including the opening, and the opening. An opening / closing unit that opens / closes the unit, an opening / closing detection unit that detects the open / closed state of the opening / closing unit, and an image pickup by the camera when an object passing through the opening is detected by the passing object detection unit. The food information acquisition unit includes a food information acquisition unit that acquires information on the food stored in the storage chamber from the image obtained by the above-mentioned food information acquisition unit. When it is detected and it is detected by the open / close detection unit that the open / close portion is switched from the open state to the closed state within a predetermined time from the time when the object is detected, the camera responds to the detection of the object. It invalidates the captured image.

As a result, the food information acquisition unit can take an image of the food passing through the opening at the timing when an object (food, the hand of the user holding the food, etc.) passing through the opening of the storage portion is detected. can. Therefore, it is possible to obtain information on the food stored in the storage room more reliably by avoiding that the food stored in the storage room is not imaged and that non-food items such as the door of the refrigerator are imaged. be able to.

According to the refrigerator of the present invention, it is possible to more reliably obtain information on the food stored in the storage chamber by taking an image of the food stored in the storage chamber at an appropriate timing.

Explanatory diagram of the configuration to check the status of food stored in the refrigerator installed in the house with a terminal device Explanatory drawing of the appearance of the refrigerator Explanatory drawing showing the structure of the inside of the refrigerator by the cross-sectional view seen from the right side. Explanatory view showing the internal structure of the refrigerator compartment by a cross-sectional view seen from the front. Configuration diagram of the control unit Flowchart showing the overall processing by the control unit Flow chart of imaging process Flow chart of weight information processing Flowchart of image analysis process at the time of warehousing Flowchart of image analysis process at the time of delivery Flow chart of weight sensor correction process First flowchart of food management process in the warehouse Second flowchart of food management processing in the warehouse Explanatory drawing of the food list in the warehouse and the best-by date standard list

The first invention comprises a containment chamber having an opening, a passing object detection unit that detects an object passing through the opening, a camera that captures an image of a range including the opening, and the opening by the passing object detection unit. It is provided with a food information acquisition unit that takes an image with the camera when an object passing through the unit is detected and acquires information on the food to be stored in the storage chamber from the image obtained by the image pickup.
According to the first invention, when an object passing through an opening of a storage chamber is detected by a passing object detection unit, the food information acquisition unit captures a range including the opening with a camera. Therefore, it is possible to take an image of the food at the timing when the user has received the food, and thereby it is possible to more reliably acquire the information of the food to be stored in the storage room.

In the second invention, the detection range of the passing object detection unit is set closer to the front side than the image pickup range portion of the camera corresponding to the opening in the direction from the opening toward the accommodation chamber.
According to the second invention, the influence of the time delay from the time when the object passing through the opening of the accommodation chamber is detected by the passing object detection unit to the time when the image is taken by the camera is reduced, and the food is subjected to the camera. The timing at which the image is created can be set more appropriately.

The third invention includes a storage capacity recognition unit that recognizes the amount of food contained in the storage chamber, and the food information acquisition unit detects an object passing through the opening by a passing object detection unit. Moreover, when the amount of food in the storage room recognized by the storage capacity recognition unit does not change, the image captured by the camera is invalidated in response to the detection of the object.
According to the third invention, when an object passing through the opening is detected by the passing object detection unit, but the amount of food in the storage chamber recognized by the storage capacity recognition unit does not change, the food is transferred to the storage chamber. It can be determined that the goods have not been received. In this case, it is highly possible that what is imaged by the camera is not the food but the opening / closing part of the accommodating part, the user's hand, or the like. Therefore, by invalidating the image captured by the camera, it is possible to prevent erroneous food information from being acquired from the image of an object other than food.

The fourth invention includes an opening / closing unit for opening / closing the opening and an opening / closing detection unit for detecting the open / closed state of the opening / closing portion, and the food information acquisition unit passes through the opening by the passing object detection unit. When it is detected by the open / close detection unit that the open / close portion is switched from the open state to the closed state within a predetermined time from the time when the object is detected, the object is detected. The image captured by the camera is invalidated.
According to the fourth invention, it is detected that the open / close part is switched from the open state to the closed state by the open / close detection unit within a predetermined time from the time when the passing object detection unit detects the object passing through the opening. As a situation, it can be assumed that the opening / closing part is closed immediately after the detection of the passing object. In this case, it can be determined that the opening / closing part is detected by the passing object detection part, not the food. Then, in this case, it is highly possible that what is imaged by the camera is not a food product but an opening / closing portion. Therefore, by invalidating the image captured by the camera, it is possible to prevent erroneous food information from being acquired from the image of an object other than food.

In a fifth aspect of the invention, the food information acquisition unit starts energizing the camera when it is detected by the open / close detection unit that the open / close unit has switched from the closed state to the open state, and the camera It is in a state where it is possible to take an image by.
According to the fifth aspect of the present invention, when it is detected that the opening / closing portion has been opened, energization of the camera is started to enable imaging, so that the imaging timing is delayed due to the start-up time of the camera. , It is possible to prevent the image of the food to be stored from failing.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. The present invention is not limited to the present embodiment.

[1. How to use the refrigerator]
FIG. 1 shows a usage mode of the refrigerator of the present embodiment. The refrigerator 1 of the present embodiment is installed in the house H and has a function of communicating with the management server 520 via the gateway 500 and the communication network 510. Further, the refrigerator 1 has a function of communicating with a terminal device (smartphone, tablet terminal, etc.) 400. The terminal device 400 has a function of communicating with the management server 520 via the gateway 500 and the communication network 510.

The control unit 100 provided in the refrigerator 1 controls the overall operation of the refrigerator 1 and generates an in-fridge food list 131 showing information on foods stored in the storage room of the refrigerator 1 to generate a management server 520. Send to. The refrigerator management application executed in the terminal device 400 is obtained by acquiring the data of the food list 131 in the refrigerator from the management server 520 or by acquiring the data of the food list 131 in the refrigerator 1 from the refrigerator 1. Based on the food list 131, information on the food stored in the refrigerator 1 is displayed on the display unit. The user P of the refrigerator 1 can confirm the status (food name, expiration date, remaining amount, etc.) of the food contained in the refrigerator 1 by visually recognizing the display of the terminal device 400.

[2. Refrigerator configuration]
The configuration of the refrigerator 1 will be described with reference to FIGS. 2 to 5. FIG. 2 is an explanatory diagram showing the appearance of the refrigerator 1. As shown in FIG. 2, the refrigerator 1 includes a main box body 2 opened in the front surface, and inside the main box body 2, a refrigerating chamber 10 (corresponding to a storage chamber of the present invention), an ice making chamber 30, and an ice making chamber are provided. A switching chamber 20, a freezer compartment 40, and a vegetable compartment 50, which are attached to the 30 and can change the temperature inside the refrigerator, are formed.

A rotary right door 11 and a left door 12 (corresponding to the opening / closing portion of the present invention) are provided in the opening on the front surface of the refrigerating chamber 10. Further, the switching chamber 20, the ice making chamber 30, the freezing chamber 40, and the vegetable compartment 50 are provided with drawers 21, 31, 41, and 51 for accommodating food, respectively.

Next, FIG. 3 is an explanatory view showing the configuration of the inside of the refrigerator 1 by a cross-sectional view seen from the right side, and FIG. 4 is an explanatory view showing the internal configuration of the refrigerator compartment 10 by a cross-sectional view seen from the front. be. With reference to FIGS. 3 and 4, the refrigerating chamber 10 is provided with an upper shelf 13, a middle shelf 14, and a lower shelf 15 that divide the refrigerating chamber 10 into sub-areas. Further, in the refrigerating chamber 10, weight sensors 13a and 13b for detecting the weight of food placed on the upper shelf 13, weight sensors 14a and 14b for detecting the weight of food placed on the middle shelf 14, and a lower shelf 15 are used. Weight sensors 15a and 15b for detecting the weight of the placed food are provided. Further, the refrigerating chamber 10 is provided with lighting units 16a to 16f for illuminating the inside of the refrigerating chamber 10, illuminance sensors 17a to 17c for detecting the illuminance in the refrigerating chamber 10, and a camera 60.

The camera 60 is arranged near the opening 10a at the upper part of the refrigerating chamber 10 so that the opening 10a is included in the imaging range, and images the range from the opening 10a to the vicinity of the front ends of the shelves 13, 14, 15. .. Further, a compressor 61, a cooling fan 62, a cooler 63, and a condenser 64, which are auxiliary machines constituting the refrigeration cycle, are provided. Further, on the back surface of the refrigerating chamber 10, a refrigerating chamber duct 70 for circulating cold air and cold air discharge ports 71 to 73 are arranged.

The refrigerating chamber 10 has an open / close sensor 18b (corresponding to the open / close detection unit of the present invention) for detecting the opening / closing of the right door 11 and an open / close sensor 18a (corresponding to the open / close detection unit of the present invention) for detecting the opening / closing of the left door 12. ) And are provided. The open / close sensor 18b outputs a close detection signal when the right door 11 is closed, and outputs an open detection signal when the right door 11 is open. Similarly, the open / close sensor 18a outputs a close detection signal when the left door 12 is closed, and outputs an open detection signal when the left door 12 is open.

The switching chamber 20 is provided with an opening / closing sensor 22 for detecting the opening / closing of the opening 20a. The open / close sensor 22 outputs a close detection signal when the drawer 21 is housed in the switching chamber 20, and outputs an open detection signal when the drawer 21 is pulled out from the switching chamber 20.

Similarly, the ice making chamber 30 is provided with an opening / closing sensor 32 for detecting the opening / closing of the opening 30a by the drawer 31. Further, the freezing chamber 40 is provided with an opening / closing sensor 42 for detecting the opening / closing of the opening 40a by the drawer 41, and the vegetable compartment 50 is provided with an opening / closing sensor 52 for detecting the opening / closing of the opening 50a by the drawer 51. The open / close sensors 32, 42, and 52 output the close detection signal and the open detection signal, similarly to the open / close sensor 22.

Next, FIG. 5 is a configuration diagram of the control unit 100. With reference to FIG. 5, the control unit 100 includes an electron configured by a CPU (Central Processing Unit) 110, an image processing unit 120, a storage unit 130, a wireless communication unit 140, a date / time recognition unit 141, an interface circuit (not shown), and the like. It is a circuit unit.

By executing the control program of the refrigerator 1 stored in the storage unit 130, the CPU 110 executes a passing object detection unit 111, a storage capacity recognition unit 112, a warehousing / delivery determination unit 113, a food information acquisition unit 114, and a food information management unit. It functions as a unit 115.

The control unit 100 includes open / close sensors 18a, 18b, 22, 32, 42, 52, illuminance sensors 17a, 17b, 17c, weight sensors 13a, 13b, 14a, 14b, 15a, 15b, a camera 60, lighting units 16a, 16b, It is connected to 16c, 16d, 16e, 16f, and refrigeration cycle accessories 61, 62, 63, 64.

Detection signals of open / close sensors 18a, 18b, 22, 32, 42, 52, illuminance sensors 17a, 17b, 17c, weight sensors 13a, 13b, 14a, 14b, 15a, 15b and the like are input to the control unit 100. .. Further, the operation of the lighting units 16a, 16b, 16c, 16d, 16e, 16f, the refrigerating cycle auxiliary machines 61, 62, 63, 64 and the like is controlled by the control signal output from the control unit 100.

Further, the control signal output from the control unit 100 controls the image pickup by the camera 60 via the image processing unit 120. The image processing unit 120 converts the video signal output from the camera 60 into a digital tone signal to generate a captured image, and stores the captured image data in the image memory 121. The image processing unit 120 processes the captured image stored in the image memory 121 according to the control signal output from the CPU 110.

The date / time recognition unit 141 recognizes the current date / time (date and time) by clocking a clock signal output from a clock circuit (not shown) provided in the control unit 100. In addition, the information of the current date and time may be acquired from the management server 520 by communication with the management server 520, and the current date and time may be recognized.

In the storage unit 130, in addition to the data of the food list 131 in the refrigerator, the stored food information 132 in which the information of the food stored in the refrigerating chamber 10 is recorded and the information of the food discharged from the refrigerating chamber 10 are recorded. The data of the delivered food information 133 is stored. Further, the storage unit 130 stores data of sample images 134 of various foods and a best-by-date list 135 showing general best-by-dates of various foods.

The passing object detection unit 111 detects an object passing through the opening 10a of the refrigerating chamber 10 based on the change in the illuminance in the refrigerating chamber 10 detected by the illuminance sensors 17a, 17b, 17c. The storage amount recognition unit 112 determines the amount of food stored in the upper shelf 13, the middle shelf 14, and the lower shelf 15 of the refrigerating chamber 10 based on the detection signals of the weight sensors 13a, 13b, 14a, 14b, 15a, and 15b. And recognize by weight.

The storage amount recognition unit 112 recognizes the weight of the food stored in the upper shelf 13 based on the detection signals of the weight sensors 13a and 13b, and determines the weight of the food stored in the middle shelf 14 based on the detection signals of the weight sensors 14a and 13b. Recognize based on the detection signal of 14b. Further, the storage capacity recognition unit 112 detects the weight of the food stored in the lower shelf 15 based on the detection signals of the weight sensors 15a and 15b.

The warehousing / delivery determination unit 113 is placed on the shelves 13, 14, 15 recognized by the capacity recognition unit 112 when an object passing through the opening 10a of the refrigerating chamber 10 is detected by the passing object detection unit 111. Based on the change in the amount (weight) of the food, it is determined whether the food is stored in the refrigerating chamber 10 and the food is discharged from the refrigerating chamber 10.

The food information acquisition unit 114 takes an image of an object that has passed through the opening 10a of the refrigerating chamber 10 by the camera 60. Then, when the warehousing / delivery determination unit 113 determines that the food is warehousing in the refrigerating chamber 10 or the food is warehousing from the refrigerating chamber 10, the image portion of the food is extracted from the captured image.

The food information acquisition unit 114 recognizes the characters contained in the image portion of the food, or extracts the characteristics of the image portion of the food, so that the food name, content, place of origin, and taste of the food received or delivered can be obtained. Get information such as deadlines. Then, the food information acquisition unit 114 records the acquired information in the warehousing food information 132 for the food determined to be warehousing (stocked food), and the warehousing food information 133 for the food determined to be warehousing (delivered food). Record in.

The food information management unit 115 is based on the weight of the food in the refrigerating room 10 recognized by the storage capacity recognition unit 112, the in-stock food information 132 and the out-of-delivery food information 133 recorded by the food information acquisition unit 114, and the like. The food information recorded in the food list 131 is updated.

[3. Processing by control unit]
Next, with reference to FIGS. 6 to 14, a process relating to the management of the food contained in the refrigerating chamber 10 executed by the control unit 100 will be described.

FIG. 6 is a flowchart showing an outline of a series of processes executed by the control unit 100. The control unit 100 has "imaging processing" in step S1, "weight information processing" in step S2, "image analysis processing" in step S3, "weight sensor correction processing" in step S4, and "food management processing in the refrigerator" in step S5. "Is executed respectively. The details of each process will be described below.

[3-1. Imaging processing]
The execution procedure of the "imaging process" will be described according to the flowchart shown in FIG. The "imaging process" is executed by the passing object detection unit 111 and the food information acquisition unit 114. Steps S10 to S15 in FIG. 7 are processes by the passing object detection unit 111. When the passing object detection unit 111 determines in step S10 that one or both of the right door 11 and the left door 12 of the refrigerating chamber 10 has been opened from the detection signals of the open / close sensors 18a and 18b, the process is performed in step S11. Proceed.

In step S11, the passing object detection unit 111 detects each illuminance (illuminance of each space partitioned by the shelves) of the upper shelf 13, the middle shelf 14, and the lower shelf 15 of the refrigerating chamber 10 by the illuminance sensors 17a, 17b, 17c. .. Then, the passing object detection unit 111 uses the detected illuminance of the upper shelf 13 as the upper reference illuminance, the illuminance of the middle shelf 14 as the middle reference illuminance, and the illuminance of the lower shelf 15 as the lower reference illuminance. The passing object detection unit 111 stores the data of the upper reference illuminance, the middle reference illuminance, and the lower reference illuminance in the storage unit 130.

For the upper reference illuminance, the middle reference illuminance, and the lower reference illuminance, one or both of the right door 11 and the left door 12 of the refrigerating chamber 10 are opened, and an object (user's hand, food, etc.) is an opening 10a of the refrigerating chamber 10. It is the illuminance of each shelf 13, 14, 15 in the state before passing through. In consideration of the error and fluctuation of the detected illuminance, the upper reference illuminance, the middle reference illuminance, and the lower reference illuminance are set to illuminance slightly lower than or slightly higher than the detected illuminance of each illuminance sensor 17a, 17b, 17c. You may. Further, even if the upper reference illuminance, the middle reference illuminance, and the lower reference illuminance used in step S13, which will be described later, and the upper reference illuminance, the middle reference illuminance, and the lower reference illuminance used in step S14 are set to different values. good.

In the following step S12, the passing object detection unit 111 detects the illuminance of the upper shelf 13, the middle shelf 14, and the lower shelf 15 by the illuminance sensors 17a, 17b, and 17c, respectively. Then, in the next step S13, the passing object detection unit 111 determines whether or not there is a shelf whose detected illuminance is lower than the reference illuminance by a predetermined level or more. Specifically, the passing object detection unit 111 determines the success or failure of each of the following conditions (1) to (3).

(1) The detected illuminance of the illuminance sensor 17a changed from "upper reference illuminance" to "lower reference illuminance-α" or less.
(2) The detected illuminance of the illuminance sensor 17b changed from "medium reference illuminance" to "medium reference illuminance-α" or less.
(3) The detected illuminance of the illuminance sensor 17c changed from "lower reference illuminance" to "lower reference illuminance-α" or less.

However, α: Determined by experiments, computer simulations, etc., assuming a decrease in illuminance that occurs when an object (food, user's hand, etc.) enters the upper shelf 13, the middle shelf 14, or the lower shelf 15. A constant for setting the threshold. In addition, α may be a common value for the upper reference illuminance, the middle reference illuminance, and the lower reference illuminance, and may be set to different values individually for the upper reference illuminance, the middle reference illuminance, and the lower reference illuminance. May be good.

Then, when at least one of the above (1) to (3) is satisfied, the passing object detection unit 111 proceeds to the process in step S20. When at least one of the above (1) to (3) is satisfied, the user can put food in the refrigerating room 10 (warehousing), take out food from the refrigerating room 10 (delivery), or refrigerate room. It is presumed that the inside of the refrigerating room 10 was modified in order to change the position of the food of 10. Then, in this case, the passing object detection unit 111 can determine that the object (user's hand, food held by the user's hand, etc.) has passed through the opening 10a of the refrigerating chamber 10.

On the other hand, when none of the above (1) to (3) is satisfied, the passing object detection unit 111 proceeds to the process in step S14. In step S14, the passing object detection unit 111 determines whether or not there is a shelf whose detected illuminance has changed from "reference illuminance-β" or less to "reference illuminance" or more. Specifically, the passing object detection unit 111 determines the success or failure of each of the following conditions (4) to (6).

(4) The detected illuminance of the illuminance sensor 17a changed from "upper reference illuminance-β" or less to "upper reference illuminance" or more.
(5) The detected illuminance of the illuminance sensor 17b changed from "medium reference illuminance-β" or less to "medium reference illuminance" or more.
(6) The detected illuminance of the illuminance sensor 17c changed from "lower reference illuminance-β" or less to "lower reference illuminance" or more.

However, β: the illuminance generated when an object (food, user's hand, etc.) that has entered the upper shelf 13, the middle shelf 14, or the lower shelf 15 passes through the opening 10a and goes out of the refrigerating chamber 10. A constant for determining the threshold value determined by experiments, computer simulations, etc., assuming an increase in. Note that β may be set in common for the upper reference illuminance, the middle reference illuminance, and the lower reference illuminance, or may be set to different values for the upper reference illuminance, the middle reference illuminance, and the lower reference illuminance. You may.

Then, when at least one of the above (4) to (6) is satisfied, the passing object detection unit 111 proceeds to the process in step S22. When at least one of the above (4) to (6) is satisfied, the user can put the food in the refrigerator compartment 10 (warehousing), take out the food from the refrigerator compartment 10 (delivery), or the refrigerator compartment 10 It is presumed that the position of the food stored in the refrigerator was changed and the food stored in the refrigerator compartment 10 was removed. Then, in this case, the passing object detection unit 111 can determine that the object (user's hand, food held by the user's hand, etc.) has passed through the opening 10a of the refrigerating chamber 10.

On the other hand, when none of the above (4) to (6) is satisfied, the passing object detection unit 111 proceeds to the process in step S15. In step S15, the passing object detection unit 111 determines from the detection signals of the open / close sensors 18a and 18b whether or not both the right door 11 and the left door 12 of the refrigerating chamber 10 are closed. Then, when the right door 11 and the left door 12 are both closed, the passing object detection unit 111 proceeds to step S15 and ends the “imaging process”. On the other hand, when at least one of the right door 11 and the left door 12 is open, the passing object detection unit 111 proceeds to the process in step S12.

Next, steps S20 to S21 and steps S22 to S23 are processes by the food information acquisition unit 114. In step S20, the food information acquisition unit 114 captures a range including the opening 10a of the refrigerating chamber 10 by the camera 60 via the image processing unit 120. In the following step S21, the food information acquisition unit 114 stores the data of the image captured in step S20 as the data of the "image at the time of warehousing" in the storage unit 130, and proceeds to the process in step S15. It should be noted that the imaging may be performed a plurality of times and the data of the plurality of images may be stored in the storage unit 130 as the data of the “image at the time of warehousing”.

Further, in step S22, the food information acquisition unit 114 captures a range including the opening 10a of the refrigerating chamber 10 by the camera 60 via the image processing unit 120. In the following step S23, the food information acquisition unit 114 stores the data of the image captured in step S22 in the storage unit 130 as the data of the “image at the time of delivery”, and proceeds to the process in step S15. It should be noted that the imaging may be performed a plurality of times and the data of the plurality of images may be stored in the storage unit 130 as the data of the “image at the time of delivery”.

The refrigerating room is based on the change in the illuminance of the upper shelf 13, the middle shelf 14, or the lower shelf 15 detected by the illuminance sensors 17a, 7b, 17c by the passing object detection unit 111 by the "imaging process" shown in FIG. An object passing through the opening 10a of 10 is detected.

Further, the “image at the time of warehousing” is an image taken in the vicinity of the opening 10a at the time when the passing object detection unit 111 detects an object that has passed through the opening 10a and entered any of the shelves 13, 14, 15. Is stored in the storage unit 130. Further, the “image at the time of delivery” which is an image captured in the vicinity of the opening 10a at the time when the passing object detection unit 111 detects an object that has passed through the opening 10a from any of the shelves 13, 14, 15. , Stored in the storage unit 130.

In step S15 of FIG. 7, when an object passing through the opening 10a is detected a plurality of times in step 13 before it is detected that the right door 11 and the left door 12 are closed, steps S20 to S21. As a result, a plurality of "images at the time of warehousing" are captured and saved. Then, the "image analysis process" is executed for each of the "images at the time of warehousing" of the foods whose warehousing is determined by the "weight information processing" described later.

Similarly, in step S15 of FIG. 7, when an object passing through the opening 10a is detected a plurality of times in step S14 before it is detected that the right door 11 and the left door 12 are closed, steps S22 to S22 to By S23, a plurality of "images at the time of delivery" are captured and stored. Then, an "image analysis process" is executed for each of the "images at the time of delivery" of the food whose delivery has been determined by the "weight information processing" described later.

[3-2. Weight information processing]
Next, the execution procedure of the "weight processing" will be described according to the flowchart shown in FIG. "Weight information processing" is executed by the warehousing / delivery determination unit 113 and the food information acquisition unit 114.

Steps S30 to S33 and S40 in FIG. 8 are processes by the warehousing / delivery determination unit 113. In step S30, the warehousing / delivery determination unit 113 recognizes the weights of the shelves 13, 14 and 15 by the storage capacity recognition unit 112 and stores them in the storage unit 130. In the following step S31, when an object passing through the opening 10a of the refrigerating chamber 10 is detected by the passing object detection unit 111 in step S13 or step S14 of FIG. 7 (the object has entered and exited the refrigerating chamber 10). When), the warehousing / delivery determination unit 113 proceeds to the process in step S32.

In step S32, the warehousing / delivery determination unit 113 recognizes the weights of the shelves 13, 14 and 15 by the storage capacity recognition unit 112. In the next step S33, the warehousing / delivery determination unit 113 determines whether or not there is a shelf in which the weight recognized in step S32 has increased with respect to the weight recognized in step S30. Then, when there is a shelf whose weight has increased (in this case, it can be determined that the food has been stored in the refrigerating chamber 10), the warehousing / delivery determination unit 113 proceeds to the process in step S34.

On the other hand, when there is no shelf whose weight has increased, the warehousing / delivery determination unit 113 proceeds from step 33 to step S40, and there is a shelf in which the weight recognized in step S32 is reduced with respect to the weight recognized in step S30. Judge whether or not. Then, when there is a shelf whose weight has been reduced (in this case, it can be determined that the food has been delivered from the refrigerating chamber 10), the warehousing / delivery determination unit 113 proceeds to the process in step S41. Further, when there is no shelf whose weight has increased (in this case, it can be determined that the food has not been warehousing in the refrigerating room 10 and the food has not been warehousing from the refrigerating room 10), the warehousing / delivery determination unit 113 may perform step S36. Proceed to process and end "weight information processing".

Steps S34 to S35 and steps S41 are processes by the food information acquisition unit 114. In step S34, the food information acquisition unit 114 estimates the storage position of the stored food from the left and right weights of the shelves 13, 14, and 15 recognized by the storage capacity recognition unit 112.

In the next step S35, the food information acquisition unit 114 records information on the weight and storage position of the stored food in the stored food information 132 (see FIG. 5), proceeds to the process in step S36, and "weight information processing". To finish. The data of the in-stock food information 132 is temporarily stored in the storage unit 130 in order to add or update the information to the in-store food list 131.

Further, in step S41, the food information acquisition unit 114 records the weight information of the delivered food in the delivered food information 133 (see FIG. 5), proceeds to the process in step S36, and ends the "weight information processing". The data of the delivered food information 133 is temporarily stored in the storage unit 130 in order to add or update the food information to the in-store food list 131.

[3-3. Image analysis processing]
Next, the execution procedure of the "image analysis process" will be described according to the flowcharts shown in FIGS. 9 and 10. The "image analysis process" is executed by the food information acquisition unit 114. FIG. 9 is a process of acquiring information from the “image at the time of warehousing” of the food warehousing in the refrigerating room 10. Further, FIG. 10 is a process of acquiring information from the “image at the time of delivery” of the food delivered from the refrigerating room 10.

First, with reference to FIG. 9, the "image analysis process" for the "image at the time of warehousing" will be described. In step S50 of FIG. 9, the food information acquisition unit 114 executes character extraction processing by the image processing unit 120 for the “image at the time of warehousing” of the warehousing food. Here, the warehousing food is the food that is determined to have been warehousing in the refrigerating chamber 10 by the warehousing / delivery determination unit 113 in step S33 of FIG. Further, the “image at the time of warehousing” of the warehousing food is a “image at the time of warehousing” captured and stored by the food information acquisition unit 114 in steps S20 and S21 of FIG. 7 for the food determined to be warehousing in step S33 of FIG. ".

In the following step S51, the food information acquisition unit 114 determines whether or not the character has been extracted from the "image at the time of warehousing" by the image processing unit 120. Then, the food information acquisition unit 114 proceeds to the process in step S52 when the character is extracted, and proceeds to the process in step S60 when the character is not extracted.

In step S52, the food information acquisition unit 114 analyzes the character information extracted by the image processing unit 120. Then, in the following step S53, the food information acquisition unit 114 acquires information on the stored food such as the food name, the expiration date, the production area, and the content from the text information. In the next step S54, the food information acquisition unit 114 records the acquired information on the stored food in the stored food information 132, and ends the “image analysis process”.

Further, in step S60, the food information acquisition unit 114 extracts the image portion of the stored food from the “image at the time of storage” by the image processing unit 120. In the following step S61, the food information acquisition unit 114 calculates the matching rate between the extracted image portion of the stored food and the sample images 134 (see FIG. 5) of various foods previously stored in the storage unit 130.

In the next step S62, the food information acquisition unit 114 presumes and acquires the food name of the sample image 134 having the highest matching rate as the food name of the stored food, and proceeds to the process in step S54. By the processing of steps S60 to S62, even when characters cannot be extracted from the "image at the time of warehousing" of the warehousing food, the food name of the warehousing food can be estimated and recorded in the warehousing food information 132.

Next, with reference to FIG. 10, the "image analysis process" for the "image at the time of delivery" will be described. In step S70 of FIG. 10, the food information acquisition unit 114 executes a character extraction process by the image processing unit 120 for the “image at the time of delivery” of the food to be delivered. Here, the delivered food is the food determined to have been delivered from the refrigerating chamber 10 by the warehousing / delivery determining unit 113 in step S40 of FIG. Further, the “image at the time of delivery” of the food to be delivered is a “image at the time of delivery” captured and stored by the food information acquisition unit 114 in steps S22 and S23 of FIG. 7 for the food determined to be delivered in step S40 of FIG. ".

In the following step S71, the food information acquisition unit 114 determines whether or not the character has been extracted from the “image at the time of delivery” by the image processing unit 120. Then, the food information acquisition unit 114 proceeds to the process in step S72 when the character is extracted, and proceeds to the process in step S80 when the character is not extracted.

In step S72, the food information acquisition unit 114 analyzes the character information extracted by the image processing unit 120. Then, in the following step S73, the food information acquisition unit 114 acquires information on the delivered food such as the food name, the expiration date, the production area, and the content from the text information. In the next step S74, the food information acquisition unit 114 records the acquired food information in the food information 133, and ends the “image analysis process”.

Further, in step S80, the food information acquisition unit 114 extracts the image portion of the food to be delivered from the “image at the time of delivery” by the image processing unit 120. In the following step S81, the food information acquisition unit 114 calculates the matching rate between the extracted image portion of the delivered food and the sample image 134 of various foods previously stored in the storage unit 130.

In the next step S82, the food information acquisition unit 114 presumes and acquires the food name of the sample image 134 having the highest match rate as the food name of the delivered food, and proceeds to the process in step S74. By the processing of steps S80 to S82, even when characters cannot be extracted from the "image at the time of delivery" of the food to be delivered, the food name of the food to be delivered can be estimated and recorded in the food information 133 to be delivered.

The image analysis process of FIG. 9 is performed on the image at the time of warehousing when the warehousing to the refrigerating room 10 is determined, and the image analysis processing of FIG. 10 is the warehousing when the warehousing from the refrigerating room 10 is determined. It is performed for the time image. Therefore, when the right door 11 or the left door 12 is detected by the passing object detection unit 111 instead of the food, the image analysis process is not performed because it is not determined to be warehousing or warehousing (the captured image becomes invalid). This makes it possible to prevent erroneous food information from being acquired from the image portion of the right door 11 or the left door 12.

[3-4. Weight sensor correction process]
According to the flowchart shown in FIG. 11, the execution procedure of the “weight sensor calibration process” for correcting the detected weight of the weight sensors 13a, 13b, 14a, 14b, 15a, 15b will be described. The "weight sensor correction process" is executed by the accommodation capacity recognition unit 112. The "weight sensor correction process" is a process for correcting (calibrating) the detection error of the weight sensors 13a, 13b, 14a, 14b, 15a, 15b that may occur due to changes over time.

In step S90 of FIG. 11, when the storage capacity recognition unit 112 determines that the food has been stored in the refrigerating room by the warehousing / delivery determination unit 113 (YES in step S33 of FIG. 8), the processing proceeds to step S91. .. In step S91, the storage capacity recognition unit 112 determines whether or not the information on the content of the stored food is recorded in the stored food information 132 (processing in step S54 in FIG. 9). Then, when the storage amount recognition unit 112 records the information of the content of the stored food (the content of the food acquired in step S53 of FIG. 9) in the stored food information 132, the processing proceeds to step S92.

In step S92, the storage capacity recognition unit 112 calculates the ratio between the weight of the stored food and the content recorded in the stored food information 132. Then, the accommodation capacity recognition unit 112 sets the calculated ratio as the correction coefficient of the weight sensor used for detecting the weight of the stored food. For example, when the stored food is stored in the upper shelf 13 and the weight is detected by the weight sensors 13a and 13b, the storage capacity recognition unit 112 sets the calculated ratio as the correction coefficient of the detected weight of the weight sensors 13a and 13b. do.

In the following step S93, the accommodation capacity recognition unit 112 stores the correction coefficient data in the storage unit 130, proceeds to the process in step S94, and ends the "weight sensor correction process". Hereinafter, the accommodation capacity recognition unit 112 treats the value obtained by multiplying the detected weights of the weight sensors 13a and 13b by the correction coefficient as the detected weight. Similarly, the storage capacity recognition unit 112 sets the correction coefficients of the weight sensors 14a and 14b when the stored food is stored in the middle shelf 14, and when the stored food is stored in the lower shelf 15, the weight is set. The correction coefficients of the sensors 15a and 15b are set.

On the other hand, if the content information is not recorded in the in-stock food information 132 in step S91, the accommodation capacity recognition unit 112 proceeds to step S94 and ends the “weight sensor correction process”. In this case, the correction coefficient is not calculated or set.

[3-5. Food management processing in the warehouse]
According to the flowcharts shown in FIGS. 12 and 13, the execution procedure of the “food management process in the refrigerator” for managing the information of the food stored in the refrigerating chamber 10 will be described. The “food management process in the refrigerator” is executed by the food information management unit 115.

The food information management unit 115 manages the information of the food stored in the refrigerating room 10 by the food list 131 in the refrigerator (see FIG. 5). As shown in FIG. 14, in the food list 131 in the refrigerator, the numbers (1, 2, 3, ...) Attached to the foods stored in the refrigerating room 10 and the information of the foods stored in the refrigerators are numbered. The food name, weight (first time, current, amount of food), warehousing / delivery date / time (first warehousing, latest warehousing, latest warehousing) inventory (1: yes, 0: none), expiration date, and place of origin are recorded. .. The food information management unit 115 generates and updates the in-store food list 131 based on the in-stock food information 132 and the outbound food information 133 recorded by the food information acquisition unit 114.

In step S100 of FIG. 12, the food information management unit 115 determines whether or not the warehousing / delivery determination unit 113 determines whether or not the food is warehousing in the refrigerating room 10 or the food is warehousing from the refrigerating room 10. When the delivery is determined, the process proceeds to step S101. In step S101, the food information management unit 115 determines whether or not the warehousing is determined, and if the warehousing is determined, proceeds to step S102. On the other hand, when the warehousing is not determined (when the warehousing is determined), the food information management unit 115 proceeds to step S110.

In step S102, the food information management unit 115 determines whether or not there has been a delivery from the refrigerating room 10 within a predetermined time. Then, if there is a warehousing from the refrigerating room 10 within a predetermined time, the process proceeds to step S103, and if there is no warehousing from the refrigerating room 10 within a predetermined time, the process proceeds to step S111. It can be recognized from the food list 131 in the refrigerator that the refrigerator has been delivered from the refrigerator within a predetermined time. Further, the predetermined time is set to, for example, about 10 minutes, assuming that the user uses a part of the food discharged from the refrigerating room 10 and returns it to the refrigerating room 10 (restocking).

In step S103, the food information management unit 115 determines whether or not the matching rate between the food delivered last time and the food received this time is equal to or higher than a predetermined value. Here, the food information management unit 115 compares the information of the previously delivered product recorded in the food list 131 in the warehouse with the product and information recorded this time in the stored food information 132, and foods. From the matching status of the name, expiration date, production area, etc., the matching rate between the food that was delivered last time and the food that was received this time is calculated.

Then, when the matching rate between the food delivered last time and the food received this time is equal to or higher than the predetermined value, the food information management unit 115 proceeds to step S104, and the food delivered last time and the food received this time are received. When the concordance rate with is less than a predetermined value, the process proceeds to step S112.

Step S104 is a process in which the matching rate between the food delivered last time and the food received this time is equal to or higher than a predetermined value, and it can be determined that the food delivered last time and the food received this time are the same. Is. In step S104, the food information management unit 115 sets the current weight of the food delivered before the predetermined time, the latest warehousing date and time, and the amount of food (difference between the first warehousing and the current weight) in the food list 131 in the warehousing. It is updated based on the information recorded in the information 132, and the process proceeds to step S105 in FIG.

Step S112 is a process in which the matching rate between the food delivered this time and the food delivered this time is less than a predetermined value, and it can be determined that the food delivered last time and the food received this time are different. In step S112, the food information management unit 115 newly registers the stored food based on the information recorded in the stored food information 132 in the stored food list 131, and proceeds to the process in step S105 of FIG.

Step S111 is a process when there is no delivery within a predetermined time. In step S111, the food information management unit 115 newly registers the stored food based on the information recorded in the stored food information 132 in the stored food list 131, and proceeds to the process in step S105 of FIG.

Step S110 is a process when it is determined that the food has been delivered from the refrigerating chamber 10. In step S110, the food information management unit 115 updates the current weight of the corresponding food (delivered food) to 0 in the in-store food list 131. Further, the food information management unit 115 updates the amount of food consumed in the food list 131 in the warehouse, updates the latest delivery time based on the information recorded in the food information 133, and steps 105 in FIG. Proceed to the process.

In step S105 of FIG. 13, the food information management unit 115 determines whether or not there is a food whose weight is currently 0 gram by the processing of step S110. Then, the food information management unit 115 proceeds to step S105 when there is a food having a weight of 0 g, and proceeds to step S120 when there is no food having a weight of 0 g.

In step S106, the food information management unit 115 updates the information of the corresponding food (food determined to have been delivered this time) in the food list 131 in the warehouse to'out of stock'(stock 0), and proceeds to step S107. Proceed with processing. Further, in step S120, the food information management unit 115 updates the information of the corresponding food (food determined to have been received this time) in the food list 131 in the warehouse to'in stock'(inventory 1), and steps. Proceed to S107.

In step S107, the food information management unit 115 determines whether or not the information on the expiration date by character recognition is recorded in the in-stock food information 132. Then, the food information management unit 115 proceeds to the process in step S108 when the information on the expiration date is recorded in the stored food information 132, and proceeds to the process in step S121 when the information on the expiration date is not recorded.

In step S108, the food information management unit 115 registers the expiration date of the food received this time in the in-store food list 131 based on the expiration date information recorded in the stored food information 132, and processes in step S109. Proceed. Further, in step S121, the food information management unit 115 searches for a general expiration date of the corresponding food with reference to the expiration date list 135 stored in the storage unit 130.

As shown in FIG. 14, the best-by-date list 135 is a list in which the food name and the best-by-date of the food are associated with each other, and the food information management unit 115 has the best-by-date associated with the food name of the food in stock this time. Search for deadlines. In the following step S122, the food information management unit 115 registers the expiration date in the data of the corresponding food in the food list 131 in the refrigerator, and proceeds to the process in step S109.

In step S109, the food information management unit 115 transmits the updated data of the food list 131 in the refrigerator to the management server 520 via the communication network 510 by the wireless communication unit 140, and ends the “food management process in the refrigerator”. do.

The management server 520 stores the data of the food list 131 in the refrigerator received from the refrigerator 1 in a storage device (not shown), and the eating status of each product recognized from the food list 131 in the refrigerator, the number of days remaining until the expiration date, etc. Is transmitted to the user's terminal device 400. As a result, the user can check the status of the products stored in the refrigerator 1 by the terminal device 400 even when he / she is out.

[4. Other embodiments]
In the above embodiment, the passing object detection unit 111 detects an object passing through the opening 10a of the refrigerating chamber 10 by the change of the illuminance in the refrigerating chamber 10 detected by the illuminance sensors 17a, 17b, 17c. As another configuration of the passing object detection unit 111, as shown in FIGS. 3 to 5, the objective sensor 80 (reflection type or transmission type) that includes the opening 10a in the detection range and directly detects an object passing through the opening 10a. Optical sensor, ultrasonic sensor, capacitance sensor, etc.) may be used. In this case, the detection range of the objective sensor 80 may be set closer to the front side of the image pickup range of the camera 60 in the direction from the opening 10a toward the inside of the refrigerating chamber 10. As a result, it is possible to prevent the image of the food to be stored from failing due to the delay time from the time when the object passing through the opening 10a is detected to the time when the image is taken by the camera 60.

Further, when the opening / closing sensor 18a detects that the left door 12 has been opened, and the opening / closing sensor 18b detects that the right door 11 has been opened, the camera 60 is energized to start energizing the camera. Imaging by 60 may be possible. According to this configuration, when the left door 12 and the right door are closed, the power supply to the camera 60 can be cut off to save power. Further, by starting the energization of the camera 60 at the timing when the right door or the left door 12 is opened, it is possible to prevent the image of the food to be stored from failing due to the delay in starting the camera 60.

Further, in the food information acquisition unit 114, the left door 12 is switched from the open state to the closed state by the open / close sensor 18a within a predetermined time from the time when the passing object detection unit 111 detects the object passing through the opening 10a. When it is detected that the right door 11 is switched from the open state to the closed state by the open / close sensor 18b, the image data captured by the camera 60 may be invalidated. According to this configuration, when the left door 12 or the right door 11 is closed, when the left door 12 or the right door 11 is detected by the passing object detection unit 111 and imaged by the camera 60, the left door 12 or the right It is possible to prevent erroneous food information from being extracted from the image of the door 11.

In the above embodiment, the storage capacity recognition unit 112 is the food stored in the shelves 13, 14, 15 of the refrigerating chamber 10 based on the weight detected by the weight sensors 13a, 13b, 14a, 14b, 15a, 15b. Recognized the amount. As another configuration of the accommodation capacity recognition unit 112, each of the refrigerating chambers 10 is based on the detected illuminance by the illuminance sensors 17a, 17b, 17c in the state where the lighting units 16a, 16b, 16c, 16d, 16e, 16f are lit. The amount of food contained in the shelves 13, 14 and 15 may be recognized. Alternatively, the amount of food contained in the shelves 13, 14, 15 of the refrigerating chamber 10 can be determined by extracting an image portion of the food from the image captured inside the refrigerating chamber 10 by the camera 60 or a separately arranged camera. You may recognize it.

In the above embodiment, the image processing unit 120 performs the processing on the captured image of the camera 60, but the CPU 110 may be provided with the function of the image processing unit 120 and the CPU 110 may perform the processing on the captured image.

In the above embodiment, the data of the sample image 134 and the expiration date list 135 are stored in the storage unit 130, but they are stored in the management server 520 and downloaded from the management server 520 to the control unit 100 of the refrigerator 1 at the time of use. You may.

Further, although the sample image 134 is stored in the storage unit 130 or the management server 520, an image recognition program in which the characteristics of each food or stored item such as color, shape, and size are learned in advance is mounted on the CPU 110. , The image portion of the stored food may be processed by an image recognition program to calculate the matching rate and presumed to be the food name of the stored food. As a result, the sample image 134 becomes unnecessary, the storage capacity stored in the storage unit 130 or the management server 520 can be reduced, and the recognition processing time for estimating the food name of the stored food can be processed quickly. The estimation of the food name of the delivered food can be processed in the same manner as described above.

In the above embodiment, for the refrigerating room 10 of the refrigerator 1, the warehousing / delivery determination unit 113 determines the warehousing / delivery of food, the food information acquisition unit 114 acquires information on the warehousing food and the warehousing food, and the food information management unit. Information on the food contained in the refrigerating room 10 was managed by 115. The switching chamber 20, the freezing chamber 40, and the vegetable compartment 50 also have a passing object detection unit that detects an object passing through the openings 20a, 40a, and 50a, and a storage capacity recognition unit that recognizes the amount of food contained. Thereby, it is possible to determine the receipt and delivery of food. Further, by providing a camera that captures an image of a range including the openings 20a, 40a, 50a and acquiring information on the in-stock food and the out-of-stock food from the captured image, the food is accommodated in the switching room 20, the freezing room 40, and the vegetable room 50. It is possible to manage food by acquiring information on the food that has been prepared.

As described above, the refrigerator according to the present invention can more reliably acquire information on the food stored in the containment chamber by photographing the food stored in the containment chamber at an appropriate timing. Therefore, the refrigerator can be obtained more reliably. It can be applied to the purpose of managing the food contained in the refrigerator.

1 Refrigerator 10 Refrigerator room 10a Opening 11 Right door 12 Left door 13 Upper shelf 13 Shelf 13a, 13b Weight sensor 14 Middle shelf 14a, 14b Weight sensor 15 Lower shelf 15a, 15b Weight sensor 16a, 16b, 16c, 16d, 16r, 16f Illumination unit 17a, 17b, 17c Illumination sensor 18a, 18b Open / close sensor 20 Switching room 20a Opening / closing sensor 30 Ice making room 30a Opening / closing sensor 32 Step 32 Opening / closing sensor 40 steps 40 Freezing room 40a Opening / closing sensor 50 Vegetables Room 50a Opening 52 Open / close sensor 60 Camera 61 Compressor 62 Cooling fan 63 Cooler 64 Condenser 100 Control unit 110 CPU
111 Passing object detection unit 112 Storage capacity recognition unit 113 Entry / exit judgment unit 114 Food information acquisition unit 115 Food information management unit 120 Image processing unit 121 Image memory 130 Storage unit 131 In-ware food list 132 In-store food information 133 Outgoing food information 134 Samples Image 135 Expiration date list 140 Wireless communication unit 141 Date and time recognition unit 400 Terminal device 500 Gateway 510 Communication network 520 Management server

Claims (5)

A containment room with an opening and
A passing object detection unit that detects an object passing through the opening, and a passing object detection unit.
A camera that captures the area including the opening, and
Food information in which an image is taken by the camera when an object passing through the opening is detected by the passing object detection unit, and information on food to be stored in the storage chamber is acquired from the image obtained by the image pickup. Acquisition department and
A storage capacity recognition unit that recognizes the amount of food stored in the storage chamber is provided.
In the food information acquisition unit, when an object passing through the opening is detected by the passing object detection unit and the amount of food in the storage chamber recognized by the storage capacity recognition unit does not change, the food information acquisition unit of the object. The image captured by the camera is invalidated according to the detection.
A refrigerator that features that. An opening / closing part that opens / closes the opening, and an opening / closing part.
It is provided with an open / close detection unit that detects the open / closed state of the open / close portion.
When the opening / closing detection unit detects that the opening / closing unit has switched from the closed state to the open state, the food information acquisition unit starts energizing the camera and is capable of taking an image with the camera. The refrigerator according to claim 1 , wherein the refrigerator is characterized by the above. A containment room with an opening and
A passing object detection unit that detects an object passing through the opening, and a passing object detection unit.
A camera that captures the area including the opening, and
An opening / closing part that opens / closes the opening, and an opening / closing part.
An open / close detection unit that detects the open / closed state of the open / close unit,
Food information in which an image is taken by the camera when an object passing through the opening is detected by the passing object detection unit, and information on food to be stored in the storage chamber is acquired from the image obtained by the image pickup. With the acquisition department ,
In the food information acquisition unit, an object passing through the opening is detected by the passing object detection unit, and the opening / closing unit is closed from the open state by the opening / closing detection unit within a predetermined time from the time when the object is detected. When it is detected that the state has been switched, the image captured by the camera is invalidated according to the detection of the object.
A refrigerator that features that. When the opening / closing detection unit detects that the opening / closing unit has switched from the closed state to the open state, the food information acquisition unit starts energizing the camera and is capable of taking an image with the camera. The refrigerator according to claim 3 , wherein the refrigerator is characterized by the above. The claim is characterized in that the detection range of the passing object detection unit is set closer to the front side of the image pickup range portion of the camera corresponding to the opening in the direction from the opening toward the accommodation chamber. The refrigerator according to any one of claims 1 to 4 .

Images