JP6335847B2 - refrigerator - Google Patents

Description

  The present invention relates to a refrigerator for storing food, and more particularly, to a refrigerator capable of grasping the state of food stored in a storage room with an image.

  Various types of foods are stored in household refrigerators as the eating habits diversify, and the capacity of refrigerators is increasing in order to secure storage capacity. As a result, it is difficult for the user to grasp the storage state in the refrigerator. In particular, in recent years, with the increase in double-income households, both couples have not accurately grasped the storage condition in the refrigerator, and there are cases where the expiration date has passed and food has been left untreated. is increasing.

  In addition, when sharing the shopping between couples, the number of foods stored in the refrigerator is not noticed, and as a result of purchasing the foods in duplicate, there is an increasing number of cases where they cannot be consumed before the expiration date and are discarded. That is, there are an increasing number of cases where food is discarded wastefully because the state of storage in the refrigerator is not grasped.

  In order to suppress the disposal of food, it is necessary to grasp the food stored in the refrigerator and its deterioration state. For that purpose, it is easiest to open the refrigerator door and check the inside of the refrigerator, but increasing the frequency of opening and closing the door will cause the temperature in the refrigerator to rise, so energy is required for re-cooling, and , It tends to cause deterioration of the food stored.

  Therefore, Patent Document 1 discloses a technique for capturing an image of a storage room of a refrigerator with an imaging device, comparing and analyzing the captured image and a reference image, and calculating and displaying an empty capacity or an empty space in the storage room. Yes. Thereby, the user can purchase food according to the empty space in the storage room. That is, it is possible to prevent the food from being purchased excessively and packed in the storage room.

  Further, in Patent Document 2, a wireless tag is attached at the time of purchase of food, and another wireless tag is attached at the time of opening the food. Based on transmission information from these wireless tags and temperature information from a temperature sensor in the refrigerator. Thus, a technique for estimating quality degradation characteristics of food is disclosed.

Japanese Patent Laying-Open No. 2014-209048 (see, for example, paragraphs 0041-0049 and FIGS. 8-9) Japanese Patent Laying-Open No. 2006-317109 (see, for example, paragraphs 0027 to 0042 and FIG. 6)

  However, the technique disclosed in Patent Document 1 cannot grasp the deterioration state of food stored in the refrigerator. Therefore, there is a problem that it is not possible to eliminate the situation where food is left in the refrigerator and discarded due to its expiration date. Moreover, since the space in a warehouse is occupied by the food which the expiration date passed, space efficiency falls, and there also exists a problem that a cooling load increases for the said food and cooling efficiency falls.

  In the technique disclosed in Patent Document 2, since it is necessary to attach two types of wireless tags to all foods to be stored and to provide a reader / writer on the refrigerator main body, there is a problem that the cost of the entire system increases. In addition, since the refrigerator is generally surrounded by metal, there is a problem that it is difficult to ensure the distance and directivity of wireless communication. Since the information obtained from the wireless tag is mainly character output data, there is also a problem that it is difficult for the user to recognize, for example, the deterioration state of food.

  The present invention has been made to solve the above-described problem, and provides a refrigerator in which a user can visually check food stored in a storage room and obtain information on deterioration of the food. With the goal.

The refrigerator of the present invention includes a storage room for storing food, a photographing device for photographing the storage room, a control device for extracting and managing food images from images in the storage room photographed by the photographing device, and a control device. With respect to the food image that has been set, a user has a time limit setting unit for setting a time limit, and a display unit. The control device includes a time change detection unit that detects a time change of the food image, a food deterioration state estimation unit that estimates a food deterioration tendency based on the time change of the food image detected by the time change detection unit, and a food deterioration Based on the deterioration tendency of food estimated by the state estimation unit, a deadline estimation unit that estimates a deadline for food, and a set temperature in the storage room in which the deadline estimated by the deadline estimation unit can be after the deadline set by the user And a preset temperature calculation unit. The control device displays both the food image and information regarding the deterioration of the food corresponding to the food image on the display unit.

  According to the present invention, since the food image and the information regarding the deterioration of the food corresponding to the food image are displayed on the display unit, the user can easily grasp the food that must be consumed early. . Thereby, for example, the user can be prompted to consume food by the expiration date, and wasteful disposal can be suppressed. Further, it is possible to reduce waste of space and energy in the storage chamber due to continuing to store foods that have passed the time limit. Moreover, since a food image can be confirmed without opening the door of the refrigerator, an increase in temperature in the storage chamber due to an increase in the frequency of opening and closing the door can be suppressed, and energy consumption can be reduced.

It is a front view which shows the structure of the refrigerator which concerns on Embodiment 1 of this invention. It is a sectional side view of the arrow direction along line segment II-II of FIG. 1 which shows the structure of the refrigerator compartment of the refrigerator which concerns on Embodiment 1 of this invention. It is a flowchart for demonstrating operation | movement of the refrigerator which concerns on Embodiment 1 of this invention. It is a figure which shows an example of the incidental information of the picked-up image of the refrigerator compartment of the refrigerator which concerns on Embodiment 1 of this invention, the food image extracted from the picked-up image, and a food image. It is a figure which shows an example of the display screen of the food image in the operation panel of the refrigerator which concerns on Embodiment 1 of this invention. It is a figure which shows an example of the display screen of the food image in the operation panel of the refrigerator which concerns on Embodiment 1 of this invention. It is a schematic diagram for demonstrating the time change detection method of the food image in the time change detection part of the refrigerator which concerns on Embodiment 1 of this invention. It is a figure which shows an example of the display screen of the food image in the operation panel of the refrigerator which concerns on Embodiment 1 of this invention. It is a sectional side view which shows the structure of the refrigerator compartment of the refrigerator which concerns on Embodiment 2 of this invention. It is a sectional side view which shows the structure of the refrigerator compartment of the refrigerator which concerns on Embodiment 3 of this invention. It is a sectional side view which shows the structure of the refrigerator which concerns on Embodiment 4 of this invention. It is a schematic diagram which shows an example of the display screen in the operation panel of the refrigerator which concerns on Embodiment 4 of this invention. It is a schematic diagram for demonstrating an example of the estimation method in the time limit estimation part of the refrigerator which concerns on Embodiment 4 of this invention.

Embodiment 1 FIG.
<Configuration>
FIG. 1 is a front view showing the configuration of a refrigerator 1000 according to Embodiment 1 of the present invention. FIG. 2 is a diagram showing the configuration of the refrigerator compartment 100 of the refrigerator 1000 according to Embodiment 1 of the present invention, and is a side sectional view in the direction of the arrow along the line segment II-II in FIG.

  As shown in FIG. 1, the refrigerator 1000 includes a plurality of storage rooms. The plurality of storage rooms are, for example, a refrigerator room 100, an ice making room 200, a switching room 300, a freezer room 400, and a vegetable room 500. Here, from the top, the refrigerating room 100, the ice making room 200, the switching room 300, the freezing room 400, and the vegetable room 500 are arranged in this order. Further, the ice making chamber 200 and the switching chamber 300 are arranged side by side. Each storage chamber is cooled by a refrigeration cycle, which will be described in a fourth embodiment.

  An operation panel 1 serving as a display unit is provided on the front surface (front surface) of the door of the refrigerator compartment 100. On the display screen of the operation panel 1, the temperature setting of each storage room, the current temperature of each storage room, time information, food image, entry / exit identification information (described later), time change of food image, and information on food deterioration It is possible to display internal information such as.

  The operation panel 1 is configured to accept user input, for example, using a touch panel. The operation panel 1 is not limited to the door of the refrigerating room 100 but may be provided on the door of another storage room or on the side surface of the refrigerator 1000.

  Moreover, the ice making room 200, the switching room 300, the freezer room 400, and the vegetable room 500 are comprised so that it can pull out to the near side of the refrigerator 1000 with the door provided in each front.

  As shown in FIG. 2, the refrigerator compartment 100 has its front side (left side in the figure) closed by a refrigerator compartment door 101. The refrigerator compartment 100 has a plurality of refrigerator compartment shelf plates 102 inside, and is partitioned into a plurality of spaces (shelves). The space below the bottom refrigerator compartment shelf 102 is a chilled chamber 110. A chilled case 111 that can be drawn forward along a guide member (not shown) such as a rail is provided inside the chilled chamber 110. A cooling air passage 1010 is provided on the back side (right side in the figure) of the refrigerator compartment 100, which will be described in the fourth embodiment.

  On the inner surface of the refrigerator compartment door 101, an in-compartment photographing device 2 as a photographing device is provided. The in-compartment photographing apparatus 2 is configured to be able to photograph from the refrigerator compartment door 101 side, which is the same as the user's viewpoint, with the visual field range from the ceiling surface of the refrigerator compartment 100 to the chilled room 110.

  In addition, the in-compartment photographing apparatus 2 is not limited to the inner surface of the refrigerator compartment door 101, and may be provided anywhere as long as it can photograph a stored item such as food placed on the refrigerator compartment shelf plate 102 over a wide range. In consideration of the view being blocked by the refrigerator compartment shelf 102, a plurality of in-compartment photographing devices 2 may be provided so that more items can be photographed.

  The refrigerator 1000 includes a control board 1001 as a control device. The control board 1001 has a processing circuit (control circuit), and is attached to the back surface of the refrigerator 1000, for example. A food image extraction unit 3, a food image management unit 4, a time change detection unit 5, and a food deterioration state estimation unit 6 are mounted on the control board 1001.

  The food image extraction unit 3 is connected to the in-compartment photographing apparatus 2 and the food image management unit 4. The food image extraction unit 3 receives, for example, an entire image (storage room image) in the refrigerator compartment 100 photographed by the in-compartment photographing apparatus 2 when the refrigerator compartment door 101 is opened and closed, and extracts a food image from the received whole image. . Furthermore, the food image extraction unit 3 adds time information (shooting time information) as supplementary information to the food image extracted from the entire image, and transmits the food image and the supplementary information to the food image management unit 4. The in-compartment photographing apparatus 2 and the food image extraction unit 3 constitute a food image detection unit that photographs the inside of the refrigerator compartment 100 and detects a food image. The time information includes date (month / day or year / month / day) information.

  The food image management unit 4 is connected to the operation panel 1, the food image extraction unit 3, and the time change detection unit 5. The food image management unit 4 receives the food image and the time information from the food image extraction unit 3, and rearranges and stores the food images in time series based on the time information. At this time, the food image management unit 4 identifies the entry or exit of the food based on the food image, and adds the identification result (entry / exit identification information) to the incidental information of the food image and stores it. Further, the food image management unit 4 transmits the food image and incidental information (time information and entry / exit identification information) to the time change detection unit 5 and the operation panel 1.

  The time change detection unit 5 is connected to the operation panel 1, the food image management unit 4, and the food deterioration state estimation unit 6. The temporal change detection unit 5 detects and stores temporal changes in the color and size of the food image transmitted from the food image management unit 4. The temporal change detection unit 5 also transmits temporal changes in the color and size of the detected food image to the food deterioration state estimation unit 6 and the operation panel 1.

  The food deterioration state estimation unit 6 is connected to the operation panel 1 and the time change detection unit 5. The food deterioration state estimation unit 6 receives the time change of the food image from the time change detection unit 5, estimates the food deterioration state based on the time change, and transmits the estimation result to the operation panel 1.

<Operation>
Next, an example of operation | movement of the refrigerator 1000 of Embodiment 1 is demonstrated with reference to FIG.1, FIG2 and FIG.3. FIG. 3 is a flowchart for explaining an operation performed each time the refrigerator compartment door 101 of the refrigerator 1000 is opened and closed.

  The operation shown in FIG. 3 is started by opening / closing the refrigerator compartment door 101. This is because when the refrigerator compartment door 101 is opened and closed, food in the refrigerator compartment 100 may be taken in and out. When opening / closing of the refrigerator compartment door 101 is detected (step S11), first, the in-compartment photographing apparatus 2 photographs the entire inside of the refrigerator compartment 100 (step S12).

  In addition, although the detection of opening / closing of the refrigerator compartment door 101 is performed using the door opening / closing detection apparatus 7 (FIG. 9), for example, this is demonstrated in Embodiment 2. FIG. In addition, after the refrigerator door 101 is closed, the in-compartment photographing apparatus 2 performs photographing while the interior is illuminated by the lighting device 8 (FIG. 9). This is also described in the second embodiment. To do.

  The in-compartment photographing apparatus 2 transmits the whole image (referred to as a storage room image) of the photographed refrigerator compartment 100 to the food image extraction unit 3 (step S13).

  Therefore, the food image extraction unit 3 receives the storage room image from the internal photographing apparatus 2 every time the refrigerator compartment door 101 is opened and closed (that is, every time the storage state in the refrigerator compartment 100 may change). Will receive. In the following description, the opening / closing time of the current (latest) refrigerator compartment door 101 will be referred to as “current door opening / closing time”, and the previous time (one time before) opening / closing of the cold storage door 101 will be referred to as “previous door opening / closing time”. Called.

  The food image extraction unit 3 stores the storage room image received from the in-compartment photographing apparatus 2 when the door is opened and closed last time, and newly stores the storage room image received from the in-compartment photographing apparatus 2 when the door is opened and closed this time. . And the image of the change part of a storeroom image is extracted by calculating the difference of the storeroom image at the time of door opening and closing this time and this time (step S14). The image of the changed portion of the storage room image is an image of food that has been received or delivered.

  For example, in the storage room image when the door was opened / closed last time, beer was placed on the refrigerator compartment shelf, and in the storage room image when the door was opened / closed this time, only the shelf where no beer was placed at the same location. In this case, a food image of beer is extracted based on the difference between the two storage room images.

  The food image extraction unit 3 adds time information, which is the current time at the time of extraction, to the food image extracted in this way as supplementary information, and the food image management unit 4 adds these food images and supplementary information. (Step S15).

  The food image management unit 4 rearranges and stores the food images transmitted from the food image extraction unit 3 in time series based on the time information that is supplementary information (step S16).

  At this time, the food image management unit 4 determines whether the food is in / out. Specifically, the food image management unit 4 analyzes the portion that has changed due to the difference when the food image extraction unit 3 extracts the food image, for example, the amount of edge change obtained by edge processing, and Incoming / outgoing food is determined by comparison with a storage room image stored in advance in an empty state (a state in which no food is stored in the refrigerator compartment 100). Then, the warehousing / departure determination result (entry / exit identification information) is added to the supplementary information of the food image and stored.

  Further, the food image management unit 4 transmits the food image and accompanying information (time information and entry / exit identification information) to the time change detection unit 5 and the operation panel 1.

  In subsequent step S17, the time change detection unit 5 detects and stores changes in the color and size of the food image transmitted from the food image management unit 4, which will be described later.

  4 (a) to 4 (d) are based on shooting data (store room image) that is shot by the inside shooting apparatus 2 and transmitted to the food image extraction unit 3 when the refrigerator door 101 is opened and closed, and the store room image. The extracted food image and its accompanying information (time information and entry / exit identification information) are shown.

  FIG. 4A shows a storage room image captured by the in-compartment photographing apparatus 2 and transmitted to the food image extracting unit 3 at time A. FIG. At the time A, the refrigerator compartment door 101 is opened and closed, but no food is stored or delivered. Since there is no change in the storage room image between time A and earlier, the food image extraction unit 3 does not generate a difference image (food image). Accordingly, the food image is not transmitted from the food image extraction unit 3 to the food image management unit 4.

  FIG. 4B shows a storage room image captured by the in-compartment photographing apparatus 2 and transmitted to the food image extraction unit 3 at time B, a food image extracted from the storage room image, and its supplementary information. Show. At time B, beer is being delivered from the uppermost refrigerator compartment shelf of the refrigerator compartment 100. The food image extraction unit 3 extracts a food image (beer) based on the difference between the storage room images at times A and B, and transmits it to the food image management unit 4 together with time information (time B). The food image management unit 4 stores the entry / exit identification information “departure” and time information (time B) together with the food image based on the determination result of the entry / exit.

  For convenience of illustration, the food image (difference image) shown in FIG. 4B also shows the refrigerator compartment shelf, but the actual food image has no refrigerator shelf and only beer.

  FIG. 4 (c) shows a storage room image photographed by the in-compartment photographing device 2 and transmitted to the food image extraction unit 3 at time C, a food image extracted from the storage room image, and its supplementary information. Show. At time C, the salad has been delivered from the bottom refrigerator compartment shelf of the refrigerator compartment 100. The food image extraction unit 3 extracts the food image (salad) from the difference between the storage room images at the times B and C, and transmits it to the food image management unit 4 together with the time information (time C). The food image management unit 4 stores the entry / exit identification information “departure” and time information (time C) along with the food image based on the determination result of the entry / exit.

  FIG. 4D shows a storage room image photographed by the in-compartment photographing device 2 and transmitted to the food image extraction unit 3 at time D, a food image extracted from the storage room image, and its supplementary information. Show. At time D, the milk is stored on the uppermost refrigerator compartment shelf of the refrigerator compartment 100, and the cake is received on the lowermost refrigerator compartment shelf. The food image extraction unit 3 extracts a food image (milk and cake) based on the difference between the storage room images at times C and D, and transmits it to the food image management unit 4 together with time information (time D). The food image management unit 4 stores the entry / exit identification information and “time entry” (time D) together with the food image based on the entry / exit determination result.

  Thus, since the food image management unit 4 stores the food image together with the time information and the warehousing / departure identification information, the food image management unit 4 can grasp the warehousing time of the food and estimates the deterioration state based on the warehousing time. be able to. That is, the food image management unit 4 grasps food (stock food) stored in the refrigerator compartment 100 as an image and holds information necessary for estimating the deterioration state of the food.

  Here, the image data stored in the food image management unit 4 is only the food image extracted by the food image extraction unit 3 (changed portion of the storage room image). Further, the storage room image (the entire image of the refrigerator compartment 100) stored in the food image extraction unit 3 is also only the latest storage room image serving as a difference reference. That is, after extracting the food image, the food image extraction unit 3 can store only the latest storage room image and can erase the rest.

  Thus, since the amount of image data stored in the food image extraction unit 3 and the food image management unit 4 is reduced, the memory capacity and the data communication load can be reduced. In addition, photographing with the in-compartment photographing device 2, extraction of food images with the food image extraction unit 3, and storage of image data in the food image management unit 4 are performed when the refrigerator compartment door 101 is opened and closed (the food has been loaded and unloaded). And only data necessary for managing food stored in the refrigerator compartment 100 is stored. Therefore, it is possible to grasp the storage state and the deterioration state of the food in the refrigerator compartment 100 with the minimum necessary operations (photographing, image processing and storage).

  Here, the storage state of food means which food is stored in the refrigerator compartment 100 at which point. Moreover, the deterioration state of food means to what extent the deterioration of food has progressed (for example, whether or not it is necessary to consume at an early stage).

  The food image management unit 4 outputs the food image data stored together with the accompanying information (time information and entry / exit identification information) to the display screen of the operation panel 1 in time series or divided into entry or exit. Can do. The user can grasp the food stored in the refrigerator compartment 100 by visually recognizing the display screen of the operation panel 1.

  FIGS. 5A and 5B are diagrams illustrating an example of a display screen displayed on the operation panel 1 by the food image management unit 4. In the display screen (timeline display screen) 21 shown in FIG. 5A, the food image extracted from the storage room image each time the refrigerator compartment door 101 is opened and closed, the time information attached to the food image, and entry / exit The identification information is displayed in time series. In this example, time information (photographing time information) is arranged in order from the newest, the time information is displayed in the left column, the food image is displayed in the center column, and the entry / exit information is displayed in the right column. it's shown.

  On the timeline display screen 21 shown in FIG. 5A, for example, the target food can be searched retroactively and the expiration date or expiration date can be estimated. Therefore, the food can be consumed before the expiration date or the expiration date, and the waste of food can be prevented.

  The display screen (entry / exit list display screen) 22 shown in FIG. 5 (b) is divided into left and right columns for entry and exit, and food images and time information are displayed in chronological order (list display). doing. In other words, in the left column, the food images whose entry / exit identification information is “Receive” and the time information thereof (that is, the entry time) are displayed in order from the latest entry time. In the right column, the food images whose entry / exit identification information is “exit” and the time information thereof (that is, the exit time) are displayed side by side in order from the latest exit time.

  On the entry / exit list display screen 22 shown in FIG. 5B, the current remaining amount of the food can be estimated from the history of receipt and delivery of the target food.

  FIG. 6 is a diagram illustrating another example of a display screen displayed on the operation panel 1 by the food image management unit 4. The food image management unit 4 uses a food image stored by itself (that is, a food image extracted every time the refrigerator door 101 is opened / closed) and time information and warehousing / departure identification information as incidental information. The food image stored in the refrigerator compartment 100 (the entry / exit identification information is not “exit”) is specified. Then, as shown in FIG. 6, a list of stock food images is displayed on the operation panel 1 as a stock food list display screen 23.

  In the inventory food list display screen 23 shown in FIG. 6, since a list of inventory food images at a certain point in time is displayed, it is clearer than the list display shown in FIG. It is possible to grasp whether food is stored.

  In addition, on the display screen of the operation panel 1, there are three tabs “Inventory food list”, “Incoming / outgoing timeline”, and “Incoming / outgoing list”, and when the “Inventory food list” tab is clicked, The stock food list display screen 23 of FIG. 6 is displayed. When the “entry / exit timeline” tab is clicked, the timeline display screen 21 shown in FIG. 5A is displayed. When the “entry / exit list” tab is clicked, the tab shown in FIG. 5B is displayed. The entry / exit list display screen 22 is displayed.

  In addition to the food image, the display screen of the operation panel 1 also includes each set temperature 25 of each storage room (the refrigerator room 100, the ice making room 200, the switching room 300, the freezer room 400, and the vegetable room 500) of the refrigerator 1000. Is displayed. In addition to the set temperature, the measured temperature of each storage room may be displayed.

  Further, the operation panel 1 may always display one of the display screens of FIGS. 5A, 5B, or 6 according to a user operation (for example, touching a touch panel or the like). May be displayed.

  Next, processing of the time change detection unit 5 and the food deterioration state estimation unit 6 will be described.

  In step S <b> 17 shown in FIG. 3, the time change detection unit 5 receives the color of the food image received from the food image management unit 4 (that is, the food image extracted from the storage room image every time the refrigerator door 101 is opened and closed) and Detect and save magnitude changes over time.

  Although there are differences depending on the type of food and the storage environment, the food stored in the refrigerator compartment 100 gradually deteriorates, and in particular, changes in color or size (or shape) appear. For example, fresh foods such as meat and fish change color to brown due to oxidation (methation). As for vegetables, broccoli gradually fades from bright green and increases in yellow, and when it further deteriorates, it changes to grayish brown due to spoilage or mold. Leafy vegetables such as spinach are deflated and shrunken due to transpiration and change in size.

  Therefore, every time a food image is received from the food image management unit 4, the time change detection unit 5 detects and stores a time change in the color and size of the received food image. Based on the time change of the color and size stored in the time change detection unit 5, the food deterioration state estimation unit 6 described later estimates the deterioration state.

  As an index for detecting a change in color, for example, when the L * a * b * color system, which is a general color index, is used, the oxidation of fresh food described above reduces the a * value (red fading), Broccoli fading is quantified as an increase in L * or b * (brightness, yellow increase), and decay is a decrease in L * (decrease in brightness). Therefore, every time the food image is received from the food image management unit 4, the time change detection unit 5 detects and stores the value of a *, the value of b *, and the value of L *.

  Further, not only the L * a * b * color system but also other color systems such as the RGB color system or the Munsell color system, color change (red fading, yellow increase, etc.) and Changes in brightness can be quantified.

  The time change detection unit 5 may store a change in color distribution. In this case, the time change detection unit 5 stores a large number of color samples in advance as a database, and calculates the similarity by comparing the color detected from the food image received from the food image management unit 4 with the color sample. Then, the entire food image is classified for each color corresponding to the most similar color sample, and the area occupancy for each color is calculated.

  FIGS. 7A and 7B are diagrams illustrating an example of the time change of the food image in the time change detection unit 5. FIG. 7A shows a food image when food (meat) is received and a color distribution detected based on the food image. FIG. 7B shows a food image after a sufficient time (T time) for the food to oxidize and a color distribution detected based on the food image. 7A and 7B, the area occupancy rate (vertical axis) for each color (horizontal axis) corresponding to each color sample in the food image is shown by a bar graph. Whenever the food image is received from the food image management unit 4, the time change detection unit 5 calculates and stores the color distribution data of the newly received food image.

  Moreover, the time change detection part 5 preserve | saves a projection area, a perimeter, the length of the longest axis etc. as a parameter | index which detects the change of a magnitude | size, for example. Moreover, a pattern may be extracted from the outline shape of food and the pattern may be stored. Here, the time change detection unit 5 detects and stores changes in the color and size of the food image, but may detect and store only the color change or only the size change. .

  In this manner, the food deterioration state estimation unit 6 estimates the food deterioration state based on the color and size change over time detected and stored by the time change detection unit 5. The estimation of the food deterioration state by the food deterioration state estimation unit 6 is preferably performed not at every opening / closing of the refrigerator compartment door 101 but at every predetermined time.

  The mode of deterioration of food varies depending on the type of food (meat, fish, vegetables, etc.), but here, the state of deterioration of the food is determined using a uniform criterion regardless of the type of food. Therefore, the food deterioration state estimation unit 6 performs determination based on a plurality of indices for one food image, and determines that “it is necessary to consume early” when at least one index exceeds a reference value. .

  For example, when the L * a * b * color system is used as the color index, the food deterioration state estimation unit 6 stores the a * value, b * value, and L * value stored in the time change detection unit 5. If at least one of the values exceeds the reference value, the food corresponding to the food image is determined to be “necessary to consume early”.

  When the color distribution shown in FIGS. 7A and 7B is used, it is based on the change in the color distribution (area occupancy for each color corresponding to each color sample) stored in the time change detection unit 5. When the area occupancy rate of a specific color exceeds the reference value, for example, it is determined that “it is necessary to consume early”.

  That is, in the example shown in FIGS. 7A and 7B, compared to FIG. 7A when the food is received, in FIG. 7B after T time, the food (meat) is oxidized by oxidation. The color is dark (actually brown), and the area occupancy of the dark color is large. Brown is the color that metmyoglobin exhibits upon oxidation or methation.

  Therefore, based on the knowledge such as “the color corresponding to the 10th color sample increases as the metrification progresses” and “the color corresponding to the 20th color sample increases as the decay progresses”, the food deterioration state A reference value is registered in advance in the estimation unit 6 for each color that reflects deterioration.

  Then, the food deterioration state estimation unit 6 has at least one of the area occupancy rates of a plurality of pre-registered colors (for example, colors corresponding to a plurality of color samples such as No. 10 and No. 20) exceeded a reference value. In this case, the food corresponding to the food image is determined as “the food may be deteriorated”.

  In addition, since the food is deflated or shrunken due to deterioration, the deterioration state can be determined based on a change in the size of the food contained in the food image. In this case, the food deterioration state estimation unit 6 determines the size of the food based on the change in the size of the food detected and stored by the time change detection unit 5 (for example, the projected area, the peripheral length, or the length of the longest axis). When the rate of decrease from the time of storage exceeds the reference value, it can be determined that the food needs to be consumed early.

  Alternatively, when the food deterioration state estimation unit 6 performs pattern matching with the pattern at the time of warehousing based on the pattern of the contour shape of the food detected and stored by the time change detection unit 5 and the error exceeds the reference time It may be determined that “it is necessary to consume early”.

  Further, in the determination based on the color distribution described with reference to FIGS. 7A and 7B, instead of obtaining the area occupancy for each color corresponding to each color sample, if an absolute area value is obtained, It is possible to make a judgment that reflects the change in size as well as the change in color.

  The food deterioration state estimation unit 6 determines the deterioration state of each food image stored in the time change detection unit 5. On the operation panel 1, the estimation result of the deterioration state by the food deterioration state estimation unit 6 is displayed together with the food image stored in the food image management unit 4.

  FIG. 8A is a diagram illustrating a display example of the deterioration state of the food on the operation panel 1. In the display example shown in FIG. 8A, the food deterioration state estimation unit 6 determines that “it is necessary to consume early” among the food images displayed on the inventory food list display screen 23 described in FIG. An alert (exclamation mark in this case) is superimposed on the food image. Thereby, the user who looked at the operation panel 1 can grasp | ascertain that the food of the food image in which the alert was displayed is a food which should be consumed at an early stage.

  In other words, it becomes obvious to the user which of the plurality of foods stored in the refrigerator compartment 100 should be consumed at an early stage. As a result, it is possible to suppress wastefully discarding food due to the expiration of the expiration date or the expiration date.

  In the display example shown in FIG. 8A, an alert is superimposed on the food image. However, the present invention is not limited to this example, and the food to be consumed early based on the estimation result of the food deterioration state estimation unit 6. As long as the user can be notified, any display may be used.

  The operation panel 1 may also display a change in color or size of the food image detected by the time change detection unit 5 as schematically shown in FIG. In this case, for example, when one food image is selected on the above-described stock food list display screen 23 (FIG. 6), a food change display screen 24 that displays a change in color or size of the food image for each time is displayed. Displayed on the operation panel 1. Thereby, the user can grasp | ascertain the degradation condition of foodstuff from an image.

  In addition, the case where the refrigerator compartment door 101 is not opened and closed over a long period of time is also assumed. Therefore, for example, even when there is only a small sign of food deterioration (for example, a change to brown) at the time of opening and closing the latest refrigerator door 101, the deterioration of the food may progress over time. Therefore, the food deterioration state estimation unit 6 estimates the food deterioration state based on changes in the color and size of each food image stored in the time change detection unit 5 at regular time intervals. Based on this, it is desirable to predict a time (notification time to the user) that should prompt the user to consume food, and display an alert on the operation panel 1 when the predicted time is reached.

<Effect>
As described above, according to the first embodiment of the present invention, a food image is extracted and managed from the image in the refrigerator compartment 100 photographed by the internal photographing device 2, and the food screen is displayed on the display screen of the operation panel 1. Both the image and information regarding the deterioration of the food (for example, information indicating the deterioration state) are displayed together. Therefore, the user can grasp the food stored in the refrigerator compartment 100, and can grasp the food to be consumed particularly quickly among them.

  As a result, it is possible to encourage the user to consume the food by, for example, the expiration date, and to suppress wasteful disposal of the food. Further, waste of space and energy due to leaving food in the refrigerator compartment 100 for a long period of time can be reduced. Further, even when the refrigerator compartment door 101 is closed, the operation panel 1 can check the stock status or the deterioration state of the food. Therefore, the temperature rise in the refrigerator compartment 100 due to the opening and closing of the refrigerator compartment door 101 is suppressed, and the energy consumption is reduced. Can be reduced.

  In addition, by displaying the entry / exit identification information and time information together with the food image on the display screen of the operation panel 1, the user can easily grasp how long a certain food has been stored in the refrigerator compartment 100. be able to.

  In addition, since the food image is stored together with the loading / unloading identification information and time information, and the deterioration state of the food is estimated based on the information, the food image is changed in consideration of the change of the food image and the time elapsed since the loading. Can be estimated with high accuracy.

  Further, each time the refrigerator door 101 is opened and closed, the in-compartment photographing device 2 captures a storage room image, and the food image extraction unit 3 extracts a food image from the storage room image. Information is updated each time there is a possibility of change. Therefore, the user can know the stock food and its deterioration state more accurately.

  Moreover, since the food deterioration state estimation part 6 estimates the deterioration state of food based on the change of the color or the size (or both) of the food image, the detection accuracy of deterioration such as oxidation (meth) and rot is improved. Can be increased.

<Other configuration examples>
In the above description, the food image management unit 4 identifies food entering or leaving, but in addition to food entering and leaving, food movement in the refrigerator compartment 100 may be identified.

  The movement of food is identified by the following method, for example. That is, the food image extraction unit 3 obtains a difference image between the storage room images at the time of opening and closing the previous door and the current door as described above. Furthermore, it is determined by matching processing whether there is a difference image of the storage room image at the time of opening and closing the previous door and the previous time and the difference image of the storage room image at the time of opening and closing the door of the previous time and this time. . As a result of the matching process, if any of the difference images of the storage room images at the previous and previous door opening / closing coincides with the difference images of the storage room images at the previous and current door opening / closing, the food is refrigerated. It is presumed that it has moved in the room 100.

  When the food deterioration state estimation unit 6 calculates the elapsed time from the receipt of the food (that is, the inventory period), the food deterioration state is estimated by including the inventory period before the food moves in the refrigerator compartment 100. And the precision of the alerting | reporting to a user can be improved.

  Further, in addition to the movement of food in the refrigerator compartment 100, there is a case where the same food is delivered once and re-entered (re-entry). This will be described in Embodiment 2.

  In the above description, the in-compartment photographing apparatus 2 is provided in the refrigerating room 100, but may be provided in a storage room other than the refrigerating room 100. For example, the in-compartment photographing apparatus 2 may be provided on the ceiling surface of the freezer compartment 400 or the vegetable compartment 500. Also in this case, a food image is extracted by the food image extraction unit 3 from the entire image of each storage room photographed by the in-chamber photographing device 2, and is stored together with the supplementary information in the food image management unit 4, and the time change detection unit 5 and A similar effect can be obtained by estimating the deterioration state by the food deterioration state estimation unit 6 and displaying the food image and the deterioration state on the operation panel 1.

Embodiment 2. FIG.
<Configuration>
FIG. 9 is a side sectional view showing the configuration of the refrigerator compartment 100 of the refrigerator 1000 according to Embodiment 2 of the present invention. 9 corresponds to a side sectional view in the direction of the arrow along the line segment II-II shown in FIG. 1, like FIG. Hereinafter, differences from the first embodiment will be mainly described. The same components as those in the first embodiment are denoted by the same reference numerals.

  As shown in FIG. 9, an illumination device 8 that illuminates the interior of the refrigerator is provided on the ceiling of the refrigerator compartment 100. In addition, a door open / close detection device 7 that detects the opening / closing of the refrigerator compartment door 101 is provided on the main body side of the refrigerator compartment 100 that contacts the refrigerator compartment door 101 with the refrigerator compartment door 101 closed.

  The detection result of the door opening / closing detection device 7 is used for lighting control of the lighting device 8. That is, the lighting device 8 is turned on when the door opening / closing detection device 7 detects the open state of the refrigerator compartment door 101, and the lighting device 8 is turned off when the door opening / closing detection device 7 detects the closed state of the refrigerator compartment door 101. The door opening / closing detection device 7 uses, for example, a magnet system in which a magnet (magnet) is embedded in the refrigerator compartment door 101 and a pair of reed switches are provided on the main body side of the refrigerator compartment 100. When the magnet approaches the vicinity of the reed switch, the reed switch is turned on, and when the magnet is separated from the reed switch, the reed switch is turned off.

  In addition to the food image extraction unit 3, the food image management unit 4, the time change detection unit 5, and the food deterioration state estimation unit 6 described in the first embodiment, the control board 1001 provided on the back surface of the refrigerator 1000 is photographed. A signal transmission device 9 and an incoming food storage unit 10 are mounted.

  The photographing signal transmission device 9 is connected to the internal photographing device 2, the door opening / closing detection device 7, and the lighting device 8. The photographing signal transmission device 9 controls the photographing timing of the internal photographing device 2 and the lighting and extinguishing timing of the lighting device 8 according to the detection result of the door opening / closing detection device 7.

  The incoming food storage unit 10 is connected to the food image management unit 4 and the food deterioration state estimation unit 6. The warehousing food storage unit 10 receives from the food image management unit 4 a food image that the food image management unit 4 has determined as “first warehousing” (described later), and stores it together with accompanying information (time information). The food image and the accompanying information stored in the warehousing food storage unit 10 are reflected in the estimation of the deterioration state in the food deterioration state estimation unit 6.

  The door open / close detection device 7 is also connected to the food deterioration state estimation unit 6. That is, the open / close state of the refrigerator door 101 detected by the door open / close detection device 7 is also reflected in the estimation of the deterioration state in the food deterioration state estimation unit 6.

<Operation>
Next, an example of the operation of the refrigerator 1000 according to the second embodiment will be described focusing on differences from the first embodiment.

  In FIG. 9, the door opening / closing detection device 7 transmits an ON signal to the photographing signal transmission device 9 when detecting that the refrigerator compartment door 101 is opened. When the photographing signal transmission device 9 receives the ON signal of the door opening / closing detection device 7, it transmits a lighting signal to the lighting device 8. When the lighting device 8 receives the lighting signal from the photographing signal transmission device 9, the lighting device 8 is turned on. In this state, the user puts in / out food in the refrigerator compartment 100 and then closes the refrigerator compartment door 101.

  When the door opening / closing detection device 7 detects that the refrigerator compartment door 101 is closed, the door opening / closing detection device 7 transmits an OFF signal to the photographing signal transmission device 9. When the photographing signal transmission device 9 receives the OFF signal of the door opening / closing detection device 7, first, the photographing signal transmission device 9 transmits a photographing instruction to the internal photographing device 2, and transmits a turn-off signal to the lighting device 8 after the photographing of the internal photographing device 2 is completed. To do. When the illumination device 8 receives the turn-off signal from the photographing signal transmission device 9, it turns off.

  The in-compartment photographing apparatus 2 photographs the inside of the refrigerator compartment 100 while the lighting device 8 is turned on, and transmits the entire image (stored compartment image) in the photographed refrigerator compartment 100 to the food image extraction unit 3. Accordingly, as in the first embodiment, the food image extraction unit 3 receives the storage room image from the in-compartment photographing apparatus 2 every time the refrigerator compartment door 101 opens and closes. The food image extraction unit 3 stores the received storage room image.

  The food image extraction unit 3 stores the storage room image received from the in-compartment photographing apparatus 2 when the door is opened and closed last time, and newly stores the storage room image received from the in-compartment photographing apparatus 2 when the door is opened and closed this time. . And the change part of an image is extracted by calculating the difference of the store room image at the time of the last door opening and closing this time.

  The food image extraction unit 3 adds time information (photographing time information) that is the current time as supplementary information to the food image extracted in this way, and the food image management and the supplementary information are added to the food image management. Send to part 4.

  The food image management unit 4 rearranges and stores the food images transmitted from the food image extraction unit 3 in time series based on the time information that is supplementary information. At this time, the food image management unit 4 determines the receipt / exit of the food by the method described in the first embodiment, and adds the food image entry / exit identification information to the incidental information and stores it.

  In the second embodiment, when the food image management unit 4 determines that a certain food image is “warehousing”, whether the food image is newly received (initial warehousing), or once evacuated and re-stocked (re-entry) Warehousing). More specifically, whether or not there is a food image (difference image) extracted at the time of opening and closing the door within a preset period in the past that matches the food image determined to be “entry”, Judge by matching process. If there is a match, it is determined as “re-entry”, and if there is no match, it is determined as “first receipt”.

  Further, the food image management unit 4 transfers the food image determined to be “first warehousing” to the warehousing food storage unit 10 together with accompanying information (time information). The warehousing food storage unit 10 stores the food image and the incidental information received from the food image management unit 4.

  Similar to the first embodiment, the time change detection unit 5 uses the color of the food image transmitted from the food image management unit 4 (that is, the food image extracted from the storage room image each time the refrigerator door 101 is opened and closed) and Detect and save magnitude changes over time. The food deterioration state estimation unit 6 estimates the food deterioration state based on the color and size temporal changes stored in the time change detection unit 5. Further, the food deterioration state estimation unit 6 displays the estimation result of the food deterioration state on the operation panel 1 together with the food image (see FIG. 8).

  In the second embodiment, when the food deterioration state estimation unit 6 estimates the deterioration state of the food, the food image determined as “re-entry” is stored in the “first storage” stored in the storage food storage unit 10. A change in color and size is detected by comparing the food image with the latest food image, and the deterioration state is estimated based on the elapsed time from the initial storage time. The initial warehousing time is the time when the food image determined to be “first warehousing” is extracted by the food image extracting unit 3.

  For example, in the case where the food first received 5 days ago is issued 4 days ago and re-stocked 3 days ago, the latest food image is compared with the food image 5 days ago, and the change in color and size is confirmed. To detect. Moreover, the elapsed time from the receipt of food to the present is counted from 5 days before. Thereby, the estimation precision of the deterioration state of foodstuffs can be improved.

  The food deterioration state estimation unit 6 further reflects the open / close state (including the open / close history) of the refrigerator door 101 that can be grasped based on the output signal of the door open / close detection device 7 to reflect the food deterioration state. Is estimated. The more frequently the refrigerator compartment door 101 is opened and closed (that is, the longer the total time that the refrigerator compartment door 101 is open), the more frequent the food is taken in and out, the greater the temperature fluctuation in the cabinet, and the higher the average temperature. It is expected that the food will deteriorate sooner.

  Therefore, when the opening / closing frequency of the refrigerator compartment door 101 is high, it is possible to improve the estimation accuracy of the deterioration state of the food by correcting the judgment standard or the judgment result based on the time change of the color or size of the food image. it can.

  For example, even if the food does not deteriorate for two weeks after warehousing, if the refrigerator door 101 is frequently opened and closed (that is, if the total time during which the refrigerator door 101 is open is long), the deterioration may occur. It is possible to estimate that the start should be 2-3 days earlier. Therefore, for example, when the total time during which the refrigerator compartment door 101 is open exceeds a preset time (minutes), the food deterioration state estimation unit 6 determines the area of the color sample (see FIG. 7) for determining deterioration. Decrease the occupancy threshold to a preset level. That is, it is determined that the food needs to be consumed at an early stage when the area occupation ratio is small.

<Effect>
As described above, according to the second embodiment of the present invention, the food image determined to be “first warehousing” is stored in the warehousing food storage unit 10, so that the food is temporarily delivered from the refrigerator compartment 100 and re-stocked. In this case, the deterioration state can be estimated based on the time of initial warehousing rather than the time of re-warehousing. Therefore, the estimation accuracy of the deterioration state of food can be improved.

  In addition, by reflecting the open / close state of the refrigerator compartment door 101 in the deterioration state estimation in the food deterioration state estimation unit 6, it is possible to estimate the deterioration state of the food in consideration of temperature fluctuations in the warehouse accompanying food in and out. become.

<Other configuration examples>
In the configuration example shown in FIG. 9, the in-compartment photographing apparatus 2 is provided on the inner surface of the refrigerator compartment door 101, but the in-compartment photographing apparatus 2 can grasp the storage state of foods placed in a wide range in the refrigerator compartment 100. It may be provided anywhere as long as it is possible. Moreover, although the illuminating device 8 was provided in the ceiling surface of the refrigerator compartment 100, the illuminating device 8 can illuminate the whole region in the refrigerator compartment 100 as much as possible, and the inside of the refrigerator compartment 100 required for imaging | photography of the storage imaging device 2 is possible. It may be provided anywhere as long as the illuminance is secured. For example, if the inside photographing device 2 and the lighting device 8 are installed close to each other, or if the lighting device 8 is built in the inside photographing device 2, there is an advantage that it is easy to photograph because the illuminance can be secured.

  Further, in the configuration example shown in FIG. 9, the in-compartment photographing device 2, the door opening / closing detection device 7, and the lighting device 8 are provided in the refrigerating room 100, but may be provided in a storage room other than the refrigerating room 100. For example, the in-compartment photographing apparatus 2 may be provided on the ceiling surface of the freezer compartment 400 or the vegetable compartment 500. Also in this case, a food image is extracted by the food image extraction unit 3 from the entire image of each storage room photographed by the in-compartment photographing apparatus 2, stored together with the supplementary information in the food image management unit 4, and determined to be the first warehousing. The food image is stored in the warehousing food storage unit 10, the deterioration state is estimated by the time change detection unit 5 and the food deterioration state estimation unit 6, and the food image and the deterioration state are displayed on the operation panel 1. An effect can be obtained.

Embodiment 3 FIG.
<Configuration>
FIG. 10 is a side sectional view showing the configuration of the refrigerator compartment 100 of the refrigerator 1000 according to Embodiment 3 of the present invention. 10 corresponds to a side sectional view in the direction of the arrow along the line II-II shown in FIG. 1, like FIG. Hereinafter, differences from the second embodiment will be mainly described. In addition, the same components as those in the first or second embodiment are denoted by the same reference numerals.

  In FIG. 10, in the third embodiment, in addition to the components described in the second embodiment, the internal temperature detection device 11 that detects the air temperature in the refrigerating chamber 100 is provided on the back wall of the refrigerating chamber 100. It has been.

  In addition, the food temperature degradation state estimation unit 6 is connected with a chamber temperature detection device 11 instead of the door opening / closing detection device 7 connected in the second embodiment. That is, when the food deterioration state estimation unit 6 estimates the food deterioration state, the temperature in the refrigerator compartment 100 detected by the internal temperature detection device 11 (internal temperature), its temperature history, and the internal temperature. It is comprised so that the cooling load of the refrigerator compartment 100 estimated from can be reflected.

<Operation>
Next, an example of the operation of the refrigerator 1000 according to the third embodiment will be described focusing on differences from the second embodiment.

  In FIG. 10, the door opening / closing detection device 7 transmits an ON signal to the photographing signal transmission device 9 when detecting that the refrigerator compartment door 101 is opened. When the photographing signal transmission device 9 receives the ON signal of the door opening / closing detection device 7, it transmits a lighting signal to the lighting device 8. When the lighting device 8 receives the lighting signal from the photographing signal transmission device 9, the lighting device 8 is turned on. In this state, the user puts in / out food in the refrigerator compartment 100 and then closes the refrigerator compartment door 101.

  When the door open / close detection device 7 detects that the refrigerator door 101 is closed, the door open / close detection device 7 transmits an OFF signal to the photographing signal transmission device 9. When the photographing signal transmission device 9 receives the OFF signal of the door opening / closing detection device 7, first, the photographing signal transmission device 9 transmits a photographing instruction to the internal photographing device 2, and transmits a turn-off signal to the lighting device 8 after the photographing of the internal photographing device 2 is completed. To do. When the illumination device 8 receives the turn-off signal from the photographing signal transmission device 9, it turns off.

  The in-compartment photographing apparatus 2 photographs the inside of the refrigerator compartment 100 while the lighting device 8 is turned on, and transmits the entire image (stored compartment image) in the photographed refrigerator compartment 100 to the food image extraction unit 3. The food image extraction unit 3 stores the received storage room image.

  The food image extraction unit 3 stores the storage room image received from the in-compartment photographing apparatus 2 when the door is opened and closed last time, and newly stores the storage room image received from the in-compartment photographing apparatus 2 when the door is opened and closed this time. . And the change part of an image is extracted by calculating the difference of the store room image at the time of the last door opening and closing this time. The food image extraction unit 3 adds time information (shooting time information) that is the current time as supplementary information to the food image extracted as described above, and adds the food image and the supplementary information to the food image management unit. 4 to send.

  The food image management unit 4 rearranges and stores the food images received from the food image extraction unit 3 in time series based on the time information that is supplementary information. At this time, the food image management unit 4 determines the receipt / exit of the food by the method described in the first embodiment, and adds the food image entry / exit identification information to the incidental information and stores it. Furthermore, the food image management unit 4 transfers the food image determined as “first storage” by the method described in the second embodiment to the stored food storage unit 10 and stores it.

  Next, the time change detection unit 5 stores the time change of the color and size of the food image transmitted from the food image management unit 4. The food deterioration state estimation unit 6 estimates the food deterioration state based on the color and size temporal changes stored in the time change detection unit 5. Further, the food deterioration state estimation unit 6 displays the estimation result of the food deterioration state on the operation panel 1 together with the food image (see FIG. 8).

  In the third embodiment, as described in the second embodiment, when the food deterioration state estimation unit 6 estimates the deterioration state of the food, the food image storage unit determines that the food image is “re-entry”. A change in color and size is detected by comparing the food image of “first warehousing” stored in 10 with the latest food image, and the deterioration state is estimated based on the elapsed time from the first warehousing time.

  Moreover, in this Embodiment 3, when the food deterioration state estimation part 6 estimates the food deterioration state, it is comprised so that the temperature history in the refrigerator compartment 100 detected by the chamber internal temperature detection apparatus 11 can be reflected. ing. For example, when the average temperature in the refrigerator compartment 100 is high, it is assumed that the food is more deteriorated than when the average temperature is low. Therefore, the estimation accuracy is improved by correcting the determination result based on the change in the color and size of the food image.

  The temperature history detected by the internal temperature detection device 11 changes depending on the storage load and cooling load in the refrigerator compartment 100 and the opening and closing of the refrigerator compartment door 101. For example, if the average temperature is higher than the cooling capacity, it is an overload, and if there is a sudden temperature rise, it can be estimated that the refrigerator compartment door 101 has been opened and closed. Therefore, when the food deterioration state estimation unit 6 estimates the food deterioration state, the storage history and cooling load in the refrigerator compartment 100 and the refrigerator compartment door 101 are reflected by reflecting the temperature history of the internal temperature detection device 11. It is possible to perform estimation in consideration of the effect of opening and closing of the.

  For example, during a period in which a certain food is stored in the refrigerator compartment 100, when the accumulated time in which the internal temperature is equal to or higher than a preset temperature exceeds a threshold, the food deterioration state estimation unit 6 performs deterioration. The area occupancy threshold of the color sample to be judged (see FIG. 7) is lowered to a preset level. That is, it is determined that the food needs to be consumed at an early stage when the area occupation ratio is small.

  As described above, the food deterioration state estimation unit 6 reflects the internal temperature history (for example, average temperature) in the estimation of the food deterioration state, and displays the estimation result on the operation panel 1. Thereby, the estimation precision of the deterioration state of foodstuffs can be improved. In addition, there is also an effect of prompting the user to suppress the cooling load by reducing the opening / closing of the refrigerator compartment door 101 as much as possible.

<Effect>
As described above, according to the third embodiment of the present invention, when the food deterioration state estimation unit 6 estimates the food deterioration state, the history of the temperatures detected by the in-case temperature detection device 11 is reflected. Accordingly, the deterioration state of the food can be estimated in consideration of the storage load and cooling load in the refrigerator compartment 100 and the influence of opening and closing of the refrigerator compartment door 101.

<Other configuration examples>
In the configuration example shown in FIG. 10, the internal temperature detection device 11 is provided on the back wall of the third-stage refrigerator compartment shelf 102, but the internal temperature detection device 11 is representative of the refrigerator compartment 100. As long as the air temperature can be detected, it may be provided on the back wall of another refrigerator compartment shelf 102 or on the refrigerator compartment door 101 side.

  For example, when estimating the deterioration state in the food deterioration state estimation unit 6, if the increase in the internal temperature due to opening / closing of the refrigeration room door 101 is emphasized, the internal temperature detection device 11 is provided on the refrigeration room door 101 side. The detection accuracy is improved. In particular, in order to quickly detect and suppress the deterioration of food due to an increase in the internal temperature, the internal temperature detector 11 is provided around the upper shelf of the refrigerator compartment door 101 where the detected temperature tends to increase. It is more effective.

  Further, in the configuration example shown in FIG. 10, the door opening / closing detection device 7 detects the opening / closing of the refrigerator compartment door 101, and based on the detection result, the illumination device 8 and the inside photographing are taken via the photographing signal transmission device 9. The apparatus 2 is controlled. However, since the open / close state of the refrigerator compartment door 101 can be estimated from the internal temperature history detected by the internal temperature detection device 11, the door open / close detection device 7 is omitted, and the detection result of the internal temperature detection device 11 is photographed. You may input into the signal transmission apparatus 9. FIG. If comprised in this way, the internal temperature detection apparatus 11 will not only reflect temperature history in estimation of the deterioration state in the food deterioration state estimation part 6, but also the illuminating device 8 and the internal imaging device based on a detection signal. 2 can be controlled, the number of parts can be reduced.

  In the configuration example shown in FIG. 10, the warehousing food storage unit 10 is provided. However, when the determination of “first warehousing” described in the second embodiment is not performed, the warehousing food storage unit 10 is omitted. Also good.

Embodiment 4 FIG.
<Configuration>
FIG. 11 is a side sectional view showing the configuration of the refrigerator 1000 according to Embodiment 4 of the present invention. 11 corresponds to a side sectional view in the direction of the arrow along the line segment II-II shown in FIG. 1, as in FIG. Hereinafter, differences from the first embodiment will be mainly described. The same components as those in the first, second, or third embodiment are denoted by the same reference numerals.

  As shown in FIG. 11, the refrigerator 1000 according to the fourth embodiment includes a refrigeration cycle that cools air supplied to each storage room (the refrigeration room 100, the ice making room 200, the switching room 300, the freezing room 400, and the vegetable room 500). And an air passage for supplying the air cooled by the refrigeration cycle to each storage chamber.

  The refrigeration cycle includes a compressor 1002, a condenser (not shown) that condenses the refrigerant discharged from the compressor 1002, a throttle device (not shown) that expands the refrigerant flowing out of the condenser, and a throttle device. And a cooler 1003 for cooling the air supplied to each storage chamber by the expanded refrigerant.

  The compressor 1002 is arrange | positioned at the lower part of the back side of the refrigerator 1000, for example. The cooler 1003 is provided on the upstream side of a cooling air passage 1010 described later. The cooling air passage 1010 is also provided with an air conveyance device 1004 for sending air cooled by the cooler 1003 to each storage room (that is, for circulating air in the refrigerator 1000).

  An air path for supplying air cooled by the refrigeration cycle to each storage room is composed of a cooling air path 1010, a return air path 1020, a refrigeration room return air path 103, and a vegetable room return air path 501. ing.

  The cooling air passage 1010 is a ventilation passage through which the air cooled by the cooler 1003 is conveyed to each storage chamber, and is formed, for example, on the back surface of the refrigerator 1000. The amount of cooling air flowing into each storage room is adjusted by a damper (only the refrigerating room damper 104 for the refrigerating room 100 shown in FIG. 11) provided in the cooling air passage 1010.

  The return air passage 1020 is a ventilation passage through which the air that has cooled each storage chamber is conveyed to the cooler 1003. The refrigerator compartment return air passage 103 is a ventilation passage through which the air that has cooled the refrigerator compartment 100 and the chilled chamber 110 is conveyed to the vegetable compartment 500. The air that has cooled the refrigerator compartment 100 and the chilled chamber 110 merges with the air that has cooled the vegetable compartment 500 in the vegetable compartment return air passage 501, and is conveyed to the cooler 1003.

  On the control board 1001 provided on the back surface of the refrigerator 1000, in addition to the components described in the first embodiment, a time limit estimation unit 12, a set temperature calculation unit 13, and a cooling control unit 14 are mounted. In addition to the configuration of the first embodiment, a time limit setting unit 15 is incorporated in the operation panel 1 provided on the door of the refrigerator compartment 100.

  The time limit setting unit 15 displays on the operation panel 1 a time limit setting screen 26 for setting a time limit (consumption time limit or best before date) for the user to consume food. FIG. 12A is a schematic diagram illustrating an example of the time limit setting screen 26. This time limit setting screen 26 is displayed on the operation panel 1 when, for example, one food image is selected on the above-described stock food list display screen 23 (FIG. 6). The time limit setting screen 26 is configured so that the user can input a time limit (expiration date or expiration date) for the selected food image.

  Further, when a new food is received in the refrigerator compartment 100, a display screen similar to the stock food list display screen 23 shown in FIG.

  The term estimation unit 12 is connected to the operation panel 1, the food deterioration state estimation unit 6, and the set temperature calculation unit 13. The deadline estimation unit 12 estimates a deadline (consumption deadline or expiration date) on which the food should be consumed based on the estimation result of the deterioration state in the food deterioration state estimation unit 6 and causes the operation panel 1 to display the estimated deadline. The result is transmitted to the set temperature calculation unit 13.

  In this case, for example, a deadline notification screen 27 as shown in FIG. 12B is displayed on the operation panel 1 for the food that is approaching the remaining period until the deadline estimated by the deadline estimating unit 12. In the time limit notification screen 27 shown in FIG. 12B, the food image and the remaining period (for example, two days) until the time limit corresponding to the food image are displayed.

  The set temperature calculation unit 13 is connected to the operation panel 1, the time limit estimation unit 12, the cooling control unit 14, and the time limit setting unit 15. The set temperature calculation unit 13 calculates a set temperature (recommended set temperature) suitable for food based on the time limit estimated by the time limit estimation unit 12 and the time limit set by the user in the time limit setting unit 15.

  Then, the set temperature calculation unit 13 displays a recommended temperature notification screen 28 as shown in FIG. 12C on the operation panel 1 in order to notify the user of the recommended set temperature. In addition, the set temperature calculation unit 13 transmits control details necessary to achieve the calculated set temperature to the cooling control unit 14.

  The cooling control unit 14 is connected to the compressor 1002, the air conveyance device 1004, the refrigerator compartment damper 104, and other storage room dampers (not shown) in addition to the set temperature calculation unit 13, and controls these devices. To do. The cooling control unit 14 receives an input of the set temperature from the set temperature calculation unit 13 and transmits a control signal to at least one of the compressor 1002, the air conveyance device 1004, the refrigerator compartment damper 104, and other storage compartment dampers. Then, cooling control of the storage room (change of cooling capacity) is performed. The refrigerating room damper 104 and other storage room dampers constitute a “blowing air volume control device”.

<Operation>
Next, an example of the operation of the refrigerator 1000 according to the fourth embodiment will be described focusing on differences from the first embodiment.

  In FIG. 11, when the refrigerator compartment door 101 is opened and closed, the in-compartment photographing apparatus 2 photographs the entire interior of the refrigerator compartment 100, and the entire image (store room image) of the photographed refrigerator compartment 100 is taken as a food image extraction unit. 3 to send. The food image extraction unit 3 extracts a food image based on the difference between the previous and current storage room images when the door is opened and closed, adds time information (shooting time information) as supplementary information, and transmits the information to the food image management unit 4. .

  The food image management unit 4 rearranges and stores the food images transmitted from the food image extraction unit 3 in time series based on the time information. At this time, the food image management unit 4 determines whether the food is in stock / out, and adds and stores the in / out identification information to the supplementary information of the food image.

  Next, the time change detection unit 5 detects and stores the time change of the color and size of the food image transmitted from the food image management unit 4. The food deterioration state estimation unit 6 estimates the food deterioration state based on the color and size temporal changes stored in the time change detection unit 5. Further, the food deterioration state estimation unit 6 displays the estimation result of the food deterioration state on the operation panel 1 together with the food image (see FIG. 8).

  In the fourth embodiment, based on the progress (deterioration tendency) of the deterioration of food estimated by the food deterioration state estimation unit 6, the time limit estimation unit 12 must consume the food (expiration date or expiration date). Deadline). And as shown in FIG.12 (b), the food image which the estimated time limit is approaching is displayed on the operation panel 1, and it alert | reports to a user, thereby preventing forgetting consumption of food and wasteful disposal of food. We are trying to suppress it.

  FIG. 13 is a schematic diagram for explaining a method for estimating the expiration date of food or the expiration date (here, the expiration date) in the expiration date estimation unit 12. FIG. 13 schematically illustrates an example in which the time limit estimation unit 12 estimates the expiration date from the temporal color change (decrease in a * value) of meat stored in the time change detection unit 5. 13 (a) and 13 (b) are schematic diagrams showing food images at the time of warehousing and after T hours, and FIG. 13 (c) is a graph showing temporal changes in a * values. In FIG.13 (c), a vertical axis | shaft is a * value of a foodstuff (meat), and a horizontal axis is time. Time 0 is when the food is received (see FIG. 7 (a)), and T time is a time sufficient for the food to be oxidized (see FIG. 7 (b)). In FIG. 13C, the solid line (curve) indicates the change in the a * value, and the broken line (straight line) parallel to the time axis indicates the consumption limit of the a * value.

  The change in the a * value shown in FIG. 13C is obtained by measuring the a * value every time the refrigerator door 101 is opened and closed, and obtaining it as a curve passing through each a * value, and decreases with time. ing. From the decrease characteristic of the a * value and the detected value of the a * value after T time, the time when the a * value reaches the consumption limit, that is, the expiration date can be estimated. The food deterioration state estimation unit 6 estimates the expiration date in this way, and notifies the user via the operation panel 1 before the expiration date is reached. The notification to the user may be a method of displaying an alert together with a food image (FIG. 8), or a method of displaying the remaining time until the expiration date (FIG. 12B).

  Thereby, the user can know the time limit before reaching the food expiry date or the expiration date. Thereby, the food can be consumed early and forgetting to consume can be prevented. That is, wasteful disposal of food can be suppressed.

  The consumption limit shown here may be a preset absolute value of a * or a decrease from an initial value, or may be an actual value reflecting characteristic data of a detected time change of a *. If a deadline (consumption deadline, best-before date, or other deadline) that matches the actual use state can be set, the same effect can be obtained.

  FIG. 13C shows an example in which the deadline is estimated from the decrease characteristic of the a * value that is a general color index. However, the target time change data is not limited to this. For example, FIG. Estimate the deterioration tendency from the change in the area occupancy rate of various color samples shown in the example or increase / decrease of a specific color sample, set the consumption limit value for that value, and estimate the deadline from the arrival time. May be.

  The deadline estimated by the deadline estimation unit 12 is displayed on the operation panel 1 and is transmitted to the set temperature calculation unit 13. Further, the time limit set by the user in the time limit setting unit 15 built in the operation panel 1 is also transmitted to the set temperature calculation unit 13. The set temperature calculation unit 13 determines the set temperature in the storage room by comparing the time limit estimated by the time limit estimation unit 12 with the time limit set by the user in the time limit setting unit 15.

  That is, when the deadline estimated by the deadline estimating unit 12 is longer than the deadline set by the user in the deadline setting unit 15, the quality of the food is assumed to be maintained until the deadline desired by the user. There is no need to change the indoor temperature setting.

  On the other hand, when the deadline (estimated deadline) estimated by the deadline estimating unit 12 is shorter than the deadline (set deadline) set by the user in the deadline setting unit 15, the estimated deadline is brought closer to the set deadline. Reduce the set temperature in the storage chamber. This is because if the temperature in the refrigerator compartment 100 decreases, the degree of temporal deterioration of the food (gradient of decrease of a *) becomes moderate.

  In the refrigerator 1000, the in-compartment air cooled by the cooler 1003 is conveyed to each storage room by the air conveying device 1004 via the cooling air passage 1010. Then, the return air that has cooled each storage chamber returns to the cooler 1003 again via the return air passage 1020. At this time, the air (for example, −30 ° C. to −25 ° C.) cooled by the cooler 1003 is distributed to each storage room by opening and closing a plurality of dampers including the refrigerator compartment damper 104. By adjusting the amount of cooling air flowing into each storage room by opening and closing the plurality of dampers, individual temperatures are set for each storage room.

  For example, the inflow damper of the freezer compartment 400 (for example, −22 ° C. to −16 ° C.) set to the lowest temperature is almost fully opened, and the inflow damper of the vegetable room 500 (for example, 3 ° C. to 9 ° C.) set to the highest temperature is Almost fully closed. And for example, by indirectly cooling the vegetable compartment 500 with the return air which cooled the refrigerator compartment 100 (for example, 0 degreeC-6 degreeC) whose temperature setting is lower than the vegetable room 500, and the chilled room 110 (for example, 0 degreeC-2 degreeC). The temperature of each storage room is adjusted.

  Here, the temperature setting of each storage room can be adjusted to about ± 2 to 3 ° C. according to overcooling or undercooling. For example, the setting of the freezer compartment 400 can be changed in the range of about −25 ° C. to −13 ° C., and the setting of the refrigerator compartment 100 can be changed in the range of about −2 ° C. to 9 ° C.

  When a specific storage room is set to a low temperature due to insufficient cooling, for example, when only the refrigerating room 100 is set to a low temperature, the flow of cooling air into the refrigerating room 100 is increased by increasing the opening of the refrigerating room damper 104. Increase the amount. Moreover, when setting several storage chambers to low temperature, it is necessary to increase the rotation speed of the compressor 1002 and the conveyance air volume by the air conveyance apparatus 1004, and to raise the cooling capacity of a refrigerating cycle. Therefore, when the plurality of storage rooms are set to a low temperature, the power consumption of the refrigerator 1000 increases.

  On the other hand, when the storage room is set to a high temperature due to overcooling, the opening of the damper is reduced for a specific storage room to reduce the inflow of cooling air. Moreover, when setting several storage chambers to high temperature, what is necessary is just to reduce the rotation speed of the compressor 1002, and the conveyance air volume by the air conveyance apparatus 1004, and the power consumption of the refrigerator 1000 reduces.

  Accordingly, the cooling control unit 14 determines the opening degree of the damper in each storage chamber including the refrigerator compartment damper 104, the rotation speed of the compressor 1002, and the air conveyance according to the set temperature calculated by the set temperature calculation unit 13. The amount of conveyance air by the apparatus 1004 is controlled. At this time, particularly when it is desired to lower the set temperature of a specific storage room, the number of rotations of the compressor 1002 and the amount of air transported by the air transport device 1004 are not increased as much as possible, but are set by increasing the damper opening of the corresponding storage room. It is desirable to reduce the temperature. Since the power consumption does not change if only the damper opening is changed, it is possible to satisfy the expiration date or expiration date desired by the user without increasing the energy consumption due to unnecessary cooling.

<Effect>
As described above, according to the fourth embodiment of the present invention, the food deterioration state estimation unit 6 estimates the food deterioration tendency, and the time limit estimation unit 12 estimates the time limit based on the estimation result. Since the user is notified of the approaching food, forgetting to consume the food can be suppressed, and wasteful disposal can be suppressed.

  Further, it is possible to reduce the waste of space and energy caused by continuously storing the expired food in the refrigerator compartment 100. That is, by distributing the cooling capacity that would have been wasted if the food was left unattended to other food and other storage chambers, the rotational speed of the compressor 1002 and the amount of air flow conveyed by the air conveying device 1004 are reduced. Therefore, energy consumption can be reduced as the entire refrigerator 1000.

  In addition, the deadline setting unit 15 receives a deadline setting by the user, and compares the set deadline with the estimated deadline by the food deterioration state estimating unit 6 to calculate the set temperature in the storage room. Quality can be maintained.

<Other configuration examples>
The configuration example shown in FIG. 11 includes a refrigeration cycle, a time limit estimation unit 12, a set temperature calculation unit 13, a cooling control unit 14, and a time limit setting unit 15 in addition to the components described in the first embodiment. The door open / close detection device 7 described in the second embodiment, the lighting device 8, the photographing signal transmission device 9, and the warehousing food storage unit 10 may be further provided, or the internal temperature detection device 11 described in the third embodiment. May be further provided.

  In any case, the food deterioration state estimation unit 6 estimates the food deterioration tendency reflecting the open / close state of the refrigerator compartment door 101 or the temperature history in the refrigerator compartment 100, and based on the estimation result of the deterioration tendency, Since the estimation unit 12 estimates the deadline (expiration date or expiration date), the deadline estimation accuracy can be improved, food can be prevented from being forgotten, and wasteful disposal can be suppressed.

DESCRIPTION OF SYMBOLS 1 Operation panel (display part), 2 In-chamber imaging device (imaging device), 3 Food image extraction part, 4 Food image management part, 5 Time change detection part, 6 Food deterioration state estimation part, 7 Door opening / closing detection apparatus, 8 Illumination device, 9 imaging signal transmission device, 10 warehousing food storage unit, 11 internal temperature detection device, 12 deadline estimation unit, 13 set temperature calculation unit, 14 cooling control unit, 15 deadline setting unit, 100 refrigeration room (storage room) 101 cold room door (door), 102 cold room shelf board, 103 cold room return air path, 104 cold room damper, 110 chilled room, 111 chilled case, 200 ice making room, 300 switching room, 400 freezer room, 500 vegetable room 501 Vegetable room return air path, 1000 refrigerator, 1001 control board (control device), 1002 compressor, 1005 cooler, 1004 air conveying device, 1010 cooling air , 1020 the return air duct.

Claims (4)

A storage room for storing food;
A photographing device for photographing the storage chamber;
A control device that extracts and manages food images from images in the storage room photographed by the photographing device;
Regarding a food image managed by the control device, a time limit setting unit in which a user sets a time limit,
With a display and
The controller is
A time change detection unit for detecting a time change of the food image;
A food deterioration state estimation unit that estimates a deterioration tendency of the food based on a time change of the food image detected by the time change detection unit;
Based on the food deterioration tendency estimated by the food deterioration state estimation unit, a deadline estimation unit that estimates a deadline for the food,
A deadline estimated by the deadline estimating unit, and a set temperature calculating unit that calculates a set temperature in the storage room that can be after the deadline set by the user,
The said control apparatus displays the said food image and the information regarding deterioration of the food corresponding to the said food image on the said display part together, The refrigerator characterized by the above-mentioned. The refrigerator according to claim 1 , wherein the time limit setting unit is provided outside the refrigerator. A compressor that compresses the refrigerant, a condenser that condenses the refrigerant discharged from the compressor, a throttle device that expands the refrigerant that has flowed out of the condenser, and a refrigerant that is expanded by the throttle device is supplied to the storage chamber. A refrigeration cycle having a cooler for cooling air
An air conveyance device for conveying the air cooled by the cooler to the storage chamber;
A blowout air volume control device for adjusting an inflow amount of air supplied to the storage room;
According to the set temperature of the storage room calculated by the set temperature calculation unit, the cooling capacity for the storage room is controlled by controlling at least one of the compressor, the air transfer device, and the blown air volume control device. The refrigerator according to claim 1 or 2 , further comprising a cooling control unit to be changed. Wherein the control device, in addition to the food image, the time limit estimator has estimated due date, and of the temperature of the set temperature calculating unit has calculated, claims 1 to 3, wherein displaying at least one The refrigerator according to any one of the preceding items.