JP2018124053A - refrigerator - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a refrigerator capable of exhibiting an effect of suppressing food waste, compared to a conventional one.SOLUTION: A refrigerator includes: a plurality of storage compartments partitioned into at least one imaging range; a plurality of interior cameras for imaging at least one of the imaging ranges, or at least one interior camera for imaging the plurality of imaging ranges; a control device for storing images of the plurality of imaging ranges in a storage device and communicated with an operation display device for displaying the images; and a clocking device for measuring a date and time. The control device stores the plurality of images related to an imaging day and time, extracts a difference of the images from the images in the same imaging range of a different imaging day and time, and notifies that a food is preserved for more than a fixed period when there is a part which does not have a fixed period difference.SELECTED DRAWING: Figure 17

Description

  The present invention relates to a refrigerator that displays storage items of a refrigerator on a portable terminal or the like.

In recent years, with the diversification of eating habits and lifestyles, various kinds of foods are stored and stored in refrigerators. Against this background, the capacity of refrigerators has been increasing, and more and more foods are stored and stored in refrigerators. In such an environment, due to the increasing number of households, including double-income households, where families have different home times and meal times, foods that were planned to be used have been eaten by another family. There was inconvenience that there was no desired food in the refrigerator.
In addition, some of the foods that are not frequently taken out may forget their existence, have deteriorated when you notice them, have to discard them, and you may have to take irrational measures in terms of environment and cost. It was.

  With respect to such problems, an operation device in a food material management system aimed at improving the usability of the refrigerator is provided by matching the actual image of the refrigerator by input / output operation of information on the food material (for example, Patent Document 1).

JP 2003-222460 A

The technique described in Patent Document 1 selects which food material is stored in which storage room by selecting a storage room to be stored after selecting the food name displayed from the food name list. I am trying to manage.
However, in view of actual use, it can be imagined that it takes a lot of time and effort to carry out the above-mentioned operations for each food item when storing a large amount of food after purchase. Not difficult.
In addition, it is estimated that there are very many cases where the consumed food does not match the input food, including input mistakes and intentional non-input. Furthermore, it is difficult to display icons for unclassifiable foods, and it is difficult to identify them from the screen. For this reason, the technique described in Patent Document 1 has a problem that the accuracy of food management becomes low.

  This invention is made | formed in order to solve the above subjects, and it aims at obtaining the refrigerator which can exhibit the effect which suppresses a foodstuff discard than before.

  The refrigerator according to the present invention includes a plurality of storage rooms partitioned into at least one photographing range, a plurality of internal cameras that photograph at least one of the photographing ranges, or at least one that photographs a plurality of the photographing ranges. Two storage cameras, a control device for storing a plurality of images in the shooting range in a storage device, communicating with an operation display device for displaying the images, and a timing device for measuring date and time, the control device comprising: , Storing a plurality of the images in association with the shooting date and time, extracting a difference between these images from the images in the same shooting range with different shooting dates and times, if there is a portion where there is no difference for a certain period, the food is for a certain period This is to notify that the information is stored.

  The refrigerator according to the present invention photographs a plurality of storage rooms partitioned into at least one photographing range, a plurality of internal cameras that photograph at least one of the photographing ranges, or a plurality of the photographing ranges. And a control device that stores at least one in-compartment camera, a plurality of images in the shooting range in a storage device, and communicates with an operation display device that displays the images, and a time measuring device that measures the date and time. The apparatus stores a plurality of the images in association with the shooting date and time, and when the specific location of the displayed image is selected in the operation display device, the images with different shooting dates and times including the specific location are stored. The difference between these images is extracted from and the period when the food at the specific location is stored or the date and time when the food stored at the specific location is taken out is displayed. That.

  The refrigerator according to the present invention can exert the effect of suppressing the discard of food materials as compared with the conventional case.

It is a front view which shows the refrigerator which concerns on Embodiment 1 of this invention. It is a front view which shows the state which opened the door of the refrigerator compartment in the refrigerator which concerns on Embodiment 1 of this invention. It is side surface sectional drawing which shows the refrigerator which concerns on Embodiment 1 of this invention. It is side surface sectional drawing which shows the refrigerator which concerns on Embodiment 1 of this invention. It is side surface sectional drawing which shows the refrigerator which concerns on Embodiment 1 of this invention. It is side surface sectional drawing which shows the refrigerator which concerns on Embodiment 1 of this invention. It is side surface sectional drawing which shows the refrigerator which concerns on Embodiment 1 of this invention. It is side surface sectional drawing which shows the refrigerator which concerns on Embodiment 1 of this invention. It is a functional block diagram of the refrigerator which concerns on Embodiment 1 of this invention. It is a flowchart which shows an example of imaging | photography and the image storage process in the refrigerator which concerns on Embodiment 1 of this invention. It is an example of the state transition of the image which the control apparatus of the refrigerator which concerns on Embodiment 1 of this invention displays on an operation display apparatus. It is another example of the state transition of the image which the control apparatus of the refrigerator which concerns on Embodiment 1 of this invention displays on an operation display apparatus. It is another example of the state transition of the image which the control apparatus of the refrigerator which concerns on Embodiment 1 of this invention displays on an operation display apparatus. It is another example of the state transition of the image which the control apparatus of the refrigerator which concerns on Embodiment 1 of this invention displays on an operation display apparatus. It is another example of the state transition of the image which the control apparatus of the refrigerator which concerns on Embodiment 1 of this invention displays on an operation display apparatus. It is another example of the state transition of the image which the control apparatus of the refrigerator which concerns on Embodiment 1 of this invention displays on an operation display apparatus. It is an example of the alert screen which the control apparatus of the refrigerator which concerns on Embodiment 1 of this invention displays on an operation display apparatus. It is a functional block diagram of the network system which concerns on Embodiment 2 of this invention. It is a flowchart which shows an example of the image display method with the portable terminal in the network system which concerns on Embodiment 2 of this invention.

Embodiment 1 FIG.
FIG. 1 is a front view showing a refrigerator according to Embodiment 1 of the present invention. FIG. 2 is a front view showing a state in which the door of the refrigerator compartment in the refrigerator is open. Moreover, FIGS. 3-8 is side surface sectional drawing which shows this refrigerator. 3 to 8, the left side is the front side of the refrigerator 1. 5 and 7 show a state where the freezer compartment 400 is drawn out, and FIGS. 6 and 8 show a state where the vegetable compartment 500 is drawn out.
The refrigerator 1 includes a plurality of storage rooms (a refrigerator room 100, a switching room 200, an ice making room 300, a freezing room 400, a vegetable room 500, and a chilled room 600).

  The refrigerator compartment 100 is provided in the uppermost part of the refrigerator 1, and the opening part of the front side is obstruct | occluded by the double door 8 so that opening and closing is possible. Further, a chilled case 601 is housed inside the refrigerator compartment 100, and the inside of the chilled case 601 is a chilled chamber 600. Below the refrigerating room 100, a drawer door type, that is, a switching room 200 and an ice making room 300 in which the storage room is drawn to the user side when the drawer door is pulled out are arranged in parallel. The switching chamber 200 is a storage chamber that can be switched from a refrigeration temperature zone (−18 ° C.) to a temperature zone such as refrigeration (3 ° C.), chilled (0 ° C.), and soft refrigeration (−7 ° C.). A freezing room 400 is provided below the switching room 200 and the ice making room 300, and a vegetable room 500 is provided below the freezing room 400. The freezer compartment 400 and the vegetable compartment 500 are also of a drawer door type, that is, the storage compartment is pulled out to the user side when the drawer door is pulled out.

  On the surface of the door 8 of the refrigerator compartment 100, there is provided an operation panel 6 composed of an operation switch 6a for adjusting the temperature and setting of each storage room, a display unit 6b for displaying the temperature of each storage room, and the like. (See also FIG. 9 described later). In addition, storage cases 201, 401, and 501 serving as storage units are installed in the switching room 200, the freezing room 400, and the vegetable room 500, respectively, and can store food. Furthermore, in the freezer compartment 400 and the vegetable compartment 500, upper trays 402 and 502 serving as storage units are installed above the storage cases 401 and 501, respectively, to improve organization in a food storage state. That is, the freezer compartment 400 and the vegetable compartment 500 have two storage units stacked in the vertical direction.

  On the back side of the refrigerator 1, a compressor 2, a cooler 3, a blower fan 4, an air passage 5, and a control device 7 are provided as a cooling mechanism for cooling each storage room. The control device 7 includes a control device and a storage device 7a storing an operation program thereof (see FIG. 9 described later), and controls the operation of the refrigerator 1. The cold air produced by the compressor 2 and the cooler 3 is blown by the blower fan 4 and is blown through the air passage 5 to the freezing room 400, the switching room 200, the ice making room 300, and the refrigerating room 100. Cool down. The vegetable compartment 500 is cooled by circulating the return cold air from the refrigerator compartment 100 from the return air passage for the refrigerator compartment, and is returned to the cooler 3 from the return air passage for the vegetable compartment (the return air passage is not shown). The temperature of each storage room is detected by a thermistor (not shown) installed in each storage room, and a damper (not shown) installed in the air passage 5 so that each storage room has a preset temperature. ), The output of the compressor 2, and the amount of air blown by the blower fan 4 are controlled.

  The refrigerator compartment 100 includes a door open / close detection switch 9 that detects the open / closed state of the door 8, door pockets 10 a, 10 b, and 10 c provided inside the door 8 (hereinafter may be collectively referred to as the door pocket 10), There are provided shelves 11a, 11b, 11c, and 11d (hereinafter sometimes collectively referred to as shelves 11) for partitioning the inside of the refrigerator compartment 100 and a water supply tank 14 for ice making.

  Furthermore, the refrigerator 1 according to Embodiment 1 is provided with an in-compartment camera 12 for photographing the storage room. Moreover, the illumination part 13 is also provided as a light source at the time of image | photographing the storage room with the camera 12 in a store | warehouse | chamber.

  Specifically, as shown in FIGS. 2 and 3, in the refrigerator compartment 100, the interior cameras 12 a and 12 b and an illumination unit 13 a that illuminates the entire refrigerator compartment 100 are provided. The in-compartment camera 12a installed on the ceiling surface photographs the stored items arranged on the shelf 11. The shelves 11a, 11b, 11c, and 11d have a dimensional relationship in which the depth of the shelf is long or equivalent from the upper stage to the lower stage in the field of view of the in-compartment camera 12a. It has a structure. Similarly, the in-compartment camera 12b installed on the ceiling surface photographs the surface on the door pocket 10 side (the side facing the inside of the refrigerator compartment 100 in the door 8). Since the door pockets 10a, 10b, and 10c have a dimensional relationship in which the depth dimension is long or equivalent from the upper stage to the lower stage in the field of view of the internal camera 12b, the internal camera 12b installed on the internal ceiling surface is An image with few blind spots can be taken. That is, in the first embodiment, the inside of the refrigerator compartment 100 is divided into three shooting ranges. More specifically, the inside of the refrigerator compartment 100 that is one photographing range is photographed by the internal camera 12a. The double door 8 is divided into two shooting ranges, a left door 8 and a right door 8, and these shooting ranges are shot by the internal camera 12b.

  Further, as shown in FIG. 3, on the back side of the refrigerator compartment 100, an in-compartment camera 12 c that takes an image of the chilled chamber 600 provided in the refrigerator compartment 100 as one imaging range is provided. In the chilled chamber 600, since the chilled case 601 forming the chilled chamber 600 is made of a transparent resin, it is possible to capture a sufficient amount of light with the light from the illumination unit 13a.

  Similarly, as shown in FIG. 4, the switching room 200 is provided on the ceiling surface thereof with an internal camera 12d that photographs the inside of the switching room 200 as one photographing range, and an illumination unit 13b that illuminates the inside of the switching room 200. Therefore, the inside of the switching room 200 can be taken.

  Moreover, as shown in FIG.5 and FIG.6, in the freezer compartment 400 and the vegetable compartment 500, the storage cameras 12e and 12f and the illumination parts 13c and 13d are each installed. In the freezer compartment 400 and the vegetable compartment 500, the interior cameras 12e and 12f are installed near the openings on the ceiling surfaces of the respective storage compartments, and trigger that the storage cases 401 and 501 are pulled out more than a certain distance. As a result, it is possible to visually recognize almost the entire area. It can be detected by the drawer detection device 30 whether or not it has been pulled out beyond a certain distance. The drawer detection device 30 is configured using, for example, a photocoupler or a proximity sensor, and by detecting the passage of the storage cases 401 and 501, it is possible to detect the distance to which the storage cases 401 and 501 are drawn, When the storage cases 401 and 501 are pulled out beyond a certain distance, it is possible to take a picture at an arbitrary position. Thereby, the in-compartment camera 12e can photograph the upper tray 402 (upper storage unit) of the freezer compartment 400 as one photographing range. Further, the in-compartment camera 12f can take an image of the upper tray 502 (upper storage portion) of the vegetable compartment 500 as one image pickup range.

  As shown in FIGS. 5 and 6, when the storage cases 401 and 501 are pulled out, the upper trays 402 and 502 are also pulled out at the same time. For this reason, in order to place food in the storage cases 401 and 501 that form the lower part of the freezer compartment 400 and the vegetable compartment, as shown in FIGS. After that it is necessary to install food. At that time, the drawer detection devices 30 are also provided for the upper trays 402 and 502, respectively, and when it is detected that the upper trays 402 and 502 are stored in the back more than a certain distance, the internal cameras 12e and 12f Take a picture. Thereby, the internal camera 12e can image the storage case 401 (lower storage unit) of the freezer compartment 400 as one imaging range. In addition, the internal camera 12f can take an image of the storage case 501 (lower storage portion) of the vegetable compartment 500 as one shooting range.

  That is, the freezer compartment 400 can photograph both the inside of the storage case 401 and the upper tray 402, and similarly, the vegetable compartment 500 can also photograph both the inside of the storage case 501 and the upper tray 502. .

  The internal camera 12 has a configuration that is covered with a transparent cover, and is configured so that stored items such as food do not come in contact with it, and is protected from impact and moisture. Further, heating means such as a heater (not shown) is provided near the internal camera 12 to prevent the internal camera 12 from condensing. Moreover, it is good also as a structure which can provide the drive part which can change the angle of up-down and right-and-left to the internal camera 12, and can image | photograph the image from which an angle differs with the internal camera 12. FIG. That is, the internal camera may be configured to swing. With this configuration, it is possible to prevent blind spots and to detect the stored items with higher accuracy. Further, for example, by placing the in-compartment camera 12 that can be swung (changing the angle in the vertical direction and moving the field of view in the front-rear direction) in the refrigeration room 100, the single in-compartment camera 12 can be refrigerated. It is also possible to photograph both the photographing range inside the room 100 and the two photographing ranges of the door 8.

FIG. 9 is a functional block diagram of the refrigerator according to Embodiment 1 of the present invention. The same components as those in FIGS. 1 to 8 are denoted by the same reference numerals.
The control device 7 acquires the temperature in each storage chamber from the thermistor provided in each storage chamber, and in accordance with a program stored in advance so that each storage chamber is maintained at the set temperature, The operation state of the fan 4 and a damper (not shown) installed in the air passage 5 is controlled. Further, the control device 7 detects the opening / closing of the door 8 by the door opening / closing detection switch 9, detects the drawer state of the storage cases 401, 501 and the upper trays 402, 502 by the drawer detection device 30, and turns on the illumination unit 13. Turns on or off.

  For example, when the control device 7 detects that the door 8 is opened, the lighting unit 13a is turned on. When the control device 7 detects that the door 8 is closed, the interior cameras 12a and 12b are turned on while the lighting unit 13a is turned on. , 12c are operated to photograph the inside of the refrigerator compartment 100 and the inside of the chilled room 600. And the control apparatus 7 will light-extinguish the illumination part 13a, after imaging | photography operation | movement is complete | finished. Further, when the control device 7 detects that the switching room 200 is closed after being pulled out by a door opening / closing detection switch (not shown), the lighting unit 13b is turned on and the inside of the switching room 200 is photographed by the internal camera 12d. To do. Then, when the photographing operation is finished, the control device 7 turns off the illumination unit 13b. In addition, the control device 7 lights the illumination units 13c and 13d at the timing when the storage cases 401 and 501 and the upper trays 402 and 403 are imaged by the above-described in-compartment cameras 12e and 12f. 13d is turned off.

The control device 7 performs arithmetic processing such as distortion correction and / or color correction on the image captured by the internal camera 12, and stores the image in the storage device 7a (In the first embodiment, An image that has been subjected to arithmetic processing such as distortion correction and / or color correction is also referred to as an image photographed by the internal camera 12). The control device 7 can communicate with the operation panel 6 to display an image of the storage location selected by the operation switch 6a on the display unit 6b of the operation panel 6. For this reason, the user can check the image of a desired location without opening the door 8 or the like.
Here, the operation panel 6 corresponds to the operation display device of the present invention.

Furthermore, the control device 7 is connected to the telecommunication line 16 by the communication means 15 provided on the upper surface of the refrigerator 1, and the portable terminal such as the tablet terminal 18 a and the mobile phone 18 b (smart phone) is connected via the telecommunication line 16. Image data can be transmitted to the terminal 18. That is, even if the user is not near the refrigerator 1, the storage status of each storage room of the refrigerator 1 can be grasped at a place the user likes, such as going out, and convenience is improved. In addition, since the user can check the space that can be stored in the refrigerator 1 at a remote place such as a shopping destination, it is possible to prevent mistakes such as the amount of food to be purchased being too large to enter the refrigerator 1. You can also
Here, the portable terminal 18 also corresponds to the operation display device of the present invention.

  The telecommunication line 16 refers to a digital or analog signal communication line such as an electric light line, infrared ray, non-infrared ray, wireless, public line, optical cable, ISDN, ADSL, Internet and satellite. The communication means 15 is an interface capable of transmitting and receiving digital or analog signals. When communicating by wire, for example, a serial interface or driver is provided, and when communicating wirelessly, for example, A communication module such as Wi-Fi (registered trademark) or Bluetooth (registered trademark) is provided.

  FIG. 10 is a flowchart showing an example of photographing and image accumulation processing in the refrigerator according to Embodiment 1 of the present invention. When the control device 7 detects that the door 8 of the refrigerating room 100 is opened or the switching room 200, the freezing room 400, and the vegetable room 500 are pulled out (S101), the control device 7 checks the storage capacity of the storage device 7a. (S102). When there is a margin in the storage capacity of the storage device 7a, the control device 7 turns on the illumination unit 13 (S104). When the storage device 7a is over capacity, the control device 7 turns on the illumination unit 13 (S104) after performing an image deletion process (S103) that deletes, for example, the oldest stored or displayed image. Note that the image deletion process may be performed after the illumination unit 13 is turned on.

  The control device 7 takes an image by operating the internal camera 12 with the illumination unit 13 turned on (S105). Here, the refrigerator 1 which concerns on this Embodiment 1 has the time measuring device 7b in the control apparatus 7, for example. For this reason, the control device 7 stores the photographed image in the storage device 7a by associating (tagging) the photographing date and time with the photographing location (S106, S107). After the photographing is finished, the control device 7 turns off the illumination unit 13 (S108).

  When the storage status of each storage chamber of the refrigerator 1 changes, that is, when adding a storage item to each storage chamber or moving a storage item between storage chambers, the user can The door 8 is opened and closed, and the switching room 200, the freezing room 400, and the vegetable room 500 are always pulled out and stored. For this reason, shooting in conjunction with opening / closing of the door 8 of the refrigerator compartment 100 and the drawer and storage of the switching chamber 200, the freezer compartment 400, and the vegetable compartment 500 can be taken images when the storage status is updated. This means that it is possible to grasp that there has been a change in the storage situation in each storage room.

FIG. 11 is an example of a state transition of an image displayed on the operation display device by the refrigerator control device according to Embodiment 1 of the present invention. In addition, FIG. 11 has shown the example of the image currently displayed on portable terminals 18, such as the tablet terminal 18a and the mobile telephone 18b (smart phone).
The control device 7 causes the portable terminal 18 to display the appearance of the refrigerator 1 as a start image (FIG. 11A). When the user touches the position of one of the displayed storage rooms of the refrigerator 1, the control device 7 causes the portable terminal 18 to display an image of the touched storage room. In the first embodiment, the storage room that has changed most recently is indicated by gentle blinking, and a display that prompts the user to touch is performed.

When the user touches the position of one of the displayed storage rooms of the refrigerator 1, the display transitions to the display of the image of each storage room. For example, when the user touches the position of the refrigerating room 100, the control device 7 displays an internal image of the refrigerating room 100 as shown in FIG. For example, when the user touches the position of the freezer compartment 400, the control device 7 displays an internal image of the freezer compartment 400 as shown in FIG. For example, when the user touches the position of the vegetable compartment 500, the control device 7 displays an internal image of the vegetable compartment 500 as shown in FIG.
FIG. 11C shows the refrigerator compartment 100 and the freezer compartment 400 connected together, and FIG. 11D shows the refrigerator compartment 100 and the vegetable compartment 500 connected together. By displaying in a row, the user can easily recall the positional relationship between the storage rooms, and the listability of the food in the refrigerator 1 can be improved.

FIG. 12 is another example of a state transition of an image displayed on the operation display device by the refrigerator control device according to Embodiment 1 of the present invention. FIG. 12 shows an operation when viewing the food stored in the door pocket 10. FIG. 12A is the same as FIG. 11B and shows the transition after FIG. 11B.
The image in the refrigerating room 100 (by the so-called swipe operation or flick operation performed on the touch panel of the smartphone or tablet terminal) is touched with the finger and moved left and right with respect to the image in FIG. When the operation of instructing the movement of the first image) is performed, the control device 7 is operated by the user from the front side of the refrigerator 1 as shown in FIGS. 12 (b) and 12 (c). An image (corresponding to the second image of the present invention) on the door pocket 10 side of the door 8 located in the opposite direction to the inside of the refrigerator compartment 100 in the observation state in which the inside is observed is displayed in a row with the image in the refrigerator compartment 100. .

  For example, when it is instructed to move the image in the refrigerator compartment 100 to the right, the control device 7 allows the user to move through the refrigerator compartment 100 from the front side of the refrigerator 1 as shown in FIG. The image on the door pocket 10 side of the door 8 on the left side in the observed observation state is displayed continuously with the image in the refrigerator compartment 100. Further, for example, when an instruction is given to move the image in the refrigerator compartment 100 to the left, the control device 7 allows the user to enter the refrigerator compartment 100 from the front side of the refrigerator 1 as shown in FIG. The image on the door pocket 10 side of the door 8 on the right side in the observation state in which the image is observed is displayed continuously with the image in the refrigerator compartment 100. At this time, as shown in FIGS. 12 (b) and 12 (c), the control device 7 includes the inside of the refrigerating room 100 and the left and right doors in an observation state in which the user observes the inside of the refrigerating room 100 from the front side of the refrigerator 1. The images in the refrigerator compartment 100 and the images of the left and right doors 8 are displayed together so as to be the same as the positional relationship with FIG.

When the operation to instruct the movement of the image in the refrigerator compartment 100 is performed, the door 8 of the actual refrigerator 1 is opened by displaying the image on the door pocket 10 side of the door 8 positioned in the opposite direction. It is possible to display images of the left and right doors 8 with the same positional relationship.
Further, when displaying the images of the left and right doors 8, the positional relationship between the inside of the refrigerator compartment 100 and the left and right doors 8 in the observation state in which the user observes the inside of the refrigerator compartment 100 from the front side of the refrigerator 1 is the same. In addition, by displaying the images in the refrigerator compartment 100 and the images of the left and right doors 8 together, it is possible to achieve a display with enhanced realism.

  Note that the operation of instructing the movement of the image in the refrigerator compartment 100 is not limited to the operation of moving the image left and right by touching a finger. For example, as shown in FIGS. 12D to 12F, the image may be moved by tapping the direction indication display 70 displayed on the screen. Here, FIGS. 12 (d), 12 (e), and 12 (f) correspond to FIGS. 12 (a), 12 (b), and 12 (c), respectively.

FIG. 12 shows an example in which the freezer compartment 400 is displayed below the image in the refrigerator compartment 100, but the freezer compartment 400 may not be displayed.
In addition, when a storage room other than the refrigerator compartment 100 has a double door, the images of the storage room and the door may be displayed as described above.

FIG. 13 is another example of the state transition of an image displayed on the operation display device by the refrigerator control device according to Embodiment 1 of the present invention. FIG. 13 shows an operation when viewing food in the freezer compartment 400 and the vegetable compartment 500. FIG. 13A is the same as FIG. 11C and shows the transition after FIG. 11C.
The image shown in FIG. 13A can be changed by touching and moving the finger up (or tapping the direction indication display 70). Specifically, when an operation of sending the image once upward with respect to FIG. 13A displaying the upper tray 402 of the freezer compartment 400 is performed, the image of the upper tray 402 of the freezer compartment 400 disappears, and the freezer An internal image of the storage case 401 (the storage part (shooting range) located below the upper tray 402), which is the lower part of the chamber 400, is displayed. This simulates that the upper tray 402 of the freezer compartment 400 has been pulled into the freezer compartment 400, and moves in the same direction as the actual work performed in the refrigerator 1, so that the movement of the image is easy to understand intuitively. . Further, when the operation of sending the image one time upward from FIG. 13B is performed, it is simulated that the storage case 401 of the freezer compartment 400 is pulled back, and the upper tray of the vegetable compartment 500 as shown in FIG. 13C. 502 images are displayed. Similarly, when the operation of sending the image upward once is performed, an image in which the lower part of the storage case 501 at the lower stage of the vegetable room 500 is seen as shown in FIG.

  In addition, although the example which displayed the image in the refrigerator compartment 100 on the upper side of the freezer compartment 400 and the vegetable compartment 500 was shown in FIG. 13, the image in the refrigerator compartment 100 does not need to be displayed.

FIG. 14 is another example of a state transition of an image displayed on the operation display device by the refrigerator control device according to Embodiment 1 of the present invention. In addition, this FIG. 14 represents the transition at the time of returning from the state which displayed the freezer compartment 400 and the vegetable compartment 500 to the state which displayed only the refrigerator compartment 100. FIG. FIG. 14 (a) is similar to FIG. 13 (d) and shows the transition after FIG. 13 (d).
As described with reference to FIG. 13, the image can be changed by touching and moving the freezer compartment 400 and the vegetable compartment 500 (or tapping the direction indication display). On the contrary, the transition which returns to the refrigerator compartment 100 from the lower stage image (storage case 501) of the vegetable compartment 500 is shown. That is, by moving the finger downward, it is simulated that the upper trays 402 and 502 and the storage cases 401 and 501 are pulled out, so that the state of each storage location can be confirmed sequentially.

  Furthermore, as shown in FIG. 14 (e), for example, by swiping the refrigeration room 100 portion downward, the entire view of the refrigeration room 100 is obtained, and regardless of the display state of the freezer room 400 and the vegetable room 500, FIG. ).

  13 and 14 may be performed only between the upper tray 402 and the storage case 401 of the freezer compartment 400, or the upper tray 502 and the storage case 501 of the vegetable compartment 500. You may carry out only between. Moreover, when the upper trays 402 and 502 are not provided in the freezer compartment 400 and the vegetable compartment 500, it is shown in FIGS. 13 and 14 between the storage case 401 of the freezer compartment 400 and the storage case 501 of the vegetable compartment 500. An image display method may be implemented.

  Here, as shown in FIGS. 12 to 14, the control device 7 displays the refrigerator icon 80 together with the image of each storage room (shooting range), and displays which part (which shooting range) of which store room is now. You may show whether you are doing. The part corresponding to the image is shown in a different color, and there is an effect that the part displayed at a glance can be recognized. As shown in FIGS. 12 to 14, the open / closed state of the door 8, the states of the storage cases 401 and 501, and the upper trays 402 and 502 (whether or not they are pulled out) may be displayed and recognized.

FIG. 15 is another example of a state transition of an image displayed on the operation display device by the refrigerator control device according to Embodiment 1 of the present invention. FIG. 15 shows a function that can display a change in image in time series.
As shown in the functional block diagram of FIG. 9 and the flowchart of FIG. 10, the image triggers that the door 8 of the refrigerator compartment 100 is opened, the switching chamber 200, the freezer compartment 400, and the vegetable compartment 500 are pulled out. And is stored in the storage device 7a together with the shooting date and time information. For this reason, by displaying the shooting date and time together with the image, it becomes easy to manage the items stored in the storage room. Here, the images from FIG. 15A to FIG. 15F are in a state in which the photographing date and time are sequentially arranged in the newest order. For example, the display of the refrigerator compartment 100 of FIG. When the operation of sending down is executed, FIG. 15B, which is an image of the previous generation, is displayed.

  As shown in this image, FIG. 15A taken at 18:00 on December 31 (December 31) and FIG. 15B taken at 16:00 on 12/31 are compared, Focus on the cake 81 placed in the middle of the refrigerator compartment 100. Since the cake that was not in FIG. 15 (b), which is the image of the previous generation, is in FIG. 15 (a), it can be seen from the image that the user was put in the refrigerator room 100 when the door opened and closed at 18:00. It can be estimated.

  As another example, a management example of beer 82 stored in the refrigerator compartment 100 can be given. Whether or not the beer 82 is sufficiently cooled can be determined based on when the beer 82 is put in the refrigerator compartment 100. If the image is traced back, it exists in FIG. 15 (d) and does not exist in FIG. 15 (e). That is, since it can be seen that the beer 82 was put in at 12:00 on 12/31, the user guessed that about 6 hours had passed in the refrigerator compartment 100 against the current time. It becomes possible. That is, the user can determine whether the beer 82 is sufficiently cooled.

  By going back to the image as described above, since a history image of the shooting date and time remains, at what timing the ingredients (contained items) entered, in other words, any ingredients have been put in the refrigerator 1 since The estimation of Takano's food management period is possible from the image, and the food storage period and the deterioration estimation can be judged.

  The input operation on the touch panel for going back to the past is performed by a swipe operation in a direction not to scroll the screen at a specific position, a tap on the image retroactive display 83 for going back, or the like, as shown by the simulated finger in FIG. Realization is possible.

FIG. 16 is another example of a state transition of an image displayed on the operation display device by the refrigerator control device according to Embodiment 1 of the present invention. Note that FIG. 16 shows a function of estimating the storage period of stored items such as food at a selected specific location using temporal changes in images.
For example, roast chicken that does not exist in FIG. 16A is stored as indicated by a broken line in FIG. 16B, and the shooting date and time is stored in the storage device 7a. The control device 7 can determine that the roast chicken has been stored in the refrigerator compartment 100 on the shooting date and time of FIG. 16B based on the difference between the images in FIG. 16A and FIG. .

  As an example, for example, an area selection operation on the touch panel of the mobile terminal 18 is performed, and an operation for specifying a location for confirming the storage state (operation for selecting a specific location) is performed. The control device 7 determines whether or not there has been a temporal change in the image of the part by selecting an arbitrary specific part. For example, when the stored item is stored in a specific location of an image several generations before, the control device 7 takes out the stored item from the specific location when the storage state changes in the most recently captured image. If there is no change, it is determined that the stored state of the stored item is maintained. Further, for example, when the stored item is stored in a specific place of the most recent image, the control device 7 stores the stored item in the specific place when the storage state changes in an image captured several generations ago. It is determined that it has been done.

  For the determination by the image, the image is divided into pixel units, pixel coordinates (x, y) representing positions in the horizontal direction (x) and the vertical direction (y) are given, and pixels of the same coordinates are The luminance L (x, y) is compared, and if the difference is equal to or greater than a preset threshold value, it is determined that there is a change, that is, the stored item has changed. Judge that there is no change in the object. In this way, it is possible to determine the presence / absence of stored items on a pixel basis.

  In FIG. 16C, as an example of selecting a specific location, it is shown that a food (region) that is anxious on the image is tapped. When a specific location is selected by the tap operation, the control device 7 determines that there is no change after comparing the difference between the previous image and the current image, for example, and as shown in FIG. The time difference between images is displayed as the storage elapsed period. Here, it is determined that the roast chicken is not moving until 18:00 of 1/1 to 18:00 of 1/8, and that the storage elapsed period is 7 days and 2 hours is displayed on the instruction unit .

  Thus, since the control apparatus 7 can judge and display the elapsed period after storing the stored items, it can give useful information to the user's consumption of ingredients such as the need to rush consumption. It becomes possible. For this reason, it is possible to reduce the risk of discarding the stored food without eating it, so that it is possible to exert an effect on the suppression of the environment and the user's food purchase cost.

  Note that the control device 7 may determine the elapsed time after the storage of the stored item by comparing the time change of the entire image, that is, by comparing the images in the same shooting range with different shooting dates and times.

FIG. 17 is an example of an alert screen displayed on the operation display device by the refrigerator control device according to Embodiment 1 of the present invention.
As shown in FIG. 16, since the control device 7 can estimate the storage elapsed period of the stored items, it passes an arbitrarily determined period such as 7 days in the refrigerator room 100 or 30 days in the freezer room. If there is a remaining food (area), an alert is displayed to notify the user that the disposal deadline is approaching. In the alert display, flashing or highlighting such as red letters is used in a portion close to the storage location, and alerting is performed by instructing the storage location.
Note that the storage period to be alerted may be set by the user for each storage room, region, and food, or the storage period recommended by the refrigerator supplier may be stored in the storage device 7a as a default. .
With this function, risks such as discarding foods that have been forgotten to exist are reduced, and it is also possible to exert an effect on the suppression of the environment and the user's food purchase costs.

  The internal camera 12 is not limited to the refrigerator room 100, the switching room 200, the freezer room 400, and the vegetable room 500, and may be provided in another storage room or storage space. In addition, the internal camera 12 can be viewed inside and at the time of opening and closing the door and case even if it is captured as a moving image instead of a still image.

Embodiment 2. FIG.
In the first embodiment, the manner of displaying an image to be displayed on the operation display device is controlled by the control device 7 of the refrigerator 1. Not only this but when using portable terminals 18, such as tablet terminal 18a and cellular phone 18b, as an operation display device, the following network system provided with refrigerator 1 and portable terminal 18 is constructed, and is displayed on portable terminal 18. The way of displaying an image may be controlled by the mobile terminal 18 itself. Configurations not described in the second embodiment are the same as those in the first embodiment, and the same configurations as those in the first embodiment are denoted by the same reference numerals as those in the first embodiment.

FIG. 18 is a functional block diagram of a network system according to Embodiment 2 of the present invention.
In the network system 700 according to the second embodiment, the refrigerator 1 and a mobile terminal 18 (tablet terminal 18a, mobile phone 18b, etc.) that are operation display devices are connected via an electric communication line 16. In the network system 700, the control device 7 of the refrigerator 1 associates (tags) the image capturing range of the image and the image capturing date / time of the image with images of a plurality of image capturing ranges, It is configured to transmit to the portable terminal 18 via the telecommunication line 16. In addition, the mobile terminal 18 has a configuration in which the mobile terminal 18 itself controls the received image in the shooting range and displays the image on the mobile terminal 18 itself as shown in FIGS. .

  As shown in FIG. 18, the portable terminal 18 configured as described above includes a communication device 515, an image storage device 516, a display device 550, a control device 507 that is a control device for an operation display device of the present invention, and an operation device. 506.

  The communication device 515 communicates with the communication means 15 of the refrigerator 1 via the electric communication line 16. The image storage device 516 stores an image of the shooting range transmitted from the refrigerator 1 (an image shot by the internal camera 12 and transmitted by associating the shooting range and shooting date / time of the image). Is. In addition, the image storage device 516 stores data such as a direction instruction display 70 and a refrigerator icon 80 displayed on the display device 550 together with the start image and the image of the photographing range shown in FIG. In the second embodiment, as described later, after the images transmitted from the refrigerator 1 are combined by the control device 507, the combined images are also stored (stored) in the image storage device 516. It has become. In the second embodiment, the above composite image is also handled as a kind of image in the shooting range.

  The display device 550 displays at least one of the images in the shooting range transmitted from the refrigerator 1 (including the synthesized image). The control device 507 causes the display device 550 to display at least one of the images in the shooting range transmitted from the refrigerator 1 (including the synthesized image). The operation device 506 is an operation performed on the mobile terminal 18 by the user (an operation for changing an image displayed on the display device), such as the swipe operation, the flick operation, and the tap operation of the direction instruction display 70 described in the first embodiment. Etc.).

  Next, an example of an image display method on the mobile terminal 18 in the second embodiment will be described.

FIG. 19 is a flowchart showing an example of an image display method on the mobile terminal in the network system according to Embodiment 2 of the present invention.
When the operation device 506 receives an image display instruction from the user, the control device 507 confirms whether an image is transmitted from the refrigerator 1 to the communication device 515 (S201). When the image is transmitted from the refrigerator 1 to the communication device 515, the control device 507 confirms the storage capacity of the image storage device 516 (S202). When the storage capacity of the image storage device 516 has a margin, the control device 507 stores (accumulates) the image transmitted from the refrigerator 1 in the image storage device 516 (S204). When the image storage device 516 is over capacity, the control device 507 performs, for example, an image deletion process (S203) for deleting the oldest stored or displayed image, and then displays the image transmitted from the refrigerator 1 as an image. The data is stored (accumulated) in the accumulation device 516 (S204).

  After storing (accumulating) the image transmitted from the refrigerator 1 in the image accumulating device 516, the control device 507 is based on the imaging range and the imaging date / time associated (tagged) with the image transmitted from the refrigerator 1. The images transmitted from the refrigerator 1 are combined and stored (stored) in the image storage device 516 (S205, 206). For example, when the image as shown in FIG. 12 is displayed on the display device 550, the control device 507 is photographed most recently on both sides of the image in the refrigerator compartment 100 that was photographed most recently (the latest photographing date and time). Then, a composite image obtained by connecting the images on the door pocket 10 side of the left and right doors 8 is created and stored in the image storage device 516. Further, for example, when displaying the images as shown in FIGS. 13 and 14 on the display device 550, the control device 507 causes the freezer compartment 400 photographed immediately below the image in the refrigerator compartment 100 photographed most recently. A composite image obtained by connecting the images of the upper tray 402, a composite image obtained by connecting the images of the storage case 401 of the freezer compartment 400, which are taken immediately below the images in the refrigerator compartment 100, which have been taken most recently, and the most recently taken images. A composite image obtained by connecting the images of the upper tray 502 of the vegetable room 500 taken immediately below the image in the refrigerator compartment 100 and a photograph taken immediately below the images in the refrigerator compartment 100 photographed most recently. A composite image obtained by connecting the images of the storage case 501 of the vegetable room 500 is created and stored in the image storage device 516.

  Thereafter, the control device 507 uses, for example, images such as the images stored in the image storage device 516 based on an instruction from the user input to the operation device 506, for example, an image as illustrated in FIGS. The display is performed on the display device 550 (S207). Here, as described above, the shooting date and time is associated with the image stored in the image storage device 516. For this reason, the control device 507 can perform the display as shown in FIG. 15 by causing the display device 550 to display the shooting date and time together with the image. Further, the control device 507 can also determine the elapsed time after storing the storage as shown in FIG. 16 by comparing images in the same shooting range with different shooting dates and times, as shown in FIG. Alert display can also be performed.

  As described above, even if the network system 700 is configured as in the second embodiment, the mobile terminal 18 that is the operation display device can perform the same image display as in the first embodiment, and is the same as in the first embodiment. An effect can be obtained.

  DESCRIPTION OF SYMBOLS 1 Refrigerator, 2 Compressor, 3 Cooler, 4 Air blower, 5 Air path, 6 Operation panel, 6a Operation switch, 6b Display unit, 7 Control device, 7a Storage device, 7b Timing device, 8 Door, 9 Door open / closed detection Switch, 10 (10a, 10b, 10c) Door pocket, 11 (11a, 11b, 11c, 11d) Shelf, 12 (12a, 12b, 12c, 12d, 12e, 12f) In-camera, 13 (13a, 13b, 13c) , 13d) Illumination unit, 14 water supply tank, 15 communication means, 16 electric communication line, 18 mobile terminal, 18a tablet terminal, 18b mobile phone, 30 drawer detection device, 70 direction indication display, 80 refrigerator icon, 81 cake (containment) Example), 82 beer (example of stored items), 83 image retroactive display, 100 refrigerator compartment, 200 switching room, 00 ice making room, 400 freezing room, 500 vegetable room, 600 chilled room, 601 chilled case, 201, 401, 501 storage case, 402, 502 upper tray, 506 operation device, 507 control device, 515 communication device, 516 image storage device 550 Display device, 700 Network system.

Claims (3)

A plurality of storage rooms partitioned into at least one imaging range;
A plurality of internal cameras that capture at least one of the imaging ranges; or at least one internal camera that captures a plurality of the imaging ranges;
A controller that stores a plurality of images in the shooting range in a storage device and communicates with an operation display device that displays the images;
A timing device that measures the date and time;
With
The controller is
Save multiple images in association with the shooting date and time,
The difference between these images is extracted from the images in the same shooting range with different shooting dates and times, and if there is a portion where there is no difference for a certain period, it is notified that the food is stored for a certain period or more. refrigerator. The controller is
2. The refrigerator according to claim 1, wherein when there is a portion having no difference for a certain period, an image indicating the portion is displayed on the operation display device. A plurality of storage rooms partitioned into at least one imaging range;
A plurality of internal cameras that capture at least one of the imaging ranges; or at least one internal camera that captures a plurality of the imaging ranges;
A controller that stores a plurality of images in the shooting range in a storage device and communicates with an operation display device that displays the images;
A timing device that measures the date and time;
With
The controller is
Save multiple images in association with the shooting date and time,
In the operation display device, when a specific part of the displayed image is selected,
The difference between these images is extracted from the images with different shooting dates and times that include the specific location, and the food stored at the specific location is taken out during the period when the food at the specific location is stored. A refrigerator that displays the date and time.