JP2021050909A - Notification method, system - Google Patents

Abstract

To provide a refrigerator capable of displaying a vacant space in a refrigerator.SOLUTION: A refrigerator includes: an imaging device for imaging an inside of a storage chamber; image storage means for storing an image imaged by the imaging device and a reference image; image analysis means for comparing and analyzing the image imaged by the imaging device and the reference image; calculation means for calculating a storage situation of the storage chamber on the basis of the analysis result of the image analysis means; and display means. A display part displays vacant capacity or a vacant space of the storage chamber on the basis of the calculation result of the calculation means. Thereby, a user can be accurately aware of the vacant capacity or vacant space of the storage chamber via the display part, and they can purchase food materials according to this vacant space, so that it is possible to prevent excessive purchase of food materials and excessive packing of the food materials. Therefore, deterioration in cooling efficiency due to excessive packing of food materials can be prevented.SELECTED DRAWING: Figure 33

Description

The present invention relates to a refrigerator and a notification method for notifying a user of the refrigerator of information.

In general, many refrigerators and refrigerated display systems of this type have been proposed. For example, in the refrigerator and the display system in the refrigerator described in Patent Document 1, a small camera is provided in a storage room such as a refrigerator room, and the food storage status in the storage room taken by the small camera can be carried via an Internet line. It is configured so that it can be confirmed by a display terminal such as a telephone.

Further, Patent Document 2 also discloses a refrigerator and a refrigerating display system similar to Patent Document 1, and the one described in Patent Document 1 is a state of foodstuffs in a storage room with a small camera when the refrigerator door is closed. It is configured to shoot.

Further, in the refrigerator and the display system in the refrigerator described in Patent Document 3, in addition to the configuration described in Patent Document 1, a display terminal is further provided on the surface of the refrigerator, and this display terminal is a fixed terminal unit incorporated in the surface of the refrigerator. It consists of a display terminal such as a mobile phone that can be removed from the surface of the refrigerator and carried around, and both terminals can display the food storage status in the storage room taken with a small camera via the Internet line on the display unit, and go out. It is configured so that you can know the food storage status in the storage room without opening the refrigerator door, not to mention when you are at home.

JP-A-2002-236798 JP-A-2007-46833 JP-A-2002-81818

As mentioned above, we have proposed many refrigerators of this type and their refrigerated display systems, that is, refrigerators and their refrigerated display systems that allow the storage status to be checked on a display terminal or the like by photographing the ingredients in the storage room. However, nothing has been put into practical use yet.

Therefore, the inventors have proceeded with studies toward the practical application of the refrigerator and its refrigerated display system as described above, and have conducted practical application verification. As a result, it was found that this kind of refrigerator and its refrigerated display system have many practical problems that must be solved in order to put it into practical use.

One of them is that although the storage status of the stored items is displayed, it is difficult to tell at a glance how much room there is in the storage space, so there is a problem of purchasing more ingredients than the storage space. If you purchase more than this storage space, you will have to stuff the food in the refrigerator, which will reduce the cooling effect.

An object of the present invention is to provide a mechanism capable of reducing a decrease in cooling efficiency of a refrigerator due to overfilling of foodstuffs.

The notification method of the present invention is a notification method for notifying a user of information, which is executed in a system including a refrigerator having at least one storage room and a display terminal capable of communicating with the refrigerator, and is a notification method of the storage room. An imaging device for imaging the inside is provided on the inner surface of the door of the storage chamber, and the notification method causes the display terminal to display the image captured by the imaging device and is based on the image captured by the imaging device. Therefore, it is determined whether or not the foodstuff is overloaded in the area where the cold air outlet is present in the storage chamber, and when it is determined that the foodstuff is overloaded, the area where the cold air outlet is present is determined. The display terminal is characterized in that information prompting the user to disperse the foodstuffs in other areas is displayed on the display terminal.

Further, the notification method of the present invention is a notification method for notifying a user of information, which is executed in a system including a refrigerator provided with at least one storage room and a display terminal capable of communicating with the refrigerator. An imaging device for imaging the inside of the room is provided on the inner surface of the door of the storage room, and the notification method causes the display terminal to display the image captured by the imaging device, and the image captured by the imaging device. Based on the above, it is determined whether or not the foodstuff is overloaded in the area where the cold air outlet is present in the storage chamber, and when it is determined that the foodstuff is overloaded, the cold air outlet is present. The display terminal is characterized in that information for notifying the user that the area is overloaded with foodstuffs is displayed.

Further, the system of the present invention is a system including a refrigerator having at least one storage chamber and a display terminal capable of communicating with the refrigerator, and the imaging device for imaging the inside of the storage chamber is the storage. The system is provided on the inner surface of the door of the room, and the system is based on the first display control means for displaying the image captured by the image pickup device on the display terminal and the image captured by the image pickup device. A means for determining whether or not the foodstuff is overloaded in the area where the cold air outlet exists, and another means for determining whether or not the foodstuff is overloaded, and when it is determined that the foodstuff is overloaded, the foodstuff in the area where the cold air outlet exists. It is characterized by including a second display control means for displaying information on the display terminal that prompts the user to disperse the information in the area.

Further, the system of the present invention is a system including a refrigerator having at least one storage chamber and a display terminal capable of communicating with the refrigerator, and the imaging device for imaging the inside of the storage chamber is the storage. The system is provided on the inner surface of the door of the room, and the system is based on the first display control means for displaying the image captured by the image pickup device on the display terminal and the image captured by the image pickup device. A means for determining whether or not the foodstuff is overloaded in the area where the cold air outlet exists, and when it is determined that the foodstuff is overloaded, the foodstuff is packed in the area where the cold air outlet exists. It is characterized by including a second display control means for displaying the information notifying the user that the amount is too much on the display terminal.

INDUSTRIAL APPLICABILITY The present invention can provide a mechanism capable of reducing a decrease in cooling efficiency of a refrigerator due to overfilling of foodstuffs.

Perspective view of the refrigerator according to the embodiment of the present invention Schematic cross-sectional view of the refrigerator A perspective view showing the inside of the refrigerator when the storage room door is opened. (A) A perspective view in which the refrigerator is integrally equipped with a display terminal, and (b) a perspective view in which the refrigerator is detachably equipped with a display terminal. An example of mounting configuration of the display terminal used in the refrigerator is shown, (a) is a sectional view when the display terminal is integrally attached, and (b) is a sectional view when the display terminal is detachably attached. An example of the display visual recognition configuration of the display terminal used in the refrigerator is shown, (a) and (b) are schematic cross-sectional views when the outer panel of the door is made of glass, and (c) is composed of the outer panel of the door with a magic mirror. Schematic cross-sectional view Vertical cross-sectional view showing a camera installation example of the refrigerator An example of mounting a camera on the refrigerator is shown, (a) is a cross-sectional view of the refrigerator door with the door closed, and (b) is a front view of the inner surface of the door when the door is opened. (A) (b) (c) Vertical cross-sectional view showing another example of camera mounting of the refrigerator. Vertical cross-sectional view showing an example of camera installation in the case of a single door of the refrigerator An example of mounting the camera of the refrigerator is shown, (a) is a cross-sectional view of the refrigerator door portion with the door closed, and (b) is a front view of the inner surface side of the door when the door is opened. Vertical cross-sectional view showing another example of camera mounting in the case of a single door of the refrigerator Explanatory drawing which shows the shooting state in the camera mounting example of FIG. Explanatory drawing showing an example of a camera attached to the refrigerator System configuration diagram of the display system in the refrigerator in the same embodiment Block diagram showing the refrigerator configuration of the display system in the refrigerator Block diagram showing the server configuration of the display system in the refrigerator Flow chart showing shooting-data storage operation when the door of the display system in the refrigerator is closed Flow chart showing shooting-data storage operation during cooling operation of the refrigerator display system Flow chart showing shooting-data storage operation when the door of the display system in the refrigerator is opened A flowchart showing another operation example of the shooting operation in FIGS. 15 to 20. A flowchart showing the case where the server device performs data storage and synthesis operations in the display system in the refrigerator. A flowchart showing the case where data storage and composition operations in the refrigerator display system are performed on the refrigerator side. A flowchart showing the case where data storage and composition operations in the display system in the refrigerator are performed on the display terminal side. Flow chart showing normal image data transmission / reception in the refrigerator display system A flowchart showing transmission / reception of image data when the image data is stored in the display terminal in the display system in the refrigerator. A flowchart showing a case where an individual image and a composite image are switched and transmitted / received in the same refrigerator display system. Flow chart showing image data transmission / reception at the time of advance transmission request in the same refrigerator display system Front view of a display terminal for explaining switching of transmission / reception routes in the display system in the refrigerator. Flow chart of image display in the refrigerator in the same refrigerator display system Flowchart of continuous playback in the refrigerator in the same refrigerator display system Flowchart for creating an image memo in the refrigerator in the same refrigerator display system Block diagram of the refrigerator availability display control unit in the refrigerator display system Flow chart showing refrigerator availability display control in the refrigerator display system Flow chart showing an example of refrigerator availability display in the refrigerator display system The figure for demonstrating the image composition processing in the display system in the refrigerator.

Hereinafter, specific and detailed embodiments of the present invention will be described, but first, the main points of the embodiments of the present invention will be described.

An embodiment of the refrigerator in the present invention includes a refrigerator main body composed of an inner box and an outer box and having an open front surface, a storage chamber formed by partitioning the refrigerator main body, and a heat insulating door covering the storage chamber. An image pickup device provided in the storage chamber or the heat insulating door to image the inside of the storage chamber, an image storage means for storing an image captured by the image pickup device and a reference image, and an image captured by the image pickup device. The display means includes an image analysis means for comparing and analyzing the image and the reference image, a calculation means for calculating the storage status of the storage room based on the analysis result of the image analysis means, and a display unit, and the display means performs the calculation. The empty capacity or the empty space of the storage chamber is displayed based on the calculation result of the means, and the display means is provided in either the refrigerator main body or the portable display terminal.

As a result, the user can accurately know the empty capacity or empty space of the storage room through the display unit, and can purchase the ingredients according to the empty capacity or the empty space. It is possible to prevent the storage room from being overloaded. Therefore, it is possible to prevent a decrease in cooling efficiency due to over-packing of ingredients.

Further, the image storage means shall store time information together with the image captured by the image pickup device, and the display means displays the images captured by the image pickup device in time series based on the time information. It is a configuration.

This makes it possible to know changes in the storage state of ingredients in the refrigerator in chronological order, and grasp the tendency of how to use the refrigerator of yourself and your family, for example, the tendency that one corner of the storage room is always empty all day and night. it can. Then, based on this, it is possible to promote more energy-saving usage by devising the food installation position in the storage room. In particular, since the refrigerator is powered on for 24 hours and consumes power, the energy saving effect is large and effective even with a slight ingenuity.

Further, the display means is configured to divide the storage chamber into predetermined areas, and when the empty capacity or the empty space is equal to or more than a predetermined value in each area, the corresponding area is emphasized and displayed.

As a result, it is highlighted that the storage space is surplus for each area, so that the space to be stored can be appropriately recognized without opening the heat insulating door.

Further, the display means is configured to divide the storage chamber into predetermined areas, and when the empty capacity or the empty space is less than a predetermined value in each area, the corresponding area is emphasized and displayed.

As a result, it is also highlighted that there is almost no storage space for each area, so that it is possible to appropriately recognize the space where the foodstuffs are overloaded without opening the heat insulating door.

Further, when there is a cold air discharge port for cooling the storage chamber in an area where the empty capacity or the empty space is less than a predetermined value in each area, the display means is overloaded with information on how to use, for example, the area. Therefore, it is configured to display usage information such as distributing ingredients to other areas.

As a result, the user can appropriately recognize the possibility of a decrease in cooling efficiency due to over-packing of the ingredients, and can obtain an opportunity for improvement in usage such as dispersing the ingredients.

Further, the storage chamber is divided into predetermined areas, and when the empty capacity or the empty space in each area is equal to or more than a predetermined value, the corresponding area is not cooled.

As a result, it is possible to realize cooling according to the stored state, and it is possible to realize further energy saving.

Further, the door opening / closing detecting means for detecting the opening / closing of the heat insulating door is provided, and when the closing of the heat insulating door is detected by the door opening / closing detecting means after a lapse of a predetermined time, an image is taken by the imaging device.

As a result, the number of times of imaging can be reduced only when the heat insulating door is closed, and the burden on the calculation means can be reduced.

Further, a refrigerator body composed of an inner box and an outer box and having an open front surface, a storage room formed by partitioning the refrigerator body, a heat insulating door covering the storage room, and the storage room or the heat insulating door. An image pickup device provided to image the inside of the storage chamber, an image storage means for storing an image captured by the image pickup device and a reference image, and an image captured by the image pickup device and a reference image are compared and analyzed. An image analysis means, a calculation means for calculating the storage state of the storage chamber based on the analysis result of the image analysis means, and a display means provided on any of a display terminal for displaying an image captured by the image pickup device. The display means is a display system in a refrigerator, which displays an empty capacity or an empty space of the storage chamber based on a calculation result of the calculation means.

As a result, the user can accurately know the empty capacity or empty space of the storage room through the display unit, wherever he or she is, for example, during shopping, and can purchase foodstuffs according to this empty capacity or empty space. , It is possible to prevent the food from being purchased too much and being packed in the storage room too much. Therefore, it is possible to prevent a decrease in cooling efficiency due to over-packing of ingredients.

Hereinafter, Embodiment 1 of the present invention will be described with reference to the drawings. The present invention is not limited to the following embodiments.

[Embodiment 1]
<1. Configuration>
1-1. Refrigerator configuration 1-1-1. Configuration of French Door Type Refrigerator First, a configuration example in the case of a French door type refrigerator will be described with reference to FIGS. 1 to 9.

In FIGS. 1 to 3, the refrigerator main body 1 has a metal (for example, iron plate) outer box 2 that opens forward, a hard resin (for example, ABS) inner box 3, and these outer boxes 2 and inner boxes 3. It is made of a heat insulating material 4 such as hard urethane foam filled with foam between the two. The refrigerator body 1 has a plurality of storage chambers formed inside the refrigerator body 1. This storage chamber is located below the refrigerating chamber 5, the switching chamber 6 located below the refrigerating chamber 5, and the ice making chamber 7, the switching chamber 6, and the ice making chamber 7 arranged side by side in the switching chamber 6 from the upper part of the refrigerator main body 1. There is a freezing room 8 and a vegetable room 9 located below the freezing room 8. Further, the front surface of the refrigerator compartment 5 is freely opened and closed by a double door 10, and the front portions of the switching chamber 6, the ice making chamber 7, the freezing chamber 8 and the vegetable compartment 9 are drawer-type doors 11, 12, and 13. , 14 (hereinafter referred to as a pull-out door) allows the door to be opened and closed freely.

There is a cooling chamber 16 on the back surface of the refrigerator body 1, a cooler 17 for generating cold air, a blower fan 18 for supplying cold air to each chamber, and a damper (air volume adjusting means) for adjusting the air volume to the refrigerating chamber. 22 is provided. Further, a compressor 19 is provided at the back of the top surface of the main body 1 of the refrigerator main body 1, and a condenser (not shown), a heat radiating pipe 20 for heat dissipation, a capillary tube 21, and the above-mentioned cooler 17 are sequentially arranged. The refrigerating cycle connected in a ring shape is filled with a refrigerant to perform a cooling operation.

Here, each of the doors 10 to 14 is filled with hard urethane foam to have heat insulating properties like the refrigerator main body 1, and in this refrigerator, among the doors 10 to 14, the storage chamber 5 which is a refrigerating chamber A display terminal 15 as a display means is provided on one front surface of the door 10. The display terminal 15 is formed of a smartphone, tablet PC, or the like capable of communicating with the Internet or the like, which will be described later, and integrally has a display unit such as a liquid crystal panel and a control unit for controlling the display unit, and is shown in FIG. 4A. As shown in the above, the door 10 is integrally attached, or as shown in FIG. 4B, the door 10 is detachably attached and detached, and is equipped by either method.

5 (a) and 5 (b) show an example of mounting configuration of the display terminal 15 on the door 10, FIG. 5 (a) shows the display terminal 15 when the display terminal 15 is integrally attached, and FIG. 5 (b) shows the display terminal 15. The case where it is attached detachably is shown.

In FIGS. 5A and 5B, the door 10 mainly has an outer panel 10a facing the front side of the refrigerator, an inner plate 10b facing the inside of the refrigerator, and an outer panel 10a and an inner plate. It is composed of a heat insulating material 4 filled between 10b and 10b. The outer panel 10a is made of a metal member such as a colored steel plate or a transparent member such as a glass plate, and the inner plate 10b is a synthetic resin member formed by vacuum forming. A raised wall portion 10b-1 is provided around the inner plate 10b so as to have a square shape when viewed from the inside of the refrigerator and project to the inside of the refrigerator. Further, as described above, for example, hard urethane foam is used for the heat insulating material 4.

Further, in the door 10 of FIG. 5A in which the display terminal 15 is integrated, a resin storage frame 23 that continuously covers the heat insulating material 4 is provided on the front side of the door 10. (If the outer panel 10a is a glass plate, it is integrally adhered to the glass plate). The display terminal 15 is fixed to the storage frame 23 via the terminal holder 24. A touch panel 25 is provided on the front side of the display terminal 15. A gap 26 is provided between the display terminal 15 and the touch panel 25 so that the touch panel 25 does not malfunction due to noise from the display terminal 15. The display terminal 15 and the touch panel 25 are fixed by attaching a frame body 27 covering the outer peripheral edge to the surface of the door 10. By providing the storage frame 23 in this way, the heat insulating material 4 does not come into direct contact with the display terminal 15, so that the display terminal 15 is exposed to a high temperature or deformed when the heat insulating material 4 is foamed. It is possible to prevent problems such as On the other hand, in the door 10 of FIG. 5B in which the display terminal 15 is detachable, a recess 28 is formed by providing a storage frame 23 on the front surface of the door 10, and the display terminal 15 is attached to and detached from the recess 28. It is configured to do.

6 (a), (b), and (c) show an example of the display perspective configuration of the door 10 and the display terminal 15, and FIGS. 6 (a) and 6 (b) show a glass plate on the outer panel 10a of the door 10. In the case of 10a-1, FIG. 3C shows a case where the outer panel 10a of the door 10 is composed of the magic mirror 10a-2.

In FIG. 6A, a smoke sheet 29 is provided between the display terminal 15 embedded in the door 10 and the glass plate 10a-1 so that only the portion facing the display terminal 15 is transparent. Even when it is off and looks blackish, the glass plate 10a-1 portion other than the display terminal 15 also looks blackish due to the smoke sheet 29, and the boundary with the display terminal 15 is not conspicuous, so that the appearance can be maintained well. Further, in FIG. 6B, a transparent liquid crystal display unit is used as the display unit of the display terminal 15, and a smoke sheet 29a, which is entirely black, is laid along the back surface of the transparent liquid crystal display unit. Can be good. Further, in FIG. 6 (c), the glass plates 10a-1 of FIGS. 6 (a) and 6 (b) are used as magic mirrors 10a-2, and the smoke sheet 29 is not required. be able to. In each case, an example is shown in which the display terminal 15 is integrally attached to the door 10. Instead of the smoke sheet 29, the back side of the glass plate 10a-1 may be painted so as to look black.

7 and 8 (a) and 8 (b) show installation examples of a camera for shooting inside the refrigerator and an illumination lamp inside the refrigerator, and FIG. 7 is a vertical cross-sectional view of the refrigerator with the door closed, FIG. a) is a cross-sectional view showing the inner surface of the door in the refrigerator with the door closed, and FIG. 8 (b) is a front view of the inner surface of the door when the door is opened.

In FIGS. 7, 8 (a) and 8 (b), the storage chamber 5 of the refrigerator is provided with shelf boards 30-1 to 30-4 in a plurality of upper and lower stages. The shelves 30-1 to 30-4 are made of a transparent material such as a transparent synthetic resin material or glass, except for the surrounding frame.

Reference numeral 31 denotes a storage container provided so as to be freely pulled out below the lowermost shelf board 30-4. Although the bottom shelf plate 30-4 is fixed because it constitutes the upper surface of the storage container 31, the shelf plates 30-1 to 30-3 are convex side walls provided on both side walls of the storage chamber 5, respectively. It is mounted on parts 32-1 to 32-4 and is removable. In particular, among the shelf boards 30-1 to 30-3 excluding the bottom shelf board 30-4, the central shelf board 30-2 is provided with a plurality of corresponding side wall convex portions 32-2 at different heights. The height of the shelves can be adjusted.

33-1 to 33-3 are door pockets provided on the inner surface of the door 10 that opens and closes the opening of the storage chamber 5, and are provided in a plurality of upper and lower stages. The door pockets 33-1 to 33-3 are also made of a transparent material.

34-1 and 34-2 are interior lighting lights, which are provided on both side walls of the front opening and the upper wall surface of the front opening of the storage chamber 5 in front of the front ends of the shelf boards 30-1 to 30-4. .. In order to save energy, the interior lighting lamps 34-1 and 34-2 are provided with a substrate on which a plurality of LEDs are arranged, for example, in the vertical direction and covered with a translucent cover.

The 35-1 and 35-2 are cameras as imaging means provided so as to be located on the front side of the storage chamber 5 like the internal lightings 34-1 and 34-2. For example, a CCD camera, a CMOS camera, or the like. The type is not particularly limited, but it is preferably resistant to dew condensation and the like.

In this embodiment, two cameras are provided, one of which is attached to the lower surface of the upper door pocket 33-1 substantially in the center in the front-rear direction, and the other 35-2 is the lowermost door pocket. It is attached near the upper part of 33-3. Here, the front-rear direction of the door pocket 33 is a direction perpendicular to the surface formed by the inner plate 10b of the door 10, and is the same as the front-rear direction of the storage chamber 5 when the door 10 is closed. is there.

Further, the upper camera 35-1 is provided at a substantially front position of the uppermost shelf board 30-1, and the lower camera 35-2 is provided at a front position of the third shelf board 30-3 in the drawing. That is, each camera is not a shelf board 30-2 mounted on a plurality of side wall convex portions 32-2 provided on the premise of height adjustment, but a shelf that is removable but not premised on height adjustment. It is provided at a position substantially in front of the plate 30-1 and the shelf plate 30-3.

As is clear from FIG. 7, all the cameras 35-1 and 35-2 are located in front of the front end of each shelf board 30-1 to 30-4, and are as shown in FIG. 8 (b). The door 10 is provided so as to be located near the open end side end Y opposite to the shaft support X, that is, substantially in the center of the refrigerator 51 in the left-right direction, and these two cameras 35-1, 35-2 are used for storage. Almost the entire area inside can be photographed. In the case of this French door example, the cameras 35-1 and 35-2 are provided on the door 10 side having a large area and are set so as to be located substantially at the center of the refrigerator 51. Further, in this example, the cameras 35-1 and 35-2 are provided on the door 10 side where the center pillar 36 (see FIG. 3) is not provided, and are set so as to be located substantially at the center of the refrigerator 51. .. As is well known, the center pillar 36 is provided on one side of the two doors 10 in the case of a French door, and the cold air in the storage chamber 5 is discharged from the gap generated between the two doors 10. It is a member provided to prevent leakage.

As for the cameras 35-1 and 35-2, as shown in FIG. 9A, a dedicated camera 35-3 in the storage container is additionally installed, or as shown in FIG. 9B, the lower camera 35 is installed. -2 can be provided so as to be movable up and down, or the lower camera 35-2 can be provided so as to be rotatable in the up-down direction as shown in FIG. Good.

Further, as is clear from FIG. 7, the interior lighting lamps 34-1 and 34-2 are provided at a position on the back side of the storage room from the cameras 35-1 and 35-2, and the interior lighting lamps 34-1 and 34-2 are provided. The light of -2 is provided so as not to directly enter the photographing section of the cameras 35-1 and 35-2.

More specifically, as shown in FIG. 8A, the photographing unit of the cameras 35-1 and 35-2 has an irradiation amount (maximum irradiation amount) in the irradiation direction of the internal lighting lamps 34-1 and 34-2. ), The irradiation amount is halved or more, and in this example, it is provided outside the range of the irradiation angle of 60 degrees.

Although not shown, this is a light source such as an LED that provides a light-shielding portion on the camera side of the interior lighting lamps 34-1 and 34-2 or constitutes the interior lighting lamps 34-1 and 34-2. It may be possible to prevent the light of the interior lighting lamps 34-1 and 34-2 from directly entering the photographing unit of the cameras 35-1 and 35-2 by arranging the lamps so as to be tilted diagonally backward. It may be selected appropriately according to the installation conditions of the camera and the like. This also applies to the single-door type refrigerator described below.

1-1-2. Configuration of Single Door Type Refrigerator FIGS. 10 to 13 show an example of mounting a camera and an interior lighting when the door 10 has a single door. In this single type door refrigerator, the same parts as those of the French door type refrigerator already described will be given the same number and the description will be omitted, and only the different parts will be described.

First, an example thereof will be described with reference to FIGS. 10 and 11 (a) and 11 (b).

In FIGS. 10 and 11 (a) and 11 (b), in the case of this single-door refrigerator, the upper camera 35-1 is a left-right split door pocket 33-, which is a general arrangement example for a single door. It is provided on the raised wall portion 10b-2 existing between 4, 33-5 and 33-6, and the lower camera 35-2 is provided on the inner surface portion of the door between the door pockets 33-7 and 33-8. .. Similar to the French door type, the cameras 35-1 and 35-2 are provided at substantially forward positions of the shelf boards 30-1 and the shelf boards 30-3, which are removable but not premised on height adjustment. Then, as shown in FIG. 10, the position is in front of the front end of each shelf board 30-1 to 30-4, and as can be understood in FIGS. 11A and 11B, the vicinity of the central portion of the door 10, that is, The refrigerator 51 is provided so as to be located at a substantially central portion in the left-right direction so that almost the entire inside of the refrigerator can be photographed by these two cameras 35-1 and 35-2.

Further, the interior lighting lights 34-1 and 34-2 are provided in the same manner as the French door type. That is, the interior lighting lamps 34-1 and 34-2 are provided on both side walls of the front opening of the storage chamber and on the upper wall surface of the front opening in front of the front ends of the shelf boards 30-1 to 30-4. The position where the light does not directly enter the photographing section of the cameras 35-1, 35-2, that is, the photographing section of the cameras 35-1, 35-2 is located in the interior lighting 34-1, 34-2. It is provided outside the range in which the irradiation amount is 1/2 or more with respect to the irradiation amount (maximum irradiation amount) in the irradiation direction, in this example, the irradiation angle of 60 degrees.

12 (a) and 12 (b) and 13 show another example of the single door type.

In the case of this example, the cameras 35-1 and 35-2 are provided near the door open end Y on the side opposite to the shaft support X of the door 10, as shown in FIGS. 12A and 12B. The height positions of the cameras 35-1 and 35-2 are the same as in the case of the single door described above. When the cameras 35-1 and 35-2 are provided near the door open end Y, the image is taken when the door 10 is opened at a predetermined angle, for example, 30 degrees. As a result, the image can be taken in a wide range up to the vicinity of the opening of the storage chamber 5 as shown by the broken line, and the image can be checked by the user by opening the door slightly from time to time. It is substantially the same as the state of viewing when the image is displayed, and when the image is visually recognized on the display terminal 15, the image can be viewed in a natural form without any discomfort.

1-1-3. Camera configuration FIG. 14 shows an example of a camera used for in-house photography. It is most preferable to install the cameras at positions separated from each other in the vertical direction as described in each of the above examples, but even if the upper and lower cameras 35-1 and 35-2 are installed close to each other as shown in FIG. As a result of verification, it was confirmed that the entire area inside can be photographed.

Therefore, in such a case, it is conceivable that the cameras 35-1 and 35-2 are integrated into the camera module 41, and by modularizing in this way, the installation can be facilitated and the cost can be reduced. It can also be modularized by using a wider-angle lens such as a fisheye lens.

1-2. Overall configuration of the refrigerator display system Next, the entire refrigerator display system using the refrigerator as described above will be described with reference to FIGS. 15 to 33.

FIG. 15 shows the overall configuration of the in-refrigerator display system 50 using the above-mentioned refrigerator. As shown in FIG. 15, the display system 50 in the refrigerator is a system for confirming the state of foodstuffs in the refrigerator 51 by a display terminal 15 or the like.

The refrigerator display system 50 includes a refrigerator 51 and a display terminal 15, and further includes a wireless adapter (communication device) 53, a gateway device (relay device) 54, a router device 55, the Internet 56, and a server device 57.

The refrigerator 51 is arranged in the user's residence A, for example, in the kitchen room in the residence A.

The wireless adapter (communication device) 53 is configured to be electrically connected to a control unit described later of the refrigerator 51 and communicate with the router device 55 via the gateway device (relay device) 54. The wireless adapter 53 receives the signal transmitted from the router device 55 and outputs the received signal to the control unit of the refrigerator 51. Based on this signal, the refrigerator 51 is configured to perform the corresponding operation.

The wireless adapter 53 also acquires identification information (for example, serial number, model number, etc.) of the refrigerator 51, image data in the refrigerator, which will be described later, from the storage unit of the refrigerator 51, and a router via a gateway device (relay device) 54. It is configured to transmit to the device 55. The wireless adapter 53 may be integrally provided with the refrigerator 51, or may be detachably configured.

The wireless adapter 53 includes a "connect" button 53a. The "connect" button 53a is for newly connecting to the gateway device (relay device) 54. When the "connect" button 53a is operated by the user, the wireless adapter 53 is configured to acquire identification information (for example, serial number or model number) of the refrigerator 51 from the storage unit of the refrigerator 51.

As described above, the display terminal 15 is, for example, a general-purpose portable terminal such as a smartphone or a tablet PC (personal computer), and in the case of the present embodiment, the display terminal 15 can be attached to and detached from the door 10 of the refrigerator 51 described above. It has a function (means) for connecting to the Internet 56 and a function (means) for communicating with the gateway device 54, which will be described later.

The display terminal 15 (for example, a smartphone) is configured to connect to the Internet 56 via a telephone line network (for example, a 3G line network). In addition to or as an alternative, for example, the display terminal 15 is configured to connect to the Internet 56 via a public wireless LAN by means of a Wi-Fi communication function.

By connecting to the Internet 56, for example, the display terminal 15 is a program for acquiring image data of foodstuffs in a refrigerator, which is the subject of the present invention, and data for performing energy-saving operation (for example, data such as the number of times the door is opened and closed). Can be obtained from the homepage of the manufacturer of the refrigerator 51 via the Internet 56. By installing the acquired program on the display terminal 15 and activating the installed program, the display terminal 15 can acquire image data and the like of foodstuffs in the refrigerator. That is, the display terminal 15 can generate and transmit an operation signal for acquiring the image data of the refrigerator 51 by operating the operation screen of the display means 15a such as the display. Details will be described later.

The display terminal 15 is also connected to the router device 55 by general-purpose communication not via the Internet 56 such as Wi-Fi communication, Bluetooth (registered trademark) communication, infrared communication, etc., and the gateway device 54 is connected via the router device 55. It is configured to make a communication connection to. A device for that purpose (for example, a Wi-Fi antenna or the like) is incorporated in the display terminal 15.

The gateway device (relay device) 54 is a device that relays communication between the wireless adapter 53 and the display terminal 15, and is, for example, a wireless adapter 53 and a specific low power wireless special small frequency band (924.0 to 928.0 MHz). ) Is used for communication. The frequency band for communication between the gateway device 54 and the wireless adapter 53 is preferably a low frequency band that reaches a long distance, and is installed in the user's residence A in which the refrigerator 51 is installed together with the router device 55.

The gateway device 54 is also configured to communicate and connect to the display terminal 15 via the router device 55. That is, the gateway device 54 is configured so that communication (first communication) can be executed only through the router device 55 without going through the display terminal 15 and the Internet 56.

The gateway device 54 is further configured to be able to connect to the Internet 56 via the router device 55. As a result, the gateway device 54 can communicate with the display terminal 15 (second communication) and also communicate with the server device 57 via the Internet 56 and the router device 55.

Therefore, the gateway device 54 communicates with the display terminal 15 via the Internet 56 (that is, the first communication only via the router device 55) and the communication via the Internet 56 (that is, via the Internet 56 and the router device 55). It is configured so that the second communication) can be executed. The reason for this configuration will be described later.

The server device 57 is prepared, for example, by the manufacturer of the refrigerator 51. The server device 57 manages information necessary for access and authentication regarding the display terminal 15, the refrigerator 51, and the wireless adapter 53. For example, the server device 57 determines whether or not the identification information of the display terminal 15 included in the communication request signal matches the identification information of the display terminal 15 stored (registered) in association with the wireless adapter 53. Authenticate (determine). Then, when these match, the server device 57 authenticates the display terminal 15 to be accessed as a legitimate display terminal, and permits communication with the refrigerator 51 in the residence A.

In addition, the server device 57 stores the above-mentioned image data of foodstuffs in the refrigerator transmitted from the refrigerator 51, data for performing energy-saving operation, and the like. These data are transmitted to the display terminal 15 based on the data call signal from the display terminal 15. That is, the user can display and confirm the food material image data in the refrigerator on the display by operating the display terminal 15.

1-3. System section configuration of the refrigerator FIG. 16 shows the system section configuration of the refrigerator used in the refrigeration system.

In FIG. 16, the refrigerator 51 includes an interface (I / F) 58, a control calculation unit 59, a storage unit 60, a door open / close detection unit 61, interior lighting 34-1, 34-2, and a camera 35-1. It includes 35-2, a fan drive circuit 62, a compressor drive circuit 63, a blower fan 18, and a compressor 19, and receives power from an AC power source 64 such as a commercial power source in the residence A, for example.

The interface (I / F) 58 exchanges data and the like between the wireless adapter 53 and the control calculation unit 59 of the refrigerator 51.

The control calculation unit 59 controls the operation of the refrigerator. Further, the control calculation unit 59 controls the cameras 35-1 and 35-2 and the lighting lamps 34-1 and 34-2 in the refrigerator to photograph the foodstuffs in the refrigerator, and temporarily stores the image data in the storage unit 60. It is stored and transmitted to the server device 57 of the Internet 56. Further, if there is a request signal for prior transmission of image data or the like from the display terminal 15 via the server device 57, the display terminal 15 is instructed to immediately transmit the signal to the display terminal 15 via the server device 57 after photographing the foodstuff. The details will be described later. Furthermore, the control calculation unit 59 determines whether or not there is any change by comparing the photographed image of the food material specified by the object specifying means with the image stored in the storage unit 60, which will be described later, and determines the determination. Based on the result, the storage period stored in the storage unit 60 is changed and corrected.

In addition to the identification code and the control program, the storage unit 60 rewrites and stores the image data of the foodstuffs in the refrigerator taken by the cameras 35-1 and 35-2 and the data for performing the energy-saving operation together with the time information for a certain period of time. Further, if there is a food material specified by the above-mentioned target object specifying means food material, the storage period of the food material is stored.

The lighting and extinguishing of the interior lighting lights 34-1 and 34-2 are controlled by the control calculation unit 59 based on the signal from the door open / close detection unit 61, and are turned on when the door 10 is opened and when the door 10 is closed. Turns off after a delay of a few seconds.

The cameras 35-1 and 35-2 are controlled by the control calculation unit 59 based on the signal from the door open / close detection unit 61, and when the door 10 is closed, the cameras take pictures of the foodstuffs in the refrigerator. The details will be described later.

The fan drive circuit 62 receives a control signal from the control calculation unit 59 and controls the drive of the rotation of the blower fan 18.

The compressor drive circuit 63 receives a control signal from the control calculation unit 59 and controls the drive of the compressor 19 such as the rotation speed.

The blower fan 18 operates under the control of the fan drive circuit 62 to generate an air flow for circulating cold air.

The compressor 19 compresses the refrigerant (not shown) circulating in the refrigerator 51 under the control of the compressor drive circuit 63.

1-4. Configuration of Server Device Next, a configuration example of the server device of the refrigerator display system according to the first embodiment will be described with reference to FIG. Here, the configuration of the server device 57 in FIG. 15 will be taken as an example.

As shown in FIG. 17, the server device 57 includes a server interface 65, a server control unit 66, and a data storage unit 67.

The server interface 65 is a communication unit that exchanges data and the like between the display terminal 15 and the router device 55 and the server device 57 via the Internet 56 under the control of the server control unit 66.

The server control unit 66 controls the overall operation of the server device 57. The server control unit 66 also combines and processes the image data of the foodstuffs transmitted from the refrigerator 51, stores it in the data storage unit 67 together with the time information, and receives a signal of a pre-transmission request for the image data from the display terminal 15. The latest image data of the image data stored in the data storage unit 67 is transmitted to the display terminal 15 each time the image data is transmitted from the refrigerator 51, and the display terminal 15 stores and rewrites the image data together with the time information for a certain period of time. Let me update. Of course, when a data call signal is transmitted from the display terminal 15 without a signal for requesting advance transmission of image data, the latest image data stored in the data storage unit 67 is transmitted to the display terminal 15.

The data storage unit 67 stores necessary data such as a management program and an application program for executing the refrigerator system. Further, the data storage unit 67 rewrites and stores the latest data of the food material image data transmitted from the refrigerator 51 by adding time information under the control of the server control unit 66. The data storage unit 67 is composed of, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like.

The server device 57 can be constructed by either a server composed of a single large computer or a cloud-type server composed of a large number of computers.

<2. Operation>
Next, the operation of the refrigerator and the display system in the refrigerator having the above configuration will be described. In each of the following explanations, the same operation will be described with the same step number.

2-1. Shooting operation of ingredients 2-1-1. Taking a picture when the door is closed FIG. 18 is a flowchart showing an operation when the food material in the storage room is taken when the door 10 of the refrigerator 51 is closed.

When the door 10 is opened to put in and take out food (Y in step 1), the interior lighting lights 34-1 and 34-2 are turned on (step 2), and when the door 10 is closed (Y in step 3), Reduce the illuminance of the interior lighting lights 34-1 and 34-2 (step 4), activate the cameras 35-1 and 35-2 for a while, for example, after a few seconds, and take a picture of the ingredients in the refrigerator (step 5). .. After that, the captured image data is stored in the storage unit 60 of the refrigerator 51 together with the time information (step 6), and the lighting lamps 34-1 and 34-2 in the refrigerator are turned off (step 7). The time information indicating the imaging date and time is acquired from the refrigerator itself or the server via communication.

Here, it is preferable to reduce the illuminance of the internal lighting lamps 34-1 and 34-2 during the above shooting, because the foodstuffs can be clearly photographed. That is, when the foodstuffs are photographed by turning on the lighting lamps 34-1 and 34-2 in the refrigerator, the reflected light is transmitted to the foodstuffs from the entire circumference except the opening portion of the storage chamber because the storage chamber surface is white. It reflects strongly (it bounces off the food with high illuminance without decreasing the illuminance), and the food becomes whitish, making it difficult to take clear pictures. However, if the illuminance of the interior lighting lamps 34-1 and 34-2 is reduced as in the above example, the influence of the reflected light from the storage interior surface can be significantly reduced, and a clear image can be taken immediately. It will be possible.

2-1-2. Photographing during cooling room cooling operation FIG. 19 is a flowchart showing an operation when the foodstuffs in the refrigerator are taken in and out and then the foodstuffs in the storage room are photographed when the refrigerating room is cooled.

When the door 10 is opened to put in and take out the ingredients (Y in step 1), the interior lights 34-1 and 34-2 are turned on (step 2), the door 10 is closed (Y in step 3), and the refrigerator compartment is used. It is determined whether or not it is in the cooling operation state (step 8). If the blower fan 18 is ON and the damper 22 is open and the refrigerating room is being cooled, the illuminance of the interior lighting lamps 34-1 and 34-2 is reduced as in the case of closing the door (step 4). ), The cameras 35-1 and 35-2 are activated to take a picture of the ingredients in the refrigerator (step 5). After that, the captured image data is stored in the storage unit 60 of the refrigerator 51 together with the time information (step 6), and the lighting lamps 34-1 and 34-2 in the refrigerator are turned off (step 7).

In this way, by taking pictures during the cooling operation of the refrigerating room, for example, even if dew condensation occurs on the surface of the foodstuff newly put in the storage room due to the temperature difference between the inside and outside of the storage room, this dew condensation is eliminated by the cooling operation. , Will disappear. Therefore, the images taken by the cameras 35-1 and 35-2 are preferable because the characters written on the food packaging container can be read clearly.

In the above operation, when the door 10 is closed (Y in step 1), the internal lighting lights 34-1 and 34-2 are turned off once, and the internal lighting lights 34-1 and 34 are turned off again before shooting. -If the lights are turned on and the illuminance thereof is lowered, the wasteful power consumption due to the continuous lighting of the internal lighting lamps 34-1 and 34-2 can be suppressed.

2-1-3. Shooting at the time of opening the door FIG. 20 is a flowchart showing an operation in the case of a single door, in which the foodstuffs in the storage room are photographed when the door 10 is opened at a predetermined angle.

When the door 10 is opened to put in and take out food (Y in step 1), the interior lighting lights 34-1 and 34-2 are turned on (step 2), and the opening angle of the door is a predetermined angle, for example, FIG. When the opening angle of 30 degrees is reached (Y in step 10), the illuminance of the interior lighting lights 34-1 and 34-2 is reduced (step 4), and the cameras 35-1 and 35-2 are activated to activate the refrigerator. Take a picture of the ingredients inside (step 5). After that, the captured image data is stored in the storage unit 60 of the refrigerator 51 together with the time information (step 6), and when the door is closed (Y in step 11), the interior lighting lights 34-1 and 34-2 are turned off (step 6). Step 7).

If the door 10 is photographed when the door 10 is opened at a predetermined angle as in this example, the storage room can be photographed in a wide range as described above, and the image becomes natural without any discomfort, which is preferable. is there.

When taking a picture when the predetermined angle is opened, the picture is taken at least twice when the door is opened and when the door is closed. At this time, the latest storage room in which the foodstuffs have been taken in and out by storing the last captured image data (during the door closing operation) in the storage unit 60 during the operation time from one door opening to closing. You can take a picture of the food condition of the door.

Needless to say, in step 10 of detecting whether or not the opening angle of the door 10 reaches a predetermined angle, if the door 10 is closed before reaching the predetermined angle, the interior lighting 34-1 , 34-2 are turned off, and the door 10 is waited to be opened again.

As described above, a typical shooting operation has been described as an example. However, in addition to the shooting operation, for example, shooting is performed at a scheduled time or when a predetermined time elapses after the previous shooting, or when a shooting instruction is given from the display terminal 15, shooting is performed. Shooting operations such as

2-1-4. Illumination at the time of shooting As described in FIGS. 18, 19, and 20, the foodstuff lighting at the time of shooting is performed by turning on the internal lighting lamps 34-1 and 34-2 and then reducing the illuminance. Irradiate and shoot. In some cases, it is conceivable to reduce the illuminance and take a picture by lighting only the interior lighting 34-1 on both sides of the storage room or the interior lighting 34-2 on the upper surface of the storage room. The reason for lowering the illuminance for shooting is as described above.

In addition, as another example of irradiating foodstuffs at the time of shooting, the following can be considered. That is, FIG. 21 is a flowchart showing a state in which the interior lighting lamps 34-1 and 34-2 are turned off and off at the time of shooting by the cameras 35-1 and 35-2. It corresponds to the operation of the broken line part.

The photographs in FIGS. 18, 19, and 20 described above are examples in which the interior lighting lamps 34-1 and 34-2 are turned on and the illuminance is reduced, but in this example, the storage room is used. Turn off one of the interior lighting lights 34-1 on both sides, in other words, make sure that only one of the interior lighting lights 34-1 on both sides of the storage room is lit, and illuminate the food to take a picture. is there. In this case, the interior lighting lamp 34-2 on the upper surface of the storage chamber may be turned on or off, but either one may be appropriately selected so that the illuminance in the refrigerator is optimal for shooting.

In FIG. 21, one of the interior lighting lights 34-1 on both sides of the storage room that is already lit, for example, the right interior lighting lamp is turned off before shooting is started by the cameras 35-1 and 35-2. (Step 12), the foodstuff in the state of being illuminated by the left chamber lighting is photographed (step 13) and stored (step 14). After that, the right internal lighting is turned on (step 15), then the left internal lighting is turned off (step 16), and the foodstuff in the state of being illuminated by the right internal lighting is photographed (step 17). ). The image data is stored (step 18), and the right internal lighting is turned off (step 19). The image data stored by irradiating the food material from one side in this way is combined into one image and stored together with the time information.

As described above, the reason why the interior lighting lamps 34-1 and 34-2 are turned on one by one to photograph the foodstuffs and combine them into one is as follows. That is, as already described, the illuminance of the internal lighting lamps 34-1 and 34-2 is reduced during shooting, but if the illuminance reduction is too large, the image becomes dark and lacks clarity. Therefore, there is a limit to the decrease in illuminance. Therefore, the illuminance is maintained to some extent, but even with that illuminance, the influence of the reflected light may remain when the volume of the storage room is small. However, as described in this example, if one of the left and right sides of the interior lighting lamp 34-1 is turned on to take a picture of the food, the influence of the reflected light can be reduced and the interior lighting lamp 34-1 is turned on. The surface on the side of the screen can be photographed with moderate brightness. Therefore, it is preferable to perform a compositing process of combining the left and right images of the foodstuff taken with this appropriate brightness to obtain a clear image as a whole.

2-2. Saving and synthesizing images Next, saving and synthesizing image data captured by each of the above operations will be described.

2-2-1. Saving and Compositing on the Server FIG. 22 shows a case where image data is saved and synthesized on the server device 57, and when the inside of the refrigerator is photographed by the operations shown in FIGS. 18 to 20 including FIG. 21, the refrigerator 51 is a wireless adapter. The 53 connects to the Internet 56 via the gateway device 54 and the router device 55 (step 20), and the individual image data captured and stored by each camera is transmitted to the server device 57 connected to the Internet 56 (step 21). .. The image data transmitted via the Internet 56 is subjected to a synthesis process (step 22) in which individual image data are combined by the server control unit 66 of the server device 57. The composited image data is saved by adding the current time information acquired from the server to the data storage unit 67 of the server device 57 as the shooting time (step 23).

If the image is synthesized by the server device 57 and stored in storage as in this example, the load on the control calculation unit 59 and the storage unit 60 on the refrigerator side can be reduced, and the Internet-compatible refrigerator can be inexpensive. Can be provided to.

2-2-2. Storage and composition on the refrigerator side FIG. 23 shows a case where image data is stored and combined on the refrigerator side, and when the inside of the refrigerator is photographed by the operations shown in FIGS. The unit 59 takes out individual image data from each camera stored in the storage unit 60, combines them, and synthesizes them (step 24). The combined image data was stored (step 25), the refrigerator 51 was connected to the Internet 56 from the wireless adapter 53 via the gateway device 54 and the router device 55 (step 20), and the stored image data was connected to the Internet 56. It is transmitted to the server device 57 (step 21). The composite image data transmitted via the Internet 56 is stored in the data storage unit 67 of the server device 57 by adding the current time information acquired from the server as the shooting time (step 23).

In this example, the load on the control calculation unit 59 and the storage unit 60 on the refrigerator side becomes large, but the image data can be transmitted and received to and from the display terminal 15 without going through the data storage unit 67 of the server device 57, which reduces the system operating cost. It can be cheap.

2-2-3. Saving and combining on the display terminal side FIG. 24 shows a case where image data is saved and combined on the display terminal 15 side, and when the inside of the refrigerator is photographed by the operations shown in FIGS. 18 to 20 including FIG. 21, the refrigerator 51 Connects to the Internet 56 from the wireless adapter 53 via the gateway device 54 and the router device 55 (step 20), and transmits image data to the display terminal 15 instead of the server (step 26). The image data transmitted via the Internet 56 is subjected to a synthesis process (step 27) of combining the image data by the display terminal 15. The composited image data is stored in the data storage unit of the display terminal 15 together with the time information held by the display terminal itself (step 28).

In the case of this example as well, although the load on the display terminal 15 increases, the load on the control calculation unit 59 and the storage unit 60 on the refrigerator side can be reduced, and an Internet-compatible refrigerator can be provided at low cost.

Although typical image data transmission and composition operations have been described above by way of example, the storage and composition locations do not necessarily have to be performed at the same location, for example, the server device 57. For example, the refrigerator 51. The image data may be combined with the above, and the image data may be saved by the server device 57, etc., which may be arbitrarily set according to the system.

2-3. Normal transmission / reception of image data 2-3-1. Transmission / reception of image data stored in the server / refrigerator FIG. 25 is a flowchart showing transmission / reception between the server device 57 or the refrigerator 51 and the display terminal 15, and shows transmission / reception of image data stored in the server device 57 or the refrigerator 51. is there.

First, the user presses a server connection button (not shown) displayed on the display of the display terminal 15 at a shopping destination, for example, and requests to read the image data stored in the data storage unit 67 of the server device 57 (not shown). Step 31). It is detected whether the image data is taken in from the server device 57 (step 32), and if the image data is not transferred, the display terminal 15 connects to the Internet 56 (step 33). The image data stored in the data storage unit 67 of the server device 57 or the storage unit 60 of the refrigerator is taken out (step 34), and the image data is transferred to the display terminal 15 such as a smartphone, which is the data caller, via the Internet 56. Is transferred (step 35). It detects whether the image data has been captured from the server device 57 (step 32), and when the image data is captured, outputs the image data to the display of the display terminal 15 (step 36).
After that, it is detected whether the enlargement button (not shown) displayed on the display of the display terminal 15 is pressed (step 37), and when it is detected that the enlargement button is pressed, the inside of the storage room output to the display is detected. Create image data by enlarging the image data by digital conversion 1.1 times or more. The enlarged and converted image data is output to the display of the display terminal 15 (step 36).

After that, it is detected whether the numeric keypad button (not shown) for local display displayed on the display is pressed (step 38), and when it is detected that the numeric keypad button is pressed, the inside of the storage room is output to the display. The enlarged image data is set as the image data of the portion designated by the numeric keypad and output to the display (step 36). Further, it is detected whether the numeric keypad button is pressed (step 38). If it is not detected, it ends.

The enlargement button and the numeric keypad button for displaying the image data at a predetermined location may be of a method of sliding the surface of the display with a finger, and such a configuration improves usability.

As described above, according to this refrigerator system, the contents of the refrigerator 51 can be checked at a glance at the shopping destination, it is possible to prevent forgetting to buy and to eliminate the waste of purchasing extra ingredients. .. Further, in a double-income household, it is possible to check the inside of the storage room 5 of the refrigerator 51 just before returning home and purchase foodstuffs or the like on the way home, which can shorten the time for the user.

In addition, by enlarging the display and displaying a predetermined location, the miniaturized display terminal 15 such as the display terminal 15 can be enlarged to display the image, so that the foodstuffs that are difficult to check on the small display terminal 15 can be surely displayed. You can see it.

2-3-2. Transmission / reception of image data saved in the display terminal FIG. 26 shows a case where the image data is already stored and saved in the display terminal 15. In this case, as described in FIG. 24, the image data is transmitted and stored in the display terminal 15, and when the reading request of the image data is made (step 31), the image data is taken in from the storage unit of the display terminal 15. Outputs image data to the display of the display terminal 15 (step 36). Subsequent enlarged display operations and local displays are as described with reference to FIG.

2-3-3. Switching of image data The images taken by the receiving cameras 35-1, 35-2 are the images taken by each camera 35-1, 35-2 (upper half or lower half of the storage room) and each of these shot images. An example is shown in which composite images (combined images of the storage chamber from top to bottom) are stored and stored, and these images can be switched and transmitted / received and displayed on the display terminal 15.

FIG. 27 is a flowchart showing the operation in such a case.

First, the user presses the individual image / composite image call button (not shown) displayed on the display of the display terminal 15 (step 41), and calls either the individual image or the composite image (step 42). ). After that, the image data of either the individual image or the composite image is taken in by the flow shown in FIG. 25 or FIG. 26 and displayed on the display of the display terminal 15.

When the image to be displayed on the display terminal 15 can be switched to either an individual image or a composite image as in this example, usually, the composite image is called in and displayed to check the food condition in the storage room at a glance. However, when it is difficult to distinguish the foodstuffs due to the fact that they are in the corner of the storage room, etc., it is possible to distinguish them reliably by switching to individual images and displaying them, which improves usability.

2-4. Transmission / reception of image data or the like when a pre-transmission request is made FIG. 28 shows a flowchart when an image pre-transmission request is made from the display terminal 15.

First, the user operates the display terminal 15 to request the pre-request signal (step 51). The server device 57 or the refrigerator 51 receives this request signal (step 52), and reads and transmits the image data (individual image data or composite image data, the same applies hereinafter) stored in the data storage unit 67 or the storage unit 60 (step 52). 53). The display terminal 15 detects whether the image data from the server device 57 or the refrigerator 51 has been taken in (step 54), and if the image data is not transmitted, the display terminal 15 connects to the Internet 56 (step 55). The server devices 57 take out the image data stored in the refrigerator 51 (step 56), and transmit the image data to the display terminal 15 via the Internet 56 (step 57). It detects whether the image data has been fetched from the server device 57 (step 54), and when the image data is detected, waits until the image data is called (N in step 58). If the image data is called (Y in step 58), the image data is output to the display of the display terminal 15 (step 59).

As a result, when the display terminal 15 requests the image data, even if the access to the server device 57 is concentrated and the transmission of the image data from the server device 57 is delayed, the image data is requested in real time at the same time as the image data request. Can be displayed on the display to check the image. Therefore, it is possible to avoid the situation of waiting for the image data to be displayed at the shopping destination, and it is possible to shorten the shopping time.

Further, if there is a pre-transmission request for an image from the display terminal 15, the server device 57 is set to transmit the image data to be transmitted each time the door 10 of the refrigerator 51 is closed, and the display terminal 15 is set to transmit the image data. If the user who has 15 is set to notify when the image data is transmitted, he / she can know this every time the door is opened / closed. Therefore, for example, when the number of times the door is opened and closed is large and the power consumption is likely to increase, the user can call the home and talk to the child or the like to refrain from using the refrigerator 51. Convenience is also improved.

2-5. Switching the transmission / reception route of image data or the like FIG. 29 shows a display display when the communication system path of the display terminal 15 is selected. The flow of sending and receiving image data is as described above, but the communication route is changed to speed up the acquisition of image data.

Specifically, the user can select whether to perform the display terminal 15 directly with the gateway device 54 via the router device 55 or with the server device 57 via the Internet 56.

In FIG. 29, 70 and 71 are communication path selection buttons displayed on the display of the display terminal 15, and 70 is a “direct” selection that directly connects the display terminal 15 to the refrigerator 51 via the router device 55 and the gateway device 54. The button 71 is a "server" selection button that further connects to the server device 57 or the refrigerator 51 via the Internet 56. 72 and 73 are data selection buttons for the data to be called, 72 is the image data of the food material, and 73 is the button for energy saving related data such as the number of times the door is opened and closed. Reference numeral 74 is a temperature measurement button for measuring the temperature of the food material and instructing the notification, and 75 is a transmission button.

When the "direct" selection button 70 selects this, the display terminal 15 is set to "direct" communication. When the user touches the transmit button 74 while the "direct" communication is set, the image data request signal of the display terminal 15 is sent directly from the display terminal 15 to the gateway device (relay device) 54 via the router device 55. Will be sent. The gateway device 54 transmits the received data request signal to the wireless adapter 53 of the refrigerator 51. When the wireless adapter 53 receives the image data request signal, it transmits it to the control calculation unit 59 of the refrigerator 51. When the control calculation unit 59 of the refrigerator 51 receives the image data request signal, it reads the image data stored in the storage unit 60 and transmits the image data to the display terminal 15 via the wireless adapter 53 and the router device 55.

Therefore, as shown by the broken line in FIG. 15, when the user is located in the residence A, in a broad sense, when the display terminal 15 is located at a position where it can directly communicate with the router device 55, the display terminal 15 is the router device 55. It communicates with the gateway device 54 and can directly acquire image data from the storage unit 60 of the refrigerator 51.

As a result, when the transmission response of the image data to the operation of the user to the display terminal 15 is delayed due to the congestion of the line due to the concentration of access to the Internet 56, the user sets the display terminal 15 to ". By switching to "direct" communication, image data can be acquired without congestion of the Internet 56 line. Therefore, the user can display the image on the display of the display terminal 15 in real time without waiting time and check the food condition in the storage room 5.

On the other hand, as shown by the solid line in FIG. 15, when the user is located outside the residence A, in a broad sense, when the display terminal 15 is located at a position where it cannot directly communicate with the router device 55, the user selects the "server" button. By touching 71, the display terminal 15 communicates with the server device 57 via the Internet 56 and reads the image data of the refrigerator 51. That is, when the user is outside the residence A, if he / she touches the "server" selection button 71 to set the "server" communication, the display terminal 15 acquires the image data of the refrigerator even if he / she is in a remote place, and the foodstuff is prepared. You can know the storage status of.

As described above, the refrigerator system in this embodiment allows the user to select either "direct" communication or "server" communication, and is "direct" when he / she is in the residence A. You can select communication and view images in a comfortable state without being affected by the congestion of the Internet line.

2-6. Reproduction of image data on a display terminal 2-6-1. Reproduction of a normal internal image FIG. 30 shows a flowchart of a normal internal image display. The user arbitrarily sets the display period or the date and time of the image in the refrigerator to be displayed from the display terminal 15 and presses the image data selection button 72. For example, a pre-transmission request is made from the display terminal 15, and an image in the refrigerator is acquired within a designated period. The flow of image data acquisition is as shown in FIG. The image in the refrigerator is stored in the display terminal 15 (step 60), and the display terminal 15 confirms the presence or absence of image data within the specified period from the stored images (step 61), and displays the image data in time series. And output to the display unit of the display terminal 15 (step 62). When the user presses the play button, the image data read in time series on the display unit of the display terminal 15 is automatically switched (step 64) and displayed to perform continuous playback. The timing of image switching is such that image switching is performed after a certain period of display time has elapsed or when there is an image feed operation by the user (step 63). The image feed operation by the user can be viewed while skipping the image in the time width by pressing the feed button or using the seek bar. This makes it possible to search for images in the warehouse by specifying the date and time, and to continuously browse the images in the warehouse in chronological order, greatly reducing the time required for image search, and by using the seek bar, this makes it possible. It is possible to effectively find the target image.

2-6-2. Change detection and reproduction in the refrigerator FIG. 31 shows a flowchart in the case where the change in the image in the refrigerator is detected and the image in the refrigerator is continuously reproduced. The image data is saved in the display terminal 15 and the image data is output to the display unit (step 66), and the user traces the displayed image in the refrigerator with a finger to specify the foodstuff or the area in the refrigerator (step 67) and automatically reproduces the image. By doing so, a specific area of the previously displayed image and the currently displayed image is compared (step 68), and when a change is detected, automatic playback is stopped. As a result, the user can easily know the change in the specific area in the refrigerator or the presence or absence of foodstuffs by performing automatic reproduction without checking the image data one by one, and can easily manage the inventory. Can be done. Further, by designating a specific area, the amount of image change detection processing can be reduced and a constant processing speed can be maintained. It is preferable that the control calculation unit 59 of the refrigerator or the server control unit 66 of the server device 57 performs the comparison of the specific areas of the previously displayed image and the current display image to detect the presence or absence of the change.

2-6-3. Creating an Image Memo in the Cabinet FIG. 34 shows a flowchart when creating a memo using the image in the refrigerator. The user can input the memo information on the internal image (step 73) by tracing on the internal image output to the display terminal 15 (step 72) and reflect this (step 74). Further, by selecting the range of tracing the image area in the refrigerator with a finger (step 75), trimming is performed (step 76), and the image is partially transmitted. As a result, the size of the transmitted image can be reduced and the transmission time can be shortened, so that it is easy to use and the memo can be created and the image in the refrigerator can be shared without hassle.

2-6-4. Display of free space or free space Next, the display of free space or free space, which is the gist of the present invention, will be described.

FIG. 33 is a block diagram of the vacant state display control unit composed of the control calculation unit 59 and the storage unit 60 of the refrigerator shown in FIG. 16, and FIG. 34 is a flowchart showing the vacant state display control.

In FIG. 33, 77 is an image storage means configured in the storage unit 60 of the refrigerator, which stores a reference image of an empty storage chamber 5 before the food is put in, and images the image each time the door is opened or closed. The image to be displayed is stored together with the time information. 78 is an image analysis means for comparing and analyzing a reference image stored by the image storage means 77 and an image captured and stored by the cameras 35-1, 35-2, 79 based on the analysis result of the image analysis means 78. 15a is a calculation means for calculating the storage status of the storage chamber 5, and 15a is a display means for displaying the empty capacity or the empty space of the storage chamber 5 based on the calculation result of the calculation means 79. It is composed of a display screen of the display terminal 15.

Next, the operation of the free state display control having the above configuration will be described with reference to the flowchart of FIG. 34.

When the image in the refrigerator is displayed on the display terminal 15 by the operation shown in FIGS. 25 to 28, the image analysis means 78 compares and analyzes the display image and the reference image (step 81), and the calculation means 79 based on the comparison analysis result. The vacancy state of the storage room 5 is calculated (step 82). Then, the free state based on the calculation result is displayed on the display means 15a (step 83). This display can be variously considered, for example, numerically displayed as the vacancy rate, or displayed as a map with different colors depending on the vacancy rate. As a result, the user can accurately know the empty capacity or the empty space of the storage room via the display means 15a, and can purchase the foodstuff according to the empty capacity or the empty space. Can be prevented from being packed too much in the storage room. Therefore, it is possible to prevent a decrease in cooling efficiency due to over-packing of ingredients.

If a certain amount of time has passed after the free status is displayed, or if there is an image feed operation by the user (steps 84 and Y), the image is switched (step 85), and steps 81 to the above are taken. Repeat 83. Therefore, it is possible to display the free state by sequentially displaying the images in chronological order based on the time information attached to the images. Therefore, the user can know the change of the vacancy state of the storage room in chronological order, and grasp the tendency of how to use the refrigerator of himself / herself and his / her family, for example, the tendency that one corner of the storage room is always vacant space throughout the day and night. it can. Then, based on this, it is possible to promote more energy-saving usage by devising the food installation position in the storage room. In particular, since the refrigerator is powered on for 24 hours and consumes power, the energy saving effect is large and effective even with a slight ingenuity.

Further, as described above, since the above image shooting is configured to take an image when the door is closed, the images taken by the cameras 35-1 and 35-2 are taken only when the door is closed, and the number of times of taking an image is reduced. This makes it possible to reduce the burden on the calculation means 79.

In addition, if the door is opened and closed multiple times in a short time (for example, 5 minutes), for example, during subdivision storage in which food is stored in several times, the door should be opened and closed a certain number of times (for example, 3 times) or more. By storing only the latest image in the image storage means 77 when the door is not opened or closed for a certain period of time (for example, 5 minutes) after the detection, the burden on the calculation means 79 can be further reduced.

FIG. 35 is a flowchart showing an example of the free state display described with reference to FIG. 34. In this method, after the calculation of the vacant state is performed in step 82 of FIG. 34, the vacant capacity or the vacant space (hereinafter, simply referred to as vacant) of the storage room obtained by the calculation is the first predetermined value, for example, the storage room. It is confirmed whether or not the vacancy is equal to or more than the first predetermined value indicating 70% (step 91), and if it is equal to or more than the first predetermined value, it is confirmed whether or not the area where the vacancy exists can be specified (step 92). If possible, the area is highlighted in the display image (step 93). For example, the empty area is displayed in a blue frame. As a result, the user can recognize the vacant area in the storage without opening the heat insulating door and looking inside, and the purchased food can be stored in the storage instantly. Therefore, when storing foods in the storage room, it is not necessary to spend time opening the heat insulating door to check the vacant area inside, so that the outflow of cold air generated during that time can be prevented, and energy saving can be improved accordingly. it can.

Further, in this embodiment, it is confirmed whether or not there is a cold air outlet in the vacant area (step 94), and if there is an outlet, the cooling capacity for the area is reduced (step 95). For example, reduce the opening of the cold air outlet. As a result, cooling can be realized according to the stored state, and energy saving can be realized by optimizing the cooling.

If the vacancy confirmation result in step 91 is equal to or less than the first predetermined value, it is confirmed whether the vacancy is less than the second predetermined value, for example, the vacancy of the storage room is less than the second predetermined value indicating 10% or less. (Step 96), if it is equal to or less than the second predetermined value, it is confirmed whether or not the area where the vacancy exists can be specified (step 97), and if it can be specified, the area is highlighted and displayed in the display image (step). 98). For example, the empty area is displayed in a red frame. As a result, the user can recognize the area where the ingredients are overloaded, and the purchased ingredients can be appropriately and instantly distributed and stored in the empty area without hesitation. Therefore, the door can be opened and the empty area inside can be stored. It is possible to eliminate the time waste of deciding the amount of food distribution by looking at the above, and prevent the outflow of cold air that occurs during that time to further improve energy saving.

Then, in this embodiment, it is confirmed whether or not there is a cold air outlet in the overloaded area (step 99), and if there is an outlet, the area is in a state of being packed with foodstuffs. Since there is a high possibility that the outlet is slightly blocked and the cooling efficiency is lowered, usage information such as distributing the foodstuffs to other areas is displayed on the display means 15a (step 100). As a result, the user can see this display and know that the area is overloaded, and can improve the usage such as distributing the ingredients in the area to other areas to improve the cooling efficiency. ..

In addition, since the refrigerator and the display system in the refrigerator illustrated in the embodiment of the present invention also have the following advantages, they will be described.

That is, in the refrigerator of this embodiment, the cameras 35-1 and 35-2 photograph the foodstuffs in the storage chamber 5 from the front side of the storage chamber 5, and at this time, the lighting lamps 13-0, 34-2 in the refrigerator. Is also provided on the front side of the storage chamber 5, and the foodstuffs in the storage chamber 5 are irradiated from the same front side as the shooting direction by the cameras 35-1 and 35-2. Therefore, the captured image is a clear image without the darkness and difficulty of seeing as in the case where the interior lighting lamps 34-1 and 34-2 are provided on the back wall of the storage chamber 5. Further, it is not necessary to ask the server device 57 for image processing to brighten and sharpen the captured image.

Further, since the cameras 35-1 and 35-2 and the interior lighting lights 34-1 and 34-2 are provided so as to be located on the front side from the front end of the shelf plates 30-1 to 30-4. The images in the storage room 5 taken by the cameras 35-1 and 35-2 are the same as the state in which the user opens the door 10 of the refrigerator 51 and actually sees it. Therefore, when the user tries to display the image data on the display terminal 15, the state of the foodstuff in the storage room 5 can be confirmed without any discomfort. Moreover, since the interior lighting lamps 34-1 and 34-2 illuminate the foodstuffs on the shelf board from the front of the front end of the shelf boards 30-1 to 30-4, they are placed on the front end portion of the shelf boards 30-1 to 30-4. The placed ingredients can be photographed brightly and clearly.

Further, since the cameras 35-1 and 35-2 are provided on the inner surface of the door 10 of the refrigerator 51, and the internal lightings 34-1 and 34-2 are provided on both side walls of the opening edge of the storage chamber 5. The cameras 35-1 and 35-2 are first located at approximately the center of the left and right sides of the storage chamber 5. Therefore, the images taken by the cameras 35-1 and 35-2 can be made into a shape closer to the state in which the user opens the door 10 of the refrigerator 51 and actually sees the images. In addition, a large distance between the cameras 35-1 and 35-2 and the foodstuffs on the shelf board can be taken to take a picture of a wide range in the storage chamber 5, and a wider range of foodstuff storage status can be displayed. It can be confirmed on the terminal 15.

Further, since the cameras 35-1 and 35-2 are provided so as to be located at substantially the center of the door pockets 33-1 and 33-2 on the inner surface of the door in the front-rear direction, not only the foodstuffs in the storage chamber 5 but also the door A part of the foodstuffs stored in the pockets 33-1 and 33-2, such as a milk carton, can be photographed. Since the user has stored it by himself / herself, he / she can recognize what it is even if only a part of it is shown. Therefore, it is possible to photograph almost all the ingredients in the storage room.

Further, if the camera is provided with the camera 14-3 dedicated to the storage container as shown in FIG. 9A, the foodstuff in the storage container 31 can be clearly photographed. Further, by making the camera 35-2 movable up and down as shown in FIG. 9B and rotating it up and down as shown in FIG. 9C, the foodstuffs in the storage container 31 are also two cameras, that is, You can shoot efficiently with a small number of cameras.

On the other hand, as is clear from FIGS. 7 and 10, the internal lighting lights 34-1 and 34-2 are provided at positions behind the cameras 35-1 and 35-2, and the internal lighting lights 34-1 and 34-2 are provided. Since the light of 34-2 is provided so as not to directly enter the photographing section of the cameras 35-1 and 35-2, the light of the internal lighting lamps 34-1 and 34-2 is provided to the cameras 35-1 and 35-2. It is possible to prevent the images of the cameras 35-1 and 35-2 from being deteriorated by entering the camera, and it is possible to always obtain a good image. This is because a light-shielding portion is provided on the camera side of the interior lighting lamps 35-1, 35-2, or the light source such as the LED constituting the interior lighting lamps 34-1 and 34-2 is tilted diagonally backward. The same effect can be obtained by adopting a configuration such as arranging.

Further, the illuminance of the interior lighting lamps 34-1 and 34-2 in the storage room is 150 lux to 200 lux, and when the illuminance is taken as it is, the foodstuffs become whitish as described in the section of the shooting operation in FIG. It will be difficult to see and distinguish. Therefore, in this embodiment, the illuminance is reduced for shooting. This enables clear shooting even with a CCD camera or a CMOS camera. The illuminance of the interior lighting lamps 34-1 and 34-2 is controlled by the control calculation unit 59 so that the illuminance is lower than the normal illuminance only when the door 10 is closed or the door 10 is opened for shooting. However, by performing such control, it is possible to maintain the same illuminance as before during normal times when shooting is not performed, and to enable clear shooting while making it easier to see the inside of the refrigerator. The decrease in illuminance may be caused by, for example, turning off a part of the interior lighting lamps 34-1 and 34-2 of either one or both of the interior lighting lamps 34-1 and 34-2, or lighting both interior lamps. Various considerations such as reducing the illuminance of the lamps 34-1 and 34-2 may be considered and may be appropriately selected.

Further, the images in the storage chamber 5 taken by the cameras 35-1 and 35-2 are transmitted to the server device 57 by transmitting the images of the respective cameras 35-1 and 35-2 to the server device 57 or the refrigerator 51. Alternatively, the images are combined and combined in the display terminal 15, but in this embodiment, the image composition process becomes easy.

That is, the cameras 35-1 and 35-2 are door inner surface portions corresponding to the front ends of the shelf boards 30-1 to 30-4, in this example, the upper shelf board 30-1 and the third shelf board 30-. Since it is provided on the inner surface of the door corresponding to the front end of No. 3, the shelves 30-1 and 30-3 are photographed by the respective cameras 35-1 and 35-2 so that the line Z in FIG. 34 becomes the boundary line. Will be taken mainly in.

Therefore, the image data in the storage chamber 5 taken by the two cameras 35-1 and 35-2 can be combined at the second-stage shelf board 30-2 portion which is the boundary line Z. That is, as shown in the "photographed raw image" of FIG. 36, the shelf boards 30-2 are commonly shown in the upper and lower images, and the curvature of the "photographed raw image" is corrected to "after the curvature correction processing". After making the "image", if the common shelf boards 30-2 moved to the upper and lower images are matched, the photographed image of the entire storage room is obtained. As a result, it is not necessary to perform extremely difficult correction processing for aligning and combining each foodstuff placed on the shelf board at the time of image combination processing, and the image combination processing is easy. Become. Therefore, the processing program for image combination can be simplified, and the capacity of the control portion of the server device 57, the refrigerator 51, or the display terminal 15 for image combination processing also needs to be large. It disappears and can contribute to speeding up the processing speed.

Further, since the two cameras 35-1 and 35-2 are provided at substantially the front positions of the shelf boards 30-1 and the shelf boards 30-3 whose heights cannot be adjusted, the height-adjustable shelf boards 30- Even if the height of No. 2 is changed, the image combination processing can be easily performed by adjusting the range of the combination processing according to the height of the shelf board 30-2 at the time of the image combination processing.

Although the refrigerator and the display system in the refrigerator of the present invention have been described above by exemplifying a specific configuration, the present invention is not limited to the above configuration. That is, it should be considered that the embodiments disclosed this time are exemplary in all respects and are not restrictive. That is, the scope of the present invention is indicated by the scope of claims rather than the above description, and it is intended that all modifications within the meaning and scope equivalent to the scope of claims are included.

For example, the vacancy status display described in the present embodiment is applied to a refrigerator and a display system in the refrigerator so that the storage status can be confirmed using the Internet, and the vacancy status is displayed on the portable display terminal 15. However, this may be displayed on a display unit provided on the refrigerator itself to display an operating state or the like, or may be applied to a single refrigerator that does not use the Internet. It should be noted that even if the display is displayed on the display unit of the refrigerator itself, it is effective in that the vacant state of the storage room can be known without opening and closing the door.

Further, although the case where the image storage means 77, the image analysis means 78, the calculation means 79 and the like are provided in the control calculation unit 59 and the storage unit 60 on the refrigerator side is illustrated, this may be provided on the server device 57 side. Alternatively, the refrigerator and the server device 57 may be shared.

Further, as the communication mode, in the above-described embodiment, the gateway device 54 is interposed in the communication path between the display terminal 15, the wireless adapter 53 of the refrigerator 51, and the Internet 56, but the gateway device 54 is not always necessary. The system configuration may be simplified by directly communicating with the display terminal 15, the wireless adapter 53 of the refrigerator 51, and the Internet 56 via the router device 55.

Further, in the above embodiment, the image data is acquired from the server device 57 when the user is outside the residence A, but the Internet 56, the router device 55, and the gateway device are acquired from the display terminal 15 without going through the server device 57. It may be configured to communicate with 54 and take in the image data stored in the storage unit 60 of the refrigerator 51 directly from the refrigerator 51. In this case and when performing "direct" communication, it is preferable that the storage unit 60 of the refrigerator 51 performs the image combining process and transmits the image data after the combined process.

Further, in this embodiment, a case where the "direct" selection button 70 and the "server" selection button 71 are displayed on the display of the display terminal 15 and the user selects "direct" communication and "server" communication is exemplified. However, this may be done by giving the display terminal 15 a function to automatically switch to "direct" communication when the radio wave of the router device 55 is caught so that "direct" communication is automatically selected, which is convenient. Can be improved.

INDUSTRIAL APPLICABILITY The present invention can prevent a decrease in cooling efficiency due to overstuffing of foodstuffs, and can provide a refrigerator that can be used efficiently. Therefore, the present invention can be applied to various refrigerators including household refrigerators.

1 Refrigerator body 2 Outer box 3 Inner box 4 Insulation 5 Refrigerator room (storage room)
6 Switching room 7 Ice making room 8 Freezing room 9 Vegetable room 10 Doors 11-14 Pull-out doors (doors)
15 Display terminal (image display means)
15a Display means 16 Cooling chamber 17 Cooler 18 Blower fan 19 Compressor 20 Heat dissipation pipe 21 Capillary tube 22 Damper 23 Storage frame 24 Terminal holder 25 Touch panel 26 Gap 27 Frame 28 Recess 29, 29a Smoke sheet 30-1 to 30- 4 Shelf board 31 Storage container 32-1-23-4 Side wall convex part 33-1-3-3-8 Door pocket 34-1,34-2 Interior lighting 35-1,35-2,35-3 Camera (imaging) means)
36 Center pillar 41 Camera module 50 In-refrigerator display system 51 Refrigerator 53 Wireless adapter (communication device)
53a "Connect" button 54 Gateway device (relay device)
55 Router device 56 Internet 57 Server device 58 Interface (I / F)
59 Control calculation unit 60 Storage unit 61 Door open / close detection unit 62 Fan drive circuit 63 Compressor drive circuit 64 AC power supply 65 Server interface 66 Server control unit 67 Data storage unit 70 "Direct" selection button 71 "Server" selection button 72 Image data Select button 73 Energy saving related data selection button 74 Send button 77 Image storage means 78 Image analysis means 79 Calculation means A Residence

Claims (8)

A notification method for notifying a user of information, which is executed in a system including a refrigerator having at least one storage room and a display terminal capable of communicating with the refrigerator.
An imaging device for imaging the inside of the storage chamber is provided on the inner surface of the door of the storage chamber.
The notification method is
The image captured by the image pickup device is displayed on the display terminal, and the image is displayed on the display terminal.
Based on the image captured by the image pickup device, it is determined whether or not the foodstuff is overloaded in the area where the cold air outlet of the storage chamber exists.
A notification method characterized in that when it is determined that the foodstuffs are overloaded, the display terminal displays information prompting the user to disperse the foodstuffs in the area where the cold air outlet exists in other areas. .. A notification method for notifying a user of information, which is executed in a system including a refrigerator having at least one storage room and a display terminal capable of communicating with the refrigerator.
An imaging device for imaging the inside of the storage chamber is provided on the inner surface of the door of the storage chamber.
The notification method is
The image captured by the image pickup device is displayed on the display terminal, and the image is displayed on the display terminal.
Based on the image captured by the image pickup device, it is determined whether or not the foodstuff is overloaded in the area where the cold air outlet of the storage chamber exists.
A notification method characterized in that, when it is determined that the foodstuffs are overloaded, information for notifying the user that the foodstuffs are overloaded in the area where the cold air outlet exists is displayed on the display terminal. The first or second claim, wherein it is determined whether or not the foodstuff is overloaded in the area where the cold air outlet exists by comparing the reference image with the image captured by the image pickup apparatus. Notification method. The notification method according to claim 3, wherein the reference image is an image obtained by capturing an image of the storage chamber in an empty state. A system including a refrigerator having at least one storage room and a display terminal capable of communicating with the refrigerator.
An imaging device for imaging the inside of the storage chamber is provided on the inner surface of the door of the storage chamber.
The system
A first display control means for displaying an image captured by the image pickup device on the display terminal, and
Based on the image captured by the image pickup device, a determination means for determining whether or not the foodstuff is overloaded in the area where the cold air outlet of the storage chamber exists, and
A second display control means for displaying on the display terminal information prompting the user to disperse the foodstuffs in the area where the cold air outlet exists to other areas when it is determined that the foodstuffs are overloaded. A system characterized by being equipped with. A system including a refrigerator having at least one storage room and a display terminal capable of communicating with the refrigerator.
An imaging device for imaging the inside of the storage chamber is provided on the inner surface of the door of the storage chamber.
The system
A first display control means for displaying an image captured by the image pickup device on the display terminal, and
Based on the image captured by the image pickup device, a determination means for determining whether or not the foodstuff is overloaded in the area where the cold air outlet of the storage chamber exists, and
When it is determined that the foodstuffs are overloaded, the second display control means for displaying the information notifying the user that the foodstuffs are overloaded in the area where the cold air outlet exists is displayed on the display terminal. A system characterized by being prepared. The determination means is characterized in that it determines whether or not the foodstuff is overloaded in the area where the cold air outlet exists by comparing the reference image with the image captured by the imaging device. The system according to 5 or 6. The system according to claim 7, wherein the reference image is an image obtained by capturing an image of the storage chamber in an empty state.

Images