EP3543630A1

EP3543630A1 - Refrigerator system

Info

Publication number: EP3543630A1
Application number: EP16921964.9A
Authority: EP
Inventors: Tsuyoshi Uchida; Takashi Ito
Original assignee: Mitsubishi Electric Corp
Current assignee: Mitsubishi Electric Corp
Priority date: 2016-11-21
Filing date: 2016-11-21
Publication date: 2019-09-25
Also published as: CN109952480B; CN109952480A; JPWO2018092310A1; JP6673498B2; WO2018092310A1; EP3543630A4; TWI638126B; TW201819832A

Abstract

There is provided a refrigerator system that enables both a user terminal used at home and outside to check the latest storage room image, and reduces chances of communication between the home and outside user terminals. Hence, the refrigerator system includes: a storage room image storage section (4) that stores a storage room image of a refrigerator (1000) output from a refrigerator image pickup device (2); a storage room image management section (5) that stores the storage room image stored in the storage room image storage section (4); a radio communication section (6); and a user terminal (10). The radio communication section (6) is capable of switching between a first communication mode in which a storage room image stored in the storage room image storage section (4) is transmitted to a cloud server (9), and a second communication mode in which a storage room image stored in the storage room image management section (5) is transmitted to the user terminal (10). The user terminal (10) receives a storage room image from the cloud server (9) in the first communication mode, and receives a storage room image from the radio communication section (6) in the second communication mode.

Description

Field

The invention relates to a refrigerator system.

Background

A known refrigerator system (see PTL 1, for example) includes an in-home network device that has a food storeroom including image pickup means for picking up an image of the inside of the food storeroom, and a local database accumulating picked up images of the inside of the food storeroom and connected to an information network. A mobile information terminal is used to access the in-home network device through an information network such as the Internet, and download and show the picked up image of the inside of the food storeroom stored in the local database on a display.

Citation List

Patent Literature

[PTL 1] JP 2002-243335 A

Summary

Technical Problem

However, in a refrigerator system such as that shown in PTL 1, when an image (hereinafter referred to as "storage room image") of the inside of a storage room of the refrigerator which is the food storeroom is checked, a user terminal constantly connects to the local database in the in-home network through an external information network such as the Internet. For this reason, firstly, communication between a communication device used at home where a refrigerator main body is installed and equipment used outside becomes more frequent, whereby communication load and communication cost increase. Secondly, since communication between the communication device used at home and the equipment used outside becomes more frequent, the risk of information leakage to the Internet and security risks such as occurrence of a security hole may increase.
The invention has been made to solve such problems. An objective of the invention is to provide a refrigerator system that enables both a user terminal used at home where a refrigerator main body is installed and a user terminal used outside to check the latest storage room image of the refrigerator main body, reduces chances of communication between a communication device used at home and equipment used outside as much as possible, reduces communication load and communication cost, and can reduce the risk of information leakage to the Internet and security risks such as occurrence of a security hole.

Solution to Problem

A refrigerator system according to the present invention includes: a refrigerator main body in which a storage room for storing food is formed; a camera configured to pick up an image of an inside of the storage room and to output the image as a storage room image; a first image storage section configured to store the storage room image output from the camera; a second image storage section configured to store the storage room image stored in the first image storage section; a communication device capable of communicating with outside; and a terminal device capable of communicating with the communication device, wherein the communication device is capable of switching between a first communication mode in which a storage room image stored in the first image storage section is transmitted to an external server, and a second communication mode in which a storage room image stored in the second image storage section is transmitted to the terminal device, and the terminal device: receives a storage room image from the external server when the communication device selects the first communication mode, and receives a storage room image stored in the second image storage section from the communication device when the communication device selects the second communication mode.

Advantageous Effects of Invention

The refrigerator system of the invention has effects of enabling both a user terminal used at home where a refrigerator main body is installed and a user terminal used outside to check the latest storage room image of the refrigerator main body, reducing chances of communication between a communication device used at home and equipment used outside as much as possible, reducing communication load and communication cost, and reducing the risk of information leakage to the Internet and security risks such as occurrence of a security hole.

Brief Description of the Drawings

Fig. 1 is a front view of a refrigerator included in a refrigerator system according to an Embodiment 1 of the present invention.
Fig. 2 is a diagram schematically showing a home-side configuration of the refrigerator system according to the Embodiment 1 of the present invention and a configuration of a refrigerating room of the refrigerator
Fig. 3 is a diagram for describing a first communication mode of the refrigerator system according to the Embodiment 1 of the present invention.
Fig. 4 is a diagram for describing a second communication mode of the refrigerator system according to the Embodiment 1 of the present invention.
Fig. 5 is a flowchart for describing an operation of the refrigerator system according to the Embodiment 1 of the present invention.
Fig. 6 is a diagram for describing a case where a storage room image is transmitted to multiple user terminals in a refrigerator system according to an Embodiment 2 of the present invention.
Fig. 7 is a diagram for describing a case where storage room images of multiple refrigerators are transmitted to a user terminal in the refrigerator system according to the Embodiment 2 of the present invention.
Fig. 8 is a diagram schematically showing a home-side configuration of a refrigerator system according to an Embodiment 3 of the present invention and a configuration of a refrigerating room of a refrigerator
Fig. 9 is a diagram showing an example of a storage room image of the refrigerator system according to the Embodiment 3 of the present invention.
Fig. 10 is a diagram showing a first example of a processed storage room image of the refrigerator system according to the Embodiment 3 of the present invention.
Fig. 11 is a diagram showing a second example of processed storage room images of the refrigerator system according to the Embodiment 3 of the present invention.
Fig. 12 is a diagram showing a third example of processed storage room images of the refrigerator system according to the Embodiment 3 of the present invention.
Fig. 13 is a flowchart for describing an operation of the refrigerator system according to the Embodiment 3 of the present invention.

Description of Embodiments

Embodiments of the invention will be described with reference to the accompanying drawings. In the drawings, the same or corresponding parts are assigned the same reference numerals, and overlapping descriptions will be simplified or omitted as appropriate. Note that the present invention is not limited to the following embodiments, and various modifications may be made without departing from the gist of the invention.

Embodiment 1

Figs. 1 to 5 are related to Embodiment 1 of the invention, where Fig. 1 is a front view of a refrigerator included in a refrigerator system, Fig. 2 is a diagram schematically showing a home-side configuration of the refrigerator system and a configuration of a refrigerating room of the refrigerator, Fig. 3 is a diagram for describing a first communication mode of the refrigerator system, Fig. 4 is a diagram for describing a second communication mode of the refrigerator system, and Fig. 5 is a flowchart for describing an operation of the refrigerator system.
The refrigerator system of Embodiment 1 of the invention includes a refrigerator 1000. The refrigerator 1000 is a refrigerator main body of the refrigerator system of Embodiment 1 of the invention. In Embodiment 1, the refrigerator 1000 has a refrigerating room 100, an ice making room 200, a switchover room 300, a freezing room 400, and a vegetable room 500. The refrigerating room 100, the ice making room 200, the switchover room 300, the freezing room 400, and the vegetable room 500 are storage rooms for storing food.
As shown in Fig. 1, the storage rooms are arranged in the order of the refrigerating room 100, the ice making room 200 and switchover room 300, the freezing room 400, and the vegetable room 500 from the top. The ice making room 200 and the switchover room 300 are arranged side by side at the same vertical position. A refrigerating room door 101 that opens and closes the refrigerating room 100 is provided on a front face of the refrigerating room 100. The refrigerating room door 101 is an example of a door that opens and closes a storage room. In addition, each of the ice making room 200, the switchover room 300, the freezing room 400, and the vegetable room 500 is configured to be drawn out to the near side of the refrigerator 1000 together with a door provided on a front face thereof.
An operation panel 1 is provided on a front face of the refrigerating room door 101. The operation panel 1 includes a panel display portion 1a. The panel display portion 1a is a main body display part capable of displaying information. The panel display portion 1a can display refrigerator information such as a set temperature and the current temperature of each storage room. The panel display portion 1a includes a liquid crystal display, for example. The operation panel 1 includes an operation portion such as a touch panel, a button, and a switch, for example. A user can operate the operation portion of the operation panel 1 to input necessary information. Note that the installation position of the operation panel 1 is not limited to the door of the refrigerating room 100, and may be a door of another storage room, or a side face of the refrigerator 1000, for example.
As shown in Fig. 2, the front face side (left side in Fig. 2) of the refrigerating room 100 is closed by the refrigerating room door 101. Note that the cross-sectional view of Fig. 2 is a section taken along A-A' in Fig. 1. Multiple refrigerating room shelf boards 102 are provided inside the refrigerating room 100. The inside of the refrigerating room 100 is partitioned into multiple spaces (shelves) in the vertical direction by the refrigerating room shelf boards 102. A space below the lowest refrigerating room shelf board 102 is a chilled room 110. A chilled case 111 is installed in the chilled room 110. The chilled case 111 can be drawn out frontward along a guide member (not shown) such as rails. Additionally, a cooling air passage 1010 is formed on the rear face side (right side in Fig. 2) of the refrigerating room 100.
A refrigerator image pickup device 2 is installed on an inner face of the refrigerating room door 101. The refrigerator image pickup device 2 is a camera that picks up an image of the inside of a storage room (refrigerating room 100 in this embodiment) and outputs a storage room image. A visual range of the refrigerator image pickup device 2 is from a ceiling surface of the inside of the refrigerating room 100 to the chilled room 110. That is, the refrigerator image pickup device 2 can pick up an image of the inside of the refrigerating room 100 from the same viewpoint as a user viewing the inside of the refrigerating room 100 from the refrigerating room door 101 side.
The refrigerator image pickup device 2 picks up an image of the inside of the refrigerating room 100 when the refrigerating room door 101 is opened or closed, for example. This is because when the refrigerating room door 101 is opened or closed, it is likely that food is brought into or taken out of the refrigerating room 100 to change the storage state inside the refrigerating room 100. The refrigerator 1000 includes a sensor that can detect opening and closing of the door. The sensor is a general magnet-type sensor, for example. Specifically, the sensor detects proximity of a magnet embedded in the refrigerating room door 101 by a pair of reed switches installed on the refrigerator 1000 main body side, for example. After detecting closure of the open refrigerating room door 101, the refrigerator image pickup device 2 picks up an image while the inside of the refrigerating room 100 is illuminated, for example.
Note that the installation position of the refrigerator image pickup device 2 is not limited to the inner face of the refrigerating room door 101 as shown in Fig. 2. The refrigerator image pickup device 2 may be installed anywhere, as long as it is a position from which images of stored goods such as food placed on the refrigerating room shelf board 102 can be picked up. Moreover, the number of installed refrigerator image pickup devices 2 is not limited to one. That is, multiple refrigerator image pickup devices 2 may be installed. By installing multiple refrigerator image pickup devices 2, it is possible to avoid creation of a blind spot due to blockage of view by food and the like stored in the refrigerating room 100.
The refrigerator 1000 includes a control board 1001. The control board 1001 is accommodated in an upper part on the rear face side of the refrigerator 1000, for example. The control board 1001 includes a control circuit and the like for performing various controls necessary for operation of the refrigerator 1000. The control board 1001 includes a microcomputer, for example, and therefore includes a processor and a memory. The control board 1001 performs preset processing by allowing the processor to execute a program stored in the memory, and thereby controls the refrigerator 1000.
The refrigerator system of Embodiment 1 of the invention includes an image server 3. The image server 3 is configured to retain, manage and transmit storage room image to the outside. The image server 3 is installed in a home (hereinafter simply referred to as "home") where the refrigerator 1000 is installed, and outside the storage room of the refrigerator 1000. Figs. 1 and 2 show an example where the image server 3 is arranged on an outer face portion of a ceiling part of the refrigerator 1000. Note that the installation position of the image server 3 is not limited to the ceiling part of the refrigerator 1000. The image server 3 may be installed on a side face or a rear face of the refrigerator 1000, or a door of the storage room, as long as it is outside the storage room of the refrigerator 1000. Moreover, the image server 3 may be contained on the control board 1001.
As in the case of the aforementioned control board 1001, the image server 3 includes a microcomputer, for example, and therefore includes a processor and a memory. In addition, as shown in Fig. 2, the image server 3 includes a storage room image storage section 4, a storage room image management section 5, and a radio communication section 6. Functions of the sections are implemented by allowing the processor in the microcomputer of the image server 3 to execute a program stored in the memory to perform preset processing, for example. Note that the image server 3 is connected to the control board 1001 and operates by receiving supply of electric power from the control board 1001, for example.
The storage room image storage section 4 is a first image storage section for storing a storage room image output from the refrigerator image pickup device 2. As mentioned earlier, upon detection of opening or closing of the refrigerating room door 101, the refrigerator image pickup device 2 picks up an overall image of the inside of the refrigerating room 100. The refrigerator image pickup device 2 outputs the picked up overall image (storage room image) of the inside of the refrigerating room 100 to the storage room image storage section 4. The storage room image output from the refrigerator image pickup device 2 is input into the storage room image storage section 4. Accordingly, every time the refrigerating room door 101 is opened or closed (i.e., every time the storage state inside the refrigerating room 100 changes), a storage room image output from the refrigerator image pickup device 2 is input into the storage room image storage section 4. Thus, the latest-state storage room image is constantly input into the storage room image storage section 4.
The storage room image storage section 4 temporarily stores the input latest storage room image. At this time, the storage room image storage section 4 temporarily stores the time when the storage room image is input into the storage room image storage section 4, together with the storage room image, as image pickup time information. Note that the image pickup time information also includes information on the date (day and month or year, day, and month) of image pickup.
The storage room image management section 5 is a second image storage section for storing the storage room image stored in the storage room image storage section 4 which is the first image storage section. The storage room image management section 5 acquires the storage room image and the image pickup time information temporarily stored in the storage room image storage section 4, from the storage room image storage section 4.
Then, the storage room image management section 5 stores the acquired storage room images by sorting them in the order of image pickup time on the basis of the image pickup time information. Thus, the storage room image picked up by the refrigerator image pickup device 2 is temporarily stored in the storage room image storage section 4, and then is moved to the storage room image management section 5 and managed. Note that the storage room image managed and stored in the storage room image management section 5 can be displayed on the panel display portion 1a of the operation panel 1 with image pickup time information.
The radio communication section 6 is a communication device capable of communicating with the outside. The radio communication section 6 uses a known radio communication technique, for example, to enable mutual communication to exchange information between the image server 3 of the refrigerator 1000 and a router device 7. Note that the communication between the image server 3 and the router device 7 is not limited to the radio system, and may be a wired system.
As shown in Figs. 3 and 4, the refrigerator system of Embodiment 1 of the invention includes a user terminal 10. The user terminal 10 is a terminal device capable of communicating with the radio communication section 6 which is a communication device. To be specific, the user terminal 10 is a personal computer (PC), a smart phone, or a tablet terminal, for example. Note that there are one or more user terminals 10, and more preferably, there are multiple user terminals 10.
The user terminal 10 includes an unillustrated terminal display portion. The terminal display portion includes a liquid crystal display, for example. Moreover, the terminal display portion may be configured of a touch panel, so that information may be displayed thereon and user operation can be received. The terminal display portion of the user terminal 10 is capable of displaying a storage room image and various information including image pickup time information.
Next, a description will be given of displaying of a storage room image of the refrigerator 1000 on the terminal display portion of the user terminal 10. In order to display a storage room image on the user terminal 10, the user terminal 10 needs to receive the storage room image. To enable reception of the storage room image by the user terminal 10, the radio communication section 6 is configured to switch between two communication modes. That is, the radio communication section 6 which is a communication device is capable of switching between a first communication mode and a second communication mode.
The first communication mode is a communication mode in which a storage room image stored in the storage room image storage section 4 which is the first image storage section is transmitted to an external server. As shown in Fig. 3, the refrigerator 1000, the image server 3, and the router device 7 are installed at home. The router device 7 is connected to and is capable of communicating with a cloud server 9 through Internet 8, for example. The cloud server 9 is the aforementioned external server. Fig. 3 exemplifies a case using the cloud server 9 formed of multiple servers (server group) connected to the Internet 8, as the external server. However, the external server is not limited to the cloud server 9, and may be a single server. The Internet 8 and the cloud server 9 are outside (hereinafter simply referred to as "outside") the home where the refrigerator 1000 is installed.
As shown in Fig. 4, the second communication mode is a communication mode in which a storage room image stored in the storage room image management section 5 which is the second image storage section is transmitted to the user terminal 10.
In initial settings, the radio communication section 6 selects the first communication mode. Hence, as shown in Fig. 3, the radio communication section 6 transmits the latest storage room image temporarily stored in the storage room image storage section 4 to the outside cloud server 9, through the router device 7 and the Internet 8. Then, the cloud server 9 stores the received storage room image. Note that at this time, image pickup time information corresponding to the storage room image may be transmitted together, and the cloud server 9 may store both the storage room image and the image pickup time information.
At this time, assuming a case where there is congestion in communication to the cloud server 9 and the storage room image cannot be uploaded (transmitted) in a single attempt, the maximum number of retries of uploading of the storage room image to the cloud server 9 from the radio communication section 6 may be set. Then, when the uploading reaches the maximum number of retries, the radio communication section 6 may upload the storage room image immediately.
If the radio communication section 6 selects the first communication mode, the user terminal 10 receives a storage room image from the external server. That is, in this case, the outside user terminal 10 accesses the cloud server 9 through the Internet 8. When the user operates the user terminal 10 to display the storage room image, the user terminal 10 transmits a storage room image-transmission request. Upon receipt of the storage room image-transmission request, the cloud server 9 transmits a storage room image and image pickup time information stored therein to the user terminal 10. The user terminal 10 receives the storage room image and image pickup time information from the cloud server 9 through the Internet 8. Then, the user terminal 10 displays the received storage room image and image pickup time information on the terminal display portion.
While the first communication mode is selected, the radio communication section 6 is capable of switching to the second communication mode upon request from the home user terminal 10. When the radio communication section 6 is switched to the second communication mode by the request from the home user terminal 10, as shown in Fig. 4, the radio communication section 6 transmits a storage room image stored in the storage room image management section 5 to the home user terminal 10, through the home router device 7. Note that at this time, as in the case of the aforementioned first communication mode, image pickup time information corresponding to the storage room image may also be transmitted together.
If the radio communication section 6 selects the second communication mode, the user terminal 10 receives a storage room image stored in the storage room image management section 5 from the radio communication section 6. That is, in this case, when the user operates the user terminal 10 to display the storage room image, the user terminal 10 transmits a storage room image-transmission request. Upon receipt of the storage room image-transmission request, the radio communication section 6 transmits a storage room image and image pickup time information stored in the storage room image management section 5 to the home user terminal 10. The home user terminal 10 receives the storage room image and image pickup time information transmitted from the radio communication section 6 through the home router device 7. Then, the user terminal 10 displays the received storage room image and image pickup time information on the terminal display portion.
Here, the router device 7 preferably includes a function of automatically setting a radio communication connection with the radio communication section 6 or the like of the image server 3. However, in case the router device 7 does not include such an automatic setting function, the radio communication section 6 should be capable of manually setting a radio communication connection between the router device 7 and the radio communication section 6 of the image server 3, by a user operation of the user terminal 10 in the second communication mode.
Note that as mentioned earlier, at home, the storage room image can also be checked on the operation panel 1 of the refrigerator 1000. For this reason, in practice, the first communication mode is likely to be used more frequently than the second communication mode. Accordingly, as mentioned earlier, the first communication mode is preferably selected in initial settings of the radio communication section 6.
It is also preferable that the radio communication section 6 select the first communication mode when a preset certain amount of time passes after selecting the second communication mode. For example, if the user goes out while the second communication mode restricted to viewing at home is set, the latest storage room image stored in the storage room image storage section 4 is not uploaded to the cloud server 9. Hence, the user cannot check the latest storage room image from outside. In case of such trouble, even when the second communication mode is set to check the storage room image at home, the first communication mode should preferably be resumed automatically after passage of a certain amount of time.
Note that the radio communication section 6 may determine which of the first communication mode and the second communication mode to select, depending on whether the user terminal 10 that transmitted the storage room image-transmission request is at home or is outside. In this case, the radio communication section 6 that received the storage room image-transmission request from the user terminal 10 checks whether the user terminal 10 that transmitted the storage room image-transmission request is at home or is outside. This checking can be done by checking the transmission route of the storage room image-transmission request, and checking whether the storage room image-transmission request was transmitted from home or from outside, for example. Additionally, the transmission route of the storage room image-transmission request can be confirmed by using a known network management command, for example.
Upon receipt of a storage room image-transmission request from an outside user terminal 10, the radio communication section 6 selects the first communication mode. Accordingly, the radio communication section 6 transmits the latest storage room image stored in the storage room image storage section 4 to the cloud server 9. Then, the outside user terminal 10 receives the storage room image from the cloud server 9.
On the other hand, upon receipt of a storage room image-transmission request from a home user terminal 10, the radio communication section 6 selects the second communication mode. Accordingly, the radio communication section 6 transmits the storage room image stored in the storage room image management section 5 to the home user terminal 10. Then, the home user terminal 10 receives the storage room image from the radio communication section 6 through the router device 7.
According to the first communication mode described above, the user can check the storage room image of the refrigerator 1000 through the Internet 8 and the cloud server 9, even from outside. Additionally, the second communication mode described above uses only a local communication network closed in a home. Hence, the user can view information such as storage room images stored in the image server 3 on the home user terminal 10, by inputting an access code of the router device 7 and an address (URL) of the image server 3 into the user terminal 10.
Note that one or both of the image server 3 and the cloud server 9 may have an authentication section for verifying whether the user terminal 10 has the authority to receive the storage room image. Fig. 3 shows an example where a user authentication device 11 as an authentication section is provided in the cloud server 9. The user authentication device 11 verifies whether the user terminal 10 has the authority to receive the storage room image of the refrigerator 1000.
Accordingly, the user authentication device 11 of the cloud server 9 previously stores, in association with an ID of the refrigerator 1000, a terminal device ID of the user terminal 10 that has authority to connect to the refrigerator 1000. The ID of the refrigerator 1000 is refrigerator identification information used to uniquely identify multiple refrigerators 1000 connected to the Internet 8. The terminal device ID is terminal device identification information used to uniquely identify multiple user terminals 10 connected to the Internet 8.
The user authentication device 11 verifies whether a mobile terminal ID of the user terminal 10 is stored, in association with the ID of the refrigerator 1000, as a user terminal having authority to receive a storage room image of the refrigerator 1000. If the mobile terminal ID of the user terminal 10 is stored, in association with the ID of the refrigerator 1000, as a user terminal having authority to receive a storage room image of the refrigerator 1000, the user authentication device 11 verifies the authority of the user terminal 10 for receiving a storage room image of the refrigerator 1000.
A storage room image is stored in one or both of the storage room image management section 5 and the cloud server 9, in association with an ID of the refrigerator 1000 in which the storage room image was picked up. Here, a storage room image stored in the cloud server 9 is associated with an ID of the refrigerator 1000 in which the storage room image was picked up, for example.
The cloud server 9 allows reception of a storage room image of the refrigerator 1000 by the user terminal 10 whose authority of receiving a storage room image of the refrigerator 1000 is verified by the user authentication device 11. Conversely, the cloud server 9 restricts reception of a storage room image of the refrigerator 1000 by the user terminal 10 whose authority of receiving a storage room image of the refrigerator 1000 is not verified by the user authentication device 11.
Since such a user authentication device 11 is provided, even if an access code (e.g., SSID and encryption key) of the router device 7 is leaked, it is possible to prevent a terminal other than the registered user terminal 10 from accessing the cloud server 9 or the image server 3 and receiving a storage room image.
Note that the user authentication device 11 described above verifies authority by use of IDs. However, the authentication method is not limited to this. Other than the ID, biometric authentication such as a fingerprint, a vein, and the iris of an eye, authentication by password, or a combination of these authentication methods may be used to verify authority.
Next, an exemplar flow of an operation of the refrigerator system configured in the above manner will be described with reference to Fig. 5. First, when the refrigerating room door 101 is opened or closed by the user, the aforementioned sensor detects the opening or closing of the refrigerating room door 101. Upon detection of the opening or closing of the refrigerating room door 101, the processing proceeds from step S11 to step S12.
In step S12, the refrigerator image pickup device 2 picks up an image of the inside of the refrigerating room 100 which is a storage room. After step S12, the processing proceeds to step S13. In step S13, the refrigerator image pickup device 2 transmits the image of the inside of the refrigerating room 100 picked up in step S12 to the storage room image storage section 4 as a storage room image. The storage room image storage section 4 stores the storage room image picked up by the refrigerator image pickup device 2 in step S12 together with image pickup time information. After step S13, the processing proceeds to step S14.
In step S14, the storage room image storage section 4 transmits the storage room image and image pickup time information received in step S13 to the storage room image management section 5. After step S14, the processing proceeds to step S15. In step S15, the storage room image management section 5 stores the storage room images transmitted from the storage room image storage section 4 by sorting them in the order of image pickup time on the basis of the image pickup time information. After step S15, the processing proceeds to step S16.
In step S16, the storage room image management section 5 transmits the storage room image and image pickup time information to the operation panel 1. The operation panel 1 displays the received storage room image and image pickup time information on the panel display portion 1a. After step S16, the processing proceeds to step S17.
In step S17, the radio communication section 6 checks whether the first communication mode is currently selected. If the first communication mode is selected, the processing proceeds to step S18. In step S18, the radio communication section 6 uploads the storage room image and image pickup time information temporarily stored in the storage room image storage section 4 to the outside cloud server 9 through the router device 7 and the Internet 8. After step S18, the processing proceeds to step S19.
On the other hand, if the first communication mode is not selected in step S17, this means that the second communication mode is currently selected. In this case, the processing proceeds directly from step S17 to step S19 without passing step S18.
In step S19 and the following step S20, the subject of the processing differs depending on whether the processing of step S18 is performed. If the processing of step S18 is performed and the storage room image is uploaded to the cloud server 9, that is, if the first communication mode is selected, the cloud server 9 checks whether a storage room image-transmission request is received from the user terminal 10.
On the other hand, if the processing of step S18 is not performed, that is, if the second communication mode is selected, the radio communication section 6 checks whether a storage room image-transmission request is received from the user terminal 10.
If a storage room image-transmission request is not received from the user terminal 10, the processing returns to step S11. On the other hand, if a storage room image-transmission request is received from the user terminal 10, the processing proceeds to step S20.
In step S20, if the first communication mode is selected, the cloud server 9 transmits the storage room image and image pickup time information to the user terminal 10. On the other hand, if the second communication mode is selected, the radio communication section 6 transmits the storage room image and image pickup time information to the user terminal 10. Then, the user terminal 10 displays the received storage room image and image pickup time information on the terminal display portion. After step S20, the processing returns to step S11.
Note that an example of providing the refrigerator image pickup device 2 in the refrigerating room 100 has been described. However, the installation point of the refrigerator image pickup device 2 is not limited to this example. The refrigerator image pickup device 2 may be provided in a storage room other than the refrigerating room 100, or may be installed simultaneously in multiple storage rooms. For example, the inside state of the vegetable room 500 is more difficult to grasp than other storage rooms, and the vegetable room 500 is a storage room that stores vegetables that are more likely to go bad. Accordingly, by installing the refrigerator image pickup device 2 in the vegetable room 500, the user can check picked up images of the inside of the vegetable room 500 from outside. Hence, the user can easily grasp the expiration date of food stored in the vegetable room 500, so that he/she is less likely to exceed the expiration date and waste food.
Additionally, by installing the refrigerator image pickup device 2 in the freezing room 400, the user can grasp the stored state of frozen meat to be the main dish, rice, or the like. Moreover, frozen food needs to be stored in the freezing room 400 after purchase. If the refrigerator image pickup device 2 is provided in the freezing room 400, the storage amount of the freezing room 400 can be shared among multiple users such as family members. Hence, the user can determine the purchase amount of frozen food from outside.
The refrigerator system configured in the above manner includes: the refrigerator 1000 in which a storage room that stores food is formed; the refrigerator image pickup device 2 that picks up an image of an inside of the storage room and outputs the image as a storage room image; the storage room image storage section 4 that stores the storage room image output from the refrigerator image pickup device 2; the storage room image management section 5 that stores the storage room image stored in the storage room image storage section 4; the radio communication section 6 that is capable of communicating with the outside; and the user terminal 10 capable of communicating with the radio communication section 6.
The radio communication section 6 is capable of switching between a first communication mode in which a storage room image stored in the storage room image storage section 4 is transmitted to the cloud server 9, and a second communication mode in which a storage room image stored in the storage room image management section 5 is transmitted to the user terminal 10. Additionally, the user terminal 10 receives a storage room image from the cloud server 9 when the radio communication section 6 selects the first communication mode, and receives a storage room image stored in the storage room image management section 5 from the radio communication section 6 when the radio communication section 6 selects the second communication mode.
For this reason, the home user terminal 10 can check all of the storage room images managed in the storage room image management section 5, without passing through outside communication networks such as the Internet 8. Moreover, the outside user terminal 10 can also check the latest storage room image through the cloud server 9.
Accordingly, the latest storage room image of the refrigerator main body can be checked from both a user terminal used at home where the refrigerator main body is installed and a user terminal used outside, chances of communication between the home radio communication section 6 and outside equipment can be reduced as much as possible, communication load and communication cost can be reduced, and the risk of information leakage to the Internet and security risks such as occurrence of a security hole can be reduced.
In addition, according to the following two reasons in particular, chances of communication between the home radio communication section 6 and outside equipment can be reduced drastically. Firstly, regardless of the number of user terminals 10, the outside equipment that the home radio communication section 6 communicates with is only the cloud server 9. Secondly, after the outside user terminal uploads the latest storage room image to the external server once to access the external server and check whether the storage room image is updated, no communication is necessary between the home image server and the outside external server until the next update of the storage room image.

Embodiment 2

Figs. 6 and 7 are related to Embodiment 2 of the invention, where Fig. 6 is a diagram for describing a case where a storage room image is transmitted to multiple user terminals in a refrigerator system, and Fig. 7 is a diagram for describing a case where storage room images of multiple refrigerators are transmitted to a user terminal in the refrigerator system.
In Embodiment 2 described herein, image server identification information used to uniquely identify an image server from among multiple image servers is assigned to the image server, in the aforementioned configuration of Embodiment 1. Hereinafter, a refrigerator system of Embodiment 2 will be described by focusing on differences from Embodiment 1.
Fig. 6 shows an exemplar state of transmitting a storage room image from a single image server 3 to multiple home user terminals 10. As shown in Fig. 6, an image server identification number 12 is assigned to the image server 3 corresponding to a refrigerator 1000. The image server identification number 12 is image server identification information used to uniquely identify a specific image server 3 from multiple image servers 3.
The user terminal 10 can connect to the image server 3 identified by the image server identification number 12, by use of the image server identification number 12 which is image server identification information. The refrigerator system of the example includes multiple user terminals 10 including a first user terminal 10a, a second user terminal 10b, and a third user terminal 10c.
When a second communication mode is selected, the multiple home user terminals 10 can connect to the image server 3 simultaneously. In the second communication mode, when each of the first user terminal 10a, the second user terminal 10b, and the third user terminal 10c receives a storage room image of the refrigerator 1000 from the image server 3, the first user terminal 10a, the second user terminal 10b, and the third user terminal 10c specify the same common image server identification number 12 to connect to the image server 3.
Thus, when multiple user terminals 10 are connected to the image server 3 of the same refrigerator 1000, the same common image server identification number 12 is used. Hence, when setting a communication connection from the user terminals 10 to the image server 3, the same setting can be used, so that setting work can be facilitated. Additionally, when a user different from a normally using user, such as a service person visiting for repair, connects to the image server 3, the common image server identification number 12 can be used. Note, however, that in this case, a special access code for the service person different from the user who owns the refrigerator 1000 should preferably be set.
Next, Fig. 7 shows an exemplar state in which a single home user terminal 10 acquires storage room images stored in multiple image servers 3. As shown in Fig. 7, in this example, multiple refrigerators 1000 including a first refrigerator 1000a and a second refrigerator 1000b are installed at home. A first image server 3a corresponding to the first refrigerator 1000a, and a second image server 3b corresponding to the second refrigerator 1000b are provided.
A first image server identification number 12a is assigned to the first image server 3a corresponding to the first refrigerator 1000a. A second image server identification number 12b is assigned to the second image server 3b corresponding to the second refrigerator 1000b. Both of the first image server identification number 12a and the second image server identification number 12b are the aforementioned image server identification information.
As in the example of Fig. 6, the user terminal 10 can connect to the image server 3 identified by the image server identification number 12, by use of the image server identification number 12 which is image server identification information. In the example of Fig. 7, when the user terminal 10 receives a storage room image of the first refrigerator 1000a from the first image server 3a, the user terminal 10 specifies the first image server identification number 12a to connect to the first image server 3a. When the user terminal 10 receives a storage room image of the first refrigerator 1000a from the second image server 3b, the user terminal 10 specifies the second image server identification number 12b to connect to the second image server 3b.
Thus, when storage room images of multiple refrigerators 1000 are checked in a line testing in a plant before shipping of products, or at an in-store demonstration where multiple refrigerators 1000 are installed, for example, storage room images of multiple refrigerators 1000 can be displayed on a single user terminal 10.
Note that although the description has been given of the image server 3, the outside cloud server 9 may also be configured to connect with multiple user terminals 10 simultaneously. With this, a storage room image of a refrigerator 1000 of parents living far away can be checked by a child's user terminal 10, as well as by the parents' user terminal 10. Additionally, the child's user terminal 10 can check both of the storage room image of the parents' refrigerator 1000 and the storage room image of the child's own refrigerator 1000.
Other configurations are the same as Embodiment 1, and therefore descriptions will be omitted.
The refrigerator system configured in the above manner also has the same effects as Embodiment 1. Moreover, even when the system includes multiple refrigerators 1000 or multiple user terminals 10, communication settings and exchange of storage room images can be easily and surely performed.

Embodiment 3

Figs. 8 to 13 are related to Embodiment 3 of the invention, where Fig. 8 is a diagram schematically showing a home-side configuration of a refrigerator system and a configuration of a refrigerating room of a refrigerator, Fig. 9 is a diagram showing an example of a storage room image of the refrigerator system, Figs. 10 to 12 are diagrams respectively showing first to third examples of processed storage room images of the refrigerator system, and Fig. 13 is a flowchart for describing an operation of the refrigerator system.
In Embodiment 3 described herein, when a first communication mode is selected, a communication device processes a storage room image stored in a first image storage section, and then transmits the storage room image to an external server, in the aforementioned configuration of Embodiment 1 or Embodiment 2. Hereinafter, a refrigerator system of Embodiment 3 will be described by using an example based on the configuration of Embodiment 1, and focusing on differences from Embodiment 1.
As shown in Fig. 8, in the refrigerator system of Embodiment 3, an image server 3 includes an image processing section 13 and a processed image storage section 14. The image processing section 13 processes a storage room image stored in a storage room image storage section 4 which is a first image storage section. To perform image processing smoothly, the image processing section 13 preferably includes a processor capable of performing high-speed image processing. The operating frequency of such a processor is specifically 400 MHz or higher, for example. The image processing section 13 transmits the processed storage room image and image pickup time information on the storage room image to the processed image storage section 14. Specific contents of storage room image processing by the image processing section 13 will be described later.
The processed image storage section 14 is configured to store the storage room image processed by the image processing section 13 until it is transmitted by a radio communication section 6. The processed image storage section 14 acquires the storage room image processed by the image processing section 13 and image pickup time information. Then, the processed image storage section 14 stores the acquired processed storage room images by sorting them in the order of image pickup time on the basis of the image pickup time information. At this time, before storing, the processed image storage section 14 may classify the acquired processed storage room images into methods of processing by the image processing section 13.
When the first communication mode is selected, the radio communication section 6 transmits the storage room image stored in the processed image storage section 14 to an outside cloud server 9 through a router device 7 and Internet 8. Accordingly, the radio communication section 6 which is a communication device transmits an image obtained by processing a storage room image stored in the storage room image storage section 4 by the image processing section 13, to the cloud server 9 which is an external server.
The cloud server 9 receives the processed storage room image transmitted from the radio communication section 6. Then, the cloud server 9 stores the received processed storage room image. The outside user terminal 10 receives the processed storage room image from the cloud server 9 through the Internet 8.
Next, specific contents of storage room image processing by the image processing section 13 will be described with reference to Figs. 9 to 12 by use of concrete examples. First, Fig. 9 shows an example of a storage room image temporarily stored in the storage room image storage section 4. That is, Fig. 9 shows an example of a storage room image that has not yet undergone processing by the image processing section 13.
Fig. 10 shows a first example of a storage room image obtained by processing the storage room image of Fig. 9 by the image processing section 13. In the first example, the image processing section 13 performs processing to lower the resolution of the storage room image. The image processing section 13 performs processing to lower the resolution of the storage room image, by performing, on an overall storage room image as in Fig. 9, equalization among pixels within a certain range to reduce the image size, and then stretching the image to the original image size while keeping the reduced resolution. The example shown in Fig. 10 is obtained by compressing the overall image in Fig. 9 to 1/8 image size, and setting the resolution to 1/8 of the original image.
Even with such a low-resolution image, the user can roughly grasp the state inside a storage room such as a refrigerating room 100. As compared to directly uploading the overall image of Fig. 9 to the cloud server 9 through the Internet 8, the processed image of Fig. 10 can reduce communication data amount. Accordingly, by adjusting the reduction rate of resolution, communication speed at the time of image upload can be increased, and the time lag of transmission of a storage room image-transmission request and reception of a storage room image can be reduced even in communication from a remote location, for example.
Next, Fig. 11 shows a second example of a storage room image processed by the image processing section 13. In the second example, the image processing section 13 performs processing to divide the storage room image into multiple parts. The image processing section 13 divides an overall storage room image as in Fig. 9 into a preset dividing number, for example. Instead, the image processing section 13 may divide the storage room image so that the divided image size becomes a preset value or smaller. The example of Fig. 10 is obtained by dividing the overall image of Fig. 9 into 16 equal-area regions.
As compared to directly uploading the overall image of Fig. 9 to the cloud server 9 through the Internet 8, since the divided images are sequentially transmitted one by one in the processed image of Fig. 11, it is possible to reduce communication data amount for each transmission, that is, for transmitting each divided image. Accordingly, communication speed at the time of image upload can be increased. Additionally, by transmitting only a divided image including a specific region such as only the lowest shelf of the storage room and only the left side, the user can check the stored state of a desired region with less communication data amount while keeping the resolution of the image.
Fig. 12 shows a third example of a storage room image processed by the image processing section 13. In the third example, the image processing section 13 performs processing to extract parts where images of food in the storage room are picked up, from a storage room image. In the third example, the image processing section 13 previously stores an initial image where a refrigerator image pickup device 2 picked up an image of the inside of the storage room when no food was stored therein. The image processing section 13 calculates a difference between a storage room image picked up by the refrigerator image pickup device 2 and the initial image. Then, the image processing section 13 extracts a part that has changed from the initial image, as a part where food is placed.
As compared to directly uploading the overall image of Fig. 9 to the cloud server 9 through the Internet 8, since the images in which food are extracted are sequentially transmitted one by one in the processed image of Fig. 12, it is possible to reduce communication data amount for each transmission, that is, for transmitting each image in which food is extracted. Accordingly, communication speed at the time of image upload can be increased. Additionally, the user can obtain images only of food stored in the storage room of the refrigerator 1000. Hence, the user can promptly and easily check the stored food. Consequently, at a shopping destination, the user can grasp the stored state from the extracted food image, so that excessive purchase or forgetting to purchase food can be prevented.
Note that other configurations are the same as Embodiment 1 or Embodiment 2, and therefore descriptions will be omitted.
Next, an exemplar flow of an operation of the refrigerator system of Embodiment 3 configured in the above manner will be described with reference to Fig. 13. Contents of steps S11 to S17 in the flowchart of Fig. 13 are the same as the contents of steps S11 to S17 in the flowchart of Fig. 5 described in Embodiment 1. Hence, descriptions of these steps will be omitted. In the operation flow of Fig. 13 in Embodiment 3, when the first communication mode is currently selected in step S17, the processing proceeds to step S21.
In step S21, the storage room image storage section 4 transmits the last storage room image and image pickup time information stored therein to the image processing section 13. After step S21, the processing proceeds to step S22.
In step S22, the image processing section 13 performs preset image processing on the storage room image received from the storage room image storage section 4. Then, the image processing section 13 transmits the processed storage room image and image pickup time information of the storage room image to the processed image storage section 14. After step S22, the processing proceeds to step S18.
In step S18, the radio communication section 6 uploads the processed storage room image and image pickup time information stored in the processed image storage section 14 to the outside cloud server 9, through the router device 7 and the Internet 8. After step S18, the processing proceeds to step S19.
On the other hand, if the first communication mode is not selected in step S17, this means that the second communication mode is currently selected. In this case, the processing proceeds directly from step S17 to step S19 without passing steps S21, S22, and S18.
In step S19 and the following step S20, the subject of the processing differs depending on whether the processing of steps S21, S22, and S18 is performed. If the processing of steps S21, S22, and S18 is performed and the processed storage room image is uploaded to the cloud server 9, that is, if the first communication mode is selected, the cloud server 9 checks whether a storage room image-transmission request is received from the user terminal 10.
On the other hand, if the processing of steps S21, S22, and S18 is not performed, that is, if the second communication mode is selected, the radio communication section 6 checks whether a storage room image-transmission request is received from the user terminal 10.
If a storage room image-transmission request is not received from the user terminal 10, the processing returns to step S11. On the other hand, if a storage room image-transmission request is received from the user terminal 10, the processing proceeds to step S20.
In step S20, if the first communication mode is selected, the cloud server 9 transmits the processed storage room image and image pickup time information to the user terminal 10. On the other hand, if the second communication mode is selected, the radio communication section 6 transmits the storage room image and image pickup time information to the user terminal 10. Then, the user terminal 10 displays the received processed or unprocessed storage room image and image pickup time information on the terminal display portion. After step S20, the processing returns to step S11.
The refrigerator system configured in the above manner not only has the same effects as Embodiment 1 or Embodiment 2, but can also transmit an image obtained by processing a picked up storage room image by the image processing section 13 to the external server in the first communication mode, to thereby reduce communication load. Hence, the required time for connecting to the cloud server 9 through the Internet 8 can be reduced.

Industrial Applicability

The invention is applicable to a refrigerator system including a refrigerator main body in which a storage room for storing food is formed, and a camera that picks up an image of the inside of the storage room and outputs the image as a storage room image, the system configured to display the storage room image output from the camera on a terminal device.

Reference Signs List

1: Operation panel
1a: Panel display portion
2: Refrigerator image pickup device
3: Image server
3a: First image server
3b: Second image server
4: Storage room image storage section
5: Storage room image management section
6: Radio communication section
7: Router device
8: Internet
9: Cloud server
10: User terminal
10a: First user terminal
10b: Second user terminal
10c: Third user terminal
11: User authentication device
12: Image server identification number
12a: First image server identification number
12b: Second image server identification number
13: Image processing section
14: Processed image storage section
100: Refrigerating room
101: Refrigerating room door
102: Refrigerating room shelf boards
110: Chilled room
111: Chilled case
200: Ice making room
300: Switchover room
400: Freezing room
500: Vegetable room
1000: Refrigerator
1000a: First refrigerator
1000b: Second refrigerator
1001: Control board
1010: Cooling air passage

Claims

A refrigerator system comprising:
a refrigerator main body in which a storage room for storing food is formed;

a camera configured to pick up an image of an inside of the storage room and to output the image as a storage room image;

a first image storage section configured to store the storage room image output from the camera;

a second image storage section configured to store the storage room image stored in the first image storage section;

a communication device capable of communicating with outside; and

a terminal device capable of communicating with the communication device, wherein

the communication device is capable of switching between a first communication mode in which a storage room image stored in the first image storage section is transmitted to an external server, and a second communication mode in which a storage room image stored in the second image storage section is transmitted to the terminal device, and

the terminal device:
receives a storage room image from the external server when the communication device selects the first communication mode, and

receives a storage room image stored in the second image storage section from the communication device when the communication device selects the second communication mode.
The refrigerator system according to claim 1, further comprising an image server installed at home and outside of the storage room of the refrigerator main body, wherein
the first image storage section, the second image storage section, and the communication device are provided in the image server.
The refrigerator system according to claim 2, further comprising an authentication section provided in one or both of the image server and the external server, the authentication section configured to verify whether the terminal device has an authority to receive a storage room image.
The refrigerator system according to claim 2 or 3, wherein:
image server identification information used to uniquely identify the image server from among a plurality of the image servers is assigned to the image server, the image server capable of connecting with a plurality of the terminal devices when the second communication mode is selected; and

when the plurality of terminal devices connect to the image server simultaneously, the plurality of terminal devices use the same image server identification information.
The refrigerator system according to any one of claims 1 to 4, wherein
the communication device selects the first communication mode in initial settings.
The refrigerator system according to claim 5, wherein
when the first communication mode is selected, the communication device is capable of switching to the second communication mode according to a request from the terminal device used at home where the refrigerator main body is installed.
The refrigerator system according to claim 6, wherein
the communication device selects the first communication mode when a preset certain amount of time passes after selecting the second communication mode.
The refrigerator system according to any one of claims 1 to 4, wherein
the communication device
selects the second communication mode upon receipt of a storage room image-transmission request from the terminal device used at home where the refrigerator main body is installed, and

selects the first communication mode upon receipt of a storage room image-transmission request from the terminal device used outside.
The refrigerator system according to any one of claims 1 to 8, wherein
the storage room image is stored in association with refrigerator identification information used to uniquely identify the refrigerator main body in which the storage room image is picked up from among a plurality of the refrigerator main bodies, in one or both of the second image storage section and the external server.
The refrigerator system according to any one of claims 1 to 9, further comprising an image processing section configured to process a storage room image stored in the first image storage section, wherein
the communication device transmits an image obtained by processing the storage room image stored in the first image storage section by the image processing section to the external server, when the first communication mode is selected.
The refrigerator system according to claim 10, wherein
the image processing section performs processing to lower a resolution of a storage room image.
The refrigerator system according to claim 10, wherein the image processing section performs processing to divide a storage room image into multiple parts.
The refrigerator system according to claim 10, wherein
the image processing section performs processing to extract a part of a storage room image in which food in the storage room is picked up.