TW201819832A - Refrigerator system - Google Patents

Abstract

A refrigerator system is provided which is capable of checking the latest storeroom image from a user terminal inside and outside a house, and reducing communication opportunities on the inside and outside of a house as much as possible. Therefore, the refrigerator system includes a storeroom image storage unit (4) for storing a storeroom image of a refrigerator (1000) which is output from an in-storeroom imaging device (2), a storeroom image management unit (5) for storing the storeroom image stored in the storeroom image storage unit (4), a radio communication unit (6) and a user terminal (10). The radio communication unit (6) is capable of performing a switching operation between a first communication mode for transmitting a storeroom image stored in the storeroom image storage unit (4) to a cloud server (9) and a second communication mode for transmitting a storeroom image stored in the storeroom image management unit (5) to the user terminal (10), The user terminal (10) receives a storeroom image from the cloud server (9) in the first communication mode, and receives a storeroom image from the radio communication unit (6) in the second communication mode.

Description

   Refrigerator system   

The present invention relates to a refrigerator system.

There is known a refrigerator system (for example, refer to Patent Document 1), which includes a home network device, which includes a food storage library for photographing a photographing device in the refrigerator, stores a captured image in the food storage refrigerator, and communicates with the information network. The local database connected to the road, through the mobile information terminal to access the home network device through the information network such as the Internet, download the food image stored in the refrigerator in the local database and display it On the monitor.

    Advance technical literature         Patent literature    

Patent Document 1: Japanese Patent Application Laid-Open No. 2002-243335

However, in the refrigerator system shown in Patent Document 1, when a user terminal serving as a mobile information terminal confirms an image in a storage room of a refrigerator serving as a food storage room (hereinafter referred to as a "storage room image"), The user terminal is always connected to the local database in the home network through an external information network such as the Internet. Therefore, first, the communication opportunities of the communication devices in the house and the devices outside the house where the refrigerator body is installed are increased, and the communication load and communication cost are increased. In addition, because there are more communication opportunities between the communication devices in the house and the machines outside the house, there is a concern that the security risks such as the risk of information leakage to the Internet and the occurrence of security breaches will increase.

The present invention is to solve the problems described above. The purpose is to obtain a refrigerator system, which can confirm the latest storage room image of the refrigerator body from user terminals in the house and outside the house where the refrigerator body is installed, and minimizes the need for communication devices in the house and equipment outside the house. Opportunities for communication can reduce communication load and communication costs, and reduce the risk of information leakage to the Internet and security risks such as security breaches.

A refrigerator system according to the present invention includes: a refrigerator body forming a storage room for storing food; a camera that captures the interior of the storage room and outputs it as a storage room image; and stores a first image of the storage room image output from the storage device. A memory unit; a second image memory unit that stores the storage room images stored in the first image memory unit of the preamble; a communication device that can communicate with the outside; a terminal device that can communicate with the preamble communication device; Switching in the communication method described later: the first communication method of transmitting the storage room image stored in the first image storage unit of the preamble to an external server, and transmitting the storage room image stored in the second image storage unit of the preamble to the preamble terminal The second communication method; the preamble terminal device, when the preamble communication device selects the preamble first communication method, receives the storage room image from the preamble external server, and when the preamble communication device selects the preamble second communication method, Receive the storeroom image stored in the second image memory section of the previous note.

In the refrigerator system of the present invention, it is possible to confirm the latest storage room image of the refrigerator body from user terminals inside and outside the house where the refrigerator body is installed, and to minimize the need for communication between the communication device in the house and the equipment outside the house. Opportunities to reduce the communication load and communication costs, and reduce the risk of information leakage to the Internet and security risks such as security breaches.

1‧‧‧ operation panel

1a‧‧‧Panel display section

2‧‧‧ camera inside the refrigerator

3‧‧‧Image Server

3a‧‧‧1st image server

3b‧‧‧2nd image server

4‧‧‧Storage Room Video Memory Section

5‧‧‧Storage room image management department

6‧‧‧Wireless Communication Department

7‧‧‧ routing device

8‧‧‧ internet

9‧‧‧ Cloud Server

10‧‧‧user terminal

10a‧‧‧1st user terminal

10b‧‧‧Second user terminal

10c‧‧‧3rd user terminal

11‧‧‧Personal authentication device

12‧‧‧Image server identification number

12a‧‧‧The first image server identification number

12b‧‧‧Second image server identification number

13‧‧‧Image Processing Department

14‧‧‧ Processing Image Memory

100‧‧‧ Refrigerator

101‧‧‧Refrigerator door

102‧‧‧Refrigerator Shelving

110‧‧‧cooling room

111‧‧‧cooling box

200‧‧‧ Ice Making Room

300‧‧‧ Switch Room

400‧‧‧freezer

500‧‧‧ Vegetable Room

1000‧‧‧ refrigerator

1000a‧‧‧The first refrigerator

1000b‧‧‧2nd refrigerator

1001‧‧‧Control Board

1010‧‧‧cooling air path

FIG. 1 is a front view of a refrigerator included in a refrigerator system according to Embodiment 1 of the present invention.

Fig. 2 is a diagram schematically showing the structure of the inside of a house and the structure of a refrigerator compartment of a refrigerator according to a first embodiment of the present invention.

Fig. 3 is a diagram illustrating a first communication method of the refrigerator system according to the first embodiment of the present invention.

Fig. 4 is a diagram illustrating a second communication method of the refrigerator system according to the first embodiment of the present invention.

Fig. 5 is a flowchart illustrating the operation of the refrigerator system according to the first embodiment of the present invention.

FIG. 6 is a diagram illustrating a case where a storage room image is transmitted to a plurality of user terminals in a refrigerator system according to Embodiment 2 of the present invention.

FIG. 7 is a diagram illustrating a case where a plurality of refrigerator storage room images are transmitted to a user terminal in a refrigerator system according to Embodiment 2 of the present invention.

Fig. 8 is a diagram schematically showing the configuration of the inside of a house and the configuration of a refrigerator compartment of a refrigerator in a refrigerator system according to a third embodiment of the present invention.

FIG. 9 is a diagram showing an example of a storage room image of a refrigerator system according to Embodiment 3 of the present invention.

Fig. 10 is a diagram showing a first example of a processed storage room image of a refrigerator system according to Embodiment 3 of the present invention.

Fig. 11 is a diagram showing a second example of a processed storage room image of the refrigerator system according to Embodiment 3 of the present invention.

FIG. 12 is a diagram showing a third example of a processed storage room image of the refrigerator system according to Embodiment 3 of the present invention.

Fig. 13 is a flowchart illustrating the operation of the refrigerator system according to the third embodiment of the present invention.

Embodiments for carrying out the present invention will be described with reference to the drawings. In each figure, the same or equivalent parts are denoted by the same symbols, and repeated descriptions are appropriately simplified or omitted. The present invention is not limited to the following embodiments, and various modifications can be made without departing from the spirit of the present invention.

    Embodiment 1.    

FIGS. 1 to 5 are Embodiment 1 of the present invention. FIG. 1 is a front view of the refrigerator included in the refrigerator system of Embodiment 1. FIG. 2 is a diagram showing the interior of the refrigerator system of Embodiment 1 FIG. 3 is a diagram illustrating a configuration of a refrigerator compartment of a refrigerator, FIG. 3 is a diagram illustrating a first communication method of the refrigerator system of the first embodiment, and FIG. 4 is a diagram illustrating a second communication method of the refrigerator system of the first embodiment. 5 is a flowchart illustrating the operation of the refrigerator system of the first embodiment.

The refrigerator system according to the first embodiment of the present invention includes a refrigerator 1000. The refrigerator 1000 is a refrigerator body in the refrigerator system according to the first embodiment of the present invention. In the first embodiment, the refrigerator 1000 includes a refrigerator compartment 100, an ice-making compartment 200, a switching compartment 300, a freezing compartment 400, and a fruit and vegetable compartment 500. The refrigerating compartment 100, the ice-making compartment 200, the switching compartment 300, the freezing compartment 400, and the fruit and vegetable compartment 500 are storage compartments for storing food.

As shown in Figure 1, these storage rooms From top to bottom, the refrigerator compartment 100, the ice-making compartment 200, the switching compartment 300, the freezing compartment 400, and the fruit and vegetable compartment 500 are arranged in this order. The ice making chamber 200 and the switching chamber 300 are arranged side by side at the same vertical position. In front of the refrigerating compartment 100, a refrigerating compartment door 101 that opens and closes the refrigerating compartment 100 is provided. The refrigerator compartment door piece 101 is an example of a door piece which opens and closes a storage room. In addition, the ice-making chamber 200, the switching chamber 300, the freezing chamber 400, and the fruit and vegetable chamber 500 are configured to be able to be pulled out to the front side of the refrigerator 1000 together with door pieces provided in front of each of them.

An operation panel 1 is provided in front of the refrigerator compartment door 101. The operation panel 1 includes a panel display portion 1a. The panel display portion 1a is a main body display portion capable of displaying information. The panel display portion 1a can display, for example, information in the refrigerator such as the set temperature and current temperature of each storage room. The panel display unit 1a includes, for example, a liquid crystal display. The operation panel 1 includes operation units such as a touch panel, buttons, and switches. The user can input necessary information by operating the operation section of the operation panel 1. In addition, the installation position of the operation panel 1 is not limited to the door sheet of the refrigerating compartment 100, and may also be a door sheet of another storage compartment, a side surface of the refrigerator 1000, or the like.

As shown in FIG. 2, the front side (left side when facing the drawing) of the refrigerator compartment 100 is closed by the refrigerator compartment door 101. The cross-sectional view shown in Fig. 2 is along the A-A 'cross-section in Fig. 1. A plurality of refrigerator compartment shelves 102 are provided inside the refrigerator compartment 100. The inside of the refrigerator compartment 100 is divided into a plurality of spaces (layers) in the vertical direction by the refrigerator compartment shelf 102 described above. The space below the refrigerating compartment shelf 102 on the lowest level is the cooling compartment 110. A cooling box 111 is provided inside the cooling chamber 110. The cooling box 111 can be pulled out along a guide member (not shown) such as a rail. In addition, a cooling air path 1010 is formed on the back side (right side when facing the drawing) of the refrigerator compartment 100.

On the inside surface of the refrigerator compartment door 101, an in-fridge imaging device 2 is provided. The imaging device 2 in the refrigerator is a camera that captures the inside of a storage room (here, the refrigerating room 100) and outputs an image of the storage room. The field of view of the imaging device 2 in the refrigerator is a range from the top surface inside the refrigerator compartment 100 to the cooling compartment 110. That is, the storage room image captured and output by the in-fridge imaging device 2 can take the same viewpoint as the user who views the inside of the refrigerator compartment 100 from the side of the refrigerator door 101 to capture the inside of the refrigerator compartment 100.

The imaging device 2 in a refrigerator images the inside of the refrigerator compartment 100, for example, when the refrigerator compartment door 101 is opened and closed. When the refrigerator compartment door 101 is opened and closed, food in the refrigerator compartment 100 may be taken out and put, and the storage state in the refrigerator compartment 100 may be changed. The refrigerator 1000 includes a sensor capable of detecting the opening and closing of the door. This sensor is, for example, a general magnet type sensor. That is, for example, the approach of the magnet embedded in the refrigerator compartment door 101 is detected by a pair of reed switches provided on the main body side of the refrigerator 1000. The imaging device 2 in the refrigerator, for example, performs imaging while detecting that the opened refrigerator compartment door 101 is closed and then lighting the refrigerator compartment 100.

The installation position of the imaging device 2 in the refrigerator is not only the surface inside the refrigerator compartment door 101 as shown in FIG. 2. The imaging device 2 in the refrigerator may be installed at any position as long as it is a position capable of photographing storage items such as food placed on the refrigerator compartment shelf 102. The number of the imaging devices 2 in the refrigerator is not limited to one. That is, a plurality of imaging devices 2 in a refrigerator can be placed. By providing a plurality of imaging devices 2 in the refrigerator, it is possible to reduce occurrence of blind spots due to obstruction of the field of view by foods and the like stored in the refrigerator compartment 100.

The refrigerator 1000 includes a control substrate 1001. The control substrate 1001 is housed in, for example, an upper portion on the back side of the refrigerator 1000. The control board 1001 includes a control circuit and the like to perform various controls necessary for the operation of the refrigerator 1000. The control substrate 1001 includes, for example, a microcomputer, and includes a processor and a memory. The control substrate 1001 executes a program stored in the memory by a processor and executes a preset process to control the refrigerator 1000.

The refrigerator system according to the first embodiment of the present invention includes an image server 3. The image server 3 is a device for storing, managing, and transmitting an image of a storage room to the outside. The image server 3 is installed in a house where the refrigerator 1000 is provided (hereinafter referred to simply as "in the house"), and is located outside the storage room of the refrigerator 1000. 1 and 2 show examples in which the image server 3 is installed on the outer surface of the top of the refrigerator 1000. The installation position of the image server 3 is not limited to the top of the refrigerator 1000. The image server 3 may be installed on the side, back, or door of the storage room of the refrigerator 1000 as long as it is outside the storage room of the refrigerator 1000. The image server 3 may be mounted on the control board 1001.

The image server 3 has, for example, a microcomputer, that is, a processor and a memory, like the aforementioned control substrate 1001. In addition, as shown in FIG. 2, the video server 3 includes a storage room video storage unit 4, a storage room video management unit 5, and a wireless communication unit 6. The functions of the above-mentioned parts are realized by a method described later. For example, in the microcomputer of the image server 3, the processor executes a program stored in the memory and executes a predetermined process. The image server 3 is connected to, for example, the control board 1001 and operates by receiving power supply from the control board 1001.

The storage room image storage unit 4 is a first image storage unit that stores storage room images output from the imaging device 2 in the refrigerator. As described above, when the opening / closing of the refrigerator compartment door 101 is detected, the imaging device 2 in the refrigerator images the entire interior of the refrigerator compartment 100. The image capturing device 2 in the refrigerator outputs the entire image (storage room image) in the refrigerated room 100 that has been captured to the storage room image storage unit 4. The storage room image output from the imaging device 2 in the refrigerator is input to the storage room image storage unit 4. Therefore, whenever the refrigerator compartment door 101 is opened and closed (that is, whenever the storage state in the refrigerator compartment 100 may be changed), the storage room image output from the in-fridge imaging device 2 is input to the storage room image memory section 4 . In this way, the latest storage room image is always input to the storage room image storage unit 4.

The storage room image storage unit 4 temporarily stores the latest storage room image that has been input. At this time, the storage room image storage unit 4 takes the time when the storage room image is input into the storage room image storage unit 4 as the shooting time information, and temporarily stores it together with the storage room image. In addition, the shooting time information also includes information on the shooting date (month day or year month day).

The storage room image management unit 5 is a second image storage unit that stores storage room images stored in the storage room image storage unit 4 as the first image storage unit. The storage room image management unit 5 obtains, from the storage room image storage unit 4, the storage room image and shooting time information temporarily stored in the storage room image storage unit 4.

Then, the storage room image management unit 5 rearranges the acquired storage room images in the order of the shooting time and memorizes them according to the shooting time information. In this way, the storage room image captured by the imaging device 2 in the refrigerator is temporarily stored in the storage room image storage unit 4 and then moved to the storage room image management unit 5 and managed by it. In addition, the storage room image managed and memorized in the storage room image management unit 5 may be displayed on the panel display portion 1a of the operation panel 1 together with the shooting time information.

The wireless communication unit 6 is a communication device capable of communicating with the outside. The wireless communication unit 6 uses, for example, a well-known wireless communication technology to perform mutual communication of information between the image server 3 and the router 7 of the refrigerator 1000. In addition, the communication between the image server 3 and the routing device 7 is not limited to a wireless method, and a wired method may also be adopted.

As shown in FIGS. 3 and 4, the refrigerator system according to the first embodiment of the present invention includes a user terminal 10. The user terminal 10 is a terminal device provided to be able to communicate with the wireless communication section 6 as a communication device. Specifically, the user terminal 10 may be, for example, a PC, a smartphone, or a tablet terminal. In addition, one or more user terminals 10 may be provided, and a plurality of user terminals 10 may be provided.

The user terminal 10 includes a terminal display section (not shown). The terminal display section includes, for example, a liquid crystal display. In addition, the terminal display portion can also be configured with a touch panel, which can not only display information but also accept user operations. The terminal display section of the user terminal 10 can display various information including storage room images and shooting time information.

Next, it will be explained that the storage room image of the refrigerator 1000 is displayed on the terminal display portion of the user terminal 10. In order to display the storeroom image on the user terminal 10, the user terminal 10 must receive the storeroom image. The wireless communication unit 6 is configured to be switchable to two communication methods so that the user terminal 10 can receive the storage room image. That is, the wireless communication unit 6 as a communication device can be switched to the first communication method and the second communication method.

The first communication method is a communication method for transmitting a storage room image stored in the storage room image storage section 4 as the first image storage section to an external server. As shown in FIG. 3, the refrigerator 1000, the image server 3, and the router 7 are installed in a house. The routing device 7 is communicably connected to the cloud server 9 via, for example, the Internet 8. The cloud server 9 is the aforementioned external server. This figure illustrates a case where a cloud server 9 constituted by a plurality of servers (server groups) connected to the Internet 8 is used as an external server. However, the external server is not limited to the cloud server 9 and may be a single server. The Internet 8 and the cloud server 9 are located outside the house inside the house where the refrigerator 1000 is installed (hereinafter referred to simply as "outside the house").

As shown in FIG. 4, the second communication method is a communication method for transmitting the storage room image stored in the storage room image management section 5 as the second image storage section to the user terminal 10.

The wireless communication unit 6 selects the first communication method in the initial setting. Therefore, as shown in FIG. 3, the wireless communication unit 6 transmits the latest storage room image temporarily stored in the storage room image storage unit 4 to the cloud server 9 outside the house via the router 7 and the Internet 8. Then, the cloud server 9 memorizes the received storage room image. In addition, at this time, the shooting time information corresponding to the storage room image may be transmitted together, and the cloud server 9 stores the storage room image and the shooting time information.

At this time, it is conceivable that communication to the cloud server 9 is blocked, and the storage room image cannot be uploaded (transmitted) at a time, and the maximum number of retries to upload the storage room image from the wireless communication unit 6 to the cloud server 9 is set. Moreover, the wireless communication section 6 can upload immediately when the maximum number of retries has been reached.

When the wireless communication unit 6 selects the first communication method, the user terminal 10 receives a storeroom image from an external server. That is, in this case, the user terminal 10 outside the house accesses the cloud server 9 through the Internet 8. When the user performs a display operation of the storage room image on the user terminal 10, the user terminal 10 transmits a storage room image transmission request. The cloud server 9 that has received the storage room image transmission request transmits the storage room image and shooting time information memorized by itself to the user terminal 10. The user terminal 10 receives the storage room image and shooting time information from the cloud server 9 through the Internet 8. Then, the user terminal 10 displays the received storage room image and shooting time information on the terminal display unit.

The wireless communication unit 6 is configured to switch to the second communication method in response to a request from the user terminal 10 located in the house when the first communication method is selected. As shown in FIG. 4, when the second communication method is switched in response to a request from the user terminal 10 located in the house, the wireless communication unit 6 passes the storage room image stored in the storage room image management unit 5 via The routing device 7 in the house is transmitted to the user terminal 10 in the house. In addition, at this time, as in the case of the first communication method described above, information on the shooting time corresponding to the image of the storeroom may be transmitted together.

When the wireless communication section 6 selects the second communication method, the user terminal 10 receives the storage room image stored in the storage room image management section 5 from the wireless communication section 6. That is, in this case, when the user performs a display operation of the storage room image in the user terminal 10, the user terminal 10 transmits a storage room image transmission request. The wireless communication unit 6 that has received the storage room image transmission request transmits the storage room image and shooting time information stored in the storage room image management unit 5 to the user terminal 10 in the house. The user terminal 10 in the house receives the storage room image and shooting time information transmitted from the wireless communication unit 6 through the router 7 in the house. Then, the user terminal 10 displays the received storage room image and shooting time information on the terminal display section.

Here, it is preferable to use a routing device 7 having a setting function for automatically setting a wireless communication connection with the wireless communication section 6 of the video server 3 or the like. However, in order to cope with the case where the routing device 7 without the automatic setting function is used, in the second communication method, the wireless communication unit 6 can manually perform the routing device 7 and video by the user operating the user terminal 10 Setting of the wireless communication connection of the wireless communication section 6 of the server 3.

In addition, as described above, the storage room image can be confirmed using the operation panel 1 of the refrigerator 1000 in the house. Therefore, in fact, the first communication method is more often used than the second communication method. Therefore, as described above, it is preferable that the wireless communication unit 6 selects the first communication method in advance in the initial setting.

In addition, the wireless communication unit 6 may select the first communication method when a predetermined period of time has passed after the second communication method is selected. For example, if the user sets the second communication method for viewing in the house and then goes out, the latest storage room image stored in the storage room image storage unit 4 will not be uploaded to the cloud server 9. Therefore, the user cannot confirm the latest storeroom image from outside the house. In order to cope with this situation, if the second communication method is set in order to check the storeroom image in the house, it is preferable to automatically change to the first communication method after a certain period of time has passed.

In addition, the wireless communication unit 6 may select either the first communication method or the second communication method depending on whether the user terminal 10 transmitting the storage room image transmission request is outside the house or inside the house. In this case, the wireless communication unit 6 that has received the storage room image transmission request from the user terminal 10 confirms whether the user terminal 10 that transmitted the storage room image transmission request is outside or inside the house. This confirmation can be performed by the following methods: for example, confirming the transmission path of the storage room image transmission request, and confirming whether the storage room image transmission request is transmitted from inside or outside the house. The transmission path of the storage room image transmission request can be confirmed using, for example, a well-known network management command.

When receiving a storeroom image transmission request transmitted from the user terminal 10 located outside the house, the wireless communication unit 6 selects the first communication method. Therefore, the wireless communication unit 6 transmits the latest storage room image stored in the storage room image storage unit 4 to the cloud server 9. Then, the user terminal 10 outside the house receives the storage room image from the cloud server 9.

On the other hand, when a storeroom image transmission request is received from the user terminal 10 located in the house, the wireless communication unit 6 selects the second communication method. Therefore, the wireless communication unit 6 transmits the storeroom image stored in the storeroom image management unit 5 to the user terminal 10 in the house. Then, the user terminal 10 in the house receives the storeroom image from the wireless communication unit 6 via the routing device 7.

With the first communication method as described above, the user can check the storage room image of the refrigerator 1000 through the Internet 8 and the cloud server 9 even outside the house. In the second communication method as described above, only a local communication network enclosed in a house is used. Therefore, if the user inputs the access code of the routing device 7 and the address (URL) of the image server 3 into the user terminal 10, the user terminal 10 in the house can read the storage room stored in the image server 3 Information such as images.

In addition, an authentication unit may be provided in one or both of the image server 3 and the cloud server 9 to authenticate whether the user terminal 10 has the authority to receive the storeroom image. Fig. 3 shows an example in which the personal authentication device 11 is provided in the cloud server 9 as an authentication unit. The personal authentication device 11 authenticates whether the user terminal 10 has the authority to receive the storage room image of the refrigerator 1000.

Therefore, the personal authentication device 11 of the cloud server 9 memorizes the ID of the refrigerator 1000 in advance and stores the terminal device ID of the user terminal 10 having the authority to connect to the refrigerator 1000 in advance. The ID of the refrigerator 1000 is refrigerator identification information that can uniquely identify the plurality of refrigerators 1000 connected to the Internet 8. The terminal device ID is terminal device identification information that can uniquely identify the plurality of user terminals 10 connected to the Internet 8.

The personal authentication device 11 confirms whether the mobile terminal ID of the user terminal 10 has been memorized to have the ID of the refrigerator 1000 to have the authority to receive the storage room image of the refrigerator 1000. If the mobile terminal ID of the user terminal 10 has been memorized as having the receiving authority of the storage room image of the refrigerator 1000 for the ID of the refrigerator 1000, the personal authentication device 11 authenticates that the user terminal 10 has the storage room image of the refrigerator 1000. permission.

The storage room image corresponds to the ID of the refrigerator 1000 that captured the storage room image, and is stored in one or both of the storage room image management unit 5 and the cloud server 9. Here, for example, the storage room image stored in the cloud server 9 corresponds to the ID of the refrigerator 1000 that captured the storage room image.

The cloud server 9 allows the user terminal 10 authenticated by the personal authentication device 11 to have the reception authority of the storage room image of the refrigerator 1000 to allow it to receive the storage room image of the refrigerator 1000. Conversely, the cloud server 9 restricts the user terminal 10 that has not been authenticated by the personal authentication device 11 to have the storage room image of the refrigerator 1000 to receive the storage room image of the refrigerator 1000.

As described above, by having the personal authentication device 11, even if the access code (such as the SSID and encryption key) of the routing device 7 is leaked, it is possible to prevent terminals other than the registered user terminal 10 from accessing the cloud server 9 or the image server The device 3 also receives images of the storage room.

The personal authentication device 11 described above is a device that authenticates authority using an ID. The authentication method is not limited to this. In addition to the ID, biometric authentication such as fingerprints, veins, eye iris, etc., or password authentication, or a combination of these authentication methods can be used to authenticate the connection authority.

Next, an example of the operation flow of the refrigerator system comprised as mentioned above is demonstrated with reference to FIG. First, when the user opens and closes the refrigerator compartment door 101, the aforementioned sensor detects the opening and closing of the refrigerator compartment door 101. When the opening and closing of the refrigerator compartment door 101 is detected, the process proceeds from step S11 to step S12.

In step S12, the imaging device 2 in the refrigerator photographs the inside of the refrigerator compartment 100 as a storage compartment. After step S12, the process proceeds to step S13. In step S13, the imaging device 2 in the refrigerator uses the internal image of the refrigerator compartment 100 that has been captured in step S12 as a storage room image, and transfers the image to the storage room image storage unit 4. The storage room image memory unit 4 stores the storage room image captured by the imaging device 2 in the refrigerator in step S12 together with the shooting time information. After step S13, the process proceeds to step S14.

In step S14, the storage room image storage unit 4 transmits the storage room image and shooting time information received in step S13 to the storage room image management unit 5. After step S14, the process proceeds to step S15. In step S15, the storage room image management unit 5 rearranges and stores the storage room images transmitted from the storage room image storage unit 4 in the order of the shooting time based on the shooting time information. After step S15, the process proceeds to step S16.

In step S16, the storage room image management unit 5 transmits the storage room image and shooting time information to the operation panel 1. The operation panel 1 displays the received storage room image and shooting time information on the panel display unit 1a. After step S16, the process proceeds to step S17.

In step S17, the wireless communication unit 6 confirms whether or not the first communication method is currently selected. When the first communication method is selected, the process proceeds to step S18. In step S18, the wireless communication unit 6 uploads the storage room image and shooting time information temporarily stored in the storage room image storage unit 4 to the cloud server 9 outside the house through the routing device 7 and the Internet 8. After step S18, the process proceeds to step S19.

On the other hand, if the first communication method is not selected in step S17, the second communication method is now selected. In this case, the process proceeds directly from step S17 to step S19 without going through step S18.

In step S19 and the following step S20, the main body that executes the processing varies depending on whether or not the processing in step S18 is performed. When the processing of step S18 is performed, and the storage room image is uploaded to the cloud server 9 (that is, when the first communication method is selected), the cloud server 9 confirms whether or not it has been received from the user terminal 10 Storage room image transmission requirements.

On the other hand, when the processing of step S18 is not performed (that is, when the second communication method is selected), the wireless communication unit 6 confirms whether or not a request for a storage room image transmission from the user terminal 10 has been received.

When the storage room image transmission request from the user terminal 10 is not received, the process returns to step S11. On the other hand, if a storage room image transmission request from the user terminal 10 has been received, the process proceeds to step S20.

When the first communication method is selected in step S20, the cloud server 9 transmits the storage room image and the shooting time information to the user terminal 10. On the other hand, when the second communication method is selected, the wireless communication unit 6 transmits the storeroom image and the shooting time information to the user terminal 10. Then, the user terminal 10 displays the received storage room image and shooting time information on the terminal display unit. After step S20, the process returns to step S11.

In the above, the case where the imaging device 2 in the refrigerator is installed in the refrigerator compartment 100 is described as an example. However, the installation position of the imaging device 2 in the refrigerator is not limited to this example. The imaging device 2 in the refrigerator may be installed in a storage room other than the refrigerating room 100, or may be installed in a plurality of storage rooms at the same time. For example, the vegetable compartment 500 is more difficult to grasp the internal conditions than other storage compartments, and is a storage compartment that stores vegetables that are liable to deteriorate. Therefore, the in-fridge imaging device 2 is installed in the vegetable room 500 so that the user can confirm the image of the inside of the vegetable room 500 from outside the house. Therefore, the user can easily grasp the expiry date of the food stocked in the vegetable compartment 500, and can reduce the possibility of discarding the food beyond the expiry date.

In addition, by installing the in-fridge imaging device 2 in the freezer compartment 400, the user can grasp the storage status of frozen meat, rice, and the like, which are the main course ingredients of the meal. In addition, frozen food must be stored in the freezer compartment 400 after being purchased. If there is an in-fridge imaging device 2 in the freezer compartment 400, since the storage amount of the freezer compartment 400 is shared by a plurality of users such as a family, the user can judge the purchase amount of frozen food outside the house.

The refrigerator system configured as described above includes a refrigerator 1000 forming a storage room for storing food, an in-fridge imaging device 2 that captures the interior of the storage room and outputs it as an image of the storage room; and a storage room that memorizes the output from the imaging device 2 in the refrigerator Image storage room image storage section 4; storage room image storage section 5 for storage room images stored in storage room image storage section 4; wireless communication section 6 capable of communicating with the outside; and wireless communication section 6 capable of communicating User terminal 10.

In addition, the wireless communication unit 6 can switch among the communication methods described later: the first communication method of transmitting the storage room image stored in the storage room image storage section 4 to the cloud server 9 and the storage room image management section 5 The second communication method for transmitting the image of the storage room to the user terminal 10. In addition, the user terminal 10 receives the storeroom image from the cloud server 9 when the wireless communication unit 6 selects the first communication method, and when the wireless communication unit 6 selects the second communication method, it transmits the wireless communication The unit 6 receives the storeroom image stored in the storeroom image management unit 5.

Therefore, from the user terminal 10 in the house, all the storeroom images managed by the storeroom image management unit 5 can be confirmed without going through a communication network outside the house such as the Internet 8. Furthermore, the user terminal 10 outside the house can also check the latest storeroom image through the cloud server 9.

Therefore, it is possible to confirm the latest storage room image of the refrigerator main body from the user terminal in the house and outside the house where the refrigerator body is installed, and to minimize the need for communication between the wireless communication unit 6 in the house and the equipment outside the house. Opportunities for communication can reduce communication load and communication costs, and reduce the risk of information leakage to the Internet and security risks such as security breaches.

In addition, the communication opportunities between the wireless communication unit 6 in the house and the devices outside the house can be significantly reduced for two reasons, which will be described later. For one, regardless of the number of user terminals 10, the only external device that the wireless communication unit 6 in the house communicates with is the cloud server 9. Second, the user terminal outside the house accesses the external server to confirm whether the storage room image has been updated. As long as the latest storage room image is uploaded to the external server once, the next time it is stored in the storage room image. Before the update, there was no need for communication between the image server in the house and the external server outside the house.

    Embodiment 2.    

6 and 7 are Embodiment 2 of the present invention, and FIG. 6 is a diagram for explaining a case where a plurality of user terminals transmit images of a storage room in a refrigerator system, and FIG. 7 is a diagram for explaining a case where a plurality of refrigerators are used in the refrigerator system. This is a diagram of the situation where the storage room image is transmitted to the user terminal.

The second embodiment described here is that in the configuration of the first embodiment described above, image server identification information that can uniquely identify the image server from among the plurality of image servers is assigned to the image server. Hereinafter, the refrigerator system according to the second embodiment will be described focusing on the differences from the first embodiment.

FIG. 6 shows an example of a situation where a storage room image is transmitted from one video server 3 to a plurality of user terminals 10 in the house. As shown in the figure, the image server 3 corresponding to the refrigerator 1000 is assigned an image server identification number 12. The image server identification number 12 is the image server identification information that can uniquely identify the specific image server 3 from the plurality of image servers 3.

The user terminal 10 can be connected to the image server 3 specified by the image server identification number 12 using the image server identification number 12 as the image server identification information. The refrigerator system in this example includes a plurality of user terminals 10: a first user terminal 10a, a second user terminal 10b, and a third user terminal 10c.

When the second communication method is selected, a plurality of user terminals 10 in the house can be connected to the image server 3 at the same time. In the second communication method, each of the first user terminal 10a, the second user terminal 10b, and the third user terminal 10c receives the image of the storage room of the refrigerator 1000 from the image server 3, respectively. In this case, the first user terminal 10a, the second user terminal 10b, and the third user terminal 10c designate a common and identical image server identification number 12 and connect to the image server 3.

As described above, when a plurality of user terminals 10 are connected to the image server 3 of the same refrigerator 1000, the same and common image server identification number 12 is used, and thus, for example, the user terminal 10 to the image is performed. When the communication connection of the server 3 is set, the same setting content can be used, and the setting operation can be easily performed. In addition, when a user other than a user who is generally used, for example, a service person who comes to repair is connected to the image server 3, the common image server identification number 12 can be used. However, in this case, it is better to set an access code dedicated to the service person different from the user who owns the refrigerator 1000 in advance.

Next, FIG. 7 shows an example of a situation in which one user terminal 10 in the house acquires images stored in the storage rooms of the plurality of image servers 3. As shown in the figure, in this example, a plurality of refrigerators 1000 are installed in a house: a first refrigerator 1000a and a second refrigerator 1000b. A first video server 3a is provided corresponding to the first refrigerator 1000a. A second image server 3b is provided corresponding to the second refrigerator 1000b.

The first video server identification number 12a is assigned to the first video server 3a corresponding to the first refrigerator 1000a. The second image server identification number 12b is assigned to the second image server 3b corresponding to the second refrigerator 1000b. The first image server identification number 12a and the second image server identification number 12b are the aforementioned image server identification information.

As in the example of FIG. 6, the user terminal 10 may be connected to the image server 3 specified by the image server identification number 12 using the image server identification number 12 as the image server identification information. In the example of FIG. 7, when the user terminal 10 receives the storage room image of the first refrigerator 1000 a from the first image server 3 a, the user terminal 10 specifies the first image server identification number 12 a. Is connected to the first image server 3a. When the user terminal 10 receives the storage room image of the first refrigerator 1000a from the second image server 3b, the user terminal 10 designates the second image server identification number 12b and connects to the second image server 3b.

In this way, for example, in the inspection of the production line of the factory before the product is shipped, one can be used in the case of confirming the image of the storage room of the plurality of refrigerators 1000, or in the store display where the plurality of refrigerators 1000 are installed, etc. The user terminal 10 displays the storage room images of the plurality of refrigerators 1000.

In addition, the above is the video server 3 in the house, but a plurality of user terminals 10 may be connected to the cloud server 9 outside the house at the same time. With this, for example, the storage room image of the refrigerator 1000 of the parents living far away can be confirmed from the user terminal 10 of the child in addition to the user terminal 10 of the parent. In addition, from the user terminal 10 of the child, both the storage room image of the parent's refrigerator 1000 and the storage room image of the child's own refrigerator 1000 can be confirmed.

The other configurations are the same as those of the first embodiment, and descriptions thereof are omitted here.

The refrigerator system configured as described above can also exhibit the same effects as those of the first embodiment. Furthermore, when the system includes a plurality of refrigerators 1000 or a plurality of user terminals 10, communication settings and storage room image transmission and reception can be easily and reliably performed.

    Embodiment 3.    

8 to 13 are the third embodiment of the present invention. FIG. 8 is a diagram schematically showing the structure of the inside of the refrigerator system and the structure of the refrigerator compartment. FIG. 9 is an image showing the storage room of the refrigerator system. Examples of the diagram, FIGS. 10 to 12 show the first to third examples of the processed storage room image of the refrigerator system, and FIG. 13 is a flowchart illustrating the operation of the refrigerator system.

The third embodiment described here is that in the configuration of the first embodiment or the second embodiment, when the first communication method is selected, the image stored in the storage room of the first image storage unit is processed and then processed. The communication device transmits to an external server. Hereinafter, a refrigerator system according to the third embodiment will be described with a configuration based on the first embodiment as an example, focusing on the differences from the first embodiment.

In the refrigerator system of the third embodiment, as shown in FIG. 8, the video server 3 includes a video processing unit 13 and a processed video storage unit 14. The image processing unit 13 processes the storage room image stored in the storage room image storage unit 4 as the first image storage unit. In order to make the image processing process smoothly, it is preferable that the image processing unit 13 includes a processor capable of high-speed processing of the image. Specifically, the operating frequency of such a processor is, for example, 400 MHz or more. The image processing unit 13 transmits the processed storage room image and the shooting time information of the storage room image to the processing image storage unit 14. The details of the image processing processing in the storage room of the image processing unit 13 will be described later.

The processing image storage unit 14 is a device that stores the storage room image processed by the image processing unit 13 before storing it in the wireless communication unit 6. The processed image storage unit 14 obtains the storage room image and the shooting time information processed by the image processing unit 13. Then, the image storage unit 14 is processed, and the acquired storage room images are rearranged in the order of shooting time and stored. At this time, the processing image storage unit 14 may classify and store the acquired storage room images according to the processing methods of the image processing unit 13 respectively.

When the first communication method is selected, the wireless communication unit 6 transmits the storage room images stored in the processing image storage unit 14 to the cloud server 9 outside the house via the router 7 and the Internet 8. Therefore, the wireless communication unit 6 as the communication device transmits the image processed by the image processing unit 13 in the storage room image stored in the storage room image storage unit 4 to the cloud server 9 as an external server.

The cloud server 9 receives the processed storage room image transmitted from the wireless communication unit 6. Then, the cloud server 9 memorizes the received image of the processed storage room. The user terminal 10 outside the house receives the processed storage room image from the cloud server 9 through the Internet 8.

Next, referring to FIGS. 9 to 12, specific examples will be described to explain the specific content of the image processing processing in the storage room of the image processing unit 13. First, Fig. 9 shows an example of a storage room image temporarily stored in the storage room image storage unit 4. That is, FIG. 9 shows an example of a storage room image before processing by the image processing unit 13.

FIG. 10 shows a first example of a storage room image after the storage room image of FIG. 9 is processed by the image processing unit 13. In this first example, the image processing unit 13 performs processing to reduce the resolution of the storeroom image. For example, the image processing unit 13 performs an averaging process between pixels in a certain range on the entire storage room as shown in FIG. 9 to reduce the image size, maintains the reduced resolution, and enlarges the original image. Size to reduce the resolution of the storeroom image. The example shown in FIG. 10 is to compress the entire image as shown in FIG. 9 to an image size of 1/8, so that the resolution is 1/8 of the original image.

Even with such a low-resolution image, the user can roughly grasp the situation in the storage room such as the refrigerator compartment 100 and the like. Compared with the case where the entire image of FIG. 9 is directly uploaded to the cloud server 9 through the Internet 8, using the processed image of FIG. 10 can reduce the amount of communication data. Therefore, by adjusting the low reduction rate of the resolution, the communication speed at the time of image uploading can be increased, and even in a remote place, for example, the time delay after the transmission of the storage room image transmission request to the reception of the storage room image can be shortened.

Next, FIG. 11 shows a second example of the storeroom image after the processing by the image processing unit 13. In this second example, the image processing unit 13 performs a process of dividing the storage room image into a plurality of numbers. The image processing unit 13 divides the entire image of the storage room as shown in FIG. 9 into, for example, a predetermined number of divisions. Alternatively, the image processing unit 13 may divide the storage room image so that the size of the divided image is smaller than a preset value. The example shown in FIG. 10 is to divide the entire image as shown in FIG. 9 into 16 regions of equal area.

Compared with the case where the entire image of FIG. 9 is directly uploaded to the cloud server 9 through the Internet 8, the processed image of FIG. 11 can be used to sequentially transfer the divided images one by one, which can reduce the One time, that is, the amount of communication data during the transmission of one divided image. Therefore, the communication speed at the time of image upload can be increased. In addition, for example, by transmitting only a segmented image that includes a specific area (such as only the bottom section of the storage room, only the left side, etc.), the original image resolution can be maintained with a small amount of communication data, and the user can confirm that he wants Check the storage status of the area.

FIG. 12 shows a third example of the storeroom image after the processing by the image processing unit 13. In this third example, the image processing unit 13 performs processing for extracting a portion of the food in the storage room from the storage room image. In this third example, for example, the image processing unit 13 previously memorizes a state in which no food is stored at all, and the imaging device 2 in the refrigerator captures an initial image in the storage room. The image processing unit 13 calculates a difference between the storage room image and the initial image captured by the imaging device 2 in the refrigerator. Then, the image processing unit 13 extracts a portion changed from the initial image as a portion where the food is arranged.

Compared to the case where the entire image is directly uploaded to the cloud server 9 through the Internet 8 in FIG. 9, the processed image in FIG. 12 is a one-by-one transmission of the extracted food images, which can reduce the number of times (also That is, the amount of communication data transmitted by extracting one image of food. Therefore, it is possible to speed up the communication speed when uploading images. In addition, the user can obtain only the images of the food in the warehouse stored in the storage room of the refrigerator 1000. Therefore, the user can quickly and easily confirm the food in the warehouse. Therefore, for example, at a shopping place, the user can extract the images from the food to grasp the status of the warehouse, and prevent buying too much or forgetting to buy ingredients.

The other configurations are the same as those of the first embodiment or the second embodiment, and descriptions thereof are omitted here.

Next, an example of an operation flow of the refrigerator system of the third embodiment configured as described above will be described with reference to FIG. 13. The contents of steps S11 to S17 in the flowchart of FIG. 13 are the same as the contents of steps S11 to S17 in the flowchart of FIG. 5 described in the first embodiment. Therefore, the description of these steps is omitted here. In the operation flow of FIG. 13 of the third embodiment, when the first communication method is currently selected in step S17, the process proceeds to step S21.

In step S21, the storage room image storage unit 4 transmits the last storage room image and shooting time information that it has memorized to the image processing unit 13. After step S21, the process proceeds to step S22.

In step S22, the image processing unit 13 applies a preset image processing process to the storage room image received from the storage room image storage unit 4. The image processing unit 13 transmits the processed storage room image and the shooting time information of the storage room image to the processing image storage unit 14. After step S22, the process proceeds to step S18.

In step S18, the wireless communication unit 6 uploads the processed storage room image and shooting time information stored in the processing image storage unit 14 to the cloud server 9 outside the house through the routing device 7 and the Internet 8. After step S18, the process proceeds to step S19.

On the other hand, if the first communication method is not selected in step S17, the second communication method is selected now. In this case, the process proceeds directly from step S17 to step S19 without going through steps S21, S22, and S18.

In step S19 and the following step S20, the main body that executes the processing depends on whether or not steps S21, S22, and S18 are performed. When the processing of steps S21, S22, and S18 is performed and the processed storage room image is uploaded to the cloud server 9, that is, when the first communication method is selected, the cloud server 9 confirms whether the A storage room image transmission request from the user terminal 10 is received.

On the other hand, when the processes of steps S21, S22, and S18 are not performed, that is, when the second communication method is selected, the wireless communication unit 6 confirms whether or not the video transmission from the store room has been received from the user terminal 10. Claim.

When the storage room image transmission request from the user terminal 10 is not received, the process returns to step S11. On the other hand, if a storage room video transmission request has been received from the user terminal 10, the process proceeds to step S20.

In step S20, when the first communication method is selected, the cloud server 9 transmits the processed storage room image and shooting time information to the user terminal 10. On the other hand, when the second communication method is selected, the wireless communication unit 6 transmits the storeroom image and the shooting time information to the user terminal 10. Then, the user terminal 10 displays the received processed or unprocessed storeroom image and shooting time information on the terminal display section. After step S20, the process returns to step S11.

The refrigerator system configured as described above can achieve the same effect as in the first embodiment or the second embodiment. In the first communication method, the captured storage room image is transmitted to the external server through the image processed by the image processing unit 13, This can reduce the communication load. Therefore, the time required to connect to the cloud server 9 through the Internet 8 can be shortened.

    [Industrial possibilities]    

The present invention includes a refrigerator body formed with a storage room for storing food, and a camera that captures the interior of the storage room and outputs it as a storage room image, and can be used in a refrigerator system that displays the storage room image output from the camera on a terminal .

Claims (13)

    A refrigerator system includes a refrigerator body forming a storage room for storing food, a camera that captures the interior of a storage room and outputs it as a storage room image, and a first image storage unit that stores the storage room image output from the storage system. ; The second image memory unit that stores the storage room images stored in the first image memory unit of the preamble; the communication device that can communicate with the outside; the terminal device that can communicate with the preamble communication device; the preamble communication device, which can be communicated later Switching between methods: The first communication method of transmitting the storage room image stored in the first image storage unit of the preamble to an external server, and transmitting the storage room image stored in the second image storage unit of the preamble to the predecessor terminal device. The second communication method; the preamble terminal device, when the preamble communication device selects the preamble first communication method, receives the storage room image from the preamble external server; when the preamble communication device selects the preamble second communication method, it receives from the preamble communication device. Storeroom image memorized in the second image memory section of the preamble.         The refrigerator system described in item 1 of the scope of patent application, further includes an image server installed in the house and outside the storage compartment of the refrigerator in the refrigerator, a first image memory unit, a second image memory unit, and a communication unit. The device is set in the previous image server.         For example, the refrigerator system described in item 2 of the patent application scope further includes an authentication unit, which is installed in one or both of the pre-image server and pre-external server, and verifies whether the pre-terminal device has the authority to receive images from the storage room. .         For example, the refrigerator system described in item 2 or 3 of the scope of patent application, the preamble image server is assigned image server identification information that can uniquely identify the image server from a plurality of preamble image servers, and when the preamble number is selected, 2 In the case of the communication method, it can be connected to a plurality of preamble terminal devices at the same time. When the plurality of preamble terminal devices are connected to the preamble video server at the same time, the preamble image server identification information used by the plurality of preamble terminal devices is the same.         As in the refrigerator system described in any one of claims 1 to 3 of the scope of patent application, the preamble communication device selects the preamble first communication method in the initial setting.         For the refrigerator system and preamble communication device described in item 5 of the scope of the patent application, when the preamble first communication method is selected, the preamble terminal device located in the house where the preamble refrigerator body is installed can be switched to the preamble. The second communication method.         For example, the refrigerator system and preamble communication device described in item 6 of the scope of patent application, after the preamble second communication method is selected, the predetermined first communication method is selected after a predetermined time has passed.         As described in any one of claims 1 to 3 of the scope of the patent application, the preamble communication device receives the image transmission request from the storage room of the prerecord terminal device located in the house where the prerecord refrigerator is installed, and selects the prerecord. The second communication method selects the first communication method in the case of receiving a video transmission request from the storeroom terminal device located outside the house.         According to the refrigerator system described in any one of claims 1 to 3, the pre-recorded storage room image corresponds to the refrigerator identification information of the pre-recorded refrigerator body which can uniquely identify the pre-recorded refrigerator body from which a plurality of pre-recorded refrigerator images are captured. , And stored in one or both of the second image memory section of the preamble and the external server of the preamble.         The refrigerator system described in any one of claims 1 to 3 of the scope of patent application, further includes an image processing unit storing image processing in a storage room of the first image memory unit of the preamble, and a preamble communication device. In the case of the communication method, the image stored in the storage room image stored in the first image memory section of the preamble and processed by the preimage image processing section is transmitted to the preamble external server.         In the refrigerator system described in item 10 of the scope of patent application, the pre-image processing unit performs processing to reduce the resolution of the storage room image.         In the refrigerator system described in claim 10 of the scope of the patent application, the pre-image processing unit performs a process of dividing the storage room image into a plurality of pieces.         According to the refrigerator system described in claim 10, the pre-image processing unit processes the portion of the food captured and imaged in the pre-storage room from the storage room.