JP2017040425A - Cold storage and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a clear inside storage image without cloudiness with no excessive latency time or excessive cost in a cold storage having a camera for photographing the interior of the cold storage and storing an in-room image photographed by the camera.SOLUTION: A photographing communication module of a cold storage 1 starts a camera 2 for photographing the interior of the cold storage so as to take a picture inside of a cold room 10 with prescribed intervals. The photographing communication module sequentially determines the presence or absence of lens cloudiness of the camera 2 with respect to the in-room image photographed by the camera 2 with prescribed intervals. When the absence of lens cloudiness is determined, the photographing communication module stops the camera 2 so as to store the latest in-room image determined that there is no lens cloudiness.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a refrigerator and a program that include a camera that captures an image of the interior and stores an interior image captured by the camera.

  Many interior image browsing systems have been proposed in which the inside of a refrigerator is photographed with a camera and the photographed interior images can be viewed with a mobile terminal or the like.

  Also, in order to use it for surveillance camera applications and user recognition, a refrigerator that can shoot both inside and outside of the warehouse with a single camera by changing the direction of the camera, and a camera for outside photography A refrigerator equipped is also proposed.

  In Patent Document 1, a camera image taken inside a warehouse is stored in a storage device, transferred to a server connected by an electric communication line at a predetermined time, stored in the server, and an image is read from the server by operating a mobile phone. A food management apparatus that obtains and outputs to a display device of a mobile phone is disclosed.

  In Patent Document 2, when a camera is attached to an opening / closing door and the door is closed, the interior is imaged, a change in the stored item is detected from the image in the warehouse, and when there is a change, it is transmitted to the server and carried. An article monitoring system that notifies a terminal and transmits image data to a mobile terminal in response to a request is disclosed.

JP 2002-236798 A JP 2007-046834 A

  However, since the camera for in-compartment photography is installed in the refrigerator, it is always cooled in the same manner as the articles in the interior. When the camera is mounted at a position that is in contact with outside air, such as an open / close door, for example, the lens is clouded by contact with outside air when the door is opened. In the conventional technology, since fogging of the lens is not taken into consideration at the time of shooting, there is a problem that when shooting is performed with the door closed, a clear image cannot be shot due to the fogging of the lens.

  Since the fogging of the lens changes depending on the temperature / humidity difference inside and outside the refrigerator and the opening time of the door, it is necessary to set a long waiting time in consideration of the worst case when waiting for the fogging to naturally disappear due to the cool air inside the cabinet after the door is closed. There is a problem that it is difficult to shoot because there is. When trying to optimize the waiting time, it is necessary to detect the temperature and humidity inside and outside the refrigerator and the opening time of the door, and to previously store the clouding elimination time as a table from these detected values. There are problems that extra sensors and memories are required and cost is increased, and that the presence or absence of fogging of the lens is not directly detected, so that accuracy is lacking.

  The present invention has been made in view of the circumstances as described above, and in a refrigerator that includes a camera that captures an image of the interior of the refrigerator and stores an image of the interior captured by the camera, extra waiting time and extra costs are incurred. The object is to obtain a clear image without cloudiness.

  In order to achieve the above object, a refrigerator according to the present invention includes one or more cameras, a cloudiness determination unit, and an image storage unit. The camera takes pictures of the inside of the refrigerator at predetermined intervals. The cloudiness determination unit determines whether or not the lens of the camera is clouded from the inside image captured by the camera. The image storage unit stores the latest internal image that the cloudiness determination unit determines that the camera lens is not cloudy.

  According to the present invention, in a refrigerator that includes a camera that captures the inside of a refrigerator and that stores an in-store image captured by the camera, the lens is not clouded by determining whether the lens is clouded from the interior image captured by the camera. By storing the latest image in the warehouse determined as “no”, it is possible to obtain a clear image in the warehouse without cloudiness without extra waiting time and extra cost.

It is a figure which shows the structural example of the refrigerator which concerns on Embodiment 1 of this invention. 3 is a diagram illustrating an example of a functional configuration of an imaging communication module according to Embodiment 1. FIG. 2 is a diagram illustrating an example of a hardware configuration of an imaging communication module according to Embodiment 1. FIG. 3 is an example of a top view of the refrigerator according to Embodiment 1. FIG. 6 is a diagram illustrating an example of a pixel array of an in-compartment image according to Embodiment 1. FIG. It is a figure which shows an example of the change of the high frequency component of the luminance signal of the image in a store | warehouse | chamber which concerns on Embodiment 1. FIG. It is a block diagram which shows the structural example of the in-box image browsing system which concerns on Embodiment 1. FIG. 6 is a flowchart illustrating an example of an operation of image storage processing according to the first embodiment. 3 is a flowchart illustrating an example of an operation of image transfer processing according to the first embodiment. 6 is a flowchart illustrating an example of an operation of image browsing processing according to the first embodiment. It is a figure which shows the example of the light and dark area arrange | positioned in the store | warehouse | chamber of the refrigerator which concerns on Embodiment 2 of this invention. It is a figure which shows an example of a function structure of the imaging | photography communication module which concerns on Embodiment 3 of this invention. FIG. 10 is a diagram illustrating an example of a hardware configuration of an imaging communication module according to Embodiment 3.

  EMBODIMENT OF THE INVENTION Below, the form for implementing this invention is demonstrated in detail with reference to drawings. In addition, the same code | symbol is attached | subjected to the part which is the same or it corresponds in a figure.

(Embodiment 1)
FIG. 1 is a diagram showing a configuration example of a refrigerator according to Embodiment 1 of the present invention. The refrigerator 1 includes, for example, a compressor, an evaporator, a circulation fan, a temperature sensor, and the like (all not shown), and cools items in the warehouse. The refrigerator 1 includes a door-type refrigerator compartment 10, a drawer-type freezer compartment 20, and a drawer-type vegetable compartment 30 as a storage for storing articles such as food. The refrigerator 1 includes a photographing communication module including a camera 2 for photographing inside the refrigerator compartment 10 and an opening / closing sensor 3 that detects opening / closing of the left door 4 and the right door 5. The camera 2 uses a wide-angle camera so that the interior can be fully photographed with the door closed.

  In the example of FIG. 1, the refrigerator 1 includes three storage boxes in the order of the refrigerator compartment 10, the freezer compartment 20, and the vegetable compartment 30 from the top, but it is sufficient that the number of storage containers is one or more. It is not limited to this. The refrigerator compartment 10 of the refrigerator 1 includes one camera 2, but two or more cameras 2 may be provided. Although the refrigerator compartment 10 is provided with two doors of the left door 4 and the right door 5, the number of doors may be one or three or more. The refrigerating room 10 includes the camera 2 and the open / close sensor 3 in the vicinity of the central end of the left door 4. However, the camera 2 only needs to be provided at a position where a necessary range in the refrigerator room 10 can be photographed. May be provided at a position where the opening and closing of the left door 4 and the right door 5 can be detected.

  When the open / close sensor 3 detects that at least one of the left door 4 and the right door 5 is closed, the photographing communication module activates the camera 2 for photographing inside the refrigerator and refrigerates at a predetermined interval (for example, every second). The inside of the room 10 is photographed. The determined intervals may not be equal. The photographing communication module sequentially determines whether or not the lens of the camera 2 is cloudy with respect to the internal images photographed at a predetermined interval by the camera 2. If the photographing communication module determines that the lens is not clouded, the photographing communication module stops the camera 2 and stores the latest image in the cabinet that is determined to have no lens clouded. For example, if it is determined that the lens image is not clouded with respect to the first image captured after the camera 2 is activated, the camera 2 stops after one shooting. That is, “shooting at a predetermined interval” includes the case of stopping at one shooting.

  The photographing communication module may be configured to photograph not only the door-type refrigerator compartment 10 but also the inside of the drawer-type freezer compartment 20 and the vegetable compartment 30. In this case, the photographing communication module is provided with the camera 2 and the open / close sensor 3 for photographing inside the freezer compartment 20 and the vegetable compartment 30 as well. When the open / close sensor 3 detects that the drawers of the freezer compartment 20 and the vegetable compartment 30 are closed, the imaging communication module activates the cameras 2 in the freezer compartment 20 and the vegetable compartment 30 and sets the freezer compartment 20 and the The inside of the vegetable room 30 is photographed.

  FIG. 2 is a diagram illustrating an example of a functional configuration of the photographing communication module according to the first embodiment. The photographing communication module of the refrigerator 1 includes a door closing detection unit 11, a photographing unit 12, a cloudiness determination unit 13, an image storage unit 14, and a communication unit 15. The door closing detection unit 11 detects the closing of the left door 4 and the right door 5 using the opening / closing sensor 3. In the case of the freezer compartment 20 and the vegetable compartment 30, the door closing detection unit 11 detects that the drawer is closed using the open / close sensor 3. When the door closing detection unit 11 detects the closing of at least one of the left door 4 and the right door 5, the imaging unit 12 uses the camera 2 to image the inside of the refrigerator compartment 10 at a predetermined interval. The cloudiness determination unit 13 sequentially determines whether or not the lens of the camera 2 is cloudy with respect to the internal images captured at intervals determined by the imaging unit 12 using the camera 2. When the cloudiness determination unit 13 determines that the lens is not clouded, the imaging unit 12 stops the camera 2. The image storage unit 14 stores the latest internal image determined that the lens is not fogged. The communication unit 15 is connected to an external server and uploads the internal image stored in the image storage unit 14.

  FIG. 3 is a diagram illustrating an example of a hardware configuration of the photographing communication module according to the first embodiment. The photographing communication module of the refrigerator 1 includes a processor 111, a memory 112, and a communication interface 113 in addition to the camera 2 and the open / close sensor 3. Each function of the photographing communication module is realized by the processor 111 executing a program stored in the memory 112. The memory 112 functions as the image storage unit 14 and stores an internal image taken by the camera 2 in addition to the program. Although FIG. 3 shows an example in which the processor 111 and the memory 112 are each configured as one, the plurality of processors 111 and the plurality of memories 112 may execute each function in cooperation. The communication interface 113 has, for example, a wireless LAN communication function (IEEE802.11b / g / n), and establishes communication by connecting to a wireless LAN router. In accordance with the program, the processor 111 detects the closing from the information indicating the opening / closing state of the door obtained from the opening / closing sensor 3, the photographing unit 12 that issues an imaging control command to the camera 2, and the warehouse obtained from the camera 2. The fogging determination unit 13 that determines whether the lens of the camera 2 is fogged from the inner image, and the function of the communication unit 15 that uploads the in-chamber image stored in the memory 112 to an external server via the communication interface 113 are executed. .

  FIG. 4 is an example of a top view of the refrigerator according to the first embodiment. FIG. 4 is an overhead view of the refrigerator 1, and the internal configuration of the refrigerator 1 is indicated by a broken line. The camera 2 and the open / close sensor 3 are arranged near the center end of the left door 4, and the processor 111, the memory 112, and the communication interface 113 are arranged on the upper back of the refrigerator 1. The camera 2 conforms to, for example, USB (Universal Serial Bus) UVC (USB Video Class) standard, and is connected to the processor 111, the memory 112, and the communication interface 113 via the USB wiring 8. For example, the USB wiring 8 is laid from the camera 2 inside the left door, passed through the hinge 6 of the left door 4, and once wired under the outer box (metal casing) of the refrigerator 1, and then the wiring extraction hole 7. To the processor 111, the memory 112, and the communication interface 113 on the upper back of the refrigerator 1. In addition to the above, the camera 2 may use a network camera. In that case, an Ethernet (registered trademark) wiring is laid instead of the USB wiring. Similarly, the open / close sensor 3 is connected to the processor 111, the memory 112, and the communication interface 113. The positions of the processor 111, the memory 112, and the communication interface 113 are not limited to the example in FIG.

  Since the camera 2 is installed in the refrigerator, it is always cooled. When the user opens the door for putting in and out the article, the lens comes into contact with warm outside air, and fogging occurs. The degree of cloudiness varies depending on the temperature and humidity of the outside air and the time that the door is open. When the door is closed and begins to cool again, the haze is cleared over time. Here, a method for determining whether or not the lens of the camera 2 is clouded will be described. The cloudiness determination unit 13 of the photographing communication module of the refrigerator 1 is extracted by a high-frequency component extraction unit that extracts a high-frequency component that indicates a spatial change in a luminance signal of a pixel in the image inside the image captured by the camera 2 and a high-frequency component extraction unit. A high-frequency component difference calculation unit that calculates a time change amount of the high-frequency component, and based on the high-frequency component extracted by the high-frequency component extraction unit and the time change amount of the high-frequency component calculated by the high-frequency component difference calculation unit, Whether the lens of the camera 2 is fogged is determined.

  FIG. 5 is a diagram illustrating an example of a pixel array of the image in the warehouse according to the first embodiment. The internal image 31 has a resolution with the number of horizontal pixels H and the number of vertical pixels V. Y11 represents the upper left pixel of the internal image 31, and YVH represents the lower right pixel. The number of pixels may be determined from the resolution with which the article in the warehouse can be identified and the cost of the camera 2, for example, H = 640, V = 480. The high frequency component extraction unit of the cloudiness determination unit 13 integrates, for example, the luminance signal difference between adjacent pixels of the in-chamber image 31 over the entire screen to obtain the high frequency component of the luminance signal in the screen. Specifically, the absolute values | Y12−Y11 |, | Y13−Y12 |,... Of the pixels adjacent in the horizontal direction of the uppermost line 1 of the internal image 31 are calculated and integrated to the pixel Y1H. Then, | Y22-Y21 |, | Y23-Y22 |,... One line below is calculated, integrated up to pixel Y2H, calculated up to the bottom line V, integrated, and the integrated value of each line is calculated. The total is used as the high frequency component of the luminance signal of the image 31 in the warehouse. Next, the high-frequency component difference calculation unit of the cloudiness determination unit 13 calculates a time change of the high-frequency component of the luminance signal of the in-compartment image 31 photographed at a predetermined interval.

  Here, an example has been described in which the high-frequency component extraction unit of the cloudiness determination unit 13 obtains a high-frequency component by integrating the luminance signal difference between pixels adjacent in the horizontal direction in the internal image 31. The present invention is not limited to this, and a luminance signal difference between adjacent pixels in the vertical direction may be used, or a luminance signal difference between adjacent pixels in both the horizontal direction and the vertical direction may be used. Or it does not limit to the difference of the luminance signal between adjacent pixels, You may obtain the high frequency component by Fourier-transforming the image 31 in a store | warehouse | chamber, and you may use the band filter which extracts a high frequency component.

  FIG. 6 is a diagram illustrating an example of a change in the high-frequency component of the luminance signal of the internal image according to the first embodiment. When the closing detection unit 11 of the photographing communication module of the refrigerator 1 detects the closing, the high-frequency component of the luminance signal of the in-chamber image 31 photographed by the photographing unit 12 using the camera 2 changes as shown in FIG. . In FIG. 6, the horizontal axis represents time, and the vertical axis represents the power of the high frequency component of the luminance signal. When the closing detection unit 11 detects the closing and the time when the camera 2 first captures the interior is 0, the lens is initially cloudy, so the interior image 31 has low definition, and the original high frequency component Can hardly be reproduced, so the power of the high frequency component becomes small. As time passes, cloudiness is removed and the image 31 in the interior becomes clear. The power of the high frequency component increases, and after the cloudiness is completely eliminated, the power of the high frequency component does not increase.

  Therefore, if the amount of time change of the high frequency component of the in-compartment image 31 becomes a very small increase, it can be determined that the cloudiness is almost eliminated. However, the amount of time change of the high-frequency component of the luminance signal of the internal image 31 is small to some extent even before the clouding immediately after closing the door starts to disappear. Since it may not be possible to determine that there is no cloudiness only with the temporal change amount of the high frequency component of the luminance signal of the internal image 31, the cloudiness determination unit 13 also takes into account the power of the high frequency component of the luminance signal of the internal image 31. Judgment. Specifically, the cloudiness determination unit 13 determines that the high-frequency component extracted by the high-frequency component extraction unit is equal to or higher than the threshold TH1 and the temporal change amount of the high-frequency component calculated by the high-frequency component difference calculation unit is less than the threshold TH2. It is determined that the lens of the camera 2 is not fogged. Alternatively, the cloudiness determination unit 13 becomes cloudy on the lens of the camera 2 when the time change amount of the high frequency component calculated by the high frequency component difference calculation unit increases from a small state, decreases again, and becomes smaller than the threshold value TH2. It may be determined that there is no. In other words, the cloudiness determination unit 13 changes from the state in which the time change amount of the high frequency component calculated by the high frequency component difference calculation unit decreases to a state where the amount of change is smaller than the threshold value TH2. It may be determined that there is no cloudiness. The threshold TH1 and the threshold TH2 are preferably set based on the actual high frequency component of the camera 2 and the time variation of the high frequency component when the door is closed.

  FIG. 7 is a block diagram illustrating a configuration example of the in-compartment image browsing system according to the first embodiment. The in-compartment image browsing system 100 includes a refrigerator 1, a router 21, a network 22, a server 23, a storage 24, and a mobile terminal 25. A communication interface 113 of the photographing communication module of the refrigerator 1 is connected to the router 21 provided in the house and is connected to the server 23 via the network 22. The internal image stored in the memory 112 of the photographing communication module is uploaded to the server 23 via the communication interface 113 and accumulated in the storage 24. A user's portable terminal 25 is also connected to the network 22.

  When the user transmits the in-store browsing request to the server 23 using the mobile terminal 25, the server 23 transfers the in-store image stored in the storage 24 in response to the in-store browsing request to the mobile terminal 25. Communication between the refrigerator 1 and the router 21 and between the router 21 and the network 22 may be wireless or wired. The refrigerator 1 also includes a home appliance control target function that communicates with a HEMS (Home Energy Management System) controller that controls energy management in accordance with a predetermined communication method, for example, ECHONET Lite. May be. The server 23 may be a HEMS controller. In the example of FIG. 7, the in-compartment image browsing system 100 includes the refrigerator 1, the router 21, the server 23, and the storage 24, but the refrigerator 1, the router 21, the server 23, and the storage 24. May be two or more.

  The mobile terminal 25 may directly access the refrigerator 1 via the network 22. In this case, when the user transmits an in-store browsing request to the refrigerator 1 using the mobile terminal 25, the processor 111 of the refrigerator 1 receives the in-store browsing request via the communication interface 113. The processor 111 reads the internal image stored in the memory 112 in response to the received internal browsing request, and transfers it to the portable terminal 25 via the communication interface 113. In this case, the internal image browsing system 100 may not include the server 23 and the storage 24.

  FIG. 8 is a flowchart illustrating an example of the operation of the image storage process according to the first embodiment. The image storage process starts when the photographing communication module of the refrigerator 1 is activated. The door closing detection unit 11 of the photographing communication module detects the door closing of the refrigerator 1 using the opening / closing sensor 3. When the door closing detection unit 11 does not detect the door closing of the refrigerator 1 (step S11; NO), step S11 is repeated to wait for the door closing of the refrigerator 1. When the door closing detection unit 11 detects the closing of the refrigerator 1 (step S11; YES), the imaging unit 12 images the inside of the refrigerator compartment 10 using the camera 2 (step S12). Here, it is assumed that the interior of the refrigerator 1 is illuminated to an illuminance suitable for photographing before photographing with a lighting fixture (not shown) and is turned off at a predetermined timing after photographing is completed. The high-frequency component extraction unit of the cloudiness determination unit 13 extracts the high-frequency component of the image in the cabinet photographed by the photographing unit 12 using the camera 2 (step S13), and the high-frequency component difference calculation unit is extracted by the high-frequency component extraction unit. The amount of time change of the high frequency component is calculated (step S14). The cloudiness determination unit 13 determines whether or not the high-frequency component extracted by the high-frequency component extraction unit is equal to or greater than the threshold TH1 and the temporal change amount of the high-frequency component calculated by the high-frequency component difference calculation unit is less than the threshold TH2. S15).

  If the high-frequency component is less than the threshold TH1 or the time variation amount of the high-frequency component is greater than or equal to the threshold TH2 (step S15; NO), the process returns to step S12 and repeats steps S12 to S15. When the high-frequency component is equal to or higher than the threshold TH1 and the temporal change amount of the high-frequency component is less than the threshold TH2 (step S15; YES), the fog determination unit 13 determines that the lens is not fogged. The image storage unit 14 stores the latest internal image determined that the lens is not fogged (step S16). If the cloudiness determination unit 13 determines that the lens is not clouded, the imaging unit 12 stops the camera 2. If the photographing communication module is not powered off (step S17; NO), the process returns to step S11 and repeats steps S11 to S17. When the photographing communication module is powered off (step S17; YES), the process is terminated.

  FIG. 9 is a flowchart illustrating an example of the operation of the image transfer process according to the first embodiment. The image transfer process starts when the photographing communication module of the refrigerator 1 and the server 23 are activated. The communication unit 15 of the photographing communication module inquires of the server 23 whether or not the image in the warehouse can be uploaded (step S21). Whether or not uploading is possible is determined by the load status of the server 23 and the availability of the storage 24. If uploading is not possible (step S22; NO), the process proceeds to step S24. When uploading is possible (step S22; YES), the communication unit 15 transmits the internal image stored in the image storage unit 14 to the server 23 (step S23). The server 23 stores the internal image received from the refrigerator 1 in the storage 24. If the photographing communication module and the server 23 are not turned off (step S24; NO), the process returns to step S21 and repeats steps S21 to S24. When the photographing communication module or the server 23 is turned off (step S24; YES), the process is terminated.

  FIG. 10 is a flowchart showing an example of the operation of the image browsing process according to the first embodiment. The image browsing process starts when the server 23 and the portable terminal 25 are activated. When the application for viewing the internal image is activated by the user (step S31), the portable terminal 25 inquires of the server 23 whether the internal image can be downloaded (step S32). Whether or not download is possible depends on the load status of the server 23 and whether or not the user is a portable terminal 25 or a user permitted to download the image in the warehouse. If download is not possible (step S33; NO), the process proceeds to step S36. If download is possible (step S33; YES), the server 23 reads the internal image from the storage 24 and transmits it to the portable terminal 25. The portable terminal 25 receives the internal image (step S34) and displays the internal image on the display screen (step S35). If the power of the server 23 and the portable terminal 25 is not OFF (step S36; NO), the process returns to step S31, and steps S31 to S36 are repeated. When the photographing communication module or the server 23 is turned off (step S36; YES), the process is terminated.

  As described above, according to the refrigerator 1 of the first embodiment, the latest interior image that is determined to be free of lens fogging is determined by determining the presence or absence of lens fogging from the interior image captured by the camera 2. As a result, it is possible to obtain a clear internal image without cloudiness without taking extra waiting time. Further, since it is possible to determine whether the lens of the camera 2 is clouded without detecting the temperature / humidity difference between the inside and outside of the refrigerator 1 and the door opening time, it is possible to obtain a clear photographed image without clouding without incurring extra costs. Can do.

(Embodiment 2)
The refrigerator 1 has a variety of items and types. For example, when the amount of articles is small and the ratio of articles in the interior image is small, most of the interior image is occupied by the wall of the refrigerator compartment 10. Generally, the wall of the refrigerator compartment 10 is uniformly a single color such as white, so that the high-frequency component extracted from the in-store image is reduced. Therefore, in the second embodiment, a light and dark area composed of a high luminance part and a low luminance part is arranged in the photographing range of the camera 2 in the refrigerator 1 so that high frequency components extracted from the image in the refrigerator are increased. It is assumed that the high luminance part and the low luminance part have a luminance difference equal to or greater than a predetermined value.

  FIG. 11 is a diagram showing an example of a light and dark area arranged in the refrigerator of the second embodiment of the present invention. In the refrigerator compartment 10 of FIG. 11A, since the light and shade area is provided on the wall 101 in the warehouse within the range photographed by the camera 2, it is possible to detect the high frequency component with higher accuracy than in the case of only the article. . In the refrigerator compartment 10 of FIG. 11 (b), since the light and shade areas are provided in front of the partition shelves 102A, 102B and 102C for storing the articles in the warehouse, the light and shade areas are not shaded by the articles, and high-frequency components are detected. Can be performed with high accuracy. Other configurations are the same as those of the refrigerator 1 of the first embodiment. Further, in the case of a configuration in which the inside of the freezer compartment 20 or the vegetable compartment 30 is photographed, the photographing range of the camera 2 provided in the freezer compartment 20 or the vegetable compartment 30 is as shown in FIGS. A clear shading region may be provided.

  As described above, according to the refrigerator 1 of the second embodiment, the high-frequency component is detected by arranging the light and shade area composed of the high luminance part and the low luminance part in the photographing range of the camera 2 in the refrigerator 1. Can be performed with high accuracy.

(Embodiment 3)
In the third embodiment, a case will be described in which the refrigerator 1 includes a plurality of cameras 2 and a single in-compartment image is generated by combining captured images of the plurality of cameras 2.

  FIG. 12 is a diagram illustrating an example of a functional configuration of the imaging communication module according to Embodiment 3 of the present invention. The photographing module of the refrigerator 1 according to the third embodiment includes an image composition unit 16 in addition to the functional configuration of the photographing communication module of the refrigerator 1 according to the first embodiment. When the door closing detection unit 11 detects the closing of at least one of the left door 4 and the right door 5, the imaging unit 12 images the interior of the refrigerator compartment 10 at a predetermined interval using the plurality of cameras 2. The image synthesizing unit 16 synthesizes the captured images taken at the same timing by the photographing unit 12 using the plurality of cameras 2 to generate one internal image. The cloudiness determination unit 13 sequentially determines whether or not the lens of the camera 2 is cloudy with respect to the internal image generated by the image composition unit 16. The image storage unit 14 stores the latest internal image determined that the lens is not fogged. Other functional configurations are the same as those of the photographing communication module of the refrigerator 1 of the first embodiment.

  FIG. 13 is a diagram illustrating an example of a hardware configuration of the imaging communication module according to the third embodiment. In the example of FIG. 13, the photographing communication module of the refrigerator 1 includes two cameras 2, a camera 2 </ b> A and a camera 2 </ b> B. For example, the camera 2A captures the upper half of the refrigerator compartment 10, and the camera 2B captures the lower half. The processor 111 executes the function of the image composition unit 16 that synthesizes the captured images taken at the same timing obtained from the camera 2A and the camera 2B according to the program, and generates one interior image in which the entire refrigerator compartment 10 is captured. . The number of cameras 2 is not limited to two and may be three or more. Other hardware configurations are the same as those of the photographing communication module of the refrigerator 1 of the first embodiment.

  As described above, according to the refrigerator 1 of the third embodiment, by determining whether or not the lens is clouded from the interior image after combining the captured images, the cloudiness determination unit 13 is provided for each camera 2. The amount of calculation of the photographing communication module can be reduced. In addition, even if there is a difference in the degree of cloudiness for each camera 2, an image can be stored at an optimal timing. Further, the use amount of the memory 112 can be reduced by using one image after the synthesis.

  Further, when the above functions are realized by sharing an OS (Operating System) and an application, or by cooperation between the OS and the application, only the part other than the OS may be stored in the medium.

  Furthermore, each program can be superimposed on a carrier wave and distributed via a communication network. For example, the program may be posted on a bulletin board (BBS, Bulletin Board System) on a communication network, and the program may be distributed via the network. Then, the above-described processing may be executed by starting these programs and executing them in the same manner as other application programs under the control of the operating system.

  In the above embodiment, when the open / close sensor 3 detects the closing of at least one of the left door 4 and the right door 5, the photographing communication module of the refrigerator 1 activates the camera 2 for photographing inside and decides. The inside of the refrigerator compartment 10 is photographed at intervals. Not only this but the imaging | photography communication module of the refrigerator 1 may image | photograph regularly, or when an internal browsing request | requirement is received from the portable terminal 25, it may image | photograph. Alternatively, the photographing communication module may include an input device that receives input from the user, and may photograph at the timing when the input of the article by the user is input to the input device. In this case, the photographing communication module may not include the open / close sensor 3 and may not include the door closing detection unit 11 as a functional configuration.

  In the above embodiment, the cloudiness determination unit 13 of the photographing communication module of the refrigerator 1 determines whether or not the lens of the camera 2 is in contact with warm outside air when the door is opened, but is not limited thereto. For example, the cloudiness determination unit 13 has a value of the high-frequency component extracted by the high-frequency component extraction unit that is smaller than the value of the high-frequency component when the lens 2 has no cloudiness in a state where there is no article in the warehouse. If the value does not change, it may be determined that the camera 2 has failed.

  1 Refrigerator, 2, 2A, 2B Camera, 3 Open / close sensor, 4 Left door, 5 Right door, 6 Hinge, 7 Wire extraction hole, 8 USB wiring, 10 Refrigerated room, 11 Closed door detection unit, 12 Shooting unit, 13 Cloudy judgment Unit, 14 image storage unit, 15 communication unit, 16 image composition unit, 20 freezer room, 21 router, 22 network, 23 server, 24 storage, 25 mobile terminal, 30 vegetable room, 31 image in the cabinet, 100 image browsing in the cabinet System, 101 wall, 102A, 102B, 102C partition shelf, 111 processor, 112 memory, 113 communication interface.

Claims (9)

One or more cameras that capture the interior of the refrigerator at regular intervals;
A cloudiness determination unit that determines the presence or absence of the lens cloudiness of the camera from the image inside the room taken by the camera,
An image storage unit that stores the latest image in the warehouse that the cloudiness determination unit has determined that the camera lens is not clouded;
Refrigerator. Further comprising a door closing detection unit for detecting the door closing of the refrigerator,
2. The refrigerator according to claim 1, wherein the camera shoots the interior of the refrigerator at a predetermined interval when the closing detection unit detects the closing. The cloudiness determination unit
A high-frequency component extraction unit that extracts a high-frequency component indicating a spatial change in a luminance signal of a pixel in the internal image;
A high-frequency component difference calculating unit that calculates a time change amount of the high-frequency component extracted by the high-frequency component extracting unit;
With
When the high-frequency component extracted by the high-frequency component extraction unit is greater than or equal to a first threshold value and the temporal change amount of the high-frequency component calculated by the high-frequency component difference calculation unit is less than a second threshold value, The refrigerator according to claim 1, wherein the refrigerator is determined not to be cloudy. The cloudiness determination unit
A high-frequency component extraction unit that extracts a high-frequency component indicating a spatial change in a luminance signal of a pixel in the internal image;
A high-frequency component difference calculating unit that calculates a time change amount of the high-frequency component extracted by the high-frequency component extracting unit;
With
2. When the time change amount of the high-frequency component calculated by the high-frequency component difference calculation unit increases from a small state and then decreases again to become less than a threshold value, it is determined that the camera lens is not cloudy. Or the refrigerator of 2.   The refrigerator according to any one of claims 1 to 4, wherein the photographing range of the camera has a light and dark area including a high luminance part and a low luminance part.   The refrigerator according to claim 5, wherein the shade area is provided on a wall of a storage in the refrigerator.   The refrigerator according to claim 5 or 6, wherein the shade area is provided on a front surface of a partition shelf of a storage in the refrigerator. An image compositing unit that synthesizes captured images captured by the plurality of cameras at the same timing to generate a single image in the cabinet;
The refrigerator according to any one of claims 1 to 7, wherein the fogging determination unit determines the presence or absence of lens fogging of the camera from the internal image generated by the image composition unit. A computer connected to one or more cameras that take pictures of the interior of the refrigerator at predetermined intervals.
A cloudiness determination unit that determines the presence or absence of lens fogging of the camera, from the image in the warehouse photographed by the camera, and
An image storage unit that stores the latest image in the warehouse that the cloudiness determination unit has determined that the camera lens is not cloudy;
Program to function as.

Images