JP2024040654A - Storage cabinet - Google Patents

Abstract

To appropriately acquire an image of an object stored in a storage cabinet.SOLUTION: A storage cabinet comprises a rotary door comprising a door pocket, a drawer door for moving a container back and forth, and an imaging part for imaging the door pocket and/or the container. The imaging part comprises an image sensor 314, a first lens 318 to be used when a range including the door pocket is imaged, a second lens 362 to be used when a range including the container is imaged, and a lens attachment/detachment part 350 for separating the first lens 318 and/or the second lens 362 from an axis LA of light incident on the image sensor 314.SELECTED DRAWING: Figure 7

Description

FIELD OF THE INVENTION The present invention relates to storage.

As background art in this technical field, the summary of Patent Document 1 below states, "[Problem] To appropriately determine whether an article has been stored or taken out. [Solution] Article In a determination device 12 that determines whether to store or take out an article in a storage device that has at least a first storage section that stores items in a horizontal direction and a second storage section that stores items in a vertical direction, a door of the first storage section is used. , and a door determination unit 12a that determines which of the doors of the second storage section is in the open state, and when it is determined that the door of the first storage section is in the open state, the horizontal direction of the article When it is determined that the door of the second storage section is in an open state, a change in the vertical position of the article is detected, and the article is stored based on the detected change. and an entry/exit determination unit 12c that determines whether or not the vehicle is in storage.
In addition, the summary of Patent Document 2 below states, ``[Problem] To provide an imaging device that can switch the imaging state. [Solution] An optical A member 80 is arranged to be able to go in and out.The optical member 80 can be moved by a drive mechanism 81 between an insertion position where it faces the image sensor 22 and a retracted position where it does not face the image sensor 22.Optical member 80 An example is an optical plate glass with a refractive index greater than 1. When the optical member 80 is inserted between the image sensor 22 and the exit surface 63, the focal point of the prism optical system 21 is positioned at a position suitable for the first photographing state. When the optical member 80 is retracted from between the image sensor 22 and the exit surface 63, the focal point of the prism optical system 21 moves to a position suitable for the second photographing state.'' .

JP2016-23854A Japanese Patent Application Publication No. 2006-119298

By the way, in the above-mentioned technology, there is a desire to more appropriately acquire images of articles stored in a storage.

In order to solve the above problems, the storage of the present invention includes a rotating door having a door pocket, a drawer door that moves a container back and forth, and an imaging section that takes an image of the door pocket and/or the container. The imaging unit includes an image sensor, a first lens used when imaging an area including the door pocket, a second lens used when imaging an area including the container, the first lens, and and/or a lens attachment/detachment part for separating the second lens from an optical axis incident on the image sensor.

FIG. 1 is a front view of the refrigerator according to the first embodiment. It is a side view of a refrigerator. FIG. 3 is a front view of the refrigerator with left and right refrigerator compartment doors open. FIG. 3 is a plan view of the refrigerator with left and right refrigerator compartment doors open. FIG. 3 is a perspective view of the camera unit viewed diagonally from below. 6 is a sectional view taken along the line VI-VI in FIG. 5. FIG. FIG. 3 is a schematic diagram showing the configuration of an optical unit and a lens switching section. It is a system configuration diagram including a camera unit of the refrigerator. It is a figure which shows the example of photographed image data and processed image data. FIG. 7 is a diagram showing another example of photographed image data and processed image data. FIG. 3 is an explanatory diagram of exposure adjustment performed by an exposure adjustment section. FIG. 6 is an explanatory diagram of exposure adjustment in a comparative example. FIG. 7 is an explanatory diagram of another exposure adjustment performed by the exposure adjustment section. FIG. 3 is a diagram showing a shooting setting screen displayed on a mobile terminal. This is an example of a time chart of lens switching control signals, etc. This is another example of a time chart of lens switching control signals, etc. This is another example of a time chart of lens switching control signals, etc. This is another example of a time chart of lens switching control signals, etc. This is another example of a time chart of lens switching control signals, etc. This is another example of a time chart of lens switching control signals, etc. This is an example of a time chart of the shooting control state, etc. This is another example of a time chart of the shooting control state, etc. FIG. 3 is a diagram showing an example of an image viewing screen displayed on a mobile terminal. FIG. 7 is a schematic plan view of a lens switching unit applied to a second embodiment. FIG. 7 is a schematic cross-sectional view of a camera unit applied to a third embodiment. FIG. 7 is a schematic diagram showing the configuration of an optical unit and a lens switching section in a third embodiment. It is a schematic diagram showing the composition of the lens switching part in a 4th embodiment. It is a schematic diagram showing the composition of the lens switching part in a 5th embodiment. FIG. 7 is a schematic plan view of a lens switching unit applied to a sixth embodiment. FIG. 7 is a schematic diagram of an optical unit in a seventh embodiment. It is a schematic diagram of the optical unit in 8th Embodiment.

[First embodiment]
<Overall configuration of refrigerator>
FIG. 1 is a front view of a refrigerator 100 (storage) according to the first embodiment.
The refrigerator 100 is a device that cools food and the like, and includes a housing 101, doors such as refrigerator compartment doors 211 and 212 (swivel doors), and a camera unit 3 (imaging section). The housing 101 has a plurality of storage chambers therein. In the example of FIG. 1, the storage compartments of the refrigerator 100 include, in order from the top, a refrigerator compartment 121, an ice-making compartment 122 lined up on the left and right, an upper freezer compartment 123, a vegetable compartment 124, and a lower freezer compartment 125. There is.

The housing 101 has a structure in which a heat insulating material (not shown) such as urethane foam is filled between an outer box 111 made of a steel plate and an inner box (not shown) made of resin. A plurality of openings 110 (see FIGS. 2 and 4) corresponding to each chamber are provided on the front side (front side) of the housing 101. For example, when food or the like is put into the refrigerator compartment 121 through the opening 110, the refrigerator compartment doors 211 and 212 are opened. Furthermore, when the refrigerator compartment doors 211 and 212 are closed, the opening 110 of the refrigerator compartment 121 is closed. In this way, the refrigerator compartment doors 211 and 212 have the function of closing the opening 110 of the housing 101. The same applies to the drawer door described later.

The refrigerator 100 includes left and right refrigerator compartment doors 211 and 212 as French doors that form a refrigerator compartment 121 together with the housing 101 . The left refrigerator compartment door 211 (left door) is rotatable about the axis of the left end hinge 211a (see FIG. 4). The same applies to the right refrigerator compartment door 212 (right door). The refrigerator 100 also includes an ice making compartment drawer door 221 (drawer door), an upper freezing compartment drawer door 231 (drawer door ), a vegetable compartment drawer door 241 (drawer door), and a lower freezer compartment drawer door 251 (drawer door). Hereinafter, these may be referred to as "drawer doors 221, 231, 241, 251."

The refrigerator compartment 121 is provided with a plurality of shelf boards 213 (see FIG. 3) that partition the refrigerator compartment 121 into predetermined areas. A plurality of door pockets 211c (see FIG. 3) for storing foods, etc. are provided on the inside of the left refrigerator compartment door 211 (the same applies to the right refrigerator compartment door 212). The ice-making compartment 122 is provided with an ice-making compartment container 223 (container) that is pulled out together with the ice-making compartment drawer door 221 . Similarly, the upper freezer compartment 123 is provided with an upper freezer container 233 (container), the vegetable compartment 124 is provided with a vegetable compartment container 243 (container), and the lower freezer compartment 125 is provided with a lower freezer container 253 (container) is provided. Note that these containers may be collectively referred to as "containers 223, 233, 243, and 253."

Although not shown, the refrigerator 100 also includes a compressor, a radiator (condenser), a capillary tube (throttling mechanism), and an evaporator. The refrigerant then circulates through the compressor, radiator, capillary tube, and evaporator in sequence, and the air in the storage room is cooled by heat exchange with the refrigerant flowing through the evaporator. There is. The camera unit 3 shown in FIG. 1 takes an image of the storage room of the refrigerator 100, and is installed on the top surface of the housing 101.

FIG. 2 is a side view of refrigerator 100.
As shown in FIG. 2, the camera unit 3 includes a main body part 31 and a support part 32. The main body 31 has a function of capturing images of the storage room and the like. The support section 32 supports the main body section 31 and is installed on the top surface of the housing 101.

An optical unit 300 is provided near the front end of the main body portion 31 . The optical unit 300 includes optical elements such as lenses that refract and focus light. Such an optical unit 300 includes, for example, a fisheye lens. Then, when the refrigerator compartment doors 211 and 212 (see FIG. 1) are opened, the optical unit 300 is placed in a state facing downward so that the camera unit 3 can take an image of the refrigerator compartment 121 (see FIG. 1). ing.

As shown in FIG. 2, the optical unit 300 is located in front of the front end of the housing 101 (the opening 110 of the housing 101). More preferably, the optical unit 300 is located further in front of the front surfaces of the closed refrigerator compartment doors 211 and 212 (see FIG. 1). Thereby, for example, when the refrigerator compartment doors 211 and 212 are opened, the refrigerator compartment 121 can easily come into the field of view of the optical unit 300. Note that it is also possible to image the vegetable compartment 124 and the like (see FIG. 1) using the camera unit 3 while the vegetable compartment drawer door 241 and other drawer doors are opened. In this way, the optical unit 300 is provided at a position where each storage chamber can be viewed from above.

FIG. 3 is a front view of the refrigerator 100 with the left and right refrigerator compartment doors 211 and 212 open.
As described above, the inner plates 211b, 212b of the left and right refrigerator doors 211, 212 are provided with a plurality of door pockets 211c, 212c for storing food, etc. 212 is opened, food and the like in the refrigerator compartment 121 and the door pockets 211c and 212c come into the field of view of the optical unit 300 of the camera unit 3 in a bird's-eye view.

The rotating partition 211d shown in FIG. 3 is a partition for suppressing cold air leakage from the gap between the refrigerator compartment doors 211 and 212. In the example of FIG. 3, in the left refrigerator compartment door 211, a rotating partition 211d is provided at the end opposite to the axis of the hinge 211a (see FIG. 4), and rotates in a predetermined manner as the refrigerator compartment door 211 opens and closes. It is supposed to be done.

Moreover, when manually photographing the refrigerator compartment 121 using the camera unit 3, the photographing button 7, which is pressed by the user, is provided on each of the left and right refrigerator compartment doors 211 and 212. In the example of FIG. 3, in the left refrigerator compartment door 211 (door), the photographing button 7 is provided at the bottom of the surface of the refrigerator compartment door 211 opposite to the hinge 211a (see FIG. 4) (right side). The same applies to the refrigerator compartment door 212). To explain from another perspective, when the left and right refrigerator doors 211, 212 are closed (see FIG. 1), one of the refrigerator compartment doors 211, 212 faces the other (rotating partition 211d in FIG. 3). A photographing button 7 is provided near the camera. Furthermore, the photographing button 7 is provided near the upper end of the drawer doors 221, 231, 241, and 251 as well.

Photographing performed when at least one of the plurality of photographing buttons 7 is pressed by the user is referred to as "manual photographing". Moreover, the photographing performed by the control unit 80 (see FIG. 8) according to the open/closed states of the refrigerator compartment doors 211, 212, the drawer doors 221, 231, 241, 251, etc. is referred to as "automatic photographing". "Automatic photography" is preferably performed when a predetermined period of time or more has elapsed after the previous photography (automatic photography or manual photography) was performed.

FIG. 4 is a plan view of the refrigerator 100 with the left and right refrigerator compartment doors 211 and 212 open.
In the example of FIG. 4, the main body 31 of the camera unit 3 is arranged slightly to the left of the center of the housing 101 of the refrigerator 100 in the left-right direction. To explain in more detail, the main body 31 of the camera unit 3 is arranged directly above the joint between the closed refrigerator compartment doors 211 and 212 (see also FIG. 1). By arranging the main body portion 31 in this manner, for example, when the refrigerator compartment doors 211 and 212 are opened, food and the like in the door pockets 211c and 212c easily enter the field of view of the camera unit 3.

Furthermore, when using the refrigerator 100, the user is often aware of the seam (boundary part) between the refrigerator compartment doors 211 and 212. Therefore, by arranging the main body part 31 directly above the joint between the refrigerator compartment doors 211 and 212, the user will not feel a sense of discomfort compared to a case where the horizontal position of the main body part 31 is outside the joint. can be suppressed.

FIG. 5 is a perspective view of the camera unit 3 viewed diagonally from below.
The main body 31 of the camera unit 3 shown in FIG. 5 includes, in addition to the optical unit 300 described above (see also FIG. 2), a lens switching section 350, a case 31b, and a camera LED 31c (Light Emitting Diode). There is. In the example of FIG. 5, the case 31b of the main body portion 31 generally has a rectangular parallelepiped shape that is elongated in the front-rear direction. A circular hole (reference numeral not shown) is provided on the lower surface near the front end of the case 31b, and the optical unit 300 is exposed through this hole. Further, a lens switching section 350 is provided on the rear side (back side) of the optical unit 300, and a camera LED 31c is further provided on the rear side (back side). Note that the lens switching section 350 switches the magnification of the optical unit 300 and the like.

The camera LED 31c irradiates light onto the refrigerator compartment 121 etc. so that the camera unit 3 can photograph the refrigerator compartment 121 (see Figure 3) and the door pockets 211c, 212c (see Figure 3) under appropriate brightness. It is a light source. Further, the camera LED 31c (illumination section) is provided outside the housing 101 of the refrigerator 100 (see FIG. 2). In this embodiment, the camera unit 3 photographs the refrigerator compartment 121 etc. (see FIG. 3) with the camera LED 31c turned on while the internal light 4 (see FIG. 8) of the refrigerator 100 is turned off. There is.

As shown in FIG. 5, by providing the optical unit 300 in front of the camera LED 31c, when the refrigerator compartment doors 211, 212, etc. are opened, the reflected light from the refrigerator compartment 121, etc. (see FIG. 3) is transmitted to the optical unit. It becomes easier to enter 300. Therefore, the food stored in the refrigerator compartment 121 etc. can be appropriately photographed. Note that the camera LED 31c is preferably located in front of the front end (opening 110 of the housing 101) of the housing 101 (see FIG. 2) of the refrigerator 100. This makes it easier for the light emitted from the camera LED 31c to enter the refrigerator compartment 121 and the like, thereby making it easier to obtain clear photographic results.

FIG. 6 is a sectional view taken along the line VI-VI in FIG.
As shown in FIG. 6, the camera unit 3 includes a case 31b, a camera LED 31c, an optical unit 300, a lens switching section 350, and a circuit board 31d. The circuit board 31d is a printed circuit board on which a camera/communication control SoC 31f (System-on-a-Chip), which will be described later, is mounted. In this way, by housing the optical unit 300, camera LED 31c, and circuit board 31d in one case 31b, the camera unit 3 can be made smaller and the length of the wiring can be reduced compared to the case where these are provided separately. Can be shortened. Furthermore, the wireless LAN unit 31g (see FIG. 8) is placed in the support section 32 in the camera unit 3, the upper part of the housing 101, the refrigerator compartment 121, etc., and is connected to the camera/communication control SoC 31f (see FIG. 8). has been done.

FIG. 7 is a schematic diagram showing the configuration of the optical unit 300 and the lens switching section 350.
In FIG. 7, the optical unit 300 includes a support substrate 312, an image sensor 314, a lens base 316, and a base lens 318 (first lens). The image sensor 314 is attached to the support substrate 312. The lens base 316 is formed into a substantially cylindrical shape and is attached to the support substrate 312. Base lens 318 is mounted on lens base 316. Thereby, light incident on the base lens 318 along the optical axis LA forms an image on the image sensor 314 via the lens base 316. Note that the optical axis LA is the central axis of the photographing range by the camera unit 3.

The image sensor 314 is an element that photoelectrically converts the light that enters through the base lens 318 and generates captured image data. As such an image sensor 314, for example, a CCD sensor (Charge Coupled Device) or a CMOS sensor (Complementary Metal Oxide Semiconductor) is used. Moreover, a wide-angle lens such as a magnifying lens or a fisheye lens can be applied to the base lens 318, for example.

Further, the lens switching section 350 (lens attachment/detachment section) includes an additional lens 362 (second lens), a lens frame 364, an actuator 366, and a driver 368. The additional lens 362 is, for example, a wide-angle lens such as a fisheye lens or a magnifying lens, and is attached to the lens frame 364.

When a fisheye lens is used as the base lens 318, the additional lens 362 has smaller distortion than the base lens 318. When the driver 368 supplies current to the actuator 366, the actuator 366 drives the lens frame 364 in the front-back direction. For example, a magnifying lens with a magnification of 2 to 3 times can be used.

Thereby, the driver 368 can insert the additional lens 362 into the optical axis LA as needed, and can separate the additional lens 362 from the optical axis LA. Hereinafter, the state in which the additional lens 362 is inserted into the optical axis LA will be referred to as the "enlargement ON state" of the lens switching unit 350, and the state in which it is separated from the optical axis LA will be referred to as the "enlargement OFF state."

FIG. 8 is a system configuration diagram including the camera unit 3 of the refrigerator 100.
As shown in FIG. 8, the refrigerator 100 includes, in addition to the camera unit 3, an interior light 4, a buzzer 5, a photographing button 7, a door sensor 10, a drawer door sensor 12, and a control section 80. ing. The interior light 4 is a light source that irradiates light into the interior of the refrigerator 100, and is provided inside the housing 101. The buzzer 5 is set at a predetermined location on the housing 101, and is sounded when, for example, the interior light 4 is turned off when photographing with the camera unit 3. The photographing button 7 (see also FIG. 3) is a button that is pressed by the user when manually photographing using the camera unit 3, as described above.

The door sensor 10 detects the open/close states of the refrigerator compartment doors 211, 212, and supplies the control unit 80 with open/close signals S211, S212 indicating the respective open/close states. Generally, a door sensor of a refrigerator detects as an "open state" if the refrigerator compartment doors 211, 212 are even slightly open, in order to warn the user if they forget to close the door. In this embodiment as well, the refrigerator 100 is equipped with such a general door sensor, but it is not particularly illustrated. The door sensor 10 shown in FIG. 8 sets the opening/closing signals S211, S212 to "1" (open state) if the opening angles of the refrigerator compartment doors 211, 212 are each a predetermined angle or more suitable for photographing. On the other hand, if the opening angles of the refrigerator compartment doors 211 and 212 are each less than the predetermined angle, opening/closing signals S211 and S212 of "0" (closed state) are output.

Further, the drawer door sensor 12 detects the open/close states of the drawer doors 221, 231, 241, and 251, and supplies the control unit 80 with open/close signals S221, S231, S241, and S251 indicating the respective open/close states. These opening/closing signals S221, S231, S241, S251 become "1" (open state) if the pull-out distance of the corresponding drawer door is equal to or longer than a predetermined length suitable for photographing the containers 223, 233, 243, 253, This signal becomes "0" (closed state) if the pull-out distance is less than the predetermined length. The control unit 80 performs various automatic photographing based on these opening/closing signals S211, S212, S221, S231, S241, and S251.

The control unit 80 is, for example, a microcomputer, and is configured to include electronic circuits such as a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and various interfaces, although not shown. . Then, the program stored in the ROM is read out and expanded to the RAM, and the CPU executes various processes. In addition to performing the above-described "automatic photographing", the control unit 80 performs "manual photographing" when the photographing button 7 is pressed. The control unit 80 includes a lens switching control unit 81 (magnification setting unit), an exposure adjustment unit 82, an open/close determination unit 83, a shooting setting unit 84, and a shooting control unit 85.

Lens switching control section 81 outputs a lens switching control signal SAD. The lens switching control signal SAD is a binary signal of ON/OFF, and controls the enlargement ON/OFF state of the lens switching section 350. The exposure adjustment section 82 adjusts the exposure of the camera unit 3. The open/close determination unit 83 determines the open/close states of the refrigerator compartment doors 211, 212 and the drawer doors 221, 231, 241, 251 based on the open/close signals S211, S212, S221, S231, S241, S251.

The shooting setting unit 84 sets the priorities of the refrigerator compartment doors 211, 212 and the drawer doors 221, 231, 241, 251 (details will be described later), and the ON/OFF status of automatic shooting and manual shooting (automatic shooting and manual shooting (whether to allow or not) etc. The photographing control unit 85 performs various photographing controls.

As described above, the camera unit 3 includes the camera/communication control SoC 31f. This is an integrated circuit designed to integrate circuits with microcomputer functions and other applied functions onto a single chip, and to work together. The camera/communication control SoC 31f performs predetermined communication with the control unit 80 and outputs a shooting command to the image sensor 314 (see FIG. 7). As a result, the captured image data GS is input from the image sensor 314 to the camera/communication control SoC 31f. Note that the camera/communication control SoC 31f is an example of an integrated circuit, and other types of circuits may be used.

The wireless LAN unit 31g transmits photographed image data GS output from the camera/communication control SoC 31f to the router 51. The router 51 is a communication relay device, and transmits the captured image data GS received from the wireless LAN unit 31g to the server 53 via the network 52. The server 53 performs predetermined processing on the photographed image data GS, and stores the processed image data GM that is the result in association with the identification information of the refrigerator 100. Moreover, when the server 53 receives a request signal for an internal image of the refrigerator 100 from the user's mobile terminal 54, the server 53 transmits the processed image data GM to the mobile terminal 54. As such a mobile terminal 54, in addition to a mobile phone or a smartphone, a tablet, a wearable terminal, or the like is used.

<Summary of operation of lens switching control unit 81>
Here, it is assumed that a fisheye lens is used as the base lens 318 and a magnifying lens is used as the additional lens 362. The fisheye lens widens the field of view and is therefore suitable for imaging the door pockets 211c and 212c, and the magnifying lens is suitable for imaging distant areas, so the containers 223, 233, 243, 253 are taken in and out together with the drawer doors 221, 231, 241, 251. Suitable for imaging. When the base lens is a magnifying lens and the additional lens is a wide-angle lens, the following magnification ON/OFF settings should be read in reverse. Note that the inventors have found that in order to properly image both the door pockets 211c and 212c and the containers 223, 233, 243, and 253, the base lens should be a magnifying lens and the additional lens should be a wide-angle lens. is preferred.

The lens switching control unit 81 of the control unit 80 switches the lens switching unit 350 (see FIG. 7 (see) is set to the enlargement OFF state. Furthermore, when either of the drawer doors 221, 231, 241, or 251 becomes open while the refrigerator compartment doors 211 and 212 are both closed, the lens switching control unit 81 turns the lens switching unit 350 into the enlargement ON state. Set. In addition, when either of the refrigerator compartment doors 211 and 212 and any of the drawer doors 221, 231, 241, and 251 are opened at the same time, the lens switching control unit 81 selects the one with the highest "priority". The enlargement ON/OFF state is set with priority given to . Note that details regarding the "priority" will be described later.

FIG. 9 is a diagram showing an example (GS1, GM1) of photographed image data GS and processed image data GM.
Note that in the captured image data GS1 of FIG. 9, the upper side of the page is the front, and the bottom side of the page is the rear, so the left and right sides are reversed with respect to FIG. The photographed image data GS1 is photographed with all the drawer doors 221, 231, 241, 251 (see FIG. 1) in the closed state and with both the left and right refrigerator compartment doors 211, 212 in the open state.

As a result, the photographed image data GS1 shown in the figure was photographed with the lens switching unit 350 (see FIG. 7) in the enlargement OFF state. In the photographed image data GS1 shown in the figure, in addition to the refrigerator compartment 21, a door pocket 211c of the refrigerator compartment door 211 on the left side and a door pocket 212c of the refrigerator compartment door 212 on the right side are also photographed.

By using a fisheye lens as the base lens 318 of the optical unit 300 (see FIG. 7), the refrigerator 100 can be imaged with a wide range of angles of view. However, in the captured image data GS1 that has not been subjected to image processing, as shown in FIG. As the distance increases, the number of pixels in the captured image decreases.

Further, the processed image data GM1 is an example of the result of image processing performed by the server 53 (see FIG. 8) on the captured image data GS1. The processed image data GM1 is an image as if the refrigerator compartment 21 and the left and right refrigerator compartment doors 211 and 212 were viewed from the front. By displaying such processed image data GM1 on the mobile terminal 54 (see FIG. 8), the user can grasp at a glance what kind of food is stored in the refrigerator compartment 21 or the like.

Although the photographed image data GS1 does not include images of the containers 223, 233, 243, and 253 (see FIG. 1), if a photograph is taken with any of the drawer doors open, the image of the corresponding container will be It is included in the area AR1 indicated by the broken line in the captured image data GS1. The area AR1 is narrower than the entire area of the captured image data GS1, and has a smaller number of pixels. Therefore, even if processed image data is obtained by enlarging the area AR1, the resolution will be low, making it difficult to understand the contents of the food or the like.

FIG. 10 is a diagram showing other examples (GS2, GM2) of photographed image data GS and processed image data GM. Photographed image data GS2 indicates that the vegetable compartment drawer door 241 is opened when both the left and right refrigerator doors 211, 212 are closed and the other drawer doors 221, 231, 251 (see FIG. 1) are closed. This photo was taken to show the condition. As a result, the illustrated photographed image data GS2 is obtained by photographing mainly the inside of the crisper container 243 with the lens switching section 350 in the enlargement ON state. In the illustrated photographed image data GS2, a portion of the refrigerator compartment doors 211 and 212, the ice making compartment drawer door 221, and the upper freezer compartment drawer door 231 are also photographed.

Further, the processed image data GM2 is an example of the result of image processing performed by the server 53 (see FIG. 8) on the captured image data GS2. The processed image data GM2 is an image that looks like the vegetable compartment container 243 viewed from the front. By displaying such processed image data GM2 on the mobile terminal 54 (see FIG. 8), the user can grasp at a glance what kind of food is stored in the vegetable compartment container 243.

As described above, since the additional lens 362 (see FIG. 7), which is a magnifying lens, was inserted into the optical axis LA when photographing the photographed image data GS2, the area of the crisper container 243 in the photographed image data GS2 is not included in the photographed image. It is wider than the area AR1 in the data GS1 (see FIG. 9). Therefore, in the processed image data GM2, high resolution can be ensured for the portion of the vegetable compartment container 243, and the user can easily understand the contents of the food and the like.
In this example, the enlarged ON state is when the vegetable compartment drawer door 241 is opened, but if any of the other drawer doors 221, 231, 251 is opened instead, the enlarged ON state is also the same. Photographed as. By selecting a suitable magnification for the additional lens 362 (see FIG. 7), which is a magnifying lens, a common magnifying lens can be applied to all containers 223, 233, 243, and 253 having different heights. can do. Of course, if the complexity of the mechanism is allowed, magnifying lenses with different magnifications may be set in each container 223, 233, 243, 253.

<Operation of exposure adjustment section 82>
FIG. 11 is an explanatory diagram of exposure adjustment performed by the exposure adjustment section 82 (see FIG. 8).
The exposure adjustment section 82 adjusts the gain when amplifying the electrical signal output from the image sensor 314 (see FIG. 7). When the lens switching unit 350 (see FIG. 7) is in the enlargement OFF state, in this embodiment, the three rectangular areas 61, 62, and At least one of 63 is used. That is, at least one of the region 61 of the left refrigerator compartment door 211, the region 62 of the right refrigerator compartment door 212, and a predetermined region 63 between these regions 61 and 62 is used. Note that the positions and sizes of the regions 61, 62, 63 may be changed as appropriate based on the opening angles of the refrigerator compartment doors 211, 212.

Here, it is preferable if the gain can be adjusted to make it easier to visually recognize the area where the food material most interesting to the user is photographed. Therefore, for example, the position of the food may be determined by recognizing the feature amount of the food, and one or more areas may be set so that the food within the field of view of the image sensor 314 is included. Further, instead of the feature amount of the food, the opening/closing of the refrigerator compartment doors 211 and 212 may be sensed, and if the doors are open, the area on the open side of the door may be used.

In the example of FIG. 11, the left refrigerator compartment door 211 is open, but the right refrigerator compartment door 212 is closed. In such a case, the exposure adjustment section 82 performs the exposure adjustment based on the average value of the brightness of the region 61 corresponding to the left refrigerator compartment door 211. That is, the exposure adjustment unit 82 adjusts the gain when amplifying the electrical signal output from the image sensor 314 (see FIG. 7) so that the average value of the brightness of the region 61 is included in a predetermined range. As a result, the region 62 of the right refrigerator compartment door 212 that is in the closed state and the region 63 that is partially separated from the left refrigerator compartment door 211 are excluded from the reference when adjusting the gain. Therefore, even if the surroundings of the refrigerator compartment are dark, the exposure adjustment section 82 can appropriately adjust the exposure without increasing the gain too much.

The area to be used for exposure adjustment can be determined using the position where the food material is present. The position where the food is present may be determined using feature amounts in image recognition, or may be determined indirectly by detecting door opening information or door opening angle information. Door opening information and door opening angle information can be acquired using a known door sensor, image recognition technology, or the like. By trying to exclude areas where there is no food, which is useful information, as much as possible from the area subject to exposure adjustment, if the brightness of the area where there is no food is too bright or too dark, the white of the area where the food is It is possible to reduce the possibility of skipping or black spots.
In this way, only the above-mentioned "partial area" including the position where the food is present in the field of view of the image sensor 314 may be used for exposure adjustment. Further, when adjusting the exposure, the weight of the "partial area" may be made larger than that of the remaining area.

In this way, when one of the right refrigerator compartment door 212 or the left refrigerator compartment door 211 is opened, the exposure adjustment unit 82 sets the area corresponding to the one described above within the field of view of the camera unit 3 as a reference. , adjust the exposure gain during shooting. Further, the exposure adjustment may be performed based on the one having the highest average value of image brightness among the three regions 61 to 63. That is, the exposure adjustment unit 82 adjusts the exposure gain during photographing based on the region having the highest average value of image brightness among the plurality of regions 61 to 63 included in the field of view of the image sensor 314. This prevents the gain from becoming too high and, in turn, suppresses whiteout (halation) in the captured image data GS.

Further, exposure is adjusted based on an area including at least a part of the left and right refrigerator doors 211 and 212 to which an open signal is input from the door sensor 6, and which has the highest average image brightness. may be performed. As a result, even in a situation where, for example, light irradiated from outside the refrigerator 100 is reflected on the surface of the refrigerator compartment door 212 in a closed state, the exposure adjustment section 82 can appropriately set the gain in the exposure adjustment. As a result, it is possible to suppress the occurrence of blown-out highlights and blown-up shadows in the captured image data GS.

FIG. 12 is an explanatory diagram of exposure adjustment in a comparative example.
In the comparative example shown in FIG. 12, one rectangular area 60 is set as a reference when adjusting exposure. For example, when the left and right refrigerator compartment doors 211, 212 are opened, the rectangular area 60 is set to include both of these refrigerator compartment doors 211, 212. When one large area 60 is used in this way, when the surroundings of the refrigerator 100 are dark, the right refrigerator compartment door 212 appears dark when photographed, so the gain is set high to increase the sensitivity to light. As a result, in the photographed image, overexposure tends to occur in the left refrigerator compartment door 211 in the open state and in the left half of the refrigerator compartment 21.

On the other hand, according to the present embodiment (see FIG. 11), when the lens switching unit 350 (see FIG. 7) is in the enlargement OFF state, the exposure The exposure adjustment section 82 selects an area that will serve as a reference when making adjustments. Thereby, it is possible to suppress overexposure in the photographed image data GS and display clear processed image data GM that is easy for the user to visually recognize on the mobile terminal 54 (see FIG. 8).

FIG. 13 is an explanatory diagram of another exposure adjustment performed by the exposure adjustment section 82 (see FIG. 8).
When the lens switching unit 350 (see FIG. 7) is in the enlargement ON state, in this embodiment, any one of the areas 66, 67, and 68 shown in FIG. I try to use one. In the example of FIG. 13, the vegetable compartment drawer door 241 is in an open state, but in this case, the exposure adjustment section 82 selects the area 66. This area 66 is approximately equal to the entire area of the vegetable compartment container 243. Similarly, when the lower freezer compartment drawer door 251 (see FIG. 1) is open, the exposure adjustment section 82 selects the area 66.

On the other hand, when only the icemaker drawer door 221 is open, the exposure adjustment section 82 selects the region 67. Furthermore, when only the upper freezer compartment drawer door 231 is open, the exposure adjustment section 82 selects the region 68. In this way, the exposure adjustment unit 82 selects one of the areas 66, 67, and 68 depending on which drawer door is in the open state, so that it is possible to prevent overexposure or underexposure from occurring in the captured image data GS. It can be suppressed. Note that the positions and sizes of the regions 66, 67, and 68 may be changed as appropriate based on the pull-out distance of the draw-out doors 221, 231, 241, and 251.

Note that the wall surface of the refrigerator compartment 21 and the inner plates 211b, 212b (see FIG. 3) of the refrigerator compartment doors 211, 212 (doors) may be white, respectively. Here, "white" is a color that is reflected over substantially the entire spectrum of visible light with a reflectance of 50% or more. From another perspective, "white" means that in the RGB color system, the brightness of each element of R (Red), G (Green), and B (Blue) is approximately the same, and the reflectance is 50% or more. The color is reflective. Similarly, the interior color of the drawer doors 221, 231, 241, 251 may also be white.

When the walls of the refrigerator compartment 21 are white, there is a tendency for the photographed results to have blown out highlights. can be suppressed. In this way, according to the present embodiment, the refrigerator compartment 21 and the like can be appropriately photographed using the camera unit 3. Therefore, the user can easily check the state of the refrigerator compartment 21 and the like by looking at the mobile terminal 54.

<Operation of shooting setting section 84>
FIG. 14 is a diagram showing a shooting setting screen 70 displayed on the mobile terminal 54.
The imaging setting unit 84 (see FIG. 8) communicates with the mobile terminal 54 via the router 51, server 53, and the like. Thereby, the photographing setting section 84 displays a photographing setting screen 70 shown in FIG. 14 on the mobile terminal 54. As shown in FIG. 14, the shooting setting screen 70 includes a plurality of buttons 71, 72, 73, 76, 77, and 78.

By operating the button 71, the user can set the ON/OFF state of automatic photography of the refrigerator compartment 121 (see FIG. 3) and the door priority PDR (priority). The door priority PDR is the priority for automatic photographing of the refrigerator compartment 121. The automatic shooting ON state is a state in which automatic shooting is permitted. The automatic shooting OFF state is a state in which automatic shooting is not permitted. In addition, the priority that can be set as the door priority PDR is any of three types: H (high), M (medium), and L (low), and H (high) has the highest priority, followed by M (medium) has the highest priority, and L (low) has the lowest priority.

The lens switching control unit 81 (see FIG. 8) determines the priority when a plurality of refrigerator doors 211, 212 and drawer doors 221, 231, 241, 251 (see FIG. 3) are detected to be open. The enlargement ON/OFF state of the lens switching unit 350 is set depending on the highest one. Further, the three drawer doors 221, 231, 241 (see FIG. 3) are collectively referred to as a "middle drawer door." By operating the button 72, the user can set the ON/OFF state of automatic photographing of containers related to these middle drawer doors and the middle drawer priority PDM (priority). Here, the middle drawer priority PDM is the priority of the middle drawer (H (high), M (middle), or L (low)).

Further, by operating the button 73, the user can set the ON/OFF state of automatic imaging of the lower freezer compartment drawer door 251 (see FIG. 3) and the lower drawer priority PDL (priority). Here, the lower drawer priority PDL is the priority (H (high), M (medium), or L (low)) of the lower freezer compartment drawer door 251.

Furthermore, by operating the button 76, the user can set the ON/OFF state of manual photography of the refrigerator compartment 121. The ON state of manual photography is a state in which manual photography is permitted, and the OFF state of manual photography is a state in which manual photography is not permitted. Similarly, by operating the button 77, the user can set the ON/OFF state of manual photography of the containers related to the middle drawer door (ie, the containers 223, 233, 243). Similarly, by operating the button 78, the user can set the ON/OFF state of manual photography of the lower freezer compartment container 253. In manual photography, whether to perform photography with magnification ON or OFF is determined using the relationship between the opening and closing states of the refrigerator compartment doors 211 and 212 and the drawer doors 221, 231, 241, and 251 described in this embodiment. It can be judged based on

<Operation of lens switching control unit 81>
FIG. 15 is an example of a time chart of the lens switching control signal SAD, etc.
The door opening/closing signal SDR in the figure is the logical sum of the opening/closing signals S211, S212 (see FIG. 8) of the refrigerator compartment doors 211, 212, and is "1" if either of the refrigerator compartment doors 211, 212 is open. , this signal becomes "0" if both the refrigerator compartment doors 211 and 212 are closed. Further, the lens switching control signal SAD is a binary signal of ON/OFF as described above, and is a signal for controlling the enlargement ON/OFF state of the lens switching section 350 (see FIG. 7).
Before time t11 in FIG. 15, the door opening/closing signal SDR is "0" (closed state), and the lens switching control signal SAD is also in the enlargement OFF state. At time t11, the door opening/closing signal SDR is "1" (open state). This means that at least one of the refrigerator compartment doors 211 and 212 is in an open state. Further, at time t12, the door opening/closing signal SDR is "0" (closed state). This means that both the refrigerator compartment doors 211 and 212 are in the closed state. During the illustrated period, the lens switching control section 81 (see FIG. 8) always sets the lens switching control signal SAD to the enlargement OFF state. That is, when there is no need to switch the lens switching control signal SAD, the lens switching control unit 81 maintains the enlargement ON/OFF state as it is.

FIG. 16 is another example of a time chart of the lens switching control signal SAD, etc.
Before time t15 in FIG. 16, the door opening/closing signal SDR is "0" (closed state), and the lens switching control signal SAD is in the enlargement ON state. Then, when the door opening/closing signal SDR becomes "1" (open state) at time t15, it becomes necessary to move the additional lens 362 away from the optical axis LA in the lens switching section 350 (see FIG. 7). Therefore, the lens switching control section 81 (see FIG. 8) sets the lens switching control signal SAD to the enlargement OFF state at time t15. After that, assume that the door opening/closing signal SDR becomes "0" (closed state) at time t16. In this case, the camera unit 3 is in a state where it does not take any pictures and there is no need to drive the lens switching section 350, so the lens switching control section 81 maintains the lens switching control signal SAD in the magnification OFF state.

FIG. 17 is another example of a time chart of the lens switching control signal SAD, etc.
The drawer opening/closing signal SDW in the figure is the logical sum of the opening/closing signals S221, S231, S241, S251 (see FIG. 8) of the drawer doors 221, 231, 241, 251 (see FIG. 3); This signal is "1" if any of the drawer doors 241, 251 is open, and "0" if all of the drawer doors 221, 231, 241, 251 are closed.

Before time t21 in FIG. 17, the drawer opening/closing signal SDW is "0" (closed state), and the lens switching control signal SAD is in the enlargement OFF state. At time t21, the drawer opening/closing signal SDW is "1" (open state). This means that at least one of the drawer doors 221, 231, 241, and 251 is in an open state. Further, at time t22, the drawer opening/closing signal SDW is "0" (closed state). This means that all the drawer doors 221, 231, 241, 251 are in the closed state. During the illustrated period, the lens switching control section 81 (see FIG. 8) always sets the lens switching control signal SAD to the enlargement ON state.

FIG. 18 is another example of a time chart of the lens switching control signal SAD, etc.
Before time t25 in FIG. 18, the drawer opening/closing signal SDW is "0" (closed state), and the lens switching control signal SAD is in the enlargement OFF state. Then, when the drawer opening/closing signal SDW becomes "1" (open state) at time t25, it becomes necessary to insert the additional lens 362 into the optical axis LA in the lens switching section 350 (see FIG. 7). Therefore, the lens switching control section 81 (see FIG. 8) sets the lens switching control signal SAD to the enlargement ON state at time t25. After that, assume that the drawer opening/closing signal SDW becomes "0" (closed state) at time t25. In this case, the camera unit 3 is in a state where it is not photographing anything, and there is no need to drive the lens switching section 350, so the lens switching control section 81 maintains the lens switching control signal SAD in the magnification ON state.

FIG. 19 is another example of a time chart of the lens switching control signal SAD, etc.
Here, on the shooting setting screen 70 (see FIG. 14), the door priority PDR is set to H (high), the middle drawer priority PDM is set to M (medium), and the lower drawer priority PDL is set to L (low). It is assumed that this has been set. For the sake of simplicity, the inside of the drawer doors 221 and 231 will not be photographed.
It is assumed that before time t31 in FIG. 19, the door opening/closing signal SDR and the drawer opening/closing signal SDW are both "0" (closed state), and the lens switching control signal SAD is in the OFF state.

Assume that the door opening/closing signal SDR becomes "1" (open state) at time t31. Therefore, the lens switching control unit 81 (see FIG. 8) maintains the lens switching control signal SAD in the magnification OFF state in order to photograph the refrigerator compartment doors 211 and 212 (see FIG. 3). Next, at time t32, it is assumed that the drawer opening/closing signal SDW becomes "1" (open state) while the door opening/closing signal SDR is kept at "1" (open state). As described above, in the example of FIG. 19, the door priority PDR is H (high), which is higher than the middle drawer priority PDM and the lower drawer priority PDL.

Therefore, the lens switching control unit 81 maintains the lens switching control signal SAD in the magnification OFF state even after time t32 in order to photograph the refrigerator compartment doors 211 and 212 (see FIG. 3). After that, assume that the drawer opening/closing signal SDW becomes "0" (closed state) at time t33, and the door opening/closing signal SDR becomes "0" (closed state) at time t34. Since there is no need to drive the lens switching section 350 (see FIG. 7) at these timings, the lens switching control section 81 maintains the lens switching control signal SAD in the magnification OFF state.

Next, when the drawer opening/closing signal SDW becomes "1" (open state) at time t35, the lens switching control section 81 selects a mode for photographing any of the drawer doors 221, 231, 241, 251 (see FIG. 3). , switches the lens switching control signal SAD to the enlargement ON state. After that, assume that the door opening/closing signal SDR becomes open at time t36. As described above, the door priority PDR is H (high), which is higher than the middle drawer priority PDM and the lower drawer priority PDL. Therefore, the lens switching control section 81 sets the lens switching control signal SAD to the magnification OFF state at time t36 in order to photograph the refrigerator compartment doors 211 and 212 (see FIG. 3) again.

After that, when the door opening/closing signal SDR becomes "0" (closed state) at time t37, the lens switching control section 81 selects one of the drawer doors 221, 231, 241, and 251 (see FIG. 3) to photograph any of the drawer doors 221, 231, 241, and 251 (see FIG. 3). The lens switching control signal SAD is switched to the enlargement ON state again. Thereafter, at time t38, the drawer opening/closing signal SDW becomes "0" (closed state), but since there is no need to drive the lens switching section 350 (see FIG. 7), the lens switching control section 81 outputs the lens switching control signal SAD. Keep the magnification ON state.

FIG. 20 is another example of a time chart of the lens switching control signal SAD, etc.
Here, in the shooting setting screen 70 (see FIG. 14), the door priority PDR is set to L (low), the middle drawer priority PDM is set to H (high), and the lower drawer priority PDL is set to M (medium). It is assumed that this has been set.
The values of the door opening/closing signal SDR and the drawer opening/closing signal SDW at times t41 to t48 in FIG. 20 are the same as the values at times t31 to t48 in FIG. However, the lens switching control signal SAD has been in the enlargement ON state since before time t41.

During the period from time t41 to time t42, the lens switching control section 81 sets the lens switching control signal SAD to the magnification OFF state in response to the door opening/closing signal SDR which is "1" (open state). When the drawer opening/closing signal SDW becomes "1" (open state) during the period from time t42 to time t43, the middle drawer priority PDM and the lower drawer priority PDL are higher than the door priority PDR, so the lens switching control unit 81 Set the lens switching control signal SAD to the enlargement ON state.

During the period from time t43 to time t44, the drawer opening/closing signal SDW becomes "0" (closed state), so the lens switching control section 81 sets the lens switching control signal SAD to the enlargement OFF state. Thereafter, during the period from time t45 to time t48, since the drawer opening/closing signal SDW is "1" (open state), the lens switching control section 81 sets the lens switching control signal SAD to the enlargement ON state in the engine. Note that in the middle of that period, the door open/close signal SDR is "1" (open state) from time t46 to t47, but the door priority PDR is lower than the middle drawer priority PDM and the lower drawer priority PDL. Therefore, it does not affect the lens switching control signal SAD.

FIG. 21 is a diagram showing an example of a time chart of the shooting control state SST and the like.
Here, it is assumed that on the shooting setting screen 70 (see FIG. 14), the door priority PDR is pre-set to H (high), the middle drawer priority PDM is pre-set to M (medium), and the bottom drawer priority PDL is pre-set to L (low).
After the photography control unit 85 of the control unit 80 causes the camera unit 3 to start photographing the refrigerator 100, it takes some time until the camera unit 3 finishes photographing. Therefore, the state in which the camera unit 3 is not photographing or not is called the photography control state SST. In other words, if the photography control state SST is ON, photography is in progress, and if the photography control state SST is OFF, photography is not being performed.

Assume that at time t51 in FIG. 21, the drawer opening/closing signal SDW switches from "0" (closed state) to "1" (open state). Then, at the same time, the lens switching control section 81 sets the lens switching control signal SAD to the enlargement ON state. Further, at time t52, which is slightly delayed from time t51, the photographing control section 85 sets the photographing control state SST to the ON state, and instructs the camera unit 3 to set the photographing control state SST to the ON state, and instructs the camera unit 3 to select one of the containers 223, 233, 243 and 253 are started.

However, it is assumed that at time t53 during the container photography control, one of the refrigerator compartment doors 211 and 212 is opened and the door opening/closing signal SDR becomes "1" (open state). Accordingly, at the same time, the lens switching control section 81 changes the lens switching control signal SAD to the enlargement OFF state. As a result, the imaging control of the container cannot be continued, so the imaging control unit 85 sets the imaging control state SST to the OFF state and cancels the imaging of the container. Further, at time t54, which is slightly delayed from time t53, the photographing control unit 85 starts photographing the refrigerator compartment door, and finishes photographing the refrigerator compartment door at time t55. Note that when the photographing control unit 85 cancels the photographing, it notifies the user that the photographing has been canceled by making the buzzer 5 (see FIG. 8) sound.

FIG. 22 is a diagram showing another example of a time chart of the shooting control state SST, etc.
Here, in the shooting setting screen 70 (see FIG. 14), the door priority PDR is set to L (low), the middle drawer priority PDM is set to H (high), and the lower drawer priority PDL is set to M (medium). It is assumed that this has been set.
Assume that at time t61 in FIG. 22, the door opening/closing signal SDR switches from "0" (closed state) to "1" (open state). Further, at time t62, which is slightly delayed from time t61, the photographing control section 85 sets the photographing control state SST to the ON state, and starts controlling the camera unit 3 to photograph the corresponding refrigerator compartment door.

However, at time t63 during the photographing control of the refrigerator compartment door, any of the drawer doors 221, 231, 241, 251 (see FIG. 3) is opened and the drawer opening/closing signal SDW is "1" (open state). Suppose that it becomes . Accordingly, at the same time, the lens switching control section 81 changes the lens switching control signal SAD to the enlargement ON state. As a result, the photographing control of the refrigerator compartment door cannot be continued, so the photographing control unit 85 sets the photographing control state SST to the OFF state and cancels the photographing of the refrigerator compartment door. Further, at time t64, which is slightly delayed from time t63, the photographing control unit 85 starts photographing the containers in the drawer, and finishes photographing the containers in the drawer at time t65. In addition, as another example, the priority level of each door may not be defined, and a lens suitable for the door that was opened last may be simply set and photographed.

<Image viewing>
FIG. 23 is a diagram showing an example of an image viewing screen 400 displayed on the mobile terminal 54.
In FIG. 23, the image viewing screen 400 includes an image display section 450 and an image selection section 420. The image display unit 450 displays any processed image data GM stored in the server 53 (see FIG. 8). The image selection section 420 is a section for selecting the processed image data GM to be displayed on the image display section 450, and includes a left door specification button 422, a refrigerator compartment specification button 424, a right door specification button 426, and an ice making compartment/upper section. It includes a freezer compartment button 428, a vegetable compartment button 430, and a lower freezer compartment button 432.

When the user taps any button in the image selection section 420, the mobile terminal 54 communicates with the server 53 (see FIG. 8), downloads the processed image data GM corresponding to the tapped button, and displays the processed image data GM in the image display section 450. to be displayed. In the illustrated example, when the right door designation button 426 is tapped, the processed image data GM of the door pocket 211c of the right door is displayed on the image display section 450.

In the illustrated example, the image selection section 420 is secured in a separate area from the image display section 450, but the image selection section 420 may be made semi-transparent and superimposed on the image display section 450. . Furthermore, instead of providing the image selection section 420, the processed image data GM corresponding to each section may be switched and displayed by swiping the image display section 450 vertically and horizontally. Thereby, the image display section 450 can be made wider, and the processed image data GM can be displayed in a larger size, so that its visibility can be improved.

[Second embodiment]
Next, a refrigerator according to a second embodiment will be described. In the following description, parts corresponding to those in the first embodiment described above may be denoted by the same reference numerals, and the description thereof may be omitted.
In the second embodiment, a lens switching section 351 shown in FIG. 24 is applied instead of the lens switching section 350 (see FIG. 7) in the first embodiment. In other respects, the configuration of this embodiment is similar to that of the first embodiment (see FIGS. 1 to 23).

FIG. 24 is a schematic plan view of the lens switching section 351 applied to the second embodiment.
In FIG. 24, the lens switching section 351 (lens attachment/detachment section) includes a driver 368, an additional lens 362, and a lens frame 364. These configurations are similar to those of the first embodiment (see FIG. 7). Further, the lens switching section 351 of this embodiment includes a rotating section 370 that rotates the lens frame 364 using electric power supplied from the driver 368.

When the rotating unit 370 rotates the lens frame 364 to the position shown by the solid line, the additional lens 362 is inserted into the optical axis LA, so that the magnification is turned on. On the other hand, when the rotating unit 370 rotates the lens frame 364 to the position shown by the broken line, the additional lens 362 is moved away from the optical axis LA, so that the magnification is turned off. According to this embodiment, since the rotating section 370 rotates the lens frame 364 in the left-right direction without moving it in the front-back direction, the rotation section 370 rotates the lens frame 364 in the left-right direction, so that The dimensions of can be reduced.

[Third embodiment]
Next, a refrigerator according to a third embodiment will be described. In the following description, parts corresponding to those in the other embodiments described above may be denoted by the same reference numerals, and the description thereof may be omitted.
In the third embodiment, the camera unit 3 shown in FIG. 25 is used instead of the camera unit 3 (see FIG. 6) in the first embodiment. In other respects, the configuration of this embodiment is similar to that of the first embodiment (see FIGS. 1 to 23).

FIG. 25 is a schematic cross-sectional view of the camera unit 3 applied to the third embodiment.
In the camera unit 3 of this embodiment, the optical unit 302 and lens switching section 352 (lens attachment/detachment section) shown in FIG. 25 are used instead of the optical unit 300 and lens switching section 350 (see FIG. 6) of the first embodiment. Applicable.

FIG. 26 is a schematic diagram showing the configuration of the optical unit 302 and the lens switching section 352 in the third embodiment.
In FIG. 26, the optical unit 302 includes a support substrate 312, an image sensor 314, a lens base 316, and a base lens 318. The configuration of these individual elements is similar to that of the first embodiment (see FIG. 7), but there is a space (no reference numeral) between the image sensor 314 and the lens base 316 into which an additional lens 362 can be inserted and removed. It is secured.

Further, the lens switching unit 352 includes an additional lens 362, a lens frame 364, an actuator 366, and a driver 368. These individual structures are similar to those of the first embodiment (see FIG. 7), except that the actuator 366 inserts and removes the additional lens 362 between the lens base 316 and the image sensor 314. According to this embodiment, as shown in FIG. 25, the lens switching section 352 is housed inside the case 31b and is not exposed to the outside. Thereby, the product of this embodiment can achieve higher robustness than that of the first embodiment.

[Fourth embodiment]
Next, a refrigerator according to a fourth embodiment will be described. In the following description, parts corresponding to those in the other embodiments described above may be denoted by the same reference numerals, and the description thereof may be omitted.
In the fourth embodiment, a lens switching section 353 (lens attachment/detachment section) shown in FIG. 27 is applied instead of the lens switching section 350 (see FIG. 7) in the first embodiment. In other respects, the configuration of this embodiment is similar to that of the first embodiment (see FIGS. 1 to 23).

FIG. 27 is a schematic diagram showing the configuration of the lens switching section 353 in the fourth embodiment. Note that the same figure shows three states ST11, ST12, and ST13 of the lens switching section 353.
In FIG. 27, the lens switching section 353 includes a lens 374 (second lens), a lens 376 (first lens), a lens frame 372, a rotating section 370, and a driver 368. Lenses 374 and 376 are attached to the lens frame 372. Here, the lens 374 is, for example, a magnifying lens, and the lens 376 is, for example, a fisheye lens.

The driver 368 and the rotating unit 370 set the attitude of the lens frame 372 to either state ST11 or ST13 by rotationally driving the lens frame 372. Note that state ST12 is a state in the middle of rotation. In state ST11, the lens 376, which is a fisheye lens, for example, is inserted into the optical axis LA, so the lens switching unit 353 is in the magnification OFF state. Furthermore, in state ST13, the lens 374, which is a magnifying lens, for example, is inserted into the optical axis LA, so the lens switching unit 353 is in the magnification ON state.

According to this embodiment, there is an advantage that lenses 374 and 376 that are optimal for the magnification ON/OFF state can be applied. For example, when photographing a refrigerator with the magnification ON, the fisheye lens can be prevented from being inserted into the optical axis LA, so that distortion of the photographed image data GS2 (see FIG. 10) in the magnification ON state can be reduced.

[Fifth embodiment]
Next, a refrigerator according to a fifth embodiment will be described. In the following description, parts corresponding to those in the other embodiments described above may be denoted by the same reference numerals, and the description thereof may be omitted.
In the fourth embodiment, a lens switching section 354 (lens attachment/detachment section) shown in FIG. 28 is applied instead of the lens switching section 350 (see FIG. 7) in the first embodiment. In other respects, the configuration of this embodiment is similar to that of the first embodiment (see FIGS. 1 to 23).

FIG. 28 is a schematic diagram showing the configuration of the lens switching section 354 in the fifth embodiment.
In FIG. 28, the lens switching section 354 includes an additional lens 362, a lens frame 380, and a shaft support 382. The pivot support 382 rotatably supports the lens frame 380 near its center. The lens frame 380 is formed into a long rod shape, and an additional lens 362 is attached to one end of the lens frame 380. The user can then rotate the lens frame 380 by moving the other end of the lens frame 380 with a finger.

When the user rotates the lens frame 380 to the position shown by the solid line, the additional lens 362 is inserted into the optical axis LA, so that the magnification is turned on. On the other hand, when the user rotates the lens frame 380 to the position shown by the broken line, the additional lens 362 is moved away from the optical axis LA, so that the magnification is turned off. According to this embodiment, since the lens frame 380 is moved by the user, a circuit for driving the lens frame 380 can be omitted, and power consumption required for driving can also be reduced.

[Sixth embodiment]
Next, a refrigerator according to a sixth embodiment will be described. In the following description, parts corresponding to those in the other embodiments described above may be denoted by the same reference numerals, and the description thereof may be omitted.
In the sixth embodiment, a lens switching section 355 shown in FIG. 29 is applied instead of the lens switching section 350 (see FIG. 7) in the first embodiment. In other respects, the configuration of this embodiment is similar to that of the first embodiment (see FIGS. 1 to 23).

FIG. 29 is a schematic plan view of the lens switching unit 355 applied to the sixth embodiment.
In FIG. 29, the lens switching section 355 (lens attachment/detachment section) includes an additional lens 362, a lens frame 364, and a driver 368. These configurations are similar to those of the first embodiment (see FIG. 7). Further, the lens switching unit 355 of this embodiment includes an actuator 386 that drives the lens frame 364 in the left-right direction.

When the actuator 386 rotates the lens frame 364 to the position shown by the solid line, the additional lens 362 is inserted into the optical axis LA, so that the magnification is turned on. On the other hand, when the actuator 386 rotates the lens frame 364 to the position shown by the broken line, the additional lens 362 is moved away from the optical axis LA, so that the magnification is turned off. According to this embodiment, the actuator 386 does not move the lens frame 364 in the front-back direction but rotates it in the left-right direction. Dimensions can be reduced.

[Seventh embodiment]
Next, a refrigerator according to a seventh embodiment will be described. In the following description, parts corresponding to those in the other embodiments described above may be denoted by the same reference numerals, and the description thereof may be omitted.
FIG. 30 is a schematic diagram of an optical unit 300 in the seventh embodiment.
In this embodiment, an IR cut filter (infrared cut filter) 320 is inserted in the middle of the optical axis LA. FIG. 30 shows two arrangement examples P10 and P12. The IR cut filter 320 may be inserted between the image sensor 314 and the base lens 318 as in arrangement example P10, or may be inserted between the additional lens 362 and the base lens 318 as in arrangement example P12. good.

The configuration of this embodiment other than the above is similar to any of the first to sixth embodiments (see FIGS. 1 to 29). In this embodiment, since the IR cut filter 320 is inserted on the optical axis LA, it is possible to suppress the phenomenon in which an infrared image appears in the captured image data GS (see FIG. 8).

[Eighth embodiment]
Next, a refrigerator according to an eighth embodiment will be described. In the following description, parts corresponding to those in the other embodiments described above may be denoted by the same reference numerals, and the description thereof may be omitted.
FIG. 31 is a schematic diagram of an optical unit 300 in the eighth embodiment.
In this embodiment, instead of the additional lens 362 in the third embodiment (see FIGS. 25 and 26), a set of an additional convex lens 390 (second lens, convex lens) and an additional concave lens 392 (second lens, concave lens) is used. It is inserted into the optical axis LA.

In this configuration, even when the base lens 318 is moved along the optical axis LA, the optical axes LA and LB can be kept parallel. The configuration of this embodiment other than that described above is the same as that of the third embodiment (see FIGS. 25 and 26). In this embodiment, by moving the base lens 318 along the optical axis LA as necessary, the focal point FP can be adjusted to the imaging target, thereby making it possible to adjust the focus of the image.

[Ninth embodiment]
Next, a refrigerator according to a ninth embodiment will be described. In the following description, parts corresponding to those in the other embodiments described above may be denoted by the same reference numerals, and the description thereof may be omitted.
The configuration of the ninth embodiment is similar to any of the first to eighth embodiments (see FIGS. 1 to 31).

However, in this embodiment, the opening/closing determining unit 83 (see FIG. 8) determines the opening/closing states of the refrigerator compartment doors 211, 212 and the drawer doors 221, 231, 241, 251 based on the captured image data GS. Thereby, the door sensor 10 and the drawer door sensor 12 shown in FIG. 8 can be omitted.

[Effects of embodiment]
As described above, according to the embodiment described above, the storage (100) has a rotating door (211, 212) having door pockets 211c, 212c, and a container (223, 233, 243, 253) that moves back and forth. The imaging unit (3) includes a drawer door (221, 231, 241, 251), and an imaging unit (3) that captures an image of the door pocket 211c, 212c and/or the container (223, 233, 243, 253). , the image sensor 314, the first lens (318, 376) used when imaging the range including the door pockets 211c, 212c, and the first lens (318, 376) used when imaging the range including the containers (223, 233, 243, 253). The second lens (362, 374, 390, 392) used and the optical axis LA that makes the first lens (318, 376) and/or the second lens (362, 374, 390, 392) incident on the image sensor 314 The lens attachment/detachment part (350-355) is detachable from the first lens (318, 376) and/or the second lens (362, 374, 390, 392). ) is separated from the optical axis LA that enters the image sensor 314 as necessary, so images of the article can be appropriately acquired.

In addition, the first lens (318, 376) has a wider angle than the second lens (362, 374), and the lens attachment/detachment part (350 to 355) is used to It is more preferable to separate the two lenses (362, 374, 390, 392) from the optical axis LA. In this way, by separating the second lens (362, 374, 390, 392) from the optical axis LA, a wide range of the door pockets 211c, 212c can be photographed.

Further, as in the eighth embodiment, it is more preferable that the second lens (390, 392) includes a convex lens (390) and a concave lens (392). Thereby, the optical axes LA and LB (see FIG. 38) that pass through the concave lens (392) can be made parallel.

Furthermore, as in the eighth embodiment, it is more preferable that the first lens (318) is movable along the optical axis LA, thereby enabling focus adjustment. Thereby, the first lens (318) can be moved to a position according to the object to be photographed, and an image of the article can be acquired more appropriately.

In addition, when the imaging unit (3) images the range including the door pockets 211c and 212c, the imaging unit has a function of adjusting exposure based on the brightness of the range including at least a part of the door pockets 211c and 212c. When (3) images a range that includes the container (223, 233, 243, 253), exposure adjustment is performed based on the brightness of the range that includes at least a part of the container (223, 233, 243, 253). It is more preferable to further include an exposure adjustment section 82 having the function of performing the following functions. Thereby, it is possible to suppress blown-out highlights and blown-up shadows in the image, and it is possible to more appropriately obtain an image of the article.

The camera further includes a photography setting section 84 that determines priorities (PDR, PDM, PDL) for the rotating doors (211, 212) and the drawer doors (221, 231, 241, 251), and the photography setting section 84 includes: When the rotating door (211, 212) and the drawer door (221, 231, 241, 251) are opened to the point where both the door pockets 211c, 212c and the container (223, 233, 243, 253) can be photographed, It is more preferable to determine whether or not to remove the second lens (362, 374, 390, 392) from the optical axis LA based on the priority (PDR, PDM, PDL). Thereby, it is possible to more appropriately acquire an image of the article stored in the side to be prioritized among the rotating doors (211, 212) or the drawer doors (221, 231, 241, 251).

Furthermore, in the above embodiment, from another point of view, the storage (100) is configured such that the rotating door (211, 212) is in the open state and the drawer door (221, 231, 241, 251) is in the closed state. The imaging unit (3 ) is provided with a magnification setting section (81) for setting the magnification of the image. Thereby, an appropriate magnification of the imaging unit (3) can be set according to the state of the storage (100), so that images of articles can be appropriately acquired.

[Modified example]
The present invention is not limited to the embodiments described above, and various modifications are possible. The embodiments described above are exemplified to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the configurations described. Furthermore, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Further, it is possible to delete a part of the configuration of each embodiment, or add or replace other configurations. Furthermore, the control lines and information lines shown in the figures are those considered necessary for explanation, and do not necessarily show all the control lines and information lines necessary on the product. In reality, almost all components may be considered to be interconnected. Possible modifications to the above embodiment include, for example, the following.

(1) Since the hardware of the control unit 80 in the above embodiment can be realized by a general computer, programs etc. that execute the various processes described above are stored in a storage medium (a computer-readable recording medium on which programs are recorded). , or distributed via a transmission channel.

(2) Each of the above-mentioned processes has been explained as a software process using a program in the above embodiment, but some or all of them can be implemented using an ASIC (Application Specific Integrated Circuit) or an FPGA (Field It may be replaced with hardware processing using a programmable gate array (Programmable Gate Array) or the like.

(3) The various processes executed in the above embodiment may be executed by the server 53, and the various data stored in the above embodiment may also be stored in the server 53.

3 Camera unit (imaging section)
81 Lens switching control section (magnification setting section)
82 Exposure adjustment section 84 Shooting setting section 100 Refrigerator (storage)
211,212 Refrigerator door (swivel door)
211c, 212c Door pocket 221 Ice maker drawer door (drawer door)
223 Ice making compartment container (container)
231 Upper freezer compartment drawer door (drawer door)
233 Upper freezer container (container)
241 Vegetable compartment drawer door (drawer door)
243 Vegetable compartment container (container)
251 Lower freezer compartment drawer door (drawer door)
253 Lower freezer container (container)
314 Image sensor 318 Base lens (first lens)
350-355 Lens switching section (lens attachment/detachment section)
362 Additional lens (second lens)
374 lens (second lens)
376 lens (first lens)
390 Additional convex lens (second lens, convex lens)
392 Additional concave lens (second lens, concave lens)
LA optical axis PDR door priority (priority)
PDM middle drawer priority (priority)
PDL lower drawer priority (priority)

Claims (7)

A rotating door with a door pocket,
A drawer door that moves the container back and forth,
an imaging unit that images the door pocket and/or the container;
The imaging unit includes:
image sensor and
a first lens used when imaging a range including the door pocket;
a second lens used when imaging a range including the container;
A storage unit comprising: a lens attachment/detachment unit that separates the first lens and/or the second lens from an optical axis that enters the image sensor. The first lens has a wider angle than the second lens,
The storage according to claim 1, wherein the lens attachment/detachment section detaches the second lens from the optical axis when capturing an image of a range including the door pocket. The storage according to claim 2, wherein the second lens includes a convex lens and a concave lens. The storage according to claim 3, wherein the first lens is movable along the optical axis, thereby enabling focus adjustment. When the imaging unit images a range including the door pocket, a function of adjusting exposure based on the brightness of the range including at least a part of the door pocket;
When the imaging unit images a range including the container, a function of adjusting exposure based on the brightness of the range including at least a part of the container;
The storage according to claim 1, further comprising an exposure adjustment section having: further comprising a shooting setting unit that determines priorities for the rotating door and the drawer door,
The photographing setting unit is configured to move the second lens from the optical axis based on the priority when the rotating door and the drawer door are opened to a state where both the door pocket and the container can be photographed. The storage according to claim 1, further comprising: determining whether or not to remove the storage. A rotating door with a door pocket,
A drawer door that moves the container back and forth,
an imaging unit that images the door pocket and/or the container;
The magnification when the rotating door is in the closed state and the drawer door is in the open state is higher than the magnification when the rotating door is in the open state and the drawer door is in the closed state. , and a magnification setting section that sets the magnification of the imaging section.

Images