JP2024038577A - Storage cabinet and camera unit - Google Patents

Abstract

To provide a storage cabinet, etc., that appropriately determine the presence or absence of a lens cover.SOLUTION: A refrigerator 100 comprises a case comprising a refrigerating chamber, a refrigerating chamber door for closing an opening of the case, and a camera unit 3 installed outside the case, and for photographing at least the refrigerating chamber, and comprises a control unit 8 for determining whether a lens cover is attached to a lens 31a of the camera unit 3 or not, on the basis of luminance in a first region in a photographic image of the camera unit 3. The first region is set so that at least a portion of the refrigerating chamber is shown in the first region in the photographic image when the lens cover is detached.SELECTED DRAWING: Figure 7

Description

TECHNICAL FIELD This disclosure relates to storage and the like.

As a technique for determining whether a camera lens cover (lens cap) remains attached based on a photographed image, for example, the technique described in Patent Document 1 is known. That is, Patent Document 1 describes a video recording device that includes a "warning display determination section that determines whether or not a lens cap is attached using the output results of the gain comparison section and the luminance signal comparison section." ing.

JP2009-027310A

There is also a possibility that photography may be performed with the lens cover attached to the camera unit in the storage, but Patent Document 1 does not describe processing that assumes such a case.

The storage according to the present disclosure includes a casing having a storage chamber, a door that closes an opening of the casing, and a camera unit that is installed outside the casing and photographs at least the storage chamber. a control unit that determines whether or not a lens cover is attached to the lens of the camera unit based on the brightness of a first area in an image taken by the camera unit; In the image, the first region is set so that at least a part of the storage room is shown in the first region.

It is a front view of a refrigerator concerning an embodiment. It is a side view of the refrigerator concerning an embodiment. FIG. 2 is a front view of the refrigerator according to the embodiment with left and right refrigerator compartment doors open. FIG. 2 is a plan view of the refrigerator according to the embodiment with left and right refrigerator compartment doors open. In the refrigerator according to the embodiment, it is a perspective view when looking up diagonally from below the camera unit with the lens exposed. In the refrigerator according to the embodiment, it is a perspective view when looking up from diagonally below the camera unit with the lens cover attached. It is a front view of the camera unit of the refrigerator concerning an embodiment. It is a sectional view taken along the line II-II in FIG. 6A in the refrigerator according to the embodiment. FIG. 1 is a system configuration diagram including a camera unit of a refrigerator according to an embodiment. This is an example of a photographed result taken by the camera unit of the refrigerator according to the embodiment, with no image processing being performed. This is an example of a photographed result taken by the camera unit of the refrigerator according to the embodiment, after predetermined image processing has been performed. FIG. 3 is an explanatory diagram of a first region and a second region in a photographed image of the refrigerator according to the embodiment. It is a flow chart of processing performed by a control part of a refrigerator concerning an embodiment. It is a flow chart of processing performed by a control part of a refrigerator concerning an embodiment.

≪Embodiment≫
FIG. 1 is a front view of a refrigerator 100 according to an embodiment.
The refrigerator 100 (storage) is a device that cools food and the like, and includes a housing 1, doors such as refrigerator compartment doors 211 and 212, and a camera unit 3. The housing 1 has a plurality of storage chambers inside thereof. In the example of FIG. 1, the refrigerator 100 is provided with, in order from the top, a refrigerator compartment 21, an ice-making compartment 22 and an upper freezer compartment 23 lined up on the left and right, a vegetable compartment 24, and a lower freezer compartment 25. There is.

The casing 1 has a heat insulating material (not shown) such as urethane foam filled between an inner box 12 made of resin (see FIG. 3) forming a storage room and an outer box 11 made of steel plate. The structure is as follows. A plurality of openings 10a (see FIG. 2) corresponding to each chamber are provided on the front side (front side) of the housing 1. For example, when food or the like is placed through the opening 10a of the refrigerator compartment 21, the refrigerator compartment doors 211 and 212 are opened. Further, when the refrigerator compartment doors 211 and 212 are closed, the opening 10a of the refrigerator compartment 21 is closed. In this way, the refrigerator compartment doors 211 and 212 (doors) have a function of closing the opening 10a of the housing 1, etc.

As shown in FIG. 1 , the refrigerator 100 includes left and right refrigerator compartment doors 211 and 212 as hinged doors that together with the housing 1 form a refrigerator compartment 21 (storage compartment). The left refrigerator compartment door 211 is rotatable about the axis of a hinge 211a (see FIG. 4). The same applies to the refrigerator compartment door 212 on the right side. Further, the refrigerator 100 includes an ice making compartment door 221, an upper freezing compartment door 231, a vegetable compartment door 241, and a lower freezing compartment door 251 as pull-out doors.

The refrigerator compartment 21 is provided with a plurality of shelf boards 213 (see FIG. 3) that partition the refrigerator compartment 21 into predetermined areas. A plurality of door pockets 211c (see FIG. 3) for storing foods, etc. are installed on the inside of the left refrigerator compartment door 211 (door) (the same applies to the right refrigerator compartment door 212). The ice-making compartment 22 shown in FIG. 1 is provided with an ice-making compartment container (not shown) that is pulled out integrally with the ice-making compartment door 221. Similarly, the upper freezer compartment 23 is provided with an upper freezer container (not shown), the vegetable compartment 24 is provided with a vegetable compartment container (not shown), and the lower freezer compartment 25 is provided with a container (not shown). A lower freezer container (not shown) is provided.

Although not shown, the refrigerator 100 includes a compressor, a radiator (condenser), a capillary tube (throttling mechanism), and an evaporator. The refrigerant circulates through the compressor, radiator, capillary tube, and evaporator in sequence, and the air in the storage room is cooled by heat exchange with the refrigerant flowing through the evaporator. There is. The camera unit 3 shown in FIG. 1 photographs at least the storage compartment (for example, the refrigerator compartment 21) of the refrigerator 100, and is installed outside the housing 1 (in FIG. 1, on the upper side of the housing 1). .

FIG. 2 is a side view of refrigerator 100.
As shown in FIG. 2, the camera unit 3 includes a main body part 31 and a support part 32. The main body part 31 is for photographing the storage room of the refrigerator 100 and the like. The support section 32 supports the main body section 31 and is installed on the top surface of the housing 1.

A lens 31a (see also FIG. 5A) is installed near the front end of the main body 31. The lens 31a is an optical element that refracts and focuses light. For example, a fisheye lens is used as such a lens 31a, but the lens 31a is not limited to this. When the refrigerator compartment doors 211 and 212 (see FIG. 3) are opened, the lens 31a is in a state facing downward so that the camera unit 3 can photograph the refrigerator compartment 21 (see FIG. 3). .

As shown in FIG. 2, the lens 31a is located in front of the front end of the housing 1 (that is, the opening 10a). More preferably, the lens 31a is located further in front of the front surfaces of the refrigerator compartment doors 211, 212 (see FIG. 1) in the closed state. Thereby, for example, when the refrigerator compartment doors 211 and 212 are opened, the refrigerator compartment 21 can easily come into the field of view of the lens 31a. Note that it is also possible to photograph the vegetable compartment 24 and the like (see FIG. 1) using the camera unit 3 while a pull-out door such as the vegetable compartment door 241 is opened.

FIG. 3 is a front view of the refrigerator 100 with the left and right refrigerator compartment doors 211 and 212 open.
As shown in FIG. 3, a plurality of door pockets 211c for storing foods, etc. are installed in the inner plate 211b of the left refrigerator compartment door 211 (the same applies to the right refrigerator compartment door 212). When the left and right refrigerator compartment doors 211, 212 are opened, the food, etc. in the refrigerator compartment 21 and the door pockets 211c, 212c come into the field of view of the lens 31a of the camera unit 3 (see FIG. 8). (see also).

In the left refrigerator compartment door 211, a square frame-shaped packing 211d is installed near the edge of the inner plate 211b. The packing 211d is a resin member for increasing the airtightness of the refrigerator compartment 21. Then, when the refrigerator compartment door 211 is closed, the packing 211d comes into contact (close contact) around the opening 10a of the housing 1 (see FIG. 4).

The side piece 211e shown in FIG. 3 is a plate-shaped member forming the side surface of the refrigerator compartment door 211, and extends in the vertical direction. In addition to the side piece 211e provided on the opposite side of the axis of the hinge 211a (see FIG. 4), the refrigerator door 211 also includes another side piece 211f (see FIG. 4) provided near the axis of the hinge 211a. We are prepared. These side pieces 211e and 211f (see FIG. 4) both extend in the vertical direction and are provided parallel to the axis of the hinge 211a (see FIG. 4). The same applies to the refrigerator compartment door 212 on the right side.

Further, when manually photographing the refrigerator compartment 21 using the camera unit 3, photographing buttons 5 that are pressed by the user are installed on each of the left and right refrigerator compartment doors 211 and 212. In the example of FIG. 3, one shooting button 5 is installed at the bottom of each of the left and right side pieces 211e and 212e.

FIG. 4 is a plan view of the refrigerator 100 with the left and right refrigerator compartment doors 211 and 212 open.
As shown in FIG. 4, hinges 211a and 212a are installed on the top surface of the housing 1. The left hinge 211a rotatably supports the refrigerator compartment door 211 (the same applies to the right hinge 212a). Moreover, in the example of FIG. 4, the camera unit 3 is arranged slightly to the left of the center of the housing 1 of the refrigerator 100 in the left-right direction. To explain in more detail, the camera unit 3 is arranged directly above the joint between the refrigerator compartment doors 211 and 212 in the closed state (see also FIG. 1). Thereby, when the refrigerator compartment doors 211 and 212 are opened, food and the like in the door pockets 211c and 212c easily enter the field of view of the camera unit 3.

FIG. 5A is a perspective view of the camera unit 3 viewed diagonally from below with the lens 31a exposed.
As shown in FIG. 5A, the main body 31 of the camera unit 3 includes a case 31b and an LED cover 31c in addition to the lens 31a described above (see also FIG. 2). The case 31b of the main body portion 31 generally has a rectangular parallelepiped shape that is elongated in the front-rear direction. A circular hole (reference numeral not shown) is provided on the lower surface near the front end of the case 31b, and the lens 31a is exposed through this hole. In the case 31b, a hole (reference numeral not shown) elongated in the left-right direction is provided on the rear side (inner side) of the lens 31a, and the LED cover 31c is fitted into this hole. The LED cover 31c is a translucent resin member for protecting the camera LED 31d (illumination section: see FIG. 6B).

FIG. 5B is a perspective view of the camera unit 3 with the lens cover 33 attached, viewed diagonally from below.
The lens cover 33 shown in FIG. 5B is a cover for protecting the lens 31a (see FIG. 5A) of the camera unit 3. The lens cover 33 is attached to cover the lens 31a from below and is removable.

Note that when the refrigerator 100 (see FIG. 1) is sold, a lens cover 33 is attached to the lens 31a (see FIG. 5A), and a protective film (not shown) is attached to prevent the lens cover 33 from coming off. Sometimes. When the user uses the refrigerator 100 and removes the protective film, the lens cover 33 is usually removed along with the protective film, but there are cases where the lens cover 33 is not removed. Furthermore, the user may reattach the lens cover 33 to the camera unit 3 for some reason. Then, there is a possibility that the user forgets to remove the lens cover 33 (with the lens cover 33 still attached) and the camera unit 3 takes a picture. Therefore, in this embodiment, a control unit 8 (see FIG. 7), which will be described later, determines whether or not the lens cover 33 is present based on the brightness of the photographed image.

FIG. 6A is a front view of the camera unit 3.
As shown in FIG. 6A, the main body part 31 of the camera unit 3 is installed above the support part 32. Further, the main body portion 31 is installed near the center of the support portion 32 in the left-right direction.

FIG. 6B is a sectional view taken along the line II-II in FIG. 6A.
As shown in FIG. 6B, the camera unit 3 includes the lens 31a (see also FIG. 5A), the LED cover 31c (see also FIG. 5A), and the case 31b, as well as a camera LED 31d and a circuit board 31e. ing.

The camera LED 31d (illumination unit) is a light source that illuminates a storage room such as the refrigerator compartment 21. That is, the camera LED 31d emits light into the refrigerator compartment 21 etc. so that the camera unit 3 can photograph the refrigerator compartment 21 (see FIG. 3) and the door pockets 211c, 212c (see FIG. 3) under appropriate brightness. It is installed on the outside of the housing 1 (in this embodiment, on the upper side of the housing 1). In this embodiment, the camera LED 31d is located in front of the front end (opening 10a of the housing 1) of the housing 1 (see FIG. 2) of the refrigerator 100. That is, the camera LED 31d (illumination section) is located in front (outside) of the opening 10a of the housing 1 (see FIG. 2) in plan view. This makes it easier for the light emitted from the camera LED 31d to enter the refrigerator compartment 21 and the like, thereby making it easier to obtain clear photographic results. Furthermore, when the lens cover 33 (see FIG. 5B) is removed, the camera unit 3 photographs the refrigerator compartment 21 illuminated with light from the camera LED 31d (a photographed image with relatively high average brightness). This increases the precision with which the control unit 8 determines whether the lens cover 33 is present based on the photographed image. For example, when at least one of the refrigerator compartment doors 211 and 212 (see FIG. 3) is opened, the camera LED 31d is turned on. A translucent LED cover 31c (see also FIG. 5A) is installed below the camera LED 31d.

The circuit board 31e is a printed circuit board on which an image sensor 31f (see FIG. 7) and a camera/communication control SoC 31g (System-on-a-Chip: see FIG. 7), which will be described later, are mounted. In this way, by housing the lens 31a, camera LED 31d, and circuit board 31e in one case 31b, the camera unit 3 can be made smaller than when these are provided separately.

FIG. 7 is a system configuration diagram including the camera unit 3 of the refrigerator 100.
As shown in FIG. 7, in addition to the camera unit 3, the refrigerator 100 includes a door sensor 4, a shooting button 5, an operation panel 6, an interior light 7, a control section 8, and a wireless LAN unit 9. We are prepared.
The door sensor 4 outputs a predetermined signal to the control unit 8 indicating the open/closed state of the refrigerator compartment doors 211 and 212 (see FIG. 1). Although one door sensor 4 is shown in FIG. 7, in reality, door sensors 4 are provided corresponding to each door of the refrigerator 100 (see FIG. 1). For example, when the left refrigerator compartment door 211 (see FIG. 1) is opened, a predetermined open signal is output from the door sensor 4 corresponding to this refrigerator compartment door 211 to the control unit 8. For example, when at least one of the refrigerator compartment doors 211 and 212 (see FIG. 3) is opened, the camera LED 31d lights up.

The photographing button 5 (see also FIG. 3) is a button that is pressed by the user when manually photographing using the camera unit 3, as described above. The operation panel 6 is a panel for inputting predetermined setting information based on user operations. The interior light 7 is a light source that illuminates the refrigerator compartment 21 (storage compartment), and is installed on the wall of the refrigerator compartment 21.

The control unit 8 is, for example, a microcomputer, and is configured to include electronic circuits such as a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and various interfaces, although not shown. . Then, the program stored in the ROM is read out and expanded to the RAM, and the CPU executes various processes. Note that the processing performed by the control unit 8 will be described later.

The camera unit 3 includes a lens 31a (see also FIG. 6B), a camera LED 31d (see also FIG. 6B), an image sensor 31f, a camera/communication control SoC 31g, and a buzzer 31h. The image sensor 31f is an element that photoelectrically converts the light that enters through the lens 31a and generates photographed image data. As such an image sensor 31f, for example, a CCD sensor (Charge Coupled Device) or a CMOS sensor (Complementary Metal Oxide Semiconductor) is used.

The camera/communication control SoC 31g is an integrated circuit in which circuits having microcomputer functions and other applied functions are integrated on one chip and are designed to work together. The camera/communication control SoC 31g performs predetermined communication with the control unit 8 and outputs a shooting command to the image sensor 31f. Thereby, data of the captured image is input from the image sensor 31f to the camera/communication control SoC 31g. Note that the camera/communication control SoC 31g is an example, and other types may be used.

The buzzer 31h emits a predetermined sound, for example, when an error occurs during photographing by the camera unit 3. Note that when the camera unit 3 starts photographing, a predetermined sound may be emitted from the buzzer 31h.

The wireless LAN unit 9 transmits data of the captured image output from the camera/communication control SoC 31g to the router 51. The router 51 is a communication relay device, and transmits captured image data received from the wireless LAN unit 9 to the server 53 via the network 52. The server 53 performs predetermined processing on the captured image data and stores it in association with the identification information of the refrigerator 100. Furthermore, when the server 53 receives a request signal for an internal image of the refrigerator 100 from the user's mobile terminal 54, the server 53 transmits data of the photographed image to the mobile terminal 54. As such a mobile terminal 54, in addition to a mobile phone or a smartphone, a tablet or a wearable terminal is used.

FIG. 8 is an example of the result of photography by the camera unit, with no image processing performed.
Note that in FIG. 8, the upper side of the page is the front, and the bottom side of the page is the rear, so the left and right sides are reversed with respect to FIG. 3, etc. (the same applies to FIGS. 9 and 10). In the example of FIG. 8, both the left and right refrigerator compartment doors 211, 212 are open, and in addition to the refrigerator compartment 21, the door pocket 211c of the left refrigerator compartment door 211, and the door pocket 212c of the right refrigerator compartment door 212. has also been photographed.

In particular, when a fisheye lens is used as the lens 31a of the camera unit 3 (see FIG. 5), images can be captured with a wide range of angles of view. However, in a state where image processing is not performed, as shown in FIG. The number of pixels decreases.

FIG. 9 shows an example of a photographed result by the camera unit 3 after predetermined image processing has been performed.
For example, by performing predetermined image processing in the server 53 (see FIG. 7) based on the photographic results shown in FIG. is converted to By displaying such an image on the mobile terminal 54 (see FIG. 7), the user can grasp at a glance what kind of food is stored in the refrigerator compartment 21 or the like.

FIG. 10 is an explanatory diagram of first areas A1, A2, A3 and second areas B1, B2 in a photographed image of a refrigerator.
10 is an area of a captured image based on electrical signals from a large number of image sensors 31f (see FIG. 7) arranged vertically and horizontally in a matrix. One corner of the rectangular area 60 (e.g., the lower left corner of the paper) is set as the origin O, the horizontal direction from the origin O is set as the x-axis, and the vertical direction is set as the y-axis. Each pixel in the captured image is identified based on the coordinate values of the x-axis and y-axis.

An image circle 61 is included in the area 60 of the photographed image. The image circle 61 is a circular area on which light incident through the lens 31a (see FIG. 7) is reflected as an image. In the rectangular region 60, black portions outside the image circle 61 are so-called vignetting regions 62 and 63. The vignetting regions 62 and 63 correspond to the image sensor 31f (see FIG. 7) at a position where almost no light enters.

In FIG. 10, three first areas A1, A2, and A3 indicated by rectangular broken line frames are used to determine whether the lens cover 33 (see FIG. 5B) remains attached to the camera unit 3. This is the area used. First, to explain the first area A1 located in the center in the left-right direction, in the photographed image with the lens cover 33 (see FIG. 5B) removed, the first area A1 includes at least one of the refrigerator compartments 21 (storage compartments). A first area A1 is set so that a portion of the image is captured. In the example of FIG. 10, the rectangular first area A1 is opened so that a part of the refrigerator compartment 21 enters the first area A1 when the left and right refrigerator doors 211, 212 (see FIG. 3) are opened. It is set in advance inside the image circle 61.

The first area A2 on the left side (right side in the left-right direction of the paper) shown in FIG. 10 is an image taken with the lens cover 33 (see FIG. 5B) removed, when the left refrigerator door 211 is opened. The setting is such that at least a part of the door pocket 211c is reflected in the first area A2. In the example of FIG. 10, when the left refrigerator compartment door 211 is opened, a rectangular first area A2 is placed inside the image circle 61 in advance so that a part of the door pocket 211c enters the first area A2. It is set. Note that the same applies to the first area A3 on the right side.

Light is irradiated from the camera LED 31d (see FIG. 7) to the refrigerator compartment 21 and the door pocket 211c in which foods and the like are stored. Therefore, if photography is performed when the left and right refrigerator doors 211, 212 are opened with the lens cover 33 (see FIG. 5B) removed, the average luminance of each of the first areas A1, A2, and A3 has a relatively high value. On the other hand, when photographing is performed with the lens cover 33 (see FIG. 5B) attached, light is reflected by the lens cover 33, so that almost no light enters the image sensor 31f (see FIG. 7). As a result, although not shown, the entire image circle 61 becomes an almost black image similar to the vignetting areas 62 and 63.

As shown in FIG. 10, a rectangular second area B1 is preset inside the vignetting area 62 in the photographed image. Similarly, a rectangular second region B2 is set in advance inside the other vignetting region 63 in the photographed image. For example, if a photograph is taken with the lens cover 33 (see FIG. 5B) still attached and the brightness of the photographed image is low overall, the control unit 8 may increase the brightness gain by adjusting the exposure. . In such a case, the vignetting areas 62 and 63 become pale black, and the average brightness of the vignetting areas 62 and 63 becomes higher than when the lens cover 33 (see FIG. 5B) is removed.

Therefore, in this embodiment, the lens cover 31 is attached to the lens 31a of the camera unit 3 (see FIG. 7) based on the brightness of the first areas A1, A2, A3 and the second areas B1, B2 in the captured image of the camera unit 3. The control unit 8 (see FIG. 7) determines whether or not (see FIG. 5B) is attached.

11A and 11B are flowcharts of processing executed by the control unit of the refrigerator (see also FIG. 7 as appropriate).
Note that FIGS. 11A and 11B mainly show processing related to determining the presence or absence of the lens cover 33 (see FIG. 5B). Further, it is assumed that at least one of the refrigerator compartment doors 211 and 212 (see FIG. 3) is opened at the time of "START" in FIG. 11A, and the camera unit 3 takes a picture. For example, when the refrigerator compartment doors 211 and 212 are opened, the camera unit 3 may automatically take a picture (automatic picture taking). Furthermore, when the photographing button 5 (see FIG. 3) is pressed by the user, the camera unit 3 may take a photograph (manual photographing). These matters are also included in this embodiment.

In step S101 of FIG. 11A, the control unit 8 captures a photographed image. That is, the control unit 8 takes in the data of the photographed image from the image sensor 31f of the camera unit 3 (see FIG. 7). Although not shown in FIG. 11A, the control unit 8 may convert the photographed image (color image) into a grayscale image, or the luminance value of each pixel of the photographed image may be changed to '1'. It may be binarized into either ' (white) or '0' (black).

Next, in step S102, the control unit 8 calculates the average brightness of each of the three first areas A1, A2, and A3 (see FIG. 10).
In step S103, the control unit 8 specifies the one with the highest average brightness among the three first areas A1, A2, and A3 (see FIG. 10). For example, when both the refrigerator compartment doors 211 and 212 (see FIG. 3) are opened, the light from the camera LED 31d (see FIG. 7) is irradiated onto the refrigerator compartment 21 and the door pockets 211c and 212c (see FIG. 3). Therefore, the average luminance of each of the first areas A1, A2, and A3 usually takes a relatively high value. However, when the lens cover 33 (see FIG. 5B) remains attached, almost no light enters the image sensor 31f (see FIG. 7), so all of the three first areas A1, A2, and A3 become dark. , the average luminance of each is also considerably lower.

Next, in step S104, the control unit 8 determines whether the average brightness of the identified first area (the first area with the highest average brightness) is equal to or higher than the target value. Here, the "target value" is a luminance threshold value that is a criterion for determining whether or not the control unit 8 finishes the exposure adjustment, and is set in advance. In step S104, if the average brightness of the first region with the highest average brightness is not equal to or higher than the target value (S104: No), the process of the control unit 8 proceeds to step S105.

In step S105, the control unit 8 determines whether a predetermined exposure limit time has elapsed since the start of photography. Note that the "exposure limit time" is a threshold value that serves as a criterion for determining whether or not the control unit 8 performs exposure adjustment (the next process in S106), and is set in advance. If the exposure time limit has not elapsed in step S105 (S105: No), the process of the control unit 8 proceeds to step S106.

In step S106, the control unit 8 adjusts the gain and shutter speed (that is, performs exposure adjustment). More specifically, the control unit 8 adjusts the gain (sensitivity to brightness) when amplifying the electrical signal from the image sensor 31f (see FIG. 7), and also adjusts the shutter speed of the camera unit 3. By performing such exposure adjustment, it is possible to suppress overexposure (halation) and underexposure in the photographed image. After performing the process of step S106, the process of the control unit 8 returns to step S101. Then, the control unit 8 captures the captured image ( S101) and exposure adjustment (S106) are repeated in sequence.

If the exposure time limit has elapsed in step S105 (S105: Yes), the process of the control unit 8 proceeds to step S107 in FIG. 11B. In addition, in step S104, even when the average brightness of the highest average brightness among the plurality of first areas A1, A2, A3 is equal to or higher than the target value (S104: Yes), the process of the control unit 8 is performed as shown in FIG. 11B. The process advances to step S107.

In step S107, the control unit 8 determines whether the average brightness of the identified first area (the first area with the highest average brightness) is less than or equal to a first predetermined value. Here, the "first predetermined value" is a brightness threshold that is a criterion for determining whether or not the lens cover 33 (see FIG. 5B) is attached to the lens 31a (see FIG. 5A) of the camera unit 3, and is It is set.

In step S107, if the average brightness of the first area (for example, first area A1) with the highest average brightness is equal to or less than the first predetermined value, the process of the control unit 8 proceeds to step S109. That is, if the average luminance of any of the three first areas A1, A2, and A3 is equal to or less than the first predetermined value, the process of the control unit 8 proceeds to step S109.

In step S109, the control unit 8 determines that the lens cover 33 (see FIG. 5B) remains attached to the lens 31a (see FIG. 5A). That is, if the average brightness of the highest average brightness among the plurality of first areas A1, A2, and A3 is equal to or lower than the first predetermined value (S107: Yes), the control unit 8 controls the lens 31a of the camera unit 3. It is determined that the lens cover 33 (see FIG. 5B) is attached to the camera (see FIG. 5A). This is because when the lens cover 33 remains attached, the light from the camera LED 31d is reflected by the lens cover 33, and almost no light enters the image sensor 31f (see FIG. 7).

Although omitted in FIG. 11B, if it is determined that the lens cover 33 (see FIG. 5B) is attached to the lens 31a (see FIG. 5A) of the camera unit 3 (S109), the control unit 8 It is preferable not to send the photographing results of No. 3 to the server 53 (see FIG. 7). This can prevent a photographed image in which substantially the entire area is black from being displayed on the mobile terminal 54 (see FIG. 7).

Although omitted in FIG. 11B, when it is determined that the lens cover 33 (see FIG. 5B) is attached to the lens 31a (see FIG. 5A) of the camera unit 3, the control unit 8 activates the buzzer 31h (see FIG. 7). ). This makes it easier for the user to notice that he or she has forgotten to remove the lens cover 33. In addition, a message such as "Recognition failure" or "Please check the presence or absence of a lens cover" may be displayed on the user's mobile terminal 54 (see FIG. 7). These matters are also included in this embodiment.

Further, in step S107, if the average brightness of the first area (for example, first area A1) with the highest average brightness is higher than the first predetermined value (S107: No), the process of the control unit 8 proceeds to step S108. .
In step S108, the control unit 8 determines whether or not any of the second regions B1 and B2 (see FIG. 10) has an average luminance equal to or higher than a second predetermined value. Here, the "second predetermined value" is a brightness threshold that is a criterion for determining whether or not the lens cover 33 (see FIG. 5B) is attached to the lens 31a (see FIG. 5A) of the camera unit 3, and is It is set. For example, a "second predetermined value" is set as a luminance threshold lower than the "first predetermined value" (S107) described above. In step S109, if there is a second area where the average luminance is equal to or higher than the second predetermined value (S108: Yes), the process of the control unit 8 proceeds to step S109.

In step S109, the control unit 8 determines that the lens cover 33 (see FIG. 5B) is attached to the lens 31a (see FIG. 5A) of the camera unit 3 (lens cover is present). That is, after performing the exposure adjustment to adjust the brightness gain of the captured image (S106 in FIG. 11A), if the average brightness of the second area is equal to or higher than the second predetermined value (S108: Yes), the control unit 8 It is determined that the lens cover 33 (see FIG. 5B) is attached to the lens 31a (see FIG. 5A) of the camera unit 3. This is because, as described above, when the captured image is entirely dark and the control unit 8 increases the brightness gain by adjusting the exposure, the vignetting areas 62 and 63 (see FIG. 10) become pale black.

Further, in step S108, if the second area where the average luminance is equal to or higher than the second predetermined value does not exist (S108: No), the process of the control unit 8 proceeds to step S110.
In step S110, the control unit 8 determines that the lens cover 33 (see FIG. 5B) is not attached to the lens 31a (see FIG. 5A) (no lens cover). In such a case, although not shown in FIG. 11B, when the opening angle of the refrigerator compartment doors 211, 212 (see FIG. 3) based on the captured image of the camera unit 3 is equal to or greater than a predetermined value, the control unit 8 Image data is uploaded to the server 53 (see FIG. 7). Then, by performing predetermined image processing in the server 53, an image (see FIG. 9) that looks as if the refrigerator compartment 21 and the left and right refrigerator compartment doors 211, 212 are viewed from the front is created on the user's mobile terminal 54 (see FIG. 7). (see).

According to the present embodiment, in addition to the first areas A1, A2, A3 included in the image circle 61 (see FIG. 10) of the captured image, the second areas B1, B2 included in the vignetting area 62 (see FIG. 10) Based on each average luminance, the presence or absence of the lens cover 33 (see FIG. 5B) can be appropriately determined. Further, since there is no particular need to separately add a configuration for determining the presence or absence of the lens cover 33, the manufacturing cost of the refrigerator 100 can be suppressed.

According to the present embodiment, when the control unit 8 determines the presence or absence of the lens cover 33, in addition to the first areas A1, A2, and A3, the second area B1, which is included in the vignetting area 62 (see FIG. 10), The average brightness of B2 is also used. As a result, even if the lens cover 33 is still attached to the camera unit 3 and the overall captured image becomes a dimly bright image due to exposure adjustment, the control unit 8 can detect the presence or absence of the lens cover 33 with high precision. Can be distinguished.

Further, according to the present embodiment, when it is determined that the lens cover 33 is attached, the control unit 8 prevents the data of the photographed image from being transmitted to the server 53 (see FIG. 7). This prevents a photographed image in which almost the entire area is black from being displayed on the user's mobile terminal 54 (see FIG. 7), so the user does not think that the camera unit 3 is broken or feels uncomfortable. can be prevented.

≪Modification example≫
Although the refrigerator 100 and the like according to the present disclosure have been described above in the embodiments, the invention is not limited to these descriptions, and various changes can be made.
For example, in the embodiment, in addition to the three first areas A1, A2, and A3, two second areas B1 and B2 are set in the area of the photographed image. The number of areas can be changed as appropriate. For example, the number of first areas in the image circle 61 (see FIG. 10) may be one or two, or may be four or more. Note that the first area is set so that a part of the area of the refrigerator compartment 21 or the door pockets 211c and 212c is captured. Furthermore, the number of second regions in the vignetting region 62 (see FIG. 10) may be one, or may be three or more. Further, in the embodiment, a case has been described in which the shapes of the first regions A1, A2, A3 and the second regions B1, B2 are each rectangular, but the shape of each region can be changed as appropriate.

Further, in the embodiment, the control unit 8 controls the lens cover 33 based on the average brightness of each of the first areas A1, A2, A3 (see FIG. 10) and the second areas B1, B2 (see FIG. 10) of the photographed image. (See FIG. 5B) has been described above, but the present invention is not limited thereto. That is, the process regarding the second areas B1 and B2 is omitted, and if the average brightness of the highest average brightness among the first areas A1, A2, and A3 is equal to or lower than the first predetermined value, the lens cover 33 is not attached. The control unit 8 may determine that it is. Further, the process regarding the first areas A1, A2, and A3 is omitted, and if there is a second area B1, B2 whose average luminance is equal to or higher than the second predetermined value, it is determined that the lens cover 33 is attached. The control unit 8 may also make the determination.

Further, in the embodiment, when the average brightness of the highest average brightness among the first areas A1, A2, and A3 is equal to or lower than the first predetermined value, the control unit 8 determines that the lens cover 33 is attached. Although the processing described above is not limited to this. For example, if the average brightness of all pixels included in the first areas A1, A2, and A3 is less than or equal to a first predetermined value, the control unit 8 may determine that the lens cover 33 is attached. Further, if the average luminance of all pixels included in the second regions B1 and B2 is equal to or higher than a second predetermined value, the control unit 8 may determine that the lens cover 33 is attached.

Further, in the embodiment, a case has been described in which the control unit 8 determines whether the lens cover 33 (see FIG. 5B) of the camera unit 3 is present, but the present invention is not limited to this. That is, even if the control unit 8 determines the presence or absence of a light-transmitting protective film (not shown) attached to cover the lens 31a (see FIG. 7) of the camera unit 3 based on the captured image. good. In this case, a translucent lens cover may be attached between the lens 31a and a protective film (not shown), or the lens cover may not be attached from the time of sale. For example, on a transparent protective film (and/or a transparent lens cover), a predetermined mark (not shown) is printed inside an area corresponding to the lens 31a, and the control unit 8 prints an image on the photographed image. When the mark is recognized, it may be determined that a protective film is present. Note that the method for determining the presence or absence of a protective film can also be applied to determining the presence or absence of a light-transmitting lens cover.

Further, in the embodiment, a case has been described in which the camera unit 3 is integrally installed in the refrigerator 100, but the present invention is not limited to this. For example, the camera unit 3 may be an optional item sold separately and configured to be detachable from the housing 1 of the refrigerator 100 (see FIG. 1).
Further, in each embodiment, a process has been described in which the control unit 8 (see FIG. 7) of the refrigerator 100 determines the presence or absence of the lens cover 33 (see FIG. 5B) based on a captured image, but the present invention is not limited to this. For example, based on the brightness of the first areas A1, A2, A3 (see FIG. 10) in the photographed image of the camera unit 3, "control" determines whether or not the lens cover 33 is attached to the lens 31a of the camera unit 3. The camera unit 3 may include a section. Even with such a configuration, the same effects as in the embodiment can be achieved.

Further, in the embodiment, a case has been described in which a fisheye lens is used as the lens 31a (see FIG. 7) of the camera unit 3, but the invention is not limited to this. For example, as the lens 31a of the camera unit 3, a general lens other than a fisheye lens may be used.
Further, in the embodiment, a case has been described in which the camera unit 3 includes the camera LED 31d (illumination section) and the camera LED 31d is installed above the housing 1, but the present invention is not limited to this. For example, an “illumination unit” that irradiates light onto the refrigerator compartment 21 may be provided separately from the camera unit 3. Further, the position of the "illumination section" can be changed as appropriate.

Furthermore, in the embodiment, a configuration has been described in which the camera unit 3 includes the buzzer 31h (see FIG. 7), but the present invention is not limited to this. That is, a buzzer may be installed at a predetermined location outside the camera unit 3 in the refrigerator 100.

Further, in the embodiment, a configuration has been described in which the camera unit 3 (see FIG. 2) that generates a photographed image is provided on the top surface of the housing 1, but the present invention is not limited to this. That is, the camera unit 3 may be provided at another predetermined position on the outside of the housing 1.

Further, in the embodiment, the refrigerator 100 is described as having French-style refrigerator compartment doors 211 and 212 (see FIG. 3), but the present invention is not limited thereto. For example, the embodiment can be applied to a refrigerator with a single-opening refrigerator compartment door (not shown) or a portable refrigerator (not shown).
Further, the "storage" to which the embodiments can be applied is not limited to a refrigerator. That is, the embodiments can be applied to various types of hinged "storage". In this case, the casing of the "storage" may be configured such that there is no particular distinction between an outer box and an inner box. Further, the embodiments can also be applied to, for example, a chest-type freezer or storage. Further, the embodiments can also be applied to storages equipped with a top-opening door (a flip-up door), a bottom-opening door, a folding door, or the like. Furthermore, the embodiments can be applied to, for example, a closet equipped with a folding door.

Furthermore, one or more computers may execute all or part of the program that implements the method for determining the presence or absence of the lens cover 33 described in the embodiment. The above program can be provided via a communication line, or can be written on a recording medium such as a CD-ROM and distributed.

Further, the embodiments are described in detail to explain the present disclosure in an easy-to-understand manner, and the embodiments are not necessarily limited to having all the configurations described. Furthermore, it is possible to add, delete, or replace some of the configurations of the embodiments with other configurations.
Further, the mechanisms and configurations described above are those considered necessary for explanation, and not all mechanisms and configurations are necessarily shown in the product.

1 Housing 3 Camera unit 4 Door sensor 5 Shooting button 6 Operation panel 7 Interior light 8 Control section 9 Wireless LAN unit 10a Opening 21 Refrigerator room (storage room)
31a Lens 31d Camera LED (lighting part)
31f Image sensor 31h Buzzer 33 Lens cover 53 Server 61 Image circle 62, 63 Vignetting area 100 Refrigerator (storage)
211,212 Refrigerator door (door)
211c, 212c Door pocket A1, A2, A3 1st area B1, B2 2nd area

Claims (9)

a casing having a storage room;
a door that closes the opening of the housing;
a camera unit installed outside the housing and photographing at least the storage room, and
comprising a control unit that determines whether a lens cover is attached to the lens of the camera unit based on the brightness of a first region in the image taken by the camera unit;
The first area is set so that at least a part of the storage room is captured in the first area in the photographed image with the lens cover removed. The storage according to claim 1, further comprising a lighting section installed outside the housing and illuminating the storage room. The storage according to claim 2, wherein the lighting section is installed above the housing and is located outside the opening of the housing in plan view. comprising a door pocket installed on the inside of the door,
In the captured image with the lens cover removed, the first area is set so that at least a part of the door pocket when the door is opened is captured in the first area. The storage according to claim 1, characterized in that: a plurality of first areas are set;
The control unit determines that the lens cover is attached to the lens when the average brightness of the highest average brightness among the plurality of first areas is equal to or lower than a first predetermined value. The storage according to claim 1. A second area is set inside the vignetting area in the captured image,
The control unit determines that the lens cover is attached to the lens if the average brightness of the second area is equal to or higher than a second predetermined value after performing exposure adjustment to adjust the gain of brightness of the photographed image. The storage according to claim 1, characterized in that it is determined that: The storage according to claim 1, wherein when the control unit determines that the lens cover is attached to the lens, the control unit does not transmit the photographing result of the camera unit to the server. Equipped with a buzzer that emits a predetermined sound,
The storage according to claim 1, wherein the control unit sounds the buzzer when determining that the lens cover is attached to the lens. A camera unit that is installed outside the housing of a storage warehouse and includes a housing that has a storage room, and a door that closes an opening of the housing, and that photographs at least the storage room.
a control unit that determines whether a lens cover is attached to the lens of the camera unit based on the brightness of a first area in the image taken by the camera unit;
In the camera unit, the first region is set so that at least a part of the storage chamber is captured in the first region in the photographed image with the lens cover removed.

Images