TW202417792A - Filming system, filming device, and refrigerator - Google Patents

Abstract

A photographic image that properly captures the interior of the refrigerator is selected from the images captured by the camera. The photographing system 7 comprises: a wide-angle camera 301, which is disposed on the top surface 20A of the refrigerator 2 and captures the refrigerator 2, wherein the refrigerator 2 has a refrigerating chamber 21 and a door 21A for opening and closing the opening 201 of the refrigerating chamber 21; a photographed object PS, which is disposed near the opening 201 of the refrigerating chamber 21 or the door 21A, and the photographic state captured by the wide-angle camera 301 changes according to the opening and closing angle of the door 21A; the state The memory unit 364 stores the first photographic state CD1, which is the photographic state of the photographed object PS photographed by the wide-angle camera 301 at a predetermined opening and closing angle of the door 21A; and the generation unit 356 selects the photographic image P1 when the door 21A is in the open state from the photographic image PB1 photographed by the wide-angle camera 301 according to the first photographic state CD1.

Description

Filming system, filming device, and refrigerator

The present disclosure relates to a photographing system, a photographing device, and a refrigerator.

Patent document 1 discloses a technology that installs a camera on the upper part of a refrigerator, performs photography when an object passing through the opening of the refrigerator is detected, and manages the food contained in the refrigerator based on the photography results. Prior art document Patent document

Patent document 1: Japanese Patent Application Publication No. 2019-070476

Invention Problems to be Solved

The present disclosure provides a photographic system, a photographic device, and a refrigerator for selecting a photographic image that appropriately captures the internal conditions of a refrigerator from images captured by a camera. Means for solving the problem

This specification contains all the contents of Japanese Patent Application No. 2022-165223 filed on October 14, 2022. The photographing system disclosed herein comprises: a camera disposed on the upper surface of a refrigerator and photographing the refrigerator, wherein the refrigerator has a storage chamber and an opening and closing body for opening and closing the opening of the storage chamber; a photographed object disposed near the opening of the storage chamber or the opening and closing body, and the photographing state photographed by the camera changes according to the opening and closing angle of the opening and closing body. a state memory unit for memorizing the photographic state of the photographed object photographed by the camera at a predetermined opening and closing angle of the aforementioned opening and closing body; and a selection unit for selecting a photographic image when the aforementioned opening and closing body is in an open state from the images photographed by the camera according to the photographic state of the photographed object photographed by the camera memorized by the state memory unit.

Furthermore, the photographing device disclosed in the present invention comprises: a camera, which is arranged on the upper surface of a refrigerator and photographs the refrigerator, wherein the refrigerator has a storage chamber and an opening and closing body for opening and closing the opening of the storage chamber; a state memory unit, which stores the photographing state of the photographed object photographed by the camera at a predetermined opening and closing angle of the opening and closing body; and a selection unit, which selects the state of the photographed object according to the above The state memory unit stores the photographic state of the photographed object photographed by the camera, and selects a photographic image when the opening and closing body is in an open state from the images photographed by the camera. The photographed object is arranged near the opening of the storage room or the opening and closing body, and the photographic state photographed by the camera changes according to the opening and closing angle of the opening and closing body.

Furthermore, the refrigerator disclosed in the present invention comprises: a camera disposed on the upper surface of the refrigerator and photographing the refrigerator, wherein the refrigerator has a storage chamber and an opening and closing body for opening and closing the opening of the storage chamber; a photographed object disposed near the opening of the storage chamber or the opening and closing body, and the photographing state photographed by the camera changes according to the opening and closing angle of the opening and closing body. a state memory unit for memorizing the photographic state of the photographed object photographed by the camera at a predetermined opening and closing angle of the opening and closing body; and a selection unit for selecting a photographic image when the opening and closing body is in an open state from the images photographed by the camera according to the photographic state of the photographed object photographed by the camera memorized by the state memory unit. Effect of the invention

The photographing system, photographing device, and refrigerator disclosed in the present invention can select a photographic image when the opening and closing body is in the open state from the images photographed by the camera according to the photographic state of the photographed object stored in the state memory unit. Therefore, according to the present invention, it is possible to select a photographic image that appropriately photographs the internal state of the refrigerator from the images photographed by the camera.

The form used to implement the invention

(Knowledge and insights underlying the present disclosure, etc.) When the inventor of the present invention thought of the present disclosure, there was already a technology for performing photography when an object passing through the opening of a refrigerator is detected, and managing the food contained in the refrigerator based on the photography results. Therefore, the inventor of the present invention discovered the problem of selecting a photographic image that appropriately photographs the internal conditions of the refrigerator, and completed the subject of the present disclosure to solve the problem. Therefore, the present disclosure provides a photography system, a photography device, and a refrigerator that select a photographic image that appropriately photographs the internal conditions of the refrigerator from an image photographed by a camera.

Below, the implementation form is described in detail with reference to the drawings. However, sometimes more detailed descriptions than necessary are omitted. For example, sometimes detailed descriptions of well-known matters or repeated descriptions of substantially the same structures are omitted. In addition, the attached drawings and the following descriptions are provided to enable those with ordinary knowledge in the relevant technical field to fully understand the present disclosure, and are not intended to limit the subject matter described in the scope of the patent application by these.

(Implementation form 1) [1-1. Configuration] [1-1-1. Configuration of article management system] FIG. 1 is a diagram showing the configuration of article management system 1 in implementation form 1. Article management system 1 is a system for managing articles contained in refrigerator 2. Article management system 1 of this implementation form manages images of articles contained in refrigerator 2 as an example of article management.

In this embodiment, the articles contained in the refrigerator 2 include food or containers for containing food, etc. Predetermined marks may also be attached to the articles.

The article management system 1 is provided with a refrigerator 2. The refrigerator 2 is provided with a main body 20 with an opening at the front. A refrigerating chamber 21, an ice-making chamber 22, a fresh freezing chamber 23, a freezer 24, and a vegetable chamber 25 are formed in the main body 20. These storage chambers are cooled by cold air. A revolving door 21A is provided at an opening 201 in front of the refrigerating chamber 21. The door 21A includes a revolving right door 21B and a revolving left door 21C. Drawers 22A, 23A, 24A, and 25A for storing articles are provided in the ice-making chamber 22, the fresh freezing chamber 23, the freezer 24, and the vegetable chamber 25, respectively. The drawers 22A to 25A are opened on the upper surface when they are set. In the following, when the drawer 22A, the drawer 23A, the drawer 24A, and the drawer 25A are collectively referred to without distinguishing them, the symbol "26A" is added and recorded as "drawer 26A". The refrigerator 21 is an example of a "storage room" corresponding to the present disclosure. In addition, the door 21A is an example of an "opening and closing body" corresponding to the present disclosure.

The article management system 1 is equipped with a camera 3. The camera 3 is installed on the top surface 20A of the main body 20 of the refrigerator 2. In addition, the top surface 20A of the main body 20 is made of metal. The camera 3 will take a picture of the refrigerator 2. The top surface 20A is an example of the "upper surface" corresponding to the present disclosure.

The article management system 1 has a terminal device 4. The terminal device 4 is a portable computer such as a smart phone. The terminal device 4 is installed with an application related to the management of articles contained in the refrigerator 2, and communicates with the article management server 5 through the function of this application. In the following description, this application is recorded as "article management application" and is attached with the symbol "411".

In FIG. 1 , a user P at home is shown by a solid line, and a user P outside the home H is shown by a dotted line. When the terminal device 4 is used by the user P at home, it communicates with the item management server 5 through the communication device 6 or without the communication device 6. Furthermore, when the terminal device 4 is used by the user P outside the home H and cannot establish a communication connection with the communication device 6, it communicates with the item management server 5 without the communication device 6.

The communication device 6 is connected to the network NW formed by a public line or the like, and communicates with the item management server 5 via the network NW. The communication device 6 functions as an interface device for connecting the refrigerator 2, the camera 3, and the terminal device 4 to the network NW. The communication device 6 constructs a local area network in the home H.

The article management system 1 includes an article management server 5. The article management server 5 is a device for managing articles contained in the refrigerator 2. The article management server 5 stores images of articles for article management. The article management server 5 is connected to the network NW. In addition, in each figure, although the article management server 5 is represented by a block, this does not necessarily mean that the article management server 5 is composed of a single device.

[1-1-2. Structure of refrigerator] FIG. 2 is a three-dimensional diagram of refrigerator 2 in the set state. FIG. 2 shows the X-axis, Y-axis, and Z-axis. The X-axis, Y-axis, and Z-axis are mutually orthogonal. The Z-axis shows the up-down direction, which is equivalent to the height direction of refrigerator 2. The X-axis and Y-axis are parallel to the horizontal direction. The X-axis shows the left-right direction. The left-right direction is equivalent to the width direction of refrigerator 2. The Y-axis shows the front-back direction. The positive direction of the X-axis shows the right direction. The positive direction of the Y-axis shows the front direction. The positive direction of the Z-axis shows the upper direction.

The refrigerator 2 has a main box 20, a door 21A, and a drawer 26A. The main box 20 is a box-shaped component that forms the body of the refrigerator 2. The main box 20 has: an outer box 202 that is opened at the front, an inner box 203 that is stored inside the outer box 202 and is opened at the front, and a heat insulating material that fills the space between the outer box 202 and the inner box 203. The outer box 202 is formed of metal such as iron or hard resin such as ABS. The upper surface of the outer box 202, that is, the top surface 20A of the main box 20, is formed of metal. The inner box 203 is formed of hard resin such as ABS. The heat insulating material can be, for example, a hard polyurethane foam. In addition, the heat insulating material is not limited to this, as long as it has heat insulation and can be filled in the space between the outer box 202 and the inner box 203.

The inner space of the inner box 203 is divided into a plurality of spaces by partition members. In this embodiment, the inner space of the inner box 203 is divided into five spaces, and the five spaces function as a refrigeration chamber 21, an ice making chamber 22, a fresh freezing chamber 23, a freezing chamber 24, and a vegetable chamber 25, respectively.

The right door 21B is pivotally supported so as to be rotatable by a right door hinge member 204. The right door hinge member 204 is a hinge member disposed between the top surface 20A of the main housing 20 and the right door 21B and the drawer 23A. The right door hinge member 204 disposed between the right door 21B and the drawer 23A is shown in FIG. 12. The left door 21C is pivotally supported so as to be rotatable by a left door hinge member 205. The left door hinge member 205 is a hinge member disposed between the top surface 20A of the main housing 20 and the right door 21B and the drawer 22A.

A second wireless communication module 206 is provided on the right door hinge member 204. The second wireless communication module 206 is a module for receiving control data for controlling the refrigerator 2 from an external device or for transmitting predetermined data to the external device.

[1-1-3. Configuration of the photographing device] The photographing device 3 is described with reference to FIGS. 3 to 7. In FIGS. 3 to 7, the X-axis, Y-axis, and Z-axis which are the same as those shown in FIG. 2 are shown. The X-axis direction corresponds to the width direction of the photographing device 3 and the left-right direction of the photographing device 3. The Y-axis direction corresponds to the front-back direction of the photographing device 3. The Z-axis direction corresponds to the up-down direction of the photographing device 3.

Fig. 3 is a side view of the camera 3 installed in the refrigerator 2 as viewed from the right. Fig. 4 is a perspective view of the limiting member 34 abutting against the front edge 20B of the main body 20 as viewed from the left front.

As shown in FIG3 and FIG4 , the photographing device 3 includes a main body 30 disposed on the top surface 20A of the main housing 20 and a photographing component 31 extending forward from the main body 30 . The photographing component 31 is disposed on the upper front portion of the main body 30 .

The main body 30 is provided with a power button 32 on the front for turning on and off the power of the photographing device 3. The power button 32 is provided with a first LED (Light emitting diode) 33 for notifying predetermined information.

The main body 30 has a limiting member 34. The limiting member 34 abuts against the front edge 20B of the main body 20, thereby limiting the position of the photographing member 31 in the front-rear direction of the refrigerator 2. The limiting member 34 is a plate-like member extending downward from the bottom surface 30A of the main body 30. The limiting member 34 extends in the width direction of the main body 30 (the width direction of the camera 3) in front of the main body 30, and has a mark 35 in the center of the width direction of the main body 30. The mark 35 indicates the center of the width direction of the camera 3. The limiting member 34 is formed to extend from the bottom surface 30A to a length shorter than the following length: the length from the top surface 20A of the main body 20 to the door gasket provided on the upper inner surface of the door 21A and roughly parallel to the top surface 20A.

Fig. 5 is a front view of the refrigerator 2 and the camera 3 installed in the refrigerator 2, viewed from the front. The refrigerator 2 shown in Fig. 5 is in a state where the door 21A is open. Fig. 6 is a side view of the refrigerator 2 and the camera 3 installed in the refrigerator 2, viewed from the right. The refrigerator 2 shown in Fig. 6 is in a state where the drawer 25A is open, i.e., has been pulled out. Fig. 7 is a plan view of the camera 3 installed in the refrigerator 2, viewed from above.

The camera 3 includes a wide-angle camera 301 and a narrow-angle camera 302. The wide-angle camera 301 and the narrow-angle camera 302 are disposed at the front end of the bottom surface 31A of the camera member 31. When the camera 3 is mounted on the refrigerator 2 and the door 21A is closed, the wide-angle camera 301 and the narrow-angle camera 302 are located in front of the door 21A.

The wide-angle camera 301 is a camera with a wider angle than the narrow-angle camera 302. When the camera 3 is mounted on the refrigerator 2, the wide-angle camera 301 photographs the refrigerator compartment 21 from the upper front of the refrigerator 2. For example, when viewed from the front of the refrigerator 2, the photographing range of the wide-angle camera 301 includes the range A1 shown in FIG. 5. The range A1 includes the door pockets 21E of the right door 21B and the left door 21C and the opening 201 of the refrigerator compartment 21 when the right door 21B and the left door 21C are opened at the maximum opening. Furthermore, when viewed from the right side of the refrigerator 2, the photographing range of the wide-angle camera 301 includes the range A3 shown in FIG. 6. Range A3 is a range from the front end of the right door 21B and the left door 21C in the open state to the front end of the shelf 21F in the front-back direction, and is a range that includes the opening 201 of the refrigerator compartment 21 in the up-down direction. The wide-angle camera 301 is an example of a "camera" corresponding to the present disclosure.

As shown in Fig. 5 to Fig. 7, the wide-angle camera 301 is a part of the shooting system 7 constituting the implementation form 1. The shooting system 7 of the implementation form 1 includes the wide-angle camera 301, the photographed object PS, the state memory unit 364, and the generation unit 356. In other words, the shooting system 7 includes: a refrigerator 2 having a photographed object PS; and a shooting device 3 having a wide-angle camera 301, a state memory unit 364, and a generation unit 356. The photographed object PS, the state memory unit 364, and the generation unit 356 are explained with reference to Fig. 8.

The narrow-angle camera 302 is a camera with a narrower angle than the wide-angle camera 301. When the camera 3 is installed on the refrigerator 2, the narrow-angle camera 302 takes a picture of the drawer 26A from the upper front of the refrigerator 2. For example, when viewed from the front of the refrigerator 2, the photographing range of the narrow-angle camera 302 includes the range A2 shown in FIG. 5. The range A2 is a range including the drawer 26A when viewed from the front of the refrigerator 2. For example, when viewed from the right of the refrigerator 2, the photographing range of the narrow-angle camera 302 includes the range A4 shown in FIG. 6. The range A4 is a range including the drawer 26A in the most pulled-out state in the front-back direction. In addition, the photographing range of the narrow-angle camera 302 may also be a range including the drawers set in the refrigerator compartment 21.

The camera device 3 has a first photo sensor 303 and a second photo sensor 304. The first photo sensor 303 and the second photo sensor 304 are provided on the main body 30. The first photo sensor 303 is a sensor for detecting the opening and closing of the right door 21B. The first photo sensor 303 is a reflective photo sensor. When the camera device 3 is installed on the refrigerator 2, the first photo sensor 303 projects light to a predetermined position of the right door 21B. Furthermore, the first photo sensor 303 outputs a signal indicating the amount of light received. The second photo sensor 304 is a sensor for detecting the opening and closing of the left door 21C. The second photo sensor 304 is a reflective photo sensor. When the camera 3 is installed on the refrigerator 2, the second photo sensor 304 projects light to a predetermined position of the left door 21C. In addition, the second photo sensor 304 outputs a detection value indicating the amount of light received.

The imaging device 3 includes a thermopile 305. The thermopile 305 is a sensor for detecting the opening and closing of the drawer 26A. The thermopile 305 is provided at the front end of the bottom surface 31A of the imaging member 31. The thermopile 305 outputs a detection value indicating the detected temperature.

The photographing device 3 includes a human body sensor 306. The human body sensor 306 may be an infrared sensor or an ultrasonic sensor. The human body sensor 306 outputs a detection value. The human body sensor 306 is provided at the front end of the bottom surface 31A of the photographing member 31.

The photographing device 3 includes an illuminance sensor 307. The illuminance sensor 307 is a sensor for adjusting the exposure of the wide-angle camera 301 and the narrow-angle camera 302. The illuminance sensor 307 outputs a detection value. The illuminance sensor 307 is provided at the front end of the photographing member 31.

The camera 3 includes a second LED 308 and a third LED 309. The second LED 308 is provided at the right front end of the bottom surface 31A of the camera member 31. The third LED 309 is provided at the left front end of the bottom surface 31A of the camera member 31. The second LED 308 and the third LED 309 illuminate the refrigerator 2 on which the camera 3 is mounted from above.

As shown in FIG. 7 , the imaging device 3 includes a first control substrate 311 and a second control substrate 312 .

The first control substrate 311 is a substrate for controlling various sensors provided in the photographing device 3 and the power supply of the photographing device 3. The first control substrate 311 is disposed inside the photographing device 3. The first control substrate 311 is connected to the power button 32, the first LED 33, the first light sensor 303, the second light sensor 304, the thermopile 305, the human body sensor 306, and the illuminance sensor 307. Furthermore, the first control substrate 311 controls these components. The first control substrate 311 is disposed behind the second control substrate 312 and the first wireless communication module 313.

The second control substrate 312 is a substrate for controlling the wide-angle camera 301 and the narrow-angle camera 302, and controlling the first wireless communication module 313. The second control substrate 312 is disposed inside the photographing device 3. The wide-angle camera 301, the narrow-angle camera 302, and the first wireless communication module 313 are connected to the second control substrate 312. Furthermore, the second control substrate 312 controls these components. The second control substrate 312 is disposed in front of the first control substrate 311 and arranged in parallel with the first control substrate 311.

The first wireless communication module 313 is a device having hardware such as an antenna or a wireless communication circuit. The first wireless communication module 313 is disposed inside the photographing component 31 to the left of the second control substrate 312. The first wireless communication module 313 is disposed in the photographing device 3 so that when the photographing device 3 is installed in the refrigerator 2, it is located in front of the opening 201 of the main body 20. More specifically, the first wireless communication module 313 is disposed in front of the limiting component 34 in the photographing device 3.

As shown in FIG. 7 , when the camera 3 is installed in the refrigerator 2 , a metal plate 314 is provided between the first wireless communication module 313 and the second wireless communication module 206 in the left-right direction.

FIG8 is a block diagram showing the structure of the camera 3. As shown in FIG8 , the first control substrate 311 has a first processor 320 such as a CPU (Central Processing Unit), a first memory 330, a first communication module 340, and an interface circuit for connecting to other devices or sensors.

The first memory 330 is a memory device for storing programs or data executed by the first processor 320. The first memory 330 is composed of a non-volatile memory device such as ROM (Read Only Memory). The first memory 330 may also include a volatile memory device such as RAM (Random Access Memory) that constitutes a working area of the first processor 320. The first memory 330 stores data processed by the first processor 320 or a control program 331 executed by the first processor 320.

The first communication module 340 has hardware such as a communication circuit or connector in accordance with a predetermined wired communication standard, and communicates with the second control substrate 312 according to the control of the first processor 320.

The first processor 320 reads and executes the control program 331, thereby turning the power of the camera 3 on and off in response to the operation of the power button 32. In addition, the first processor 320 reads and executes the control program 331, thereby turning on and off the first LED 33, the second LED 308, and the third LED 309. In addition, the first processor 320 reads and executes the control program 331, thereby sending the detection values output by each sensor connected to the first control substrate 311 to the second control substrate 312 through the first communication module 340.

The second control substrate 312 includes a second processor 350 such as a CPU, a second memory 360, a second communication module 370, and an interface circuit for connecting to other devices or sensors.

The second memory 360 is a memory device for storing programs or data executed by the second processor 350. The second memory 360 is composed of a non-volatile memory device such as a ROM. In addition, the second memory 360 may also include a volatile memory device such as a RAM that constitutes a working area of the second processor 350. The second memory 360 stores data processed by the second processor 350, a control program 361 executed by the second processor 350, or a camera ID (Identification) 362. Camera ID 362 is information that uniquely identifies the camera 3.

The second communication module 370 has hardware such as a communication circuit or connector in accordance with a predetermined wired communication standard, and communicates with the first control substrate 311 according to the control of the second processor 350.

The second processor 350 reads and executes the control program 361 stored in the second memory 360, thereby functioning as the first communication control unit 351, the second communication control unit 352, the opening and closing determination unit 353, the shooting control unit 354, the recording control unit 355, and the generating unit 356. In addition, the second processor 350 reads and executes the control program 361 stored in the second memory 360, thereby causing the second memory 360 to function as the state storage unit 364.

The first communication control unit 351 communicates with the item management server 5 via the first wireless communication module 313 .

The second communication control unit 352 communicates with the first control board 311 through the second communication module 370. The second communication control unit 352 receives detection values of various sensors connected to the first control board 311 from the first control board 311.

The opening/closing determination unit 353 determines whether the right door 21B is in the open state or the closed state based on the detection value of the first photo sensor 303 received by the second communication control unit 352. When the second communication control unit 352 has received a detection value different from the detection value corresponding to the closed state, the opening/closing determination unit 353 determines that the right door 21B is in the open state. On the other hand, when the second communication control unit 352 has received a detection value corresponding to the closed state, the opening/closing determination unit 353 determines that the right door 21B is in the closed state.

The opening/closing determination unit 353 is similar to the determination for the right door 21B. The second communication control unit 352 determines whether the left door 21C is in the open state or the closed state according to the detection value of the second photo sensor 304.

The opening/closing determination unit 353 determines whether the drawer 26A is in an open state or a closed state based on the detection value of the thermopile 305 received by the second communication control unit 352. When the second communication control unit 352 has received the detection value of the thermopile 305 indicating a temperature below a predetermined temperature, the opening/closing determination unit 353 determines that the drawer 26A is in an open state. On the other hand, when the second communication control unit 352 has received the detection value of the thermopile 305 indicating a temperature exceeding the predetermined temperature, the opening/closing determination unit 353 determines that the drawer 26A is in a closed state.

When the opening/closing determination unit 353 determines that the door 21A or the drawer 26A is in the open state, the photography control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to start photography. In addition, when the photography control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to start photography, the second communication control unit 352 sends lighting request information to the first control substrate 311. The lighting request information is information requesting the lighting of the second LED 308 and the third LED 309. When the first processor 320 receives the lighting request information, it lights the second LED 308 and the third LED 309 to notify the user P that photography has started.

When the opening/closing determination unit 353 determines that the door 21A and the drawer 26A are in a closed state, the photography control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to terminate photography. In addition, after the photography control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to terminate photography, the second communication control unit 352 sends light-off request information to the first control substrate 311. The light-off request information is information requesting the second LED 308 and the third LED 309 to be turned off. When the first processor 320 receives the light-off request information, it turns off the second LED 308 and the third LED 309, and notifies the user P that photography has been terminated.

When the opening/closing determination unit 353 determines that the door 21A or the drawer 26A is in the open state, the video recording control unit 355 starts recording the shooting results of the wide-angle camera 301 and the narrow-angle camera 302. That is, the video recording control unit 355 starts recording the video data 363 in the second memory 360. The video data 363 includes the first video data DV1 showing the shooting results of the wide-angle camera 301 and the second video data DV2 showing the shooting results of the narrow-angle camera 302.

When the opening/closing determination unit 353 determines that the door 21A and the drawer 26A are in the closed state, the recording control unit 355 ends the recording of the shooting results of the wide-angle camera 301 and the narrow-angle camera 302 .

The state memory unit 364 pre-stores the first photographic state CD1 and the second photographic state CD2. The first photographic state CD1 is a photographic state of the photographed object PS photographed by the wide-angle camera 301 at a predetermined opening and closing angle of the door 21A. When the opening and closing angle is zero degrees, the door 21A is displayed as being closed. The larger the opening and closing angle, the more the door 21A is displayed as being opened. The predetermined opening and closing angle is included in the range of, for example, 60 degrees to 100 degrees. The predetermined opening and closing angle is, for example, 90 degrees. In the following description, the state in which the door 21A is at a predetermined opening and closing angle (for example, 60 degrees to 100 degrees) is recorded as the "first reference state". The first photographic state CD1 is, for example, that the photographic image PA1 photographed by the wide-angle camera 301 in the first reference state includes at least one of the position, size, and shape of the image corresponding to the photographed body PS. In the first embodiment, the first photographic state CD1 is, for example, that the photographic image PA1 photographed by the wide-angle camera 301 in the first reference state includes the size of the image corresponding to the photographed body PS. The first photographic state CD1 is an example of "the photographic state of the photographed body photographed by the camera at a predetermined opening and closing angle of the opening and closing body" corresponding to the present disclosure.

The photographed object PS is arranged near the opening of the refrigerator room 21, the door 21A, and the photographic state photographed by the wide-angle camera 301 changes according to the opening and closing angle of the door 21A. The photographed object PS is further described with reference to FIGS. 9 to 14.

The second photographic state CD2 is a photographic state of the subject photographed by the narrow-angle camera 302 at a predetermined pull-out amount of the drawer 26A. When the pull-out amount is zero, the drawer 26A is displayed as being closed. The larger the predetermined pull-out amount is, the more the drawer 26A is displayed as being opened. The case where the pull-out amount is 100% is the case where the drawer 26A is displayed as being fully open. The predetermined pull-out amount is included in the range of, for example, 70% to 100%. The predetermined pull-out amount is, for example, 80%. In the following description, the state where the drawer 26A is at a predetermined pull-out amount (for example, 70% to 100%) is recorded as the "second reference state". The second photographic state CD2 is, for example, that the photographic image PA2 photographed by the narrow-angle camera 302 in the second reference state includes at least one of the position, size, and shape of the image corresponding to the photographed object. In the first and second embodiments, the second photographic state CD2 is, for example, that the photographic image PA2 photographed by the narrow-angle camera 302 in the second reference state includes the size of the image corresponding to the photographed object. The photographed object is, for example, arranged at the upper end of the drawer 26A, and the photographic state photographed by the narrow-angle camera 302 changes according to the amount of the drawer 26A being pulled out.

The generating unit 356 selects the first photographic image P1 from the photographic image PB1 included in the dynamic image shown in the first video data DV1 of the video data 363 according to the first photographic state CD1. The first photographic image P1 is a photographic image when the door 21A is in the open state. The photographic image PB1 is an example of "an image photographed by a camera" corresponding to the present disclosure. The first photographic image P1 is an example of "a photographic image when the opening and closing body is in the open state" corresponding to the present disclosure. Furthermore, the generating unit 356 selects the second photographic image P2 from the photographic image PB2 included in the dynamic image shown in the second video data DV2 of the video data 363 according to the second photographic state CD2. The second photographic image P2 is a photographic image when the drawer 26A is in an open state. Then, the generating unit 356 generates the management data D1 from the first photographic image P1 and the second photographic image P2. The management data D1 is data uploaded to the item management server 5 and is used to manage the items contained in the refrigerator 2. The first communication control unit 351 sends the management data D1 generated by the generating unit 356 to the item management server 5. The generating unit 356 is an example of a "selecting unit" corresponding to the present disclosure. In this embodiment, although the description is made with respect to the case where the photographic device 3 is equipped with the generating unit 356, it is not limited to this. For example, the item management server 5 may also be equipped with the generating unit 356. In this case, the management data D1 includes the first video data DV1 and the second video data DV2.

In addition, in the first embodiment, the generation unit 356 selects the first photographic image P1 from the photographic image PB1 included in the dynamic image shown in the first video data DV1 of the video data 363 according to the first photographic state CD1, but the present invention is not limited to this. The generation unit 356 may also select the photographic image PB1 with the least projection of the user P from the photographic image PB1 included in the dynamic image shown in the first video data DV1 of the video data 363 according to the first photographic state CD1 as the first photographic image P1. In this case, the generation unit 356 further selects the first photographic image P1 from the first photographic image P1 selected according to the first photographic state CD1 as follows. That is, the generating unit 356 recognizes the image of the user P by a method such as pattern matching, and selects the photographed image PB1 with the smallest area of the recognized image of the user P as the first photographed image P1.

Similarly, in the first embodiment, the generator 356 selects the second photographic image P2 from the photographic image PB2 included in the dynamic image shown in the second video data DV2 of the video data 363 according to the second photographic state CD2, but the present invention is not limited thereto. The generator 356 may select the photographic image PB2 in which the user P is least reflected as the second photographic image P2 from the photographic images PB2 included in the dynamic image shown in the second video data DV2 of the video data 363 according to the second photographic state CD2.

[1-1-4. Composition of the photographed body] Next, referring to FIG. 9, an example of the composition of the photographed body PS is described. FIG. 9 is a diagram showing the composition of the fourth embodiment of the photographed body PS in the embodiment 1. In the following description, the photographed body PS of the fourth embodiment is recorded as the photographed body PS4. The photographed body PS4 is composed of a connecting member connecting the left end of the right door 21B and the right end of the opening of the refrigerator compartment 21. The larger the opening and closing angle of the right door 21B, the larger the length LD of the photographed body PS4 in the left-right direction.

In this case, the state memory unit 364 stores the length LD in the shot image PA1 shot by the wide-angle camera 301 when the opening and closing angle is set to 60 degrees to 100 degrees as an example of the first shooting state CD1. In addition, the generation unit 356 selects the first shot image P1 when the right door 21B is in the open state from the shot image PB1 included in the dynamic image shown in the first video data DV1 of the video data 363 based on the length LD in the shot image PA1 stored in the state memory unit 364.

In addition, the generating unit 356 may select the first photographic image P1 when the right door 21B is in the open state from the photographic image PB1 included in the dynamic image shown in the first video data DV1 of the video data 363, based on the area of the photographic object PS4 in the photographic image PA1 stored in the state storage unit 364. In this case, the state storage unit 364 may store the area of the photographic object PS4 in the photographic image PA1 photographed by the wide-angle camera 301 when the opening and closing angle is set to 60 to 100 degrees as an example of the first photographic state CD1.

In FIG. 9, although the photographed object PS4 is described as being composed of a connecting member connecting the left end of the right door 21B and the right end of the opening of the refrigerator compartment 21, the photographed object PS4 may also be composed of a connecting member connecting the right end of the left door 21C and the left end of the opening of the refrigerator compartment 21. In addition, in FIG. 9, FIG. 10, FIG. 12, and FIG. 14, the same X-axis, Y-axis, and Z-axis as those shown in FIG. 2 are illustrated.

Next, the configuration of the photographed object PS is described with reference to FIGS. 10 to 16 . In addition, in the following description, the description is made for a case where the photographed object PS is arranged near the opening of the refrigerator compartment 21 or the right door 21B and is photographed by the wide-angle camera 301 . In addition, in the following description, the description is made for a case where the state memory unit 364 stores the first photographic object image MK1 as the first photographic state CD1 . In addition, the first photographic object image MK1 is described with reference to FIGS. 15 to 16 . The first photographic object image MK1 is an example of a “photographic object image” corresponding to the present disclosure.

Fig. 10-Fig. 11 are diagrams showing the configuration of the first embodiment of the photographed object PS in the embodiment 1. In the following description, the photographed object PS of the first embodiment is described as the photographed object PS1. The left figure of Fig. 10 is a front view of the refrigerator 2 and the camera 3 as viewed from the front, and corresponds to the upper part of Fig. 5.

The right figure of FIG. 10 is an enlarged view of the area AR1 of the left figure of FIG. 10. As shown in the right figure of FIG. 10, the photographed object PS1 is arranged in the door pocket 21E, and the aforementioned door pocket 21E is set at the top of the right door 21B. In FIG. 10-FIG. 11, although the case where the photographed object PS1 is arranged in the door pocket 21E of the right door 21B is described, the photographed object PS1 can also be arranged in the door pocket 21E of the left door 21C. The photographed object PS1 is, for example, a flat plate member AT3 formed in a rectangular shape. The flat plate member AT3 is fixed to the upper end of the side member 21S of the door pocket 21E through the connecting member AT2 and the mounting member AT1. The side member 21S is a member constituting the side of the door pocket 21E that is close to the opening of the refrigerator compartment 21 (the left side in FIG. 10 ). The flat member AT3, the connecting member AT2, and the mounting member AT1 are integrally constituted as the photographed member AT. The photographed member AT is formed of, for example, resin. The connecting member AT2 and the mounting member AT1 are examples of "fixed members" corresponding to the present disclosure.

The photographic plane PF1 is a plane photographed by the wide-angle camera 301 among the photographed subjects PS1. The photographic plane PF1 is fixed to the side member 21S of the door pocket 21E in a manner substantially orthogonal to the photographing direction DR of the wide-angle camera 301 when the right door 21B is at a predetermined opening and closing angle (e.g., 90 degrees). In other words, the photographed subject PS1 is arranged on the right door 21B so that the wide-angle camera 301 is located on the normal line of the center of the photographic plane PF1, that is, on a one-point link of the photographing direction DR shown in FIG. 10 .

As shown in FIG. 10 , since the photographed object PS1 is arranged in the uppermost door pocket 21E provided in the right door 21B, the distance between the wide-angle camera 301 and the photographed object PS1 can be shortened compared to, for example, the case where the photographed object PS1 is arranged in another door pocket 21E. As a result, the wide-angle camera 301 can appropriately shoot the first photographic object image MK1 formed on the photographic plane PF1 of the photographed object PS1 in the first reference state. Furthermore, as shown in FIG. 10 , since the photographed object PS1 is disposed at the side member 21S of the door pocket 21E of the right door 21B, the distance between the wide-angle camera 301 and the photographed object PS1 can be shortened compared to, for example, the case where the photographed object PS1 is disposed at other positions of the door pocket 21E. For example, the other positions of the door pocket 21E are the positions of the other side of the door pocket 21E (the right side in FIG. 10 ). As a result, the wide-angle camera 301 can appropriately capture the first photographic object image MK1 formed on the photographic plane PF1 of the photographed object PS1 in the first reference state. Furthermore, as shown in FIG. 10 , since the photographed object PS1 is disposed at the upper end of the side member 21S of the door pocket 21E of the right door 21B, it is possible to suppress the situation where the wide-angle camera 301 and the photographed object PS1 are blocked by the items stored in the refrigerator 2. As a result, the wide-angle camera 301 can appropriately shoot the first photographic object image MK1 formed on the photographing plane PF1 of the photographed object PS1 in the first reference state.

Fig. 11 is a three-dimensional diagram of the photographed object PS1. As shown by the dotted line in Fig. 11, a first photographed object image MK1 is formed on the photographing plane PF1 of the photographed object PS1, and the first photographed object image MK1 includes identification information showing the type of articles stored in the refrigerator compartment 21. For example, a sticker on which the first photographed object image MK1 is formed is attached to the photographing plane PF1.

The state memory unit 364 stores the first photographic object image MK1. The first photographic object image MK1 is an example corresponding to the first photographic state CD1. The generation unit 356 selects the first photographic image P1 when the right door 21B is in the open state from the captured image PB1 included in the dynamic image shown in the first video data DV1 of the video data 363 based on the first photographic object image MK1 stored in the state memory unit 364. In addition, the generating unit 356 can also select the first photographic image P1 when the right door 21B is in the open state from the photographic image PB2 included in the dynamic image shown in the second video data DV2 of the video data 363 according to the first photographic object image MK1 stored in the state storage unit 364. The first photographic object image MK1 is further described with reference to FIGS. 15-16.

As shown in FIG. 11 , the mounting member AT1 includes a first member AT11, a second member AT12, and an opening and closing member AT13. The opening and closing member AT13 is composed of a spring and the like, and can open and close the first member AT11 and the second member AT12. That is, the first member AT11 and the second member AT12 are set to an open state and inserted into the side member 21S of the door pocket 21E, and the first member AT11 and the second member AT12 are set to a closed state, thereby fixing the photographed body PS1 to the side member 21S of the door pocket 21E.

FIG. 12-FIG. 13 are diagrams showing the configuration of the second embodiment of the photographed object PS in the embodiment 1. In the following description, the photographed object PS of the second embodiment is described as the photographed object PS2. As shown in FIG. 12, the photographed object PS2 is configured as a hinge cover AS. The left figure of FIG. 12 shows a state where the hinge cover AS is not installed, and the right figure of FIG. 12 shows a state where the hinge cover AS is installed. In the left figure of FIG. 12, since the hinge cover AS is not installed, the right door hinge member 204 is exposed. The right door hinge member 204 supports the right door 21B axis so that it can rotate. The right door hinge component 204 is an example corresponding to the "hinge component".

The hinge cover AS is formed in an L-shape in a planar view, and a photographic surface PF2 is formed on the upper surface. A first photographic object image MK1 is formed on the photographic surface PF2, and the first photographic object image MK1 includes identification information showing the type of items stored in the refrigerator compartment 21. For example, a sticker having the first photographic object image MK1 is attached to the photographic surface PF2. The first photographic object image MK1 is further described with reference to FIGS. 15-16.

FIG. 13 is a perspective view of the hinge cover AS. As shown in FIG. 13 , a notch AS1 is formed on the hinge cover AS. The notch AS1 is engaged with the upper part of the frame of the drawer 23A. The hinge cover AS is formed of, for example, resin. In addition, the hinge cover AS may be the same color as the main box 20 or the hinge, or may be a different color. In addition, the state memory unit 364 and the generation unit 356 perform the same processing as that described with reference to FIG. 10-FIG. 11.

FIG. 14 is a diagram showing the structure of the third embodiment of the photographed body PS in the embodiment 1. In the following description, the photographed body PS of the third embodiment is described as the photographed body PS31, the photographed body PS32, and the photographed body PS33. The photographed body PS31, the photographed body PS32, and the photographed body PS33 correspond to an example of the photographed body PS of the third embodiment, respectively. As shown in FIG. 14, the photographed body PS31, the photographed body PS32, and the photographed body PS33 are respectively arranged near the opening of the refrigerator compartment 21. The photographed body PS31 is arranged on the lower right side of the inner surface of the refrigerator compartment 21. In other words, the photographed body PS31 is formed by a part of the lower right side surface of the inner surface of the refrigerator compartment 21. The photographing surface PF31 is formed integrally with the photographed body PS31. The first photographing object image MK1 is formed on the photographing surface PF31, and the first photographing object image MK1 includes identification information showing the type of items stored in the refrigerator compartment 21. For example, a sticker having the first photographing object image MK1 is attached to the photographing surface PF31.

Furthermore, the photographed body PS32 is arranged at the right end portion of the lower surface of the inner surface of the refrigerating chamber 21. In other words, the photographed body PS32 is constituted by a portion of the right end portion of the lower surface of the inner surface of the refrigerating chamber 21. The photographing plane PF32 is integrally constituted with the photographed body PS32. A first photographic object image MK1 is formed on the photographing plane PF32, and the first photographic object image MK1 includes identification information indicating the type of items stored in the refrigerating chamber 21. For example, a sticker on which the first photographic object image MK1 is formed is attached to the photographing plane PF32.

Furthermore, the photographed body PS33 is the right end portion of the upper surface of the front member in the drawer 23A of the fresh freezing chamber 23 disposed below the refrigerating chamber 21. In other words, the photographed body PS33 is constituted by a portion of the right end portion of the upper surface of the front member in the drawer 23A of the fresh freezing chamber 23 disposed below the refrigerating chamber 21. The photographing plane PF33 is integrally constituted with the photographed body PS33. The first photographic object image MK1 is formed on the photographing plane PF33, and the first photographic object image MK1 includes identification information indicating the type of items stored in the refrigerating chamber 21. For example, a sticker on which the first photographic object image MK1 is formed is attached to the photographing plane PF33.

In addition, the state storage unit 364 and the generation unit 356 execute the same processing as that described with reference to Figures 10 and 11.

In FIG. 14, although the case where the photographed object PS31 is arranged on the lower right side of the inner surface of the refrigerating chamber 21 is described, the photographed object PS31 may be arranged on the lower left side of the inner surface of the refrigerating chamber 21. In addition, in FIG. 14, although the case where the photographed object PS32 is arranged on the right end of the lower surface of the inner surface of the refrigerating chamber 21 is described, the photographed object PS32 may be arranged on the left end of the lower surface of the inner surface of the refrigerating chamber 21. In FIG. 14, although the case where the photographed object PS33 is arranged on the right end of the upper surface of the front member in the drawer 23A of the fresh freezing chamber 23 arranged below the refrigerating chamber 21 is described, it is not limited to this. The photographed object PS33 may be disposed at the left end portion of the upper surface of the front member in the drawer 22A of the ice making room 22 disposed below the refrigerating room 21.

Next, the first photographic object image MK1 is described with reference to FIGS. 15 and 16. FIG. 15 is a diagram showing the first embodiment of the first photographic object image MK1 of the photographed object PS in the embodiment 1. As shown in FIG. 15, the first photographic object image MK1 includes a rectangular outer frame image MF and an identification image MD formed inside the outer frame image MF. The identification image MD is identification information corresponding to the type of items stored in the refrigerator compartment 21 of the refrigerator 2. The outer frame image MF is formed in black, for example.

The identification image MD is composed of a first rectangular image PD1 of a first color and a second rectangular image PD2 of a second color different from the first color, and the arrangement pattern of the first rectangular image PD1 and the second rectangular image PD2 displays identification information. The first color is, for example, white, and the second color is, for example, black. The size of the first rectangular image PD1 is the same as the size of the second rectangular image PD2, and the first rectangular image PD1 and the second rectangular image PD2 are, for example, squares. In other words, the width WP1 of the first rectangular image PD1 and the width WP2 of the second rectangular image PD2 are the same as the height HP of the first rectangular image PD1 and the second rectangular image PD2.

The coarseness HF in the short side direction (up and down direction) of the outer frame image MF and the coarseness WF in the long side direction (left and right direction) of the outer frame image MF are respectively the same as, for example, the width WP1 of the first rectangular image PD1, the width WP2 of the second rectangular image PD2, the height HP of the first rectangular image PD1, and the height HP of the second rectangular image PD2. The coarseness HF in the short side direction (up and down direction) of the outer frame image MF and the coarseness WF in the long side direction (left and right direction) of the outer frame image MF are respectively determined based on the resolution of the wide-angle camera 301 and the distance between the wide-angle camera 301 and the first photographic object image MK1 at a predetermined opening and closing angle (e.g., 90 degrees) of the right door 21B. Furthermore, the coarseness HF in the short side direction of the outer frame image MF and the coarseness WF in the long side direction of the outer frame image MF are determined according to the shooting range or viewing angle of the wide-angle camera 301. For example, the greater the resolution of the wide-angle camera 301, the smaller the coarseness HF in the long side direction of the outer frame image MF and the coarseness WF in the short side direction of the outer frame image MF can be. Furthermore, the longer the distance between the wide-angle camera 301 and the first photographic object image MK1 at a predetermined opening and closing angle (e.g., 90 degrees) of the right door 21B, the greater the coarseness HF in the long side direction of the outer frame image MF and the coarseness WF in the short side direction of the outer frame image MF must be set. Furthermore, the larger the shooting range or viewing angle of the wide-angle camera 301, the larger the thickness HF in the long-side direction of the outer-frame image MF and the thickness WF in the short-side direction of the outer-frame image MF must be. In the first photographic object image MK1 shown in FIG. 15 , the thickness HF in the long-side direction of the outer-frame image MF and the thickness WF in the short-side direction of the outer-frame image MF are, for example, 6 mm, respectively.

The first rectangular image PD1 and the second rectangular image PD2 are arranged along the long side direction (here, the left-right direction) of the outer frame image MF. In FIG. 15 , five first rectangular images PD1 and four second rectangular images PD2 are arranged as the identification image MD. The first rectangular image PD1 is displayed as "0", and the second rectangular image PD2 is displayed as "1". That is, the identification image MD is a binary number displayed as 9 digits. The identification image MD is composed of an information image MDA and a coding image MDB. The information image MDA is composed of five rectangular images PD, and the coding image MDB is composed of four rectangular images PD. The rectangular image PD is the first rectangular image PD1 or the second rectangular image PD2. The information image MDA corresponds to the identification information for displaying the types of items stored in the cold storage room 21 of the refrigerator 2. The coded image MDB displays the code for error correction of the identification information displayed by the information image MDA. Since the information image MDA is composed of 5 rectangular images PD, a 5-digit binary number is displayed. That is, the information image MDA displays 2 to the fifth power, that is, 32 types of identification information. For example, the information image MDA shown in FIG. 15 displays a 5-digit binary number of "01010".

The size of the first rectangular image PD1 and the size of the second rectangular image PD2 are determined based on the resolution of the wide-angle camera 301 and the distance between the wide-angle camera 301 and the first photographic object image MK1 at a predetermined opening and closing angle (e.g., 90 degrees) of the right door 21B. In addition, the size of the first rectangular image PD1 and the size of the second rectangular image PD2 are determined based on the shooting range or viewing angle of the wide-angle camera 301. For example, the larger the resolution of the wide-angle camera 301, the smaller the size of the first rectangular image PD1 and the size of the second rectangular image PD2 can be. In addition, the longer the distance between the wide-angle camera 301 and the first photographic object image MK1 at a predetermined opening and closing angle (e.g., 90 degrees) of the right door 21B, the larger the size of the first rectangular image PD1 and the size of the second rectangular image PD2 must be set. Furthermore, the larger the shooting range or viewing angle of the wide-angle camera 301 is, the larger the size of the first rectangular image PD1 and the size of the second rectangular image PD2 must be. In the first photographic object image MK1 shown in FIG. 15 , the size of the first rectangular image PD1 and the size of the second rectangular image PD2 are squares with one side being 6 mm.

Fig. 16 is a diagram showing a second embodiment of the first photographic image MK1 of the photographed object PS in the embodiment 1. The first photographic image MK1 shown in Fig. 16 is different from the first photographic image MK1 shown in Fig. 15 in that the recognition image MD has two rows of rectangular images PD in the vertical direction.

The information image MDA is composed of 10 rectangular images PD, and the coded image MDB is composed of 8 rectangular images PD. Among the 8 rectangular images PD constituting the coded image MDB, the upper 4 rectangular images PD show the codes for error correction of the identification information displayed by the upper 5 rectangular images PD among the 10 rectangular images PD constituting the information image MDA. Furthermore, among the 8 rectangular images PD constituting the coded image MDB, the lower 4 rectangular images PD show the codes for error correction of the identification information displayed by the lower 5 rectangular images PD among the 10 rectangular images PD constituting the information image MDA.

Although the identification image MD is described with reference to the case where it has two rows of rectangular images PD in the vertical direction in FIG16 , the present invention is not limited thereto. The identification image MD may have three or more rows of rectangular images PD in the vertical direction.

In FIG. 15 and FIG. 16, although the outer frame image MF is described as being formed in black, the present invention is not limited thereto. For example, the outer frame image MF may be formed in blue. Furthermore, although the first color is described as white and the second color is black, the present invention is not limited thereto. For example, the first color may be yellow and the second color may be blue.

The photographed object PS1, the photographed object PS2, the photographed object PS31, the photographed object PS32, and the photographed object PS33 described with reference to Fig. 10 to Fig. 14 each have the first photographed object image MK1. Also, as shown in Fig. 15 to Fig. 16, the first photographed object image MK1 includes a rectangular outer frame image MF and an identification image MD formed inside the outer frame image MF. Then, the generating unit 356 performs the following processing.

That is, the generating unit 356 first determines whether the outer frame image MF can be detected from each of the captured images PB1 included in the dynamic image shown in the first recorded video data DV1 of the recorded video data 363. And, if the outer frame image MF can be detected from the captured image PB1, the generating unit 356 determines whether the identification information shown in the identification image MD inside the detected outer frame image MF can be read. Then, if the identification information shown in the identification image MD can be read and the read identification information is consistent with the identification information included in the first photographic object image MK1 stored in the state storage unit 364, the generating unit 356 selects the captured image as the first photographic image P1. Furthermore, when the outer frame image MF can be detected from each of the captured images PB1 included in the moving image shown in the first video data DV1, the generating unit 356 notifies the user P that the wide-angle camera 301 has photographed the subject PS by lighting, flashing, or making a sound, etc. of an LED. Furthermore, when the outer frame image MF can be detected from each of the captured images PB2 included in the moving image shown in the second video data DV2, the generating unit 356 notifies the user P that the narrow-angle camera 302 has photographed the subject PS by lighting, flashing, or making a sound, etc. of an LED. The situation where the outer frame image MF can be detected from each of the captured images PB1 is an example corresponding to "the situation where it is determined that the aforementioned camera has photographed the aforementioned subject".

Furthermore, in order to allow the generation unit 356 to easily detect the outer frame image MF from each of the captured images PB1 included in the dynamic image shown in the first video data DV1, it is desirable that the color of the outer frame image MF is different from the color of the inner surface of the refrigerating chamber 21 and the color of the items stored in the refrigerating chamber 21. Therefore, the color of the outer frame image MF is preferably black, blue, or the like.

In addition, a photographed object not shown in the figure is also arranged at the upper end of the drawer 26A, and the photographed object has a second photographic object image MK2, and the second photographic object image MK2 has the same structure as the first photographic object image MK1 described with reference to Figures 15 and 16. Therefore, the state storage unit 364 stores the second photographic object image MK2 as the second photographic state CD2. In addition, the generation unit 356 performs the following processing.

That is, the generating unit 356 first determines whether the outer frame image MF can be detected from each of the captured images PB2 included in the dynamic image shown in the second video data DV2 of the video data 363. And, if the outer frame image MF can be detected from the captured image PB2, the generating unit 356 determines whether the identification information shown in the identification image MD inside the detected outer frame image MF can be read. Then, if the identification information shown in the identification image MD can be read, and the read identification information is consistent with the identification information included in the second photographic object image MK2 stored in the state storage unit 364, the generating unit 356 selects the captured image PB2 as the second photographic image P2. Furthermore, the generating unit 356 notifies the user P that the narrow-angle camera 302 has photographed the subject PS by lighting, flashing, or making sounds of an LED, for example, when the outer-frame image MF can be detected from each of the captured images PB2 included in the dynamic image shown in the second video data DV2. The fact that the outer-frame image MF can be detected from each of the captured images PB2 is an example corresponding to "the fact that it is determined that the aforementioned camera has photographed the aforementioned subject".

[1-1-5. Configuration of the item management server] Figure 17 is a block diagram showing the configuration of the terminal device 4 and the item management server 5. The item management server 5 has a processor such as a CPU, namely a server processor 50, a server memory 51, and a server communication module 52. The server processor 50 is connected to the server memory 51 and the server communication module 52.

The server memory 51 is a memory device that stores programs or data executed by the server processor 50. The server memory 51 is composed of a non-volatile memory device. In addition, the server memory 51 may also include a non-volatile memory device that constitutes a work area of the server processor 50. The server memory 51 stores data processed by the server processor 50, a control program 511 executed by the server processor 50, and an item management DB (database) 512. The item management DB 512 has one record for each shooting device 3. This record includes the camera ID 362, the image data of the first photographed image P1, and the image data of the second photographed image P2.

The server communication module 52 has a communication circuit or connector according to a predetermined communication specification, and communicates with the refrigerator 2, the camera 3, and the terminal device 4 through the network NW.

The server processor 50 reads and executes the control program 511 stored in the server memory 51, thereby functioning as the server communication control unit 501 and the server processing unit 502. The server communication control unit 501 communicates with the refrigerator 2, the camera 3, and the terminal device 4 through the server communication module 52. The server processing unit 502 performs various information processing using the camera 3 and the terminal device 4 as clients. The server processing unit 502 can also use the refrigerator 2 as a client.

[1-1-6. Configuration of terminal device] Next, the configuration of the terminal device 4 is described. The terminal device 4 includes a processor such as a CPU, namely a terminal processor 40, a terminal memory 41, a terminal communication module 42, and a touch panel 43.

The terminal memory 41 is a memory device that stores programs or data executed by the terminal processor 40 in a non-volatile manner. The terminal memory 41 is composed of a non-volatile memory device. In addition, the terminal memory 41 may also include a RAM that constitutes a work area of the terminal processor 40. The terminal memory 41 stores data processed by the terminal processor 40, a control program executed by the terminal processor 40, and an item management application 411.

The terminal communication module 42 has a communication circuit or connector according to a predetermined communication specification, and communicates with the item management server 5 according to the control of the terminal processor 40. The touch panel 43 has a display panel and a touch sensor that is overlapped or integrated with the display panel.

The terminal processor 40 functions as an application execution unit 400. The application execution unit 400 communicates with the item management server 5 through the terminal communication module 42. The application execution unit 400 causes the touch panel 43 to display a user interface related to the management of items contained in the refrigerator 2. In the following description, this user interface is referred to as "application UI". The display control unit 402 of this embodiment displays images of items contained in the refrigerator 2 by displaying the application UI. The application execution unit 400 accepts various inputs from the user P through the application UI.

[1-2. Operation] Next, the operation of each part of the item management system 1 is explained.

[1-2-1. Actions related to video recording] First, the actions of the camera 3 related to video recording will be described. Fig. 18 is a flowchart FA showing the actions of the camera 3.

The open/close determination unit 353 determines whether the door 21A or the drawer 26A is in an open state (step SA1).

When the opening/closing determination unit 353 determines that the door 21A or the drawer 26A is not in the open state (step SA1: No), the second processor 350 returns the process to step SA1.

When the opening/closing determination unit 353 determines that the door 21A or the drawer 26A is in the open state (step SA1: Yes), the photography control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to start photography (step SA2).

Next, the recording control unit 355 starts recording the shooting results of the wide-angle camera 301 and the narrow-angle camera 302 (step SA3).

Next, the opening/closing determination unit 353 determines whether the door 21A and the drawer 26A are closed (step SA4). When the opening/closing determination unit 353 determines that the door 21A and the drawer 26A are not closed (step SA4: No), the determination of step SA4 is performed again.

On the other hand, when it is determined that the door 21A and the drawer 26A are closed (step SA4: Yes), the recording control unit 355 ends the recording of the shooting result (step SA5). As a result, the recording data 363 is generated in the second memory 360.

Next, the photography control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to finish photography (step SA6).

[1-2-2. Actions related to the transmission of management data] Next, the actions related to the transmission of management data D1 will be described. FIG. 19 is a flowchart showing the actions of the camera 3 and the item management server 5. In FIG. 19 , flowchart FB shows the actions of the camera 3, and flowchart FC shows the actions of the item management server 5.

The generating unit 356 determines whether the recorded video data 363 is stored in the second memory 360 (step SB1). When the generating unit 356 determines that the recorded video data 363 is not stored (step SB1: No), the determination of step SB1 is performed again.

On the other hand, when the generating unit 356 determines that the video data 363 has been stored (step SB1: yes), the first photographic image P1 is selected from the captured image PB1 contained in the dynamic image shown by the first video data DV1 of the video data 363 stored in the second memory 360 according to the first photographic state CD1 stored in the state memory unit 364 (step SB2).

Step SB2 is described in detail. The first photographic image P1 is a photographic image when the right door 21B or the left door 21C is opened at a predetermined opening and closing angle (e.g., 90 degrees).

The generation unit 356 first determines whether the outer-frame image MF can be detected from each of the captured images PB1 included in the dynamic image shown in the first video data DV1 of the video data 363. And, when the outer-frame image MF can be detected from the captured image PB1, the generation unit 356 determines whether the identification information shown in the identification image MD inside the detected outer-frame image MF can be read. Then, when the identification information shown in the identification image MD can be read and the read identification information is consistent with the identification information included in the first photographic object image MK1 stored in the state memory unit 364, the generation unit 356 selects the captured image PB1 as the first photographic image P1.

Returning to the description of the flowchart FB, the generating unit 356 selects the second photographic image P2 from each of the photographic images PB2 included in the dynamic image shown in the second video data DV2 of the stored video data 363 (step SB3).

Step SB3 is described in detail. The second photographic image P2 is a photographic image when the drawer 26A is in the open state.

The generation unit 356 first determines whether the outer-frame image MF can be detected from each of the captured images PB2 included in the dynamic image shown in the second video data DV2 of the video data 363. And, when the outer-frame image MF can be detected from the captured image PB2, the generation unit 356 determines whether the identification information shown in the identification image MD inside the detected outer-frame image MF can be read. Then, when the identification information shown in the identification image MD can be read and the read identification information is consistent with the identification information included in the second photographic object image MK2 stored in the state memory unit 364, the generation unit 356 selects the captured image PB2 as the second photographic image P2.

The generating unit 356 determines whether at least one of the first photographic image P1 and the second photographic image P2 has been selected (step SB4).

When the generating unit 356 determines that the first photographic image P1 and the second photographic image P2 cannot be selected (step SB4: No), it proceeds to the process of step SB7.

On the other hand, when the generating unit 356 determines that at least one of the first photographic image P1 and the second photographic image P2 has been selected (step SB4: Yes), it generates the management data D1 (step SB5).

When the first photographic image P1 and the second photographic image P2 are selected, the generating unit 356 generates management data D1 including image data of the first photographic image P1, image data of the second photographic image P2, and photographic device ID 362. When the first photographic image P1 is selected, but the second photographic image P2 cannot be selected, the generating unit 356 generates management data D1 including image data of the first photographic image P1 and photographic device ID 362. When the second photographic image P2 is selected but the first photographic image P1 cannot be selected, the generating unit 356 generates management data D1 including the image data of the second photographic image P2 and the photographic device ID 362.

Next, the first communication control unit 351 transmits the management data D1 generated by the generating unit 356 to the article management server 5 (step SB6).

Next, the generating unit 356 deletes the recorded video data 363 which is set as the processing object after step SB2 from the second memory 360 (step SB7).

As shown in the flowchart FC, the server communication control unit 501 determines whether the management data D1 has been received from the camera 3 (step SC1). When the server communication control unit 501 determines that the management data D1 has been received (step SC1: yes), the server processing unit 502 updates the item management DB 512 according to the management data D1 received by the server communication control unit 501 (step SC2).

Step SC2 is described in detail. The server processing unit 502 specifies the record of the camera ID 362 included in the management data D1 from the item management DB 512, and updates the image data of the first photographic image P1 in the specified record to the image data of the first photographic image P1 included in the management data D1. The server processing unit 502 specifies the record of the camera ID 362 included in the management data D1 from the item management DB 512, and updates the image data of the second photographic image P2 in the specified record to the image data of the second photographic image P2 included in the management data D1.

[1-2-3. Actions related to the application UI] Next, the actions of the item management system 1 related to the application UI are described. FIG. 20 is a flowchart showing the actions of the terminal device 4 and the item management server 5. In FIG. 20, flowchart FD shows the actions of the terminal device 4, and flowchart FE shows the actions of the item management server 5.

As shown in the flowchart FD, the application execution unit 400 determines whether to request the image data stored in the item management DB 512 from the item management server 5 (step SD1). For example, when the item management application 411 is activated, the terminal communication control unit 401 makes an affirmative determination in step SD1.

When the application execution unit 400 determines that the image data stored in the item management DB 512 is required (step SD1: Yes), it sends the request data for the image data to the item management server 5 (step SD2). The request data includes the camera ID 362 stored in the terminal memory 41. In addition, the terminal device 4 obtains the camera ID 362 at a predetermined time point.

As shown in the flowchart FE, the server communication control unit 501 receives the request data (step SE1). Then, the server processing unit 502 obtains the image data corresponding to the camera ID 362 included in the request data received by the server communication control unit 501 from the item management DB 512 (step SE2). Then, the server communication control unit 501 sends the image data obtained by the server processing unit 502 to the terminal device 4 as a response to the request data (step SE3).

Referring to flowchart FD, the application execution unit 400 receives image data from the item management server 5 (step SD3). Next, the application execution unit 400 displays the application UI on the touch panel 43, and the application UI includes the image shown by the image data received in step SD3 (step SD4).

[1-3. Effects, etc.] As described above, referring to FIGS. 1 to 20, the shooting system 7 in the embodiment 1 includes: a wide-angle camera 301, which is arranged on the top surface 20A of the refrigerator 2 and takes a picture of the refrigerator 2. The refrigerator 2 has a refrigerating chamber 21 and a door 21A for opening and closing the opening 201 of the refrigerating chamber 21; a photographed object PS, which is arranged near the opening 201 of the refrigerating chamber 21 or the door 21A, and the photographic state photographed by the wide-angle camera 301 is in accordance with the door 21A. The state memory unit 364 stores the first photographic state CD1, which is the photographic state of the photographed object PS photographed by the wide-angle camera 301 at the predetermined opening and closing angle of the door 21A; and the generation unit 356 selects the photographic image P1 when the door 21A is in the open state from the photographic image PB1 photographed by the wide-angle camera 301 according to the first photographic state CD1.

Accordingly, according to the photographic state of the photographed object PS photographed by the wide-angle camera 301 at the predetermined opening and closing angle of the door 21A, i.e., the first photographic state CD1, the first photographic image P1 when the door 21A is in the open state is selected from the photographic image PB1 photographed by the wide-angle camera 301. Thus, by appropriately setting the predetermined opening and closing angle, a photographic image that appropriately photographs the interior state of the refrigerator 2 can be selected as the photographic image when the door 21A is in the open state, i.e., the first photographic image P1. The predetermined opening and closing angle is, for example, 60 degrees to 100 degrees.

Furthermore, in the photographing system 7, the photographed object PS1 is arranged in the door 21A so that when the door 21A is at the aforementioned predetermined opening and closing angle, the photographing plane PF1 of the photographed object PS1 is substantially orthogonal to the photographing direction of the wide-angle camera 301.

Thus, since the photographic plane PF1 of the photographed object PS1 is approximately orthogonal to the photographing direction of the wide-angle camera 301 when the door 21A is opened and closed at the aforementioned predetermined angle, the photographed object PS1 can be properly photographed by the wide-angle camera 301 when the door 21A is opened and closed at the aforementioned predetermined angle.

Furthermore, in the photographing system 7, the photographed object PS1 is provided with a connecting member AT2 and a mounting member AT1, and the connecting member AT2 and the mounting member AT1 are disposed in the door pocket 21E of the door 21A and fix the photographed object PS1 to the door pocket 21E.

Thus, since the photographed object PS1 is fixed to the door pocket 21E, the photographing state photographed by the wide-angle camera 301 changes according to the opening and closing angle of the door 21A. Therefore, the photographed object PS1 can be arranged at an appropriate position.

Furthermore, in the photographing system 7, a right door hinge member 204 is provided for rotatably supporting the right door 21B, and the photographed subject PS2 is a hinge cover AS constituting the right door hinge member 204.

According to this, since the photographed object PS2 is formed as the hinge cover AS of the right door hinge member 204, the photographing state photographed by the wide-angle camera 301 changes according to the opening and closing angle of the door 21A. Therefore, the photographed object PS2 can be arranged at an appropriate position.

Furthermore, in the photographing system 7, a notch AS1 is formed in the hinge cover AS, and the notch AS1 is fitted into the upper portion of the frame of the drawer 23A below the right door.

According to this, since the notch AS1 formed in the hinge cover AS is fitted into the upper part of the frame of the drawer 23A below the right door, the hinge cover AS can be fixed to the right door hinge member 204. Therefore, the photographed object PS2 can be arranged at an appropriate position.

Furthermore, in the photographing system 7, a first photographic object image MK1 is formed on the photographing plane PF1 of the photographed object PS1, and the first photographic object image MK1 includes identification information, and the identification information indicates the types of items stored in the refrigerating chamber 21 and at least one of the refrigerating chambers 21 storing the items. The state memory unit 364 stores the first photographic object image MK1, and the generating unit 356 selects the first photographic image P1 when the door 21A is in an open state from the photographic image PB1 photographed by the wide-angle camera 301 based on the first photographic object image MK1 stored in the state memory unit 364.

Accordingly, from the captured images PB1 captured by the wide-angle camera 301, for example, the captured image PB1 that appropriately captures the first photographic object image MK1 is selected as the first photographic image P1, thereby appropriately selecting the first photographic image P1.

Furthermore, in the photographing system 7, the first photographic object image MK1 includes a rectangular outer frame image MF and an identification image MD. The identification image MD is formed inside the outer frame image MF and corresponds to identification information. The identification information is identification information showing the types of items stored in the refrigerating chamber 21 of the refrigerator 2.

According to this, from among the photographed images PB1 photographed by the wide-angle camera 301, for example, the photographed image PB1 in which the outer-frame image MF and the identification image MD are appropriately photographed is selected as the first photographed image P1, thereby making it possible to appropriately select the first photographed image P1.

Furthermore, in the photographing system 7, the identification image MD is composed of a first rectangular image PD1 of a first color and a second rectangular image PD2 of a second color different from the first color, and the first rectangular image PD1 and the second rectangular image PD2 are arranged in a pattern that displays the aforementioned identification information.

According to this, by appropriately setting each of the first color and the second color, the identification information indicated by the arrangement pattern of the first rectangular image PD1 and the second rectangular image PD2 can be accurately detected.

Furthermore, in the photographing system 7, the size of the first rectangular image PD1 is the same as the size of the second rectangular image PD2, and the first rectangular image PD1 and the second rectangular image PD2 are arranged along the long side direction of the outer-frame image MF.

Accordingly, since the size of the first rectangular image PD1 is the same as the size of the second rectangular image PD2, the identification information can be efficiently expressed by the arrangement pattern of the first rectangular image PD1 and the second rectangular image PD2. Furthermore, since the first rectangular image PD1 and the second rectangular image PD2 are arranged along the long side direction of the outer frame image MF, the first rectangular image PD1 and the second rectangular image PD2 can be arranged closely.

Furthermore, in the photographing system 7, the size of the first rectangular image PD1 and the size of the second rectangular image PD2 are determined based on the resolution of the wide-angle camera 301 and the distance between the wide-angle camera 301 and the first photographic object image MK1 at a predetermined opening and closing angle of the door 21A.

Accordingly, for example, the higher the resolution of the wide-angle camera 301, the smaller the size of the first rectangular image PD1 and the second rectangular image PD2 can be. Also, for example, the longer the distance between the wide-angle camera 301 and the first photographic object image MK1 at a predetermined opening and closing angle of the door 21A, the larger the size of the first rectangular image PD1 and the second rectangular image PD2 can be set. In this way, the size of the first rectangular image PD1 and the size of the second rectangular image PD2 can be appropriately determined.

Furthermore, in the photographing system 7, the generating unit 356 selects the photographed image PB1 as the first photographed image P1 when it is possible to detect the outer frame image MF included in the photographed image PB1 photographed by the wide-angle camera 301, and when the identification information shown in the identification image MD of the first photographed object image MK1 included in the photographed image PB1 photographed by the wide-angle camera 301 is consistent with the identification information shown in the identification image MD of the first photographed object image MK1 stored in the state memory unit 364.

Accordingly, when the outer frame image MF included in the captured image PB1 photographed by the wide-angle camera 301 can be detected, the captured image PB1 is an image that was photographed appropriately. Furthermore, when the identification information shown in the identification image MD of the first photographed object image MK1 included in the captured image PB1 is consistent with the identification information shown in the identification image MD of the first photographed object image MK1 stored in the state memory unit 364, the captured image PB1 is an image that was photographed more appropriately. Therefore, the first photographed image P1 can be easily and appropriately selected.

Furthermore, in the photographing system 7, when the generating unit 356 determines that the wide-angle camera 301 or the narrow-angle camera 302 has photographed the subject PS, it notifies the user P that the wide-angle camera 301 or the narrow-angle camera 302 has photographed the subject PS.

According to this, since the user P is informed that the wide-angle camera 301 or the narrow-angle camera 302 has photographed the subject PS, the user P can easily recognize that the wide-angle camera 301 or the narrow-angle camera 302 has photographed the subject PS. Therefore, the convenience of the user P can be improved.

Furthermore, the photographing device 3 in the embodiment 1 comprises: a wide-angle camera 301, which is disposed on the top surface 20A of the refrigerator 2 and photographs the refrigerator 2, wherein the refrigerator 2 has a refrigerating chamber 21 and a door 21A for opening and closing the opening 201 of the refrigerating chamber 21; a state memory unit 364, which stores a first photographing state CD1, wherein the first photographing state CD1 is a wide-angle photographed state of the photographed object PS1 at a predetermined opening and closing angle of the door 21A. The photographic state photographed by the camera 301; and the generating unit 356, according to the first photographic state CD1, selects the photographic image when the door 21A is in the open state, that is, the first photographic image P1, from the photographic image PB1 photographed by the wide-angle camera 301, the photographed object PS1 is arranged near the opening 201 of the cold storage room 21 or the door 21A, and the photographic state photographed by the wide-angle camera 301 changes according to the opening and closing angle of the door 21A.

Accordingly, the shooting device 3 in implementation form 1 can exert the same effect as the shooting system 7 in implementation form 1.

(Implementation Form 2) Next, Implementation Form 2 will be described. In the description of Implementation Form 2, the same symbols are attached to the components that are the same as the components of each part of the article management system 1 of Implementation Form 1, and the detailed description is appropriately omitted.

[2-1. Configuration] FIG. 21 is a diagram showing the configuration of the article management system 1A in the second embodiment. The article management system 1A in the second embodiment includes a refrigerator 2A, an article management server 5, and a terminal device 4. Although the refrigerator 2 and the camera 3 in the first embodiment are separate entities, the refrigerator 2A in the second embodiment includes a camera 3A. That is, the refrigerator 2A and the camera 3A in the present embodiment are not separate entities.

FIG. 22 is a block diagram showing the structure of the refrigerator 2A in this embodiment. The refrigerator 2A has a third control substrate 220. The third control substrate 220 has a third processor 230 such as a CPU, a third memory 240, a third communication module 250, and an interface circuit for connecting to other devices or sensors.

The third memory 240 is a memory device for storing programs or data executed by the third processor 230. The third memory 240 is composed of a non-volatile memory device. In addition, the third memory 240 may also include a volatile memory device constituting a working area of the third processor 230. The third memory 240 is for storing data processed by the third processor 230 or a control program 241 executed by the third processor 230.

The third communication module 250 has communication hardware such as a communication circuit or a connector in accordance with a predetermined wired communication standard, and communicates with the first control substrate 311A according to the control of the third processor 230.

The third control substrate 220 is connected to the second wireless communication module 206, the cooling unit 207, the first opening/closing sensor 208, the second opening/closing sensor 209, the third opening/closing sensor 210, the fourth opening/closing sensor 211, the fifth opening/closing sensor 212, and the sixth opening/closing sensor 213.

The cooling unit 207 has a mechanism for cooling each storage chamber of the refrigerator 2A, such as a compressor, a condenser, a capillary tube, a cooler, a cooling fan for sending the cold air generated by the cooler to each storage chamber, and an air gate for diverting the cold air sent by the cooling fan. The cooling unit 207 cools each storage chamber of the refrigerator 2A according to the control of the third control board 220.

The first opening/closing sensor 208 to the sixth opening/closing sensor 213 are constituted by, for example, a Hall IC. The first opening/closing sensor 208 is a sensor for detecting the opening/closing sensor of the right door 21B. The second opening/closing sensor 209 is a sensor for detecting the opening/closing sensor of the left door 21C. The third opening/closing sensor 210 is a sensor for detecting the opening/closing sensor of the drawer 22A. The fourth opening/closing sensor 211 is a sensor for detecting the opening/closing sensor of the drawer 23A. The fifth opening/closing sensor 212 is a sensor for detecting the opening/closing sensor of the drawer 24A. The sixth opening/closing sensor 213 is a sensor for detecting the opening/closing of the drawer 25A. Each opening/closing sensor outputs the detection value to the third control substrate 220.

The third processor 230 reads and executes the control program 241, thereby communicating with an external device through the second wireless communication module 206. In addition, the third processor 230 reads and executes the control program 241, thereby cooling each storage chamber of the refrigerator 2A with the cooling unit 207. In addition, the third processor 230 reads and executes the control program 241, thereby sending the detection value output by each of the first opening and closing sensors 208 to the sixth opening and closing sensors 213 to the first control substrate 311A.

The refrigerator 2A includes the camera 3A. Compared with the camera 3, the camera 3A does not include the first photo sensor 303, the second photo sensor 304, the thermopile 305, the human body sensor 306, the second LED 308, and the third LED 309.

Furthermore, the photographed object PS is disposed near the opening of the refrigerator compartment 21 of the refrigerator 2A, the right door 21B, or the left door 21C, and the photographic state photographed by the wide-angle camera 301 changes according to the opening and closing angles of the right door 21B and the left door 21C. Furthermore, the photographed object PS is a part of the refrigerator 2A disclosed herein.

The photographing device 3A of the second embodiment includes a first control substrate 311A instead of the first control substrate 311. The first control substrate 311A is connected to the power button 32, the first LED 33, and the illuminance sensor 307. Furthermore, the photographing device 3A of the second embodiment includes a second control substrate 312A instead of the second control substrate 312. The wide-angle camera 301, the narrow-angle camera 302, and the first wireless communication module 313 are connected to the second control substrate 312A.

The first control substrate 311A is disposed in the same position as the first control substrate 311 in the camera 3 in the camera 3. Furthermore, the second control substrate 312A is disposed in the same position as the second control substrate 312 in the camera 3 in the camera 3. Furthermore, in the present embodiment, the wide-angle camera 301, the narrow-angle camera 302, the power button 32, the first LED 33, the illuminance sensor 307, and the first wireless communication module 313 are disposed in the same position as the position in the embodiment 1.

The first control substrate 311A includes a first processor 320A such as a CPU, a first memory 330A, a first communication module 340A, and an interface circuit for connecting to other devices or sensors.

The first memory 330A is the same memory as the first memory 330. The first memory 330A stores data processed by the first processor 320A or a control program 331A executed by the first processor 320A.

The first communication module 340A has communication hardware such as a communication circuit or a connector in accordance with a predetermined wired communication standard, and communicates with the second control substrate 312A and the third control substrate 220 via cables.

The first processor 320A reads and executes the control program 331A, thereby turning the power of the camera 3A on and off in response to the operation of the power button 32. The first processor 320A reads and executes the control program 331A, thereby turning the first LED 33 on and off. The first processor 320A reads and executes the control program 331A, thereby sending the detection value output by the illuminance sensor 307 to the second control substrate 312A through the first communication module 340A. The first processor 320A reads and executes the control program 331A, thereby sending the detection values of the first opening and closing sensor 208 to the sixth opening and closing sensor 213 received from the third control substrate 220 to the second control substrate 312A through the first communication module 340A.

The second control substrate 312A includes a second processor 350A such as a CPU, a second memory 360A, a second communication module 370A, and an interface circuit for connecting to other devices or sensors.

The second memory 360A is the same memory as the first memory 330. The second memory 360A stores data processed by the second processor 350A or a control program 361A executed by the second processor 350A.

The second communication module 370A has communication hardware such as a communication circuit or a connector in accordance with a predetermined communication standard, and communicates with the first control substrate 311A according to the control of the second processor 350A.

The second processor 350A reads and executes the control program 361A stored in the second memory 360A, thereby functioning as the first communication control unit 351, the second communication control unit 352A, the opening/closing determination unit 353A, the shooting control unit 354, the recording control unit 355, and the generating unit 356. Furthermore, the second processor 350A reads and executes the control program 361A stored in the second memory 360A, thereby causing the second memory 360A to function as the state storage unit 364.

The refrigerator 2A in the second embodiment includes a wide-angle camera 301, a photographed object PS, a state storage unit 364, and a generation unit 356.

The second communication control unit 352A communicates with the first control board 311A via the second communication module 370A. The second communication control unit 352A receives detection values of various sensors from the first control board 311A.

The opening/closing determination unit 353A determines whether the right door 21B is in the open state or the closed state based on the detection value of the first opening/closing sensor 208 received by the second communication control unit 352A. When the second communication control unit 352A has received a detection value different from the detection value corresponding to the closed state, the opening/closing determination unit 353 determines that the right door 21B is in the open state. On the other hand, when the second communication control unit 352 has received a detection value corresponding to the closed state, the opening/closing determination unit 353 determines that the right door 21B is in the closed state.

The opening/closing determination unit 353A makes the same determination as for the right door 21B. The second communication control unit 352 determines whether the left door 21C is in the open state or the closed state according to the detection value of the second opening/closing sensor 209.

The opening/closing determination unit 353A determines whether the drawer 26A is in an open state or a closed state according to the detection values of the third opening/closing sensor 210 to the sixth opening/closing sensor 213 received by the second communication control unit 352. The opening/closing determination unit 353A makes a determination similar to the determination of the right door 21B.

[2-2. Action] The actions of each part of the article management system 1A in the implementation form 2 are the same as the actions of each part of the article management system 1 in the implementation form 1. That is, the action of the camera 3A related to the video recording is the same as the action of the camera 3 in the implementation form 1. In addition, the action of the camera 3A related to the management data D1 is the same as the action of the camera 3 in the implementation form 1.

[2-3. Effects, etc.] As described above, referring to FIG. 21-FIG. 22, the refrigerator 2A in the embodiment 2 is equipped with: a wide-angle camera 301, which is arranged on the top surface 20A of the refrigerator 2 and photographs the refrigerator 2. The refrigerator 2 has a refrigerator compartment 21 and a door 21A for opening and closing the opening 201 of the refrigerator compartment 21; a photographed object PS, which is arranged near the opening 201 of the refrigerator compartment 21 or the door 21A, and the photographic state photographed by the wide-angle camera 301 is in accordance with the door 21A. The state memory unit 364 stores the first photographic state CD1, which is the photographic state of the photographed object PS photographed by the wide-angle camera 301 at the predetermined opening and closing angle of the door 21A; and the generation unit 356 selects the photographic image P1 when the door 21A is in the open state from the photographic image PB1 photographed by the wide-angle camera 301 according to the first photographic state CD1.

Accordingly, the refrigerator 2A in implementation form 2 can exert the same effect as the shooting system 7 in implementation form 1.

(Other embodiments) As described above, embodiments 1 and 2 are described as examples disclosed in this application. However, the technology disclosed in this application is not limited thereto, and can also be applied to embodiments that have been modified, replaced, added, omitted, etc. In addition, each component described in embodiments 1 and 2 can be combined to create a new embodiment. Therefore, other embodiments are exemplified below.

In the above-mentioned embodiments 1 and 2, although the description is made with respect to the case where the photographed subject PS1 is arranged in the door pocket 21E of the right door 21B, the photographed subject PS1 may also be arranged in the door pocket 21E of the left door 21C.

In the above-mentioned embodiments 1 and 2, although the subject PS1 is described as being arranged in the door pocket 21E provided at the top of the door 21A, the subject PS1 may be arranged in the door pocket 21E of the door 21A. For example, the subject PS1 may be arranged in the second or third door pocket 21E provided in the door 21A from the top.

In the above-mentioned embodiments 1 and 2, although the description is made of the case where the photographed body PS1 is fixed to the side member 21S of the door pocket 21E of the door 21A, the photographed body PS1 only needs to be arranged in the door pocket 21E of the door 21A. For example, the photographed body PS1 may also be fixed to the front member. In FIG5, the front member is the member on the positive direction side of the Y axis of the door pocket 21E, that is, the member on the front side. In another example, the photographed body PS1 may also be fixed to the bottom member. In FIG5, the bottom member is the member on the negative direction side of the Z axis of the door pocket 21E, that is, the member on the lower side.

In the above-mentioned embodiments 1 and 2, the identification image MD is composed of the information image MDA and the coded image MDB, but the identification image MD may be composed of only the information image MDA. In other words, the identification image MD may not include the coded image MDB.

In the above-mentioned embodiments 1 and 2, although the first rectangular image PD1 and the second rectangular image PD2 are described as being square, the first rectangular image PD1 and the second rectangular image PD2 may be rectangular.

In the above-mentioned embodiments 1 and 2, although the outer frame image MF is described as being formed in black, it is not limited to this. For example, the outer frame image MF may also be formed in blue. Moreover, although the description is made for the case where the first color is white and the second color is black, it is not limited to this. For example, the first color may be yellow and the second color may be blue. In order to enable the generation unit 356 to easily detect the outer frame image MF from each of the captured images PB1 contained in the dynamic image shown in the first video data DV1, it is ideal that the color of the outer frame image MF is different from the color of the inner side surface of the refrigerator room 21 and the color of the items stored in the refrigerator room 21. Therefore, the color of the outer frame image MF is preferably black, blue, etc.

In the above-mentioned second embodiment, the first photo sensor 303, the second photo sensor 304, the thermopile 305, the human body sensor 306, the second LED 308, and the third LED 309 are not provided. In other embodiments of the second embodiment, any of these elements may be provided.

In the above-mentioned embodiments 1 and 2, the case where the widths of the right door 21B and the left door 21C are different is illustrated. In other embodiments, the widths of the right door 21B and the left door 21C may be the same, and the width of the left door 21C may be longer than the width of the right door 21B.

In the above-mentioned embodiments 1 and 2, the camera 3 and 3A are provided with the second LED 308 and the third LED 309. In other embodiments, the second LED 308 and the third LED 309 may be formed by one LED.

In the above-mentioned embodiment 2, the first opening/closing sensor 208 to the sixth opening/closing sensor 213 are exemplified as being constituted by a Hall IC. However, the first opening/closing sensor 208 and the second opening/closing sensor 209 may also be other sensors such as an angle sensor, a camera, a temperature sensor, etc. Furthermore, the third opening/closing sensor 210 to the sixth opening/closing sensor 213 may also be other sensors such as an acceleration sensor, a mechanical switch, an angle sensor, a camera, a temperature sensor, etc.

In other embodiments, the camera 3 may also be equipped with a subject detection unit. In this case, the recording control unit 355 may also control the start and end of recording based on the fact that the subject detection unit has detected the subject. In this way, the capacity of the recording data can be reduced. In addition, the processing speed can be improved. The subject detection unit is an example corresponding to the "selection unit". The operation of the camera 3 related to the recording based on the subject is described. FIG. 23 is a flowchart FF showing the operation of the camera 3.

The open/close determination unit 353 determines whether the door 21A or the drawer 26A is in an open state (step SF1).

When the opening/closing determination unit 353 determines that the door 21A or the drawer 26A is not in the open state (step SF1: No), the second processor 350 returns the process to step SF1.

When the opening/closing determination unit 353 determines that the door 21A or the drawer 26A is in the open state (step SF1: Yes), the photography control unit 354 causes the wide-angle camera and the narrow-angle camera 302 to start photography (step SF2).

Next, the object detection unit determines whether the object can be detected (step SF3).

When the subject detection unit determines that the subject cannot be detected (step SF3: No), the second processor 350 advances the processing to step SF7.

When the subject detection unit determines that the subject has been detected (step SF3: Yes), the recording control unit 355 starts recording the shooting results of the wide-angle camera 301 and the narrow-angle camera 302 (step SF4). In addition, it is also possible to start recording only the shooting results of the wide-angle camera 301 based on the subject corresponding to the wide-angle camera 301. Similarly, it is also possible to start recording only the shooting results of the narrow-angle camera 302 based on the subject corresponding to the narrow-angle camera 302. Furthermore, when the subject detection unit determines that the subject can be detected, the subject detection unit can also notify the user that the subject can be detected (the recording of the photographic result has started). Accordingly, since the user is notified that the subject has been detected (the recording of the photographic result has started), the user can easily recognize that the recording of the photographic result has started. As a result, the photographic noise can be reduced and the convenience of the user can be improved.

When the recording control unit 355 starts recording the shooting results of the wide-angle camera 301 and the narrow-angle camera 302, the subject detection unit determines whether the subject can be detected (step SF5).

When the subject detection unit determines that the subject has been detected (step SF5: yes), the determination of step SF5 is performed again.

On the other hand, when the subject detection unit determines that the subject cannot be detected (step SF5: No), the recording control unit 355 ends the recording of the shooting result (step SF6). As a result, the recording data 363 is generated in the second memory 360. In addition, the recording data 363 may also include a plurality of recording data. For example, a plurality of recording data may be generated in the second memory 360 in one shooting. In this case, the generation unit 356 may also perform the processing of steps SB1 to SB7 on the plurality of recording data. Furthermore, when the subject detection unit determines that the subject cannot be detected, the subject detection unit may also notify the user P that the subject cannot be detected (recording of the photographic result has been completed). Thus, since the user P is notified that the subject cannot be detected (recording of the photographic result has been completed), the user P can easily recognize that recording of the photographic result has been completed. As a result, the convenience of the user P can be improved.

Next, the opening/closing determination unit 353 determines whether the door 21A and the drawer 26A are closed (step SF7). When the opening/closing determination unit 353 determines that the door 21A or the drawer 26A is not closed (step SF7: No), the second processor 350 returns the process to step SF3.

On the other hand, when the opening/closing determination unit 353 determines that the door 21A or the drawer 26A is closed (step SF7: Yes), the photography control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to terminate photography (step SF8).

In other embodiments, the recording control unit 355 may also terminate the recording of the photographic result after a predetermined time (for example, 1 minute) has passed since the start of the recording of the photographic result.

In other implementation forms, the generating unit 356 may also compare the image captured by the camera with the pre-stored image of the door being in the open state to select the image of the door being in the open state.

In other embodiments, the camera 3 may also be provided with a camera confirmation unit. The camera confirmation unit is used to confirm that the camera 3 has been set at the predetermined position of the refrigerator 2 when the user sets the camera 3 in the refrigerator 2. In this case, when the camera confirmation unit detects the photographed object in the pre-memorized area, it can also notify the user that the camera 3 has been set at the predetermined position of the refrigerator 2 (for example, the center position of the refrigerator 2 in the left and right direction) by lighting, flashing, or making sounds of the LED, or notifying the terminal device 4. Accordingly, the user can easily recognize that the camera 3 has been set at the predetermined position of the refrigerator 2, and can proceed to the next setting operation. Thus, the user's convenience can be improved. Furthermore, when the photographic object cannot be detected in the pre-stored area, the photographic device confirmation unit may notify the user that the photographic device 3 is not set at the predetermined position of the refrigerator 2 by lighting, flashing, or making sounds of an LED, or by notifying the terminal device 4. Accordingly, the user can easily recognize that the photographic device 3 has not been set at the predetermined position of the refrigerator 2. Thus, the convenience of the user can be improved. Furthermore, the photographic device confirmation unit may make the notification to the user different when the photographic device 3 is set at the predetermined position of the refrigerator 2 and when it is not set at the predetermined position of the refrigerator 2. For example, the LED may be lit when the photographic device 3 is set at the predetermined position of the refrigerator 2, and the LED may be flashed when it is not set at the predetermined position of the refrigerator 2. According to this, the user can easily recognize that the camera 3 has been set at the predetermined position of the refrigerator 2 and that it has not been set at the predetermined position of the refrigerator 2. Thus, the convenience of the user can be improved.

In other embodiments, the photographic device 3 may also be provided with a subject confirmation unit. The subject confirmation unit is used to confirm that the subject has been placed in the predetermined position of the refrigerator 2 when the user places the subject in the refrigerator 2. In this case, the subject confirmation unit may also notify the user that the subject has been placed in the predetermined position of the refrigerator 2 by lighting, flashing, or making sounds of an LED, or notifying the terminal device 4, etc., when the subject information (information obtained from the photographed subject) is consistent with the information of the storage room memorized in advance. Accordingly, the user can easily recognize that the subject has been placed in the predetermined position of the refrigerator 2, and can proceed to the next setting operation. Thus, the user's convenience can be improved. Furthermore, when the information of the photographed object (information obtained from the photographed object) is inconsistent with the information of the storage room stored in advance, the object confirmation unit may notify the user that the photographed object has not been set at the predetermined position of the refrigerator 2 by lighting, flashing, or making a sound of an LED, or by notifying the terminal device 4 (for example, displaying the information of the photographed object). Accordingly, the user can easily recognize that the photographed object has not been set at the predetermined position of the refrigerator 2. Thus, the convenience of the user can be improved. Furthermore, the object confirmation unit may make different notifications to the user when the photographed object has been set at the predetermined position of the refrigerator 2 and when the subject has not been set at the predetermined position of the refrigerator 2. For example, the LED can be turned on when the object to be photographed is set at the predetermined position of the refrigerator 2, and can be turned on when the object to be photographed is not set at the predetermined position of the refrigerator 2. Thus, the user can easily recognize that the object to be photographed has been set at the predetermined position of the refrigerator 2 and that the object to be photographed has not been set at the predetermined position of the refrigerator 2. Thus, the convenience of the user can be improved.

In other embodiments, the photographed object may also include text or illustrations that can provide information about the storage room to be installed (e.g., the text "refrigerator" or an illustration of a refrigerator). Accordingly, when the user installs the photographed object in the refrigerator 2, the storage room to be installed can be easily identified. Thus, the user's installation errors can be suppressed and the user's convenience can be improved.

In other embodiments, in addition to the functional units of the second processors 350 and 350A in embodiments 1 and 2, the second processors 350 and 350A may also function as a detection unit for detecting objects. During the period from when the recording control unit 355 starts recording with the wide-angle camera 301 and the narrow-angle camera 302 until it ends, the detection unit may also extract images of objects entering and leaving the refrigerating chamber 21 from the dynamic images. Furthermore, when the objects are marked, the detection unit may also detect the marked objects.

In other embodiments, in addition to the functional units of the second processor 350 in embodiment 1, the second processor 350 can also function as a person presence determination unit. The person presence determination unit determines whether a person exists around the refrigerator 2 based on the detection value of the human body sensor 306. More specifically, the person presence determination unit determines that a person exists around the refrigerator 2 when the detection value of the human body sensor 306 corresponds to the detection value of the human body presence, and determines that there is no person around the refrigerator 2 when this is not the case. When the second processor 350 functions as a human presence determination unit, the photography control unit 354 starts photography with the wide-angle camera 301 and the narrow-angle camera 302 when the human presence determination unit determines that a human is present, and stops photography with the wide-angle camera 301 and the narrow-angle camera 302 when the human presence determination unit determines that no human is present. Furthermore, when the second processor 350 functions as a human presence determination unit, the video recording control unit 355 starts video recording when the human presence determination unit determines that a human is present, and stops video recording when the human presence determination unit determines that no human is present. Thus, even if the opening/closing determination unit 353 makes an erroneous determination, the shooting results of the wide-angle camera 301 and the narrow-angle camera 302 can still be properly recorded.

In other embodiments, in addition to the functional units of the second processors 350 and 350A in embodiments 1 and 2, the second processors 350 and 350A may also function as an entry and exit determination unit. In this case, the entry and exit determination unit changes the boundary line used for entry and exit determination in response to the opening and closing angles of the left door 21C and the right door 21B to determine the entry and exit of the items. In addition, the entry and exit determination unit changes the boundary line used for entry and exit determination in response to the pulling degree of the drawer 26A to determine the entry and exit of the items.

In other embodiments, the first photographic image P1 may be subjected to image correction and uploaded to the article management server 5. An example of image correction is correction by viewpoint conversion.

In the above embodiment, the first photographic image P1 and the second photographic image P2 are used for display purposes of the terminal device 4. However, the first photographic image P1 and the second photographic image P2 may also be used for other purposes such as detecting items contained in the refrigerator 2 or 2A.

In other embodiments, the compartments formed in the main box 20 of the refrigerator 2, 2A may be accommodating rooms other than the above five types, or may be less. In other embodiments, the number of doors provided in the opening 201 of the refrigerating chamber 21 may also be one.

The first processor 320, 320A, the second processor 350, 350A, the server processor 50, and the terminal processor 40 may be composed of a single processor or a plurality of processors. These processors may also be programmed to implement hardware corresponding to the functional units. That is, these processors may also be composed of, for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

The configuration of the camera 3, 3A, refrigerator 2, 2A, terminal device 4, and article management server 5 shown in FIG8, FIG17, and FIG22 is only an example, and the specific assembly form is not particularly limited. That is, it is not necessary to assemble individually corresponding hardware in each part, and it can also be set as a configuration in which the functions of each part are realized by a processor executing a program. In addition, part of the functions realized by software in the above-mentioned implementation form can also be set as hardware, or part of the functions realized by hardware can also be realized by software.

The step units of the actions shown in FIG. 18 , FIG. 19 , FIG. 20 , and FIG. 23 are divided according to the main processing contents for easy understanding of the actions, and the actions are not limited by the division method or name of the processing units. It can also be further divided into more step units according to the processing contents. In addition, it can also be divided so that one step unit contains more processing. In addition, the order of the steps can also be appropriately replaced within the scope of not hindering the main purpose of the present disclosure.

In addition, since the above-mentioned embodiments are used to illustrate the embodiments of the technology in the present disclosure, various changes, substitutions, additions, omissions, etc. can be made within the scope of the patent application or its equivalent scope.

(Note) The above description of the implementation form reveals the following technology.

(Technique 1) A photographing system comprising: a camera disposed on an upper surface of a refrigerator and photographing the refrigerator, wherein the refrigerator has a storage chamber and an opening and closing body for opening and closing the opening of the storage chamber; a photographed object disposed near the opening of the storage chamber or the opening and closing body, wherein the photographing state photographed by the camera corresponds to the opening and closing angle of the opening and closing body. The state memory unit stores the photographic state of the photographed object photographed by the camera at the predetermined opening and closing angle of the opening and closing body; and the selection unit selects the photographic image when the opening and closing body is in the open state from the images photographed by the camera according to the photographic state of the photographed object photographed by the camera stored in the state memory unit. According to this configuration, the photographic image when the opening and closing body is in the open state is selected from the images photographed by the camera according to the photographic state of the photographed object photographed by the camera at the predetermined opening and closing angle of the opening and closing body. Therefore, by appropriately setting the predetermined opening and closing angle, a photographic image that appropriately captures the interior condition of the refrigerator can be selected as a photographic image when the aforementioned opening and closing body is in the open state.

(Technique 2) A photographing system as described in Technique 1, wherein the photographed object is arranged on the opening and closing body so that when the opening and closing body is at the predetermined opening and closing angle, the photographing plane of the photographed object is substantially orthogonal to the photographing direction of the camera. According to this configuration, since when the opening and closing body is at the predetermined opening and closing angle, the photographed object is substantially orthogonal to the photographing direction of the camera, when the opening and closing body is at the predetermined opening and closing angle, the photographed object can be appropriately photographed by the camera.

(Technique 3) A shooting system as described in Technique 2, wherein the aforementioned object to be photographed is arranged in the door pocket of the aforementioned opening and closing body, and the aforementioned shooting system has a fixing member for fixing the aforementioned object to the aforementioned door pocket. According to this structure, since the aforementioned object to be photographed is fixed to the door pocket of the aforementioned opening and closing body, the photographic state of the aforementioned object to be photographed by the aforementioned camera will change in response to the opening and closing angle of the aforementioned opening and closing body. Therefore, the aforementioned object to be photographed can be arranged in an appropriate position.

(Technique 4) A photographing system as described in Technique 1, wherein a hinge member is provided to support the aforementioned opening and closing body axis so as to be rotatable, and the aforementioned photographed object is constituted as a hinge cover of the aforementioned hinge member. According to this configuration, since the aforementioned photographed object is constituted as a hinge cover of the aforementioned hinge member, the photographic state of the aforementioned photographed object photographed by the aforementioned camera changes in accordance with the opening and closing angle of the aforementioned opening and closing body. Thus, the aforementioned photographed object can be arranged at an appropriate position.

(Technique 5) A shooting system as described in Technique 4, wherein a notch is formed in the hinge cover, and the notch is engaged with the upper part of the frame of the drawer below the storage room. According to this structure, since the notch formed in the hinge cover is engaged with the upper part of the frame of the drawer below the storage room, the hinge cover can be fixed to the hinge member. Therefore, the photographed object can be arranged in an appropriate position.

(Technique 6) A photographing system as described in any one of Techniques 1 to 5, wherein a photographing object image is formed on the photographing surface of the photographed object, the photographing object image corresponds to identification information, the identification information indicates the type of items stored in the storage room and at least one of the storage rooms storing the items, the state memory unit stores the photographing object image, and the selection unit selects a photographing image when the opening and closing body is in the open state from the images photographed by the camera based on the photographing object image stored in the state memory unit. According to this configuration, from among the images photographed by the camera, for example, an image that properly photographs the image of the photographic object is selected as an image taken when the opening and closing body is in the open state, thereby properly selecting an image taken when the opening and closing body is in the open state.

(Technique 7) A photographing system as described in Technique 6, wherein the photographed object image includes a rectangular outer frame image and an identification image, the identification image is formed inside the outer frame image and corresponds to identification information, and the identification information is a display of the type of items stored in the storage room of the refrigerator. According to this configuration, from the images photographed by the camera, for example, a photographed image that properly captures the outer frame image and the identification image is selected as a photographed image when the opening and closing body is in the open state, thereby appropriately selecting a photographed image when the opening and closing body is in the open state.

(Technique 8) A shooting system as described in Technique 7, wherein the identification image is composed of a first rectangular image of a first color and a second rectangular image of a second color different from the first color, and the arrangement pattern of the first rectangular image and the second rectangular image is to display the identification information. According to this configuration, by appropriately setting each of the first color and the second color, the identification information shown by the arrangement pattern of the first rectangular image and the second rectangular image can be correctly detected.

(Technique 9) A shooting system as described in Technique 8, wherein the size of the first rectangular image is the same as the size of the second rectangular image, and the first rectangular image and the second rectangular image are arranged along the long side direction of the outer frame image. According to this configuration, since the size of the first rectangular image is the same as the size of the second rectangular image, the identification information can be efficiently expressed in the arrangement pattern of the first rectangular image and the second rectangular image. Furthermore, since the first rectangular image and the second rectangular image are arranged along the long side direction of the outer frame image, the first rectangular image and the second rectangular image can be arranged closely.

(Technique 10) A shooting system as described in Technique 8 or Technique 9, wherein the size of the first rectangular image and the size of the second rectangular image are determined based on the resolution of the camera and the distance between the camera and the image of the photographic object at the predetermined opening and closing angle of the opening and closing body. According to this configuration, for example, the higher the resolution of the camera, the smaller the size of the first rectangular image and the size of the second rectangular image can be. Also, for example, the longer the distance between the camera and the image of the photographic object at the predetermined opening and closing angle of the opening and closing body, the larger the size of the first rectangular image and the size of the second rectangular image are set. In this way, the size of the first rectangular image and the size of the second rectangular image can be appropriately determined.

(Technique 11) A photographing system as described in any one of Techniques 7 to 10, wherein the selection unit selects the image photographed by the camera as the photographed image when the opening and closing body is in the open state when the outer frame image included in the image photographed by the camera can be detected and the identification information shown in the identification image of the photographed object image included in the image photographed by the camera is consistent with the identification information shown in the identification image of the photographed object image stored in the state storage unit. According to this configuration, when the outer frame image included in the image photographed by the camera can be detected, the image photographed by the camera is an image photographed appropriately. Furthermore, when the identification information shown in the identification image of the photographed object image contained in the image photographed by the camera is consistent with the identification information shown in the identification image of the photographed object image stored in the state memory unit, the image photographed by the camera is a more appropriately photographed image. Thus, the photographed image when the switch is in the open state can be easily and appropriately selected.

(Technique 12) A shooting system as described in any one of Techniques 1 to 11, wherein the selection unit notifies a user that the camera has photographed the subject when it is determined that the camera has photographed the subject. According to this configuration, since the user is notified that the camera has photographed the subject when it is determined that the camera has photographed the subject, the user can easily recognize that the camera has photographed the subject. Thus, the convenience of the user can be improved.

(Technique 13) A photographing device comprising: a camera disposed on the upper surface of a refrigerator and photographing the refrigerator, wherein the refrigerator has a storage chamber and an opening and closing body for opening and closing the opening of the storage chamber; a state memory unit for storing the photographing state of the photographed object photographed by the camera at a predetermined opening and closing angle of the opening and closing body; and a selection unit for selecting the state of the photographed object according to the previous The photographic state of the photographed object photographed by the camera is stored in the state memory unit, and a photographic image when the opening and closing body is in the open state is selected from the images photographed by the camera. The photographed object is arranged near the opening of the storage room or the opening and closing body, and the photographic state photographed by the camera changes according to the opening and closing angle of the opening and closing body. According to this configuration, the same effect as the shooting system described in the technique 1 can be exerted.

(Technique 14) A refrigerator comprising: a camera disposed on an upper surface of the refrigerator and used to photograph the refrigerator, wherein the refrigerator has a storage chamber and an opening and closing body for opening and closing the opening of the storage chamber; a photographed object disposed near the opening of the storage chamber or the opening and closing body, wherein the photographing state photographed by the camera corresponds to the opening and closing angle of the opening and closing body. The state memory unit stores the photographic state of the photographed object photographed by the camera at the predetermined opening and closing angle of the opening and closing body; and the selection unit selects the photographic image when the opening and closing body is in the open state from the images photographed by the camera according to the photographic state of the photographed object photographed by the camera stored in the state memory unit. According to this structure, the same effect as the shooting system described in technology 1 can be exerted. Industrial applicability

As described above, the photographing system, photographing device, and refrigerator of the present invention can be used to select a photographic image that appropriately captures the internal conditions of the refrigerator from images taken by a camera.

1,1A: Item management system 2,2A: Refrigerator 3: Camera device (shooting device) 3A: Shooting device 4: Terminal device 5: Item management server 6: Communication device 7: Shooting system 20: Main cabinet 20A: Top surface 20B: Front edge 21: Refrigerator (storage room) 21A: Door (opening and closing body) 21B: Right door 21C: Left door 21E: Door pocket 21F: Shelf 21S: Side member 22: Ice making room 22A,23A,24A,25A,26A: Drawers 23: Fresh freezing room 24: Freezer room 25: Vegetable room 30: Main body 30A: Bottom 31: Photographic component 31A: Bottom 32: Power button 33: 1st LED 34: Limit component 35: Marker 40: Terminal processor 41: Terminal memory 42: Terminal communication module 43: Touch panel 50: Server processor 51: Server memory 52: Server communication module 201: Opening 20A: Top surface 202: Outer box 203: Inner box 204: Right door hinge component (hinge component) 205: Left door hinge component 206: 2nd wireless communication module 207: Cooling unit 208: 1st open/close sensor 209: 2nd open/close sensor 210: 3rd open/close sensor 211: 4th open/close sensor 212: 5th open/close sensor 213: 6th open/close sensor 220: 3rd control board 230: 3rd processor 240: 3rd memory 241: Control program 250: 3rd communication module 301: Wide-angle camera (camera) 302: Narrow-angle camera 303: 1st light sensor 304: 2nd light sensor 305: Thermopile 306: Human body sensor 307: Illuminance sensor 308: 2nd LED 309: 3rd LED 311,311A: 1st control board 312,312A: 2nd control board 313: 1st wireless communication module 314: Metal plate 320,320A: 1st processor 330,330A: 1st memory 331,331A: Control program 340,340A: 1st communication module 350,350A: 2nd processor 351: 1st communication control unit 352,352A: 2nd communication control unit 353,353A: Open/close judgment unit 354: Photography control unit 355: Recording control unit 356: Generation unit (selection unit) 360,360A: 2nd memory 361,361A: Control program 362: Camera ID 363: Video data 364: State memory 370,370A: Second communication module 400: Application execution unit 401: Terminal communication control unit 402: Display control unit 411: Item management application 501: Server communication control unit 502: Server processing unit 511: Control program 512: Item management DB A1,A2,A3,A4: Range AR1: Area AS: Hinge cover AS1: Notch AT: Photographed component AT1: Mounting component AT2: Connecting component (part of the fixed component) AT3: Flat component (part of the fixed component) AT11: 1st component AT12: 2nd component AT13: Open/Close component CD1: 1st shooting state (the shooting state of the object being photographed by the camera at the predetermined opening/closing angle of the opening/closing body) CD2: 2nd shooting state D1: Management data DR: Shooting direction DV1: 1st recording data DV2: 2nd recording data FA, FB, FC, FD, FE, FF: Flowchart H: Home HF: Coarseness HP: Height LD: Length MD: Identification image MDA: Information image MDB: Coded image MK1: 1st shooting object image (shooting object image) MK2: 2nd shooting object image MF: Frame image NW: Network P: User P1: 1st photographic image (photographed image when the shutter is open) P2: 2nd photographic image PA1, PA2: Photographed image PB1: Photographed image (image photographed by the camera) PB2: Photographed image PD: Rectangular image PD1: 1st rectangular image PD2: 2nd rectangular image PF1, PF2, PF31, PF32, PF33: Photographic surface PS, PS1, PS2, PS31, PS32, PS33, PS4: Photographed object SA1~SA6, SB1~SB7, SC1, SC2, SD1~SD4, SE1~SE3, SF1~SF8: Steps WP1, WP2: Width WF: Coarseness X: X axis Y: Y axis Z: Z axis

FIG. 1 is a diagram showing the structure of the article management system in the embodiment 1. FIG. 2 is a perspective view of the refrigerator in the embodiment 1. FIG. 3 is a side view of the camera in the embodiment 1 as viewed from the right. FIG. 4 is a perspective view of the limiting member abutting against the front edge of the main body in the embodiment 1 as viewed from the left front. FIG. 5 is a front view of the refrigerator and the camera in the embodiment 1 as viewed from the front. FIG. 6 is a side view of the refrigerator and the camera in the embodiment 1 as viewed from the right. FIG. 7 is a plan view of the camera installed in the refrigerator in the embodiment 1 as viewed from above. FIG. 8 is a block diagram showing the structure of the camera in the embodiment 1. FIG. 9 is a diagram showing the configuration of the fourth embodiment of the photographed object in the first embodiment. FIG. 10 is a diagram showing the configuration of the first embodiment of the photographed object in the first embodiment. FIG. 11 is a diagram showing the configuration of the first embodiment of the photographed object in the first embodiment. FIG. 12 is a diagram showing the configuration of the second embodiment of the photographed object in the first embodiment. FIG. 13 is a diagram showing the configuration of the second embodiment of the photographed object in the first embodiment. FIG. 14 is a diagram showing the configuration of the third embodiment of the photographed object in the first embodiment. FIG. 15 is a diagram showing the first embodiment of the photographed object image of the photographed object in the first embodiment. FIG. 16 is a diagram showing a second embodiment of a photographic object image of a photographed object in embodiment 1. FIG. 17 is a block diagram showing the configuration of a terminal device and an article management server in embodiment 1. FIG. 18 is a flowchart showing the operation of a photographic device in embodiment 1. FIG. 19 is a flowchart showing the operation of a photographic device and an article management server in embodiment 1. FIG. 20 is a flowchart showing the operation of a terminal device and an article management server in embodiment 1. FIG. 21 is a diagram showing the configuration of an article management system in embodiment 2. FIG. 22 is a block diagram showing the configuration of a refrigerator in embodiment 2. FIG. 23 is a flowchart showing the operation of a photographing device.

3: Photographic equipment

32: Power button

33: 1st LED

301: Wide-angle camera (camera)

302: Narrow-angle camera

303: 1st photo sensor

304: Second photo sensor

305: Thermopile

306: Human body sensor

307: Illuminance sensor

308: 2nd LED

309: 3rd LED

311: 1st control substrate

312: Second control substrate

313: No. 1 wireless communication module

320: Processor 1

330: Memory 1

331: Control program

340: 1st communication module

350: Second processor

351: 1st Communication Control Unit

352: Second communication control unit

353: Open/Close Judgment Unit

354: Photography Control Department

355: Video Control Department

356: Generation Department (Selection Department)

360: Memory 2

361: Control program

362: Camera device ID

363: Video recording

364: State memory unit

370: 2nd communication module

CD1: The first photographic state (the photographic state of the photographed object being photographed by the camera at the predetermined opening and closing angle of the opening and closing body)

CD2: Second photography state

DV1: 1st video data

DV2: 2nd video data

MK1: 1st photography object image (photography object image)

MK2: Second photography subject image

P1: The first photographic image (the photographic image when the switch is open)

P2: Second photographic image

PA1,PA2: Take pictures

PB1: Take pictures (pictures taken by the camera)

PB2: Take pictures

Claims (14)

A photographing system, comprising: A camera, arranged on the upper surface of a refrigerator, and photographing the refrigerator, wherein the refrigerator has a storage chamber and an opening and closing body for opening and closing the opening of the storage chamber; A photographed object, arranged near the opening of the storage chamber or the opening and closing body, wherein the photographing state photographed by the camera changes in accordance with the opening and closing angle of the opening and closing body; A state memory unit, which stores the photographing state of the photographed object photographed by the camera at a predetermined opening and closing angle of the opening and closing body; and The selection unit selects the photographic image when the switch is in the open state from the images photographed by the camera according to the photographic state of the photographed object stored in the state storage unit. As in claim 1, the photographic object is arranged on the opening and closing body so that when the opening and closing body is at the predetermined opening and closing angle, the photographing plane of the photographed object is approximately orthogonal to the photographing direction of the camera. As in claim 2, the photographic object is arranged in a door pocket of the opening and closing body, and the photographic system has a fixing member for fixing the photographed object to the door pocket. The shooting system of claim 1 is provided with a hinge member that supports the aforementioned opening and closing body axis to be rotatable, and the aforementioned photographed object is constituted as a hinge cover of the aforementioned hinge member. As in claim 4, a shooting system is provided, wherein a notch is formed in the hinge cover, and the notch is engaged with the upper portion of the frame of the drawer below the storage room. A photographing system as claimed in any one of claims 1 to 5, wherein a photographing object image is formed on the photographing surface of the photographed object, the photographing object image includes identification information, and the identification information indicates the type of items stored in the storage room and at least one of the storage rooms storing the items. The state memory unit stores the photographing object image, and The selection unit selects a photographing image when the opening and closing body is in an open state from the images photographed by the camera based on the photographing object image stored in the state memory unit. A photographing system as claimed in claim 6, wherein the photographed object image comprises: a rectangular outer frame image; and an identification image formed inside the outer frame image and corresponding to identification information, wherein the identification information indicates the type of items stored in the storage compartment of the refrigerator. A photographing system as claimed in claim 7, wherein the identification image is composed of a first rectangular image of a first color and a second rectangular image of a second color different from the first color, and the configuration of the first rectangular image and the second rectangular image is such that the identification information is displayed. As in claim 8, the size of the first rectangular image is the same as the size of the second rectangular image, and the first rectangular image and the second rectangular image are arranged along the long side of the outer frame image. A photographing system as claimed in claim 9, wherein the size of the first rectangular image and the size of the second rectangular image are determined based on the resolution of the camera and the distance between the camera and the photographic object image at a predetermined opening and closing angle of the opening and closing body. A photographing system as claimed in claim 7, wherein the selection unit selects the image photographed by the camera as a photographed image when the opening and closing body is in an open state, when the selection unit can detect the outer frame image included in the image photographed by the camera and the identification information shown in the identification image of the photographed object image included in the image photographed by the camera is consistent with the identification information shown in the identification image of the photographed object image stored in the state storage unit. As in the photographing system of claim 1, the selection unit, when determining that the camera has photographed the subject, notifies the user that the camera has photographed the subject. A photographing device, comprising: A camera, arranged on the upper surface of a refrigerator, and photographing the refrigerator, wherein the refrigerator has a storage chamber and an opening and closing body for opening and closing the opening of the storage chamber; A state memory unit, which stores the photographic state of the photographed object photographed by the camera at a predetermined opening and closing angle of the opening and closing body; and A selection unit, which selects a photographic image when the opening and closing body is in an open state from the images photographed by the camera according to the photographic state of the photographed object photographed by the camera stored in the state memory unit, The photographed object is arranged near the opening of the storage room or the opening and closing body, and the photographic state photographed by the camera changes according to the opening and closing angle of the opening and closing body. A refrigerator, comprising: A camera, arranged on the upper surface of the refrigerator, for photographing the refrigerator, wherein the refrigerator has a storage chamber and an opening and closing body for opening and closing the opening of the storage chamber; A photographed object, arranged near the opening of the storage chamber or the opening and closing body, wherein the photographic state photographed by the camera changes in accordance with the opening and closing angle of the opening and closing body; A state memory unit, for storing the photographic state of the photographed object photographed by the camera at a predetermined opening and closing angle of the opening and closing body; and The selection unit selects the photographic image when the switch is in the open state from the images photographed by the camera according to the photographic state of the photographed object stored in the state storage unit.

Images