KR101662087B1 - Refrigerator and Controlling Method for the same - Google Patents

Abstract

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a refrigerator and a control method thereof, and more particularly, to a refrigerator and a control method thereof that can provide information about food stored in a refrigerator even if a user does not open the door.
According to an embodiment of the present invention, there is provided a refrigerator comprising: a main body of a refrigerator, a storage chamber formed by a fixed heat insulating wall and provided with an opening; A door rotatably installed in the main body and opening / closing the opening; A drawer provided in the storage chamber; (1) in which a food is stored in an outer space of the drawer (2) and a region in which food is stored in an inner space of the drawer (2) A camera fixed to the camera body; (First region image) and the second region image (second region image) for the first region in the photograph through the photographing time point of the photograph taken together with the first region and the second region And a control unit for separating and storing the refrigerant separately and separately.

Description

[0001] DESCRIPTION [0002] Refrigerator and Controlling Method for the Same [

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a refrigerator and a control method thereof, and more particularly, to a refrigerator and a control method thereof that can provide information about food stored in a refrigerator even if a user does not open the door.

2. Description of the Related Art Generally, a refrigerator is a device for supplying cold air generated by driving a refrigeration cycle, and is a device for storing food in a low temperature state. In the case of a conventional refrigerator, only the function of storing food in a low temperature state can be performed. However, in recent years, there is an increasing need for additional functions in addition to the food storage function.

The refrigerator is a device for storing and storing a certain object. In order to check the inside of the refrigerator, the refrigerator door must be opened and checked. In addition, when the user tries to buy goods from markets or marts, he or she can not buy the same food or buy necessary food if the amount and type of the food stored in the refrigerator can not be grasped.

Japanese Patent Publication No. 3450907 and Japanese Patent Laid-Open No. 2004-183987 disclose techniques for photographing the inside of a refrigerator by installing a camera on a door. Further, according to Japanese Patent Application Laid-Open No. 2001-294308, a technique is disclosed in which a camera is installed in a room, a drawer, and a door, respectively.

However, according to the related art, there is a problem that it is necessary to use a plurality of cameras in order to capture various storage areas because the range of the camera installed in the refrigerator is limited.

Therefore, it is necessary to increase the shooting efficiency of the camera by increasing the range of the area photographed by the camera, while reducing the number of cameras installed in the hall.

Also, according to the prior art, there are various problems in acquiring images for the inside of the drawer. For example, concrete methods of obtaining photographed images at a desired point, troubles about condensation generated in the camera, and troubles for improving the power consumption by the camera have not been specifically made.

In particular, according to the related art, there is a problem that when there are a plurality of storage areas including a drawer in a hull, there is lack of specificity to provide optimal information to the user considering the specificity and positional relationship of each area.

The present invention basically aims to solve the above-mentioned problems.

The present invention provides a storage room display device for a refrigerator that provides information on food stored in a refrigerator even if a user does not open the refrigerator door and a control method thereof .

Also, it is an object of the present invention to provide a refrigerator and a control method thereof that allow a user to intuitively recognize information about food stored in each of a plurality of storage areas in a storage room.

The present invention also provides a refrigerator which can intuitively recognize a positional relationship between a plurality of storage areas and can provide stored food information that substantially matches the user's visual point of view. Control method.

In addition, through the embodiment of the present invention, the driving time of the camera and the photographing time of the camera are interlocked with the opening of the door and / or the rotation angle of the door to reduce the power consumption by the camera, And to provide a refrigerator and its control method capable of acquiring stored information.

In addition, through the embodiments of the present invention, it is intended to prevent the quality of a photograph taken by a camera from being lowered by condensation.

Also, it is possible to provide the user with the latest information on the food stored in the refrigerator through the embodiment of the present invention.

In addition, through an embodiment of the present invention, an administrator or a user can directly correct an area of a photo displayed to a user, thereby providing an optimal food storage information to a user.

Also, it is an object of the present invention to provide a refrigerator and a control method thereof, which can provide optimal food storage information to a user by preventing a camera from being shaken by installing a camera in a fixed body.

In addition, through the embodiments of the present invention, a refrigerator and its control method capable of effectively grasping and storing food storage information in the drawer while simplifying the configuration by grasping the state information of the drawer by software are provided.

The present invention also provides a refrigerator and its control method capable of providing the latest food storage information for each of a plurality of storage areas inside a storage room including a drawer by continuous shooting through one camera I want to.

In addition, through the embodiment of the present invention, it is possible to provide a food product which is stored in an inner space of a drawer and another storage area in which at least a part of the drawer overlaps vertically or in an inner space of another drawer in which at least part of the drawer is vertically overlapped And to provide a refrigerator and its control method capable of providing storage information using a single camera.

Also, it is possible to prevent degradation of the quality of the photographs taken by the camera due to condensation through the embodiments of the present invention.

Also, it is possible to provide a refrigerator which can reduce the cost of the refrigerator by limiting the number of cameras installed in the living room through one embodiment of the present invention.

Also, it is an object of the present invention to provide a refrigerator and a control method thereof that can effectively prevent an increase in power consumption due to a camera.

Also, it is an object of the present invention to provide a refrigerator and its control method capable of minimizing the load on the control unit and the memory unit and effectively and efficiently processing photographs taken continuously.

To achieve the above object, according to an embodiment of the present invention, there is provided a refrigerator comprising: a storage chamber formed in a body of a refrigerator by a fixed heat insulating wall and provided with an opening; A door rotatably installed in the main body and opening / closing the opening; A drawer provided in the storage chamber; (1) in which a food is stored in an outer space of the drawer (2) and a region in which food is stored in an inner space of the drawer (2) A camera fixed to the camera body; (First region image) and the second region image (second region image) for the first region in the photograph through the photographing time point of the photograph taken together with the first region and the second region And a control unit for separating and storing the refrigerant separately and separately.

Preferably, the control unit determines a photograph corresponding to the update condition for the first area photograph and the second area photograph from the photographed photographs, and the update condition is different according to the photographing time point.

Preferably, the control unit updates the first area photograph and the second area photograph individually and independently.

Preferably, the control unit determines a photograph taken at a specific time point when the door is opened after the door is opened, and extracts and stores an image for updating the first area.

Preferably, the control unit determines a photographed photograph at a specific time point when the drawer is drawn after being drawn, and extracts and stores an image for updating the second area.

Preferably, the control section separates the first region photograph and the second region photograph separately from each other in a plurality of photographs successively photographed, and separately divides and stores them separately.

The first region is an area where the food is placed by at least one shelf, and the first area may be located at the upper part of the second area.

And a display provided in front of the door to separately display the first area photograph and the second area photograph, and the first area photograph and the second area photograph are independently updated.

The storage area is provided with a third area which is provided at a lower portion of the drawer so as to vertically overlap with the drawer in a state that the drawer is drawn out and is separated from the first area and the second area to store food .

The first region and the second region are photographed together when the drawer is drawn out, and the first region and the third region may be photographed together when the drawer is drawn in.

 Preferably, the control unit separates the one area photograph and the third area photograph (third area photograph) from each other and separately categorizes and stores the one area photograph.

Preferably, the control unit updates the first area photograph, the second area photograph, and the third area photograph separately.

To achieve the above object, according to an embodiment of the present invention, there is provided a refrigerator comprising: a storage chamber formed in a body of a refrigerator by a fixed heat insulating wall and provided with an opening; A door rotatably installed in the main body and opening / closing the opening; A drawer provided in the storage chamber; (1) in which a food is stored in an outer space of the drawer (2) and a region in which food is stored in an inner space of the drawer (2) A camera fixed to the camera body; (First area photograph) and the second area photograph (second area photograph) for the first area are separated from the photograph through the photographing time of the photograph in which the first area and the second area are photographed together A control unit for individually dividing and storing data; And a display for displaying the first area photograph and the second area photograph so as to be distinguished from each other.

The display may display a first area frame to which the first area photograph is allocated and displayed and a second area frame to which the second area photograph is allocated and displayed.

The first region may be a region where food is stored by a shelf in an upper space above the drawer.

It is preferable that at least one of the first area photograph and the second area photograph stored in the controller is displayed after the size is corrected to match the first area frame and the second area frame size.

The storage area is provided at a lower portion of the drawer so as to vertically overlap with the drawer in a state where the drawer is drawn out, and a third area is provided in which the food is separated from the first area and the second area .

Wherein the control section separates the first region photograph and the third region photograph (third region photograph) from the photograph, through the photographing time point of the photograph taken together with the first region and the third region, Classification and storage.

And the third area frame to which the third area picture is allocated and displayed is displayed on the display so that the third area frame is divided below the first area frame.

The width of the first area frame and the width of the third area frame may be the same, and the third area photograph may be displayed after being corrected according to the width of the third area frame.

The drawers may include two drawers symmetrically disposed to the left and right, respectively, and the second area photographs may be separated and stored separately from each other.

The display may display an image captured through the camera and an adjustment line for adjusting a position of a separation line for separating the image into a plurality of regions.

Preferably, the adjustment line includes at least one horizontal adjustment line and at least one vertical adjustment line, and the adjustment line is movable through manipulation of the user's display.

To achieve the above object, according to an embodiment of the present invention, there is provided a refrigerator comprising: a storage chamber formed in a body of a refrigerator by a fixed heat insulating wall and provided with an opening; A door rotatably installed in the main body and opening / closing the opening; A drawer provided in the storage chamber; (1) in which a food is stored in an outer space of the drawer (2) and a region in which food is stored in an inner space of the drawer (2) A camera fixed to the camera body; (First area photograph) for the first area in a photograph taken at a specific point in time after the door is opened after the door is opened, And a controller for separating, dividing, storing, and updating the first area photograph and the second area photograph separately from each other.

Wherein the door comprises a left door and a right door, wherein the drawer is provided in each of the left and right sides of the storage room, and the control unit controls the portion for the left drawer and the portion for the right drawer It is desirable to separate, separate, store and update them individually.

And a door sensor for sensing an opening angle of the left door and the right door, respectively.

Preferably, the door sensor is provided at a connection portion of a hinge portion that connects the upper surface of the main body and the upper surface of the door to allow the door to pivot relative to the main body.

To achieve the above object, according to an embodiment of the present invention, there is provided a refrigerator comprising: a storage chamber formed in a body of a refrigerator by a fixed heat insulating wall and provided with an opening; A door rotatably installed in the main body and opening / closing the opening; A camera fixed to the storage compartment and photographing the inside of the storage compartment to update food information in the storage compartment; A door switch for detecting opening and closing of the door; A door sensor for sensing an opening angle of the door; According to a detection result of the door switch, a switching point between a standby mode of the camera and a driving mode for starting shooting is judged. According to the detection result of the door sensor, And a control unit for determining a reference time point for selecting the reference time point.

To achieve the above object, according to an embodiment of the present invention, A drawer movably provided in the storage chamber and provided with a marker; A camera for photographing the drawer outside the drawer; The position of the marker is sensed in the photograph successively photographed through the camera, and the state information of the drawer including at least one of the drawing degree, the drawing direction, the moving direction, the stopping state, and the moving state of the drawer is determined And a controller for controlling the operation of the refrigerator. Since the marker is moved integrally with the drawer, the state information of the drawer can be grasped through the position of the marker.

To achieve the above object, according to an embodiment of the present invention, A drawer movably provided at a lower portion of the storage compartment and having a marker; A camera fixed to the ceiling of the storage room and adapted to photograph the inside of the drawer from the outside of the drawer; The position of the marker is sensed in the photograph successively photographed through the camera, and the state information of the drawer including at least one of the drawing degree, the drawing direction, the moving direction, the stopping state, and the moving state of the drawer is determined And a controller for controlling the operation of the refrigerator.

To achieve the above object, according to an embodiment of the present invention, there is provided a refrigerator comprising: a storage chamber formed in a body of a refrigerator by a fixed heat insulating wall and provided with an opening; A door rotatably installed in the main body and opening / closing the opening; A drawer provided in the storage chamber; At least one shelf disposed in the storage chamber and positioned above the drawer; (First region) in which food is stored by the shelf in a space above the outer side of the drawer, and a region (second region) in which food is stored in the inner space of the drawer, And a camera fixed to a ceiling of the storage compartment between the front end of the shelf and the opening.

The above-described embodiments of the present invention may be embodied with each other as long as they are not mutually exclusive or exclusive.

According to the present invention, it is not necessary for the user to open the door to obtain information on the food stored in the refrigerator, so that the cold air contained in the storage chamber can be prevented from leaking. Therefore, it is possible to prevent unnecessary loss of cold air, thereby improving the energy efficiency of the refrigerator.

In addition, since the present invention provides the user with the latest information on the food stored in the refrigerator, the reliability of the food information provided to the user can be improved.

In addition, since the present invention can provide information on foods stored in various locations by a single camera, only a structure for installing one camera is added, so that the design can be easily performed. In particular, the cost of using the camera can be reduced.

According to the present invention, the camera installed in the refrigerator is prevented from being dewed, so that a picture taken by the camera can be stably provided to the user.

According to the present invention, an individual can remotely grasp the current state of the refrigerator, and can receive food related information necessary for an individual from an external provider.

According to the present invention, the image captured by the camera with respect to the interior of the drawer can be provided in a form similar to that of a user actually using the refrigerator.

Further, according to the present invention, since the images are overlapped in the spatial arrangement, positions that can not be viewed at a time can be provided to the user as a flat image on one screen.

1 is a front view of a refrigerator according to the present invention;
Figure 2 shows the door open in Figure 1;
Figure 3 shows a third area disposed at the bottom of the storage compartment;
4 is a control block diagram of an embodiment according to the present invention;
5 is a view for explaining an angle of view of a camera;
Fig. 6 is a side sectional view of Fig. 5; Fig.
FIG. 7 is a screen shot on the camera in the state of FIG. 6; FIG.
8 is a view for explaining the positioning of the camera.
9 is a sectional view showing a main part of a refrigerator.
10 is a view for explaining a manner of operation of the door sensor;
11 is a diagram specifically showing a left hinge portion and a right hinge portion.
12 is a screen provided to a user;
13 is a view for explaining a method of adjusting a picture photographed by a camera;
14 is a perspective view showing a camera.
Fig. 15 is a view showing a main part of a camera; Fig.
Fig. 16 is a sectional view of Fig. 14; Fig.
17 is a view for explaining the arrangement of heaters;
18 is a view for explaining a photograph taken in a state in which the camera is mounted on the refrigerator with the camera turned up and down.
Fig. 19 is a view for explaining a photograph taken when the camera is horizontally rotated in a refrigerator; Fig.
20 is a view showing a state in which a camera housing is assembled;
21 is a front view of the first housing;
22 is a front view of the second housing;
23 is a front view showing a state in which a camera is installed in the first housing;
24 is a side sectional view of Fig.
25 is a sectional view showing a state in which the camera housing is installed in the inner case;
26 is a table comparing the consumed power of the camera in the present invention.
FIG. 27 is a diagram comparing the supplied standby current with the driving current. FIG.
Figs. 28 and 29 are diagrams for explaining the starting point of photographing of the camera and the continuous photographing of the camera; Fig.
30 shows one embodiment of a draw detection unit;
31 is a view for explaining a method of sensing a movement of a drawer in the draw detection unit of FIG. 30;
32 is a diagram showing a state in which markers are displayed on a drawer;
Fig. 33 (a) is a photograph showing a state in which the left and right drawers are drawn out; Fig.
Fig. 33 (b) shows a state in which the left drawer is drawn and the right drawer is drawn. Fig.
34 is a flowchart illustrating a method of operating a refrigerator according to an embodiment of the present invention.
35 is a flowchart for explaining a marker recognition and tracking method according to an embodiment of the present invention;
36 is a view for explaining a marker for detecting a draw-in and-out point of a drawer according to an embodiment of the present invention;
37 is a view for explaining the shape of a marker according to an embodiment of the present invention.
38 is a flowchart illustrating a method of detecting a drawer closing operation according to an embodiment of the present invention.
FIG. 39 is a flowchart for explaining the operation of the refrigerator performed when the completion of the close operation of the drawer according to the embodiment of the present invention is detected. FIG.
40 is a flowchart illustrating a method of controlling an image of a specific area inside a refrigerator taken at a specific time according to an embodiment of the present invention.
41 illustrates various types of markers;
42 is a view for explaining a marker position recognition method;
Fig. 43 shows the extent to which the drawers are opened; Fig.
44 is a view for explaining movement of a marker;
45 is a control flow diagram according to an embodiment of the present invention;
46 is a control flowchart for explaining a modification of Fig. 45; Fig.
47 is a control flowchart for explaining another modification of Fig. 46; Fig.
FIG. 48 is a control flowchart illustrating another modification of FIG. 45; FIG.
FIG. 49 is a modification of FIG.
50 is a control flowchart according to another embodiment of the present invention;
51 is a view for explaining a process of updating an image with two doors and two drawers for opening and closing the storage room;
52 is a ladder diagram for explaining a refrigerator operating method according to another embodiment of the present invention.
53 is a ladder diagram for explaining a refrigerator operation method according to another embodiment of the present invention.
54 is a ladder diagram for explaining a refrigerator operation method according to another embodiment of the present invention.
55 is a ladder diagram for explaining a refrigerator operation method according to another embodiment of the present invention.
56 is a view for explaining the operation of the heater of the camera;
FIG. 57 is a view for explaining experimental results on condensation generated by temperature in a transparent window of a camera; FIG.
58 is a sectional view of a transparent window;
Figs. 59 and 60 schematically show a structure in which a camera is installed in an inner case; Fig.
61 is a view showing another type of refrigerator to which the present invention is applicable;
62 is a screen provided to the user from the refrigerator according to FIG.
Fig. 63 is a view for explaining a method of adjusting a picture photographed by a camera in a refrigerator according to Fig. 61;

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

FIG. 1 is a front view of a refrigerator according to the present invention, and FIG. 2 is a door opened in FIG. This will be described below with reference to Figs. 1 and 2. Fig.

The refrigerator according to the present invention includes a top mount type in which a freezing compartment and a refrigerating compartment in which food is stored are divided into upper and lower portions and a freezing compartment is disposed on the upper side of the refrigerating compartment; The present invention is equally applicable to a side-by-side refrigerator.

However, in the present embodiment, a bottom freezer-type in which the freezing compartment and the refrigerating compartment are partitioned upward / downward for convenience of explanation and the freezing compartment is disposed on the lower side of the refrigerating compartment will be mainly described.

The main body of the refrigerator includes an outer case 10 that forms an overall appearance when viewed from the outside and an inner case 12 that forms a storage room 22 in which food is stored. A predetermined space may be formed between the outer case 10 and the inner case 12 to form a passage through which cool air is circulated. Meanwhile, a space between the outer case 10 and the inner case 12 is filled with a heat insulating material, so that the inside of the storage chamber 22 can be maintained at a relatively low temperature as compared with the outside.

A refrigerant cycle device for circulating a refrigerant to generate cold air is installed in a machine room (not shown) formed between the outer case 10 and the inner case 12. By using the refrigerant cycle device, the freshness of the foodstuff can be maintained while keeping the inside of the refrigerator at a low temperature. The refrigerant cycle device includes a compressor for compressing a refrigerant, an evaporator (not shown) for phase-converting refrigerant in a liquid state into a gaseous state to cause heat exchange with the outside, and the like.

The refrigerator is provided with doors (20, 30) for opening and closing the storage room. At this time, each of the doors may include a freezing chamber door 30 and the refrigeration chamber door 20, and one end of each door is rotatably installed to the main body of the refrigerator by a hinge. The freezer compartment door 30 and the refrigerator compartment door 20 may have a plurality of openings. That is, as shown in FIG. 2, the refrigerator compartment door 20 and the freezer compartment door 30 may be installed so as to open toward both corners of the refrigerator toward the front.

Between the outer case 10 and the inner case 12, a foaming agent is filled, and the space between the outer case and the storage chamber 22 can be insulated.

The storage room (22) forms a space insulated from the outside by the inner case (12) and the door (20). The storage room 22 can be provided with a space for insulation from the outside when the door 20 hermetically seals the storage compartment 22. In other words, It is a space which is isolated from the outside through the heat insulating wall and the heat insulating wall by the case 10,

Cooling air supplied from the machine room can flow into the storage room 22, so that the food stored in the storage room 22 can be maintained at a low temperature.

The inner case 12 may form a barrier 60 on the bottom of the storage chamber 22. A barrier 60 is provided at the lower end of the storage compartment 22 to separate the freezing compartment from the refrigerating compartment. The barrier 60 may have a predetermined thickness and may be formed on the inner case 12. The barrier 60 may extend in the horizontal direction.

The storage room (22) may include a shelf (40) on which food is placed. At this time, a plurality of shelves (40) are provided, and food can be mounted on each shelf (40). The shelf 40 can divide the inside of the storage compartment horizontally.

The storage chamber 22 is provided with a drawer 50 which can be drawn in or drawn out. Food and the like are received and stored in the drawer (50). Two of the drawers 40 may be disposed in the storage chamber 22 to the left and right. The user may open the left door of the storage compartment 22 to approach the drawer disposed on the left side. On the other hand, the user may open the right door of the storage compartment 22 to access the drawer disposed on the right side.

A space for storing food may be formed in the barrier 60. Since the barrier 60 is not provided in the door but is provided in the inner case 12, the barrier 60 is not moved according to the rotation of the door and is held in a fixed state. Therefore, the user can stably store and draw food.

The storage space 22 is divided into a space located above the shelf 40, a space formed by the drawer 50, and a space formed in the barrier 60, Can be partitioned into a plurality of parts.

At this time, each space is a storage space disposed inside a storage compartment (22), and cool air supplied to the storage compartment (22) can be moved to each space. That is, since each space is divided to such a degree that cold air can be moved to each other, it has a different meaning from the storage room described above.

In particular, unlike a storage room where each space is an insulated space, there may be a temperature difference in one storage room but not a space that is insulated from each other.

The cool air supplied to one storage chamber does not move freely to another storage chamber, but the cool air supplied to one storage chamber can freely move into each partitioned space provided in one storage chamber. That is, the cool air located on the upper side of the shelf 40 is movable into the space formed by the drawer 50.

In addition, the present invention may include one camera 70 for photographing the inside of the storage room 22. [ At this time, the camera 70 is provided to keep the fixed position, and it is possible to photograph the same part.

Particularly, the camera 70 is installed on the upper wall of the inner case 12 so as to face downward, so that the camera 70 can photograph the food stored in the storage room 22. The photographed image may have a form that the user looks down upon when viewing the user's actual refrigerator.

Particularly, the camera 70 is installed at a position corresponding to the inside of the drawer when the drawer is in the fully opened state, so that the photographed picture can provide a screen similar to that of the user viewing the drawer from above.

At this time, the storage room 22 may include a first area 42 in which food is placed on the shelf 40, and a second area 52 defined by an inner space of the drawer 50. And a third region 62 that overlaps the second region 52 when the drawer 50 is drawn out, which is a storage space different from the first region and the second region. This part will be described in more detail later.

The first region 42 and the second region 52 may be entirely different in height within the storage chamber 22. That is, the first region 42 and the second region 52 in this order.

The photographs taken by the camera 70 or various information about the refrigerator can be provided to the user through the display 14 provided on the front of the refrigerator. The user can also control the refrigerator through the display 14.

The cool air supplied to the storage chamber 22 can be moved to the first area 42 and the second area 52, and the cool air located in each area can be moved to another area.

3 is a view showing a third region disposed at the bottom of the storage chamber. This will be described below with reference to Fig.

The barrier 60 may be provided with a depressed portion so that the food can be stored in the portion, and this region can be referred to as the third region 62. The third region 62 is a space formed in the barrier 60.

At this time, the third region is a storage space disposed at a position different from the first region and the second region, and can provide another space for storing food to the user.

In the third region 62, it is possible to store food having a high frequency of use such as eggs. For this purpose, it is possible to provide a cover 68 for opening and closing the third region 62. It is preferable that the cover 68 is made of a transparent material so that the food stored in the third region 62 can be photographed even if the cover 68 is covered when the camera 70 photographs from above .

On the other hand, the drawer 50 is disposed above the barrier 60. Therefore, the user can access the third area 62 when the drawer 50 is drawn into the lower space of the shelf.

On the other hand, in a state in which the drawer 50 is drawn out from the lower space of the shelf, the drawer 50 is arranged above the third area 62, and the user approaches the third area 62 Can not.

That is, in a state in which the drawer 50 is drawn out toward the user, the drawer 50 and the third region 62 are overlapped with each other. At this time, when the user looks down the drawer 50, the third area 62 is not shown to the user, and the inside of the drawer 50 can be seen.

On the other hand, in a state in which the drawer 50 is not drawn out toward the user, the drawer 50 and the third region 62 are arranged so as not to overlap with each other. When the user looks down on the third area 62, the interior of the drawer 50 is not shown to the user, and only the third area 62 can be shown.

The first region 42, the second region 52, and the third region 62 may have different elevations in the storage chamber 22 as a whole. That is, the first region 42, the second region 52, and the third region 62 in this order.

The cool air supplied to the storage compartment 22 can be moved to the first area 42, the second area 52 and the third area 62, Can also be moved.

That is, the area taken by the camera 70 may be a plurality of storage spaces in the storage room 22. The camera 70 photographs photographs related to the inside of one storage room 22, but information about a plurality of compartmented storage spaces may be included in one photograph.

At this time, the camera 70 can take a photograph so that a plurality of storage spaces are contained in one photograph. In particular, it is possible for one storage space to be photographed according to the shooting time.

For example, a camera arranged in a fixed form photographs a first area and a second area in accordance with a relationship between a point of time when a picture is taken and a position of the door or the drawer, As shown in FIG. Details of this will be described later.

4A is a control block diagram of an embodiment according to the present invention. This will be described below with reference to FIG.

The present invention may include a control unit 100 for dividing and managing one photograph taken by the camera 70 into a plurality of images. At this time, the image may be a part of the photograph that has been processed or corrected by the control unit 100.

The control unit 100 may instruct the camera 70 to take a picture and receive a picture taken by the camera 70. [

The controller 100 may also provide a portion of the divided image to the display 14 to provide the user with up-to-date information about the food stored in the storage room 22. At this time, the display 14 may be installed on the front surface of the refrigerator or may be a separate device separated from the refrigerator. That is, the user can receive the image of the storage room 22 through the external communication terminal such as a mobile phone and obtain the information.

At this time, the control unit 100 divides the photographs taken by the camera 70 into the respective regions, and divides the images into a plurality of independent images, and provides the separated images to the display 14. At this time, the photograph selected by the controller 100 may include information about the latest storage room state in which the user finally accesses the storage room 22 and draws food in the storage room or puts food in the storage room.

The embodiment of the present invention may include a door switch 110 that can detect whether the door 20 is opened or closed in the storage room 22. At this time, the door switch 110 is provided in the outer case 10, and when the door 20 is brought into contact with the outer case 10, the door 20 seals the storage room 22 can do. Also, if the door 20 is not in contact with the outer case 10, the door 20 can be detected as opening the storage compartment 22.

In addition, the embodiment of the present invention may include a door sensor 120 for sensing the rotation angle of the door 20. At this time, the door sensor 120 may sense the rotation direction of the door 20 and the rotation angle of the door 20. [ For example, when the door 20 rotates by a predetermined angle or more, the door sensor 120 may be changed to detect that the door 20 is rotated by a predetermined angle or more have. Further, when the door 20 is rotated in a specific direction, it is possible to sense a rotation direction of the door 20 by sensing a change in a pulse generated in a direction. Of course, the door sensor 120 can be modified into various forms other than the above-described system.

In addition, the door sensor 120 may include a light emitting portion and a light receiving portion to determine whether the light emitted from the light emitting portion is transmitted to the light receiving portion, and to sense the rotation angle of the door 20.

In particular, the door sensor 120 may be driven only when the door 20 determines that the storage room 22 is opened in the door switch 110.

On the other hand, when the door sensor 120 determines that the door 20 has been rotated at a specific angle, it is also possible to take a picture with the camera 70. In this case, even if the opening of the door 20 is sensed by the door switch 110, the camera 70 does not immediately start photographing, but when the door sensor 120 reaches a certain angle, .

When the photographing of the camera 70 is started, it is also possible to continuously photograph a photograph at predetermined time intervals. Of course, if the door sensor 120 determines that the door 20 has been opened by a predetermined angle, the camera 70 can continuously photograph the image until the photographing stop instruction is provided instead of a single photograph Do.

The control unit 100 may include a drop sensing unit 130 for sensing whether the drawer 50 is drawn or drawn. In this case, the drop sensing unit 130 may refer to a portion in which the photograph taken by the camera 70 reads an image in the control unit 100. [ The drop sensing unit 130 can sense not only the movement of the drawer 50 but also the movement direction of the drawer 50.

In particular, the drop detection unit 130 may be implemented by software. The drop sensing unit 130 may sense the position of the drawer 50 using the information photographed in the photograph taken by the camera 70. [

The drawer 50 is provided with a recognition mark called a marker to indicate whether the drawer 50 has entered a specific section through a photographed picture of the marker or stopped when the marker 50 has entered, Can be obtained. The marker will be described later in detail.

Since the markers are displayed on the drawers, they have the same movement as the drawers 50. Therefore, various information regarding the movement of the drawer 50 can be obtained by using the information of the photo taken by the camera 70, without having a separate sensor for detecting the movement of the drawer.

The photograph taken by the camera 70 may be stored in the storage unit 18. At this time, the storage unit 18 may be provided in a refrigerator or in another device separate from the refrigerator. The storage unit 18 may be disposed in a refrigerator together with the display 14 or may be provided in a terminal connected to a network with a server or a refrigerator connected to a network with a refrigerator and a device other than the refrigerator together with the display 14 have.

At this time, the storage unit 18 may not store all the photographs taken by the camera 70. For example, when the control unit 100 does not generate a command to select a specific picture, the storage unit 18 deletes the previously stored picture and stores the currently taken picture (First In , First Out). The storage unit 18 stores only a part of the photographs taken by the camera 70, thereby reducing the storage capacity.

4B is a control block diagram of a modified form of an embodiment according to the present invention. This will be described below with reference to FIG. 4B.

4B, the drawer sensing unit 130 may detect the movement of the drawer 50 separately without using the information obtained from the photograph taken by the camera 70, unlike FIG. 4A.

For example, the drop sensing unit 130 may have the form of a plurality of Hall sensors. A plurality of hall sensors may be provided in the movement path of the drawer (50). When the drawer 50 is moved, it senses a change in the plurality of hall sensors, and can detect the position and the moving direction of the drawer 50.

When the drawer is moved by the drawer sensing unit 130 and the camera 70 photographs the picture, it is possible to grasp the time when the inside of the drawer is filled with the picture taken by the camera 70 . That is, the drop sensing unit 130 may sense a point in time when information about an article stored in the drawer can be displayed on a photograph taken by the camera.

The drop sensing unit 130 senses the movement of the drawer and can detect when the user determines that the drawer is in the latest state after using the drawer. That is, the drop detection unit 130 may detect a point in time when the user includes the latest information that the user has finished using the drawer. That is, it is possible to indirectly grasp the completion time of the goods in and out, or the end point of the article update.

5 is a view for explaining the angle of view of the camera. This will be described below with reference to FIG.

In FIG. 5, for convenience of explanation, the drawer 50 is drawn out so that the camera 70 can photograph the second area 52, and the shelf is omitted.

The camera 70 may be disposed on the upper wall of the inner case 12 so as to be inclined toward the rear wall of the inner case 12. [

On the other hand, the photograph taken by the camera 70 may have a horizontal angle of view along the width direction of the storage room 22 and a vertical angle of view along the forward and backward directions.

At this time, the horizontal viewing angle of the camera 70 is set such that the food stored in the inner space of the two drawers 50, that is, the food in the second area 52, can be seen while the two drawers 50 are drawn out .

The range of the horizontal angle of view according to the camera 70 may be such that both ends of the shelf 40 are at least partially included. This is because it is preferable to photograph both ends of the shelf 40 from the camera 70 in order to acquire information about the food placed on the shelf 40.

As the angle of view of the camera 70 changes, the range inside the storage compartment shown in the photograph changes.

Fig. 6 is a side sectional view of Fig. 5, and Fig. 7 is a view taken on the camera in the state of Fig. This will be described below with reference to Figs. 6 to 7. Fig.

In the inner case 12, an opening 14 is formed in front of the storage compartment 22 so that the user can access the storage compartment 22. The user can insert food into the storage compartment 22 or take food out of the storage compartment 22 through the opening 14. The door (20) can open / close the opening (14).

The camera 70 may be installed outside the drawer to take a picture while the drawer is moving or stopping, and photographs the opened state, the closed state, and the moving state of the drawer. That is, when the camera 70 performs continuous shooting, a state in which the position of the drawer is changed according to a photographing time may be displayed on a photograph taken by the camera 70.

In addition, the camera 70 may provide a screen to the user that includes the latest state of goods in and out of the storage room 22.

Preferably, the camera 70 is installed within a range 1 between the opening 14 and the front end of the shelf 40. It is preferable that the shelf 40 is installed in a region (1, see FIG. 8) between the front end positioned at the foremost position of the front ends and the opening 14 when there are a plurality of shelves 40. In other words, it is preferable to be installed on the ceiling of the storage room corresponding to this area. Since the camera 70 must selectively photograph the first area and the second area 52 or the first area and the third area 62, Can not be moved to the rear wall of the inner case (12).

6 shows a vertical angle of view of the camera 70. One end (front end) of the vertical angle of view is preferably arranged so as to be able to photograph the front end of the drawer 50 in a state that the drawer 50 is open Do. Further, it is preferable that the other end (rear end) of the vertical angle of view is arranged so as to be able to take a picture to the rear end of the shelf at the uppermost side.

Since the camera 70 includes the second area 52 by the drawer 50 in the range of the vertical angle of view of the camera 70, Can be obtained.

As shown in FIG. 7, the camera 70 can take a picture of the first area 42 and the second area 52 with a single photograph. At this time, the camera 70 can take photographs of the two drawers 50 disposed on the right and left sides. In addition, when a plurality of shelves are provided, it can be seen that the first area 42 and the second area 52, which are divided by the plurality of shelves, can be taken together.

The camera 70 may be installed inside the storage room 22 and take a picture including information about the inside of the storage room 22. [ Since the camera 70 is installed inside the storage room, it is not moved. In other words, the camera itself is not moved because it is generally fixed to the storage room of the main body which is always fixed. Therefore, when the camera 70 photographs a picture, the camera 70 is always in a stopped state.

The photograph taken by the camera 70 includes the first area 42 and the second area 52 or includes the first area 42 and the third area 62.

When the camera 70 photographs the photograph including the first area 42 and the second area 52, the user can recognize the food stored on the upper side of the shelf 40, Information about the food stored in the lower portion 50 can be obtained.

On the other hand, when the camera 70 photographs the photograph including the first area 42 and the third area 62, the user can view the food stored on the upper side of the shelf 40, Information about the food stored in the barrier 60 can be obtained.

The camera 70 photographs the interior of one storage room 22 and transmits photographed photographs, and the controller 100 can acquire a plurality of storage space information.

The camera 70 is disposed above the first area 42, the second area 52, and the third area 62 to take a picture while looking downward.

Fig. 8 is a view for explaining camera positioning. This will be described below with reference to FIG.

As shown in FIG. 8, the camera can be mounted at the positions of L1, L2, L3, and L4.

L1 is an outer space defined by the outer case 10 to the outside of the storage chamber 22. [ When the camera is installed at the position of L1, the user uses the refrigerator while approaching the storage room 22 while the door is opened. However, there is a problem that the camera is hidden by the user.

As disclosed in the above-mentioned Japanese Patent Laid-Open Publication, when the camera is installed outside the refrigerator and the inside of the storage room is photographed, the image interference by the user is too large, especially the drawer is blocked and the movement of the drawer can not be tracked It is impossible to grasp the inner state of the drawer and the motion state of the drawer in the embodiment of the present invention. In addition, the L1 position may indicate that the camera is located on the door. In this case, since the door is moved, the camera can be moved at the time of photographing. Therefore, it is very difficult to obtain a stable and clear photograph.

L2 is the interior of the storage room 22 and means the upper wall of the storage room 22, that is, the ceiling. In the embodiment of the present invention, the position corresponds to the position where the camera is installed, but no problem is obscured by the user. Further, since the upper wall of the storage chamber 22 corresponds to the inner case 12, it is convenient to dispose the electric wire for supplying electricity to the camera. Since the inner case 12 is generally fixed at all times, it can be stably mounted in a configuration in which the camera is fixed.

L3 is the inside of the storage chamber 22 and the side wall of the storage chamber 22. [ At this time, the camera is installed so as to be able to take a photograph from one side of the storage room 22 toward the other side of the storage room 22. The camera must be installed on the side wall of the inner case 12 in order to be disposed at the middle of the storage room 22 based on the height of the storage room 22 rather than the upper end of the storage room 22.

When the camera is installed on one side wall, the camera lens must be installed to face the other side wall. Therefore, there is a problem that an asymmetrical image is obtained due to the difference in the distance between the left and right sides of the inside of the storage room.

L4 is the inside of the storage chamber 22 and means one end of the shelf 40. [ In order to position the camera relative to the height of the storage compartment 22, it must be supported by a specific structure, in which case the camera can be fixed to one end of the shelf 40.

In this case, since the distance between the drawer 50 and the camera is shorter than L2, there is a problem that a camera having a larger angle of view than the state of L2 must be used. In addition, there arises a problem that the degree of distortion of the obtained photograph is increased. In addition, various problems such as inconvenience to the user, such as installing a wire to be supplied to the camera in the shelf 40 may occur. And, the shelf 40 is generally movable. Therefore, when the food is put on the shelf, the camera itself may be shaken.

The camera applied to the embodiment of the present invention is capable of providing a performance of 20 fps at a VGA level, with an angle of view of 120 degrees.

9 is a cross-sectional view showing the main part of the refrigerator. This will be described below with reference to FIG.

A basket (21) capable of storing food can be provided inside the door (20). The basket (21) may be installed on a surface of the door (20) facing the storage room (22).

There may be a problem that the basket 21 is shown on a photograph taken by the camera 70. [ Therefore, in the embodiment of the present invention, it is possible to detect whether the door 20 has been rotated by a predetermined angle or more through the door sensor 120, and to take a picture through the camera 70. Of course, it is also possible to transmit the information about the viewpoint from the door sensor 120 to the controller 100 so that the photographer can select a photograph taken at a point adjacent to the point of time when the door sensor 120 sensed.

For example, the constant angle [theta] can be from 60 degrees to 80 degrees. Of course, the angle can be variously changed according to the capacity of the refrigerator, the size of the door, or the length of the front and rear of the basket 21.

The predetermined angle? May be an angle at which the basket 21 may not be displayed on the picture when the camera 70 photographs the picture. The predetermined angle? It can be varied depending on the width of the left and right and the length of the front and rear.

The door switch 110 for detecting whether the door 20 is open or closed may be installed above the refrigerator. At this time, when the door switch 110 is pressed by the door 20, the door 20 senses that the door 20 is closed, and if the door 20 is not pressed by the door 20, It can detect that it is open.

A left hinge portion 300 for rotatably mounting the left door 20 to the refrigerator and a right hinge portion 320 for rotatably mounting the right door 20 to the refrigerator. The left hinge unit 300 and the right hinge unit 320 may include a portion that is coupled to the door 20 and that is provided on the outer case 10, It can be divided into connecting parts connecting two parts. That is, the door coupling portions 300c and 302c, the outer case coupling portions 300a and 320a, and the coupling portions 300b and 320b connecting the door coupling portions 300c and 302c.

A left door sensor 120L for detecting the rotation of the left door 20 may be installed on the left hinge 300. [

The right hinge unit 320 may be provided with a right door sensor 120R for detecting the rotation of the right door 20.

The left door sensor 120L and the right door sensor 120R can independently detect a rotation angle of each door.

10 is a view for explaining the manner of operation of the door sensor. This will be described below with reference to Fig.

The door sensor 120 may include a light emitting unit 122 for emitting light and a light receiving unit 124 for receiving light emitted from the light emitting unit 122.

The light emitted from the light emitting unit 122 is reflected through the upper surface of the door 20 and can be transmitted to the light receiving unit 124. In this case, it is also possible to be made of a material with good reflection so that the reflection of light can be stably performed.

When the light emitted from the light emitting unit 122 is transmitted to the light receiving unit 124, the door 20 may be detected as being rotated by an angle smaller than an angle at which the door sensor 120 is installed. That is, the door 20 may be rotated at an angle smaller than a certain angle? Or may not be rotated in a state where the storage chamber 22 is closed.

On the other hand, if the light emitted from the light emitting unit 122 is not transmitted to the light receiving unit 124, the door 20 can be detected as being rotated by an angle larger than the angle at which the door sensor 120 is installed. That is, the door 20 may be rotated in a state where the door 20 is larger than a certain angle? To allow the user to access the storage room 22.

9, the left hinge portion 300 and the right hinge portion 320 may be mounted on the uppermost surface of the outer case 10 of the main body and the uppermost surface of the door 20, respectively. Accordingly, as the door 20 rotates with respect to the rotation centers 302 and 322, the uppermost surface of the door or a part of the upper surface of the door (e.g., a stepped portion) can perform the function of the light receiving portion. Of course, the left door sensor 120L and the right door sensor 120R may be installed in the respective connecting portions 300b and 320b to perform a light emitting function. Because of the positional relationship between the sensors 120L and 120R and the door, the sensors 120L and 120R can recognize the time when the door is opened at a predetermined angle? And the time when the door is closed at a certain angle? .

11 is a view showing the left hinge portion and the right hinge portion in detail. In Fig. 11, the other components of the refrigerator are omitted for convenience of explanation. This will be described below with reference to FIG.

The door sensor may be disposed on the front surface of the inner case, that is, in front of the opening (i.e., the position where the user stands) to sense the rotation of the door.

A left door sensor 120L for detecting the rotation of the left door 20 may be installed on the left hinge 300. [

The right hinge unit 320 may be provided with a right door sensor 120R for detecting the rotation of the right door 20.

The left door sensor or the right door sensor may be disposed in an area between the opening and the respective rotation centers of the respective hinge parts. That is, at the connection portions 300b and 320b of the respective hinge portions.

The door sensor 120 may be connected to other components by a connector c and a wire to receive electricity or to transmit signals to the outside. At this time, the door sensor 120 may be connected to the control unit 100 to provide the acquired signal information.

The left hinge unit 300 may include a rotation center 302 in which the left door 20 is rotated. At this time, the left door sensor 120L may be disposed on the right side with respect to the rotation center 302. [ The right space from the rotation center 302 is located at a position where the upper surface of the left door 20, particularly the left door 20, is positioned or not positioned below the left door sensor 120L in accordance with the rotation angle of the left door 20 It is a space that can be. Therefore, it is preferable that the left door sensor 120L is located at the connection part 300b of the left hinge part 300. [ If the size of the door or the size of the basket is changed through this, the predetermined angle? Can be flexibly changed. That is, the right space of the connection portion 300b can be easily changed, and the position of the left door sensor 120L in the connection portion 300b can be easily changed.

The wire (w) connected to the left door sensor 120L may be connected through the rotation center 302. Further, the wire (w) can be connected to the refrigerator body through the outer case (10).

The right hinge portion 320 may have a rotation center 322 at which the right door 20 is rotated. At this time, the right door sensor 120R may be disposed on the left side with respect to the rotation center 322. [ The left space from the rotation center 322 is a space where the right door 20 may not be positioned or positioned below the right door sensor 120R according to the rotation angle of the right door 20. [ Therefore, for the same reason, it is preferable that the right door sensor 120R is installed at the connection portion 320b of the right hinge portion 320. [

The time point sensed by the door sensor 120 may be a capture time point of each region in the storage room 22, specifically, the first region 42 or the third region 62 . That is, the point of time sensed by the door sensor 120 may indicate the capture time of a photograph of the external storage space of the drawer 50.

Meanwhile, since the door sensor 120 irradiates and receives light, the door sensor 120 irradiates light downward. The light irradiated downward is again reflected upward by the door 20.

The door sensor 120 and the door 20 can be kept at a short interval so that the door sensor 120 can receive a sufficient amount of light. At this time, the vertical distance between the door sensor 120 and the door 20 can be maintained at a maximum of 20 mm.

12 is an example of a screen provided to a user. This will be described below with reference to FIG.

12A shows a case in which the first area 42, the second area 52 and the third area 62 are all provided to the user and FIG. 12B shows that the second area 52 and the third area 62 are provided to the user, FIG. 12C shows a form in which the first area 42 and the second area 52 are provided to the user.

First, description will be made with reference to Fig. 12A.

One photograph taken by the camera 70 shows a first image showing the first region 42, a second image showing the second region 52, and a third image showing the third region 62 The third image can be divided into three. That is, a single photograph or a plurality of photographs can be classified into areas. Here, the first image, the second image, and the third image may each be a shape cut from a photographed photograph to a region showing the corresponding region.

Such separated photographs can be independently provided to the user as shown in FIG. 12A. That is, photographs for the respective regions are individually provided to the user, so that the user can easily acquire information on the foods stored in the corresponding regions.

The display 14 may be arranged such that a plurality of storage areas can be shown in a frame.

A second image showing the second area 52 is arranged on the left side and a right side on the upper side of one square frame respectively and a third image showing the third area 62 is arranged on the left side Respectively.

A plurality of regions may be arranged in the display 14 in an image having the same width in one frame. For this, the control unit may cut the photographed photograph, correct the size of the photographed image, and provide the corrected image to the display 14. At this time, the corrected image is shown at the corresponding position of the frame.

At least some of the images provided on the display may be calibrated to have the same width to have a planar arrangement effect by having the user have a feel similar to the viewing of the storage room with the actual refrigerator door open.

The display 14 can provide the food information in a state that the actual user can not see at a time by providing the two overlapping areas through one screen.

In particular, since the second image and the third image are arranged to overlap in a state in which the drawer is drawn out, the user can not simultaneously acquire the information of the two storage spaces in a state in which the drawer is drawn out. However, the display 14 may simultaneously provide two storage space information.

The control unit 100 can update (replace) each image individually and provide the user with information on the type of food accommodated in each area.

For example, if the portion corresponding to the second image is upgraded through a photograph taken by the camera, and the portion corresponding to the third image does not need to be upgraded, only the second image can be replaced.

On the contrary, the portion corresponding to the third image is upgraded through the photograph taken by the camera, and when the portion corresponding to the second image does not need to be upgraded, only the third image can be replaced.

It is possible that the second image and the third image are upgraded independently of each other in one frame regardless of other images.

It is also possible that only the second image arranged on the left side among the two second images arranged on the right and left sides is upgraded or only the second image arranged on the right side is upgraded. It is also possible to upgrade the third image separately on the left and right sides.

That is, the display 14 provides a frame in which a plurality of images are provided, so that the user can be provided with information on the articles stored in the storage room, and the plurality of images can be separately upgraded regardless of other images. In particular, since the second image and the third image are images selected from different photographs, the controller can determine which area the corresponding photo represents and upgrade each image.

Wherein the second image and the third image have a width that is the same or substantially similar in size. That is, the portions corresponding to the second image and the third image obtained in the photograph can be converted into the same width and provided to the display by the correction of the control unit.

If the size of the second image is different from the size of the third image, the number of pixels of the second image and the third image, that is, the size of the third image may be differently selected. Otherwise, it is possible to select the second image and the third image with the same number of pixels, and to correct the second image and the third image to have different sizes when they are displayed on the display 14.

It is possible to arrange the second image and the third image vertically so as to have a planar sense of vertical arrangement. The area originally indicated by the second image is arranged above the area shown by the third image. Thus, the user can intuitively recognize the image provided by the display, so that the information provided by the image can be easily grasped.

Of course, the display 14 may provide the user with a screen in which the first image and the third image are simultaneously replaced. Alternatively, the first image and the third image may remain the same, and the second image may be replaced with a new one.

The refrigerator compartment shelf shown in FIG. 12A means the first area 42, the vegetable compartment means the second area 52, and the multi-compartment corner can refer to the third area 62. Of course, it is also possible to have various arrangements and various forms according to the preference of the user or the convenience of the image provider.

On the other hand, the screen of FIG. 12A may be provided to the display 14 installed in the refrigerator, but may also be provided to an external device other than a refrigerator, for example, an external terminal device such as a smart phone. Therefore, the user can obtain information about the food stored in the refrigerator while going out from the house, and utilize it for shopping. Therefore, even if the user does not open the door, the user can grasp the type of food stored in the refrigerator. The screen provided to the user can be displayed by dividing one storage room into a plurality of storage areas. Therefore, the user can easily acquire the information classified into the positions of the foods stored in the respective positions in one storage room.

The image shown in the vegetable room shows an area from the front end of the drawer 50 to the front end of the shelf 40 when the drawer 50 is pulled out. An image shown in the multi-storage corner shows an area of a portion overlapping when the drawer 50 is pulled in a state that the door 20 is not covered by the door. The shelf shows the upper area of the shelf (40) to the user.

In an embodiment of the present invention, the screens provided to the user do not show one storage room as they are, but the separated images are arranged in the respective separated portions, so that the user can easily acquire the information. In other words, the photographs shown in FIG. 7 are not used as they are, but the individual regions are separated and used independently and individually in one photograph.

A screen of a form different from that of FIG. 12A may be provided as shown in FIG. 12B. In this case, it is possible for the camera 70 to take a picture including the second area 52 and the third area 62. [ Otherwise, even if the camera 70 captures a photograph including both the first area 42, the second area 52 and the third area 62, the user can not recognize the image corresponding to the two areas Can be provided.

Another type of screen may be provided as shown in FIG. 12C. In this case, it is possible for the camera 70 to take a picture including the first area 42 and the second area 52. Otherwise, even if the camera 70 captures a photograph including both the first area 42, the second area 52 and the third area 62, the user can not recognize the image corresponding to the two areas Can be provided.

That is, according to an embodiment of the present invention, it is also possible to provide the related image to the user after shooting all three areas. However, as shown in FIGS. 12B and 12C, it is possible to provide at least the image corresponding to the internal storage area of the drawer, that is, the image corresponding to the second area and the area other than the second area to the user.

13 is a view for explaining a method of adjusting a picture photographed by a camera. This will be described below with reference to FIG.

FIG. 13 is an example of a screen displayed on the display 14. FIG. Of course, if the refrigerator is not provided with a display, FIG. 13 may refer to a screen displayed on a separate display device of the operator when the refrigerator is manufactured. And may be a screen displayed on the user's mobile phone.

When the camera 70 is installed, the camera 70 may not be installed at a desired position due to assembly tolerances.

Therefore, in the embodiment of the present invention, it is possible to change the adjustment line 15 for dividing each region through a photograph or an image taken by the camera 70 by the operator or the user.

The adjustment line 15 may include a vertical adjustment line and a horizontal adjustment line. Vertical adjustment lines can be placed at the center or at both ends of the image. The user can move the vertical adjustment line up and down through the buttons shown on the screen and move the horizontal adjustment line to the left and right to select a portion where the image provided on the display 14 is cut.

On the other hand, a limit line 16 which defines the movable range of the adjustment line 15 can also be shown. At this time, the limit line 16 may be arranged to have a predetermined distance left and right around the vertical adjustment line. In addition, the limit line 16 can be arranged to have a predetermined distance up and down about the horizontal adjustment line.

If the image of the desired degree is not obtained despite the movement of the adjustment line 15 with the limit line 16, it is judged to be defective and the installation position of the camera 70 can be adjusted.

As shown in FIG. 13, in the case where two drawers are provided in the storage chamber, the vertical adjustment line can be disposed between the two drawers.

And the horizontal adjustment line can be disposed at the front end of the shelf.

In a state in which the movement of the horizontal adjustment line and the vertical adjustment line is completed, the lower left end of the intersection of the horizontal adjustment line and the vertical adjustment line may constitute an image of the second region or the left region of the third region. In this case, the image may be from the horizontal adjustment line to a specific number of pixels in the horizontal direction and from the vertical adjustment line to a specific number of pixels in the vertical direction.

The right lower end of the intersection of the horizontal adjustment line and the vertical adjustment line may constitute an image for the second region or the right region of the third region. In this case, the image may be from the horizontal adjustment line to a specific number of pixels in the horizontal direction and from the vertical adjustment line to a specific number of pixels in the vertical direction.

The upper side of the horizontal adjustment line may constitute the image of the first region. The image may be from the horizontal adjustment line to a specific number of pixels in the vertical direction.

On the other hand, as shown in FIG. 13, the left and right photographs of the third area corresponding to the vegetable compartment are separately shown, and it is also possible to confirm whether or not the photograph corresponding to the draw is acquired.

When the adjustment by the user or the operator is completed, the user can input that the movement of the adjustment line 15 is completed by clicking the input completion button. Therefore, when the installation error of the camera is small, it is possible to adjust the position of the vertical line and / or the horizontal line separating each region in the photographed photograph. As a result, parts corresponding to specific areas can be clearly displayed. Of course, in the case where there are a plurality of areas divided into upper and lower and left and right, a plurality of the above-mentioned horizontal adjustment lines or vertical adjustment lines may be provided.

Fig. 14 is a perspective view showing the camera, Fig. 15 is a view showing the main part of the camera, and Fig. 16 is a sectional view of Fig. This will be described below with reference to Figs. 14 to 16. Fig.

The camera 70 may include a camera module 71 including a camera lens 71a and a body 71b and a case 73 or 74 in which the camera module 71 is accommodated.

The camera module 71 may include an image sensor for converting image information passed through the camera lens 71a into a digital signal.

The case may include a first case 73 and a second case 74 coupled to the first case 73 to form a predetermined space therein. The camera body 71b and the camera lens 71a may be accommodated in the first case 73 and the second case 74. The coupling between the first case (73) and the second case (74) can be maintained by a separate sealing member so that moisture or dust does not flow into the interior.

Since the camera body 71b and the camera lens 71a are inserted into the space formed by the first case 73 and the second case 74, the inner case 12 and the outer case 10 The blowing agent does not come into contact with the blowing agent filled in between. Also, the camera module 71 can be protected without directly contacting cool air stored in the storage chamber 22. [

The first case 73 may include a transparent window 80 provided at a front end of the camera lens 71a and a heater 84 for providing heat to the transparent window 80. [ The transparent window 80 is preferably made of a transparent material.

It is possible to prevent the transparent window 80 from being dewed by the heat provided by the heater 84.

If the heater 84 is installed on the camera lens 71a to prevent dew formation, the camera lens 71a may be deformed. The possibility of occurrence of aberration and permanent damage due to such deformation is increased. Therefore, in the embodiment of the present invention, the transparent window 80 is not directly brought into contact with the camera range 71a.

As the material of the transparent window 80, both plastic and glass can be used for camera image acquisition, but plastic has a low thermal conductivity. Accordingly, when plastic is used for the transparent window 80, power consumption can be increased as compared with the case of using glass. Therefore, it is preferable that the transparent window 80 is made of a glass material to reduce power consumption.

The heat supplied by the heater 84 can be easily conducted to the portion of the transparent window 80 where the heater 84 does not directly contact. For example, when the thermal conductivity is low, the temperature of the portion adjacent to the heater 84 is high while the temperature of the portion away from the heater 84 is low, so that it is difficult to prevent condensation. Further, since more heat energy must be supplied by the heater 84, the energy efficiency is lowered.

On the other hand, since the heater 84 directly contacts the transparent window 80, heat transfer efficiency can be improved.

The heater 84 is disposed on the opposite side of the surface on which the transparent window 80 is exposed to the storage chamber 22 so that the heat generated by the heater 84 does not pass through the transparent window 80, (22).

In particular, the heater 84 may include a hot-wire coil for generating heat, and the hot-wire coil may be in surface contact with the transparent window 80.

It is preferable that the transparent window 80 and the camera lens 71a are spaced apart from each other by a predetermined gap g. This is for the purpose of considering deformation or assembly tolerance that can be generated by supplying heat to the transparent window 80. [

In addition, the camera 70 may include a cable 79 that receives electricity supplied from outside and transmits a signal relating to the obtained photograph. The cable 79 may electrically connect the camera module 71 to an external component.

Meanwhile, the case is provided with a first seat piece 75 capable of fixing the camera 70 to other components. At this time, the first seat piece 75 is not provided regardless of which of the first case 73 and the second case 74 is provided.

The first seat piece 75 may include a surface having a predetermined area, and the camera 70 may be disposed at a desired installation position with a desired angle of inclination.

The first seat piece 75 is provided on both sides of the camera 70, and the camera 70 can be fixed to the left and right sides by external components.

The second seat 74 may be formed with a second seating piece 76 having a predetermined area and formed to be stepped. The second seat piece 76 is provided at a different position from the first seat piece 75 so that the camera 70 can be moved to a specific position at a predetermined angle So that they can be stably bonded. Accordingly, the camera 70 can be fixed to the other components through the first seat 75 and the second seat 76 with three or more fulcrums at a predetermined angle. Details of the supporting structure of the camera 70 will be described later.

The resolution of the camera 70 can be VGA resolution, and the resolution can be VGA (640x480) and 30-megapixel resolution in consideration of image data transmission speed and processing time.

 The interface of the camera 70 is a USB type and can be driven using a 5V, 500mA USB power source.

 Also, the current value supplied to the camera 70 may be 87 mA in the standby mode and 187 mA in the operating mode (the heater is 67 mA). Therefore, the power consumption proportional to the supplied current value can be consumed. At this time, the operation of the camera 70 is divided into a standby mode and an operation mode. In order to prevent condensation on the surface of the transparent window 80, the heater 84 can be always powered.

Meanwhile, when the rated voltage is applied, the camera 70 operates in a standby mode. If the door switch 110 detects that the door 20 is opened, the camera 70 switches to the operation mode and photographs the picture. That is, when power is supplied to the refrigerator, the camera operates in a standby mode. When the door switch 110 detects that the door 20 is opened, the camera can be switched to an operation mode for photographing.

17 is a view for explaining the arrangement of the heaters. This will be described below with reference to FIG.

The photographing angle of the camera 70 may be 115 ° for a horizontal angle of view, 95 ° for a vertical angle of view, and 145 ° for a diagonal angle of view. That is, the camera 70 can be arranged such that the horizontal angle of view is larger than the vertical angle of view. 17) of the storage compartment 22 is longer than the length of the storage compartment 22 in the anteroposterior direction (the vertical direction in FIG. 17), food information on the storage compartment 22 Can be obtained.

Meanwhile, the heater 84 may be formed along the outline of the transparent window 80. At this time, the heater 84 may be arranged to form a square so as not to block the angle of view of the camera 70.

Further, when the heater 84 has a rectangular shape, heat can be supplied to the transparent window 80 with a relatively large area, so that the energy efficiency of the heater 84 can be improved.

In particular, when a clearance g is provided between the camera lens 71a and the transparent window 80 as shown in FIG. 16, the angle of view with respect to the transparent window 80 becomes a rectangular shape as shown in FIG. Since the heater 84 is in contact with one surface of the transparent window 80 if there is no clearance g and the transparent window 80 and the camera lens 71a are in contact with each other, Or a square having a length equal to or greater than the diameter of the square.

However, as described above, since the gap g exists and the vertical angle of view and the horizontal angle of view are different from each other, it is preferable that the heater 84 has a rectangular shape longer than the length of the other side.

FIG. 18 is a view for explaining a photograph taken in a state in which the camera is mounted on the refrigerator in a state where the camera is turned in a vertical direction (forward and backward direction), FIG. 19 is a view for explaining a photograph taken in a state where the camera is horizontally FIG. This will be described below with reference to FIGS. 18 and 19. FIG.

FIG. 18A shows a state in which the camera is upwardly moved upward by 5 degrees, FIG. 18B is set in a desired position, and FIG. The degree of photograph shown in FIG. 18 can provide the user with information about the food stored in the storage room, but can not provide the user with the desired information if it is turned at a larger angle.

FIG. 19A shows a state in which the camera is rotated by 4 degrees to the left, FIG. 19B shows a state in which the camera is rotated two degrees to the left, and FIG. 19C shows a state in which the camera is installed at a desired position.

As shown in FIG. 19, when a camera is installed in a vertical direction as shown in FIG. 18, a photograph that is not installed at a desired angle in a horizontal direction as shown in FIG. 19 provides a relatively largely distorted picture.

As can be seen from FIGS. 18 and 19, even if the installation angle of the camera is small, distortion of the photographed image can be greatly increased. Therefore, it is necessary to manage the installation angle of the camera so as to minimize the assembly tolerance. Of course, the fine error can be adjusted through the adjustment line described with reference to FIG. 13, so that it is possible to provide an optimum photograph.

FIG. 20 is a front view of the first housing, FIG. 22 is a front view of the second housing, and FIG. 23 is a front view showing a state in which the camera is installed in the first housing. Fig. 24 is a side cross-sectional view of Fig. 23, and Fig. 25 is a cross-sectional view of the camera housing in an inner case. This will be described below with reference to FIGS. 20 to 25. FIG.

And a camera housing for installing the camera 70 in the inner case 12. [ That is, a camera housing for housing the camera shown in FIG. 14 and fixing the camera to the ceiling of the storage room may be provided.

The camera housing includes a first housing 400 coupled to a lower surface of a ceiling of the inner case 12, a second housing 410 coupled to an upper surface of the ceiling of the inner case 12, The third housing 420 and the fourth housing 430, which are coupled to the first housing 420 and the second housing 430, respectively. Here, the first housing 400 may be coupled to the top surface of the ceiling of the storage room.

The third housing 420 can close the front of the camera 70 such that the other part of the camera 70 except for the transparent window 80 is not directly exposed to the storage room 22. That is, the third housing 420 may function as a cover covering the front surface of the camera 70. At this time, the third housing 420 and the fourth housing 430 may be constituted by one component. Of course, the fourth housing 430 may be a deco housing interposed between the third housing 320 and the first housing 400.

The camera 70 is assembled in the first housing 400 to control the degree of tilting of the camera 70 in the horizontal direction and the degree of tilting of the camera 70 in the vertical direction .

As shown in FIG. 21, the first housing 400 may include a first seat 402 on which the first seat 75 of the camera 70 is seated. At this time, the first seating part 402 may be provided with two pieces, so that the first and second seating parts 75 may be respectively fastened.

The first seat piece 75 may be disposed at right angles to the left and right sides of the camera 70 so that the camera 70 may be inclined at a desired angle in a state where the camera 70 is installed in the inner case 12 .

The first seating part 402 is provided with a fastening hole 404 so that the first seating part 75 and the first seating part 402 can be fixed by screws. The first seating portions 402 are disposed at different heights in the horizontal direction, so that the camera 70 can be stably fixed.

The first housing 400 may include a second seating portion 408 on which the second seating piece 76 of the camera 70 can be seated. At this time, the second seating portion 408 is formed to have a predetermined area so that the degree of tilting of the camera 70 can be fixed while making surface contact with the second seating piece 76.

That is, the camera 70 may be combined with the first housing 400 while contacting the first housing 400 by a plurality of surfaces, thereby fixing the camera 70 to a desired angle. In other words, the first housing 400 can be fixed to have three or more supporting points and can be supported at a predetermined angle with respect to the first housing 400. Of course, the horizontal angle may be fixed to be parallel to the first housing 400, and the vertical angle may be fixed to a predetermined angle.

The first housing 400 may include a fastening hole 406 to be coupled to the second housing 410 by a screw.

As shown in FIG. 22, the second housing 410 has a contact surface 416 that can be in surface contact with the upper surface of the ceiling of the inner case 12. At this time, the contact surfaces 416 are generally rectangular and widely distributed, so that the second housing 410 can be stably mounted on the top surface of the ceiling of the inner case 12. Accordingly, 410 have a surface parallel to the ceiling of the inner case 12 of the contact surface 416. So that the reference line or reference plane for the installation angle of the camera can be the same as the ceiling surface of the inner case 12.

The second housing 410 may have a receiving space 412 in which the cable 79 of the camera 70 can be received. A through hole 418 is formed at one side of the accommodation space 412 so that a wire connected from other components of the refrigerator can be exposed to the accommodation space 412. At this time, the cable 79 may be connected to an electric wire connected from other components of the refrigerator in the accommodation space 412. Accordingly, electricity can be supplied to the camera 70 through the cable 79, and signals for photographs taken by the camera 70 can be transmitted to other components.

The second housing 410 may have a coupling part 414 coupled to the first housing 400. The fastening part 414 can be coupled to the fastening hole 406 by a screw. It is preferable that the coupling portion 414 does not cause an error due to assembly when the coupling portion 414 is engaged with the coupling hole 406. Therefore, it may be formed to protrude by a predetermined height so as to be in surface contact with the fastening hole 406.

23 and 24, when the camera 70 is assembled to the first housing 400, the camera 70 may be disposed at an inclined angle by a predetermined angle. As shown in FIG. 24, one side 400a of the first housing 400 is shaped to be in contact with the inner case 12. As shown in FIG. The camera 70 is inclined at a predetermined angle in comparison with the one side face 400a of the first housing 400. [ The one side surface 400a is coupled to the ceiling bottom surface of the inter case 12 so as to be parallel to the contact surface 416 of the second housing 410. [ Accordingly, when the camera 70 is coupled to the first housing 400 and the first housing 400 is coupled to the second housing 420, the installation angle of the camera 70 with respect to the ceiling of the storage room Can be significantly reduced.

25, the second housing 410 is disposed in a space between the inner case 12 and the outer case 10 so as to be exposed to a space where the foamed liquid is filled in the space between the inner case 12 and the outer case 10 do. The second housing 410 is disposed outside the storage chamber 22, but is disposed in a space defined by the outer case 10. The second housing 410 is fixed to the inner case 12 and is finally fixed by the foamed liquid filled between the inner case 12 and the outer case 10 .

The first housing 400 may be coupled to the second housing 410 with the inner case 12 interposed therebetween. Since the first housing 400 is coupled to the second housing 410 in a state where the second housing 410 is fixed at a specific position of the inner case 12, Can be fixed.

The third housing 420 and the fourth housing 430 may be coupled to the second housing 410 while the camera 70 is inclined at a predetermined angle to the first housing 400 . As described above, the third housing 420 and the fourth housing 430 are constituted by one component, and the first housing 400 and the second housing 420 function as the third housing 420, May be combined.

The third housing 420 can expose only the transparent window 80 of the camera 70 to the storage compartment 22 and prevent the other parts from being exposed. Therefore, it is possible to prevent the camera 70 from being affected by moisture, dust, cold air or the like existing in the storage chamber 22. [

The structure of the housing of the camera, the installation direction of the camera housing with respect to the inner case, the installation form, and the like also have a technical meaning to reduce condensation generated in the transparent window 80. A detailed description will be given later.

FIG. 26 is a table comparing the consumed power of the camera in the present invention, and FIG. 27 is a diagram comparing the supplied standby current with the driving current. FIG. 26A shows a state in which current is supplied to the camera in the comparative example, and FIG. 27B shows a state in which current is supplied to the camera in the present invention. This will be described below with reference to Figs. 26 and 27. Fig.

In the comparative example, a standby current of 70 mA is supplied to the camera 70, and a current of 50 mA is further supplied to the camera 70 when the door is opened, so that a driving current of 120 mA is supplied to the camera 70. In the comparative example, when the camera 70 is opened 25 times a day, 253 Wh is consumed per month, and when it is opened 50 times a day, 254 Wh is consumed per month.

On the other hand, in the embodiment of the present invention, a standby current of 20 mA is supplied to the camera 70, and 100 mA is additionally supplied at the time when the door is opened, so that a driving current of 120 mA can be supplied to the camera 70. When the camera 70 is opened 25 times a day, 74 Wh is consumed per month, and when it is opened 50 times a day, 76 Wh is consumed per month.

Comparing two experimental results, it can be seen that the power consumed by the camera 70 can be reduced by about 70% by lowering the quiescent current. Therefore, it has been confirmed that the power consumed by the camera 70 can be greatly reduced by lowering the standby current on the assumption that the driving current is the same. This is because the door opening time is relatively short when the refrigerator is operated for 24 hours continuously. Therefore, when the current value required for photographing in the operation mode is constant, the current consumption can be reduced as the current value applied to the camera is lowered in the standby mode.

When the standby current is lowered, the timing at which the driving current is supplied to the camera 70 is delayed as shown in FIG. This is because the delay time must be increased due to the difference between the standby current value and the driving current value. In other words, when the mode is switched from the standby mode to the drive mode, the photographing is not performed immediately, and the predetermined delay time elapses, so that the photograph can not be taken.

Therefore, the time point at which the camera 70 actually takes a photograph is delayed. If the difference between the time when the instruction to take a picture to the camera 70 is received and the time when the picture is actually taken by the camera 70 is actually large, the picture actually taken by the camera 70 actually gives the user food information May not be provided.

In other words, as the standby current is lower, the power consumed by the camera 70 is reduced. However, the delay time may be long in raising the driving current for driving the camera 70. Therefore, it is desirable to calculate the normal time that elapses from when the user can take a picture at a desired point in time, and to select the standby current value so that the delay time is shorter than the corresponding time. For this reason, according to the embodiment of the present invention, it is preferable to raise the current applied stepwise to the drive current value at the standby current value. That is, it is possible to gradually increase the current value applied to the relatively high driving current value at a relatively low standby current value with a sufficient delay time. That is, it is possible to prevent the current value from rising sharply. Therefore, the power consumption can be reduced and the stability of the camera can be improved.

Therefore, in the embodiment of the present invention, as shown in FIGS. 28 and 29, the camera 70 is controlled to start driving at the time when the door 20 is opened. That is, when it is determined that the door is opened through the door switch, the mode is switched from the standby mode to the driving mode. However, the effective photograph of the photograph taken by the camera is after at least a predetermined time has elapsed at the time when the door is opened. This is because the door will open and the user will have to physically take more than a few seconds to update the food, even if the delay time is long enough to get the picture at the desired point. Therefore, in the present embodiment, it can be said that the point of time when the camera is switched from the standby mode to the driving mode is performed by the detection result of the door switch. The reference point for selecting a valid photograph may be said to be performed by the detection result of the door sensor.

FIG. 28 is a view for explaining the start time of photographing of the camera and the continuous photographing of the camera. This will be described below with reference to FIG.

In the present invention, as shown in FIG. 4, the door switch may be included. When the opening of the storage compartment 22 is sensed by the door switch 110, the control unit 100 may instruct the camera 70 to start taking a picture. That is, when the door 20 is moved or rotated to open the storage room 22, the camera 70 can start photographing.

On the other hand, when the door switch 110 detects the airtightness of the storage compartment 22, the control unit 100 may instruct the camera 70 to stop taking pictures. That is, when the door 20 is moved or rotated to close the storage compartment 22, the camera 70 can end photographing.

As shown in FIG. 28, when electricity is supplied to the refrigerator by an external power source, a standby current is supplied from the camera 70 (S1). At this time, the camera 70 may be in a standby mode. In the standby mode, the camera 70 does not take a picture.

The opening of the door 20 may be sensed by the door switch 110 (S10).

If it is determined that the door 20 opens the storage compartment 22, the camera 70 may start photographing (S14). In particular, the camera 70 can take a picture continuously at predetermined time intervals.

At this time, since the camera 70 is supplied with a current larger than the current supplied in the standby mode, the camera 70 can be operated in the driving mode. In order to supply a current larger than the standby current for driving the camera 70, the driving current must be supplied for a predetermined time, i.e., a delay time. That is, in order to increase the current supplied to the camera 70, a certain time is actually required. After a delay time of a predetermined time has elapsed after a current for performing a driving mode is supplied to the camera 70, a driving current is supplied to the camera 70, so that the camera 70 can photograph the picture .

Since the camera 70 is switched from the opening time of the door 20 to the driving mode for photographing, the starting time of photographing is fast, and the risk of not being able to take a picture at a specific time is reduced. If the current value is increased at a desired point while supplying the standby current until the desired time, the photographing time of the photograph is delayed due to the delay time.

In addition, the number of pictures taken in one second by the camera 70 can be reduced. Assuming that the camera 70 takes 30 pictures in one second, it is very unlikely that a change in the storage room 22 occurs within a time interval of photographing, that is, 1/30 second. The inside of the storage room 22 can be changed by the user's operation because a general person is less likely to end the operation for 1/30 second with respect to the storage room 22. That is, the time interval taken by the camera 70 can be relatively increased. Therefore, the camera 70 can be used at a relatively low cost, so that the cost of the refrigerator can be reduced.

The photograph taken by the camera 70 may be stored in the storage unit 18 (S16). Also, if the number of pictures taken in one second by the camera 70 is reduced, the number of pictures stored in the storage unit 18 is reduced, and the storage unit 18 can be operated more efficiently.

Meanwhile, the door sensor 120 may detect whether the door 20 is rotated at a specific angle (S18).

If it is not detected that the door 20 is rotated by the door sensor 120, the user may delete the picture of the picture stored in the storage unit 18 before the predetermined time (S90). That is, it is possible to delete the unprocessed photograph to provide to the user, thereby preventing the capacity of the storage unit 18 from becoming large.

On the other hand, if the door sensor 120 determines that the door 20 has been rotated at a certain angle, a photograph taken at that point of time is selected (S92). At this time, the selected photograph is processed as an effective photograph by the control unit 100 and can be provided to the user. For example, a photograph processed by the control unit 100 may be transmitted to the display 14.

It takes a certain period of time to perform the operation of rotating the door 20 in order for the user to access the storage room 22. Since the delay time of the camera 70 has elapsed during the time, the picture can be taken at a time when the door 20 is rotated at a specific angle. Accordingly, the controller 100 can select a photograph at a desired point in time.

If the camera 70 is switched from the stand-by mode to the drive mode so as to take a picture at the time the door sensor 120 senses it, there is a problem that the storage room state after the delay time has elapsed .

On the other hand, since the camera 70 starts shooting at the time when the door 20 is opened, a standby current of a relatively small size can be supplied to the camera 70. If the standby current is supplied in a small amount, the delay time till the driving current is supplied. However, in the control flow described above, since the photographing is started early in the camera 70, .

FIG. 29 is a view for explaining the start time of photographing of the camera and the continuous photographing of the camera. This will be described below with reference to FIG.

29 is different from FIG. 28 in that the picture selection time is not determined according to the rotation state of the door 20, but the picture selection time is determined according to the state of the drawer 50. The other parts are the same, and the same parts are omitted for convenience of explanation. However, it can be said that the operation of the drawer 50 is premised on the opening of the door 20. [ The reference point of time for selecting a specific photograph may be determined by the judgment of the door sensor, and the reference point of time for selecting the photograph for the specific area in the specific photograph may be determined by the judgment of the door sensor and the judgment of the door sensing unit 130 It will be possible. Therefore, in any case, the determination result of the door sensor is required to determine the reference time point of the effective photograph.

In order to store food in the drawer 50 or take out the food stored in the drawer 50 to the outside, the user can take out the drawer 50. At this time, the user can take out the drawer 50 for a sufficient length to use the drawer 50.

After the user has finished using the drawer 50, the drawer 50 can be pushed in. The time when this state occurs may mean the specific state described in S19.

Since the camera 70 starts to take a picture at the time when the door 20 is opened in the door switch 110, if the drawer 50 is placed in a specific state, And can be provided to the user (S92).

When the door 20 is opened, a photograph is taken by the camera 70, and movement of the drawer 50 can be monitored from the time when the photograph is taken, that is, when the door 20 is opened.

FIG. 30 is a diagram illustrating an embodiment of the draw detection unit, and FIG. 31 is a view for explaining a method of sensing the movement of the draw in the draw detection unit of FIG. This will be described below with reference to Figs. 30 and 31. Fig.

Referring to FIG. 30, the drawer sensing unit 130 may be implemented as a separate component instead of being provided in the control unit 100.

The drop sensing unit 130 may include one light emitting unit and two light receiving units. That is, the time when the light emitted from one light emitting unit is incident on the two light receiving units is analyzed, and the moving direction of the drawer 50 can be sensed.

A plurality of slits 134 may be continuously arranged in the drawer 50 to prevent the light emitted from the light emitting unit of the draw sensing unit 130 from being reflected when the slit 134 is incident on the slit 134. Therefore, light can be sequentially incident on the two light receiving portions.

31, the x-axis denotes time, and the y-axis denotes a signal when light is received by two light-receiving units (Photo TR1 and Photo TR2). 31A, light is received before Photo TR2 in Photo TR1, and light is received before Photo TR2 in Photo TR2 in FIG. 31B.

31B shows a moving state in which the user draws out to use the drawer 50. In FIG. 31A, when the user finishes using the drawer 50 and draws the drawer 50 to the original position State. ≪ / RTI >

31A, the controller 100 determines that it is time to acquire a picture photographed by the camera 70. When the drawer 50 receives a signal from the camera 70, That is, a photograph corresponding to the third area. That is, the acquisition time point of the photograph corresponding to the third area can be determined according to the detection or determination result of the drop sensing unit 130. [ Of course, in the obtained photograph, the portion corresponding to the third region can be selected, separated and updated independently and separately from the portion corresponding to the other region.

32 is a diagram showing a state in which markers are displayed on a drawer. This will be described below with reference to FIG.

30 and 31, movement of the drawer 50 can be detected by using a marker, which is a specific indication for the drawer 50, without using a separate machine or an electronic device .

The drawer 50 may be provided with various types of markers (patterns having black and white colors). The camera 70 analyzes the movement of the marker through the photographed photograph, and can read whether the drawer 50 is moved or moved. At this time, analysis of the marker can be performed in the drop sensing unit 130 of the control unit 100. That is, the drop sensing unit 130 can determine the state of the drawer 50 by analyzing the movement of the marker in the photographs continuously shot. Accordingly, the acquisition time point of the photograph corresponding to the third area can be determined according to the detection or determination result of the dwarf detection unit 130. [ Of course, in the obtained photograph, the portion corresponding to the third region can be selected, separated and updated independently and separately from the portion corresponding to the other region.

In Fig. 32, the marker includes an image in which the shape of the camera is simplified and a character called " smart sensor ". On the other hand, the marker may include various patterns, and may be provided so that black and white intersect with each other, so that the marker 70 can be recognized by the camera 70.

At this time, the marker can be displayed on the front side of the drawer 50 so that the marker 70 can be recognized by the camera 70.

Fig. 33 (a) is a photograph showing a state in which left and right drawers are drawn out, and Fig. 33 (b) is a drawing showing a state in which the left drawer is drawn and the right drawer is drawn. 33 (a) and 33 (b).

In Fig. 33, the marker is provided at the center of the upper side of the drawer 50, and shows an example of a wavy pattern.

The first area 42 may be represented in a photograph taken by the camera 70. [ In addition, one of the second region 52 and the third region 62 may be shown in one photograph.

That is, as shown in FIG. 33 (a), photographs taken in the state in which the left and right drawers are drawn out show the first region 42 disposed on the left and right sides and the second region 52 disposed on the left and right sides.

On the other hand, as shown in FIG. 33 (b), the photograph taken in the state in which only the right drawer is drawn includes the first region 42, the left third region 62 and the right second region 52 Can be shown.

That is, when the drawer 50 is drawn out, the third region 62 is not shown in the photograph but the second region is shown. On the other hand, when the drawer 50 is drawn in, the second region 52 is not shown in the photograph and the third region 62 is shown.

The camera 70 takes a picture at the same position. The drawer 50 can be moved so that the picture acquired by the camera 70 according to the position of the drawer 50 can take pictures of various areas.

33 (b), if the drawer 50 provided on the left side is completely drawn into the lower space of the shelf 40, the marker can not be recognized by the camera 70. [ That is, no markers are shown in the photographs taken by the camera 70.

A method of detecting the movement of the drawer using the marker will be described with reference to FIGS. 34 to 40. FIG.

34 is a flowchart illustrating a method of operating a refrigerator according to an embodiment of the present invention.

34, if the opening of the door 20 is detected by the door switch 110, the camera 70 starts photographing (S2101), and the controller 100 performs a marker recognition operation and a marker tracking And starts operation (S2103).

Hereinafter, the steps S2101 and S2103 will be described in more detail with reference to FIG.

35 is a flowchart for explaining a marker recognition and tracking method according to an embodiment of the present invention.

Referring to FIG. 35, when the door 20 is opened, the camera 70 starts photographing (S2301).

The controller 100 controls the camera 70 to acquire an internal photograph of the drawer 50 (S2303).

Next, the control unit 100 processes the obtained photograph (S2305), and determines whether a predetermined marker is recognized in the obtained photograph (S2307).

The control unit 100 may compare whether the marker recognized from the obtained photograph is the same as the marker stored in advance in order to determine whether or not the predetermined marker is recognized.

The predetermined marker may be determined by the operator or the user, and may be stored in advance in the storage unit 18 or the like.

The marker may be formed in a searchable area in a photograph taken from the camera. Hereinafter, with reference to Figs. 36 and 37, the marker that can be recognized and tracked by the control unit 100 will be described in detail.

FIG. 36 is a view for explaining a marker for detecting a draw-in point of a drawer according to an embodiment of the present invention. FIG.

Referring to FIG. 36 (a), a marker may be formed in an area that can be searched by the camera 70 from the time when the drawer 50 starts to be opened. For example, if the camera 70 is located at the top of the refrigerator and the plane of the drawer 50 can be photographed, the marker may be located at the front end of the upper end of the drawer 50. In addition, the field of view of the camera 70 can be set so that the drawer 50 can be photographed from a completely closed state to a fully open state of the drawer 50.

When the marker is positioned at the upper end of the drawer 50, when there is no obstruction at the upper end of the drawer 50 at the time when the drawer 50 is closed after the storage of the stored contents in the drawer 50 is completed And is not limited thereto.

36 (b), when the camera 70 looks at the drawer 54 and the drawer 50 in which the drawer 50 is stored, the camera 70 draws the drawer The marker 50 can be photographed in the entire range from the state in which the marker 50 starts to be opened to the state in which the drawer 50 is completely opened.

37 is a view for explaining a shape of a marker according to an embodiment of the present invention.

Referring to FIG. 37, the marker may be formed in a shape in which the same shapes of two colors are crossed and repeated.

Referring to FIG. 37 (a), the marker may be formed by arranging a plurality of alternating strips of white square and black square alternately.

Referring to FIG. 37 (b), the markers may be formed in a form in which a plurality of strips alternately repeating a white oblique rectangle and a black oblique rectangle are alternately arranged.

As described above, the marker of the repeated type is advantageous in that the marker can be recognized and tracked by the control unit 100 even if a part of the marker is hidden from the camera shooting range. However, the marker is not limited to a repeated shape, and is not limited to a plurality of colors. As described above, the marker suffices to be able to recognize the movement of the interior of the refrigerator, such as the drawer 50, having mobility.

Referring again to FIG. 35, when a predetermined marker is recognized, the controller 100 detects movement of the recognized marker (S2311). The control unit 100 may analyze the photographs obtained continuously or periodically to determine whether the recognized marker moves or stops. For example, the control unit 100 can determine whether the marker 50 is moved or stopped when the opening operation of the drawer 50 is continued or completed, or when the closing operation of the drawer 50 is started, continued or completed .

Alternatively, if the predetermined marker is not recognized, the control unit 100 notifies the user that the marker recognition has failed (S2309).

For example, if an obstacle is located in the marker, the camera may not acquire the photograph including the marker, or the controller 100 may not be able to recognize the marker from the acquired photograph. In this case, the control unit 100 can inform the user through the display 14 or the like of information indicating that the marker recognition has failed and information indicating how to succeed in recognizing the marker, such as removing obstacles.

If the marker recognition is unsuccessful, the control unit 100 may repeat the step of recognizing the predetermined marker by processing the newly acquired photograph through the camera.

Referring again to FIG. 34, when a marker closing operation is detected (S2105), the control unit 100 determines whether the inside of the drawer (the third area 62) And obtains the final photograph (S2107). At this time, the final picture inside the drawer acquired by the control unit 100 may be the inside of the drawer immediately before the drawer closing operation is detected, or the inside of the drawer immediately after the drawer closing operation is detected, But is not limited thereto.

The detection of the drop closing operation will be described with reference to FIG.

FIG. 38 is a flowchart for explaining a method of detecting a draw-close operation according to an embodiment of the present invention.

Referring to FIG. 38, the controller 100 determines whether the end of the opening operation of the drawer 50 is detected (S2501). For example, when the controller 100 recognizes the marker from the photograph of the previously acquired drawer 50, tracks the movement of the marker, and detects that the movement of the marker is stopped, the opening operation of the drawer 50 It can detect that it has been terminated.

Next, the control unit 100 determines whether or not the start of the closing operation of the drawer 50 is sensed (S2503). For example, when the marker stops moving and starts moving again in the opposite direction, the control unit 100 can sense that the closing operation of the drawer 50 is started.

The operation of the controller 100 that senses the door closing operation according to another embodiment of the present invention will be described with reference to FIG.

FIG. 39 is a flowchart for explaining the operation of the refrigerator performed when the completion of the door closing operation according to the embodiment of the present invention is detected.

If the end of the door closing operation is detected (S2701), the control unit 100 determines whether or not the final image acquisition through the marker recognition is successful (S2703).

The control unit 100 may sense that the closing operation of the drawer 50 is completed through the image processing of the draw sensing unit 130. [

The control unit 100 notifies the user that the marker is not recognized normally due to an obstruction or the like, and fails to recognize the marker if the final image acquisition fails (S2705).

The control unit 100 can inform the user that the marker has failed to be recognized through the display 14 or the like. This can guide the user to remove the obstruction and induce the drawer to accept normal markers and final image acquisition.

Referring again to FIG. 34, the control unit 100 displays the final image using the display 14 or the like (S2109). At this time, the form provided to the user may be a portion excerpted from only the portion corresponding to the third region 62 in the obtained final photograph. Next, the final image control according to another embodiment of the present invention will be described with reference to FIG.

FIG. 40 is a flowchart illustrating a method of controlling an image of a specific area inside a refrigerator taken at a specific time according to an embodiment of the present invention.

Referring to FIG. 40, the control unit 100 obtains a final photograph (S2901), and can cut out an image corresponding to a desired area in the final photograph. The acquired final image is stored in the storage unit 18 (S2903).

Then, the control unit 100 determines whether the user input for displaying the final image is received through the user's input or the like (S2905).

For example, at the time of acquiring a final photograph, the control unit 100 may form an icon-shaped user input unit on the display 14 configured with a touch screen, It is possible to determine whether the input is received or not. The user input unit may include a physical key or the like as described above in addition to the icon formed on the touch screen, but the present invention is not limited thereto.

When the control unit 100 receives the user input for displaying the processed final image in the final photograph, the control unit 100 displays the final image (S2907). The control unit 100 can display the final image using the display 14. [ The display of the final image can be said to replace the existing image. Therefore, the update of the final image is possible.

Of course, updating of the final image may be performed automatically regardless of the user's input.

According to the embodiment of the present invention, the refrigerator can recognize whether a drawer or the like is moving by recognizing a marker previously formed on a drawer or the like. Also, the refrigerator can acquire the final state of the stored product stored in the drawer in the form of an image using a camera installed therein. At this time, the refrigerator can determine the optimal time point at which the final storage state of the drawer and the like can be obtained by using the marker formed on the drawer or the like.

Hereinafter, an installation position of the camera and a photograph that can be obtained therefrom will be described.

The embodiment of the present invention is characterized in that a camera 70 is installed outside the drawer 50, that is, in the storage chamber 22, and a draw body 50, which forms the drawer 50, (Including the front and rear side walls, the upper side of both side walls, and the like) can be continuously photographed to simultaneously recognize the storage state of the drawer and the moving state of the drawer. Of course, the photographs taken by the camera 70 are taken together to the upper space of the drawer 50, that is, the portion corresponding to the first area 42.

Basically, photographs taken by the camera should include both the opening and closing operation of the drawer and the path of the drawer. Therefore, the camera should be installed in a position where it can photograph the path of the opening and closing of the drawer from the outside of the drawer. That is, the camera is preferably installed at a position where at least a state in which the drawer is closed and a state in which the drawer is completely opened can be photographed with one photograph.

Further, the camera 70 may be divided into another storage area of the storage room as well as the drawer area, for example, a shelf area where the food or the article is stored, So that one camera can be configured to cover another storage area including a drawer area, that is, a plurality of storage areas.

For this, the drawer 50 is preferably formed on the bottom surface of the refrigerator storage compartment. The drawer 50 may include a front wall and two side walls, a rear wall and a bottom wall, and an upper opening. The camera 70 should be provided at the upper portion of the drawer so that at least a part of the interior of the drawer and the upper portion of the front wall of the drawer in a state in which the drawer is fully opened are exposed.

Then, a marker made of a specific pattern is formed or attached at any position on the front wall of the drawer exposed on the camera or on the upper surface of both side walls.

At this time, since the camera 70 is fixed at a predetermined position, it is necessary to configure the position of the printing unit of the drawer 50 to change relatively. That is, when the camera 70 receives the drive command from the control unit, the camera 70 continuously captures images at a constant speed and sequentially transmits the photographed internal images to the control unit 100. The control unit 100 tracks the position of the printing unit from the photographs relating to storage of the in-store articles received from the camera 70, and determines the information about the drawer's movement, as well as the open state of the drawer.

The range that the camera can take can be determined by the angle of view of the camera, which is composed of a vertical angle of view, which means a shooting range for the vertical direction, and a horizontal angle of view, which determines a shooting range for the horizontal direction.

One end of the vertical viewing angle of the camera 70 should include at least the drawer front wall in a state in which the drawer is fully opened and the other end in the draw closed state, Up to the part of the upper surface of the drawer that covers the upper opening of the drawer.

At this time, it is possible to simultaneously recognize another storage space in addition to the drawer internal area by one camera. Another embodiment of the present invention is characterized in that a drawer is opened and a separate storage space (hereinafter referred to as " A storage compartment floor compartment). In this case, the vertical angle of view of the camera 70 is set to be such that another storage space including the inner area of the drawer (bottom storage compartment) can be recognized even in the shooting range including the fully opened state and the fully closed state It is possible.

Meanwhile, another embodiment of the present invention with respect to the vertical angle of view of the camera 70 may simultaneously photograph the drawer area and the shelf area. Therefore, the other end range of the vertical angle of view can be determined so as to include any one of the plurality of shelves arranged at regular intervals. In the case where the camera vertical angle includes the shelf area, a preferable example may include up to a part of the shelf at the uppermost stage among the plurality of shelves, more preferably at least the upper shelf To the rear end.

According to these embodiments, one camera captures the storage state of at least two storage spaces including the inner area of the drawer with one photograph taken by one camera.

For this purpose, a desirable installation position of the camera is required for the drawer formed at the bottom of the storage compartment. The inventors of the present invention first examined the installation position of the camera disclosed or disclosed through the data published at the time of filing of the present invention. It was examined whether the inside of the storage room could be photographed by installing it on the top of the door outside the refrigerator storage room.

When the camera is installed outside the storage room (door), the camera is always exposed to the external temperature, and condensation is not formed around the camera, so that there is an advantage that a separate condensation prevention structure is unnecessary. However, it is difficult to photograph the inside of the storage room with the camera mounted on the door because the work for storing the goods in the storage space inside the drawer and the storage room is always performed with the door open. Particularly, one object of the present invention is to obtain not only the movement state of the drawer outside the drawer but also the storage state of the article in the drawer. However, the user's work in the drawer is always in a state in which the door is opened, and in a state in which the door is closed, the drawer is also closed.

Therefore, in order to photograph at least the inside of the drawer and the state of the drawer, it is preferable to arrange the storage room so that the storage room can be photographed in a main storage room where there is a drawer to be photographed, Do.

On the other hand, according to one aspect of the present invention, in a case where the camera is directed from the lower portion of one of the shelves positioned above the drawer toward the opening of the main storage room, the photographing of the shelf area is impossible It is difficult to install and wire the drive unit for driving the camera on the shelf, and a wide-angle camera must be used in order to capture all of the path of the drawer.

The inventors of the present invention have found that, when a camera is installed on a side wall of a storage room in a range in which a drawer can be exposed, the image showing the storage state of the inside of the drawer is distorted due to the asymmetric image due to the difference in distance from the camera in the left- I could see that it was severe. Therefore, it can be seen that the satisfaction of users can be greatly reduced.

It is an object of the present invention to detect the opening / closing state and the degree of the drawer by tracking the moving state of the drawer without a separate sensor. In the asymmetric image, The image can be photographed in a relatively enlarged state (image distortion). Therefore, the size (pixel size) occupied by the marker portion may become large with respect to the total size of one image, so that the position change may not be revealed clearly. In addition, since the range to be processed for determining the change of the marker position is increased, the image processing time may become longer, so that the continuously transmitted image may not be missed or processed.

Accordingly, it has been found that the installation of the camera on the sidewall or edge of the storage compartment already known at the time of filing of the present invention does not meet the intent of the present invention.

In one embodiment of the present invention, the camera is installed on the top wall of the storage compartment corresponding to the upper opening of the drawer, and the state in which the drawer is fully opened and the drawer is fully opened The direction of the camera is directed toward the inside of the storage compartment toward the rear wall side of the storage compartment, and the focus of the camera is inclined at a certain angle so as to face a certain point on the rear wall of the storage compartment. Thus, with respect to the above-described embodiments of the invention relating to the vertical angle of the camera of the present invention, an image with minimized distortion can be obtained.

In another preferred embodiment of the present invention, the upper end of the upper shelf of the shelves disposed above and below the storage compartment at the front opening of the storage compartment is located at the upper wall of the storage compartment, To be projected. The direction of the camera is directed toward the inside of the storage compartment toward the rear wall side of the storage compartment, and the focus of the camera is inclined at a predetermined angle so as to face a certain point on the rear wall of the storage compartment.

Preferably, the camera is located at a central portion of the upper portion of the storage chamber, and a printing unit located on the upper surface of the drawer front wall or both side walls is provided on the upper surface of the drawer front wall. More preferably, the upper surface of the drawer front wall corresponding to the camera position provided on the upper surface of the storage chamber is provided.

In a refrigerator which opens and closes an opening of one storage room by one or more doors, that is, left and right doors, the camera is installed in a storage room upper surface (ceiling front surface) at a position where the left and right doors are bounded, , The printing unit is positioned at a portion near the right and left boundaries. That is, in view of the inside of the storage compartment, it is preferable that the left drawer is positioned on the upper right side of the front wall or the upper side wall of the right side wall, and the right drawer is located on the upper left side of the front wall or the upper side of the left side wall. In other words, it is preferable that printing portions are formed at positions close to the vertical lower portion of the camera. This is because there is a high risk of distortion of the printed portion as the print head is further away from the vertical lower portion. An example of this is shown in Figure 41B.

In this case, when the printing unit is provided on both sides of the upper surface of the front wall of the left and right drawers, the drawers used for the storage room opened and closed by the left and right doors can be commonly used regardless of the left and right positions. So that the efficiency can be increased.

As described above, according to the camera position and the position of the printing unit according to the present invention, since the moving path from the closed state of the drawer to the open state is included in the shooting range of the fixed camera, It is possible to detect not only the opening and closing state of the lower door but also the opening and closing degree and the stop or movement of the drawer, and at the same time, the storage state of the articles in the drawer can be detected or confirmed by one camera.

In addition, in addition to the area inside the drawer, the bottom storage bin installed on the bottom of the storage room can be checked while the drawers are closed, so that a plurality of storage areas can be recognized by one camera.

In addition, when the view angle of the camera is enlarged to the shelf area, three storage areas can be selectively recognized by one camera from the storage room bottom bin to the front shelf area as well as the drawer interior area.

On the other hand, since the camera provided on the upper surface of the storage compartment and the printing unit formed or attached on the upper portion of the drawer are arranged in a substantially vertical alignment, The space occupied by the printing unit can be relatively minimized and the time for tracking and determining the position of the printing unit through the image processing can be minimized.

Hereinafter, a marker tracking recognition method formed in a drawer will be described.

A printing unit (marker or Printing paper) recognizable by image reading can be installed at a specific portion of the drawer body exposed to the camera. By tracking the position of the marker part, it is possible to detect the opening / closing degree of the drawer, and the open and closed states. The marker may be exposed to the camera from the closed position to the fully opened position on the movement path of the drawer so that the state of the marker can be determined by the position of the marker. That is, various state information of the drawer can be determined through the change of the position of the marker in the pictures taken continuously.

As described above, the print structure may have a pattern of a specific type in order to track and recognize the movement state or open / close state of the drawer. At this time, the printing unit can be disposed at various positions on the upper surface of the drawer exposed to the camera.

It is preferable that the printing unit is installed at a portion of the top surface of the drawer or at the top of both side walls of the drawer, which can be the boundary of the maximum opening portion of the drawer.

The pattern of the printing unit may be formed in a pattern in which patterns having a certain size and a predetermined shape are repeated in a color contrasting with each other. The shape of the pattern may be various shapes such as a square, a triangle, a circle, and a wavy shape.

Such a printing unit may cause a deterioration in the overall feel of the refrigerator, and may have a sense of discomfort depending on the user. Therefore, it is necessary to minimize the repetition of the pattern so that there is no sense of irritation to the user's feelings, and consequently, the printing unit can be considered to be able to appeal to the user.

However, the limitations of this design aspect are not limited to the design problem, but the situation of the drawer can not be precisely recognized. That is, it affects the recognition rate of the print portion of the drawer, which ultimately results in an inability to accurately detect the position of the drawer.

That is, considering the design factor of greatly reducing the size of the printing unit, it is possible to select the degree of repetition of about two to four times without repeating the pattern several times with a pattern of a certain shape. In this case, the control unit for processing the image of the inside of the room may not correctly recognize the boundary of the printing unit (marker) in the entire image.

Therefore, according to a preferred embodiment of the present invention for solving such a problem, since the drawer is usually made of bright color, transparent or translucent, it is possible to select a color corresponding thereto. Referring to FIG. 41A, the length L1 in the vertical direction with respect to the moving direction (forward and backward travel) of the drawer can be set to be longer than the length L2 in the draw moving direction. A first color part having a predetermined length in the vertical direction and having the same color as the first color part in the front or rear of the first color part with respect to the advancing direction of the drawer, And is repeatedly formed to have the same color, the same color as the first hue, and the same length, with approximately the same thickness. That is, it is possible to sequentially arrange the first color part and the second color part. In this case, it is more preferable that the length L1 in the vertical direction is larger than the length L2 in the draw direction in the outer size of the marker arranged with the first color portion, the second color portion and the first color portion.

According to another embodiment of the marker of the present invention, as shown in FIG. 41B, both ends of the first hue portion at the front and rear ends may be connected to each other to surround the second hue portion. The third embodiment of the marker type shown above can be an example showing this structure.

That is, in the preferred embodiment of the marker according to the present invention, the length L1 in the vertical direction with respect to the draw moving direction is larger than the length L2 in the draw moving direction, (Dark color) with respect to the drawer, and the bright color portion contrasts with the dark color.

Since the printing unit can be configured in a small size, it is possible to meet the requirements for user's aesthetics, and the size of the printing unit is relatively small in one entire image photographed by the camera. Accordingly, the image processing area for detecting the change in the position of the printing unit is reduced so that the image processing speed can be improved. This is advantageous in that the image processing speed of the control unit matches the image capturing speed of the continuous image capturing and transmission, and the accuracy of acquiring the image at an accurate time is further improved. For example, if the size of a portion to be processed in a series of photographed pictures is large, the control unit takes a heavy load. Therefore, a very expensive control unit should be used. However, if the size of the portion to be processed is small, the load of the control unit can be reduced. Therefore, even if a low-cost control unit is used, the image can be processed effectively.

On the other hand, as shown in FIG. 41B, it is preferable that the marker includes characters, symbols, or patterns so that the user can intuitively grasp the use thereof by viewing the marker.

Hereinafter, the recognition of the marker coordinates of the drawer and the camera image processing method will be described.

42 is a view for explaining a marker position recognizing method. Hereinafter, a marker position recognition method of the drawer will be described with reference to FIG.

When a user wants to store (stock) or ship goods in the drawer, the user first opens the door. When the door is detached from the front of the storage room, the door switch detects the door switch, and the control unit receives the signal of the door switch to drive the camera in the waiting room installed in the hall. The camera which receives the driving command of the control unit applies the camera driving voltage instead of the standby voltage state, and starts the continuous shooting when the camera is ready to be driven.

The user will then attempt to open the drawer by grasping the handle on the front wall of the drawer attached to the refrigerator compartment and pulling the drawer. At this time, since the camera is continuously photographing, the camera captures the open state of the drawer at regular intervals and sends the image to the control unit.

The control unit receives an image sent from the camera. The control unit divides the entire image into predetermined pixels and assigns coordinate values to the camera at predetermined intervals. And analyzes the coordinates of the marker on the coordinates of the image.

Accordingly, when an image photographed in a state in which the drawer is closed is sent to the control unit, the corresponding image marker portion will not be exposed, and the marker coordinate at this time will have a coordinate of zero.

When the user opens a closed drawer, the camera will shoot with the marker exposed, and the camera can take a picture with the marker. When this image is sent to the control unit, the control unit recognizes the coordinate value of the marker on the whole image and compares it with the previous position. For example, when the coordinate value of the marker is changed from '0' to '50' as an example of a certain coordinate value, 'A' point in FIG. 42 can determine that the drawer is moving from the changed coordinate value have. In particular, if it changes from '0', it can be judged that the drawer has just begun to open.

When the user continues to open the drawer, the controller can determine that the drawer is being opened from the fact that the drawer is moving through a series of processes as described above and the coordinate value changes to a larger value.

When the user opens the drawer to a fully open state, the controller recognizes that the coordinate value is no longer changed and determines that the drawer is fully open or the drawer is open and stopped. For example, the point of 'B' in FIG. 42 is a point where the coordinate value of the marker does not change from '90' to '90', so that it is recognized that the drawer is stopped or the 'drawer is completely opened' It can be judged.

After a drawer is opened, whether it is fully open or partially open, it can be determined that the user is doing some work on the drawer when it is in a frozen state. Even when the drawer is stopped, the camera continuously captures the high-end image and sends it to the control unit, and the control unit monitors whether there is a change in the coordinates of the marker.

When the user finishes the operation of picking up the article from the drawer or removing the article from the drawer, the user closes the drawer again. At this time, from the image photographed by the camera, the control unit confirms that the marker coordinate value for the entire image has changed And judges that the drawer is moving again. For example, the point 'C' in FIG. 42 is a point where the coordinate value of the marker changes from '90' to '70' and the size of the coordinate value is changed in the direction of reducing the size. It is immediately after the user completes the work on the drawer.

If the coordinate value of the marker is '0' in the same way, that is, when the marker is not recognized from the image, it is judged that the drawer is completely closed.

From the images taken by the camera, the control unit determines the position of the marker by the coordinates of one entire image, and determines whether the drawer is moved or stopped, whether the drawer is closed or opened, whether the drawer is opened, And how the drawer begins to close.

Hereinafter, the camera image processing and the desired image acquisition method according to the marker position of the drawer will be described. Describes a concrete method of processing and acquiring the inner image of the drawer from the moving state of the drawer, in particular, a method of obtaining the image at the time when the user finishes the work on the drawer.

When it is determined that the marker coordinates of the drawer change to '0' or more and move in the direction in which the drawer is opened, the control unit detects only the coordinate change of the intracorporeal image captured by the camera and sent to the control unit, It buffers in the temporary buffer and does nothing for each storage partitioned in the photo.

When the drawer is continuously opened and reaches the fully opened state or is released to a certain degree, the situation in which the user performs work on the drawer proceeds. The control unit judges such a state from the fact that the coordinates of the marker are not changed. The control unit moves the image corresponding to the draw area from the storage area divided from the entire image to the temporary buffer do. That is, in the section corresponding to the 'D' point in the figure, the controller cuts a necessary portion of the entire image in the high-resolution area and moves it to the temporary buffer for the draw area. '.

In other words, the image processing method is different between the section where the drawer is opened and the section where the drawer is at rest. The reason for this is that the user's action on the draw reflects that the draw will be in the stopped state. In this way, the image processing time for tracking the coordinate change can be increased so much that the data processing efficiency can be improved.

On the other hand, when the drawer starts to be closed again in the stopped state, the control unit determines that the coordinate value changes to a smaller value immediately after the user completes the work, and at that point, The most recent image is temporarily selected from among the temporarily buffered images. That is, it captures an image of the area inside the drawer immediately after the user completes the operation.

The image of the storage state of the most recent draw area thus captured is transferred to the network server by replacing the image of the corresponding area of the display connected to the control unit directly or directly to the control unit. When the user wants the latest image stored in the network server, the latest image is provided to the user through the display installed in the refrigerator or the mobile terminal connected to the network server.

Next, a method of searching for the position of the marker by selecting a specific section, rather than searching for the position of the marker with respect to the entire photograph, is described.

That is, it is possible to set a search range for the draw area. It is possible to distinguish the opening degree of the drawer on the path of the drawer by the section and track the marker in each section. As a result, it is possible to shorten the processing (analysis) time of the photographs taken and transmitted from the camera, and to match the image processing speed and the camera photographing speed.

Generally, a drawer used in a refrigerator does not draw the entire drawer so that the drawer is detached from the drawer even if the drawer is completely opened. That is, a distance at which the drawer can be drawn out normally, usually, the opening distance is generally about 50% of the drawer full depth (drawrow front and back length with respect to the drawer moving direction).

The draw-out distance may be determined as an area to which the coordinates of the draw marker should be given in the entire image processed by the control unit. In this case, the controller can grasp the position of the drawer from the time when the drawer is drawn. In particular, in order to detect a change in the marker coordinates of the drawer, the control unit determines a data size to be processed by the size corresponding to the draw-out distance. In addition, in order to detect the change of the coordinates, the control unit must compare two images at the same time, and the processing time of such a process may be burdensome to cope with the speed of photographing by the camera.

It is preferable that the photographing speed of the camera is determined so that the speed of photographing the inside of the camera (time interval of photographing) of the camera is substantially matched with the speed at which the control unit judges and processes the transmitted image.

In this way, the position of the marker can be determined without delay and the position of the drawer can finally be determined with respect to the photographs taken by the camera. In addition, the control unit processes the image of each area partitioned by each storage space in the entire photograph of the inside of the room, and it is not difficult to image and process a new image continuously sent from the camera. That is, according to whether the door is open or the door is opened from the whole image, the image for each region is cut and moved to the temporary buffer for each region. In this case, all the images retrieved by the camera can be processed without missing It will be done.

However, since the data of the two images (the previous image and the current image) as described above must be processed, the processing capability of the control unit must be increased. Therefore, if the processing capacity of the control unit is increased in accordance with such conditions, have. In addition, in order to process an image when the drawer is not opened, the processing capability becomes excessively large, and efficient part selection becomes impossible.

In order to solve this problem, the inventors of the present invention have focused attention on the fact that it is practically difficult to sufficiently grasp the inside of the drawer when the draw-out distance of the drawer is not completely opened, that is, The image corresponding to the draw area is separated step by step according to the drawing distance of the drawer, thereby providing a method of differently processing the data.

Figure 43 shows the extent to which the drawer is open. 43A is a state in which the drawer is opened to the maximum, that is, about 50% of the drawer length is opened, FIG. 43B is about 30% of the drawer is opened, and FIG. Approximately 10% of the length is open.

For example, as shown in FIG. 43C, when the user draws a drawer and opens only about 10% of the entire length in the forward and backward directions, the article stored in the drawer can not be fully grasped. When you open about 30%, you will be able to grasp the storage status of the inside of the drawer to some extent, and this state is enough to recognize the drawer as open.

As shown in FIG. 43A, when the drawer is fully opened, about 50% of the drawer is drawn. In this case, the inside of the drawer can be fully grasped. In other words, the open state of about 30% ~ 50% should be enough for the user to store or unload the goods to the drawer, so that the drawer can be viewed as being substantially opened.

In view of this point, it is judged that the drawer is not opened when the draw-out distance is less than 30%, and the image processing is differentiated when the draw-out distance is more than 30% The data load to be processed can be greatly reduced.

The controller determines an image size corresponding to the droplet movement trajectory by imaging the image taken by the camera, determines a pixel size up to a time or distance at which the drawer is drawn by about 30% in the predetermined draw moving area, .

That is, when the drawer is pulled out to the range corresponding to FIGS. 43A to 43B, the control unit can judge the movement of the coordinates of the marker. In this case, the load of the control unit required is reduced as compared with the case of determining the movement of the coordinates of the marker from the range corresponding to FIGs. 43A to 43C.

It is determined whether or not the position of the marker has reached the position of the pixel corresponding to the distance of 30% in the section where the drawer is 30% pulled out from the closed state. In other words, the marker coordinates of the previous photograph and the marker coordinates of the current photograph are compared with each other (the precondition for judging the change of the position of the marker) is not performed and only the position of the marker is within the predetermined pixel range. .

This is because it is not necessary to compare two pieces of photograph data in the section, and only one piece of photograph data is processed. Therefore, it is not necessary to have a large data processing capability of the controller, and it is possible to cope with the camera photographing speed smoothly.

On the other hand, when the above-mentioned drawer is opened more than the 30% level, that is, when the marker reaches a position above a certain pixel in the photograph processed by the control section, two pictures The coordinates of the marker can be judged by comparing the photographed image with the photographed image thereafter).

By limiting the search interval in this way, the size of the picture is about 2/5 of the size of the image, while the search interval is not limited. Furthermore, since the size of a photograph corresponding to two photographs to be processed is about 4/5 of that of a photograph of a single photograph, the data to be processed by the control unit is smaller or substantially the same Level.

Accordingly, since the data processing speed for tracking the coordinate change value of the two photographs can correspond to the speed at which the camera is photographed, it is possible to perform data processing (coordinate tracking or monitoring) It is possible to prevent the time delay phenomenon of the processing speed even when the processing capacity of the image processing apparatus is not increased any more, so that a desired image can be obtained at an accurate time.

As a result, it is possible to divide a part of the distance that the drawer can draw from the photograph taken by the camera into the first section and the second section, as shown in FIG.

In the first section, the position of the marker with respect to the pixel of the photographed image can be determined. In this case, the first section can determine which pixel position the marker is located on in a single photograph, without comparing the coordinate values of the markers.

On the other hand, even if the drawer is actually opened in the first section, it is also possible to distinguish the drawer as being substantially closed since it is difficult for the user to draw or bring the item into the drawer.

The second section may be set as a 'search range' for comparing coordinate values of the markers shown in two consecutively photographed photographs. In the second section, since the two photographs are compared, it is possible to detect the change in the coordinates of the marker. For example, if the markers are in the same position in two consecutive photographs, it can be judged that the drawers stopped during the time the two photographs were taken. If the position of the marker is changed in two consecutive photographs, it can be judged that the drawer has been moved during the time the two photographs were taken. At this time, it is possible to judge whether the drawer is moved in the closing direction or the drawer opening direction according to the change direction of the marker position.

In the second section, it is also possible to distinguish the state in which the drawer is actually opened, since the user is exposed to the opening that the user can draw or carry the article to the drawer.

45 is a control flowchart according to an embodiment of the present invention. This will be described below with reference to FIG.

It is determined that the door 20 opens the storage compartment 22 (S10). At this time, the opening / closing of the door 20 can be detected by the door switch 110.

When the door 20 is opened, the door sensor 120 may be driven. That is, the door sensor 120 can sense whether the door 20 is rotated by a predetermined angle? Or more. That is, the door sensor 120 is not operated unless the door 20 is opened, and operation can be started when the door 20 is opened.

When the door 20 is opened, the camera 70 is operated to start photographing (S14). At this time, the camera 70 can photograph 10 pictures at a predetermined time interval, for example, 1 second.

The photograph taken by the camera 70 is stored in the storage unit 18 (S16). At this time, if a plurality of photographs are taken by the camera 70, recently photographed photographs may be stored, and the photographs taken before the photographs may be deleted due to capacity limitation of the storage unit 18.

Meanwhile, it is possible to detect whether the drawer 50 is opened by the draw detection unit 130 (S20). The draw detection unit 130 may detect that the drawer 50 is drawn out from the lower space of the shelf 40.

When it is detected by the draw detection unit 130 that the drawer 50 is closed, a photograph taken at that time or a photograph taken when the drawer closest to the time is selected as a final picture (a state in which the drawer is opened) S40). Since the camera 70 takes a predetermined number of pictures per second, it is also possible that the photographer does not photograph the picture at the time when the drawer 50 starts to close. When the drop sensing unit 130 determines movement of the drawer 50 by analyzing movement of the marker represented by the photograph taken by the camera 70, It is possible to select.

The control unit 100 divides the final photograph into the first area 42, the second area 52 and the third area 62 (S42). At this time, the controller 100 can identify the number of pixels on the basis of the adjustment line 15, divide each area, and obtain a final image of each area.

As described above, when the drawer 50 is drawn out, the second region 52 and the third region 62 overlap each other.

On the other hand, since the information that the drawer 50 has been taken out is obtained in S20, it can be determined that the control unit 100 has obtained the second image.

The information on the second area 52 may be updated (S46). The control unit 100 may transmit a second image showing the second area 52 to the display 14 so as to change the corresponding part based on the final photograph.

On the other hand, if it is determined in S30 that the drawer 50 does not start to be closed, a photograph can be taken again as in S14.

The state in which the drawer 50 is not closed at S30 can be regarded as the end of the access to the second area 52 by the user. That is, the user can store new food in the second region 52 or take out the stored food from the second region 52 while the drawer 50 is stopped.

On the other hand, the state in which the drawer 50 is closed can be predicted that the user finishes the access to the second area 52 and draws the drawer 50 into the lower space of the shelf 40.

On the other hand, if the door 50 is not opened in S20, it can be determined whether or not the door 20 is about to close (S60).

At this time, the door sensor 120 can detect whether or not the door 20 is about to be closed.

When the user finishes accessing the storage room 22, the user closes the door 20 and hermetically closes the storage room 22. That is, the user can store new food in the storage room 22 or take out the stored food from the storage room 22 in a state where the door 20 opens the opening 14.

If the user closes the door 20, it can be determined that the user maintains the stored state of the food until the access to the storage room is terminated and the door 20 is opened again.

If it is determined in S60 that the door 20 is about to be closed, the control unit 100 selects a photograph taken at that time or a photograph that is closest to the time at that time as a final picture (state in which the drawer is closed) (S70). That is, the control unit 100 may select a photo at the time point when the photographing of the camera 70 ends or at a point adjacent to the point of time as the final photograph.

The controller 100 divides the final photograph into the first area 42, the second area 52 and the third area 62 (S72). At this time, the final photograph may include a first image showing the first region 42. The latest state of the food stored in the first region 42 is because the user has finished accessing the storage room 22. Accordingly, the user can obtain accurate food information for the first region 42 through the obtained first image.

On the other hand, since it is determined in S60 that the drawer 50 has been pulled in, the controller can determine that the first image and the third image have been obtained. As described above, when the drawer 50 is drawn out, the second region 52 and the third region 62 overlap each other, but the first region and the third region are not overlapped with each other This is because it is a position where photographing is possible together.

The information about the first area 42 and the third area 62 may be updated (S76). The control unit 100 transmits a first image showing the first region 42 and a third image showing the third region 62 to the display 14 and displaying the corresponding portion on the basis of the final photograph .

Meanwhile, although not shown in the drawing, the door 20 may be rotated to close the storage compartment 22, and when the storage compartment 22 is closed, the driving of the camera 70 may be stopped. At this time, the door switch 110 may sense whether the door 20 closes the storage compartment 22.

That is, in the present invention, considering the time when the user ends the use, the user obtains information about the food stored in the corresponding region by processing the photograph obtained at that time. Accordingly, the user can obtain accurate information on the food stored in the storage room 22.

On the other hand, the door sensor 120, the door switch 110, and the drop sensing unit 130 may be installed on the left and right sides, respectively. In this case, individual information on the right and left doors and the left and right drawers can be obtained. If the drop sensing unit 130 senses the movement of the drawer through the analysis of the marker in the photographed photograph, the drawer sensing unit 130 is one, while the markers are installed in the two drawers .

Therefore, if only the left door is opened based on this information, it is possible to update only the left part of one photograph. On the other hand, when only the right door is opened, it is possible to update only the right portion of one photograph.

Of course, when the left door and the right door are opened together, it is also possible to update the left and right portions of a single photograph and provide them to the display 14, if the above conditions are satisfied.

46 is a control flowchart for explaining the modification of Fig. This will be described below with reference to Fig.

In the modification of Fig. 46, S60 is specifically divided into S62 and S64, and only differences will be described in detail.

That is, whether or not the door 20 is just before the door 20 is closed may be determined by determining whether the door 20 is rotated in a direction to close the door 20 or by determining whether the door 20 is rotated by a predetermined angle or more have.

In other words, the door 20 is rotated in the closing direction (S60), and when the door is rotated to a certain angle or less (S64), the photographing by the camera 70 can be terminated.

In order to obtain information on the food stored in the storage room 22 in the picture photographed by the camera 70, interference with the basket 20 installed in the door 20 as well as the door 20 should not occur. Accordingly, the expression 'immediately before the door 20 is closed' means that the door 20 and other components are not displayed on the photograph taken by the camera 70.

47 is a control flowchart illustrating another modification of Fig. Hereinafter, description will be given with reference to FIG. 47, but only the difference from FIG. 46 will be described.

47, the rotation direction of the door 20 is not determined, unlike in FIG.

If it is determined in S12 that the door has been rotated by a certain angle or more, the door 20 is rotated in a direction to open at the moment. Therefore, when the user normally rotates the door 20 again and the door 20 is rotated by a predetermined angle, the door 20 may be closed by the user.

Therefore, in another modification, S62 in FIG. 46 is omitted. If it is determined in S20 that the drawer is not opened, S64 is immediately performed to determine whether the door has been rotated to a certain angle or less.

FIG. 48 is a control flowchart illustrating another modification of FIG. 45; FIG. This will be described below with reference to FIG.

48, a control flow for detecting the movement of the drawer 50 by a marker will be described. For the convenience of explanation, the overlapping portions are omitted and only the difference portions are described.

S20 and S30 as a whole can be specifically divided into the steps of S22, S24 and S26.

After the photograph is stored in the storage unit 18 at S16, it is determined whether the camera 70 recognizes the marker (S22).

Since the camera 70 photographs the photograph from the upper side to the lower side of the inner case 12, the camera 70 may not be able to see the marker. That is, when the handle portion of the drawer 50 is covered at the front end of the shelf 40, that is, when the drawer 50 is not pulled out at all, the marker may not be recognized.

If the marker is recognized in the photograph taken on the camera 70, it can be determined whether the marker is stopped (S24). At this time, it is also possible to determine the movement of the marker only in the second section by distinguishing the movement path of the marker from the first section and the second section, as described above.

Since the camera 70 photographs a plurality of photographs at predetermined time intervals, it is possible to determine whether the marker is moved by analyzing the photographs by comparing them. Since the marker is displayed on the drawer 50, it can be determined that the drawer 50 is also moved when the marker is moved.

Then, it is determined whether the marker has moved a predetermined distance or more from the position where the drawer 50 is closed (S26).

If the marker has not moved over a certain distance, it is not possible to obtain a proper image for the second area 52 because the drawer 50 is not sufficiently opened. For example, in the photograph taken with the drawer 50 taken out in 1/3 of the drawer 50, the inner space of the drawer 50 is not sufficiently shown, and as the image obtained in the photograph, It is impossible to obtain the information about the image 52. At this time, it may be determined whether or not the marker has entered the search range for determining the movement of the marker.

If it is determined that the marker has moved a predetermined distance or more, the user can see that the access to the second area 52 is terminated and the use of the second area 52 is completed.

Therefore, the photographs obtained at the time of the state in which the three conditions are satisfied or at the time adjacent to the time may include information on the food stored in the second region 52.

Fig. 49 is a view showing a modification in which Fig. 45 is modified in a different manner. This will be described below with reference to FIG.

In FIG. 49, unlike FIG. 45, there is a difference that the camera is not driven immediately when the door 20 is opened in the door switch 110.

When the door switch 110 determines that the door 20 is opened, the door sensor 120 is driven.

Then, the door sensor 120 determines whether the door 20 has been rotated by a predetermined angle or more (S12). At this time, when the door 20 is not rotated more than a predetermined angle, the camera 70 does not take a picture.

When the door 20 is rotated over a predetermined angle, the camera 70 takes a picture (S14). At this time, the camera 70 can photograph 10 pictures at a predetermined time interval, for example, 1 second.

At this time, the photographed image includes the first image for the first region 42. On the other hand, since the door 20 is rotated by a certain angle or more, interference of the basket installed on the door 20 may not occur in the photograph. That is, the shape of the basket installed on the door 20 may not be displayed on the photograph.

Further, it is possible to detect whether or not the door 20 is about to close at the door sensor 120 at S60. The term 'immediately before the door 20 is closed' means that the door 20 is rotated in a direction in which the door 20 is closed so that the door 20 is rotated at a certain angle or less.

The camera 70 can take a picture and stop photographing until the door 20 is closed (S49). In FIG. 45, the camera 20 photographs a picture up to the moment when the door 20 is closed and the door switch 110 is pressed. However, in the modified example, when the door 20 is rotated to a certain angle by the door sensor 120, the camera 70 stops photographing. In this case, the predetermined angle may mean an angle to prevent the interference of the picture photographed by the camera 70 by the door 20, the basket installed on the door 20, and the like.

50 is a control flowchart according to another embodiment of the present invention. This will be described below with reference to FIG. The control flow of FIG. 50 can be continued at intervals of a predetermined time when it is determined that the door 20 is opened by the door switch 110.

First, it is determined whether the door 20 is opened by a predetermined angle (?) By the door sensor 120 (S100).

When the door 20 is opened by a predetermined angle or more, it is determined that no interference occurs in the door 20, and the camera 70 takes a picture (S110). At this time, the camera 70 can photograph a single photograph.

Then, it is determined whether the drawer 50 has been drawn more than the first predetermined distance (S120). Here, the first predetermined distance may mean a distance at which food information about the second area 52 can be sufficiently obtained when a photograph is taken by the camera 70. For example, the first predetermined distance may be 2/3 or 1/2 of the entire length of the drawer 50.

The draw-out distance of the drawer 50 can be detected by the draw detector 130.

If the controller 50 determines that the drawer 50 has been drawn more than the first predetermined distance, the controller 100 divides the pictures into areas and divides them into a plurality of images (step S122).

Since the drawer 50 is a photograph obtained in a drawn-out state, the first image and the second image can be obtained in the photograph.

The controller 100 may update the first image and the second image (S124).

On the other hand, if it is determined in step S120 that the drawer 50 has not been drawn more than the first predetermined distance, it can be determined whether the drawer 50 is drawn below the second set distance in step S130.

Here, the second set distance may mean a distance at which food information about the third area 62 can be sufficiently obtained when the camera 70 photographs the photograph. For example, the second setting distance may be 1/3 of the entire length of the drawer 50. Of course, it is also possible to select the second set distance as '0', and to select the state in which the drawer 50 is completely drawn into the lower space of the shelf.

If it is determined that the drawer 50 has been drawn below the second predetermined distance, the controller 100 divides the pictures into areas and divides them into a plurality of images (S132).

Since the drawer 50 is a photograph obtained in a drawn-out state, the first image and the third image can be obtained in the photograph.

The controller 100 may update the first image and the third image (S134).

50, the control is repeated when the door 20 is opened. When the door 20 is closed, the door 20 is completely closed. A final image for the food stored in the storage room 22 can be obtained since no more updates to the image are generated in the terminated state

51 is a view for explaining a process of updating an image with two doors and two drawers for opening and closing the storage room. This will be described below with reference to FIG.

The overlapping contents in the above embodiment will be briefly described.

When electricity is supplied to the refrigerator, it can be individually determined whether the left door or the right door is opened in the door switch 110 installed at the left and right sides, respectively (S302 and S303).

If the door switch 110 determines that either the left door or the right door is open (S303), the camera 70 starts photographing (S304).

Either the left door or the right door is opened, or the camera 70 can take a picture even if both the left door and the right door have a time interval or both are opened at the same time. This is because, if it is determined that the door is opened through any one of the door switches, lighting is turned on in the high room.

When the drawer disposed on the left side is drawn more than 130 mm (S310), an image related to the left drawer is captured (S312). The image related to the left drawer may mean the portion corresponding to the left drawer among the photographed pictures, and may mean the entire photographed image.

At this time, 130 mm can be changed by the refrigerator manufacturer or user, but it can mean a draw-out distance that allows the user to draw the drawer and access the storage space inside the drawer.

If the left drawer is drawn less than 130 mm (S310), the previous image captured by the camera 70 is maintained (S310).

The left door sensor detects the opening angle of the left door. At this time, if the left door is opened at an angle smaller than 80 degrees, an image of the left multi-storage space provided in the lower space in which the left drawer is drawn is captured (S332).

At this time, 80 degrees may mean an angle at which the various structures installed in the left door and the left door do not interfere with each other when the storage space is photographed by the camera. It is possible to change 80 degrees according to the specific structure and size of the refrigerator.

If the left door is opened at an angle larger than 80 degrees, it is possible to maintain the previous image captured by the camera 70 (S336).

In addition, it is possible to detect that the left door is closed by the door switch 110 (S334).

On the other hand, it is possible for the right drawer and the right door to implement the same type of control flow as described in the left drawer and the left door. That is, the same contents are applied in S320 to S324 and S340 to S346, so a detailed description will be omitted.

When the left door and the right door are closed, it is possible to process the captured image in the control unit 100 and display the processed image on the display 14 (S350). At this time, the display can be a display installed in the refrigerator.

Then, the camera can stop photographing (S352).

On the other hand, the refrigerator is connected to the WiFi to determine whether communication is possible (S354). If the communication is possible, the refrigerator can transmit the image to the external server (S356).

The server can determine whether the user is synchronized with the application of the terminal used (S358), and if synchronized, display the image on the display 14 of the terminal.

Accordingly, even if the user is not in front of the refrigerator, the user can receive information on the article stored in the refrigerator through the user terminal.

If the refrigerator is not connected to the WiFi so that it can not communicate with the outside or is not synchronized with the application of the terminal, it is preferable to keep the previous image on the display 14 of the user terminal and provide the user with the immediately preceding image Do.

Hereinafter, the components provided for taking a picture and acquiring a picture at a desired point of time will be schematically described.

When the door switch detects that the user has started to open the door, the control unit starts driving the camera to photograph the inside of the room. The camera is continuously taken at a frame rate per second. Opening the door of the user means taking out a desired item from the refrigerator or storing a new item in the refrigerator. Thus, by continuously photographing the inside of the refrigerator, It is possible to monitor whether there is a change in the state of the storage space.

The storage room is divided into various shelves, drawers, or baskets to form a storage space, and the storage space is stored and stored in a storage space according to the type and contents of an article or a container.

The camera can be divided into a shelf area, a drawer area, and other storage areas, which are separated by a shelf, for the storage space. Accordingly, the user is required to check the storage state of the article in the area immediately after the user completes the drawing or receiving operation. If two drawers are provided and two other storage areas are provided parallel to both sides, the user can provide information that allows the user to check each of the two drawers and two other storage areas through the camera .

In one embodiment of the present invention, the other storage area is formed in the bottom of the storage room and is located in front of the drawer storage area so that at least a part of the storage space overlaps when the drawer is opened, present.

When the door switch senses the opening of the door, the control unit turns on the door sensor included in the refrigerator hinge assembly to sense the movement of the door according to the rotation of the door. The door sensor may be always on, but it is also possible that the door sensor is turned on only when the door is opened in the door switch.

When the door is opened beyond a certain angle, the controller determines that the door is substantially opened so that the user can access the storage room and carry the goods in or out, and determines the storage status of the shelf area as an object to be upgraded.

When the drawer is opened by the drawer sensing unit and the drawer is opened, the storage state of the drawer area is determined as an object to be upgraded. If the drawer is not opened, .

The door sensor of the embodiment of the present invention senses when a certain angle has elapsed in the process of opening and closing the door, and sends the related information to the control unit. Here, the door sensor can detect whether the door is opened or closed or opened.

In the embodiment of the present invention, when the on signal of the door switch is sensed, the control unit turns on the door sensor, the door sensor detects when the door passes over a certain angle, and sends the corresponding point of time to the control unit . The controller determines that the door is opened at this time. Specifically, the door is rotated at a predetermined angle to open the door.

On the other hand, when the door is opened, the door that has been opened is closed when the user finishes withdrawing or wearing the article. When the door passes the door sensor, the door sensor sends a signal to the controller, It is judged that the door is just before closing. Specifically, at this time, the door is rotated by a predetermined angle to open, but is rotated to close.

A process of photographing an image in a hood from a camera in the process of opening and closing the door and upgrading the image taken immediately after the user completes the entry and exit of the article will be described in detail with reference to an embodiment of the present invention.

When the refrigerator door switch is turned on with the door closed, the control unit turns on the camera and the door sensor. The camera may be switched from the standby mode to the driving mode, and the door sensor may be switched to a state capable of determining the opening angle of the door.

The camera can continuously photograph the interior of the refrigerator at a predetermined speed per a predetermined time. The photographed image is sent to the control unit, and the control unit temporarily buffers the transmitted image as an image including the entire image.

The image [Picture (or photograph)] and the image (Image) used in this specification can be used separately from each other.

More specifically, a picture (Picture or photograph) means raw data transmitted immediately after being photographed by a camera, and an image is a picture (photograph or photograph) that is raw data, May refer to data that is buffered, corrected, or transmitted.

If it is determined that the door switch is turned on and the door is opened at a predetermined angle and the control unit determines that the door is opened, the control unit determines whether the entire image contains the storage state of the product .

For example, when the drawer is opened, the control unit displays an area (shelf area) where the stored entire image is stored on the refrigerator shelf and a drawer storage area (drawer area) It is judged that it contains the storage state of the article. On the other hand, if it is determined that the drawer is closed, the control unit displays the stored entire image in the storage space internal area (storage area), which is a region formed in the refrigerator shelf area and the front end of the drawer, It is judged that it contains the storage state of the article.

Here, the photograph of the entire refrigerator storage room photographed and transmitted by the camera is divided into three different storage spaces, and the shelf area is always included, and the drawer area and the storage area are divided into three areas, Depending on the open / closed state of the lower part. The draw area and the storage area do not coexist with each other in the photograph.

Accordingly, one entire picture transmitted from the camera includes at least two areas, one of which is a shelf area, and the rest of the storage area corresponds to a drawer open / close state.

The present invention is intended to capture at least two different spaces including at least a drawer, such as a drawer inner space and another storage space, by a single camera to transmit the latest I / O status of the goods to the user, A method of photographing only the area and the shelf area and performing an upgrade may be another embodiment of the present invention and it is also possible to photograph only the draw area and the storage space in a fixed manner as an embodiment of the present invention .

On the other hand, partitioning one whole photograph of the storage room constituting one independent heat insulating space in the refrigerator into storage areas in the storage room can be determined by the number of doors opening and closing the storage room. For example, in a refrigerator that opens and closes the storage room by one door, the storage area to be photographed may be roughly divided into two or three, and when the storage room is opened and closed by two right and left doors, It can be broadly classified into four types or six types.

At this time, the drawer area and the storage area may be divided into left and right areas, respectively, and may be managed as two divided areas in the entire photograph, and the shelf areas may be integrated into one area and managed. In this case, all of the parts partitioned in the whole photograph may be composed of five kinds.

The shelf area is connected to the storage spaces facing the right and left doors. In the shelf area, articles, food or vegetables may be stored on the left and right sides. Even if only one of the right and left doors is opened, it is possible to integrate the entire shelf area and replace the entire shelf area with the latest data (data), since the door can be opened or closed by opening one of the right and left door This is necessary. In consideration of this point, even if only one of the left and right doors is opened, the entire shelf area can be updated.

More specifically, if it is determined that the drawer is opened when the door is opened, the control unit temporarily separates the shelf area and the draw area from the transferred whole picture and buffers them temporarily. And, when the user finishes the goods in and out of the drawer area and / or the shelf area and tries to close the drawer, that is, when the drawer opens and then starts to move while stopped, Captures or cuts a necessary part of the draw area in the captured image and stores it in the storage part as an image in the storage state of the latest draw area. At the same time, the captured image is upgraded by replacing the captured image with the latest image with respect to the drawer area in the refrigerator high image displayed on the refrigerator display. In addition, the control unit sends the image of the captured draw area to the server system connected to the network.

On the other hand, if it is determined that the door is not opened and the door is closed when the door is opened, the controller determines that the entire image of the transmitted image includes the shelf area and the storage area. Therefore, the shelf area and the storage area are separated from each other and temporarily buffered.

When the user wears the goods in the storage area and / or the shelf area, completes the shipment and closes the door, the door sensor senses when the door passes the specified angle and sends a signal to the control part. The control unit determines that the door is closed immediately before the door is closed by the signal transmitted from the door sensor (that is, it can be expressed as just before the picture is blocked by an obstacle such as a door) Captures a necessary part in the area and stores it in the memory as an image of the latest storage state of the storing area. At the same time, the captured image is upgraded by replacing the captured image with respect to the storage area in the refrigerator high image displayed on the display.

At this time, the upgrade may mean that the image is replaced with the latest image. In addition, the control unit sends the image of the captured draw area to the server system connected to the network.

On the other hand, if the drawer is in the open or closed state, or the shelf area is determined to be just before the door is closed, the most recent image among the buffered images is captured and stored as a recent storage state of the shelf area in the memory.

The displayed image of the refrigerator display is replaced by the captured image (either a recent shelf area image, an image of a recent drawer area or an image of a recent storage area, or both), or transmitted to the server system or communicated with the refrigerator The transfer of the user to the mobile device may be simultaneously performed while the captured image is stored in the refrigerator memory, or may be performed at a time when the user requests it, or may be performed by the user's request signal for the individual areas.

Here, one image transmitted from the camera may be buffered as one image by the control unit. If it is determined that the one image is photographed in a state in which the door is opened, it may be determined whether the one image includes a draw area or a storage area in the one image depending on the state of the drawer opening / closing. When such a determination is made, the corresponding image is extracted and buffered in a separate temporary buffer.

If it is determined that one image is photographed at the time when the drawer starts to be closed, the most recent image among the temporarily buffered images is stored in the memory, and if the new entry and exit by the user is completed, the image is replaced with the latest image . Also, if it is determined that the door is just before closing, the most recent image among the images of the temporarily buffered storage area is stored in the memory.

In the meantime, the shelf area can extract only the shelf area from the most recent image immediately before the door is closed and store it in the memory, and prepare a temporary buffer for the separate shelf area so as to extract and buffer the shelf area only It is possible.

Hereinafter, various embodiments will be described with reference to FIGS. 52 to 55, in which the most recent image of the refrigerator obtained by the above-described various methods is transmitted to an individual's mobile terminal or a provider server and utilized.

52 is a ladder diagram for explaining a refrigerator operating method according to another embodiment of the present invention.

Referring to FIG. 52, when it is determined that the final picture of the specific area inside the refrigerator has been acquired through the steps described with reference to FIG. 34 (S1101), the controller 100 displays the final image, 2) (S1103). The terminal 2 may include a smart phone, a personal digital assistant (PDA), a tablet PC, and the like, but is not limited thereto. At this time. The control unit 100 may transmit not only the final image inside the refrigerator but also various information related to the final image described below to the terminal 2. [

At this time, the control unit 100 may transmit the final image to the terminal 2 as soon as the final image is obtained through the final photograph, store the final image in the storage unit 18, An image stored in an event format may be transmitted to the terminal 2, but the present invention is not limited thereto.

Subsequently, the terminal 2 displays the final image received according to the user's selection (S1105). For example, the terminal 2 may execute an application capable of receiving and displaying an image of a refrigerator according to a user's selection, or may perform a multimedia service to display an image of a received refrigerator.

According to the embodiment of the present invention, even if the user does not look into the refrigerator directly, the image of the inside of the refrigerator can be confirmed by using the mobile terminal.

53 is a ladder diagram for explaining a refrigerator operation method according to another embodiment of the present invention.

Referring to FIG. 53, when it is determined that the final picture of the specific area inside the refrigerator has been acquired as described above (S1301), the control unit 100 generates an image by dividing the final pictures into areas, To be stored in the storage unit 18 (S1303).

Then, the terminal 2 transmits a final image request command to the refrigerator (S1307) when a user input for displaying a current image of the refrigerator, that is, a final image of the refrigerator specific area is received (S1305).

The controller 100 controls the communication unit (not shown) so that the image of the refrigerator that is most recently captured and stored in the storage unit 18 is transmitted to the terminal 2 as a response to the final image request command 270 (S1309).

According to the embodiment of the present invention, the communication between the mobile terminal and the refrigerator is performed only when the user needs it, thereby being more efficient and economical. Hereinafter, only a case where an image or the like is transmitted as a response to a request will be described. However, the case where an image or the like is transmitted in an event format as described with reference to FIG. 35 is also included. It is not.

54 is a ladder diagram for explaining a refrigerator operation method according to another embodiment of the present invention.

Referring to FIG. 54, when it is determined that the final photograph of the specific area inside the refrigerator has been acquired as described above (S1501), the controller 100 controls the final image obtained from the final photograph to be stored in the storage unit 18 (S1503).

Then, when the provider server such as the market server 3 receives the user input for displaying the current image of the inside of the refrigerator (S1505), the market server 3 transmits a final image request command to the refrigerator (S1507) . At this time, the market server 3 may transmit a final image request command to the refrigerator according to a predetermined period without user input.

The controller 100 causes the market server 3 to transmit the final image of the latest internal image of the refrigerator taken from the image stored in the storage unit 18 as a response to the image request command S1509).

The market server 3 receiving the final image inside the refrigerator analyzes the final image (S1511). The market server 3 analyzes the final image and finds a stored item currently stored in the corresponding refrigerator and a stored item that is not currently stored in the refrigerator. The market server 3 analyzes the stored item, which has been stored in the corresponding refrigerator in the past, It is possible to extract information necessary for service provision.

The market server 3 transmits the insufficient article list extracted through the final image analysis to the corresponding refrigerator (S1513).

55 is a ladder diagram for explaining a refrigerator operation method according to another embodiment of the present invention.

Referring to FIG. 55, when it is determined that the final photograph of the specific region inside the refrigerator has been acquired as described above (S1701), the control unit 100 controls the final image corrected from the final photograph to be stored in the storage unit 18 (S1703).

Subsequently, when a business server such as the broadcasting station server 4 receives a user input for displaying a final image in the refrigerator (S1705), the broadcasting station server 4 transmits a final image requesting request to the refrigerator (S1707) . At this time, the broadcasting station server 4 may transmit a final image requesting request to the refrigerator according to a predetermined period without user input.

The control unit 100 causes the broadcasting station server 4 to transmit the final image in the refrigerator that is most recently captured and stored among the images stored in the storage unit 18 as a response to the final image request command (S1709).

The broadcasting station server 4 receiving the final image analyzes the final image (S1711). The broadcasting station server 4 analyzes the final image, identifies the stored contents currently stored in the corresponding refrigerator, and extracts information necessary for providing the service by analyzing dishes that can be created using the stored contents.

The broadcasting station server 4 transmits recipe information and the like of the dish that can be made using the currently stored storage through the final image analysis to the corresponding refrigerator (S1713).

The control unit 100 displays the recipe information of the dish that can be made using the currently stored stored food received from the broadcasting station server 4 on the display 14 installed in the refrigerator (S1715)

According to the embodiment of the present invention as described above, there is a merit that the related business operator can accurately grasp the inside situation of the refrigerator located in the home or business and provide refrigerator related services suitable for the home or business.

FIG. 56 is a view for explaining the operation of the heater of the camera, and FIG. 57 is a view for explaining experimental results on condensation generated by temperature in a transparent window of a camera. This will be described below with reference to FIG. In Fig. 56, the heater of the type described in Figs. 14 to 17 will be described as a reference.

56 (a) is a graph in which power is intermittently supplied to the heater 84 and is intermittently operated, and FIG. 56 (b) is a graph showing a state in which the heater 84 is continuously supplied with power Graph.

FIG. 56 (a) shows a method of maintaining the temperature of the transparent window 80 at an equilibrium state such as the temperature of the storage chamber 22 without the usual control, and then applying a momentary high power at the time of necessity of taking a picture of the refrigerator. Particularly, the condenser generated in the transparent window 80 can be removed by operating the heater 84 immediately before photographing.

The door 20 is opened so that the transparent window 80 can contact the moisture contained in the warm outside air entered through the door 20. At this time, since the transparent window 80 is relatively lower in temperature than the outside air, the moisture contained in the outside air condenses in the transparent window 80, causing condensation.

If the heater 84 is momentarily driven to remove dew generated by the condensation phenomenon, there may be a problem that it takes time until the dew of the transparent window 80 is actually removed. Therefore, if the time required for photographing is faster than the time when the dew is removed, dew may be formed on the transparent window 80.

On the other hand, FIG. 56 (b) shows a method in which the heater 84 is continuously driven to supply heat to the transparent window 80. Since the temperature of the transparent window 80 is always maintained at a temperature higher than the condensation occurrence temperature, condensation is not always generated in the transparent window 80. Accordingly, even if the door 20 is opened and the transparent window 80 contacts the outside air, the moisture contained in the outside air is not dewed in the transparent window 80. Therefore, it is possible to prevent dew from being formed on the transparent window 80 at a moment when the camera 70 needs to take a picture instantaneously.

In driving the heater 84, there is no problem of image quality due to condensation, and power consumption of the refrigerator must be prevented from rapidly increasing due to power consumption of the heater 84. It is also desirable to take into consideration that the heat supplied by the heater 84 should not affect the internal temperature.

The inventors of the present invention have considered that it is necessary to operate the heater with a longer time to turn off the heater in order to minimize the power consumption of the heater and not affect the refrigerator. However, if the heater off time is set to be long, the target temperature is set in the transparent window of the camera, and the temperature of the refrigerator outside temperature is set to 32 degrees Celsius and the relative humidity is set to 85 percent (dew point temperature is about Celsius 29 degrees to 30 degrees) (see Fig. 57). In FIG. 57A, the y-axis indicates the amount of condensation generated in the transparent window, and the y-axis in 57b indicates the percentage of the condensation amount actually generated at the corresponding temperature with respect to the present steam amount.

The on / off cycle of the heaters derived from repeated experiments could deteriorate the image quality of the pictures taken by the camera under special temperature conditions in the room.

Even if an appropriate on / off cycle of the heater is found, the user's refrigerator usage pattern may vary unpredictably during the time the heater is turned off.

For example, if the user has opened the refrigerator door for a long time, the cover glass (the cover glass may mean the above-mentioned transparent window or the various types of windows that are placed in front of the camera lens A large amount of condensation is generated on the surface. In such a case, even if the cover glass is heated by operating the heater, evaporation takes a long time. E Also, evaporation of already generated dew from the cover glass consumes a lot of time as well as a lot of power.

Also, when the storage compartment is a refrigerating compartment, the internal temperature is usually maintained between 0 and 7 degrees Celsius, and the internal volume is also larger than the cover glass surface area. The heat supplied by the heater is inevitably leaked by the cold air inside the storage chamber. Therefore, a large amount of heat must be applied for a short time in order to evaporate the dew generated in the cover glass. In order to supply such heat intensively for a short period of time, a large capacity heater is used, and the instantaneous power increases greatly, so that various problems may occur in a state where power is poor.

Therefore, it is preferable that the condensation amount of the cover glass not exceeding a predetermined amount is continuously generated from the beginning. In the present invention, power is applied so that the heater is always operated without on / off cycle.

The method of applying power to the heater at all times may be such that the power is applied to the camera heater immediately when power is applied to the refrigerator, or the power is always applied to the heater through the refrigerator control unit during normal operation have.

Next, the heating temperature of the cover glass will be described. As a result of the quality evaluation based on the image sample obtained by the temperature gauge, it was confirmed that there is no big problem in the quality of the sample image until the temperature of the center portion of the cover glass is about ½ of the temperature of the dew point. The fact that there is no problem in the image quality means that it is possible to grasp the kinds and quantity of the articles stored in the storage room through the photographs taken.

Especially, the quality of the image samples obtained at the third point (13.5 °) in FIG. 57B was applicable. Of course, it is possible to maintain the heating temperature of the cover glass at a temperature higher than the dew point, but it is possible to supply power to the heater so that the heating temperature of the cover glass is maintained at about 13 degrees to about 30 degrees.

On the other hand, the experimental results show that the heating temperature of the cover glass differs according to the outer size of the cover glass under the condition of the same applied power, and there is no significant difference in the thickness of the cover glass. Therefore, since the image quality is satisfied at a half of the dew point temperature in the same cover glass size, it is preferable to select the lower limit as the upper limit point in terms of the power consumption (applied power) of the heater.

The applied electric power in the vicinity of the dew point temperature (29.2 degrees) when heating the cover glass was 0.45 W, and the applied electric power at the level of about 0.58 times (17.1 degrees Celsius) of the dew point temperature was 0.27 W. Therefore, since the consumed power is about 0.5 times in the range of 0.5 to 0.7 of the dew point temperature, the power consumption can be greatly reduced.

The mid-band of the dew point temperature can be determined at approximately 12 to 17 degrees Celsius under ambient conditions that can normally vary. Accordingly, it is preferable that a current value is applied to the heater to maintain the temperature of the center portion of the transparent window at about 12 to 17 degrees Celsius.

That is, in the present invention, the temperature of the cover glass is maintained at a temperature lower than the dew point, not the dew point, so that the power consumed by the heater can be reduced.

58 is a sectional view of the transparent window. This will be described below with reference to FIG.

The transparent window 80 may be hydrophilic coated. At this time, the transparent window 80 may mean the cover 90 described above.

Even if dew is formed instantaneously in the transparent window 80, the contact angle? Of water can be formed within 15 degrees due to the hydrophilic coating as shown in Fig. The hydrophilic coating reduces the surface tension of water on the surface. This is because the water can be spread widely on the surface of the transparent window 80. Thus, the camera can be fixed to the ceiling of the storage room so that the angle between the transparent window and the ceiling surface of the storage room is between about 10 and 20 degrees.

Therefore, it is possible to minimize the distortion of the photographed image due to the water formed in the transparent window 80. In addition, the hydrophilic coating can be advantageous in terms of energy efficiency since it does not need to be controlled after supplying the electric power once after it is manufactured.

59 and 60 are views showing a structure in which a camera is installed in an inner case. 59A and 60A are views of the side of the storage compartment, and FIGS. 59B and 60B are views looking up the upper side of the inner case. This will be described below with reference to Fig. 59 and Fig. Through these embodiments, it is possible to bypass the path of the outside air to the cover 90 to minimize the occurrence of condensation in the cover 90.

The camera 70 may be installed on the protrusion 500 protruding downward from the ceiling of the inner case 12. [ The protrusion 500 may protrude downward as compared with other portions of the inner case 12. In this case, These protrusions 500 may be implemented through the camera housings described above.

At this time, as shown in FIG. 59, the camera 70 is installed in the protruding portion 500 so as to face the inner wall of the storage compartment 22. The camera 70 can take a picture of the inside of the storage room 22 through the cover 90. Accordingly, the cover 90 is disposed at a position facing the rear wall of the storage compartment 22 from the protrusion 500.

At this time, the moisture contained in the outside air is contacted with the cover 90, and the dew is generated.

However, since the cover 90 is disposed to face the rear wall of the storage compartment 22, the path through which the outside air reaches the cover 90 can be long.

59b in order to allow the outside air passing through the door 20 to pass through the storage compartment 22 and contact the cover 90. At this time, the outside air comes into contact with the cool air of the storage compartment 22 Some of the contained water may condense in the other part. Therefore, the outside air can reach the cover 90 while gradually decreasing the amount of moisture in the storage chamber 22.

That is, even if the door 20 is open, the inside of the storage compartment 22 is kept at a lower temperature than the outside of the storage compartment 22, and immediately the temperature does not change to the same temperature as the outside air. .

In other words, if the cover 90 is disposed at a position where the cover 90 is relatively difficult to contact with the outside air, the amount of dew generated in the cover 90 may be reduced, and the point at which dew is generated may be delayed.

First, the outside air comes into contact with the front face 501 of the protrusion 500 while entering the inside. At this time, the outside air in contact with the front face 501 enters the inside of the room with the path changing to the left and right of the protrusion. This outside air can reach the cover 90 while touching the side surface 502 of the protrusion 500. Therefore, a large amount of moisture can be condensed before reaching the cover 90. [

The outside air in contact with the front face 501 of the protruding portion 500 can be lowered and raised by the bottom face 503 of the protruding portion. Accordingly, a large amount of water can be condensed while moving to the front surface 501 and the bottom surface 503.

In addition, the cover 90 is mounted obliquely in a form surrounded by a front surface 501, a side surface 502 and a bottom surface 503. Therefore, the path of the outside air and the cover 90 are not perpendicular to each other. Therefore, it is also possible to prevent the outside air from reaching the cover 90 and rapidly condensing.

In particular, the width of the protrusion 500, in particular, the lateral width of the front surface 5010, can be wider than the width of the camera 70.

The embodiment shown in Fig. 60 is very similar to the embodiment shown in Fig. However, it can be said that the camera is mounted so that it looks at the vertical bottom. Therefore, the same effect as the example shown in Fig. 59 can be obtained.

Also, the protrusion 500 may include a rear surface 504. It is preferable that the vertical length of the front face 501 is larger than the vertical length of the rear face 504. [ That is, it is possible that the front face 501 is further protruded downward from the inner case 12.

Therefore, in order to allow the outside air to reach the cover 90, the path is longer due to the width of the protrusion 500 in the horizontal direction, and the path is longer due to the height difference of the protrusion 500 in the vertical direction .

The prevention of condensation in the cover 90 through the structure of the protrusion 500 or the structure for mounting the camera is highly related to the installation position of the camera. That is, since the camera can be installed in an angle range inclined by a predetermined angle backward from the vertical in the vertical lower part in the ceiling of the storage room. Therefore, it is possible to effectively circumvent the inflow path of the outside air while effectively photographing the entire area in the storage room in the storage room.

In the prior art, there has been disclosed a technique in which a camera is mounted on a side wall of a main body storage compartment, an upper outer side of a main body, a door or an upper portion of a door.

In the prior art, when the camera is mounted on the side wall of the storage room, there is provided a depression for accommodating the camera on the side wall, and a camera is mounted on the depression. That is, the camera is prevented from protruding into the inside of the storage room. It is presumed that this is a configuration for avoiding inconvenience to the user when the shelf or the article of the storage room is taken out.

However, in the present invention, the camera is installed on the ceiling of the inner case 12 so that the user is not disturbed when the user uses the storage room 22. This is because there is no reason for the camera to collide with the camera because the camera is not positioned on the movement path of the article or shelf when the user desires to withdraw the article from the storage room.

Further, in the present invention, the cover 90 of the camera 70 is arranged to look backward, so that the path of the outside air reaching the cover 90 can be extended.

59A, the cover 90 is arranged so as to face rearward, and the protrusion 500 protrudes downward, so that the outside air has a three-dimensional movement path for reaching the cover 90, The path becomes longer.

60A, the cover 90 is disposed so as to face downward, but a front face 504 horizontally disposed in the opening of the protrusion 500 is formed on the front face 504 at both ends of the front face 504, And a side surface 508 disposed vertically with respect thereto. At this time, the front surface 504 and the side surface 508 are provided so as to protrude downward from the protruding portion 500, so that the path of the outside air reaching the cover 90 can be made long.

That is, the high-temperature air flowing from the outside into the storage compartment opening reaches the cover 90 of the camera straightly and does not come into contact with the cover 90, and bypasses left and right along the protrusion 500 and arrives. Therefore, the outside air is heat-exchanged with the cold air in the hearth, and as the temperature is lowered, the amount of saturated water vapor becomes smaller, so that the water vapor contained therein is condensed and mixed with the hot air. When the cover 90 reaches the cover 90, the cover 90 can be sufficiently evaporated by the heater that is heating the cover 90 because of the reduced amount of water vapor.

Accordingly, the power to be supplied to the heater 84 is reduced, and the power consumption consumed by the entire refrigerator is improved.

In the embodiment shown in FIG. 59 or 60, a cabinet has a storage compartment formed therein; Wherein the storage compartment comprises an access opening at the front and a shelf area consisting of an upper wall, a lower wall, both side walls, and a rear wall made of a heat insulating material and being partitioned by a plurality of shelves, and a drawer having at least one drawer Comprising a region; At least one door which is in close contact with the front surface of the storage compartment and opens and closes the storage compartment; A camera device installed in the upper chamber wall between an opening of the storage chamber and a front edge of the shelf installed in the storage chamber;

The camera device includes a camera module part having a camera window formed with a predetermined distance from the lens and accommodating an electrical part necessary for driving a camera including a camera lens, And a camera housing part which seats and fixes the part at a predetermined position inside thereof;

The camera housing part includes a fixing surface in contact with the upper wall of the storage compartment, a front surface formed on the opening side of the storage compartment, side surfaces connecting the front and rear surfaces, And a top surface on which an opening is formed to expose a camera window of the module unit, wherein a front surface of the camera housing unit is formed higher than an opening formed on an upper surface of the camera housing unit .

The upper surface of the camera housing part may be inclined at a predetermined angle toward the opposite direction of the opening of the storage room with respect to the upper wall surface of the storage room.

The camera housing part may further include a rear surface on a rear wall side of the storage room opposite to the front surface, and the height of the front surface may be higher than the height of the rear surface.

The opening of the upper surface of the camera housing part may be formed at a position lower than the side surface of the camera housing part.

And a housing installation part having a recessed portion corresponding to a recessed space of the upper wall of the storage chamber.

The housing mounting part may be mounted on the upper wall of the storage room together with the refrigerator insulator filling, and the camera housing part may be fixed to the housing mounting part.

The camera module part fixed to the camera housing part can be received in a depression formed in the housing mounting part.

The camera module part accommodated in the depression of the housing mounting part may be fixed to the camera housing part so as to be spaced from the inner surface of the housing mounting part.

The front and both side surfaces of the camera housing part may be formed higher than the opening position of the upper surface of the camera housing.

The upper surface of the camera housing part may be formed so that the opening side of the storage chamber is further away from the storage chamber upper wall surface than the rear wall side of the storage chamber.

The camera module unit has a front portion and a rear portion opposite to each other, in which a camera window is formed, and at least one flat surface is formed on at least one of the front portion and the rear portion .

Wherein the camera housing part is formed with a receiving part for receiving the camera module, and the receiving part is provided with a seat which is inclined at a predetermined angle with respect to the upper wall surface of the storage room, And a seating part.

The depressed portion of the housing mounting portion may include a seating portion that is inclined at a predetermined angle with respect to the upper wall surface of the storage compartment to place a flat surface formed on the camera module portion.

A flat surface may be formed around a camera window of a front portion of the camera module unit and the flat surface may be formed to be approximately equal to a height of a center portion of the camera lens.

61 is a view showing another type of refrigerator to which the present invention is applicable. This will be described below with reference to FIG.

The refrigerator of FIG. 61 is a side-view type refrigerator, and the left and right sides are different storage rooms. That is, the storage room on the left side is a freezing room, the storage room on the right side is a freezing room, and the storage room on the left side is a freezing room.

A camera (70) may be installed on the ceiling of the storage room (22) so as to face the lower part of the storage room (22).

In addition, the storage room 22 may be provided with a drawer 50 capable of storing articles and capable of drawing out the articles.

Other contents are omitted because they are the same as those of the refrigerator described above.

62 is a screen provided to the user from the refrigerator according to FIG. This will be described below with reference to Fig.

The user may be provided with an image of the article stored in the drawers 50 and 51 and an image of the article stored on the shelf 40. [ In the refrigerator, one drawer 50 may be provided. Thus, an image for one drawer can also be provided in Fig. However, two drawers 50 and 51 may be provided above and below the baake kx shown in FIG. In this case, it is also possible that images relating to the interior of each drawer are provided.

In the example described with reference to FIG. 2, the drawers are provided on the left and right sides, respectively. However, in FIG. 62, the upper and lower drawers may be respectively provided. Accordingly, the draw area may be two, and the lower drawer 50 may protrude forward as compared with the upper drawer 51. [ In this case, the markers may be similarly provided in each of the drawers 50 and 51.

Thus, food information stored in a plurality of shelf areas and a plurality of drawer areas can be easily grasped through a single camera.

And a method of upgrading the image provided on the user screen are the same as those of the refrigerator described above, and thus the description thereof will be omitted.

FIG. 63 is a view for explaining a method of adjusting a picture photographed by a camera in a refrigerator according to FIG. 61; FIG. This will be described below with reference to FIG.

Since one drawer is usually provided or each of the drawers is vertically disposed, one horizontal adjustment line 15 may be provided in the refrigerator of the type shown in FIG. Of course, two horizontal adjustment lines may be provided to distinguish between the two drawers. It is also possible to arrange two vertical adjustment lines arranged in parallel to adjust the position of the adjustment line. Of course, unlike the one shown in FIG. 63, it is also possible to select a desired image using only the horizontal adjustment line without using the vertical adjustment line.

The user may be provided with a limit line 16 indicating a limit range for moving the adjustment line 15, wherein the limit line 16 can be arranged in parallel with a predetermined interval.

The correction of the error through the adjustment line 15 can be said to be the same as or similar to the above embodiment.

It is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

10: outer case 12: inner case
20: Door 40: Shelf
42: first region 50: drawer
52: second area 60: barrier
62: third area 70: camera
100:

Claims (33)

A cabinet in which a cool air supply unit for maintaining low temperature is accommodated, an opening is formed in a front surface of the cooler,
A door rotatably mounted in the cabinet to open and close the opening;
A storage compartment partitioned by the door and the cabinet, the storage compartment being formed as one independent cold storage space for storing food therein; And
And a camera unit installed in the storage room to capture an image of food stored in the storage room,
The camera unit includes:
A housing fixed to the storage chamber,
A camera which is seated on the housing and is kept in the photographing direction,
A transparent window provided on a front surface of the housing,
And a heater provided inside the transparent window,
Wherein the heater is continuously supplied with power and continues to operate. The method according to claim 1,
Wherein when electric power is applied to the refrigerator, current is supplied to the heater, and the heater heats the transparent window. The method according to claim 1,
Wherein the heater heats the transparent window while the storage chamber is lower than an external temperature. The method according to claim 1,
Wherein the heater heats the transparent window regardless of opening / closing of the door. The method according to claim 1,
Wherein the heater continuously supplies heat to the transparent window without interruption. The method according to claim 1,
Wherein the heater heats the transparent window by receiving a predetermined electric power. The method according to claim 1,
Wherein the heater continuously discharges heat at a temperature higher than a temperature of the storage compartment. The method according to claim 1,
Wherein a temperature of the transparent window is distributed in a range higher than a set temperature of the storage room. The method according to claim 1,
Wherein the heater supplies heat so that the transparent window is maintained at a dew condensation temperature or more. 10. The method of claim 9,
Wherein the condensation occurrence temperature is a temperature at which condensation may occur in the transparent window when the door is opened in the storage compartment. The method according to claim 1,
Wherein the temperature of the heater is maintained higher than the temperature of the transparent window. The method according to claim 1,
Wherein the temperature of the transparent window is lower than the temperature of the heater and is maintained higher than the condensation occurrence temperature. delete The method according to claim 1,
Wherein the heater supplies heat so as to be maintained at half or more of the condensation occurrence temperature of the transparent window. The method according to claim 1,
Wherein the heater supplies heat to maintain the transparent window at 12 to 17 degrees Celsius. 16. The method of claim 15,
Wherein the temperature of the central portion of the transparent window is maintained at 12 to 17 degrees Celsius. The method according to claim 1,
Wherein the heater supplies heat to the transparent window at a temperature lower than a condensation occurrence temperature of the transparent window. 18. The method of claim 17,
Wherein the condensation occurrence temperature is a temperature at which condensation may occur in the transparent window when the door is opened in the storage compartment. The method according to claim 1,
Wherein the heater is installed on a surface of the transparent window facing the lens of the camera. The method according to claim 1,
Wherein the heater is sealed from the storage room by the housing and the transparent window. The method according to claim 1,
Wherein the camera is enclosed by the housing and the transparent window and is sealed from the storage compartment. The method according to claim 1,
The housing includes:
A first housing, a second housing, and a sealing member disposed between the first housing and the second housing. The method according to claim 1,
Wherein the transparent window is made of glass. The method according to claim 1,
And the lens of the camera is spaced apart from the transparent window. The method according to claim 1,
Wherein the heater is spaced apart from the lens of the camera. The method according to claim 1,
Wherein the heater is disposed such that a portion of the heater contacting the transparent window has a rectangular shape. The method according to claim 1,
Wherein the heater is not disposed at the center of the transparent window but is disposed along the periphery of the transparent window. The method according to claim 1,
Wherein the transparent window has a rectangular shape. The method according to claim 1,
Wherein the transparent window has a length in a vertical direction and a length in a horizontal direction. The method according to claim 1,
Wherein the camera unit is disposed to protrude from the storage compartment so as to view a direction in which the camera lens is to be photographed. The method according to claim 1,
Wherein the transparent window is hydrophilic coated. 32. The method of claim 31,
Wherein the contact angle of water in the transparent window is within 15 degrees. delete

Images