US20160201981A1

US20160201981A1 - Refrigeration appliance with a camera module

Info

Publication number: US20160201981A1
Application number: US14/913,390
Authority: US
Inventors: Stephan Kempfle; Gerhard Wetzl
Original assignee: BSH Bosch und Siemens Hausgeraete GmbH
Current assignee: BSH Hausgeraete GmbH
Priority date: 2013-08-23
Filing date: 2014-07-25
Publication date: 2016-07-14
Also published as: EP3036490A1; CN105612397B; WO2015024735A1; DE102013216764B3; CN105612397A

Abstract

A refrigeration appliance has a movably mounted camera module that contains an image sensor with a matrix of light-sensitive elements. The image sensor is oriented so that the shot is taken by column-by-column scanning of the light-sensitive elements in a matrix perpendicularly to a direction of movement of the camera module.

Description

The present invention relates to a refrigeration appliance with a movably mounted camera module.
The movement of movable, digital camera modules produces image errors which have an adverse effect on the quality of a recorded image.
The phenomenon of what is known as the rolling shutter effect occurs particularly when photographing movable objects using a camera module that records line by line or column by column. The individual pixels of a matrix of light-sensitive elements are exposed one after the other in time within a short period with camera modules that record line by line or column by column. The resulting time delay when exposing the pixels of a line or column one after the other means that a moving object is mapped with its instantaneous location in the line or column exposed at one time point. With a line by line or column by column recording however the moving object has already moved from one position to the next position when the next line is scanned so that when all the lines or columns are put together a moving object cannot be mapped once as a whole but line by line in different positions at different time points. As the occurrence of the rolling shutter effect is simply the result of the relative movement between the object to be exposed and the camera module, the phenomenon also occurs when photographing a non-moving object using a moving camera module.
It is the object of the invention to specify a refrigeration appliance with a movable camera module, which avoids image errors.
This object is achieved by the subject matter having the features as claimed in the independent claim. Advantageous embodiments of the invention are set out in the figures, the description and the dependent claims.
According to one aspect of the invention the object is achieved by a refrigeration appliance with a movably mounted camera module that comprises an image sensor with a matrix of light-sensitive elements, the image sensor being oriented so that the recording is captured by column by column scanning of the light-sensitive elements in the matrix perpendicular to the direction of movement of the camera module. The matrix of the light-sensitive elements forms an m×n matrix, preferably an n×n matrix, m and n each being natural numbers.
The direction of movement of a refrigeration appliance door when closing follows a circular path due to a hinge between the refrigeration appliance door and the housing of the refrigeration appliance. When the light-sensitive elements in the matrix are scanned column by column perpendicular to the direction of movement of the camera module along the circular path, the position of the imaged interior of the refrigeration appliance remains roughly the same with the relative movement between camera module and the contents of the refrigeration appliance. The contents of the refrigeration appliance can thus be captured in an image as the refrigeration appliance is closed without the rolling shutter effect phenomenon manifesting itself. The photographic recording does not have to be post-processed using software. Nor is it necessary to detect the closing speed of the refrigeration appliance door. When shopping later for example it is possible to determine the contents of the refrigeration appliance by means of the image recording. It is therefore possible to record the contents of the refrigeration appliance with a moving camera module without the rolling shutter effect occurring in the recording.
A refrigeration appliance refers in particular to a domestic refrigeration appliance, in other words a refrigeration appliance used for domestic management in a domestic context or in catering, which serves in particular to store food and/or beverages at specified temperatures, for example a refrigerator, upright freezer, combined refrigerator/freezer, chest freezer or wine chiller cabinet.
In one advantageous embodiment the camera module comprises a trigger unit for activating the image sensor by means of a trigger signal. The trigger unit can switch the image sensor to trigger mode, in which the light-sensitive elements are activated at predefined times. This has the technical advantage for example that time-delayed, triggered activation of the image sensor allows the light-sensitive elements to be exposed simultaneously at a predefined time.
In a further advantageous embodiment the camera module is configured to activate a column of the light-sensitive elements for a predefined recording time on receipt of the trigger signal. Activation of the light-sensitive elements for a predefined recording time preferably lasts around a few seconds to a few minutes. It is particularly preferable to activate the light-sensitive elements for a predefined recording time in the range from 5 seconds to 3 minutes. A refrigeration appliance is generally in a state with the refrigeration appliance door open for this time range for goods to be removed or for the refrigeration appliance to be filled. This has the technical advantage for example that the goods in the interior of the refrigeration appliance are exposed regardless of the length of time for which the refrigeration appliance is open.
In one advantageous embodiment the camera module is configured to activate a first column of the light-sensitive elements for a predefined recording time at a first time point and to activate a second column of the light-sensitive elements for the predefined recording time at a second time point. The camera module is preferably configured to activate the nth line for a predefined recording time at an nth time point. At the nth time point n columns of the matrix of light-sensitive elements are activated simultaneously. This has the technical advantage for example that the photographic recording of the goods to be exposed in the interior of the refrigeration appliance can be captured by a number of columns of the light-sensitive elements.
In a further advantageous embodiment the camera module is configured to activate the columns of the light-sensitive elements sequentially with a temporal offset for the predefined recording time in each instance. The camera module activates successive columns one after the other so that all the columns of the matrix are activated for a further predefined recording time and capture the recording of the goods to be exposed in the interior of the refrigeration appliance. This has the technical advantage for example that all the columns of the light-sensitive elements are available for a common time period to capture the goods to be exposed.
In one advantageous embodiment the camera module is configured to reset the columns of light-sensitive elements column by column before activation. The resetting of the light-sensitive elements before activation corresponds to ensuring that no previously captured image information remains in the light-sensitive elements with a new photographic recording being superimposed. This has the technical advantage for example that a captured photographic recording corresponds to the actual contents of the refrigeration appliance at one time point.
In a further advantageous embodiment the camera module is configured to read the columns of light-sensitive elements column by column after activation. Column by column reading allows the captured image information to be read in columns and lines, in particular in the form of a matrix structure. It is thus possible to locate an over or under-exposed or defective light-sensitive element. This has the technical advantage for example that the read image information can be assigned to the individual light-sensitive elements.
In one advantageous embodiment the camera module comprises a processing facility for the digital processing of the image sensor recording. The processing facility puts the read columns together to form a photographic recording, which shows the contents of the refrigeration appliance when it is closed. The image sensor recording is displayed on a monitor, which is arranged for example in the outer region of the refrigeration appliance door. It is also possible to forward the recording to an addressee by way of the Internet or a mobile radio device. This has the technical advantage for example that an in situ recording of the contents of the refrigeration appliance can be seen by an addressee from outside or remotely at any time.
In a further advantageous embodiment the camera module comprises a storage unit for storing the recording digitally. The recordings captured one after the other in time can be stored and document the consumption over time of the goods stored in the refrigeration appliance. This has the technical advantage for example of facilitating domestic management relating to the consumption of foods.
In one advantageous embodiment the camera module comprises a flash facility for exposing the recording. When a refrigeration appliance is filled with goods at one time the illumination in the interior of the refrigeration appliance may be covered by the goods with the result that not enough light reaches the image sensor for exposure. The flash facility ensures that there is a sufficient quantity of light for the recording. This has the technical advantage for example that a recording can be performed when the refrigeration appliance is closed regardless of the quantity of goods contained therein.
In a further advantageous embodiment the camera module is configured to activate the flash facility at a time point when all the columns of light-sensitive elements are activated. The flash facility exposes all the columns with the same quantity of light, so that comparable contrasts are captured in the individual columns. This has the technical advantage for example that the composite recording shows only slightly different contrasts from column to column.
In one advantageous embodiment the movably mounted camera module is arranged in a refrigeration appliance door. A number of camera modules are preferably arranged in the refrigeration appliance door. For example a camera module is assigned to each chiller compartment in the refrigeration appliance. By arranging the camera module in the refrigeration appliance door it is possible to protect the camera module from fluids escaping from the goods. This has the technical advantage for example that the camera module has access to the goods stored in the refrigeration appliance and cannot be covered by individual goods. The camera module is also protected from penetrating fluids.
In a further advantageous embodiment the movably mounted camera module is arranged on a side of the refrigeration appliance door opposite a stop side of the refrigeration appliance door. The refrigeration appliance door is connected to the housing of the refrigeration appliance by way of a hinge at one side. The side of the housing opposite the hinge forms the stop side, against which the side of the refrigeration appliance door facing the interior of the refrigeration appliance comes to rest when the refrigeration appliance is closed. This side of the refrigeration appliance door is referred to in the following as the inside. The camera module is preferably arranged in a region of the inside which extends from the center to the outer edge of the refrigeration appliance door opposite the hinge. It is particularly preferable for the camera module to be arranged further from the center than from the outer edge. The position of the arrangement of the camera module in the refrigeration appliance door determines the image region for recordings by the image sensor in the camera module. This has the technical advantage for example that a predetermined image region can be captured by the image sensor in the interior of the refrigeration appliance.
In one advantageous embodiment the image sensor is configured as a CMOS sensor. CMOS sensors have light-sensitive elements arranged in a matrix and serving to capture a two-dimensional image with little energy outlay. CMOS sensors are produced miniaturized and can therefore be used as a camera module in a device. This has the technical advantage for example that a camera module can be integrated in a refrigeration appliance without taking up much space.
In a further advantageous embodiment the CMOS sensor comprises a color filter. The color filter preferably has filters for the colors red, green and blue. In particular the color filter can be configured as a Bayer filter. The color filter can also be configured as a different color filter or can contain white pixels for example. The color filter allows a photographic recording that reproduces the perception of the human eye. This has the technical advantage for example that the contents of the refrigeration appliance can be captured in color.

Exemplary embodiments of the invention are illustrated in the drawing(s) and are described in more detail below. In the drawing(s):

FIG. 1 shows a schematic view of a refrigeration appliance;

FIG. 2 shows a schematic diagram of plan views of a refrigeration appliance with the refrigeration appliance door closed and open;

FIG. 3 shows a schematic diagram of a plan view of a refrigeration appliance during closing with a camera module mounted in the refrigeration appliance door;

FIG. 4a shows the recording of contents of the refrigeration appliance with line by line capturing of the light-sensitive elements;

FIG. 4b shows the recording of contents of the refrigeration appliance with column by column capturing of the light-sensitive elements; and

FIG. 5 shows a schematic diagram of the mode of operation of the image sensor in trigger mode with a flash capturing all the light-sensitive elements.

FIG. 1 shows a refrigeration appliance 100 with two refrigeration appliance doors 111. The upper refrigeration appliance door 111 closes off the interior of a chiller compartment (not shown) and the lower refrigeration appliance door 111 closes off a freezer compartment (not shown). The refrigeration appliance door 111 here adjoins the housing 123 of the refrigeration appliance 100.
FIG. 2 shows a plan view of the refrigeration appliance 100 from above with the refrigeration appliance door 111 closed and open. A camera module 105 is arranged in the refrigeration appliance door 111. When the refrigeration appliance door 111 is closed the direction of movement 109 follows a circular path. The refrigeration appliance door 111 is connected to the housing 123 of the refrigeration appliance 100 by way of a hinge (not shown). The camera module 105 is arranged perpendicular to the direction of movement 109. As a result while the refrigeration appliance door 111 is closing the image sensor is directed or focused onto an image region 125 as shown in FIG. 3.
FIG. 3 illustrates schematically the direction of movement 109 along a circular path, the direction of movement 109 following an x axis. A putative x-y-z coordinates system is rotated according to the progress of the circular path at its x-y axes when the refrigeration appliance door is closed. The image sensor 107 of the camera module 105 is arranged perpendicular to the direction of movement 109 and faces in a horizontal direction. The y axis runs in a horizontal direction while the z axis (not shown) runs in a vertical direction. The arrangement of the camera module 105 perpendicular to the direction of movement 109 results in an image region 125 captured by the camera module.
FIG. 4a shows a recording of a pot 127 captured line by line as the refrigeration appliance door 111 of the refrigeration appliance 100, as shown in FIG. 1, closes, while FIG. 4b shows a recording captured column by column. When capturing takes place line by line as shown in FIG. 4a the rolling shutter effect phenomenon occurs but it does not occur when capturing takes place column by column as in FIG. 4b . Processing of the read columns using software is not necessary due to the arrangement of the camera module 105 perpendicular to the direction of movement 109, as shown in FIGS. 2 and 3.
FIG. 5 illustrates schematically the progress in a camera module 105 when the refrigeration appliance door 111 closes. At a first time point t1 a first column of the camera module is reset 130 and the trigger unit 113 activates a trigger signal at the image sensor 107 of the camera module 105. At the first time point t1 a first column of the camera module 105 is activated. At a second time point t2 a second column of the camera module 105 is reset and then activated. The resetting 130 and activation of the columns is continued until the nth column is activated at an nth time point. After the nth column has been reset and then activated, a flash 138 is activated by the flash facility 119 for the time of a flash duration TB. The flash 138 here extends over n activated, for example all, columns. The exposure 132 of the columns extends over an exposure time T, which commences with the start of the flash 138. The exposure time T extends over a longer period than the flash duration TB. At the end of the flash duration TB there is provision for a charge transfer 136 at the first column of the camera module 105, followed by a reading 134 of the first column. A charge transfer 136 and a reading 134 of the respective column are then performed one after the other in time at the second column until the nth column is activated. When the nth column is activated, the exposure time T ends, with the charge transfer 136 and reading 134 at the nth column being performed thereafter.
All the features described and illustrated in conjunction with individual embodiments of the invention can be provided in different combinations in the inventive subject matter in order to achieve their advantageous effects simultaneously.
The scope of protection of the present invention is defined by the claims and is not restricted by the features described in the description or illustrated in the figures.

LIST OF REFERENCE CHARACTERS

100 Refrigeration appliance
107 Image sensor
105 Camera module
109 Direction of movement
111 Refrigeration appliance door
113 Trigger unit
115 Processing facility
119 Flash facility
120 Stop side
125 Image region
127 Pot
129 Read unit
130 Resetting
132 Exposure
134 Reading
136 Charge transfer
138 Flash
T Recording time
t1 First time point
t2 Second time point
TB Flash duration

Claims

1-15. (canceled)

16. A refrigeration appliance, comprising:

a movably mounted camera module having an image sensor with a matrix of light-sensitive elements, said image sensor oriented so that a recording is captured by column-by-column scanning of the light-sensitive elements in a matrix perpendicular to a direction of movement of said camera module.

17. The refrigeration appliance according to claim 16, wherein said camera module has a trigger unit for activating said image sensor by means of a trigger signal.

18. The refrigeration appliance according to claim 17, wherein said camera module is configured to activate a column of the light-sensitive elements for a predefined recording time on receipt of the trigger signal.

19. The refrigeration appliance according to claim 17, wherein said camera module is configured to activate a first column of the light-sensitive elements for the predefined recording time at a first time point and to activate a second column of the light-sensitive elements for the predefined recording time at a second time point.

20. The refrigeration appliance according to claim 17, wherein said camera module is configured to activate columns of the light-sensitive elements sequentially with a temporal offset for the predefined recording time in each instance.

21. The refrigeration appliance according to claim 16, wherein said camera module is configured to reset columns of the light-sensitive elements column-by-column before activation.

22. The refrigeration appliance according to claim 16, wherein said camera module is configured to read columns of the light-sensitive elements column-by-column after activation.

23. The refrigeration appliance according to claim 16, wherein said camera module contains a processing facility for digital processing of the recording of said image sensor.

24. The refrigeration appliance according to claim 16, wherein said camera module contains a storage unit for storing the recording digitally.

25. The refrigeration appliance according to claim 16, wherein said camera module contains a flash facility for exposing the recording.

26. The refrigeration appliance according to claim 25, wherein said camera module is configured to activate said flash facility at a time point when all columns of the light-sensitive elements are activated.

27. The refrigeration appliance according to claim 16, further comprising a refrigeration appliance door, said movably mounted camera module is disposed in said refrigeration appliance door.

28. The refrigeration appliance according to claim 27, wherein said movably mounted camera module is disposed on a side of said refrigeration appliance door opposite a stop side of said refrigeration appliance door.

29. The refrigeration appliance according to claim 16, wherein said image sensor is a CMOS sensor.

30. The refrigeration appliance according to claim 29, wherein said CMOS sensor has a color filter.