DE102015213779A1 - Photographing method, camera and household refrigerator - Google Patents

Abstract

The method is used for image recording, wherein in the process, an image sensor (5) is fully sensitive, a shutter (9) is opened, then the shutter (9) is closed and the image sensor (5) read line by line with closed shutter (9) becomes. A camera (1) with an image-wise reading image sensor (5) and a shutter (9) is designed to carry out the method. A household refrigerating appliance (K) has at least one usable space (N) which can be closed by a door (T), a camera (1) arranged in the door (T) and a triggering device (W) which is adapted to direct the camera (1) Reaching at least one predetermined angular position (φ) of the door (T) to trigger. The invention is particularly advantageously applicable to image recordings of a useful space of a household appliance, in particular a refrigerator (CiF, "Camera-in-Fridge"). The invention is particularly advantageous applicable to built in a door of the refrigerator cameras.

Description

The invention relates to a method for image recording. The invention also relates to a camera with a rolling shutter image sensor and a shutter. The invention is particularly advantageously applicable to image recordings of an interior of a household appliance, in particular a refrigerator (CiF, "Camera-in-Fridge"). The invention is particularly advantageous applicable to built in a door of the refrigerator cameras.

An image acquisition of an image sensor with several (eg 640 × 480) sensor cells or sensor elements is typically divided into four phases: (a) deletion ("reset") of the sensor elements, (b) exposure of the sensor elements, (c) charge transfer on the image sensor and ( d) line by line time-shifted reading of the data. During reading, the sensor elements must not be exposed any further. Electronic shutter methods are used. Depending on the type of sensor, either the so-called "rolling shutter" method or the so-called "global shutter" method is used.

With the global shutter method, the deletion and the subsequent exposure or sensitization of all sensor elements takes place simultaneously. At the end of the exposure, a charge transfer is performed simultaneously for all sensor elements, by means of which the exposure for all sensor elements is terminated simultaneously. The readout of the individual rows of sensor elements is then carried out separately, e.g. consecutively line by line. The simultaneous exposure of all sensor elements and thus also of all lines has the advantage that the image of a moving object is reproduced without distortion. However, image sensors using the global shutter method are more complex than rolling shutter image sensors.

With the rolling shutter method, the erasing, the exposure, the charge transfer and the readout of the image sensor are usually slightly offset line by line. Since this results in a time offset between the exposure of successive sensor lines, the image of a moving object is displayed distorted.

Some rolling shutter image sensors can be operated in a so-called "global start mode" in which the deletion and the subsequent exposure of all sensor elements and thus also all sensor lines is started simultaneously, while the charge transfer and the readout continues offset in lines , Therefore, there is an extended exposure time from the read-out line to the next read-out line. In particular, in order to avoid an uneven distribution of the image brightness, but possibly also an image distortion, with the beginning of the exposure, which is the same for all lines, a flash is used which does not take longer than the exposure time of the first read line. Outside the flash duration, however, no light should reach the image sensor. As a result, the global start mode is particularly suitable for taking pictures in the dark (for example, night scenes) using a flash.

In particular, low-cost cameras with a CMOS image sensor usually use the rolling shutter method.

This is also the case, for example, with cameras which are installed in a refrigerator and generally record an image of the interior of the refrigerator during each opening and / or closing operation of the door, e.g. of food or other items currently in the refrigerator. The interior may be the useful space of the refrigerator and / or an inside of the door. Due to the typically high angular velocity when the door is opened, this leads to an image distortion which makes it possible to use the recorded image, e.g. considerably more difficult for an object recognition.

It is the object of the present invention to overcome the disadvantages of the prior art, at least in part, and in particular to provide a cheaper possibility for sharply recording moving objects by means of a rolling shutter image sensor, in particular for refrigerators, in particular refrigerators, in particular domestic refrigerators ,

This object is achieved according to the features of the independent claims. Preferred embodiments are in particular the dependent claims.

The object is achieved by a method for image recording in which a rolling shutter image sensor is switched completely sensitive, a shutter is opened, then the shutter is closed and the image sensor is read line by line with the shutter closed.

This method has the advantage that all the individual sensor elements or sensor cells of the image sensor are sensitive or can be exposed simultaneously for the absorption of light and, by closing the shutter, all the individual sensor elements receive or expose light for the same period of time can be. When the shutter is closed, a charge transfer from the sensor elements can be carried out line by line with a time delay and can then be read out line by line with a time shift without causing a positional error or an uneven distribution of image brightness. Thus, at a low exposure time, a sharp image can be obtained even with an inexpensive image sensor using the rolling shutter method and with a cheap shutter.

The fact that the image sensor is a rolling shutter image sensor comprises, in particular, that it operates or can be operated according to the rolling shutter method. In the case of the rolling shutter image sensor, in contrast to a global shutter image sensor, the charge transfer is performed by the sensor elements in a time-delayed manner.

In particular, the fact that the rolling shutter image sensor is switched completely sensitive means that all the sensor elements are switched at the same time sensitive. It is basically possible, the sensor elements with a time delay -. Line by line - sensitive, as long as this sensitive switching is completed only at a given time (for example, before or with the opening of the shutter) and then all the sensor elements can be exposed simultaneously.

It is an alternative development that all sensor elements are switched sensitive at the same time, for example by performing the reset or the erase for all sensor elements simultaneously. This may for example be the case when the image sensor is operable in the global start mode, but now no flash is needed. The image sensor can therefore work or be operable according to the rolling shutter method with the global start mode. In this development, the sensitive switching before, during or even after the opening of the closure can be performed. The order of the method steps that the image sensor is switched completely sensitive and that the shutter is opened, so in principle is not fixed.

Under a closure, in particular a light-tight, mechanically movable element is referred to, which lies in the camera in the beam path in front of the image sensor. During the exposure, the shutter will open for the duration of the preset shutter speed at which the light hits the image sensor. In particular, the closure is electrically operable, e.g. when energized opens and is energized (and thus in its idle state) is closed.

The rolling shutter image sensor is in particular a digital image sensor. It is an embodiment that the rolling shutter image sensor is a CMOS sensor. A CMOS sensor is particularly inexpensive and also readable line by line.

The line-by-line reading can also be a column-by-column read-out.

In particular, in the case where all the sensor elements are switched sensitive at the same time, it is preferable that the shutter is opened simultaneously with the sensitivity of the image sensor (in particular rolling shutter image sensor with global-start mode). Thus, a time delay between a triggering of the method and an image acquisition can be made particularly short.

In particular, in the case where all the sensor elements are switched sensitive at the same time, it is an advantageous embodiment that the image sensor (in particular rolling shutter image sensor with global-start mode) or all of its individual sensor elements is directly (ie without targeted delay) are switched sensitive after opening the shutter. Thus, it can be ensured that an opening movement of the shutter is not absorbed by the image sensor.

However, the sensor elements can also be switched sensitive before opening the shutter. Since no light reaches the sensor cells in the time between the sensitive switching and the opening of the shutter, this development is also suitable for providing sharp images of moving objects by means of a rolling shutter image sensor. The rolling shutter image sensor can do without the global start mode.

It is a further embodiment that the charge transfer and the readout of the image sensor are delayed with respect to a signal for closing the shutter. This ensures that a closing movement of the shutter is not absorbed by the image sensor.

It is still another embodiment that the opening of the shutter and the sensitivity switching of the image sensor after a triggering or triggering of the method are specifically delayed, e.g. for 3 ms. Thus, the beginning of the opening of the shutter and the sensitive switching can be set particularly accurately.

The field of application of the method is basically not limited and may be e.g. for image capturing interiors of household appliances, in particular refrigerators (e.g., a refrigerator and / or a freezer), in industrial facilities (e.g., for monitoring production lines), in vehicles, etc.

The task is also solved by a camera with a line by line charge-transferring and readable image sensor ("rolling shutter image sensor") and with a shutter, wherein the camera is designed to carry out the method as described above. The camera can be formed analogously to the method, and vice versa, and gives the same advantages.

It is an embodiment of the fact that the camera has a shutterless, equipped with the rolling shutter image sensor camera module, which is housed in a provided with a light passage opening and otherwise light-tight housing, and that the closure is inserted into the light passage opening.

The camera module may have a so-called "imager" or be an imager. The imager has, in addition to the actual image sensor, at least one electronic unit for operating the image sensor and a memory for storing at least one recorded image. The electronics of the imager can e.g. Sensitize and read the sensor elements.

The camera module may further include a digital signal processor coupled to the imager, e.g. perform image processing of the image data generated by the imager. The digital signal processor may e.g. set certain modes and settings of the imager and control the imager.

The camera module may also have at least one interface for e.g. to connect to an external unit such as a central control unit, a trip device, a network, etc.

The camera module and the shutter are different components of the camera, especially independently installed components.

The camera can be provided for installation in a household appliance, in particular a cooling device (CiF), "camera-in-fridge". The camera can be provided in particular for installation on a door of the household appliance with a view into a utility space and / or for installation on a body of the household appliance with an inside view of the door. If, for example, every time the door is opened and / or closed, at least one receptacle is made of the useful space of the household appliance and / or the inside of the door, the changes between different images or images that arise in the course of using the household appliance can be based on a change Setting of goods in the household appliance and a corresponding removal of the goods are inferred. This information can be further processed for different purposes.

However, the field of application of the camera is not limited thereto but may be used for industrial equipment, vehicles, etc.

The image or recording created by the camera can be transmitted to an image processing device. The image processing device is located in the household appliance or outside the household appliance, e.g. in on an external server.

The object is further achieved by a household cooling device (eg a refrigerator and / or a freezer), comprising: at least one usable space closable by a door, at least one camera and a triggering device, which is adapted to at least one camera reaching at least one predetermined angular position of the door trigger.

It is an embodiment that the triggering device has a corresponding angle sensor (for example a Hall sensor) or is connected to an angle sensor which generates a signal when the at least one predetermined angular position is reached. The triggering device can then output a trigger signal to the at least one camera for triggering the image acquisition (including, if necessary, for opening and closing the shutter).

The trigger signal causes, for example, that the sensor elements are deleted and thus are made sensitive or exposing or are.

The household refrigerator may have a screen on the door to display at least one image taken by the at least one camera.

The above-described characteristics, features and advantages of this invention, as well as the manner in which they are achieved, will become clearer and more clearly understood in connection with the following schematic description of an embodiment which will be described in detail in conjunction with the drawings.

1 shows a sectional view in plan view of a refrigerator with a camera arranged in a door;

2 shows the camera in a sectional view; and

3 shows a diagram for the sequence of a method for image acquisition by the arranged in the door of the refrigerator camera.

1 shows a sectional view in plan view of a household refrigerator in the form of a refrigerator K with a work space N, the front-side feed opening B by means of a pivotable door T is closed. In the door T is a camera 1 arranged, which can also be referred to as a CiF arrangement. In addition, a camera can be arranged on the body, which is directed to the inside of the door T (o. Fig.).

The refrigerator K further comprises a serving as a triggering device control device W, which is adapted to the camera 1 during an opening and / or closing operation of the door T with reaching a predetermined ("release -") angular position φ the door T trigger. The control device W may, for example, have an angle sensor (not shown) or be connected to an angle sensor which outputs a signal when the trigger angle position φ is reached, for example a magnetic sensor such as a Hall sensor, a switch, etc.

The camera 1 is arranged in the door T that it is directed in the triggering angular position φ or even a short time later in the work space N. Then the field of vision of the camera 1 in other words directed to the work space N. With triggering of the camera 1 This can make at least one image of the work space N or at least take a picture of the work space N.

From the camera 1 recorded images can be transmitted to the control device W, for example via a USB interface. The control device W can transmit the images wirelessly or by wire to an image processing device (not shown), which is located, for example, in a cloud or on a dedicated server.

2 shows the camera 1 in section. The camera 1 has a housing 2 in which a shutterless camera module 3 is housed. The camera module 3 has an imager 4 with a rolling shutter CMOS image sensor 5 and electronics 6 on. The CMOS image sensor 5 is by means of electronics 6 readable line by line. The camera module 3 can be an optic 7 be upstream.

The camera module 3 is through a light passage opening 8th of the housing 2 directed. In the light passage opening 8th is a lock 9 used. Is the closure 9 closed, no light penetrates through the light passage opening 8th on the CMOS image sensor 5 of the camera module 3 , The camera 1 also has a controller 10 on which the lock 9 and the imager 4 controls. The control 10 can via an interface (o. Fig., eg a USB interface) of the camera 1 as shown to be coupled to the controller W. The controller can also be functional in the electronics 6 be integrated.

3 shows a diagram of a possible sequence of a method for image acquisition by the camera 1 , If a user opens the door T (with the door T typically being pivoted by 90 ° in about one second) and reaches the triggering angular position φ, an electrical signal S0 (the .theta.) Is generated by the control device W at a time t0 by means of the angle sensor can also be referred to as trigger or trigger signal S0) to the controller 10 output. The control 10 opens at a later time t1 practically at the same time the shutter 7 and also switches (after erasing or "resetting") all individual sensor elements of the CMOS image sensor 5 sensitive to light. Advantageously, with opening of the closure 9 Already switched all sensor elements sensitive.

Opening the shutter 9 is applied by applying an electrical signal S1 (eg, a voltage) to the shutter 9 causes, which is closed in its rest position. The closure 9 can be opened almost instantaneously.

Sensitivity of the CMOS image sensor 5 or its individual sensor elements can by means of the electronics 6 be executed.

The time duration t1-t0 can be predetermined by a switching inertia and / or be set in a targeted manner. It can e.g. three milliseconds.

The closure 7 is closed again at a time t2> t1 by the electrical signal S1 no longer being applied to the shutter 7 is created. Therefore, for a ("exposure") time period t2-t1, all individual sensor elements of the CMOS image sensor become 5 exposed. From the time t2, the individual sensor elements may initially still be switched sensitive, but receive in the light-tight housing 2 no light anymore. Therefore, until their charge transfer and readout, they remain in the state predetermined by the exposure time period t2-t1.

At a time t3> t2 the electronics begins 6 , the charge of the sensor elements of the rolling shutter CMOS image sensor 5 transfer line by line and read line by line. Because after closing the shutter 9 no light on the CMOS image sensor 5 does not occur, resulting from the movement of the door T resulting image error. The recorded and read picture is very sharp. Upon completion of the read operation at a time t4> t3, the camera is 1 ready for use again.

Of course, the present invention is not limited to the embodiment shown.

That's how the camera works 1 Upon reaching a predetermined tripping angle position φ of the door T record an image sequence.

Generally, "on", "an", etc. may be taken to mean a singular or a plurality, in particular in the sense of "at least one" or "one or more" etc., unless this is explicitly excluded, e.g. by the expression "exactly one", etc.

Also, a number may include exactly the specified number as well as a usual tolerance range, as long as this is not explicitly excluded.

LIST OF REFERENCE NUMBERS

1

 camera

2

 casing

3

 camera module

4

 Imager

5

 CMOS image sensor

6

 electronics

7

 optics

8th

 Light aperture

9

 shutter

10

 control

B

 loading port

K

 fridge

N

 utility space

S0

 trigger signal

S1

 Electrical signal to the lock

S2

 reading process

T

 door

t0-t4

 time

W

 control device

φ

 Shutter-angle position

Claims (9)

Image recording method in which - a rolling shutter image sensor ( 5 ) is fully sensitive, - a closure ( 9 ) is opened, - then the shutter ( 9 ) and - the rolling shutter image sensor ( 5 ) with the closure closed ( 9 ) is read line by line. Method according to claim 1, wherein the closure ( 9 ) simultaneously with the sensitive switching of the image sensor ( 5 ) is opened. Method according to Claim 1, in which the image sensor ( 5 ) immediately after opening the closure ( 9 ) is switched sensitive. Method according to one of the preceding claims, in which the readout of the image sensor ( 5 ) is delayed from a signal to close the shutter. Method according to one of the preceding claims, in which the opening of the closure ( 9 ) and the sensitivity of the image sensor ( 5 ) are delayed after a triggering (S0) of the method. Camera ( 1 ) with a rolling shutter image sensor ( 5 ) and a closure ( 9 ), characterized in that the camera ( 1 ) is designed for carrying out the method according to one of the preceding claims. Camera ( 1 ) according to claim 6, characterized in that - the camera ( 1 ) a lockless, with the image sensor ( 5 ) equipped camera module ( 3 ), which in one with a light passage opening ( 8th ) and otherwise light-tight housing ( 2 ), and that - the closure ( 9 ) in the light passage opening ( 8th ) is used. Camera ( 1 ) according to one of claims 6 or 7, characterized in that the image sensor ( 5 ) is a CMOS sensor. Household refrigerating appliance (K) with a door (T), comprising - at least one camera ( 1 ) according to one of claims 7 or 8 and - a triggering device (W), which is adapted to the camera ( 1 ) with reaching at least one predetermined angular position (φ) of the door (T) trigger.

Images