EP3783291A1 - Fridge and / or freezer - Google Patents

Abstract

The present invention relates to a refrigerator and / or freezer with a device body in which there is a cooled interior and with a door by means of which the cooled interior can be closed, with a camera assigned to the refrigerator and / or freezer being provided by means of which the cooled interior can be captured photographically, and furthermore an image processing unit is provided, by means of which images of the interior captured by the camera can be corrected and / or improved.

Description

The present invention relates to a refrigerator and / or freezer with a device body in which there is a cooled interior and with a door by means of which the cooled interior can be closed.

It is known from the prior art to arrange a camera on the inside of a door of a refrigerator and / or freezer, which camera is arranged to detect the content of the cooled interior or the cooled interior as such.

In practice, however, undesired distortions often occur in images of the interior captured by a camera, which result from the viewing angle of the camera, distortions caused by the lens, for example fish-eye effects, or an inclined arrangement of the camera relative to the interior, etc.

Such distortion degrades the quality of the recorded images and makes subsequent image processing and analysis more difficult.

The present invention is based on the object of developing a refrigerator and / or freezer of the type mentioned at the outset such that the image quality can be improved with little effort when capturing the interior.

This object is achieved by a refrigerator and / or freezer with the features of claim 1.

A refrigerator and / or freezer according to the invention has a device body in which a cooled interior is located and a door by means of which the cooled interior can be closed. According to the invention, a camera assigned to the refrigerator and / or freezer is provided, by means of which the cooled interior can be captured photographically. Furthermore, an image processing unit is provided, by means of which images of the interior space captured by the camera can be corrected and / or improved.

It has been found to be advantageous if the image processing unit is designed to recognize predefined fixed reference structures in at least one image of the interior captured by the camera and to correct optical distortions in the at least one image based on these reference structures.

The image processing unit is preferably designed to carry out an image transformation.

Furthermore, it has proven to be advantageous if the refrigerator and / or freezer also has a display unit on which images captured by the camera and / or images processed by the image processing unit can be displayed.

The camera can be mobile or movable relative to the refrigerator and / or freezer, for example if the camera is part of a smartphone, or it can also be permanently (detachable or non-detachable) installed on or in the refrigerator and / or freezer.

The image processing unit can also be provided internally in the camera and / or externally, preferably in the cloud or a PC.

The display unit (also referred to as a display) can also be mobile / movable relative to the refrigerator and / or freezer, the display unit preferably being part of a smartphone, or fixed (detachable or non-detachable) on or in the refrigerator and / or freezer be installed.

If the units / components mentioned are not installed on or in the refrigerator and / or freezer, then the refrigerator and / or freezer preferably has a transmission unit for communication with these units / components.

The image processing unit is preferably designed to use an edge detection method to recognize horizontal structures, preferably storage areas or shelves for chilled / frozen goods, in the interior as reference structures. The edge detection can be done pixel by pixel.

The display can be arranged in or on the refrigerator and / or freezer and / or a transmission unit can be provided that transmits the camera image to an external display, such as a smart phone or PC. It is therefore also possible that the display is not part of the device, but is designed as an external unit, such as a cell phone or computer screen. Accordingly, a transmission unit can be provided which transmits the camera image to an external display.

The invention also relates to a method for recording the cooled interior of a refrigerator and / or freezer according to the invention, with the camera recording an image of the interior which is corrected and / or improved using the image processing unit, preferably by adding predefined fixed reference structures in the image are preferably recognized by an edge recognition method and optical distortions in the image are corrected using these reference structures.

Horizontal structures, preferably storage areas or shelves for chilled / frozen goods, are preferably recognized in the interior as reference structures by means of the image processing unit. A deviation of the recognized horizontal structures in the image recorded by the camera from the actual horizontal is then measured. For example, a rotational deviation of 5 ° can be recorded, the reference structures / storage shelves appear crooked / at an angle in the image.

On the basis of the measured deviation, a corresponding correction factor is then determined, on the basis of which the image captured by the camera is corrected and / or improved.

For example, it can be determined that the image is rotated 5 ° to the left, so that the shelves appear crooked, although they are arranged horizontally in reality. The correction factor could thus consist of rotating the image by 5 ° to the right, so that the horizontal shelves appear horizontal again (0 °) and the image is thus corrected.

The device can have a display which is connected to the camera in such a way that the image of the interior or of the objects recorded by the camera is shown on the display. The corrected image can also be displayed, or both.

Said image processing unit can be arranged inside the camera.

It is also conceivable and encompassed by the invention that the image processing unit is arranged inside or outside the refrigerator and / or freezer or is located in a cloud. It therefore does not have to be a component of the refrigerator and / or freezer, but can be designed as an external unit that is in communication with the refrigerator and / or freezer.

At this point it should be noted that the terms “a” and “an” do not necessarily refer to exactly one of the elements, although this represents a possible embodiment, but can also designate a plurality of the elements. The use of the plural also includes the presence of the element in question in the singular, and conversely, the singular also includes several of the elements in question.

• Fig. 1 ein verzerrtes Bild eines Innenraums, in dem die den Regalböden entsprechenden horizontalen Linien als Referenzstrukturen erkannt worden sind. Die eigentlich horizontalen Kanten der Regalböden erscheinen schief.
• Fig. 2 eine korrigierte Version des Bilds aus Fig. 1. Die horizontalen Kanten der Regalböden werden horizontal dargestellt.
• Fig. 3 verdeutlicht das Prinzip der Bildtransformation.

Further details and advantages of the invention will be explained in more detail using an exemplary embodiment described below and using the drawings. Here shows

• Fig. 1 a distorted image of an interior space in which the horizontal lines corresponding to the shelves have been recognized as reference structures. The actually horizontal edges of the shelves appear crooked.
• Fig. 2 a corrected version of the image Fig. 1 . The horizontal edges of the shelves are displayed horizontally.
• Fig. 3 illustrates the principle of image transformation.

Fig. 1 shows on the left-hand side a camera image of an interior space in which various objects are arranged.

The exemplary embodiment relates to a cabinet-like refrigerator and / or freezer with a body and a door, which together delimit a cooled interior.

At this point, it is pointed out that the invention is not limited to cabinet-type refrigerators and / or freezers, but rather includes any desired refrigerator and / or freezer, such as, for example, chests.

The device according to the exemplary embodiment has a body in which there is a cooled interior. This can be closed by means of a door that can be pivoted about a vertical pivot axis.
The door can be hinged on the right or left or it can be a device that has two doors that swing open in opposite directions.

In the present embodiment, the camera is arranged in / on the door of the device. The camera can always be activated so that it is switched on during the entire door opening or closing or so that it is only switched on during a partial section or only at a certain door opening angle.

The camera is not necessarily fixed to the door, but can be removed from it. The camera is self-adjusted / calibrated in such a way that a sufficiently good picture of the interior is obtained.

• Erfassen von horizontalen Kanten sowie der Winkel, in denen diese verlaufen, in dem Bild;
• Berechnen der Höhe sowie des Winkels einer jeden Kante, vorzugsweise in einem zentralen Abschnitt des Bildes;
• Vorhersagen des Verlaufs der Kanten über die gesamte Höhe / Breite des Innenraums des Geräts;
• Erfassen einer Abweichung der erfassten horizontalen Kanten (z.B. +/- 5°) von der tatsächlichen Horizontale (0°);
• Bestimmung eines entsprechenden Korrekturfaktors (z.B. Rotation um +5°), um die erfassten Kanten an die tatsächliche Horizontale (0°) anzugleichen;
• Korrektur des Bildes anhand des Korrekturfaktors, z.B. mittels Tranformation.

Images from the camera are transmitted to an image processing unit / control unit, which carries out the following steps:

• Detecting horizontal edges and the angles at which they run in the image;
• Computing the height as well as the angle of each edge, preferably in a central portion of the image;
• Predicting the course of the edges over the entire height / width of the interior of the device;
• Detection of a deviation of the detected horizontal edges (eg +/- 5 °) from the actual horizontal (0 °);
• Determination of a corresponding correction factor (eg rotation by + 5 °) in order to adjust the detected edges to the actual horizontal (0 °);
• Correction of the image based on the correction factor, for example by means of transformation.

Fig. 1 reflects on the right-hand side the step "Detecting horizontal edges and the angles at which they run in the image". The light lines in front of a black background in the displayed image analysis result show the recorded edges, which correspond to the shelves / storage areas of the device. Instead of horizontal edges, any other reference structures can also be recorded, for example also the side walls of the device, etc. The basic idea of the invention is to use known and constant structures to correct the image.

• Anpassung der Farben / Helligkeit
• Rotation und / oder Translation des Bildes
• Kantenerkennung; vorzugsweise werden beispielsweise bei der Erkennung der Kanten von Einlegeböden diese in der Mitte des Bildes erfasst und der weitere Verlauf der Kanten / Einlegeböden wird entsprechend extrapoliert / vorhergesagt
• Winkelvorhersage: Verwendung der berechneten Winkel / Kantenverläufe, um den weiteren Verlauf der Kanten vorherzusagen
• Bildtransformation: matrixartige Aufteilung eines Bildes in Reihen (z.B. von Pixeln) und Korrektur des Bildes, indem jede Reihe anhand eines Korrekturfaktor angepasst wird
• Anpassung an Vorlagen: Bewegen des Bildes und Zoomen, um das erfasste Bild an eine Vorlage (z.B. die bekannten Abmessungen des Innenraums) anzupassen. Bereits bestehende verschwommene Bilder können zudem dazu verwendet werden, Lücken in einem Bild (z.B. in den Ecken des Bildes nach einer Rotation) zu füllen.

The analysis and / or correction of the image can include:

• Adjustment of colors / brightness
• Rotation and / or translation of the image
• Edge detection; Preferably, for example, when the edges of shelves are recognized, they are recorded in the center of the image and the further course of the edges / shelves is extrapolated / predicted accordingly
• Angle prediction: Use of the calculated angles / edge courses to predict the further course of the edges
• Image transformation: matrix-like division of an image into rows (e.g. of pixels) and correction of the image by adjusting each row using a correction factor
• Adaptation to templates: moving the image and zooming in order to adapt the captured image to a template (e.g. the known dimensions of the interior). Existing blurred images can also be used to fill gaps in an image (e.g. in the corners of the image after a rotation).

Fig. 2 shows a corrected version of the image Fig. 1 . The horizontal edges of the shelves are displayed horizontally. The light lines in front of a black background in the image analysis result on the left-hand side show the recorded edges, which correspond to the shelves / storage areas of the device and have been rotated accordingly so that they now appear horizontally.

The correspondingly corrected image of the interior can be seen on the right-hand side. In this example the image has been rotated to the left until the edges of the shelves are displayed horizontally.

During the correction, the image is preferably transformed, with each row of the image being corrected with the determined correction factor, e.g. being rotated by a predicted angle.

In Fig. 3 the principle of image transformation is illustrated. So on the left is in Fig. 3 shown that the image transformation occurs line by line, each line being a line of pixels. Each individual line / row of the image is thus corrected with a certain correction factor, in Fig. 3 rotated a certain angle so that the edges of the shelves (known to be horizontal) are also shown horizontally in the image.

Thus, with the present invention, distortion can be easily removed and the image quality can be increased.

Claims (12)

Refrigerator and / or freezer with a device body in which there is a cooled interior and with a door by means of which the cooled interior can be closed, characterized in that a camera assigned to the refrigerator and / or freezer is provided, by means of which the cooled interior can be captured photographically, and that an image processing unit is also provided, by means of which images of the interior captured by the camera can be corrected and / or improved. Refrigerator and / or freezer according to claim 1, characterized in that the image processing unit is designed to recognize predefined fixed reference structures in at least one image of the interior captured by the camera and to correct optical distortions in the at least one image using these reference structures. Refrigerator and / or freezer according to Claim 1 or 2, characterized in that the image processing unit is designed to carry out an image transformation. The refrigerator and / or freezer according to one of the preceding claims, characterized in that the refrigerator and / or freezer furthermore has a display unit on which images captured by the camera and / or images processed by the image processing unit can be displayed. The refrigerator and / or freezer according to one of the preceding claims, characterized in that the camera is mobile relative to the refrigerator and / or freezer, wherein the camera is preferably part of a smartphone, or fixed on or in the refrigerator and / or Freezer is installed. Cooling and / or freezing device according to one of the preceding claims, characterized in that the image processing unit is provided internally in the camera and / or externally, preferably in the cloud. Refrigerator and / or freezer according to one of the preceding claims, characterized in that the display unit is mobile relative to the refrigerator and / or freezer, wherein the display unit is preferably part of a smartphone, or fixed on or in the refrigerator and / or Freezer is installed. Refrigerator and / or freezer according to one of the preceding claims, characterized in that the image processing unit is designed to use an edge detection method to recognize horizontal structures, preferably storage areas or shelves for chilled / frozen goods, in the interior as reference structures. Refrigerator and / or freezer according to one of the preceding claims, characterized in that the image correction comprises at least one of the following steps: - Adjustment of colors / brightness - Rotation and / or translation of the image - Edge detection - Angle prediction and / or prediction of the course of edges - Image transformation - Adaptation to templates Method for recording the cooled interior of a refrigerator and / or freezer according to one of claims 1 to 9, characterized in that an image of the interior is recorded by means of the camera, which is corrected and / or improved by means of the image processing unit, preferably by using predefined fixed Reference structures in the image are preferably recognized by an edge recognition method and optical distortions in the image are corrected on the basis of these reference structures. Method according to claim 10, characterized in that horizontal structures, preferably storage areas or shelves for chilled / frozen goods, are recognized in the interior as reference structures by means of the image processing unit, a deviation of the recognized horizontal structures in the image recorded by the camera from the actual horizontal is measured, and a corresponding correction factor is determined on the basis of the measured deviation, on the basis of which the image captured by the camera is corrected and / or improved. Method according to one of Claims 10 or 11, characterized in that the image correction comprises at least one of the following steps: - Adjustment of colors / brightness - Rotation and / or translation of the image - Edge detection - Angle prediction and / or prediction of the course of edges - Image transformation - Adaptation to templates

Images