EP3932282B1 - Rinsing device with image detection device - Google Patents

Description

The invention relates to a dishwashing device with an image capture device

The interior of a dishwasher can be monitored using a camera and the corresponding image evaluation in order to optically monitor and evaluate the degree of load, types of load, degree and types of soiling, etc. in order to improve washing performance.

The EP3 847 945 A1 (State of the art according to Art. 54(3) EPC)
discloses a dishwashing device with an image monitoring device, which has an image sensor with a first sensor area and a second sensor area, an interior area of the dishwashing device being able to be visually detected by means of the first sensor area and a dosing device housed in the washing compartment door being able to be visually detected by means of the second sensor area.

The DE 10 2017 118 023 A1 discloses a dishwashing device with a first camera for detecting items to be washed arranged in the upper basket and with a second camera for detecting items to be washed arranged in the lower basket.

The invention has the task of creating an improved dishwashing device with an improved image capture device.

According to the invention, this object is achieved by a dishwashing device with an image capture device with the features of the main claims. Advantageous refinements and further developments of the invention result from the following subclaims.

In addition to an enlarged viewing angle of a static camera, the advantages that can be achieved with the invention include savings in costs and materials, as well as increased stability of both the image capture device and the portion of the dishwashing device in which the image capture device is installed. The image capture device of the dishwashing device presented here does not require any additional movable mechanical elements, but still allows the views of the upper and lower baskets, including the items to be washed, to be captured. It is therefore not necessary for the camera to be mechanically movable in various ways in order to achieve large-area visual coverage and to enlarge a limited viewing angle of the camera. This reduces the costs and design complexity of the camera system. At the same time, the use of injection molded lenses enables individual elements of the optics to be combined, which results in further cost reductions.

A dishwashing device with an image capture device for detecting a dishwashing interior according to claim 1 is presented.

The dishwashing device can be, for example, a dishwasher with various load carriers for items to be washed, for example an upper basket, a lower basket and a cutlery drawer. In order to visually monitor the items to be washed arranged in the load carriers, such as dishes, cutlery or pots, as well as the washing process, a camera, for example, can be installed in the dishwasher as part of the image capture device. The image sensor can be a sensor typically used in digital cameras with an array of pixels for detecting light. The optical device can optically divide the area captured by the image capture device, which can also be referred to as the field of view of the camera, into two interior areas, the first interior area being sensed by the first sensor area of the image sensor and the second interior area being sensed by the second sensor area. In a subsequent step, the image processing device is designed to provide corresponding images of the interior of the sink. Such an image capture device can advantageously be used to realize multi-perspective optics for a static camera. This means, for example, that there is no need for a movable camera, the mechanics of which would under certain circumstances be subject to strong mechanical stress, for example if the door is unintentionally slammed hard by users of the device. In order to be able to withstand such physical stress, a movable camera would have to put in considerable design effort so that the mechanics would not be damaged. Such effort is advantageously not necessary with the image capture device presented here. Additional openings in an inner door panel of the dishwasher door can also be largely avoided. Instead, with appropriately designed static optics, two or more different ones can be created Views of the interior of a dishwasher can be captured simultaneously with the image sensor, so that neither multiple cameras nor additional mechanical components are required.

According to one embodiment, the first sensor area and the second sensor area can directly adjoin one another. For example, one half of the pixels of the image sensor can sense the incident first light from the first interior region of the sink interior and provide a corresponding first sensor signal, while the other half of the pixels can sense the incident second light and provide a corresponding second sensor signal. Advantageously, both the first sensor area and the second sensor area can be arranged together on an image sensor, whereby material and costs can be saved.

According to a further embodiment, the image capture device can have a housing in which the image sensor and the optical device can be arranged rigidly attached. For example, such a housing can be shaped to protect the image sensor and the optical device from moisture and additionally or alternatively from vibrations. The image sensor and the optical device can be arranged rigidly and immovably in the housing, which advantageously enables the use of a cost-effective camera and the amount of work involved in installing the image capture device can be kept as low as possible.

In addition, the housing can be rigidly arranged on the door. For example, the housing, and with it the image sensor and the optical device, can be installed in the door between an outer door panel and an inner door panel with a view of a rear panel, or a rear wall, of the interior. The inner door panel can, for example, have an opening that enables the interior of the sink to be captured by the image capture device. By rigidly fixing the housing to the door, the number of openings required can advantageously be minimized, which can increase the stability of the entire door.

According to the invention, the dishwashing device comprises an upper basket and a lower basket, with the upper basket being arranged in the first interior area and the lower basket in the second interior area when the dishwashing device is in an operational state. For example, the field of view of the image capture device can be divided horizontally into two halves when the dishwashing device is ready for operation. The halves can optimally represent the upper basket and the lower basket, and thus also the respective items to be washed, as well as a middle spray arm. Advantageously, items to be washed, such as dishes, can be distributed evenly in the dishwasher and still be reliably captured by the image capture device.

According to one embodiment, the optical device can comprise a first mirror element for deflecting the first light and a second mirror element for deflecting the second light. For example, two mirrors that are installed directly in the detection area, or field of view, of the image capture device can divide the detection area horizontally into two halves into two further mirrors, which can be arranged at a slightly steeper angle. As a result, both the first interior area and the second interior area can advantageously be completely captured.

According to a further embodiment, the optical device can comprise at least one prism for deflecting the first light and the second light. The prism can, for example, divide the camera's field of view horizontally in half by directing the light coming from the dishwashing compartment accordingly. The halves can optimally represent an upper basket and a lower basket and thus also the respective items to be washed and the middle spray arm. Advantageously, by using a prism, the number of components used can be reduced and thus work can be saved.

In addition, the image processing device can be designed to rotate the first image and the second image before providing them. Since, unlike attachments for 3D photos on smartphones, the motifs, for example the upper and lower baskets, lie on top of each other, the image sensor can be arranged in portrait format in order to achieve the best possible coverage of both superimposed motifs on the sensor. In this way, an image can be created on the sensor that, with two motifs, can be divided in the middle and can show one motif on one side and the second motif on the other. For example, the raw camera image in a 3:4 page format, i.e. a portrait orientation, can show an image of the upper basket in the upper half and an image of the lower basket in the lower half. Due to a non-parallel beam path caused by the fisheye optics with an aspherical frontal lens and the reflections in the prism or on the mirrors, the partial images can be upside down. For further processing, the image can be divided into two views in the middle and these can be rotated by 180° to align the images correctly. Similarly, the overall image can first be aligned and then divided horizontally into its two views. Advantageously, an image visible to a user can correspond in its orientation to the real orientation of the dishwashing device.

The flushing device can have an evaluation device for evaluating the first image and second image provided by the image processing device The evaluation device can be set up to evaluate the images with regard to one or more characteristics, for example the degree of loading of the respective load carriers, the type of loading on the respective load carriers, the amount of soiling or type of soiling of the items to be washed on the respective load carriers. The evaluation can be carried out separately, in particular for individual load carriers, such as the upper basket and lower basket. The control device of the washing device can be set up to select a washing program and/or to change or adapt it during the course of the washing program, depending on the results of the evaluation device.

In addition, the flushing device can have a display device for displaying the first image and second image provided by the image processing device. The display unit can be, for example, a display that can be arranged on an outer surface of the dishwashing device and on which the images of the dishwashing interior captured by the image capture device can be displayed. This has the advantage that a user can gain insight into the process within the dishwashing device at any time and can therefore, for example, monitor the degree of load, types of load and, additionally or alternatively, the degree and types of soiling.

According to a further embodiment, the image capture device can have a transmission device for sending the first image and the second image to a mobile device. The transmitting device can, for example, send the images captured by the image capture device to a mobile device, such as a smartphone or a tablet, using a wireless connection. This means that a user can advantageously view and monitor the flushing process and related information at any time and from any location.

Even if the described approach is described using a household appliance, the device described here can be used in connection with a commercial or professional device.

Figur 1

eine schematische Darstellung eines Ausführungsbeispiels einer Bilderfassungseinrichtung mit einer verspiegelten Optikeinrichtung;

Figur 2

eine schematische Darstellung eines Ausführungsbeispiels eines Spülgeräts mit einer Bilderfassungseinrichtung mit einer verspiegelten Optikeinrichtung;

Figur 3

eine schematische Darstellung eines Ausführungsbeispiels einer Bilderfassungseinrichtung mit einem Prisma;

Figur 4

eine schematische Darstellung eines Ausführungsbeispiels eines Kamerarohbilds;

Figur 5

eine schematische Darstellung eines ersten Bilds des ersten Innenraumbereichs; und

Figur 6

eine schematische Darstellung eines zweiten Bilds des zweiten Innenraumbereichs.

An exemplary embodiment of the invention is shown purely schematically in the drawings and is described in more detail below. It shows

Figure 1

a schematic representation of an exemplary embodiment of an image capture device with a mirrored optical device;

Figure 2

a schematic representation of an exemplary embodiment of a flushing device with an image capture device with a mirrored optical device;

Figure 3

a schematic representation of an exemplary embodiment of an image capture device with a prism;

Figure 4

a schematic representation of an exemplary embodiment of a raw camera image;

Figure 5

a schematic representation of a first image of the first interior area; and

Figure 6

a schematic representation of a second image of the second interior area.

In the following description of favorable exemplary embodiments of the present approach, the same or similar reference numerals are used for the elements shown in the various figures and having a similar effect, with a repeated description of these elements being omitted.

Figure 1 shows a schematic representation of a section of a dishwashing device with an exemplary embodiment of an image capture device 100 with a mirrored optical device 105. In this exemplary embodiment, the image capture device 100 is partially enclosed by a housing 107, the image capture device 100 being rigid and immovable in the housing 107 and the housing 107 being rigid is arranged in a door 110 of a dishwasher. This arrangement reduces the negative impact of a user opening or slamming the door 110 on the mechanics of the image capture device 100 to a minimum. The image capture device 100 is arranged between an outer door panel 112 and an inner door panel 114, only by way of example, so that the image capture device 100 captures the washing interior 115 of the dishwasher. For this purpose, in this exemplary embodiment, part of the inner door panel 114 is shaped as a transparent pane through which light from the flushing interior 115 can fall onto the image capture device 100.

The washing interior 115 of the washing device has a first interior area 117 and a second interior area 120, the first interior area 117 and the second interior area 120 being spaced apart from one another. The optical device 105 is designed to detect first light 119 originating from the first interior area 117 and second light 122 originating from the second interior area 120.

In this exemplary embodiment, the first light 119 falls on a first mirror element 125 of the optical device 105 and is deflected by the first mirror element 125 onto an optional further first mirror element 127. The first mirror element 125 and the further first mirror element 127 are arranged so that the first light 119 shines in the direction of a Image sensor 130 is guided and hits a first sensor area 132 here. According to this process, in this exemplary embodiment, the second light 122 is directed to a second sensor area 134 of the image sensor 130 by deflecting the second light 122 from a second mirror element 135 and a further second mirror element 137.

According to one exemplary embodiment, the first sensor area 132 and the second sensor area 134 directly border one another. For example, a line of the image sensor 130 is assigned to the first sensor area 132 and a directly adjacent line of the image sensor 130 is assigned to the second sensor area 134. According to one exemplary embodiment, the assignment of the pixels of the image sensor 130 to the first sensor area 132 and the second sensor area 134 is dependent on the optical device 105 and is previously known.

The image sensor 130 is designed to provide first sensor signals 140 in accordance with the incident first light 119 on the first sensor area 122 and second sensor signals 142 for an image processing device 145 in accordance with the incident second light 122 on the second sensor area 134. The image processing device 145 is designed to provide a corresponding image that images the first interior area using the first sensor signals 140 and to provide a corresponding image that images the second interior area 120 using the second sensor signals 142.

Figure 2 shows a schematic representation of an exemplary embodiment of a dishwashing device 200 with an image capture device 100 with a mirrored optical device 105. This can be the one in Figure 1 described image capture device and the described optical device act.

In this exemplary embodiment, too, the detection device 100 is arranged in a door 110, the door 110 closing a washing interior 115 of the washing machine 200. By way of example only, an upper basket 205 and a spray arm 210 are arranged in the first interior area 117 of the washing interior 115. In addition, in this exemplary embodiment, a lower basket 215 is arranged in the second interior area 120. By way of example only, items to be washed 220 are arranged in both the upper basket 205 and the lower basket 215, which can be cups and plates, for example. The image capture device 100 is designed to capture the first interior area 117 and the second interior area 120 and to provide corresponding image information for a display device 225 of the dishwashing device 200. In this exemplary embodiment, the display device 225 is formed as a display in the door 110 of the dishwasher 200. Alternatively or additionally, a display device arranged externally to the washing device 200 can also be used.

Unlike the camera attachments, in which a photo of the same subject is taken from a slightly offset perspective, here a photo of two different subjects, for example of the upper basket 205 in the first interior area 117 and the lower basket 215 in the second interior area 120, is generated simultaneously. The image capture device 100, which can also be referred to as a camera including optics, is installed in the door 110 between the outer door panel 112 and the inner door panel 114 with a view of a rear wall 230, which can also be referred to as the rear panel, of the sink interior 115. In this exemplary embodiment, two mirrors or mirror elements 127, 137, which are installed directly in the camera's field of view, divide the camera's field of view horizontally into two halves between two further mirror elements 125, 135. These are arranged at a slightly steeper angle merely as an example and capture the beam paths of the first light 119 and the second light 122, which emanate from the first interior area 117 and the second interior area 120. This results in an optimal view for both halves of the image of the baskets arranged in the first interior area 117 and the second interior area 120 and the respective items to be washed 220, as well as the middle spray arm 210 below the upper basket 205.

Figure 3 shows a schematic representation of an exemplary embodiment of an image capture device 100 with a prism 300. This can be the image capture device described in the previous figures, with the prism 300 being used instead of or as a supplement to the mirrors mentioned.

In this exemplary embodiment, too, the image capture device 100 is installed in the door 110 between the outer door panel 112 and the inner door panel 114 with a view towards the rear wall 230 of the sink interior 115. However, in this exemplary embodiment, the optical device 105 comprises the prism 300 for dividing the beam path to the image sensor 130 instead of the mirror elements described in the previous figures. For example only, the prism 300 divides the field of view of the image capture device 100 horizontally into two halves. Thus, analogous to the processes described in the previous figures, the first interior area 117 and the second interior area 120 are detected by the image capture device 100. The upper basket 205, which is arranged only as an example in the first interior area 117, and the lower basket 215, which is arranged only as an example in the second interior area 120, as well as the items to be washed in the baskets 220 and the spray arm 210 are also recorded and optionally displayed as images on the display device 225. In addition, the image capture device 100 in this exemplary embodiment includes a transmitting device 305 for sending images of the flushing interior 115 to a mobile device 310, which is a smartphone by way of example.

In principle, errors caused by light refraction must be taken into account in all optical systems and corrected if possible. With prisms in particular, chromatic aberrations, i.e. color errors, can occur, which, if they have a disruptive effect, can be eliminated with appropriate measures.

Figure 4 shows a schematic representation of an exemplary embodiment of a raw camera image 400. In this figure, the basic structure of the resulting image on the previously described image sensor, which can also be referred to as a camera sensor, is shown in portrait orientation. Only in the following steps, which follow in the following Figures 5 and 6 are described, the raw camera image 400 shown here is further processed.

The raw camera image 400 shown in this figure consists of an upper partial image 405 and a lower partial image 410, both of which directly adjoin one another. Unlike attachments for 3D photos, such as those used for smartphones, the motifs shown in the upper part of the image 405 and in the lower part of the image 410, which are designed as upper basket 205 and lower basket 215 only as an example, lie one on top of the other. That is why the image sensor described in the previous figures is arranged in portrait format in order to achieve the best possible coverage of both superimposed motifs on the sensor. This creates a raw camera image 400, which can also be referred to as an image, which is divided in the middle with two motifs and shows one motif on one side and the second motif on the other. Just as an example, the raw camera image 400 in a 3:4 page format, which corresponds to a portrait orientation, shows the upper basket 205 in the upper partial image 405 and the lower basket 215 in the lower partial image 410. Due to the non-parallel beam path caused by the fisheye optics with an aspherical frontal lens and the reflections in Prism as it is in Figure 3 was described, or on mirror elements, as in the Figures 1 and 2 described, the partial images 405, 410 of the camera raw image 400 are upside down in this exemplary embodiment.

Figure 5 shows a schematic representation of a first image 500 of the first interior area 117. Through the in Figure 1 Image processing device 145 described in Figure 4 The raw camera image described is divided into two partial images in the middle for further processing and rotated by 180° in order to align the images correctly. Alternatively, in another exemplary embodiment, the overall image can first be aligned and then divided horizontally into its two views.

The first image 500 shown in this figure shows the first interior area 117 of the Figures 1 to 3 described flushing interior, in this exemplary embodiment an upper basket 205 with items to be washed 220 arranged therein and a spray arm 210. The first image 500 shown here can be displayed in a display device, as in the Figures 2 and 3 described, displayed to a user and/or evaluated by image processing algorithms for certain characteristics (level of loading, soiling, types of loading, etc.). Depending on the evaluation of these characteristics, the course of the cleaning process can be changed or adjusted.

Figure 6 shows a schematic representation of a second image 600 of the second interior area 120. Analogous to that in Figure 5 The first image described is also the second image 600 compared to the one in Figure 4 The raw camera image described above is shown rotated by 180°. The second image 600 shows only as an example a lower basket 215 arranged in the second interior area 120 with items to be washed 220 arranged therein. The second image 600 shown here can be used together with the one in Figure 5 described first image in a display device, as in the Figures 2 and 3 described, displayed to a user and/or evaluated by image processing algorithms for certain characteristics (level of loading, soiling, types of loading, etc.). Depending on the evaluation of these characteristics, the course of the cleaning process can be changed or adjusted.

Optionally, different filters can be applied to the individual views at this point in order to optimize the individual representation of the images for further processing by image recognition algorithms.

Claims (7)

1. Rinsing device (200) having a interior washing compartment (115) in which an upper rack (205) and a lower rack (215) are arranged, and having an image recording device (100) for recording the interior washing compartment (115), wherein the interior washing compartment (115) has a first interior region (117) and a second interior region (120), wherein the first interior region (117) and the second interior region (120) are spaced apart from one another, wherein the interior washing compartment (115) can be closed with a door (110), and wherein the image recording device (100) has the following features:

   an image sensor (130) having a first sensor region (132) for providing first sensor signals (140) and a second sensor region (134) for providing second sensor signals (142);

   an optical device (105) for guiding the first light (119) incident from the first interior region (117) to the first sensor region (132) and from the second light (122) incident from the second interior region (120) to the second sensor region (134); and

   an image processing device (145), which is designed to provide, using the first sensor signals (140), a first image (500) that depicts the first interior region (117) and to provide, using the second sensor signals (142), a second image (600) that depicts the second interior region (120),

   characterised in that

   the optical device (105) comprises a first mirror element (125) for deflecting the first light (119) and a second mirror element (135) for deflecting the second light (122),
   or

   the optical device (105) comprises at least one prism (300) for deflecting the first light (119) and the second light (122),

   and in that the upper rack (205) is arranged in the first interior region (117) and the lower rack (215) is arranged in the second interior region (120) when the rinsing device (200) is ready for operation.
2. Rinsing device (200) according to claim 1, wherein the first sensor region (132) and the second sensor region (134) are directly adjacent to one another.
3. Rinsing device (200) according to one of the preceding claims, wherein the image recording device (100) has a housing (107) in which the image sensor (130) and the optical device (105) are arranged in a rigidly fastened manner.
4. Rinsing device (200) according to claim 3, wherein the housing (107) is rigidly arranged on the door (110).
5. Rinsing device (200) according to one of the preceding claims, wherein the image processing device (145) is designed to rotate the first image (500) and the second image (600) before providing them.
6. Rinsing device (200) according to one of the preceding claims, having a display device (225) for displaying the first image (500) and the second image (600) provided by the image processing device (145).
7. Rinsing device (200) according to one of the preceding claims, wherein the image recording device (100) has a transmitting device (305) for transmitting the first image (500) and the second image (600) to a mobile device (310).

Images