EP3844665A1 - Interaktionseinrichtung - Google Patents
InteraktionseinrichtungInfo
- Publication number
- EP3844665A1 EP3844665A1 EP19753002.5A EP19753002A EP3844665A1 EP 3844665 A1 EP3844665 A1 EP 3844665A1 EP 19753002 A EP19753002 A EP 19753002A EP 3844665 A1 EP3844665 A1 EP 3844665A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- interaction
- projection
- determined
- camera
- interaction device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
- G06V10/12—Details of acquisition arrangements; Constructional details thereof
- G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
- G06V10/147—Details of sensors, e.g. sensor lenses
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/68—Food, e.g. fruit or vegetables
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/22—Character recognition characterised by the type of writing
- G06V30/224—Character recognition characterised by the type of writing of printed characters having additional code marks or containing code marks
Definitions
- the invention relates to an interaction device for a domestic appliance.
- the invention relates to an interaction device for flexible control of one or more household appliances.
- DE 10 2016 100 949 A1 shows a household appliance with a projector for displaying a control element and a camera for detecting an interaction of a user with the control element.
- a barcode of a product can also be scanned using the camera in order to prepare the household appliance for processing the product.
- US 6 559 882 B1 proposes an oven which scans a food to be thawed by means of an infrared camera and provides a thermal image of the food during thawing.
- the camera can also scan a barcode of a dish or recognize the dish itself to control defrosting.
- An interaction device comprises a projector for providing a projection; a first camera for determining an interaction of an object with the projection; a second camera for scanning an image of the object; and a processing device configured to determine the object based on a scanned image.
- the detection of the object can thus be improved centrally and one or more household appliances, in particular kitchen appliances, can be provided with a reference to the detected object or can be controlled depending on the detected object.
- the object can in particular comprise an ingredient that is consumed in the preparation of a dish.
- the ingredient as an object can be assigned to different kitchen appliances in succession or simultaneously.
- the ingredient can first be stored in a pantry, refrigerator or freezer. It can then be removed by a user and processed on the basis of a recipe, for example using a mixer or a mixer. The ingredient can then be cooked together with other ingredients, for example in an oven or on a stove.
- the kitchen appliances mentioned or other kitchen appliances can be controlled in an improved manner as a function of the detected object.
- an inventory of objects present in a household can be better supported by the central object recognition by means of the interaction device.
- the object can be determined in different ways.
- the object is determined on the basis of the image.
- an outline, a shape, a size, a color or a texture of the object on the image can be recognized and compared with reference data.
- the object can be assigned to a class of objects to which further information can be assigned, for example a price, a content of a nutrient or a usual shelf life. Unpackaged products, in particular, can thus be better identified.
- the object is determined on the basis of a label attached to the object.
- the label can in particular be attached to a packaging of the object.
- the packaging itself can also be recognized.
- the packaging or the label can be provided with, in particular, printed information, which can, for example, provide text, numerical or alphanumeric information about the object.
- the object can also be determined on the basis of a logo or trademark on the label or packaging.
- a machine-readable identifier is attached to the object, the packaging or the label. This identifier can include, for example, a bar code or a QR code (two-dimensional raster code).
- the coded information can be mapped to the object directly or by means of an assignment by a database and optionally further information which can be assigned to the object. It is further preferred that the interaction device determines a quantity of the determined object. Different embodiments are possible for this. It is common to all embodiments that the quantity can be determined in any units that are applicable to the object, for example as length, area or hollow dimension, number or weight. Additional units are also possible.
- the amount of the determined object is determined based on the scanned image.
- a shape, a color, a size or a texture of the object can be taken from the image and assigned to an object based on previously determined information.
- a differentiation is made between two different objects.
- the first camera is set up to determine an image of said first object, while the second camera is set up to detect an interaction of a second, other object with the projection.
- a set of the determined first object can then be determined based on the interaction.
- the second object can in particular include a hand or another body part of a user.
- the amount can be specified by the user, for example by making a predetermined gesture or interacting with the projection.
- the objects can be captured simultaneously or one after the other.
- the projection comprises an input field for a quantity.
- the quantity can be specified numerically, for example, using a number field.
- an analog input can be supported in which, for example, a slide or rotary control can be changed by a user.
- a symbolic input field can also be provided, in particular for quantities that are difficult to quantify, such as a splash or a pinch.
- an input field for a unit can be provided. Units of the information can be variable in order to support a metric, an Anglo-American or another measurement system. For example, kitchen measures can be supported that include information such as a teaspoon, a tablespoon, a knife tip, a wine glass, a cup, a large, a shock or an egg weight.
- the amount of the determined object is determined by means of a measuring device in the area of the projection.
- the measuring device can in particular comprise a scale, which can be embedded, for example, in an object such as a worktop or a work board.
- the scale can also be part of cookware or cutlery.
- the measured value determined by the scale is preferably transmitted to the interaction device wirelessly.
- the processing device of the interaction device can be provided regardless of location and can be connected to other components by means of a wireless communication device in particular.
- the processing device can be implemented as a server or service, for example in a cloud.
- a system comprises an interaction device described herein and a central point with a communication device.
- the central point is set up to carry out determinations of objects for a large number of interaction devices. Processing or storage resources of the central point can thereby be made available to a large number of interaction devices.
- data collection about objects can be kept better in order to assign an attribute to an object to be recognized or recognized.
- the assignment can be carried out with improved learning ability, since a large number of scans and determinations of objects can be carried out, the success or failure of which can be determined and stored for future use.
- a method comprises steps of providing a projection; scanning an area in which the projection lies; scanning an image of a first object in the area of the projection; detecting an interaction of a second object with the projection; and determining the first object based on the scanned image.
- the method can in particular be carried out partially or completely by means of an interaction device or a system described here.
- the method can be present, in particular, in the form of a computer program product with program code means, and a processing device of the interaction device or of the system can be configured as a programmable microcomputer or microprocessor. rocontroller.
- a processing device of the interaction device or of the system can be configured as a programmable microcomputer or microprocessor. rocontroller.
- Features or advantages of the method can be transferred to the device or the system and vice versa.
- the interaction points to a class of objects, the first object being assigned to the class.
- different cucumbers can be assigned to a class that includes different cucumbers.
- a relatively small number of manual entries can be sufficient to provide sufficient reference data so that further objects of the same class can be recognized automatically.
- a set of the determined first object can be removed from an inventory list.
- the inventory list can in particular comprise a list of available household supplies.
- the stocks preferably include ingredients or consumables that can be used to prepare a dish.
- Certain quantities of the specific first object can also be added to a procurement list.
- the procurement list can optionally be transmitted to a user's mobile device, for example to a smartphone.
- the procurement list can also be made available, for example, to a central storage location, such as on a server or in the cloud, for retrieval by a user.
- a predetermined quantity of the first object is only added to the procurement list if a further predetermined quantity of the first object has previously been consumed.
- a partially or fully automatic order for one or more first objects can be controlled based on the procurement list.
- the processing device can be set up to carry out a method described here in whole or in part.
- the processing device can comprise a programmable microcomputer or microcontroller and the method can be in the form of a computer program product with program code means.
- the computer program product can also be stored on a computer-readable data carrier. Additional features or advantages of the method can be transferred to the device or vice versa.
- Figure 1 shows a system with an interaction device
- Figure 2 illustrates a flow diagram of a method.
- FIG. 1 shows a system 100 with an interaction device 105.
- the interaction device 105 is preferably arranged in a household, more preferably in a kitchen of the household.
- One task of the interaction device 105 is preferably to control one or more domestic appliances 110, which can in particular comprise a kitchen appliance.
- the interaction device 105 can optically scan a first object 115 and / or a second object 120.
- the two objects 115, 120 can be identical to one another or different from one another. If they are different from one another, it is preferred that the first object 115 comprises a food, an ingredient or a similar consumable of a household, while the second object 120 preferably a body part of a user, in particular a hand, or a user-controlled pointing device includes.
- the interaction device 105 comprises a projector 125, a first camera 130 and a second camera 135.
- the projector 125 is set up to throw a projection 140 onto a surface 145.
- the surface 145 can in particular be that of a work surface or a work board 150.
- the interaction device 105 is preferably mounted vertically above the surface 145 and directed downwards.
- the interaction device 105 can be arranged in the region of an upper cabinet or an extractor hood.
- an oblique or lateral orientation of the interaction device 105 with respect to a customary position of a user on the work surface can also be provided.
- the surface 145 can also include, for example, a side wall of a kitchen furniture or a wall of a room in a household. Other surfaces 145 can also be used.
- the first camera 130 is set up to take an image of the first object 115
- the second camera 135 is set up to allow the second object 120 to interact with the projection 140 determine.
- the projection 140 can comprise a control surface, the touch of which can be determined by the second object 120, for example a hand or a finger of a user, by means of the second camera 135.
- the projector 125, the first camera 130 and the second camera 135 can also be arranged in a different constellation than the one shown.
- the first camera 130 is preferably set up to scan an image in the visible light wavelength range.
- the second camera 135 can comprise a depth camera, which can work in particular in an invisible wavelength range of light. This range can include an infrared range.
- An image of the first object 115 and / or the second object 120 provided by the second camera 135 can additionally include depth information. This can be determined, for example, on the basis of the speed of light propagation between an installed light source and the second camera 135 or by means of stereoscopy.
- the interaction device 105 further preferably comprises a processing device 155, which is further preferably provided locally in the area of the projector 125 and the cameras 130 and 135.
- a preferably wireless communication interface 160 can be provided.
- processing device 155 or a further processing device 155 is communicatively connected via communication interface 160.
- the processing device 155 can be part of a central point 165, which can be designed in particular as a server or service, for example in a cloud.
- Further devices can also be communicatively connected via the communication interface 160.
- communication with a device 170 can be made possible, which can be assigned to a user of the system 100 as a smart device, for example.
- Device 170 may include, for example, a smartphone, tablet, or similar general-purpose computer.
- a measuring device 175 is provided, which can in particular be part of an object, on the surface 145 of which the projection 140 is thrown, for example a worktop or the work board 150.
- the measuring device 175 can be set up to determine a quantity, in particular a mass, of the first object 115.
- the measurement result can be processed further by means of the processing device 155 of the interaction device 105.
- the interaction device 105 not only be used to control one or more of the domestic appliances 110, but also to recognize and manage the first object 115.
- the administration can include, in particular, an assignment to one of the domestic appliances 110 or a storage or stock holding.
- FIG. 2 shows a flow chart of an exemplary method 200.
- the method 200 can be carried out by means of the interaction device 105 or by means of the system 100.
- the steps of method 200 do not necessarily have to be carried out in the order shown.
- the projection 140 can be effected on the surface 145 by means of the projector 125.
- the projection 140 can comprise different optical information, for example a pattern, a symbol, a control surface or an animation.
- the first object 115 is scanned by the first camera 130 and an image of the first object 115 is provided.
- the first object 115 can be determined in a step 215. The determination can be made in particular on the basis of the image provided.
- the first object 115 is recognized itself, in another embodiment a packaging of the first object 115 or a label attached to the first object 115 or to its packaging.
- the label can in particular comprise a textual specification of the first object 115, a barcode or a QR code.
- a step 220 the second object 120 is scanned by means of the second camera 135 and an interaction with the projection 140 or a visual information output via the projection 140 is determined.
- the scanning of the first object 115 and the determination of the interaction of the second object 120 with the projection 140 run parallel to one another, from can be done separately or in different degrees of mutual integration.
- the first object 115 and the second object 120 can be scanned one after the other and individually.
- the first object 1 15 can be scanned together with the second object 120 by means of the first camera 130, and then the interaction of the second object 120, with or without the first object 1 15, with the projection 140 by means of the second Camera 135 can be determined.
- the first object 115 can also be used to express an interaction with the projection 140. For this purpose, it can be guided by the second object 120.
- an external signal can be received, which can originate in particular from the measuring device 175.
- the first object 115 can first be fed to the measuring device 175, for example by placing it on the surface 145.
- the signal preferably indicates an amount (a quantum) of the first object 115.
- a quantity of the first object 115 can be determined. This refers in particular to the amount that is consumed, for example, during a cooking process or as part of a serving and is then no longer available to the household.
- the amount can be determined in different ways. An automatically recognized quantity does not have to be identical to an actually used quantity. In this case, a manual correction can take place, in particular by means of the interaction device 105. A correction of a quantity determined by means of the measuring device 175 can also take place in this way. In another embodiment, the amount can be estimated. If the first object 115 comprises, for example, a nutmeg, of which only a small part is usually used for the preparation of a dish, a customary or average value can be estimated as a quantity.
- a step 235 the information determined in this way can be processed further. In particular, it can be seen how much of the first object 115 is still available in the household. If necessary, a predetermined amount of the first object 115 can be placed on a procurement list.
- the procurement list can be the basis of a semi-automatic or automatic order.
- the list may also be communicated to device 170 to allow for user acquisition.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Vascular Medicine (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| DE102018214391.9A DE102018214391A1 (de) | 2018-08-27 | 2018-08-27 | Interaktionseinrichtung |
| PCT/EP2019/071072 WO2020043439A1 (de) | 2018-08-27 | 2019-08-06 | Interaktionseinrichtung |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| EP3844665A1 true EP3844665A1 (de) | 2021-07-07 |
Family
ID=67620420
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP19753002.5A Withdrawn EP3844665A1 (de) | 2018-08-27 | 2019-08-06 | Interaktionseinrichtung |
Country Status (3)
| Country | Link |
|---|---|
| EP (1) | EP3844665A1 (de) |
| DE (1) | DE102018214391A1 (de) |
| WO (1) | WO2020043439A1 (de) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102022111680B3 (de) | 2022-05-10 | 2023-08-24 | Beckhoff Automation Gmbh | Verfahren zum Betreiben eines Automatisierungssystems, Steuerungssystem und Automatisierungssystem |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6559882B1 (en) | 1999-09-02 | 2003-05-06 | Ncr Corporation | Domestic appliance |
| JP2017506169A (ja) * | 2014-02-20 | 2017-03-02 | マーク オレイニク | ロボット調理キッチン内の食品調製のための方法及びシステム |
| DE102016100949B4 (de) | 2016-01-20 | 2020-03-26 | Vorwerk & Co. Interholding Gmbh | Elektrisches Haushaltsgerät und Verfahren zu dessen Steuerung |
| US20180053263A1 (en) * | 2016-08-17 | 2018-02-22 | Wipro Limited | Method and system for determination of quantity of food consumed by users |
-
2018
- 2018-08-27 DE DE102018214391.9A patent/DE102018214391A1/de not_active Withdrawn
-
2019
- 2019-08-06 EP EP19753002.5A patent/EP3844665A1/de not_active Withdrawn
- 2019-08-06 WO PCT/EP2019/071072 patent/WO2020043439A1/de not_active Ceased
Also Published As
| Publication number | Publication date |
|---|---|
| DE102018214391A1 (de) | 2020-02-27 |
| WO2020043439A1 (de) | 2020-03-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2783550B1 (de) | Konfiguration von betriebsgeräten für leuchtmittel | |
| DE102014101079B4 (de) | Verfahren zum Betrieb einer Küchenmaschine und Küchenmaschine | |
| EP2637476B1 (de) | Kochfeld | |
| EP3415061B1 (de) | Haushaltsgerät | |
| DE102007005484B4 (de) | Verfahren zur Auswahl zumindest eines Arbeitsprogramms zur Durchführung in zumindest einem Nahrungsmittelbehandlungsraum | |
| EP3963262A1 (de) | Verfahren zum betreiben eines gargeräts und gargerät | |
| WO2015172877A1 (de) | Vorrichtung zur bedienung eines elektronischen geräts | |
| WO2016119072A1 (de) | Heissgetränkeautomat mit sensor | |
| DE102016107617A1 (de) | Verfahren zum Betreiben eines Gargeräts und Gargerät | |
| DE102014200298A1 (de) | Verfahren zum Aufbauen einer drahtlosen Kommunikationsverbindung zwischen wenigstens zwei Haushaltsgeräten | |
| DE102017209841A1 (de) | Anzeigesystem, Dunstabzug und Verfahren zur Anzeige zumindest eines Zustandes auf einem Kochfeld | |
| DE10353299A1 (de) | Verfahren zur Regelung einer Heizeinrichtung zur Nahrungsmittelzubereitung und geeignete Vorrichtung | |
| EP3844665A1 (de) | Interaktionseinrichtung | |
| DE102015211425A1 (de) | Steuereinheit und Verfahren zur Ermittlung eines Rezeptes | |
| DE102005023468B4 (de) | Gargerät | |
| EP2923533B1 (de) | Konfiguration von betriebsgeräten für leuchtmittel | |
| DE102015101707B4 (de) | Kochsystem und Verfahren zum Betreiben eines Kochsystems | |
| DE102018212441A1 (de) | Auftaumatte, Haushaltsgerät mit einer Auftaumatte, Verfahren zum Auftauen und Verfahren zum Zubereiten von Lebensmitteln | |
| EP1498057A1 (de) | Verfahren zum Bestimmen von Parametern eines Garprozesses eines Nahrungsmittels und dieses benutzende Steuervorrichtung | |
| EP3762654A1 (de) | Interaktionsmodul | |
| DE102011080246A1 (de) | Speisenhandhabungs-Küchenutensil | |
| DE102018108017A1 (de) | Bewirtungssystem, Gargerät und Verfahren zum Garen eines Produkts | |
| EP4176426A1 (de) | Lebensmittelzubereitungsvorrichtung und verfahren zur lebensmittelzubereitung | |
| BE1031026B1 (de) | Verfahren zum Betrieb eines Kochsystems sowie Kochfeld und Gargeschirr | |
| BE1030737B1 (de) | Verfahren zum Betrieb eines Kochsystems sowie Kochfeld und Gargeschirr |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
| PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
| 17P | Request for examination filed |
Effective date: 20210329 |
|
| AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
| DAV | Request for validation of the european patent (deleted) | ||
| DAX | Request for extension of the european patent (deleted) | ||
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
| 17Q | First examination report despatched |
Effective date: 20230406 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN WITHDRAWN |
|
| 18W | Application withdrawn |
Effective date: 20230619 |