DE102018221639A1 - Process for improving object recognition through machine learning - Google Patents

Abstract

In order to provide a generic method with improved properties with regard to ease of use, a method for improving object recognition, in particular objects (36a) within a domestic appliance useful space (14a), is proposed by machine learning, with many image recordings (34a) being marked in machine learning Objects (36a) of an object category can be evaluated.

Description

The invention relates to a method for improving object recognition by machine learning according to claim 1.

A method for object detection is already known from the prior art. The object recognition is carried out in the method by comparing a current image of an object of an object category with many different images of objects of different object categories. The object is thus classified by means of the image acquisition, that is to say assigned to an object category, and identified. The different images of objects of different object categories are stored in a storage unit and are unchangeably specified. It is not known to expand a number of images of the objects in the storage unit and thus to enable machine learning.

The object of the invention is in particular to provide a generic method with improved properties with regard to ease of use. The object is achieved according to the invention by the features of claim 1, while advantageous refinements and developments of the invention can be found in the subclaims.

A method is proposed for improving object recognition, in particular objects within a domestic appliance useful space, by machine learning, with machine learning evaluating many, in particular different and / or identical, image recordings of marked objects of a, in particular specific and / or predefined, object category become.

With such a configuration, a high level of operating convenience can be achieved in particular. In particular, reliable object recognition and a low probability of a misassignment can be made possible, even of objects that were previously in particular unknown and / or which are regionally unique. A database of objects, in particular food objects, can be built up, in particular in a storage unit, through which, in particular, regional and / or brand-specific variations of individual objects can be highly reflected and / or taken into account. In particular, a high identification rate of, in particular known, objects and / or a high number of identifiable objects can be achieved, in particular by enlarging a database stored in the storage unit, which in particular has a large number of images stored in the storage unit. In particular, an operator can be enabled to expand object recognition to generically available objects and / or objects that are of particular interest to him, which in particular can provide a high degree of flexibility and / or a high level of operating convenience. Additional expenditure for an operator when using the domestic appliance usable space can be avoided in particular, as a result of which in particular a high number of image recordings and / or images can be achieved. During normal and / or everyday use, in particular of the household appliance usage space, it is particularly advantageous to be able to set up and / or expand a large database, in particular to enable and / or facilitate later identification.

In particular, for object detection, an object of an, in particular specific and / or predefined, object category, in particular within the household appliance useful space, is identified by means of optical and / or acoustic and / or physical detection methods. For example, the object, in particular by means of the object recognition, by means of at least one, in particular abstract, image processing and / or by means of edge recognition and / or by means of at least one transformation and / or by means of size recognition and / or by means of color recognition and / or by means of text recognition and / or by means of optical Character recognition can be identified. Since in particular a configuration of the marking could be defined, it can in particular be optimized for the recognition of these features and in particular can be free of variability inherent in the object. For object detection, at least one image recording of an object, in particular one, in particular a specific, object category, with a plurality of images of objects, which in particular could be stored and / or stored in at least one storage unit, is advantageously identified and / or classified.

An “image recording” of an object is to be understood in particular to mean an, in particular current and advantageously photographic, recording of the object, in particular to be identified, which is present in particular in an unprocessed form and on which, in addition to the, in particular to be identified, object at least the marking is depicted and in particular at least one further object could be depicted. The further object could, for example, be at least one further object and / or at least one marking and / or at least one background and / or at least one wall delimiting the domestic appliance useful space.

A “mapping” of an object is to be understood in particular as a processed image of the object that is advantageously stored in at least one storage unit. For example, the image of the object, in particular in a comparison to an image of the object, could be free of further objects. Alternatively or additionally, the image of the object, in particular in a comparison to an image recording of the object, in particular could only have a section of an image recording in which the object and / or the marking can be recognized. For example, the image recording could have a predefined shape and / or a predefined dimension and / or a predefined format.

For example, the method could be used in at least one autonomous driving, for example in at least one self-propelled lawn mower and / or in at least one self-propelled vacuum cleaner and / or in at least one self-propelled motor vehicle. The method could be used, for example, in at least one tool room, such as at least one tool cabinet and / or in at least one tool storage room, and / or in at least one storage room, such as in at least one storage room and / or in at least one cellar room. The method is advantageously used in at least one household appliance and preferably in at least one household appliance user.

A “household appliance usable space” is to be understood in particular as a spatial area which is defined and / or enclosed at least in large part by at least one household appliance and which is intended in particular for storing food. The household appliance usable space could, for example, be a cooking space and / or a usable space of a dishwasher and / or a usable space of a cleaning machine and / or a usable space of a beverage storage device. The beverage storage device could, for example, be a wine storage cabinet and / or a beverage storage cabinet. The utility room is advantageously a cold room, in particular a cold room and / or a freezer room.

“At least for the most part” is intended to mean in particular a proportion, in particular a mass proportion and / or a volume proportion and / or a proportion of a number, of at least 70%, in particular at least 80%, advantageously at least 90% and preferably at least 95 % are understood.

In particular, the household appliance usable space and advantageously the household appliance is part of at least one system which is designed in particular as a household appliance system and advantageously as a household refrigerator system. The household appliance could, for example, have at least one cleaning appliance and / or at least one beverage storage appliance and / or at least one dishwasher and / or at least one cooking appliance. The household appliance advantageously has at least one refrigeration appliance, in particular at least one household refrigeration appliance, which could in particular be designed as a refrigerator and / or freezer. A household appliance designed as a household refrigeration appliance is particularly advantageously provided for cooling refrigerated goods, in particular foodstuffs such as beverages and / or meat and / or fish and / or milk and / or milk products and / or fruit and / or vegetables, in the at least one operating state , in particular to ensure a longer shelf life of the refrigerated goods. The household appliance designed as a household refrigerator can be, in particular, a freezer and advantageously a refrigerator and / or freezer.

In particular, the system has the household appliance and, in particular via the household appliance, in particular the domestic appliance useful space. The household appliance has, in particular, the domestic appliance useful space and delimits the domestic appliance useful space in at least one operating state, in particular at least to a large extent and advantageously completely.

The system advantageously has at least one control unit which is provided in particular for the, in particular automatic, implementation of the method. A “control unit” is to be understood in particular to mean an electronic unit which, in at least one operating state, carries out at least the method, in particular automatically. The control unit has, in particular, at least one computing unit and, in particular, in addition to the computing unit, at least one control storage unit in which, in particular, at least one control and / or regulating program is stored, which is provided in particular for execution by the computing unit.

The control unit could, for example, be at least partially, in particular at least in large part and advantageously completely integrated in the household appliance. Alternatively or additionally, the control unit could, for example, be at least partially, in particular at least in large part and advantageously completely integrated in at least one external unit. The external unit could be a mobile device and / or a server, for example. The mobile device could, for example, be a cell phone and / or a cell phone and / or a Be a laptop and / or a tablet. In particular, the system could have the external unit.

In particular, the system has at least one memory unit, on which in particular a large number of images of objects, in particular from the same and / or different object categories, are stored. For example, the storage unit could be at least partially, in particular at least in large part and advantageously completely integrated in the household appliance and / or in the external unit.

In particular, the system has at least the database, which for example could be at least partially, in particular at least in large part and advantageously completely integrated in the household appliance and / or in the external unit. The database is in particular at least partially, in particular at least in large part and advantageously completely integrated in the storage unit and is preferably defined and / or formed by the images of objects stored in the storage unit, in particular from the same and / or different object categories.

In particular, the control unit has at least one communication unit, which is provided for, in particular wireless, communication with the storage unit and / or with the external unit. In particular, the storage unit and / or the external unit has at least one communication unit which is provided for, in particular wireless, communication with the control unit.

The system has, in particular, at least one sensor unit, which is provided, in particular, for detecting at least one image of at least one object, in particular within the household appliance usable space. For the detection of the image recording, the sensor unit could, for example, have at least one camera, such as a video camera and / or a photo camera and / or a night vision camera.

Object recognition, and in particular machine learning, is improved in particular by expanding and / or enlarging at least one existing database, which is, in particular predefined, in particular stored in the storage unit. In particular, the database has a large number of images of different and / or the same objects from different and / or the same object categories.

The phrase “machine learning” is to be understood in particular to mean that an image of the object is automatically produced and / or generated on the basis of an image recording of an object and in particular is automatically added to the database, in particular the object category to which the object is to be assigned. In particular, the object is automatically classified depending on the image recording of the object and in particular assigned to an object category. The object is identified, in particular automatically, in dependence on the image recording of the object and, in particular additionally with knowledge of the object category, and in particular is specifically stored in the database. In particular, the image, which is generated, in particular automatically, from the image recording, is assigned at least one object category and advantageously at least one object to the object category when stored in the database. “Automatic” should in particular be understood to mean automatically and / or avoiding the influence of an operator.

An “object category” is to be understood in particular as a generic term for a specific group of objects which are in particular similar and in particular have at least one common feature, advantageously at least two, particularly advantageously at least three, preferably at least four and particularly preferably a plurality of common features. In particular, two objects of the same object category differ by a maximum of one feature, in particular by a maximum of two, advantageously by a maximum of three and preferably by a maximum of four features. For example, at least one object category could include and / or be milk and / or milk products and / or fruit and / or vegetables and / or meat and / or fish and / or juice and / or soft drinks and / or alcoholic beverages and / or carbohydrate sources.

An “object” is to be understood in particular as a special object of an object category. In particular, the object defines the object category together with at least one further object. For example, the object could form and / or define a smallest unit of the object category. In the case of an object category called “milk” in particular, different objects of the object category could differ, for example, by a fat content, for example at least a first object of the object category full fat milk and at least a second object of the same object category semi-fat milk and at least a third object of the same object category skimmed milk could be. As an alternative or in addition, in particular in the case of an object category referred to as “milk”, for example different objects of the object category could differ in packaging, such as packaging in bottles and / or in the tetra pack and / or in canisters. For example, especially in the case of a “dairy product” designated object category, at least a first object of the object category yogurt and at least a second object of the same object category quark and at least a third object of the same object category cottage cheese.

In the method, the object category is recognized, in particular automatically, by means of the marking of the object, which is why the object category is predefined, in particular by marking the object, and in particular a, in particular lengthy, categorization of the object by comparison with images of the database is not necessary.

The marking could, for example, have at least one bar code and / or at least one marker and / or at least one image and / or at least one image section and / or at least one text and / or at least one text section and / or at least one character and / or at least one character . In particular, the marking contains at least one, advantageously sufficient, information relating to an assignment of the object to an object category. In particular, the marking could have at least one latching element and / or at least one clip and / or at least, in particular one type, label.

For example, the marking, in particular at least in the case of a marking, in particular a type having a label, could be arranged on the object. The marking could, for example, be glued to the object and / or attached to the object. For example, the marking, in particular at least in the case of a marking having at least one latching element and / or at least one clip, could be attached and / or attached to an area in the household appliance, such as in and / or on and / or above an installation unit of the household appliance be. For example, the marking could have at least one fixed area in the household appliance usable space, which could be assigned to an object category, in particular visible to an operator, such as by labeling the area and / or by applying a marking to the area.

“Provided” is to be understood to mean, in particular, specially programmed, designed and / or equipped. The fact that an object is provided for a specific function should in particular be understood to mean that the object fulfills and / or executes this specific function in at least one application and / or operating state.

It is also proposed that the object category be predefined, in particular by means of the marking of the object. In particular, the marking of the object provides at least one input for categorizing the object in an object category. In particular, the object category is known even before the object is identified. In particular, an identification of an object category can be dispensed with in the object recognition and / or in the identification of the object, since this is known in particular in advance. This enables, in particular, rapid and / or reliable object recognition, in which, in particular, an in particular lengthy categorization of the object by comparison with images of the database can be dispensed with.

The objects could, for example, be automatically marked and / or already marked. For example, a marking could already be made on the object during the manufacture of the object. A marking could in particular be attached to at least one packaging of the object and / or printed on at least one packaging of the object and / or defined by at least one packaging of the object, in particular with regard to shape and / or shape. The objects are preferably marked manually, as a result of which an operator can in particular be able to insert only regionally known and / or rare objects into the database by machine learning and thereby enable such objects to be identified later. In particular, the objects as such and in particular each object are marked individually and advantageously provided with at least one marking, in particular formed by at least one marker. In this way, in particular a high identification rate can be achieved, since the marking can in particular be assigned directly to the object and a misassignment of the marking to another object, in particular another object category, can in particular be excluded.

It is also proposed that the objects, in particular manually, be marked by positioning them in at least one specific area assigned to the object category. In particular, at least one area is marked, in particular within the domestic appliance useful space, and in particular provided with at least one marking, in particular formed by at least one marker. In particular, this enables efficient object recognition.

It is further proposed that the objects be marked on the object by manually applying at least one marker, in particular one formed by at least one marker. The marking, in particular formed by at least one marker, is in particular separate from the object and / or from a packaging of the object educated. In particular, the marking, in particular formed by at least one marker, is attached to the object, in particular detachably without tools. For example, the marking, in particular formed by at least one marker, could be attached to the object by means of at least one sticky note and / or by means of at least one sticky note. By attaching the marking directly to the object, a high identification rate can be achieved and / or a misassignment of the marking to another object, in particular another object category, can in particular be excluded.

For example, when creating the image recording of one of the objects, only one recording of the object could be recorded, in particular while avoiding a simultaneous recording of the marking assigned to the object. The marking assigned to the object could, for example, only be used to indicate to an operator an area in which the object is to be positioned. When taking the image of one of the objects, a recording of at least the object and the marking assigned to the object is preferably recorded. In particular, both the object and the marking are visible and / or identifiable and / or recognizable on the image recording. In particular, this enables reliable object detection. In particular, the marking can also be easily assigned to the object at a later point in time.

Furthermore, it is proposed that the marking, in particular automatically, formed in particular by at least one marker, is recognized in the image recording and the object is assigned to the object category in dependence on the marking, in particular automatically. As a result, in particular an efficient procedure for object detection can be provided, which in particular enables rapid and / or reliable and / or efficient object detection to be achieved. In particular, an operator influence can be avoided, which in particular makes it easy to operate. In particular, the object assigned to the object category is stored in the storage unit in accordance with the object category, which in particular enables simple later identification.

For example, an image of the object could be generated by storing the, in particular complete, image recording, which in particular could include both the object and the marking. An image of the object is preferably produced by cutting out at least one region of the image recording identified by the marking. The marking in particular indicates the area which is to be cut out of the image recording, in particular in order to produce the image of the object. As a result, in particular each image recording of objects can be stored in a standardized manner, whereby in particular a uniform and / or clear database and in particular simple and / or fast and / or reliable and / or efficient object recognition can be achieved.

In addition, it is proposed that different image recordings of different objects of the same object category be recorded and saved for later identification of an, in particular further, object of this object category. For example, different image recordings of a recorded object could differ with regard to a position of the recorded object relative to a reference point and / or with regard to an orientation of the recorded object relative to a reference point. Alternatively or additionally, different image recordings of a recorded object could differ, for example with regard to a zoom factor and / or with regard to a degree of detail and / or with regard to a degree of resolution and / or with regard to sharpness. As a result, in particular a large number of images of objects of the same object category can be generated, whereby in particular a database with a wide spectrum of images can be provided and / or optimized object recognition can be made possible.

It is also proposed that different image recordings of the same object be recorded and stored for later identification of an, in particular further, object of the same type. In particular, two objects “of the same type” are to be understood as two objects which, in particular in at least a large part and advantageously in all object features, are advantageously identical. In particular, the objects “of the same type” could differ with regard to a position of the recorded object relative to a reference point and / or with regard to an orientation of the recorded object relative to a reference point. As a result, an, in particular further, object of the same type can be identified, in particular independently of a positioning and / or independently of a relative orientation, in particular relative to at least one reference point, as a result of which in particular a high level of operating comfort can be achieved.

It is also proposed that, in particular, different and / or the same, image recordings of different and / or the same objects of the same object category and for later identification of an object of the same Object category can be saved. As a result, in particular a high identification rate and / or a low probability of incorrect and / or inadequate identification can be achieved, which in particular enables a high level of satisfaction for an operator. In particular, a wide-ranging database can be set up, by means of which even regionally different objects of the same object category can be identified.

For example, image recordings of objects of an object category could in particular only be recorded in one, in particular single, household appliance usable space and, in particular for later identification, could be stored in a memory unit assigned to the domestic appliance usable space, which could in particular be integrated in a household appliance having the domestic appliance usable space. Image recordings of different and / or identical objects of the same object category and / or of different object categories are advantageously recorded in different domestic appliance rooms and, in particular for later identification and advantageously after generating an image of the object by cutting out at least one area of the image recording identified by the marking, be stored in a common storage unit, in particular all household appliance utility rooms. A “common” storage unit of at least two different household appliance rooms is to be understood in particular to mean a storage unit which is in particular equally assigned to the household appliance rooms. The common storage unit could, for example, be at least largely and advantageously completely integrated in at least one external unit, advantageously in exactly one external unit. The external unit could in particular be an external server. In particular, the image recordings stored in the form of images of a respective object, in particular from all customers and / or from all household appliance utility rooms, are collected in the storage unit and in particular form training data for the respective object. In particular, the object recognition is re-trained by means of this training data, which is why the respective object can be identified and / or identified in the variability contained in the training data during a later identification, in particular by the control unit. In particular, there is a variance of the training data generated in this way, in particular automatically, in particular if there are a sufficient number of images and / or image recordings from different domestic appliance utility rooms. For example, different operators could use objects of an object category called “milk” of different types, such as differing in fat content, and / or from different manufacturers, and / or these objects in different orientations and / or positions relative to at least one reference point in the place the marked area. As a result, a large-scale and / or wide-ranging database can be set up, in particular in the shortest possible time, as a result of which, in particular, optimal object recognition and / or high ease of use can be achieved. In particular, such a storage unit, which is particularly systematically constructed and in particular has a large number of images of objects, in particular food objects, and in particular both of objects known nationally and of regionally restricted objects, can be added value, which in particular can be monetized independently .

A particularly high level of operating comfort can be achieved in particular by a system for object detection, with a control unit which is provided for carrying out the method according to the invention. In particular, the control unit is provided for identifying at least one object using the method according to the invention.

It is further proposed that the system, in particular household appliance system and advantageously household refrigeration device system, have at least one household appliance, in particular at least the household appliance, which has the control unit. This makes it particularly easy for an operator to operate in the private area.

Optimized ease of use can be achieved in particular by a household appliance, in particular a household refrigerator, of the system.

The method is not intended to be limited to the application and embodiment described above. In particular, the method for fulfilling a mode of operation described here can have a number that differs from a number of individual elements, components and units mentioned here.

Further advantages result from the following description of the drawing. Exemplary embodiments of the invention are shown in the drawing. The drawing, the description and the claims contain numerous features in combination. The person skilled in the art will expediently also consider the features individually and combine them into useful further combinations.

• 1 ein System mit einer externen Einheit und mit einem Haushaltsgerät in einem geschlossen Zustand einer Gerätetür des Haushaltsgeräts in einer schematischen perspektivischen Darstellung,
• 2 das System mit der externen Einheit und mit dem Haushaltsgerät in einem geöffneten Zustand der Gerätetür in einer schematischen perspektivischen Darstellung,
• 3 einen vergrößerten Ausschnitt aus 2 und eine Sensoreinheit des Systems in einer schematischen Darstellung,
• 4 eine von der Sensoreinheit aufgenommene Bildaufnahme eines Objekts in einer schematischen Darstellung,
• 5 ein Abbild des in der Bildaufnahme aus 4 darstellten Objekts in einer schematischen Darstellung,
• 6 das Abbild aus 5 und die externe Einheit in einer schematischen Darstellung,
• 7 eine stark vereinfachte Darstellung einer in der externen Einheit gespeicherten Datenbank anhand von drei ausgewählten Beispielen in einer schematischen Darstellung,
• 8 das System mit der externen Einheit, mit dem Haushaltsgerät und mit einer Vielzahl an weiteren Haushaltsgeräten in einer schematischen Darstellung,
• 9 einen vergrößerten Ausschnitt aus 2 und eine Sensoreinheit des Systems in einer schematischen Darstellung und
• 10 ein Diagramm eines Verfahrens zu einer Verbesserung einer Objekterkennung durch maschinelles Lernen in einer schematischen Darstellung.

Show it:

• 1 1 shows a system with an external unit and with a household appliance in a closed state of an appliance door of the household appliance in a schematic perspective illustration,
• 2nd the system with the external unit and with the household appliance in an open state of the appliance door in a schematic perspective view,
• 3rd an enlarged section 2nd and a sensor unit of the system in a schematic representation,
• 4th an image recording of an object recorded by the sensor unit in a schematic illustration,
• 5 an image of the in the image capture 4th represented object in a schematic representation,
• 6 the image from 5 and the external unit in a schematic representation,
• 7 a greatly simplified representation of a database stored in the external unit based on three selected examples in a schematic representation,
• 8th the system with the external unit, with the household appliance and with a variety of other household appliances in a schematic representation,
• 9 an enlarged section 2nd and a sensor unit of the system in a schematic representation and
• 10th a diagram of a method for improving object detection by machine learning in a schematic representation.

1 shows in particular a system 10a , which is designed in particular as a household appliance system. In the present embodiment, the system 10a especially designed as a household refrigeration system.

The system 10a has at least one household appliance 12a on, which is designed in particular as a household refrigerator. The home appliance 12a is designed in particular as a refrigerator and as a freezer, in particular as a fridge-freezer combination device. In particular, the household appliance 12a specifically for the system 10a provided and advantageous specifically for the system 10a educated.

The home appliance 12a has in particular a device body 16a on. The device body 16a defines, in particular, at least partially a household appliance usable space 14a (see. 2nd ). The home appliance room 14a is designed in particular as a cold room and advantageously as a cooling room. In particular, the household appliance 12a at least one and advantageously exactly one appliance door 18a on. The appliance door 18a is particularly pivotable relative to the device body 16a stored. When the appliance door is closed 18a , which in particular in 1 is shown closes the appliance door 18a especially the household appliance room 14a .

The device body 16a in particular at least partially defines a further household appliance usage space (not shown). The further household appliance room is in particular designed as a cold room and advantageously as a freezer room. In particular, the household appliance 12a at least one and advantageously exactly one further appliance door 20a on. The further appliance door 20a is particularly pivotable relative to the device body 16a stored. In a closed state, the other appliance door 20a closes the further appliance door 20a especially the other household appliances.

The other household appliance usable space is in an installed position, in particular with respect to a vertical direction 22a below the home appliance room 14a arranged. The device body is in an installed position 16a especially on a footprint 24a set up. The footprint 24a is in the present embodiment in particular a floor, in particular a room of a house.

The system 10a has in particular at least one and advantageously exactly one control unit 26a on. The control unit 26a is in the present embodiment in particular within the household appliance 12a arranged. In particular, the control unit 26a is especially part of the household appliance 12a and / or at least for the most part and advantageously completely in the household appliance 12a integrated. In particular, the household appliance 12a the control unit 26a on.

In particular, the system 10a at least one and advantageously exactly one external unit 28a on. The external entity 28a is especially outside of the household appliance 12a arranged and / or separate from the household appliance 12a educated. In the present embodiment, the external unit 28a in particular as an, in particular external, server. Alternatively or additionally, the external unit could 28a in particular be designed as a mobile device. In particular, the control unit 26a and the external entity 28a for, in particular wireless, communication (cf. 1 and 2nd ).

The system 10a has in particular at least one and advantageously exactly one memory unit 30a on. The storage unit 30a is especially outside of the household appliance 12a arranged and / or separate from the household appliance 12a educated. The external entity 28a has in particular the memory unit 30a on. The storage unit 30a is particularly part of the external unit 28a and / or at least to a large extent and advantageously completely in the external unit 28a integrated.

The system is advantageous 10a , in particular by means of the control unit 26a , provided for object detection. The control unit is in particular for object detection 26a in particular to carry out a, in particular based on the 2nd to 9 described, method for improving object detection through machine learning.

The system 10a has, in particular for object detection, at least one and advantageously exactly one sensor unit 32a on (cf. 3rd ). The sensor unit 32a is particularly within the household appliance 12a arranged. In particular, the sensor unit 32a especially part of the household appliance 12a and / or at least to a large extent and advantageously completely in the sensor unit 32a integrated. In particular, the household appliance 12a the sensor unit 32a on.

The sensor unit 32a is particularly useful for detection of at least one image 34a at least one object 36a , especially within the household appliance room 14a , intended. In particular for the detection of image acquisition 34a points the sensor unit 32a in particular at least one camera, advantageously at least a plurality of and preferably a plurality of cameras on (not shown), which, in particular, are distributed, in particular within the domestic appliance useful space 14a are arranged.

The sensor unit 32a is especially for control by the control unit 26a intended. In particular, the control unit controls and / or regulates 26a the sensor unit in particular in at least one operating state and advantageously for carrying out the method 32a .

The control unit 26a performs the method, in particular in at least one operating state. The method is particularly intended to improve object recognition through machine learning. The method is advantageous for improving object detection, in particular objects 36a within the home appliance room 14a , provided by machine learning.

In the process, machine learning, especially by the control unit 26a , especially many different and / or the same images 34a marked objects 36a evaluated in an object category. In particular, in particular by the control unit 26a , many different image recordings for a specific object category in machine learning 34a marked objects 36a evaluated this object category.

In the method, the object category, in particular before an object is identified 36a , predefined, and advantageously manually. The object category is marked by marking the objects 36a this object category, in particular by means of at least one marker 38a , predefined.

In the present embodiment, the objects 36a especially marked manually. The objects 36a can be marked manually in different ways. In the following two of these types are presented together for reasons of scarcity. When the method is carried out, in particular real, there is in particular one, in particular only, type of marking 38a sufficient.

In particular, the objects 36a by positioning the objects 36a in at least one and advantageously exactly one specific area assigned to the object category 42a marked. The specific area assigned to the object category 42a in particular by marking an installation unit 44a of the household appliance 12a marked.

The home appliance 12a has in particular at least one and advantageously at least the set-up unit 44a on (cf. 2nd and 3rd ). In the present embodiment, the household appliance 12a especially several installation units 44a , of which in particular only one is described below. The erection unit 44a could, for example, at least one, especially on the appliance door 18a arranged, door racks (see. 2nd and 9 ). For example, the erection unit 44a at least one, especially within the household appliance room 14a and especially on the device body 16a arranged, have shelf (cf. 2nd ). The method is described below in particular on a shelf of the installation unit 44a described. A description in the case of a door rack of the installation unit 44a proceeds in particular analogously to this.

To mark the specific area assigned to the object category 42a in particular on the installation unit 44a a marker 38a attached, advantageously manually. Alternatively, the marker could be 38a in particular predefined on the installation unit 44a be attached, for example at the factory. The objects 36a are in particular by, in particular manual, positioning in the specific area assigned to the object category 42a marked and in particular assigned to the object category.

In particular, alternatively or in addition to the marking by positioning the objects 36a , the objects 36a especially by manually placing at least one marker 38a on the object 36a be marked. The mark 38a is, especially manually, on the object to be marked 36a attached itself.

In the present exemplary embodiment, the marking has 38a at least one icon. In the case of the “milk” object category described in the present exemplary embodiment, the symbol could be a cow, for example.

Especially after marking the object 36a , is in the process, in particular by the control unit 26a and advantageously by means of the sensor unit 32a , take a picture 34a recorded (cf. 3rd and 4th ). On the image acquisition 34a is in particular the specific area assigned to the object category 42a velvet in the area 42a arranged objects 36a and especially additionally in the area 42a arranged further objects shown. In particular, in the method, in particular by the control unit 26a and advantageously by means of the sensor unit 32a , when taking the picture 34a one of the objects 36a a shot of at least the object 36a and the object 36a assigned marker 38a added.

In the, especially recorded, image recording 34a is, in particular by the control unit 26a , the mark 38a recognized. In particular, in particular by the control unit 26a , the object 36a depending on the marking 38a assigned to the, in particular certain, object category. The mark 38a is in particular for an assignment of the with the marking 38a provided object 36a provided for an object category.

In particular, the marking 38a , in particular additionally, provided for an area to be cut out 46a to mark. In the process, in particular, the image is captured 34a by cutting out at least one through the marking 38a marked area 46a an image 40a of the object 36a generated, in particular for later storage in the external unit 28a and advantageous in the storage unit 30a .

Starting from the, especially in 4th illustrated, image acquisition 34a is, in particular by the control unit 26a , in particular by cutting out at least one through the marking 38a marked area 46a a, especially in 5 depicted, image 40a of the object 36a generated. An image processing processed in this way is used in particular in the method 34a of the object 36a especially in the form of an image 40a of the object 36a in the storage unit 30a stored, especially for later identification (cf. 6 ).

In particular, in the method, in particular by the control unit 26a and advantageously by means of the sensor unit 32a , different pictures 34a of different objects 36a the same object category, in particular in different time intervals when using the household appliance 12a . The particularly recorded images 34a of different objects 36a the same object category, especially by the control unit 26a , especially for later identification of an object 36a this object category, especially in the storage unit 30a , saved.

In particular, in the method, in particular by the control unit 26a and advantageously by means of the sensor unit 32a , different pictures 34a from the same object 36a recorded, in particular in different time intervals of use of the household appliance 12a and advantageously in different orientations relative to the sensor unit 32a and / or in different positions relative to the sensor unit 32a . The particularly recorded images 34a from the same object 36a be, especially by the control unit 26a , especially for later identification of an object 36a of the same type, especially in the storage unit 30a , saved.

In particular, in the method, in particular by the control unit 26a and advantageously by means of the sensor unit 32a , Picture taking 34a of different objects 36a the same object category, in particular in different time intervals when using the household appliance 12a . Different objects 36a The same object category could, for example, be in a packaging and / or in a shape and / or in a size and / or in a material property, such as fat content and / or sugar content and / or degree of maturity and / or expiry date, and / or in a manufacturer differentiate (cf. 7 ). The particularly recorded images 34a of different objects 36a the same object category, especially by the control unit 26a , especially one later identification of an object 36a the same object category, especially in the storage unit 30a , saved.

In the process, in particular by the control unit 26a , especially image recordings 34a of different objects 36a the same object category in different household appliance rooms 14a recorded (cf. 8th ). In particular, the respective image recordings 34a , especially by the control unit 26a , one image each 40a of an object 36a by cutting out at least one through the marking 38a marked area 46a the image acquisition 34a generated. A respective one, in particular from the image acquisition 34a generated, image 40a of a particular object 36a is in particular in a common storage unit 30a , advantageous in the storage unit 30a , which is stored, in particular, at least in large part and advantageously completely in the external unit 28a is integrated.

In an alternative embodiment, a respective control unit could 26a which, in particular in the present exemplary embodiment, at least to a large extent and advantageously completely in the corresponding household appliance 12a is integrated, in particular at least in large part and advantageously completely in the external unit 28a integrated and especially for all household appliances 12a be provided together (not shown).

In particular, in particular alternatively or in addition to the creation of an image 40a of an object 36a by cutting out at least one through the marking 38a marked area 46a the image acquisition 34a who have favourited picture taking 34a as such in the storage unit 30a are saved, in particular because any partially existing parts of other objects in this image recording 34a are usually not distracting, as they are particularly in contrast to the actual object 36a are statistically highly variable and can be used as a background in particular.

A, in particular temporal, sequence of the method is in particular in 10th shown. In particular, in an object marking step 48a an object 36a one, in particular certain, object category. The object 36a is in particular by means of the marking 38a marked as such, in particular by attaching the marking 38a on the object 36a . The, especially marked, object 36a is particularly in a contribution step 50a in the household appliance room 14a , in particular in any area of the household appliance room 14a , introduced and advantageous in the home appliance room 14a , in particular in any area of the household appliance room 14a , positioned.

In an alternative, in particular temporal, sequence of the method could, for example, in an area marking step 52a at least one specific area assigned to an object category 42a be marked. In particular, the specific area assigned to the object category could 42a by attaching the mark 38a on one, especially the sensor unit 32a facing section of the area 42a . In particular for marking an object, in particular to be marked 36a , becomes the object to be marked in particular 36a especially in a positioning step 54a in the home appliance room 14a , in particular in the specific area assigned to the object category 42a , positioned.

In one step 56a is, in particular by the control unit 26a and advantageously by means of the sensor unit 32a , at least one picture 34a of the marked object 36a added. In particular, in the admission step 56a when taking the picture 34a of the object 36a a shot of at least the object 36a and the object 36a assigned marker 38a added.

In particular, in an assignment step 58a , especially by the control unit 26a , in the image acquisition 34a the mark 38a recognized and the object 36a depending on the marking 38a assigned to an object category. In one processing step 60a is, in particular by the control unit 26a , especially an image 40a of the object 36 generated. The image 40a of the object 36a is in the processing step 60a , especially by the control unit 26a , in particular by cutting out at least one through the marking 38a marked area 46a the image acquisition 34a generated.

In one save step 62a is, in particular by the control unit 26a , the image 40a especially in the storage unit 30a which particularly in the external unit 28a is integrated, saved.

In particular, in order to achieve a further improvement in object recognition through machine learning, the specific area assigned to an object category could, for example 42a be designed such that a defined and / or standardized background is the corresponding object 36a of other objects 36a and / or of other objects, which are particularly evident in the image recording 34a could be located. This could be done, for example, by a household appliance designed in particular as a refrigerator 12a known white background. For example, at least one segmentation technique, such as in particular a segmentation technique known as “color based segmentation”, could allow the object to be corresponding 36a cut out and / or identify in an automated and pixel-precise manner, especially against this background.

In particular, there can be an improvement in object recognition through machine learning with regard to customized objects 36a be made possible, in particular by an operator manually entering at least one object name, for example by means of at least one operator interface and / or by means of at least one application, and in particular subsequently the control unit 26a enables, in particular by means of the sensor unit 32a , especially a sufficient number of pictures 34a of the object 36a and / or images 40a of the object 36a from different perspectives. This can be done, for example, by normal and / or everyday use of the household appliance, in particular a refrigerator 12a respectively. In particular, the object 36a in different orientations relative to a reference point and / or relative to the sensor unit 32a in the marked area 42a be put. The control unit 26a can in particular recognize the orientations presented in this way, as well as variations thereof, and in particular objects in a later identification 36a to recognize and / or identify a respective operator.

In particular, a constant improvement in object recognition can be achieved through machine learning, in particular through continuous acquisition of image recordings 34a and storage of images 40a , in particular also for those already in the database and / or in the storage unit 30a stored training data of at least one object 36a . This allows the training data of the object 36a which, in particular, are already in the database and / or in the storage unit 30a can be stored, in particular can be expanded to include additional regional characteristics and / or changing packaging. As an alternative or in addition, this can in particular improve the accuracy of an identification.

Reference symbol list

10th

system

12

Home appliance

14

Household appliance room

16

Device body

18th

Appliance door

20th

More appliance door

22

Vertical direction

24th

Footprint

26

Control unit

28

External unit

30th

Storage unit

32

Sensor unit

34

Image acquisition

36

object

38

mark

40

image

42

Area

44

Installation unit

46

Area

48

Object marking step

50

Submission step

52

Area marking step

54

Positioning step

56

Admission step

58

Assignment step

60

Processing step

62

Storage step

Claims (15)

Method for improving object recognition, in particular objects (36a) within a domestic appliance useful space (14a), by machine learning, with machine learning evaluating many image recordings (34a) of marked objects (36a) of an object category. Procedure according to Claim 1 , characterized in that the object category is predefined. Procedure according to Claim 1 or 2nd , characterized in that the objects (36a) are marked manually. Method according to one of the preceding claims, characterized in that the objects (36a) are marked by positioning in at least one specific area (42a) assigned to the object category. Method according to one of the preceding claims, characterized in that the Objects (36a) are marked by manually applying at least one marking (38a) to the object (36a). Method according to one of the preceding claims, characterized in that when the image recording (34a) of one of the objects (36a) is taken, a recording of at least the object (36a) and the marking (38a) assigned to the object (36a) is recorded. Procedure according to Claim 6 , characterized in that the marking (38a) is recognized in the image recording (34a) and the object (36a) is assigned to the object category depending on the marking (38a). Procedure according to Claim 6 or 7 , characterized in that an image (40a) of the object (36a) is produced by cutting out at least one region (46a) of the image recording (34a) identified by the marking (38a). Method according to one of the preceding claims, characterized in that different image recordings (34a) of different objects (36a) of the same object category are recorded and stored for later identification of an object (36a) of this object category. Method according to one of the preceding claims, characterized in that different image recordings (34a) of the same object (36a) are recorded and stored for later identification of an object (36a) of the same type. Method according to one of the preceding claims, characterized in that image recordings (34a) of different objects (36a) of the same object category are recorded and stored for later identification of an object (36a) of the same object category. Method according to one of the preceding claims, characterized in that image recordings (34a) of different objects (36a) of the same object category are recorded in different household appliance use rooms (14a) and stored in a common storage unit (30a). System for object detection, with a control unit (26a), which at least according to an implementation of the method Claim 1 is provided. System according to Claim 13 , characterized by at least one household appliance (12a) which has the control unit (26a). Household appliance, in particular household refrigeration appliance, according to the system (10a) Claim 14 .

Images