EP3152498A1 - Gargerät mit lichtmusterprojektor und kamera - Google Patents
Gargerät mit lichtmusterprojektor und kameraInfo
- Publication number
- EP3152498A1 EP3152498A1 EP15727630.4A EP15727630A EP3152498A1 EP 3152498 A1 EP3152498 A1 EP 3152498A1 EP 15727630 A EP15727630 A EP 15727630A EP 3152498 A1 EP3152498 A1 EP 3152498A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- light pattern
- cooking
- camera
- cooking appliance
- cooking chamber
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000010411 cooking Methods 0.000 title claims abstract description 157
- 235000013305 food Nutrition 0.000 claims description 72
- 238000011156 evaluation Methods 0.000 claims description 29
- 230000003287 optical effect Effects 0.000 claims description 13
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000000034 method Methods 0.000 description 20
- 238000005259 measurement Methods 0.000 description 16
- 230000008859 change Effects 0.000 description 11
- 230000008569 process Effects 0.000 description 11
- 235000015895 biscuits Nutrition 0.000 description 8
- 238000001816 cooling Methods 0.000 description 6
- 238000011161 development Methods 0.000 description 6
- 238000010438 heat treatment Methods 0.000 description 6
- 230000008901 benefit Effects 0.000 description 5
- 238000001514 detection method Methods 0.000 description 5
- 238000003780 insertion Methods 0.000 description 4
- 230000037431 insertion Effects 0.000 description 4
- 230000005855 radiation Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 239000011521 glass Substances 0.000 description 2
- 235000012054 meals Nutrition 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 235000008429 bread Nutrition 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000004907 flux Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000001678 irradiating effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 235000013372 meat Nutrition 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000007493 shaping process Methods 0.000 description 1
- 235000012976 tarts Nutrition 0.000 description 1
Classifications
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F24—HEATING; RANGES; VENTILATING
- F24C—DOMESTIC STOVES OR RANGES ; DETAILS OF DOMESTIC STOVES OR RANGES, OF GENERAL APPLICATION
- F24C7/00—Stoves or ranges heated by electric energy
- F24C7/08—Arrangement or mounting of control or safety devices
- F24C7/081—Arrangement or mounting of control or safety devices on stoves
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F24—HEATING; RANGES; VENTILATING
- F24C—DOMESTIC STOVES OR RANGES ; DETAILS OF DOMESTIC STOVES OR RANGES, OF GENERAL APPLICATION
- F24C7/00—Stoves or ranges heated by electric energy
- F24C7/08—Arrangement or mounting of control or safety devices
- F24C7/082—Arrangement or mounting of control or safety devices on ranges, e.g. control panels, illumination
- F24C7/085—Arrangement or mounting of control or safety devices on ranges, e.g. control panels, illumination on baking ovens
Definitions
- the invention relates to a cooking appliance comprising a cooking chamber with a feed opening closable by means of a door, a light pattern projector firmly arranged with respect to the cooking space for generating a light pattern, a camera for taking images of an area which can be illuminated by the light pattern and one with the Camera coupled evaluation for calculating a three-dimensional shape of at least one object, which is located in the irradiated by the light pattern area, by means of a light pattern evaluation.
- the invention is particularly advantageously applicable to furnaces.
- the invention is particularly advantageously applicable to household appliances.
- EP 2 530 387 A1 discloses an oven with a device for detecting a three-dimensional shape of food on a baking tray of the oven.
- the device contains at least one laser, which is arranged or can be arranged above a cooking chamber of the oven. A laser beam of the laser is directed downwards.
- the device further comprises at least one camera, which is arranged or can be arranged above a baking tray of the oven. The camera is arranged or can be arranged in a front section of the oven.
- the baking sheet and the camera are mechanically coupled, so that the camera and the baking sheet are synchronously movable.
- An upper side of the baking sheet is located in a field of view of the camera.
- An angle between a central axis of a field of view of the camera and the laser beam is predetermined. Further disclosed is the apparatus for detecting the three-dimensional shape of the food on the baking tray.
- EP 2 149 755 A1 discloses an oven for heating food products comprising a cooking chamber to receive the product via a feed opening and a product feature extraction system designed to extract at least one product feature representative of a configuration of the product the system comprising: at least one camera adapted and arranged to receive plan views of the product, and at least one contour plane unit adapted to extract or highlight contour planes of at least a portion of the product and, as the case may be , an object that is ready for insertion provided with the product in the cooking chamber, and a Marinmerkmais extraction unit for extracting the at least one product feature on the basis of the top view of the product and contour levels of the product.
- a method of operating a furnace for heating a food product comprising the steps of: a) extracting a product feature of a product to be heated in a chamber of the oven by picking up at least one top view of a product by means of at least one camera, extracting and / or Highlighting contour planes of at least a portion of the product and, as the case may be, an object intended to be inserted into the cooking cavity along with the product, using at least one contour level unit, and b) extracting the at least one a product feature based on the plan and contour levels, based on at least one product feature, and optionally secondary data representing a physical configuration of the product, preferably at least one of product temperature, product weight, and product density, for automatic control or for heating of the product.
- EP 1 921 384 A1 discloses a device for determining the temperature inside a food.
- the device has at least one temperature sensor for detecting at least one surface temperature of the food and / or an ambient temperature of the food, in particular at a measuring location within a cooking chamber surrounding the food, preferably with an ambient temperature sensor arranged at the measuring location.
- the device comprises at least one distance sensor for detecting one or a plurality of distances between the distance sensor on the one hand and one or a plurality of distance measuring points on the surface of the food on the other hand.
- the device comprises at least one time measuring device for detecting the time during preparation of the food and at least one calculating device for calculating the temperature inside the food from the surface temperature of the food and / or ambient temperature, the distance or the plurality of distances, the time and a Initial temperature of the food. Further disclosed is a method for determining the temperature inside a food.
- DE 197 48 062 A1 discloses a method and a device for the three-dimensional, optical measurement of objects. According to this, the measuring system must be calibrated in the case of optical, areal-working, three-dimensional measuring methods, since the geometric parameters of the system for carrying out the triangulation calculation must be known.
- the method makes it possible to adjust the measuring system to a different measuring field size even after calibration.
- the measuring system is adapted to different measuring field sizes such that the geometric changes performed on the system can be precisely determined and the for the Triangulation authoritative parameters can be calculated without recalibration.
- the calibration is now carried out with a measuring field size, which is selected solely under the aspects of favorable manufacturability of the calibration device and easily manageable dimensions. The once calibrated system can then be set to a wide range of measuring distances and measuring volumes, in particular very large ones.
- WO 00/70303 discloses a method and apparatus for imaging three-dimensional objects comprising a pattern light source that projects a focused image onto an object by passing light through an optical grating and a downstream projection lens, either continuously or stroboscopically. An application to household appliances or cooking appliances is not disclosed.
- DE 10 2006 005 874 A1 discloses a device and a method for non-contact measurement of in particular cylindrical objects on surfaces. For this purpose, it is proposed to generate a line on the surface with the aid of a laser whose reflection is measured by a camera. After recording the line, it is moved several times parallel to itself and the recording is repeated. In this way, by sequentially shifting the line, a shadow image of the object arranged on the surface is generated. It is also possible to separate the multi-line triangulation and the shadowing from each other. For multi-line triangulation, a fixed laser or other radiation source may be used. The Shadowing can be performed by two equally stationary radiation sources, for example a series of LEDs, simultaneously or sequentially. By using a fixed structure of radiation sources and camera simplifies and reduces the mechanical structure. An application to household appliances or cooking appliances is not disclosed.
- a cooking device comprising a cooking chamber with a feed opening closable by means of a door, (at least) a fixed projector with respect to the cooking space (hereinafter referred to as "light pattern projector" without restriction of generality) for generating a light pattern, at least) a camera for capturing images from an area that can be irradiated by the light pattern and an evaluation device coupled to the camera for calculating a three-dimensional shape of at least one object that is in the area that can be irradiated by the light pattern, by means of a light pattern evaluation, wherein the light pattern projector for irradiating a light pattern is arranged in the cooking chamber, the camera is arranged fixedly in particular with respect to the cooking chamber, the camera arranged to record images from an irradiated by the light pattern region of the cooking chamber even when the cooking chamber is closed i st and the evaluation device for the repeated calculation of the three-dimensional shape of the at least one object, which is located in the area of the cooking chamber
- the cooking appliance has the advantage that the depth information can serve as a parameter for automatic programs.
- a possible change in volume of the cooking product during the cooking process during operation of the cooking appliance can be detected and can influence the control of the cooking parameters, for example a cooking chamber temperature. So for example, a Targeh a bread and a shrinkage behavior of a piece of meat detected and possibly used to control the cooking appliance.
- the basically known method of patterned or structured light (“structure light”) is used Due to the degree of deformation of the light pattern on the object, a three-dimensional model of this object can be calculated by means of the evaluation device A theoretical optimum of the resolution would be at as large an angle as possible, but with increasing approximation to this optimum, e will deteriorate Ine visibility of the projected light pattern on the object surface and thus its detectability by the camera. A position determination is possible only for those points which are visible on the one hand from the camera and on the other hand can be illuminated by the light pattern projector (ie not lying in a shadow).
- the cooking appliance may be an oven or have such a furnace, in particular an oven.
- the cooking chamber may then also be referred to as oven room.
- the oven may be a standalone oven or part of an oven / hob combination or cooker.
- the furnace may additionally or alternatively have an embodiment as an oven microwave and / or steam treatment functionality.
- the cooking appliance is a household appliance, in particular in the sense of 'white goods'.
- the light pattern projector together with the camera and the evaluation device, may also be referred to as a so-called "3D scanner.”
- the light pattern projector emits at least one light pattern, eg a stripe and / or dot pattern, but is not limited thereto other arbitrary patterns of light are generated, eg annular shaped patterns, wave patterns, etc.
- a pattern is particularly chosen so that it matches the desired resolution of the three-dimensional image.
- the camera may be a digital camera. It may take individual pictures and / or picture sequences, in particular videos.
- the evaluation device may be an independent device of the cooking appliance, e.g. in the form of electronics, in particular on a separate board. It may alternatively be integrated in another device of the cooking appliance, e.g. in a central control facility. This further device may then in particular be able to carry out the evaluation in addition.
- an optical axis of the light pattern projector and an optical axis of the camera are at an angle between 20 ° and 30 ° to each other.
- the light pattern projector and the camera are arranged behind a wall or muffle of the cooking chamber, in particular in a predefined distance.
- these two components can be thermally insulated sufficiently thermally against the cooking chamber.
- the cooking chamber wall may have a respective window for the light pattern projector and for the camera.
- the windows may be covered with transparent glass.
- the light pattern projector and the camera are arranged behind a ceiling of the cooking chamber.
- a food support eg a baking sheet or a grid
- This position has the further advantage that cooling air conducted via the ceiling (eg for cooling electronics arranged above the cooking chamber) can also be used for cooling the light pattern projector and the camera.
- the distance between the light pattern projector and the camera behind the oven wall or muffle can be muffle-specific.
- different light patterns can be introduced into the cooking chamber by means of the light pattern projector. As a result, a determination of the three-dimensional shape of the at least one object can be carried out with a particularly small error.
- alternating punctiform and stripe-shaped light patterns can be irradiated and evaluated.
- different dot patterns and / or different stripe patterns can be radiated into the oven. This can be done in a predetermined sequence or if a measured depth resolution does not provide sufficient results.
- the light pattern projector has at least one picture-like screen or screen for shaping the light pattern.
- the pixel-like screen may e.g. a liquid crystal screen or LCD screen.
- the pixel-like screen may itself generate light as a structural unit to sufficiently irradiate the cooking space with the light pattern.
- the pixel-like screen may be e.g. be backlit by at least one separate light source so that it can be used as 'variable aperture'. The latter case enables particularly high luminous fluxes.
- the light emitted by the light pattern projector and received by the camera may be visible light and / or infrared light.
- the advantage of the infrared light is that a viewer looking into the cooking chamber does not see the light pattern. It is a development that the 3D scanner can be calibrated. It is an embodiment of at least one calibration mark in the muffle. On the basis of the known position of the at least one calibration mark relative to the at least one object to be measured, a distance of the object and thus also its size or shape can be determined more accurately.
- At least one calibration mark on a food support eg on a baking tray or a grid, etc.
- a calibration mark on a food support may be located in particular on a utility or storage surface of the food support for food.
- this calibration mark may be have a known size, so that can be determined by means of the size recorded by the camera, a distance to the camera.
- a calibration mark may be a colored mark and / or a predetermined shaped mark.
- the calibration marks may also be defined geometric features, e.g. Functional areas of the cooking chamber wall or muffle such as insertion projections.
- the calibration mark (s) may also serve to determine at which level the object to be measured is located.
- the calibration takes place in the closed muffle or in the closed cooking space, in particular at the beginning of a cooking process. This minimizes any influence on the measurement from the environment.
- it is advantageous to specify preferred shelves in the 3D-measuring cooking space ("3D-scan"), preferably in a lower third of the muffle has the advantage that even large-scale and / or large-volume objects can be reliably detected and measured.
- the 3D measurement of the object takes place advantageously after the calibration, but in principle, the calibration may also be dispensed with.
- the cooking appliance is equipped with a Einschuberkennung.
- the cooking appliance may then, when it detects that a food support is on an unfavorable for a 3D measurement insertion level, a hint signal and / or a display to issue a user. Also, the cooking device may then prevent a 3D scan.
- the evaluation device is adapted to recognize a kind of a food. This allows, inter alia, an automatic adaptation of cooking parameters to the food (eg as part of a cooking program) and / or an adjustment of a user guidance to the food (eg by displaying suitable for the detected food Garparametern and / or cooking programs).
- the cooking appliance is set up to perform a product recognition on the basis of an image analysis of images taken by the camera (without SD measurement, also referred to as "image recognition" hereinafter)
- the product identification on the basis of the 3D scan can additionally or A product recognition based on a combined image recognition and a 3D measurement enables a higher probability of recognition through the additional height or depth information, which may for example enter into an image recognition algorithm as information,
- the evaluation device is adapted to This information can be used by the cooking appliance, for example, to set or adjust cooking parameters, for example, a heat output of a top and / or bottom heat or a Activation and / or a setting of a heating capacity of a convection heating.
- the evaluation device is set up to recognize a type of accessory, in particular cooking utensils, for example, a roaster or the like stored on a food support, in which the food to be cooked is located. So it may be measurable whether the food is in an open roasting pan or the roasting pan is closed. This information may also be used by the cooking appliance to set or adjust cooking parameters, optionally including the option of choosing the cooking method used. If the roaster is closed, the cooking appliance may rely on an input by the user regarding the type of content.
- a type of accessory in particular cooking utensils, for example, a roaster or the like stored on a food support, in which the food to be cooked is located. So it may be measurable whether the food is in an open roasting pan or the roasting pan is closed. This information may also be used by the cooking appliance to set or adjust cooking parameters, optionally including the option of choosing the cooking method used. If the roaster is closed, the cooking appliance may rely on an input by the user regarding the type of content.
- the evaluation device is adapted to detect a core temperature of an object.
- the core temperature can be calculated by correlating with a volume change determined by the 3D scan during a cooking process, knowing the type of food to be cooked. For example, one may dispense with an independent core temperature sensor or a roasting spit. It is preferred for a particularly high reliability of the determination of the core temperature development that the food has a nearly homogeneous structure.
- the core temperature is thus determined in the present case by means of a 3D scan, not as described in EP 1 921 384 A1 by means of one or more distance sensors.
- the evaluation device is coupled to a control device of the cooking appliance and the control device is set up to adjust an operation of the cooking appliance on the basis of at least one object parameter determined by the evaluation device.
- the associated object can, as already mentioned above, be food, an accessory and / or a food support.
- Object parameters can be eg a position, shape, volume or type etc. of the object.
- the 3D information determined by one or more 3D scans can be used in particular for automation of cooking, cooking and baking processes. As already mentioned, it is possible to perform the 3D scan during a cooking process. The then determined 3D information or 3D data not only use the Garholderkennung, but also a possible adjustment of the cooking parameters. Thus, a cooking process or cooking process can be individually adapted to the food. If accurate detection of the food is not possible, input from a user can be taken into account. For this purpose, the cooking appliance may ask the user to enter more information about the food in the cooking appliance.
- the cooking appliance has a screen on which at least one three-dimensional image of at least one object recorded by the camera can be displayed.
- a three-dimensional representation of the contents of the cooking chamber is offered on a screen. This allows a particularly informative presentation of information for a user.
- the screen may e.g. be present on a front side or an upper side of the cooking appliance.
- the screen may be a touch-sensitive touch screen or touch screen.
- One way to operate the cooking appliance is to perform a calibration before starting a meal treatment (e.g., a cooking cycle). Furthermore, shortly before the beginning of a food treatment, a 3D measurement or a 3D scan of the food can be carried out in order to detect its initial geometry. Also, prior to the start of a meal treatment, object recognition may be performed with respect to a type of the item to be cooked, a type of accessory and / or a kind of a product carrier.
- At least one 3D scan may also take place during the food treatment, in particular several 3D scans in eg periodic intervals.
- the detected shape change of the food for example, a product detection, a detection of a treatment end and / or a determination of a core temperature may be carried out.
- the light pattern projector is also provided for the illumination of the cooking chamber. For example, it may illuminate the cooking chamber for the view of a user and irradiate the light pattern into the cooking chamber only for comparatively short periods in between. So can be dispensed with a separate light source for Garraumbeleuchtung.
- FIG. 2 shows a sketch of a reconstruction of a shape of a mold by means of an SD
- FIG. 3 shows a sectional side view of a cooking appliance equipped with a 3D scanner according to the invention.
- FIG. 4 shows a diagram of a profile of a temperature and a volume of a heated object with a cooking product determination by means of a cooking appliance according to the invention.
- 1 shows an arrangement ("3D scanner") for determining a three-dimensional shape of at least one object O (“3D scanner”), comprising a light pattern projector 1 directed onto the object, a camera 2 directed towards the object O, a control device C for operating the light pattern projector 1 and for calculating a three-dimensional shape of the object O on the basis of at least one image received by the camera 2 by means of a light pattern evaluation.
- a screen 3 for viewing the object O 'calculated by the controller C is provided.
- the light pattern projector 1 generates a predetermined light pattern L, eg, a stripe or dot pattern.
- the light pattern projector 1 emits its light in a light beam having a first optical axis A1.
- the camera 2 typically a digital camera, has a field of view F with a second optical axis A2, which is oriented obliquely to the first optical axis A1 of the light pattern projector 1.
- the camera 2 is oriented obliquely to the light pattern projector 1. It regards a region of the object O irradiated or irradiated by the light pattern L.
- the light pattern projector 1 here has a light source Q, e.g. a field of LEDs, which is followed by a pattern generating element in the form of a transparent, freely programmable LCD surface D.
- a pattern generating element in the form of a transparent, freely programmable LCD surface D.
- a corresponding, in particular complementary, light pattern L is radiated from the LCD surface D.
- an LED screen may already serve as the light source (not shown), in which case a separate light source may be dispensed with because of the backlighting integrated therein.
- FIG. 2 shows by way of example how light in the form of a vertical column or line G is radiated from the light pattern projector 1 onto the object O.
- On the object O thus appears a distorted by the surface contour of the object O projection P (G) of this line G.
- the camera 2 takes due to their inclination relative to the light pattern projector 1 an image of this projection P (G), which reproduces the distortion.
- the camera 2 stores the projection P (G) as correspondingly positioned pixels B or "pixels" of a matrix resulting from a matrix-type construction of individual sensors of a sensor array S of the camera, for example a CCD sensor array given by the deviation of the pixels B from a vertical line.
- a light beam r can be specified in space, from which the light incident on the respective individual sensor comes, and if, in addition, an assignment of the pixels to the projection P (G) visible from the pixels and thus also to the corresponding lines G exists, then points on the object surface can be distinguished by a simple beam path. Reconstruct level intersection.
- the depth resolution depends on an angle W between the light beam r leading to the pixel B and the direction of the column or line G.
- a theoretical optimum with respect to the resolution would be at as large an angle W as possible.
- the visibility of the projection P (G) on the surface of the object O and hence its detectability in the camera image deteriorates. Since a reconstruction is possible only for those points on the surface of the object O which are visible on the one hand by the camera 2 and on the other hand can be illuminated by the light pattern projector 1, a compromise is made here.
- Such a 3D measurement or 3D scan is fundamentally known and will therefore not be discussed further below.
- FIG. 3 shows a sectional side view of a cooking appliance equipped with a 3D scanner in the form of a baking oven 4.
- the oven 4 has a cooking chamber 6 bounded by a oven muffle 5.
- the oven muffle 5 has at the front a closable by means of a oven door 7 feed opening 8, through which objects, in particular in the form of food to be cooked 01 in the cooking chamber 6 can be spent.
- a cooking chamber temperature T is adjustable by means of one or more, in particular electrically operable, radiator (o. Abb.).
- On a ceiling 9 of the oven muffle 5 are two viewing windows 10 and 1 1, which may be covered with transparent glass panes, for example.
- a light pattern projector 1 eg with a LCD display for pattern generation
- a camera 2 behind the viewing window 1 1 a camera 2.
- These are thermally protected by their distance from the oven muffle 5.
- the light pattern projector 1 and the camera 2 can be further cooled by this cooling air flow.
- the light pattern projector 1 and the camera 2 are laterally offset from one another.
- the light pattern projector 1 and the camera 2 are fixedly arranged with respect to the cooking chamber 6 and thus do not move with, for example, upon actuation of the oven door 7.
- the light pattern projector 1 radiates through the viewing window 10 a light pattern L in the cooking chamber 6, in such a way that from a predetermined distance from the ceiling 9 practically the entire horizontal surface of the cooking chamber 6 with the light pattern L can be illuminated. This may be e.g. in a lower half or in a lower third of the cooking chamber 6 be the case.
- the camera 2 takes pictures from an at least partially irradiated by the light pattern region of the cooking chamber 6
- the oven 4 further has an evaluation device 12 coupled to the camera 2 for calculating a three-dimensional shape, for example of the food 01 and a food support 02, which are located in the area that can be illuminated by the light pattern L, by means of a light pattern evaluation. This is based on a 3D measurement on the basis of at least one image taken by the camera 2.
- the light pattern projector 1, the camera 2 and the evaluation device 12 together form the 3D scanner.
- the evaluation device 12 may, as shown here, be functionally integrated in a central control device of the oven 4 or may be coupled as a separate unit with a control device.
- a 3D scan including e.g. a recording of an image of the projection P (G) of the light pattern L by means of the camera 2 and from this a calculation of the three-dimensional shape of the food 01 and the food support 02, can be carried out with the oven door 7 or loading opening 8 closed.
- a calibration can be performed.
- the food support 02 has on its upper side one or more eg colored calibration marks K, which have a known size and can be easily identified.
- the size of the calibration marking (s) K recorded by the camera 2 can be used to distance it from the ceiling 9 and thus, for example, to use the insertion markers used. identify the level.
- too high shelf level oven 4 may give an indication to a user, such as a display on a front screen 3 a control panel 13.
- At least one calibration mark may also be located on the oven muffle 4.
- an initial 3D measurement of the food 01 to calculate its original shape may be displayed in the screen 3.
- the calculated shape may be used to determine the food to be cooked 01 by the cooking appliance 4, in particular in addition to an image recognition of the food to be cooked by the camera 2.
- one type of the food support 02 may also be used be recognized.
- one or more cooking parameters of the cooking cycle may be adjusted.
- a cooking time and / or the cooking space temperature T can be adjusted.
- the oven 4 may adjust the cooking cycle, e.g. change the cooking time and / or the oven temperature, including turning off the heaters.
- the light pattern projector 1 may irradiate different light patterns L into the cooking space 6.
- FIG. 4 shows a diagram of a profile of a temperature T and a volume V of the food to be cooked 01 with a Gargutbeées by means of example of the cooking device. 4
- the food 01 is purely exemplary of a dough product, such as a round tarte flambée or a biscuit in a Springform. Only by means of an image recognition by the camera 2, a distinction between these two types of food 01 not In a two-dimensional view from above (top view), both the tarte cake and the biscuit look circular. In addition, both are similar in color. However, both types of food require a different specific baking environment. Treating the tarte cake like the biscuit gives an unsatisfactory result, which also applies to the reverse case. By means of the SD measurement by means of the 3D scanner 1, 2, 12, the cooking appliance 4 additionally receives spatial information about the food 01.
- This spatial information of the initial state of the food to be cooked 01 before the cooking process may already be sufficient to distinguish the flat tart from the higher biscuit.
- the type of food to be cooked 01 may be determined by means of the 3D scanner 1, 2 12 from the change of its shape, in particular a change AV of its volume V.
- the cooking chamber temperature T has an initial value Ts at an initial time ts of the cooking cycle, e.g. Room temperature. With continuous time t, the cooking chamber temperature T increases due to at least one activated heating, as shown by the curve T1 + T2 equal for Tarte and biscuit. If the cooking chamber temperature T reaches a desired temperature Td1 for biscuit at a time td which is below a target temperature Td2 for tarte cake, a further 3D measurement is carried out in order to determine the type of the food item 01.
- the cooking appliance 4 can subsequently increase, for example, its cooking chamber temperature T to the associated desired value Td2, as indicated by the temperature curve T2.
- the cooking process ends at an associated end time te2.
- the cooking appliance 4 may in the following maintain its cooking chamber temperature T at the associated desired value Td1, as indicated by the temperature curve T1.
- the cooking process ends at an associated end time te1.
- a number may include exactly the specified number as well as a usual tolerance range, as long as this is not explicitly excluded.
Landscapes
- Engineering & Computer Science (AREA)
- Chemical & Material Sciences (AREA)
- Combustion & Propulsion (AREA)
- Mechanical Engineering (AREA)
- General Engineering & Computer Science (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
Description
Claims
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PL15727630T PL3152498T3 (pl) | 2014-06-05 | 2015-06-03 | Urządzenie do obróbki termicznej z projektorem wzorca świetlnego i kamerą |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE102014210672.9A DE102014210672A1 (de) | 2014-06-05 | 2014-06-05 | Gargerät mit Lichtmusterprojektor und Kamera |
PCT/EP2015/062349 WO2015185608A1 (de) | 2014-06-05 | 2015-06-03 | Gargerät mit lichtmusterprojektor und kamera |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3152498A1 true EP3152498A1 (de) | 2017-04-12 |
EP3152498B1 EP3152498B1 (de) | 2020-11-11 |
Family
ID=53366015
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP15727630.4A Active EP3152498B1 (de) | 2014-06-05 | 2015-06-03 | Gargerät mit lichtmusterprojektor und kamera |
Country Status (7)
Country | Link |
---|---|
US (1) | US10228145B2 (de) |
EP (1) | EP3152498B1 (de) |
CN (1) | CN106461230B (de) |
DE (1) | DE102014210672A1 (de) |
ES (1) | ES2835724T3 (de) |
PL (1) | PL3152498T3 (de) |
WO (1) | WO2015185608A1 (de) |
Families Citing this family (65)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE102014113389A1 (de) * | 2014-09-17 | 2016-03-17 | Pilz Gmbh & Co. Kg | Verfahren und Vorrichtung zum Identifizieren von Strukturelementen eines projizierten Strukturmusters in Kamerabildern |
USD787041S1 (en) | 2015-09-17 | 2017-05-16 | Whirlpool Corporation | Gas burner |
US10837651B2 (en) | 2015-09-24 | 2020-11-17 | Whirlpool Corporation | Oven cavity connector for operating power accessory trays for cooking appliance |
US11777190B2 (en) | 2015-12-29 | 2023-10-03 | Whirlpool Corporation | Appliance including an antenna using a portion of appliance as a ground plane |
WO2017170519A1 (ja) * | 2016-03-29 | 2017-10-05 | パナソニックIpマネジメント株式会社 | 加熱調理器 |
JPWO2017170319A1 (ja) * | 2016-03-29 | 2019-02-14 | パナソニックIpマネジメント株式会社 | 加熱調理器 |
WO2017170318A1 (ja) * | 2016-03-29 | 2017-10-05 | パナソニックIpマネジメント株式会社 | 加熱調理器 |
CN107296507A (zh) * | 2016-04-15 | 2017-10-27 | 松下知识产权经营株式会社 | 烹调辅助方法以及烹调辅助系统 |
DE102016206483A1 (de) * | 2016-04-18 | 2017-10-19 | Convotherm Elektrogeräte GmbH | Verfahren zum Feststellen einer Reinigungsnotwendigkeit und Qualitätsmanagement-Überwachungssystem eines gewerblichen Gargeräts, und gewerbliches Gargerät |
AU2016403900B2 (en) * | 2016-04-20 | 2022-09-29 | Vorwerk & Co. Interholding Gmbh | System for the preparation of at least one food product and method for operating the relevant system |
US10145568B2 (en) | 2016-06-27 | 2018-12-04 | Whirlpool Corporation | High efficiency high power inner flame burner |
DE102016012036A1 (de) | 2016-10-11 | 2018-04-12 | Diehl Ako Stiftung & Co. Kg | Gargutträger zum Aufnehmen von Gargut in einem Gargerät und Gargerät mit einem solchen Gargutträger |
DE102016221446A1 (de) * | 2016-11-02 | 2018-05-03 | BSH Hausgeräte GmbH | Kalibrieren eines Sauerstoffsensors eines Haushaltsgeräts |
EP3346190B1 (de) | 2017-01-10 | 2020-05-06 | Electrolux Appliances Aktiebolag | Lebensmittelzubereitungseinheit |
US10551056B2 (en) | 2017-02-23 | 2020-02-04 | Whirlpool Corporation | Burner base |
US10451290B2 (en) | 2017-03-07 | 2019-10-22 | Whirlpool Corporation | Forced convection steam assembly |
US10660162B2 (en) | 2017-03-16 | 2020-05-19 | Whirlpool Corporation | Power delivery system for an induction cooktop with multi-output inverters |
DE102017206056A1 (de) | 2017-04-10 | 2018-10-11 | BSH Hausgeräte GmbH | Betreiben eines Gargeräts |
KR102366006B1 (ko) * | 2017-06-20 | 2022-02-23 | 삼성전자주식회사 | 오븐 |
WO2019033089A1 (en) * | 2017-08-11 | 2019-02-14 | Brava Home, Inc. | CONFIGURABLE COOKING SYSTEMS AND METHODS |
CN109166054A (zh) * | 2017-08-16 | 2019-01-08 | 嘉易烤株式会社 | 烹饪鱼肉类时碳化时点管理服务提供系统 |
CN107692840A (zh) * | 2017-09-06 | 2018-02-16 | 珠海格力电器股份有限公司 | 电器具的显示方法及装置,电器具 |
DE102017121084A1 (de) * | 2017-09-12 | 2019-03-14 | Rational Aktiengesellschaft | Gargerät sowie Verfahren zur Erkennung der Belegung eines Einschubes in einem Garraum |
KR102400018B1 (ko) * | 2017-09-29 | 2022-05-19 | 삼성전자주식회사 | 자동 조리 장치 및 방법 |
US20190110638A1 (en) * | 2017-10-16 | 2019-04-18 | Midea Group Co., Ltd | Machine learning control of cooking appliances |
US10591218B2 (en) | 2017-10-27 | 2020-03-17 | Whirlpool Corporation | Oven having an imaging device |
US10605463B2 (en) | 2017-10-27 | 2020-03-31 | Whirlpool Corporation | Cooking appliance with a user interface |
US10523851B2 (en) | 2018-02-19 | 2019-12-31 | Haier Us Appliance Solutions, Inc. | Camera assembly for an oven appliance |
JPWO2019198621A1 (ja) * | 2018-04-09 | 2021-04-15 | パナソニックIpマネジメント株式会社 | 加熱調理器 |
CN108742170B (zh) * | 2018-05-08 | 2020-08-18 | 华南理工大学 | 一种烤箱的智能识别烹饪系统 |
WO2019246490A1 (en) | 2018-06-21 | 2019-12-26 | Prince Castle LLC | Infrared toaster |
US10627116B2 (en) | 2018-06-26 | 2020-04-21 | Whirlpool Corporation | Ventilation system for cooking appliance |
US10619862B2 (en) | 2018-06-28 | 2020-04-14 | Whirlpool Corporation | Frontal cooling towers for a ventilation system of a cooking appliance |
CN113170545B (zh) * | 2018-07-09 | 2023-12-29 | 博莱沃家居公司 | 烤箱内摄像头以及计算机视觉系统和方法 |
US10837652B2 (en) | 2018-07-18 | 2020-11-17 | Whirlpool Corporation | Appliance secondary door |
DE102018124378B4 (de) * | 2018-10-02 | 2023-03-23 | BIBA - Bremer Institut für Produktion und Logistik GmbH | Vorrichtung und Verfahren zur Prozessüberwachung mehrerer Teiglinge in einer Prozesskammer sowie eine Prozesskammer mit einer derartigen Vorrichtung |
CN109330356A (zh) * | 2018-11-09 | 2019-02-15 | 珠海格力电器股份有限公司 | 烹饪器具 |
WO2020116814A1 (en) * | 2018-12-06 | 2020-06-11 | Samsung Electronics Co., Ltd. | Heating cooker including three dimensional measuring device |
JP7290415B2 (ja) * | 2018-12-06 | 2023-06-13 | 三星電子株式会社 | 三次元計測装置および加熱調理器 |
DE102018221749A1 (de) * | 2018-12-14 | 2020-06-18 | BSH Hausgeräte GmbH | Backofen und Steuerverfahren |
US11287140B2 (en) | 2019-01-04 | 2022-03-29 | Whirlpool Corporation | Cooking appliance with an imaging device |
DE102019201332A1 (de) * | 2019-02-01 | 2020-08-06 | BSH Hausgeräte GmbH | Haushalts-Gargerät und Verfahren zum Betreiben eines Haushalts-Gargeräts |
DE102019203259A1 (de) | 2019-03-11 | 2020-09-17 | BSH Hausgeräte GmbH | Optisches Erkennen eines Garguts |
US11677901B2 (en) * | 2019-03-12 | 2023-06-13 | Marmon Foodservice Technologies, Inc. | Infrared toaster |
DE102019107846A1 (de) * | 2019-03-27 | 2020-07-16 | Miele & Cie. Kg | Verfahren zum Betreiben eines Gargeräts und Gargerät |
DE102019107812A1 (de) * | 2019-03-27 | 2020-10-01 | Miele & Cie. Kg | Verfahren zum Betreiben eines Gargeräts und Gargerät |
DE102019107859A1 (de) * | 2019-03-27 | 2020-07-09 | Miele & Cie. Kg | Verfahren zum Betreiben eines Gargeräts und Gargerät |
DE102019107834A1 (de) * | 2019-03-27 | 2020-07-16 | Miele & Cie. Kg | Verfahren zum Betreiben eines Gargeräts und Gargerät |
DE102019204533A1 (de) * | 2019-04-01 | 2020-10-01 | BSH Hausgeräte GmbH | Verfahren zum Zubereiten eines Garguts mit optisch angezeigten Gargutzonen des Garguts, Gargerät und Computerprogrammprodukt |
DE102019204531A1 (de) | 2019-04-01 | 2020-10-01 | BSH Hausgeräte GmbH | Haushaltsgerät und Verfahren zum Bestimmen von Konturinformation von Gut |
EP3758439B1 (de) * | 2019-06-25 | 2022-10-26 | Electrolux Appliances Aktiebolag | Verfahren und system zur steuerung eines ofens und ofen zum erwärmen von lebensmitteln |
DE102019209198A1 (de) * | 2019-06-26 | 2020-12-31 | Robert Bosch Gmbh | Haushaltsgerät |
DE102019210426B3 (de) * | 2019-07-15 | 2020-12-10 | BSH Hausgeräte GmbH | Steuereinheit und Verfahren zur Auswertung von Bilddaten in einem Hausgerät |
US10819905B1 (en) * | 2019-09-13 | 2020-10-27 | Guangdong Media Kitchen Appliance Manufacturing Co., Ltd. | System and method for temperature sensing in cooking appliance with data fusion |
JP7236644B2 (ja) * | 2019-10-01 | 2023-03-10 | パナソニックIpマネジメント株式会社 | 加熱調理器 |
DE102019216682A1 (de) * | 2019-10-29 | 2021-04-29 | BSH Hausgeräte GmbH | Bestimmen eines Ziel-Verarbeitungszustands eines zu behandelnden Garguts |
US20210186032A1 (en) * | 2019-12-19 | 2021-06-24 | Whirlpool Corporation | Monitoring system |
DE102020107568B4 (de) * | 2020-03-19 | 2022-02-03 | Miele & Cie. Kg | Verfahren zur Ansteuerung eines Gargeräts und Gargerät |
CN113491454A (zh) | 2020-04-06 | 2021-10-12 | 沙克忍者运营有限责任公司 | 能定位在支撑表面上的烹饪系统 |
KR102341238B1 (ko) * | 2020-08-21 | 2021-12-20 | 주식회사 비욘드허니컴 | 사람간 협업을 위한 자동화 조리기기 시스템 |
DE102020126249A1 (de) * | 2020-10-07 | 2022-04-07 | Welbilt Deutschland GmbH | Gargerät, insbesondere gewerbliches Gargerät |
US11940153B2 (en) | 2020-12-01 | 2024-03-26 | GMG Products, LLC | Fuel conditioner for grill |
US11747087B2 (en) * | 2021-01-06 | 2023-09-05 | Bsh Home Appliances Corporation | Household appliance including reflective door |
CN113208449B (zh) * | 2021-05-31 | 2022-10-11 | 广东美的厨房电器制造有限公司 | 用于烹饪设备的控制方法、控制装置以及烹饪设备 |
USD1005769S1 (en) | 2021-09-08 | 2023-11-28 | Newage Products Inc. | Oven |
Family Cites Families (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4224743A (en) * | 1978-06-19 | 1980-09-30 | Alternative Pioneering Systems, Inc. | Food dehydrating machine |
US4588297A (en) | 1982-06-14 | 1986-05-13 | Nippon Steel Corporation | Optical profile measuring method |
DE19748062C2 (de) | 1997-10-31 | 2000-10-05 | Bernward Maehner | Verfahren und Einrichtung zur dreidimensionalen optischen Vermessung von Objekten |
JP2002544510A (ja) | 1999-05-14 | 2002-12-24 | 3ディーメトリックス,インコーポレイテッド | カラー構造化された光3d撮像システム |
JP2001099615A (ja) * | 1999-09-30 | 2001-04-13 | Nippon Crucible Co Ltd | 対象物距離測定装置および対象物3次元形状測定装置 |
DE50113144D1 (de) | 2000-06-28 | 2007-11-29 | Bosch Gmbh Robert | Vorrichtung zum bildlichen Erfassen von Stückgütern |
JP3826111B2 (ja) * | 2003-06-06 | 2006-09-27 | 株式会社東芝 | 加熱調理器 |
US7131529B2 (en) * | 2003-07-01 | 2006-11-07 | Casa Herrera, Inc. | Oven conveyor alignment system apparatus and method |
US10687391B2 (en) * | 2004-12-03 | 2020-06-16 | Pressco Ip Llc | Method and system for digital narrowband, wavelength specific cooking, curing, food preparation, and processing |
DE102006005874C5 (de) | 2005-05-11 | 2017-05-18 | Carl Zeiss Automated Inspection GmbH | Verfahren zum berührungsfreien Vermessen |
JP2007192518A (ja) * | 2006-01-23 | 2007-08-02 | Matsushita Electric Ind Co Ltd | 高周波加熱装置 |
ATE432446T1 (de) | 2006-11-02 | 2009-06-15 | Electrolux Home Prod Corp | Vorrichtung und verfahren zum bestimmen der temperatur im inneren eines garguts |
DE102008024731B4 (de) | 2008-05-19 | 2020-08-20 | BAM Bundesanstalt für Materialforschung und -prüfung | Verfahren und Vorrichtung zur Sinterung eines Objektes unter Bestimmung des geometrischen Oberflächenprofils des Objekts |
EP2149755B1 (de) | 2008-07-30 | 2012-12-05 | Electrolux Home Products Corporation N.V. | Ofen und Betriebsverfahren dafür |
UA106979C2 (uk) * | 2009-03-05 | 2014-11-10 | Пресско Текнолоджі, Інк. | Система обробки для нагріву їжі, спосіб приготування, висушування або консервації харчового об'єкта та спосіб обробки харчового об'єкта |
DE102010003115A1 (de) * | 2010-03-22 | 2011-09-22 | BSH Bosch und Siemens Hausgeräte GmbH | Gargerät |
GB2486165A (en) * | 2010-11-30 | 2012-06-13 | St Microelectronics Res & Dev | Oven using a Single Photon Avalanche Diode (SPAD) array |
EP3220058A1 (de) | 2011-06-03 | 2017-09-20 | Electrolux Home Products Corporation N.V. | Kochofen mit einer vorrichtung zur detektion der dreidimensionalen form von lebensmitteln auf einem lebensmittelträger |
EP2798273A1 (de) * | 2011-12-26 | 2014-11-05 | Arçelik Anonim Sirketi | Ofen mit einem optischen erkennungsmittel |
EP2844187B1 (de) * | 2012-05-03 | 2016-10-12 | 3Shape A/S | Automatisierte herstellung eines zahnersatzes |
-
2014
- 2014-06-05 DE DE102014210672.9A patent/DE102014210672A1/de not_active Withdrawn
-
2015
- 2015-06-03 WO PCT/EP2015/062349 patent/WO2015185608A1/de active Application Filing
- 2015-06-03 EP EP15727630.4A patent/EP3152498B1/de active Active
- 2015-06-03 PL PL15727630T patent/PL3152498T3/pl unknown
- 2015-06-03 ES ES15727630T patent/ES2835724T3/es active Active
- 2015-06-03 US US15/315,791 patent/US10228145B2/en active Active
- 2015-06-03 CN CN201580029952.9A patent/CN106461230B/zh active Active
Also Published As
Publication number | Publication date |
---|---|
EP3152498B1 (de) | 2020-11-11 |
US10228145B2 (en) | 2019-03-12 |
US20170115008A1 (en) | 2017-04-27 |
ES2835724T3 (es) | 2021-06-23 |
CN106461230A (zh) | 2017-02-22 |
WO2015185608A1 (de) | 2015-12-10 |
DE102014210672A1 (de) | 2015-12-17 |
PL3152498T3 (pl) | 2021-05-31 |
CN106461230B (zh) | 2019-04-12 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3152498B1 (de) | Gargerät mit lichtmusterprojektor und kamera | |
DE102016215550A1 (de) | Feststellen eines Bräunungsgrads von Gargut | |
WO2018188913A1 (de) | Betreiben eines gargeräts | |
DE102016107617A1 (de) | Verfahren zum Betreiben eines Gargeräts und Gargerät | |
DE102019120008B4 (de) | Verfahren zum Betreiben eines Gargeräts und Gargerät | |
DE102015211954B4 (de) | Verfahren und Vorrichtung zum berührungslosen Vermessen einer Objektoberfläche | |
DE102013110642A1 (de) | Gargerät mit Kamera und Verfahren zur Einschuberkennung | |
DE102014114901A1 (de) | Gargerät sowie Verfahren zur Erfassung eines Prozessparameters eines Garprozesses | |
EP4229336A1 (de) | Verfahren zur bestimmung des zeitpunktes für die reinigung eines garraums eines gargeräts | |
WO2010130567A2 (de) | Kochfeld und verfahren zum erhitzen von auf dem kochfeld aufgestellten kochgefässen | |
DE102019107834A1 (de) | Verfahren zum Betreiben eines Gargeräts und Gargerät | |
DE102013114227A1 (de) | Gargerät mit Beschickungserkennung | |
DE102014210673A1 (de) | Bestimmen einer Einschubebene eines Gargutträgers | |
DE102011075187B3 (de) | Kochfeld | |
DE102018207785A1 (de) | Verfahren und Testsystem zur Bestimmung der Effektivität einer Dunstabzugsvorrichtung | |
DE10125247C1 (de) | Haushaltsgerät mit einem Garraum | |
DE102019107828B4 (de) | Verfahren zum Betreiben eines Gargeräts und Gargerät | |
EP3537898B1 (de) | Steuereinheit, nahrungsmitteldrucker und verfahren zur steuerung eines nahrungsmitteldruckers | |
EP3767580A1 (de) | Steuereinheit und verfahren zur auswertung von bilddaten in einem hausgerät | |
DE102015105128A1 (de) | Verfahren und Vorrichtung zur Messung des Glanzgrads und/oder der Mattheit von Gegenständen | |
DE102015206437B3 (de) | Vorrichtung zur Bestimmung der thermischen Ausdehnung und/oder Gefügeumwandlungen von Proben | |
WO2019170447A1 (de) | Interaktionsmodul | |
EP4323721A1 (de) | Verfahren zum betreiben eines gargeräts und gargerät | |
DE102019107815B4 (de) | Verfahren zum Betreiben eines Gargeräts und Gargerät | |
DE102017121084A1 (de) | Gargerät sowie Verfahren zur Erkennung der Belegung eines Einschubes in einem Garraum |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20170105 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20190506 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20200619 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 1333844 Country of ref document: AT Kind code of ref document: T Effective date: 20201115 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 502015013793 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: GERMAN |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20201111 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210311 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210211 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201111 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201111 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210212 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210311 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201111 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201111 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210211 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG9D |
|
REG | Reference to a national code |
Ref country code: ES Ref legal event code: FG2A Ref document number: 2835724 Country of ref document: ES Kind code of ref document: T3 Effective date: 20210623 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201111 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201111 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201111 Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201111 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201111 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201111 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201111 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 502015013793 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201111 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
26N | No opposition filed |
Effective date: 20210812 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201111 Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201111 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201111 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201111 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20210630 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20210603 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20210630 Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20210603 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20210630 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210311 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20210630 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MM01 Ref document number: 1333844 Country of ref document: AT Kind code of ref document: T Effective date: 20210603 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20210603 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20150603 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230504 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201111 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20230622 Year of fee payment: 9 Ref country code: DE Payment date: 20230630 Year of fee payment: 9 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: TR Payment date: 20230530 Year of fee payment: 9 Ref country code: PL Payment date: 20230518 Year of fee payment: 9 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: IT Payment date: 20230630 Year of fee payment: 9 Ref country code: GB Payment date: 20230622 Year of fee payment: 9 Ref country code: ES Payment date: 20230719 Year of fee payment: 9 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201111 |