WO2023094078A1

WO2023094078A1 - Food preparation assembly comprising a camera assembly

Info

Publication number: WO2023094078A1
Application number: PCT/EP2022/079081
Authority: WO
Inventors: Weimin Xiao; Jingwei Tan; Wen SUN
Original assignee: Philips Domestic Appliances Holding B.V.
Priority date: 2021-11-26
Filing date: 2022-10-19
Publication date: 2023-06-01

Abstract

Provided is a food preparation assembly (100) comprising a food preparation chamber (102), and a camera assembly (104) for obtaining one or more images inside the food preparation chamber. The camera assembly is controllable to adjust an image characteristic of the one or more images. The food preparation assembly comprises a radiation assembly (106A, 106B, 108A, 108B) controllable to adopt a radiation status in which the radiation assembly emits radiation comprising visible light inside the food preparation chamber. One or more processors (112) is or are configured to: identify the radiation status; and control the camera assembly to adjust the image characteristic using a predetermined relationship for adjusting the image characteristic according to the identified radiation status. Further provided is a method of controlling a camera assembly suitable for obtaining one or more images inside a food preparation chamber of a food preparation assembly, a computer program for implementing the method, and a method of determining a relationship for adjusting an image characteristic of one or more images inside a food preparation chamber of a food preparation assembly.

Description

FOOD PREPARATION ASSEMBLY COMPRISING A CAMERA ASSEMBLY

FIELD OF THE INVENTION

This invention relates to a food preparation assembly having a food preparation chamber and a camera assembly for obtaining one or more images inside the food preparation chamber.

The invention further relates to a method of controlling a camera assembly suitable for obtaining one or more images inside a food preparation chamber of a food preparation assembly, and a computer program for implementing the method.

The invention yet further relates to a method of determining a relationship for adjusting an image characteristic of one or more images inside a food preparation chamber of a food preparation assembly.

BACKGROUND OF THE INVENTION

Due to busy daily life, people usually have very limited time for home cooking.

Moreover, many people lack the cooking experience necessary to obtain satisfactory results. There is accordingly growing demand for smart food preparation assemblies capable of delivering food prepared according to the user’s preference. Benefits of such smart food preparation assemblies include greater user convenience, better taste and nutritional value of the food prepared using such food preparation assemblies, and minimizing or avoidance of harmful substances being generated in a cooking process using such food preparation assemblies.

Including a camera in the food preparation assembly, for example by integrating a camera into a kitchen appliance, can play an important role in smart assisted food preparation, e.g. cooking. For example, a food type can be recognized by food image recognition and a cooking setting can be determined based on food image analysis results. Other applications of cameras in food preparation assemblies include cooking process control, browning control, remote monitoring, burning prevention, and content for social media sharing.

Due to the above-mentioned benefits, camera-integrated kitchen appliances have recently been introduced onto the market. However, various challenges remain in terms of obtaining images which can be reliably used for, in particular, food recognition, cooking process control, browning control, remote monitoring, and burning prevention.

US 2018/324908 Al discloses systems and methods relating to a cooking appliance having a heating element disposed within a cooking chamber and operable to selectively emit waves at any of a plurality of powers and/or peak wavelengths, a camera operable to capture an image of the cooking chamber, and a computing device operable to supply power to the heating element to vary the power and/or peak wavelength of the emitted waves and generate heat within the cooking chamber, and instruct the camera to capture the image when the heating element is emitting at a stabilized power and/or peak wavelength.

WO 2016/179424 Al discloses a connected oven, including a set of in-cavity sensors and a processor configured to automatically identify foodstuff within the cooking cavity, based on the sensor measurements, and automatically operate the heating element based on the foodstuff identity.

SUMMARY OF THE INVENTION

The invention is defined by the claims.

According to examples in accordance with an aspect of the invention, there is provided a food preparation assembly comprising: a food preparation chamber; a camera assembly for obtaining one or more images inside the food preparation chamber, the camera assembly being controllable to adjust an image characteristic of the one or more images; a radiation assembly controllable to adopt a radiation status in which the radiation assembly emits radiation comprising visible light inside the food preparation chamber; and one or more processors configured to: identify the radiation status; and control the camera assembly to adjust the image characteristic using a predetermined relationship for adjusting the image characteristic according to the identified radiation status.

Using such a predetermined relationship to adjust the image characteristic, such as exposure and/or white balance, according to the identified radiation status of the radiation assembly can assist to provide faster and/or more repeatable image characteristic adjustment relative to, for instance, adjustment based only on an optical condition detected inside the food preparation chamber when obtaining the image(s).

In some embodiments, the camera assembly may be configured to obtain a sequence of images of the inside of the food preparation chamber, e.g. a sequence of images constituting a video of the inside of the food preparation chamber.

Alternatively or additionally, the camera assembly may be configured to obtain a still image of the inside of the food preparation chamber.

By identifying the radiation status(es) in advance and determining the predetermined relationship for adjusting the image characteristic, e.g. camera assembly capture setting(s) and/or image processing setting(s), according to the radiation status, the image(s) can, for example, be brought into closer conformity with a “real” view of the food ingredients perceptible with the naked eye, etc.

By bringing the image(s) into closer conformity with such a “real” view of the food ingredients by controlling the camera assembly to, for instance, adjust the exposure and/or white balance using the predetermined relationship, more precise/re liable food recognition, cooking process control, browning control, doneness status determination, remote monitoring, and/or burning prevention may be achieved.

In some embodiments, the radiation assembly is controllable to select the radiation status from a plurality of radiation statuses, and the one or more processors is or are configured to identify the selected radiation status, with the predetermined relationship permitting adjusting of the image characteristic according to the identified selected radiation status.

The image characteristic can accordingly be adjusted according to which of the plurality of radiation statuses is selected.

In embodiments in which the radiation assembly is controllable to select the radiation status from the plurality of radiation statuses, the plurality of radiation statuses can comprise: a first radiation status; and a second radiation status, with the first radiation status causing a first optical condition in the food preparation chamber, and the second radiation status causing a second optical condition in the food preparation chamber which is different from the first optical condition.

In such embodiments, the predetermined relationship may be determined such that the image characteristic is adjusted to compensate for the difference between the first optical condition and the second optical condition.

By compensating for the difference between the first and second optical conditions, the image(s) obtained when the first radiation status is adopted can be comparable to the image(s) obtained when the second radiation status is adopted.

For example, should the optical condition include an intensity of visible light inside the food preparation chamber, adjustment of the image characteristic includes adjustment of the exposure of the image(s) to compensate for the difference in intensity. Alternatively or additionally, should the optical condition include a spectral composition of the light inside the food preparation chamber, adjustment of the image characteristic includes adjustment of the colour, e.g. white, balance of the image(s) to compensate for the difference in spectral composition.

It is noted, for the avoidance of doubt, that the plurality of radiation statuses can include further radiation statuses in addition to the above-described first and second radiation statuses, such as a third radiation status, a fourth radiation status, a fifth radiation status, and so on. In such embodiments, the predetermined relationship may be determined such that the image characteristic is adjusted to compensate for differences between the respective optical conditions caused by selection of each of the radiation statuses.

In some embodiments, the radiation assembly comprises at least one lamp for illuminating the inside of the food preparation chamber. In such embodiments, the radiation status comprises an illumination status of the, e.g. each of the, at least one lamp.

The illumination status can, for instance, be an on/off status of the, e.g. each of the, at least one lamp. Alternatively or additionally, the illumination status can be an intensity of the illumination provided by the, e.g. each of the, at least one lamp and/or the illumination status can be a measure of the spectral composition of the illumination provided by the, e.g. each of the, at least one lamp.

In some embodiments, the radiation assembly comprises at least one heating element arranged to radiate inside the food preparation chamber. In such embodiments, the radiation status comprises a heating status of the, e.g. each of the, at least one heating element.

The heating status can, for instance, be an on/off status of the, e.g. each of the, at least one heating element.

In embodiments in which the radiation status comprises the illumination status of the at least one lamp and the heating status of the at least one heating element, the illumination status can be an on/off status of the at least one lamp, with the heating status being an on/off status of the at least one heating element.

Alternatively or additionally, the heating status can be a degree of heating provided by the, e.g. each of the, at least one heating element and/or the heating status can be a measure of the spectral composition of the radiation provided by the, e.g. each of the, at least one heating element.

The food preparation assembly may comprise a control unit having a control configuration arranged to control the radiation assembly to adopt the radiation status. In such embodiments, the one or more processors is or are configured to identify the radiation status based on said control configuration.

Thus, the radiation status can be obtained, in other words directly obtained, from the control unit controlling the radiation assembly. This may advantageously obviate sensing of an optical condition inside the food preparation chamber in order to adjust the image characteristic.

In embodiments in which the control configuration comprises a control sequence which controls the radiation status of the radiation assembly to change over time, the one or more processors may be configured to control the camera assembly to adjust the image characteristic (using the predetermined relationship) to coincide with the change of the radiation status. This may enable particularly responsive adjustment of the image characteristic which may be especially useful, for example, when the camera assembly is configured to obtain a sequence of images, e.g. video, during the change of the radiation status.

In some embodiments, the radiation status, e.g. each of the radiation statuses when there the radiation status is selectable from a plurality of radiation statuses, is associated with a power consumption of the radiation assembly, and the one or more processors is or are configured to identify the radiation status based on the (respective) power consumption of the radiation assembly.

The food preparation assembly may comprise a sensor for sensing radiation of the radiation assembly. In such embodiments, the one or more processors is or are configured to identify the radiation status based on the sensed radiation. In this way, the radiation status can be identified according to the radiation detected via the sensor.

In some embodiments, the camera assembly comprises an image capture module, and the one or more processors may be configured to control at least one image capture setting of the image capture module according to the identified radiation status to adjust the image characteristic.

The identified radiation status may thus be used to adjust the acquisition of the image(s).

The at least one image capture setting may, for instance, comprise an aperture size, a shutter speed and/or a sensitivity setting.

The food preparation assembly may comprise an image processing module, and the one or more processors may be configured to control at least one image processing setting of the image processing module according to the identified radiation status to adjust the image characteristic.

The image processing module can, for example, be included in the camera assembly, a user device, such as a smart phone or tablet computer, and/or a cloud-based server.

In such embodiments, the identified radiation status is used to adjust the processing of the image(s) by the image processing module.

The at least one processing setting may, for instance, comprise a colour balance setting. Such a colour balance setting can, for example, be a white balance setting.

In at least some embodiments, the food preparation assembly comprises or is a domestic food preparation assembly. For example, the food preparation assembly comprises a domestic kitchen appliance.

Examples of such a domestic kitchen appliance include an air fryer, oven, steamer, pressure cooker and rice cooker.

Particular mention is made of embodiments in which the camera assembly is integral, in other words built-in, to such a domestic kitchen appliance.

In alternative embodiments, the camera assembly is a removable or detachable component of the food preparation assembly, e.g. domestic food preparation assembly. A window into the food preparation chamber may be provided, and the camera assembly may be arranged such as to obtain the image(s) inside the food preparation chamber via the window in order to assemble the food preparation assembly, e.g. the domestic food preparation assembly.

For instance, the camera assembly, and optionally at least one of the one or more processors, may be included in user device separate from a kitchen appliance comprising the food preparation chamber and the radiation assembly, such as a smart phone or tablet computer. The processor(s) may alternatively or additionally be included in a cloud-based server. The food preparation assembly, e.g. domestic food preparation assembly, may be assembled by arranging the camera assembly such as to obtain the image(s) inside the food preparation chamber via the window. In some embodiments, the food preparation chamber and the radiation assembly are included in a kitchen appliance, e.g. domestic kitchen appliance, and the processor(s) included in the user device and/or the cloud-based server is or are in communication, e.g. wireless communication, with the kitchen appliance, e.g. with the above -de scribed control unit, in order to enable the processor(s) to identify the radiation status and control the camera assembly to adjust the image characteristic of the one or more images using the predetermined relationship.

According to another aspect there is provided a method of controlling a camera assembly suitable for obtaining one or more images inside a food preparation chamber of a food preparation assembly, the food preparation assembly further comprising a radiation assembly controllable to adopt a radiation status in which the radiation assembly emits radiation comprising visible light inside the food preparation chamber, the method comprising: identifying the radiation status; and controlling the camera assembly to adjust an image characteristic of the one or more images using a predetermined relationship for adjusting the image characteristic according to the identified radiation status.

The method may further comprise operating the camera assembly to obtain one or more images inside the food preparation chamber.

The image characteristic may be adjusted according to the identified radiation status in the obtained one or more images.

In some embodiments, the radiation assembly is controllable to select the radiation status from a plurality of radiation statuses, and the identifying comprises identifying the selected radiation status, with the predetermined relationship permitting adjusting of the image characteristic according to the identified selected radiation status.

In some embodiments, the radiation assembly comprises at least one lamp for illuminating the inside of the food preparation chamber. In such embodiments, the identifying comprises identifying an illumination status of the, e.g. each of the, at least one lamp. In some embodiments, the radiation assembly comprises at least one heating element arranged to radiate inside the food preparation chamber. In such embodiments, the identifying comprises identifying a heating status of the, e.g. each of the, at least one heating element.

In embodiments in which the radiation assembly comprises the at least one lamp and the at least one heating element, the identifying may comprise identifying the illumination status of the at least one lamp and identifying the heating status of the at least one heating element. In such embodiments, the illumination status can, for example, be an on/off status of the at least one lamp, with the heating status being an on/off status of the at least one heating element.

The food preparation assembly may comprise a control unit having a control configuration arranged to control the radiation assembly to adopt the radiation status. In such embodiments, the identifying may comprise identifying the radiation status based on the control configuration.

In some embodiments, the method comprises determining a power consumption of the radiation assembly, with the identifying comprising identifying the radiation status based on the determined power consumption.

The food preparation assembly may comprise a sensor for sensing radiation of the radiation assembly. In such embodiments, the identifying may comprise identifying the radiation status based on the sensed radiation.

In some embodiments, the camera assembly comprises an image capture module, and the controlling comprises controlling at least one image capture setting of the image capture module according to the identified radiation status to adjust the image characteristic. The at least one image capture setting may, for instance, comprise an aperture size, a shutter speed and/or a sensitivity setting.

Alternatively or additionally, the food preparation assembly may comprise an image processing module, and the controlling comprises controlling at least one image processing setting of the image processing module according to the identified radiation status to adjust the image characteristic. The at least one processing setting may, for instance, comprise a colour balance setting. Such a colour balance setting can, for example, be a white balance setting.

According to yet another aspect there is provided a computer program comprising computer program code which is configured, when said computer program is run on one or more processors included in a food preparation assembly further comprising a food preparation chamber, a camera assembly for obtaining one or more images inside the food preparation chamber, and a radiation assembly controllable to adopt a radiation status in which the radiation assembly emits radiation comprising visible light inside the food preparation chamber, to cause said one or more processors to implement the method according to any of the embodiments described herein.

One or more non-transitory computer readable media may be provided, which non- transitory computer readable media have a computer program stored thereon, with the computer program comprises computer program code which is configured, when the computer program is run on the one or more processors, to cause the one or more processors to implement the method according to any of the embodiments described herein.

The one or more processors can be included in a kitchen appliance included in the food preparation assembly, in a user device, for example a smart phone or tablet computer, separate from such a kitchen appliance, and/or in a cloud-based server.

According to a further aspect there is provided a method of determining a relationship for adjusting an image characteristic of one or more images inside a food preparation chamber of a food preparation assembly comprising a radiation assembly, the one or more images being obtained using a camera assembly, the method comprising: identifying at least one radiation status of the radiation assembly in which radiation comprising visible light is emitted inside the food preparation chamber; measuring, for each of the at least one radiation status, an optical condition; and determining, using said optical condition measured for each of the identified at least one radiation status, said relationship for adjusting the image characteristic according to the identified at least one radiation status.

This method may be used to establish/determine the above-described predetermined relationship for adjusting the image characteristic according to the identified radiation status.

More generally, embodiments described herein in relation to the food preparation assembly and the method of controlling the camera assembly may be applicable to the method of determining the relationship, and embodiments described herein in relation to the method of determining the relationship may be applicable to the food preparation assembly and the method of controlling the camera assembly.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the invention, and to show more clearly how it may be carried into effect, reference will now be made, by way of example only, to the accompanying drawings, in which:

FIG. 1 schematically depicts a food preparation assembly according to an example;

FIG. 2 provides a block diagram of a camera assembly according to an example;

FIG. 3 provides a flowchart of a method of operating a camera assembly according to an example;

FIG. 4 graphically represents adjustment of shutter speed and white balance according to radiation statuses of a radiation assembly as cooking progresses;

FIG. 5 provides a flowchart of an exemplary method of determining a relationship for adjusting an image characteristic of one or more images; and

FIG. 6 provides a flowchart for illustrating how the relationship determined via the method shown in FIG. 5 can be applied to an exemplary method of operating a camera assembly. DETAILED DESCRIPTION OF THE EMBODIMENTS

The invention will be described with reference to the Figures.

It should be understood that the detailed description and specific examples, while indicating exemplary embodiments of the apparatus, systems and methods, are intended for purposes of illustration only and are not intended to limit the scope of the invention. These and other features, aspects, and advantages of the apparatus, systems and methods of the present invention will become better understood from the following description, appended claims, and accompanying drawings. It should be understood that the Figures are merely schematic and are not drawn to scale. It should also be understood that the same reference numerals are used throughout the Figures to indicate the same or similar parts.

Provided is a food preparation assembly, which can also be referred to as a food preparation system, comprising a food preparation chamber, and a camera assembly for obtaining one or more images inside the food preparation chamber. The camera assembly is controllable to adjust an image characteristic of the one or more images. The food preparation assembly comprises a radiation assembly controllable to adopt a radiation status in which the radiation assembly emits radiation comprising visible light inside the food preparation chamber. One or more processors is or are configured to: identify the radiation status; and control the camera assembly to adjust the image characteristic using a predetermined relationship for adjusting the image characteristic according to the identified radiation status. Further provided is a method of controlling a camera assembly suitable for obtaining one or more images inside a food preparation chamber of a food preparation assembly, a computer program for implementing the method, and a method of determining a relationship for adjusting an image characteristic of one or more images inside a food preparation chamber of a food preparation assembly.

Obtaining one or more images inside the food preparation chamber is desirable for a number of reasons. In particular, the one or more images can be used for recognizing food present in the food preparation chamber, cooking process monitoring, and/or cooking process control based on analysis of the image(s).

Consistent image quality is particularly important for such uses. However, ensuring sufficiently consistent quality of the image(s) can be challenging, particularly due to changing optical conditions within the food preparation chamber resulting from adjustments in operation of the radiation assembly also included in the food preparation assembly, which radiation assembly emits radiation comprising visible light inside the food preparation chamber.

For example, heating element(s) included in the food preparation assembly may shift between on and off statuses during cooking. The illumination intensity and colour temperature in the food preparation chamber may therefore vary significantly due to the heating element(s) shifting between such statuses. If, for instance, the white balance and exposure are not adjusted, both the colour and brightness of the image(s) may be correspondingly significantly affected. As a result, the capability of the image(s) to represent the appearance of the food may be adversely affected. This may present difficulty in, in particular, food recognition, cooking process control, browning control, doneness status determination, remote monitoring, and/or burning prevention, etc. using the image(s).

Whilst the camera assembly can, for instance, itself detect an optical condition within the food preparation chamber and adjust an image characteristic of the image(s) based on such a detected optical condition, there may be a lag to such adjustment and/or the adjustment may not be sufficiently repeatable to ensure sufficiently consistent quality of the image(s).

The present disclosure is based on the insight that identifying the radiation status of the radiation assembly, and using the predetermined relationship to adjust the image characteristic, such as exposure and/or white balance, according to the identified radiation status can assist to provide faster and/or more repeatable image characteristic adjustment relative to, for instance, adjustment based only on an optical condition detected inside the food preparation chamber when obtaining the image(s).

The thus obtained image(s) can therefore be usefully employed in, for example, food recognition, cooking process control, browning control, doneness status determination, remote monitoring, and/or burning prevention based on analysis of the image(s).

In at least some embodiments, the present disclosure provides a method of compensating for change(s) in illuminance and/or colour temperature caused by the radiation assembly, e.g. heating elements and/or illumination system/lamps, shifting between statuses.

Such a method may, for example, adjust exposure and/or white balance during cooking for a camera assembly-integrated kitchen appliance, e.g. cooking device.

FIG. 1 schematically depicts a food preparation assembly 100 according to an example. The food preparation assembly 100 comprises a food preparation chamber 102 in which food ingredients 103 are receivable.

The food preparation assembly 100 comprises a camera assembly 104 for obtaining one or more images inside the food preparation chamber 102. Depending on how the image(s) is or are being used, e.g. for food recognition, cooking process monitoring, and/or cooking process control, the camera assembly 104 may be configured to obtain a sequence of images, e.g. a sequence of images constituting a video. Alternatively or additionally, the camera assembly 104 may be configured to obtain a still image of the inside of the food preparation chamber 102.

The camera assembly 104 may be arranged such as to enable obtaining of the one or more images of the food ingredients 103 received in the food preparation chamber 102.

In the non-limiting example shown in FIG. 1, the camera assembly 104, or at least an optical sensing element of the camera assembly 104, is arranged at or proximal to the ceiling of the food preparation chamber 102 in order to obtain image(s) of the food ingredients from above. Alternatively or additionally, the camera assembly 104, or at least the optical sensing element thereof, is arranged, for instance, at a side of the food preparation chamber 102 between the base and the ceiling of the food preparation chamber 102.

The food preparation assembly 100 comprises a radiation assembly 106A, 106B, 108A, 108B configured to emit radiation comprising visible light inside the food preparation chamber 102. In particular, the radiation assembly 106A, 106B, 108A, 108B is controllable, e.g. by a user of the food preparation assembly 100, to adopt a radiation status in which the radiation assembly 106A, 106B, 108A, 108B emits radiation comprising visible light inside the food preparation chamber 102.

In some embodiments, such as that shown in FIG. 1, the radiation assembly 106A, 106B, 108A, 108B comprises at least one lamp 106A, 106B for illuminating the inside of the food preparation chamber 102. In such embodiments, the radiation status can comprise an illumination status of the, e.g. each of the, at least one lamp 106A, 106B.

The lamp(s) 106A, 106B can, for example, be controlled by the user, e.g. by operating a suitable user interface included in the food preparation assembly 100 for enabling the user to control the lamp(s) 106A, 106B. This may, for example, assist the user to visually monitor the process taking place in the food preparation chamber 102. The light from the lamp(s) 106A, 106B may thus alter an optical condition within the food preparation chamber 102.

The lamp(s) 106A, 106B may, for example, be or comprise light emitting diode(s), although any suitable lamp type, such as filament lamps, can be contemplated.

In the non-limiting example shown in FIG. 1, the lamps 106A, 106B are arranged at or proximal to the ceiling of the food preparation chamber 102, although other position(s) for such lamp(s) 106A, 106B can be contemplated provided the lamp(s) 106A, 106B can illuminate at least part of the inside of the food preparation chamber 102.

The illumination status can, for instance, comprise an on/off status of the, e.g. each of the, at least one lamp 106A, 106B.

Alternatively or additionally, the illumination status can comprise an intensity of the illumination provided by the, e.g. each of the, at least one lamp 106A, 106B.

Alternatively or additionally, the illumination status can comprise a measure of the spectral composition of the illumination provided by the, e.g. each of the, at least one lamp 106A, 106B.

When the at least one lamp 106A, 106B comprises a plurality of lamps 106A, 106B, such as two (as shown in FIG. 1), three, four, five, six or more lamps 106A, 106B, one or more of these lamps 106A, 106B may be operable independently of the other(s). In such an example, the illumination status may comprise a combination of the individual illumination statuses of each of the independently operable lamp(s) 106A, 106B.

In some embodiments, such as that shown in FIG. 1, the radiation assembly 106A, 106B, 108A, 108B comprises at least one heating element 108A, 108B arranged to radiate inside the food preparation chamber 102. In such embodiments, the radiation status can comprise a heating status of the, e.g. each of the, at least one heating element 108A, 108B.

The at least one heating element 108A, 108B may be used to warm or cook the food ingredients 103 received in the food preparation chamber 102. However, the radiation emitted by the heating element(s) 108A, 108B may include visible light. Such visible light may alter an optical condition within the food preparation chamber 102.

The heating element(s) 108A, 108B may, for example, be or comprise resistive heating element(s), although any suitable heating element type can be contemplated.

In the non-limiting example shown in FIG. 1, the heating elements 108A, 108B are arranged at or proximal to the ceiling of the food preparation chamber 102, although other position(s) for such heating element(s) 108A, 108B can be contemplated provided the heating element(s) 108A, 108B can emit heat radiation into at least part of the inside of the food preparation chamber 102.

The heating status can, for instance, be an on/off status of the, e.g. each of the, at least one heating element 108A, 108B.

Alternatively or additionally, the heating status can be a degree of heating provided by the, e.g. each of the, at least one heating element 108A, 108B and/or the heating status can be a measure of the spectral composition of the radiation provided by the, e.g. each of the, at least one heating element 108A, 108B.

When the at least one heating element 108A, 108B comprises a plurality of heating elements 108A, 108B, such as two (as shown in FIG. 1), three, four, five, six or more heating elements 108A, 108B, one or more of these heating elements 108A, 108B may be operable independently of the other(s). In such an example, the heating status may comprise a combination of the individual heating statuses of each of the independently operable heating element(s) 108A, 108B.

In embodiments in which the radiation status comprises the illumination status of the at least one lamp 106A, 106B and the heating status of the at least one heating element 108A, 108B, the illumination status can, for example, be an on/off status of the at least one lamp 106A, 106B, with the heating status being an on/off status of the at least one heating element 108A, 108B.

In some embodiments, such as that shown in FIG. 1, the food preparation assembly 100 comprises a control unit 110 having a control configuration arranged to control the radiation assembly 106A, 106B, 108A, 108B to adopt the radiation status.

Thus, the radiation status can be obtained, in other words directly obtained, from the control unit 110 controlling the radiation assembly. This may advantageously obviate sensing of an optical condition inside the food preparation chamber 102 for the purpose of adjusting the image characteristic.

In some embodiments, the food preparation assembly 100 comprises a user interface (not visible in FIG. 1) configured to permit user selection of the radiation status. In such embodiments, the control configuration can, for example, correspond to or be generated from the user selection.

In a non-limiting example, the control unit 110 is configured to receive a recipe input, e.g. a recipe instruction input, and generate the control configuration based on the received recipe input. In other words, the control configuration can be automatically generated by the control unit 110 from the recipe input.

In some embodiments, the control configuration comprises a control sequence which controls the radiation status of the radiation assembly 106A, 106B, 108A, 108B over time.

The recipe input can be inputted in any suitable manner. For example, the recipe input can be received by the control unit 110 from a recipe database in response to a user selection from the recipe database.

More generally, due to the above-mentioned influence of the radiation assembly 106A, 106B, 108A, 108B, e.g. comprising the lamp(s) 106A, 106B and/or the heating element(s) 108A, 108B, an optical condition, e.g. an intensity and/or spectral composition of visible light, inside the food preparation chamber 102 may change according to the radiation status of the radiation assembly 106A, 106B, 108A, 108B.

For this reason, the food preparation assembly 100 comprises one or more processors 112 configured to identify the radiation status, and control the camera assembly 104 to adjust the image characteristic using a predetermined relationship for adjusting the image characteristic according to the identified radiation status.

Using such a predetermined relationship to adjust the image characteristic, such as exposure and/or white balance, according to the identified radiation status of the radiation assembly 106A, 106B, 108A, 108B can assist to provide faster and/or more repeatable image characteristic adjustment relative to, for instance, adjustment based only on an optical condition detected inside the food preparation chamber 102 when obtaining the image(s).

In some embodiments, the radiation assembly 106A, 106B, 108A, 108B is controllable to select the radiation status from a plurality of radiation statuses, and the one or more processors 112 is or are configured to identify the selected radiation status, with the predetermined relationship permitting adjusting of the image characteristic according to the identified selected radiation status.

In embodiments in which the radiation assembly is controllable to select the radiation status from the plurality of radiation statuses, the plurality of radiation statuses can comprise: a first radiation status; and a second radiation status, with the first radiation status causing a first optical condition in the food preparation chamber 102, and the second radiation status causing a second optical condition in the food preparation chamber 102 which is different from the first optical condition.

In a non-limiting example, the food preparation assembly 100, for example the abovedescribed control unit 110 included in the food preparation assembly 100, is configured to control the heating element(s) 108A, 108B according to a cooking program in which the heating element(s) 108A, 108B is or are controlled such that the food preparation chamber 102 is at a first temperature for a first predetermined duration, and then, e.g. when a doneness level of the food ingredients 103 is approaching, is at as second temperature lower than the first temperature.

In this example, the radiation status of the radiation assembly 106A, 106B, 108A, 108B, and in particular the heating status of the heating element(s) 108A, 108B, changes from a first radiation status to a second radiation status in order to implement the temperature reduction from the first temperature to the second temperature.

The first radiation status may, for instance, correspond to both of the heating elements 108A, 108B being on, with the second radiation status corresponding to one of the heating elements 108A, 108B being on and the other of the heating elements 108B, 108A being off.

The alteration in the optical condition, e.g. intensity and/or spectral composition of visible light, inside the food preparation chamber 102 resulting from the radiation status changing from the first radiation status to the second radiation status may nonetheless be compensated for by the one or more processors 112 controlling the camera assembly 104 to adjust the image characteristic. In particular, the predetermined relationship used by the processor(s) 112 may be determined such that the image characteristic is adjusted to compensate for the difference between the first optical condition and the second optical condition.

This compensating may, in turn, improve the image-based control which can be exerted over the cooking process, e.g. selection/estimation of the time required at the second predetermined temperature. This is because the image(s) obtained when the radiation assembly 106A, 106B, 108A, 108B adopts the first radiation status may be more reliably compared with the image(s) obtained when the radiation assembly 106A, 106B, 108A, 108B adopts the second radiation status.

In embodiments in which the control configuration comprises the above-mentioned control sequence which controls the radiation status of the radiation assembly 106A, 106B, 108A, 108B to change over time, the one or more processors 112 may be configured to control the camera assembly 104 to adjust the image characteristic to coincide with the change of the radiation status. This may enable particularly responsive adjustment of the image characteristic which may be especially useful, for example, when the camera assembly 104 is configured to obtain a sequence of images, e.g. constituting a video, during the change of the radiation status.

Whilst the processor(s) 112 is or are shown in FIG. 1 receiving data, e.g. the control configuration, from the control unit 110, this is not intended to be limiting. In other embodiments, which may be an alternative or in addition to the control configuration being used by the processor(s) 112 to identify the radiation status, the food preparation assembly 100 may include a sensor (not visible) for sensing radiation of the radiation assembly 106A, 106B, 108A, 108B. In such embodiments, the one or more processors 112 is or are configured to identify the radiation status based on the radiation sensed by the sensor.

Alternatively or additionally, the one or more processors 112 may be configured to identify the radiation status based on a power consumption of the radiation assembly 106A, 106B, 108A, 108B.

Referring to FIG. 2, in some embodiments, the camera assembly 104 comprises an image capture module 114, and the one or more processors 112 is or are configured to control at least one image capture setting of the image capture module 114 according to the identified radiation status to adjust the image characteristic. The identified radiation status may thus be used to adjust the acquisition of the image(s).

The at least one image capture setting can, for example, comprise or be an aperture size, a shutter speed and/or a sensitivity setting.

The camera assembly 104 may, in at least some embodiments, further comprise an image processing module 116. In such embodiments, the image processing module 116 may process the image(s) acquired by the image capture module 114, thereby to obtain the image(s). The image processing module 116 may alternatively or additionally be included in a user device, such as a smart phone or tablet computer, separate from a kitchen appliance in which the food preparation chamber 102 is included, and/or in a cloud-based server.

As an alternative or in addition to the above -de scribed control over the at least one capture setting of the image capture module 114 included in the camera assembly 104, the one or more processors 112 may be configured to control at least one image processing setting of the image processing module 116 according to the identified radiation status to adjust the image characteristic. Thus, the identified radiation status can be used to adjust the processing of the image(s) by the image processing module 116 included in the camera assembly 104.

For example, in order to implement the above-described compensation in the case of adoption of the second radiation status resulting in a higher visible light intensity in the food preparation chamber 102 compared to that resulting from adoption of the first radiation status, the aperture size may be reduced and/or the shutter speed increased and/or the sensitivity decreased to compensate for this higher intensity of visible light.

Alternatively or additionally, the colour balance, e.g. white balance, may be adjusted to compensate for a different spectral composition of visible light resulting from adoption of the second radiation status after adoption of the first radiation status.

More generally, the one or more processors 112, the image processing module 116 and/or the control unit 110 can be implemented in numerous ways, with software and/or hardware, to perform the various functions required. The processor(s) 112, the image processing module 116 and/or the control unit 110 may, for example, employ one or more microprocessors programmed using software (e.g., microcode) to perform the required functions. Examples of processor components that may be employed in various embodiments of the present disclosure include, but are not limited to, conventional microprocessors, application specific integrated circuits (ASICs), and field -programmable gate arrays (FPGAs).

In various implementations, the one or more processors 112, the image processing module 116 and/or the control unit 110 may be associated with one or more storage media such as volatile and non-volatile computer memory such as RAM, PROM, EPROM, and EEPROM. The storage media may be encoded with one or more programs that, when executed on one or more processors and/or controllers, perform the required functions. Various storage media may be fixed within a processor or controller or may be transportable, such that the one or more programs stored thereon can be loaded into the one or more processors 112, the image processing module 116 and/or the control unit 110.

Whilst the image processing can be implemented on the image processing module 116 included in the camera module 104, this is not intended to be limiting, and in other examples the image processing module 116 may be alternatively or additionally included in a cloud-based server and/or in a separate user device, such as a smart phone or tablet computer.

In at least some embodiments, the food preparation assembly 100 is a domestic food preparation assembly 100, e.g. a domestic food preparation assembly comprising a domestic kitchen appliance.

Examples of such a domestic kitchen appliance include an air fryer, oven, steamer, pressure cooker or rice cooker. In some embodiments, the camera assembly 104 is integral to such a domestic kitchen appliance.

In alternative embodiments, the camera assembly 104 is a removable or detachable component of the food preparation assembly 100, e.g. domestic food preparation assembly. A window (not visible) into the food preparation chamber 102 may be provided, and the camera assembly 104 may be arranged such as to obtain the image(s) inside the food preparation chamber 102 via the window in order to assemble the food preparation assembly 100, e.g. the domestic food preparation assembly 100.

For instance, the camera assembly 104, and optionally at least one of the one or more processors 112, may be included in a separate user device, such as a smart phone or tablet computer. The processor(s) 112 may alternatively or additionally be included in a cloud-based server. The food preparation assembly 100, e.g. domestic food preparation assembly 100, may be assembled by arranging the camera assembly 104 such as to obtain the image(s) inside the food preparation chamber 102 via the window.

In some embodiments, the food preparation chamber 102 and the radiation assembly 106A, 106B, 108A, 108B are included in a kitchen appliance, e.g. domestic kitchen appliance, and the processor(s) 112 included in the user device and/or the cloud-based server is or are in communication, e.g. wireless communication, with the kitchen appliance, e.g. with the above-described control unit 110, in order to enable the processor(s) 112 to identify the radiation status and control the camera assembly 104 to adjust the image characteristic of the one or more images.

In a non-limiting example, the food preparation assembly 100 is a kitchen appliance, e.g. cooking appliance, in which the camera assembly 104, the radiation assembly 106A, 106B, 108A, 108B, e.g. comprising heating element(s) 108A, 108B and/or illumination system/lamps 106A, 106B, processor(s) 112 and control unit 110 are integral components, as shown in FIG. 1. In such an example the kitchen appliance can also include a data processing unit (e.g. comprising the abovedescribed image processing module 116, and a data analysis module), and a storage unit for storing data.

FIG. 3 provides a flowchart of a method 200 of operating a camera assembly according to an example. The camera assembly may be suitable for obtaining one or more images inside a food preparation chamber of a food preparation assembly further comprising a radiation assembly controllable to adopt a radiation status in which the radiation assembly emits radiation comprising visible light inside the food preparation chamber. The food preparation assembly may, for example, be the food preparation assembly 100 according to any of the embodiments described above.

The method 200 comprises identifying 202 the radiation status, and controlling 204 the camera assembly to adjust an image characteristic of the one or more images using a predetermined relationship for adjusting the image characteristic according to the identified radiation status.

The method 200 may further comprise operating 206 the camera assembly to obtain one or more images inside the food preparation chamber.

The image characteristic may be adjusted according to the identified radiation status in the obtained one or more images, as previously described.

Whilst the operating 206 is shown in FIG. 3 taking place after the controlling 204 the camera assembly to adjust an image characteristic of the one or more images using the predetermined relationship, these operations can be implemented in any suitable order.

For example, the camera assembly may be operated to acquire the image(s) and the image processing setting(s) may be subsequently adjusted such that at least part of 204 can take place after 206.

In some embodiments, the radiation assembly is controllable to select the radiation status from a plurality of radiation statuses, and the identifying 202 comprises identifying the selected radiation status, with the predetermined relationship permitting adjusting of the image characteristic according to the identified selected radiation status.

In embodiments in which the radiation assembly is controllable to select the radiation status from the plurality of radiation statuses, the plurality of radiation statuses can comprise: a first radiation status; and a second radiation status, with the first radiation status causing a first optical condition in the food preparation chamber, and the second radiation status causing a second optical condition in the food preparation chamber which is different from the first optical condition. In such embodiments, the predetermined relationship may be determined such that the image characteristic is adjusted to compensate for the difference between the first optical condition and the second optical condition, as previously described.

FIG. 4 graphically represents adjustment of shutter speed and white balance according to statuses of a radiation assembly as cooking progresses. In this non-limiting example, the two lamps 106A, 106B are continuously illuminating the inside of the food preparation chamber 102 during the entire cooking process, but the heating element 108A and the heating element 108B simultaneously shift between on and off radiation statuses in order to control the temperature within the food preparation chamber 102 during the cooking process.

When the first radiation status (“Status 1”) is adopted by both of the heating elements 108A, 108B being on, the illumination intensity in the food preparation chamber 102 may be higher and the colour temperature may be lower than when the second radiation status (“Status 2”) is adopted by both of the heating elements 108A, 108B being off. Accordingly, the shutter speed may be increased, in this case such that the exposure time decreases from 1/100 s to 1/150 s, and the white balance may be adjusted from 5500 K to 7500 K in response to the radiation status changing from the second radiation status (“Status 2”) to the first radiation status (“Status 1”). The adjustment in these settings may assist to compensate for the change in the illumination intensity and colour temperature caused by the change in radiation status.

These first and second radiation statuses are shown in Table 1. The optical condition, in this case illumination intensity and color temperature, in the food preparation chamber 102 resulting from each of the radiation statuses is measured for each of the first and second radiation statuses before the cooking process. Suitable image capture setting(s) and/or image processing setting(s) are then determined, as also shown in the Table 1.

Table 1

During the cooking process, as also illustrated in FIG. 4, the radiation status can be identified from the power consumption of the radiation assembly 106A, 106B, 108A, 108B and/or obtained from the control unit 110, then the image capture setting(s) and/or the image processing setting(s) is or are selected according to Table 1.

A computer program comprising computer program code may be configured, when said computer program is run on one or more processors 112 included in a food preparation assembly 100 further comprising a food preparation chamber 102, a camera assembly 104 for obtaining one or more images inside the food preparation chamber, and a radiation assembly 106A, 106B, 108A, 108B controllable to adopt a radiation status in which the radiation assembly emits radiation comprising visible light inside the food preparation chamber 102, to cause said one or more processors 112 to implement the method 200 according to any of the embodiments described herein.

The one or more processors 112 can be included in a kitchen appliance, e.g. a kitchen appliance comprising the food preparation chamber 102 and the radiation assembly 106A, 106B, 108A, 108B, which kitchen appliance is included in the food preparation assembly. Alternatively or additionally, the one or more processors 112 can be included in a user device, for example a smart phone or tablet computer, separate from such a kitchen appliance, and/or in a cloud-based server.

FIG. 5 provides a flowchart of a method 300 of determining a relationship for adjusting an image characteristic of one or more images inside a food preparation chamber of a food preparation assembly comprising a radiation assembly; the one or more images being obtained using a camera assembly.

The food preparation assembly may, for example, be the food preparation assembly 100 according to any of the embodiments described above.

The method 300 comprises identifying 302 at least one radiation status of the radiation assembly in which radiation comprising visible light is emitted inside the food preparation chamber, and measuring 304, for each of the at least one radiation status, an optical condition. The optical condition can, for instance, be measured inside the food preparation chamber 102, e.g. using one or more suitable optical sensors.

The method 300 further comprises determining 306, using the optical condition measured for each of the identified at least one radiation status, said relationship for adjusting the image characteristic according to the identified at least one radiation status.

This method 300 may be used to determine the above -de scribed predetermined relationship for adjusting the image characteristic according to the identified radiation status.

FIG. 6 provides a flowchart for illustrating how the method 300 shown in FIG. 5 can be applied to an exemplary method 200 of operating a camera assembly. FIG. 6 may, for example, be regarded as illustrating a workflow of the development of embodiments of the present disclosure.

The method 300 may be regarded as being included in a product development phase, and the method 200 can be regarded as being included in a cooking phase, e.g. during use, of the food preparation assembly 100.

At least one radiation status of the radiation assembly in which radiation comprising visible light is emitted inside the food preparation chamber is identified in 302. This identifying 302 may comprise identifying all potential combinations of components capable of forming part of the radiation assembly, in other words the heating element(s) and/or the illumination system/lamp(s), and the possible radiation statuses of the radiation assembly which can be selected by controlling these components.

In 304, the optical condition is measured for each of the possible radiation statuses identified in step 302. The optical condition can, for instance, be measured inside the food preparation chamber 102, e.g. using one or more suitable optical sensors, as previously described.

The determining 306, using the optical condition measured for each of the identified at least one radiation status, the relationship for adjusting the image characteristic according to the identified at least one radiation status is then implemented. Turning to the method 200 included in the cooking phase in the example shown in FIG. 6, the identifying 202 the radiation status comprises obtaining 202A the individual radiation statuses of all components capable of forming part of the radiation assembly, in other words the heating element(s) and/or the illumination system/lamp(s). This can be achieved by obtaining the individual statuses from the above -de scribed control unit and/or via sensing the radiation provided by the component(s). The radiation status, in other words the overall or “combination” status, of the radiation assembly can then be identified in 202B.

The camera assembly is then controlled in 204 to adjust an image characteristic of the one or more images using the above-described predetermined relationship determined in 306.

In summary, the present disclosure proposes to control a camera assembly to adjust an image characteristic, e.g. exposure and/or white balance, for image acquisition and processing to obtain image(s)/video that can reliably reflect food appearance.

Variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality.

The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

If the term "adapted to" is used in the claims or description, it is noted the term "adapted to" is intended to be equivalent to the term "configured to".

Any reference signs in the claims should not be construed as limiting the scope.

Claims

22 CLAIMS:

1. A food preparation assembly (100) comprising: a food preparation chamber (102); a camera assembly (104) for obtaining one or more images inside the food preparation chamber, the camera assembly being controllable to adjust an image characteristic of the one or more images; a radiation assembly (106A, 106B, 108A, 108B) controllable to adopt a radiation status in which the radiation assembly emits radiation comprising visible light inside the food preparation chamber, the radiation assembly being controllable to select the radiation status from a plurality of radiation statuses comprising a first radiation status and a second radiation status, the first radiation status causing a first optical condition in the food preparation chamber, and the second radiation status causing a second optical condition in the food preparation chamber which is different from the first optical condition; and one or more processors (112) configured to: identify the selected radiation status; and control the camera assembly to adjust the image characteristic using a predetermined relationship for adjusting the image characteristic according to the identified selected radiation status, wherein the predetermined relationship is determined such that the image characteristic is adjusted to compensate for the difference between the first optical condition and the second optical condition.

2. The food preparation assembly (100) according to claim 1, wherein the first radiation status and the second radiation status are adopted in a control sequence that controls the radiation status of the radiation assembly (106A, 106B, 108A, 108B) to change over time, the one or more processors (112) being configured to control the camera assembly to adjust the image characteristic, using the predetermined relationship, to coincide with the change of the radiation status.

3. The food preparation assembly (100) according to claim 2, wherein the camera assembly (104) is configured to obtain a sequence of images of the inside of the food preparation chamber (102) during said change in radiation status.

4. The food preparation assembly (100) according to any of claims 1 to 3, wherein the radiation assembly (106A, 106B, 108A, 108B) comprises at least one lamp (106A, 106B) for illuminating the inside of the food preparation chamber (102), and wherein the radiation status comprises an illumination status of the at least one lamp.

5. The food preparation assembly (100) according to any of claims 1 to 4, wherein the radiation assembly (106A, 106B, 108A, 108B) comprises at least one heating element (108A, 108B) arranged to radiate inside the food preparation chamber (102), and wherein the radiation status comprises a heating status of the at least one heating element.

6. The food preparation assembly (100) according to any of claims 1 to 5, comprising a control unit (110) having a control configuration arranged to control the radiation assembly (106A, 106B, 108A, 108B) to adopt the radiation status, and wherein the one or more processors (112) is or are configured to identify the radiation status based on said control configuration.

7. The food preparation assembly (100) according to any of claims 1 to 6, comprising a sensor for sensing radiation of the radiation assembly (106A, 106B, 108A, 108B), and wherein the one or more processors (112) is or are configured to identify the radiation status based on the sensed radiation.

8. The food preparation assembly (100) according to any of claims 1 to 7, wherein the camera assembly (104) comprises an image capture module (114), and wherein the one or more processors (112) is or are configured to control at least one image capture setting of the image capture module according to the identified radiation status to adjust the image characteristic.

9. The food preparation assembly (100) according to claim 8, wherein the at least one image capture setting comprises an aperture size, a shutter speed and/or a sensitivity setting.

10. The food preparation assembly (100) according to any of claims 1 to 9, comprising an image processing module (116), and wherein the one or more processors (112) is or are configured to control at least one image processing setting of the image processing module according to the identified radiation status to adjust the image characteristic; optionally wherein the image processing module is included in at least one of the camera module (104), a user device, and a cloud-based server.

11. The food preparation assembly (100) according to claim 10, wherein the at least one processing setting comprises a colour balance setting; optionally wherein the colour balance setting is a white balance setting.

12. A method (200) of controlling a camera assembly suitable for obtaining one or more images inside a food preparation chamber of a food preparation assembly, the food preparation assembly further comprising a radiation assembly controllable to adopt a radiation status in which the radiation assembly emits radiation comprising visible light inside the food preparation chamber, the radiation assembly being controllable to select the radiation status from a plurality of radiation statuses comprising a first radiation status and a second radiation status, the first radiation status causing a first optical condition in the food preparation chamber, and the second radiation status causing a second optical condition in the food preparation chamber which is different from the first optical condition, the method comprising: identifying (202) the selected radiation status; and controlling (204) the camera assembly to adjust an image characteristic of the one or more images using a predetermined relationship for adjusting the image characteristic according to the identified selected radiation status, wherein the predetermined relationship is determined such that the image characteristic is adjusted to compensate for the difference between the first optical condition and the second optical condition.

13. The method (200) according to claim 12, wherein the first radiation status and the second radiation status are adopted in a control sequence that controls the radiation status of the radiation assembly to change over time, the controlling (204) the camera assembly to adjust, using the predetermined relationship, the image characteristic coinciding with the change of the radiation status.

14. The method (200) according to claim 12 or claim 13, comprising operating (206) the camera assembly to obtain one or more images inside the food preparation chamber.

15. The method (200) according to claim 14 when dependent from claim 13, wherein the operating (206) comprises obtaining a sequence of images of the inside of the food preparation chamber during said change in radiation status.

16. A computer program comprising computer program code which is configured, when said computer program is run on one or more processors included in a food preparation assembly further comprising a food preparation chamber, a camera assembly for obtaining one or more images inside the food preparation chamber, and a radiation assembly controllable to adopt a radiation status in which the radiation assembly emits radiation comprising visible light inside the food preparation chamber, to cause said one or more processors to implement the method of any of claims 12 to 15.

17. A method (300) of determining a relationship for adjusting an image characteristic of one or more images inside a food preparation chamber of a food preparation assembly comprising a 25 radiation assembly, the one or more images being obtained using a camera assembly, the method comprising: identifying (302) a first radiation status and a second radiation status of the radiation assembly in each of which radiation comprising visible light is emitted inside the food preparation chamber; measuring (304), for each of the at least one radiation status, an optical condition such that a first optical condition is measured for the first radiation status and a second optical condition is measured for the second radiation status; and determining (306), using said first and second optical conditions respectively measured for each of the first and second radiation statuses, said relationship for adjusting the image characteristic according to the identified radiation status such that the image characteristic is adjusted to compensate for a difference between the first optical condition and the second optical condition.