CN118369542A

CN118369542A - Food preparation assembly including camera assembly

Info

Publication number: CN118369542A
Application number: CN202280077650.9A
Authority: CN
Inventors: 肖卫民; 谭径微; 孙雯
Original assignee: Fansongni Holdings Ltd
Current assignee: Fansongni Holdings Ltd
Priority date: 2021-11-26
Filing date: 2022-10-19
Publication date: 2024-07-19
Also published as: WO2023094078A1

Abstract

A food preparation assembly (100) is provided that includes a food preparation chamber (102) and a camera assembly (104) for obtaining one or more images of an interior of the food preparation chamber. The camera assembly is controllable to adjust image characteristics of one or more images. The food preparation assembly comprises a radiation assembly (106 a,106b,108a,108 b) controllable to adopt a radiation state in which the radiation assembly emits radiation comprising visible light inside the food preparation chamber. The one or more processors (112) are configured to: identifying a radiation state; and controlling the camera assembly to adjust the image characteristic using a predetermined relationship for adjusting the image characteristic in accordance with the identified radiation status. A method of controlling a camera assembly adapted to obtain one or more images of a food preparation chamber interior of a food preparation assembly, a computer program for implementing the method, and a method of determining a relationship for adjusting image characteristics of one or more images of a food preparation chamber interior of a food preparation assembly are also provided.

Description

Food preparation assembly including camera assembly

Technical Field

The present invention relates to a food preparation assembly having a food preparation chamber and a camera assembly for obtaining one or more images of the interior of the food preparation chamber.

The invention also relates to a method of controlling a camera assembly adapted to obtain one or more images of the interior of a food preparation chamber of a food preparation assembly, and a computer program for implementing the method.

The invention also relates to a method of determining a relationship for adjusting image characteristics of one or more images of the interior of a food preparation chamber of a food preparation assembly.

Background

People often cook in the home for very limited time due to busy daily lives. In addition, many people lack the cooking experience necessary to obtain satisfactory results. Accordingly, there is an increasing demand for intelligent food preparation assemblies capable of delivering prepared food products according to the preferences of users. Benefits of such intelligent food preparation assemblies include: the convenience for users is greater, the taste and nutritional value of the food prepared using such a food preparation assembly is better, and the generation of harmful substances during cooking using such a food preparation assembly is minimized or avoided.

The inclusion of a camera in a food preparation assembly, such as by integrating the camera into a kitchen appliance, may play an important role in intelligent auxiliary food preparation (e.g., cooking). For example, a food item type may be identified by food item image recognition, and a cooking setting may be determined based on food item image analysis results. Other applications of cameras in food preparation assemblies include cooking process control, browning control, remote monitoring, anti-burn, and social media sharing content.

Due to the above benefits, camera-integrated kitchen appliances have recently been introduced into the market. However, various challenges remain in obtaining images that can be reliably used, inter alia, for food identification, cooking process control, browning control, remote monitoring, and anti-burn.

US2018/324908 A1 discloses a system and method related to a cooking appliance having: a heating element disposed within the cooking chamber and operable to selectively emit waves of any of a plurality of power and/or peak wavelengths; a camera operable to capture an image of the cooking chamber; and a computing device operable to power the heating element to change the power and/or peak wavelength of the emitted wave and generate heat within the cooking chamber, and instruct the camera to capture an image while the heating element is emitting at a steady power and/or peak wavelength.

WO 2016/179424 A1 discloses a connected oven comprising a set of intra-cavity sensors and a processor configured to automatically identify food in a cooking cavity based on sensor measurements and to automatically operate a heating element based on food identification.

Disclosure of Invention

The invention is defined by the claims.

According to an example of an aspect of the invention, there is provided a food preparation assembly comprising: a food preparation chamber; a camera assembly for obtaining one or more images of an interior of the food preparation chamber, the camera assembly being controllable to adjust image characteristics of the one or more images; a radiation assembly controllable to adopt a radiation state in which the radiation assembly emits radiation including visible light inside the food preparation chamber; and one or more processors configured to identify a radiation state and control the camera assembly to adjust the image characteristic using a predetermined relationship for adjusting the image characteristic according to the identified radiation state.

Using such predetermined relationships to adjust image characteristics (e.g., exposure and/or white balance) based on the identified radiation status of the radiation assembly may help provide faster and/or more repeatable image characteristic adjustments relative to, for example, adjustments based solely on optical conditions detected inside the food preparation chamber when the image is obtained.

In some embodiments, the camera assembly may be configured to obtain a sequence of images of the interior of the food preparation chamber, e.g., a sequence of images of a video that constitutes the interior of the food preparation chamber.

Alternatively or additionally, the camera assembly may be configured to obtain a still image of the interior of the food preparation chamber.

By pre-identifying the radiation status and determining a predetermined relationship for adjusting image characteristics (e.g., camera assembly capture settings and/or image processing settings) based on the radiation status, the image may be more consistent with, for example, a "real" view of the macroscopic perceivable food ingredient.

By controlling the camera assembly to use predetermined relationships, such as adjusting exposure and/or white balance, to make the image more consistent with such "true" views of the food ingredients, more accurate/reliable food identification, cooking process control, browning control, doneness status determination, remote monitoring, and/or anti-burn may be achieved.

In some embodiments, the radiation assembly is controllable to select a radiation state from a plurality of radiation states, and the one or more processors are configured to identify the selected radiation state, wherein the predetermined relationship allows for adjustment of the image characteristic in accordance with the identified selected radiation state.

The image characteristics may be adjusted accordingly depending on which of a plurality of radiation states is selected.

In embodiments where the radiating assembly is controllable to select a radiating state from a plurality of radiating states, the plurality of radiating states may include: a first radiation state; and a second radiation state, wherein the first radiation state causes a first optical condition in the food preparation chamber and the second radiation state causes a second optical condition in the food preparation chamber that is different from the first optical condition.

In such embodiments, the predetermined relationship may be determined such that the image characteristics are adjusted to compensate for the difference between the first optical condition and the second optical condition.

By compensating for the difference between the first optical condition and the second optical condition, the image obtained when the first radiation state is employed may be comparable to the image obtained when the second radiation state is employed.

For example, if the optical condition includes the intensity of visible light inside the food preparation chamber, the adjustment of the image characteristic includes an adjustment of the image exposure to compensate for the difference in intensity. Alternatively or additionally, if the optical condition comprises a spectral composition of light inside the food preparation chamber, the adjustment of the image characteristic comprises an adjustment of a color (e.g. white) balance of the image to compensate for the difference in spectral composition.

It should be noted that for the avoidance of doubt, the plurality of radiation states may include further radiation states, such as a third radiation state, a fourth radiation state, a fifth radiation state, etc., in addition to the first radiation state and the second radiation state described above. In such embodiments, the predetermined relationship may be determined such that the image characteristics are adjusted to compensate for differences between the respective optical conditions caused by each of the selected radiation states.

In some embodiments, the radiation assembly comprises at least one lamp for illuminating the interior of the food preparation chamber. In such an embodiment, the radiation state comprises, for example, an illumination state of each of the at least one lamp.

The illumination state may be, for example, an on/off state of each of the at least one lamp.

Alternatively or additionally, the illumination state may be an illumination intensity provided by, for example, each of the at least one lamp, and/or the illumination state may be a measure of a spectral composition of the illumination provided by, for example, each of the at least one lamp.

In some embodiments, the radiation assembly comprises at least one heating element arranged to radiate inside the food preparation chamber. In such embodiments, the radiation state comprises, for example, a heating state of each of the at least one heating element.

The heating state may be, for example, an on/off state of each of the at least one heating element.

In embodiments where the radiant state comprises an illumination state of the at least one lamp and a heating state of the at least one heating element, the illumination state may be an on/off state of the at least one lamp, wherein the heating state is an on/off state of the at least one heating element.

Alternatively or additionally, the heating state may be a degree of heating provided by, for example, each of the at least one heating element, and/or the heating state may be a measure of a spectral composition of radiation provided by, for example, each of the at least one heating element.

The food preparation assembly may comprise a control unit having a control configuration arranged to control the radiation assembly to adopt the radiation state. In such embodiments, the one or more processors are configured to identify the radiation status based on the control configuration.

The radiation status can thus be obtained from a control unit controlling the radiation assembly, in other words directly. This may advantageously avoid sensing optical conditions inside the food preparation chamber for adjusting image characteristics.

In embodiments where the control configuration includes a control sequence that controls the change in the radiation state of the radiation assembly over time, the one or more processors may be configured to control the camera assembly (using a predetermined relationship) to adjust the image characteristics to conform to the change in the radiation state. This may enable particularly sensitive adjustment of image characteristics, which may be particularly useful, for example, when the camera assembly is configured to obtain a sequence of images (e.g., video) during a change in radiation state.

In some embodiments, the radiation state (e.g., each radiation state when the radiation state may be selected from a plurality of radiation states) is associated with a power consumption of the radiation assembly, and the one or more processors are configured to identify the radiation state based on the (respective) power consumption of the radiation assembly.

The food preparation assembly may include a sensor for sensing radiation of the radiation assembly. In such embodiments, the one or more processors are configured to identify a radiation state based on the sensed radiation.

In this way, the radiation status can be identified from the radiation detected via the sensor.

In some embodiments, the camera assembly includes an image capture module, and the one or more processors may be configured to control at least one image capture setting of the image capture module to adjust the image characteristic in accordance with the identified radiation status.

Thus, the identified radiation status may be used to adjust the acquisition of the image.

The at least one image capture setting may include, for example, aperture size, shutter speed, and/or sensitivity settings.

The food preparation assembly may include an image processing module, and the one or more processors may be configured to control at least one image processing setting of the image processing module to adjust the image characteristic in accordance with the identified radiation status.

The image processing module may be included, for example, in a camera component, a user device (such as a smart phone or tablet computer), and/or a cloud-based server.

In such embodiments, the identified radiation status is used to adjust the processing of the image by the image processing module.

The at least one processing setting may for example comprise a color balance setting. Such a color balance setting may be, for example, a white balance setting.

In at least some embodiments, the food preparation assembly comprises or is a household food preparation assembly. For example, the food preparation assembly includes a household kitchen appliance.

Examples of such home kitchen appliances include air fryers, ovens, steamers, autoclaves and rice cookers.

In particular, an embodiment is mentioned in which the camera assembly is integrated with (in other words built-in) such a household kitchen appliance.

In an alternative embodiment, the camera assembly is a removable or detachable component of a food preparation assembly (e.g., a home food preparation assembly). A window into the food preparation chamber may be provided and the camera assembly may be arranged to obtain an image of the interior of the food preparation chamber, for example via the window, in order to assemble a food preparation assembly, for example a domestic food preparation assembly.

For example, the camera assembly and optionally at least one of the one or more processors may be included in a user device (such as a smart phone or tablet) separate from the kitchen appliance including the food preparation chamber and the radiation assembly. Alternatively or additionally, the processor may be included in a cloud-based server. A food preparation assembly (e.g., a home food preparation assembly) may be assembled by arranging a camera assembly to obtain an image of the interior of the food preparation chamber via a window.

In some embodiments, the food preparation chamber and the radiation assembly are included in a kitchen appliance (e.g., a home kitchen appliance), and a processor included in the user device and/or the cloud-based server communicates (e.g., wirelessly communicates) with the kitchen appliance (e.g., with the control unit described above) to enable the processor to identify the radiation status and control the camera assembly to adjust image characteristics of one or more images using a predetermined relationship.

According to another aspect, there is provided a method of controlling a camera assembly adapted to obtain one or more images of an interior of a food preparation chamber of a food preparation assembly, the food preparation assembly further comprising a radiation assembly controllable to adopt a radiation state in which the radiation assembly emits radiation comprising visible light within the food preparation chamber, the method comprising: identifying a radiation state; and controlling the camera assembly to adjust image characteristics of the one or more images using a predetermined relationship for adjusting image characteristics according to the identified radiation status.

The method may further include operating the camera assembly to obtain one or more images of the interior of the food preparation chamber.

The image characteristics may be adjusted based on the radiation conditions identified in the acquired one or more images.

In some embodiments, the radiation assembly is controllable to select a radiation state from a plurality of radiation states, and the identifying includes identifying the selected radiation state, wherein the predetermined relationship allows for adjustment of the image characteristic in accordance with the identified selected radiation state.

In some embodiments, the radiation assembly comprises at least one lamp for illuminating the interior of the food preparation chamber. In such embodiments, identifying includes identifying, for example, an illumination state of each of the at least one lamp.

In some embodiments, the radiation assembly comprises at least one heating element arranged to radiate inside the food preparation chamber. In such embodiments, identifying includes identifying a heating state of each of the at least one heating element, for example.

In embodiments where the radiation assembly includes at least one lamp and at least one heating element, identifying may include identifying an illumination state of the at least one lamp and identifying a heating state of the at least one heating element. In such an embodiment, the illumination state may be, for example, an on/off state of the at least one lamp, wherein the heating state is an on/off state of the at least one heating element.

The food preparation assembly may comprise a control unit having a control configuration arranged to control the radiation assembly to adopt the radiation state. In such embodiments, identifying may include identifying the radiation status based on the control configuration.

In some embodiments, the method includes determining a power consumption of the radiating component, wherein identifying includes identifying a radiation state based on the determined power consumption.

The food preparation assembly may include a sensor for sensing radiation of the radiation assembly. In such embodiments, identifying may include identifying a radiation state based on the sensed radiation.

In some embodiments, the camera assembly includes an image capture module, and the controlling includes controlling at least one image capture setting of the image capture module to adjust the image characteristic in accordance with the identified radiation status. The at least one image capture setting may include, for example, aperture size, shutter speed, and/or sensitivity settings.

Alternatively or additionally, the food preparation assembly may comprise an image processing module and the controlling comprises controlling at least one image processing setting of the image processing module to adjust the image characteristic in dependence of the identified radiation status. The at least one processing setting may for example comprise a color balance setting. Such a color balance setting may be, for example, a white balance setting.

According to yet another aspect, there is provided a computer program comprising computer program code configured to cause one or more processors comprised in a food preparation assembly, the food preparation assembly further comprising a food preparation chamber, a camera assembly for obtaining one or more images of the interior of the food preparation chamber, and a radiation assembly, the radiation assembly being controllable to assume a radiation state in which the radiation assembly emits radiation comprising visible light inside the food preparation chamber, when the computer program is run on the one or more processors.

One or more non-transitory computer-readable media may be provided having a computer program stored thereon, the computer program comprising computer program code configured to cause one or more processors to implement a method according to any of the embodiments described herein when the computer program is run on the one or more processors.

The one or more processors may be included in kitchen appliances in the food preparation assembly, in user equipment (e.g., a smart phone or tablet) separate from such kitchen appliances, and/or in a cloud-based server.

According to another aspect, there is provided a method of determining a relationship for adjusting image characteristics of one or more images of a food preparation chamber interior of a food preparation assembly comprising a radiation assembly, the one or more images being obtained using a camera assembly, the method comprising: identifying at least one radiation state of the radiation assembly, wherein radiation comprising visible light is emitted within the food preparation chamber; measuring an optical condition for each of the at least one radiation state; and determining the relationship for adjusting image characteristics according to the identified at least one radiation state using the measured optical condition for each of the identified at least one radiation state.

The method may be used to establish/determine the above-mentioned predetermined relationship for adjusting the image characteristics in accordance with the identified radiation status.

More generally, embodiments described herein related to a food preparation assembly and a method of controlling a camera assembly may be applicable to a method of determining a relationship, and embodiments described herein related to a method of determining a relationship may be applicable to a food preparation assembly and a method of controlling a camera assembly.

Drawings

For a better understanding of the invention, and to show more clearly how it may be carried into effect, reference will now be made, by way of example only, to the accompanying drawings in which:

FIG. 1 schematically depicts a food preparation assembly according to an example;

FIG. 2 provides a block diagram of a camera assembly according to an example;

FIG. 3 provides a flow chart of a method of operating a camera assembly according to an example;

FIG. 4 graphically illustrates adjusting shutter speed and white balance as cooking progresses according to the radiation status of the radiation assembly;

FIG. 5 provides a flowchart of an exemplary method of determining a relationship for adjusting image characteristics of one or more images; and

FIG. 6 provides a flow chart illustrating an exemplary method of how the relationship determined by the method shown in FIG. 5 may be applied to operate a camera assembly.

Detailed Description

The present invention will be described with reference to the accompanying drawings.

It should be understood that the detailed description and specific examples, while indicating exemplary embodiments of the apparatus, system, and method, are intended for purposes of illustration only and are not intended to limit the scope of the invention. These and other features, aspects, and advantages of the apparatus, system, and method of the present invention will become better understood with regard to the following description, appended claims, and accompanying drawings. It should be understood that the figures are merely schematic and are not drawn to scale. It should also be understood that the same reference numerals are used throughout the drawings to designate the same or similar parts.

A food preparation assembly, which may also be referred to as a food preparation system, is provided that includes a food preparation chamber and a camera assembly for obtaining one or more images of the interior of the food preparation chamber. The camera assembly is controllable to adjust image characteristics of one or more images. The food preparation assembly includes a radiation assembly controllable to adopt a radiation state in which the radiation assembly emits radiation including visible light inside the food preparation chamber. The one or more processors are configured to: identifying a radiation state; and controlling the camera assembly to adjust the image characteristic using a predetermined relationship for adjusting the image characteristic in accordance with the identified radiation status. A method of controlling a camera assembly adapted to obtain one or more images of a food preparation chamber interior of a food preparation assembly, a computer program for implementing the method, and a method of determining a relationship for adjusting image characteristics of one or more images of a food preparation chamber interior of a food preparation assembly are also provided.

For a number of reasons, it is desirable to obtain one or more images of the interior of the food preparation chamber. In particular, one or more images may be used to identify food products present in the food preparation chamber, cooking process monitoring, and/or cooking process control based on image analysis.

For such applications, stable image quality is particularly important. However, ensuring a sufficiently consistent image quality can be challenging, particularly due to variations in optical conditions inside the food preparation chamber caused by adjustments to the operation of a radiation assembly also included in the food preparation assembly that emits radiation including visible light inside the food preparation chamber.

For example, a heating element included in the food preparation assembly may be switched between on and off states during cooking. As the heating element switches between these states, the illumination intensity and color temperature in the food preparation chamber may thus vary significantly. For example, if the white balance and exposure are not adjusted, the color and brightness of the image may be correspondingly significantly affected. Thus, the ability of the image to represent the appearance of the food product may be adversely affected. This can be particularly difficult in food product identification using images, cooking process control, browning control, doneness status determination, remote monitoring, and/or anti-burn.

While the camera assembly may itself detect optical conditions within the food preparation chamber and adjust image characteristics of the image based on such detected optical conditions, for example, such adjustments may be subject to hysteresis and/or the adjustments may not be sufficiently repeatable to ensure that the quality of the image is sufficiently consistent.

The present disclosure is based on the recognition that identifying the radiation status of the radiation assembly and adjusting image characteristics (such as exposure and/or white balance) using a predetermined relationship according to the identified radiation status may help provide faster and/or more repeatable image characteristic adjustment relative to, for example, an adjustment based solely on the optical conditions detected inside the food preparation chamber when the image is obtained.

Thus, the image thus obtained may be effectively used for example for food identification, cooking process control, browning control, doneness status determination, remote monitoring and/or anti-burn based on image analysis.

In at least some embodiments, the present disclosure provides a method of compensating for changes in illuminance and/or color temperature caused by a radiant assembly (e.g., a heating element and/or an illumination system/lamp) transitioning between states.

For example, such methods may adjust exposure and/or white balance for a kitchen appliance (e.g., a cooking device) in which the camera assembly is integrated during cooking.

Fig. 1 schematically depicts a food preparation assembly 100 according to an example. The food preparation assembly 100 comprises a food preparation chamber 102 in which a food ingredient 103 may be accommodated.

The food preparation assembly 100 includes a camera assembly 104 for obtaining one or more images of the interior of the food preparation chamber 102. Depending on how the images are used, for example, for food item identification, cooking process monitoring, and/or cooking process control, the camera assembly 104 may be configured to obtain a sequence of images, for example, a sequence of images that constitute a video. Alternatively or additionally, the camera assembly 104 may be configured to obtain still images of the interior of the food preparation chamber 102.

The camera assembly 104 may be arranged to enable one or more images of the food ingredients 103 contained in the food preparation chamber 102 to be obtained.

In the non-limiting example shown in fig. 1, the camera assembly 104, or at least the optical sensing elements of the camera assembly 104, are arranged at or near the top plate of the food preparation chamber 102 in order to obtain an image of the food ingredients from above. Alternatively or additionally, the camera assembly 104, or at least the optical sensing element thereof, is arranged at one side of the food preparation chamber 102, e.g. between the base and the ceiling of the food preparation chamber 102.

The food preparation assembly 100 includes radiation assemblies 106A, 106B, 108A, 108B configured to emit radiation including visible light inside the food preparation chamber 102. In particular, the radiation assemblies 106A, 106B, 108A, 108B can be controlled, for example, by a user of the food preparation assembly 100 to adopt a radiation state in which the radiation assemblies 106A, 106B, 108A, 108B emit radiation including visible light inside the food preparation chamber 102.

In some embodiments, as shown in fig. 1, the radiation assembly 106A, 106B, 108A, 108B includes at least one lamp 106A, 106B for illuminating the interior of the food preparation chamber 102. In such embodiments, the radiation state may include, for example, an illumination state of each of the at least one lamps 106A, 106B.

The lights 106A, 106B may be controlled, for example, by a user, such as by operating a suitable user interface included in the food preparation assembly 100 to enable the user to control the lights 106A, 106B. This may help a user visually monitor the processes occurring in the food preparation chamber 102, for example. The light from the lamps 106A, 106B may thus change the optical conditions within the food preparation chamber 102.

The lamps 106A, 106B may be or include light emitting diodes, for example, but any suitable lamp type is contemplated, such as incandescent lamps.

In the non-limiting example shown in fig. 1, the lights 106A, 106B are disposed at or near the ceiling of the food preparation chamber 102, but other locations of such lights 106A, 106B are contemplated, as long as the lights 106A, 106B can illuminate at least a portion of the interior of the food preparation chamber 102.

The illumination state may, for example, include an on/off state of each of the at least one lights 106A, 106B.

Alternatively or additionally, the illumination state may comprise an illumination intensity provided by, for example, each of the at least one lamps 106A, 106B.

Alternatively or additionally, the illumination state may comprise a measure of the spectral composition of the illumination provided by, for example, each of the at least one lamps 106A, 106B.

When at least one lamp 106A, 106B comprises a plurality of lamps 106A, 106B, such as two (as shown in fig. 1), three, four, five, six or more lamps 106A, 106B, one or more of these lamps 106A, 106B may operate independently of the other lamps. In such examples, the illumination state may include a combination of individual illumination states of each of the independently operable lamps 106A, 106B.

In some embodiments, such as the embodiment shown in fig. 1, the radiation assembly 106A, 106B, 108A, 108B includes at least one heating element 108A, 108B arranged to radiate inside the food preparation chamber 102. In such embodiments, the radiant state may include, for example, a heating state of each of the at least one heating element 108A, 108B.

At least one heating element 108A, 108B may be used to heat or cook the food ingredients 103 contained in the food preparation chamber 102. However, the radiation emitted by the heating elements 108A, 108B may include visible light. Such visible light may alter the optical conditions inside the food preparation chamber 102.

The heating elements 108A, 108B may be or include resistive heating elements, for example, but any suitable type of heating element is contemplated.

In the non-limiting example shown in fig. 1, the heating elements 108A, 108B are arranged at or near the top plate of the food preparation chamber 102, but other locations of such heating elements 108A, 108B are contemplated, as long as the heating elements 108A, 108B are capable of emitting thermal radiation into at least a portion of the interior of the food preparation chamber 102.

The heating state may be, for example, an on/off state of each of the at least one heating element 108A, 108B.

Alternatively or additionally, the heating state may be a degree of heating provided by, for example, each of the at least one heating elements 108A, 108B, and/or the heating state may be a measure of a spectral composition of radiation provided by, for example, each of the at least one heating elements 108A, 108B.

When at least one heating element 108A, 108B includes a plurality of heating elements 108A, 108B, such as two (as shown in fig. 1), three, four, five, six, or more heating elements 108A, 108B, one or more of these heating elements 108A, 108B may operate independently of the other elements. In such examples, the heating state may include a combination of individual heating states of each of the independently operable heating elements 108A, 108B.

In embodiments in which the radiant state includes an illumination state of the at least one lamp 106A, 106B and a heating state of the at least one heating element 108A, 108B, the illumination state may be, for example, an on/off state of the at least one lamp 106A, 106B, wherein the heating state is an on/off state of the at least one heating element 108A, 108B.

In some embodiments, as shown in fig. 1, the food preparation assembly 100 includes a control unit 110 having a control configuration arranged to control the radiation assemblies 106A, 106B, 108A, 108B to adopt a radiation state.

Thus, the radiation status may be obtained from the control unit 110 controlling the radiation assembly, in other words directly. This may advantageously avoid sensing optical conditions inside the food preparation chamber 102 in order to adjust image characteristics.

In some embodiments, the food preparation assembly 100 includes a user interface (not visible in fig. 1) configured to allow a user to select the irradiation status. In such embodiments, the control configuration may be generated, for example, corresponding to or from a user selection.

In a non-limiting example, the control unit 110 is configured to receive recipe input (e.g., recipe instruction input) and generate a control configuration based on the received recipe input. In other words, the control configuration may be automatically generated by the control unit 110 from recipe inputs.

In some embodiments, the control configuration includes a control sequence that controls the radiation state of the radiation assemblies 106A, 106B, 108A, 108B over time.

Recipe input may be entered in any suitable manner. For example, the control unit 110 may receive recipe input from a recipe database in response to a user selection from the recipe database.

More generally, due to the above-described effects of the radiation assemblies 106A, 106B, 108A, 108B (e.g., including the lamps 106A, 106B and/or the heating elements 108A, 108B), the optical conditions (e.g., intensity and/or spectral composition of visible light) inside the food preparation chamber 102 may vary depending on the radiation status of the radiation assemblies 106A, 106B, 108A, 108B.

For this reason, the food preparation assembly 100 includes one or more processors 112 configured to identify a radiation status and control the camera assembly 104 to adjust image characteristics using a predetermined relationship to adjust the image characteristics in accordance with the identified radiation status.

Using such predetermined relationships to adjust image characteristics (e.g., exposure and/or white balance) based on the identified radiation status of the radiation assemblies 106A, 106B, 108A, 108B may help provide faster and/or more repeatable image characteristic adjustments relative to, for example, adjustments based solely on the optical conditions detected inside the food preparation chamber 102 when the images are obtained.

In some embodiments, the radiation assemblies 106A, 106B, 108A, 108B are controllable to select a radiation state from a plurality of radiation states, and the one or more processors 112 are configured to identify the selected radiation state, wherein the predetermined relationship allows for adjustment of image characteristics in accordance with the identified selected radiation state.

In embodiments where the radiating assembly is controllable to select a radiating state from a plurality of radiating states, the plurality of radiating states may include: a first radiation state; and a second radiation state, wherein the first radiation state causes a first optical condition in the food preparation chamber 102 and the second radiation state causes a second optical condition in the food preparation chamber 102 that is different from the first optical condition.

In a non-limiting example, the food preparation assembly 100 (e.g., the above-described control unit 110 included in the food preparation assembly 100) is configured to control the heating elements 108A, 108B according to a cooking program in which the heating elements 108A, 108B are controlled such that the food preparation chamber 102 is at a first temperature for a first predetermined duration and then at a second temperature that is lower than the second temperature, e.g., when the doneness level of the food ingredient 103 is approaching.

In this example, the radiation state of the radiation assemblies 106A, 106B, 108A, 108B and in particular the heating state of the heating elements 108A, 108B is changed from a first radiation state to a second radiation state in order to achieve a temperature decrease from a first temperature to a second temperature.

The first radiation state may, for example, correspond to both heating elements 108A, 108B being on, the second radiation state corresponds to one of the heating elements 108A, 108B being on and the other of the heating elements 108B, 108A being off.

The change in optical conditions (e.g., changes in intensity and/or spectral composition of visible light) within the food preparation chamber 102 resulting from the change in radiation status from the first radiation status to the second radiation status may still be compensated for by the one or more processors 112 controlling the camera assembly 104 to adjust the image characteristics. In particular, a predetermined relationship used by the processor 112 may be determined such that the image characteristics are adjusted to compensate for differences between the first optical condition and the second optical condition.

Such compensation may in turn improve image-based control, which may be implemented during cooking, e.g. selecting/estimating the time required at the second predetermined temperature. This is because the images obtained when the radiation assemblies 106A, 106B, 108A, 108B adopt the first radiation state may be more reliably compared to the images obtained when the radiation assemblies 106A, 106B, 108A, 108B adopt the second radiation state.

In embodiments where the control configuration includes the above-described control sequence that controls the change in the radiation state of the radiation assemblies 106A, 106B, 108A, 108B over time, the one or more processors 112 may be configured to control the camera assembly 104 to adjust the image characteristics to conform to the change in the radiation state. This may enable particularly sensitive adjustment of image characteristics, which may be particularly useful, for example, when the camera assembly 104 is configured to obtain a sequence of images (e.g., constituting a video) during a change in radiation state.

Although processor 112 is shown in fig. 1 as receiving data (e.g., control configuration) from control unit 110, this is not intended to be limiting. In other embodiments, which may be an alternative or in addition to the control configuration used by the processor 112 to identify the radiation status, the food preparation assembly 100 may include a sensor (not visible) for sensing radiation of the radiation assemblies 106A, 106B, 108A, 108B. In such embodiments, the one or more processors 112 are configured to identify the radiation status based on the radiation sensed by the sensor.

Alternatively or additionally, the one or more processors 112 may be configured to identify the radiation status based on the power consumption of the radiation components 106A, 106B, 108A, 108B.

Referring to fig. 2, in some embodiments, the camera assembly 104 includes an image capture module 114, and the one or more processors 112 are configured to control at least one image capture setting of the image capture module 114 to adjust the image characteristic in accordance with the identified radiation status. Thus, the identified radiation status may be used to adjust the acquisition of the image.

The at least one image capture setting may for example comprise or be an aperture size, a shutter speed and/or a sensitivity setting.

In at least some embodiments, the camera assembly 104 can also include an image processing module 116. In such embodiments, the image processing module 116 may process the image acquired by the image capture module 114 to obtain an image. The image processing module 116 may alternatively or additionally be included in a user device (such as a smart phone or tablet) separate from the kitchen appliance including the food preparation chamber 102 and/or in a cloud-based server.

In addition to or instead of the above-described control of at least one capture setting of the image capture module 114 included in the camera assembly 104, the one or more processors 112 may be configured to control at least one image processing setting of the image processing module 116 to adjust the image characteristic in accordance with the identified radiation status.

Thus, the identified radiation status may be used to adjust the processing of the image by the image processing module 116 included in the camera assembly 104.

For example, to perform the above-described compensation in the event that the use of the second radiation state results in a higher intensity of visible light in the food preparation chamber 102 than the use of the first radiation state, the aperture size may be reduced and/or the shutter speed may be increased and/or the sensitivity reduced to compensate for the higher intensity of visible light.

Alternatively or additionally, a color balance (e.g., white balance) may be adjusted to compensate for the different spectral composition of visible light resulting from the second state of radiation being employed after the first state of radiation is employed.

More generally, one or more of the processor 112, the image processing module 116, and/or the control unit 110 may be implemented in a variety of ways using software and/or hardware to perform the various functions required. The processor 112, image processing module 116, and/or control unit 110 may, for example, employ one or more microprocessors programmed with software (e.g., microcode) to perform the desired functions. Examples of processor components that may be employed in various embodiments of the present disclosure include, but are not limited to, conventional microprocessors, application Specific Integrated Circuits (ASICs), and Field Programmable Gate Arrays (FPGAs).

In various embodiments, the one or more processors 112, image processing modules 116, and/or control units 110 may be associated with one or more storage media (e.g., volatile and non-volatile computer memory, such as RAM, PROM, EPROM and EEPROM). The storage medium may be encoded with one or more programs that, when executed on one or more processors and/or controllers, perform the desired functions. The various storage media may be fixed in a processor or controller or may be removable such that one or more programs stored thereon may be loaded into one or more processors 112, image processing modules 116, and/or control units 110.

While image processing may be implemented on the image processing module 116 included in the camera module 104, this is not intended to be limiting, and in other examples, the image processing module 116 may alternatively or additionally be included in a cloud-based server and/or a separate user device (such as a smart phone or tablet).

In at least some embodiments, the food preparation assembly 100 is a household food preparation assembly 100, such as a household food preparation assembly comprising a household kitchen appliance.

Examples of such home kitchen appliances include air fryers, ovens, steamers, autoclaves or rice cookers.

In some embodiments, the camera assembly 104 is integral with such a home kitchen appliance.

In an alternative embodiment, the camera assembly 104 is a removable or detachable component of the food preparation assembly 100 (e.g., a home food preparation assembly). A window (not visible) may be provided into the food preparation chamber 102 and the camera assembly 104 may be arranged to obtain an image of the interior of the food preparation chamber 102 via the window in order to assemble the food preparation assembly 100, e.g. the home food preparation assembly 100.

For example, the camera assembly 104 and optionally at least one of the one or more processors 112 may be included in a separate user device (such as a smart phone or tablet). Alternatively or additionally, the processor 112 may be included in a cloud-based server. The food preparation assembly 100 (e.g., the home food preparation assembly 100) may be assembled by arranging the camera assembly 104 to obtain an image of the interior of the food preparation chamber 102 via a window.

In some embodiments, the food preparation chamber 102 and the radiation assemblies 106A, 106B, 108A, 108B are included in a kitchen appliance (e.g., a home kitchen appliance), and the processor 112 included in the user device and/or the cloud-based server communicates (e.g., wirelessly communicates) with the kitchen appliance (e.g., with the control unit 110 described above) to enable the processor 112 to identify the radiation status and control the camera assembly 104 to adjust the image characteristics of one or more images.

In a non-limiting example, the food preparation assembly 100 is a kitchen appliance, such as a cooking appliance, wherein the camera assembly 104, the radiation assembly 106A, 106B, 108A, 108B (e.g., including the heating elements 108A, 108B and/or the lighting system/lights 106A, 106B), the processor 112, and the control unit 110 are integrated components, as shown in fig. 1. In such examples, the kitchen appliance may further include a data processing unit (e.g., including the image processing module 116 and the data analysis module described above) and a storage unit for storing data.

Fig. 3 provides a flow chart of a method 200 of operating a camera assembly according to an example. The camera assembly may be adapted to obtain one or more images of an interior of a food preparation chamber of the food preparation assembly, the food preparation assembly further comprising a radiation assembly controllable to adopt a radiation state in which the radiation assembly emits radiation comprising visible light inside the food preparation chamber. The food preparation assembly may be, for example, the food preparation assembly 100 according to any of the embodiments described above.

The method 200 includes identifying 202 a radiation status and controlling 204 a camera component to adjust image characteristics of one or more images using a predetermined relationship for adjusting image characteristics according to the identified radiation status.

The method 200 may also include operating 206 a camera assembly to obtain one or more images of the interior of the food preparation chamber.

As previously described, image characteristics may be adjusted based on the radiation conditions identified in the acquired image or images.

Although operation 206 shown in fig. 3 occurs after controlling 204 the camera component to adjust the image characteristics of one or more images using a predetermined relationship, the operations may be performed in any suitable order.

For example, the camera assembly may be operated to acquire an image, and the image processing settings may be subsequently adjusted such that at least a portion of 204 may occur after 206.

In some embodiments, the radiation assembly is controllable to select a radiation state from a plurality of radiation states, and identifying 202 includes identifying the selected radiation state, wherein the predetermined relationship allows for adjustment of the image characteristic in accordance with the identified selected radiation state.

In embodiments where the radiating assembly is controllable to select a radiating state from a plurality of radiating states, the plurality of radiating states may include: a first radiation state; and a second radiation state, wherein the first radiation state causes a first optical condition in the food preparation chamber and the second radiation state causes a second optical condition in the food preparation chamber that is different from the first optical condition. In such embodiments, the predetermined relationship may be determined such that the image characteristics are adjusted to compensate for the difference between the first optical condition and the second optical condition, as previously described.

Fig. 4 graphically illustrates adjusting shutter speed and white balance according to the state of the radiation assembly as cooking proceeds. In this non-limiting example, the two lamps 106A, 106B continuously illuminate the interior of the food preparation chamber 102 throughout the cooking process, but the heating element 108A and the heating element 108B transition between an on-radiating state and an off-radiating state simultaneously in order to control the temperature within the food preparation chamber 102 during the cooking process.

When a first radiant state ("state 1") is employed by two on heating elements 108A, 108B, the illumination intensity in the food preparation chamber 102 may be higher and the color temperature may be lower than when a second radiant state ("state 2") is employed by two off heating elements 108A, 108B. Thus, the shutter speed may be increased, in which case the exposure time is reduced from 1/100s to 1/150s, and the white balance may be adjusted from 5500K to 7500K in response to the radiation state changing from the second radiation state ("state 2") to the first radiation state ("state 1"). Adjustment of these settings may help compensate for changes in illumination intensity and color temperature caused by changes in radiation conditions.

These first and second radiation states are shown in table 1. The optical conditions in the food preparation chamber 102, in this case the illumination intensity and the color temperature, resulting from each of the radiation states are measured for each of the first radiation state and the second radiation state prior to the cooking process. Appropriate image capture settings and/or image processing settings are then determined, as also shown in table 1.

TABLE 1

During the cooking process, as also shown in fig. 4, the radiation status may be identified from the power consumption of the radiation assemblies 106A, 106B, 108A, 108B and/or obtained from the control unit 110, and then the image capturing settings and/or the image processing settings are selected according to table 1.

A computer program comprising computer program code may be configured to cause one or more processors 112 included in the food preparation assembly 100 to implement the method 200 according to any of the embodiments described herein when the computer program is run on the one or more processors, the food preparation assembly further comprising a food preparation chamber 102, a camera assembly 104 for obtaining one or more images of the interior of the food preparation chamber, and radiation assemblies 106A, 106B, 108A, 108B controllable to assume a radiation state in which the radiation assemblies emit radiation comprising visible light inside the food preparation chamber 102.

The one or more processors 112 may be included in a kitchen appliance, such as a kitchen appliance including the food preparation chamber 102 and the radiation assemblies 106A, 106B, 108A, 108B, which kitchen appliance is included in the food preparation assembly. Alternatively or additionally, the one or more processors 112 may be included in a user device (e.g., a smart phone or tablet) separate from such kitchen appliance and/or in a cloud-based server.

FIG. 5 provides a flow chart of a method 300 of determining a relationship for adjusting image characteristics of one or more images of a food preparation chamber interior of a food preparation assembly including a radiation assembly; the one or more images are obtained using a camera assembly.

The food preparation assembly may be, for example, the food preparation assembly 100 according to any of the embodiments described above.

The method 300 comprises the following steps: identifying 302 at least one radiation state of the radiation assembly in which radiation comprising visible light is emitted inside the food preparation chamber; and measuring 304 optical conditions for each of the at least one radiation state. For example, the optical condition may be measured inside the food preparation chamber 102, for example, using one or more suitable optical sensors.

The method 300 further comprises determining 306 the relation for adjusting the image characteristic according to the identified at least one radiation state using the measured optical condition for each of the identified at least one radiation state.

The method 300 may be used to determine the predetermined relationship described above for adjusting image characteristics based on the identified radiation status.

Fig. 6 provides a flowchart illustrating how the method 300 shown in fig. 5 may be applied to an exemplary method 200 of operating a camera assembly. For example, fig. 6 may be considered to be a workflow illustrating the development of an embodiment of the present disclosure.

Method 300 may be considered to be included in a product development stage and method 200 may be considered to be included in a cooking stage (e.g., during use) of food preparation assembly 100.

At least one radiation state of the radiation assembly is identified at 302, in which radiation comprising visible light is emitted inside the food preparation chamber. The identification 302 may include identifying all potential combinations of components that can form part of the radiation assembly (in other words, the heating element and/or the illumination system/lamp), as well as possible radiation conditions of the radiation assembly that may be selected by controlling these components.

In 304, an optical condition is measured for each of the possible radiation states identified in step 302. As previously described, one or more suitable optical sensors may be used, for example, to measure optical conditions inside the food preparation chamber 102.

The measured optical conditions for each of the identified at least one radiation state are then used to determine 306 a relationship for adjusting the image characteristic in accordance with the identified at least one radiation state.

Turning to the method 200 included in the cooking phase in the example shown in fig. 6, identifying 202 the radiation status includes obtaining 202A individual radiation status of all components (in other words, heating elements and/or lighting systems/lamps) that can form part of the radiation assembly. This may be achieved by obtaining a separate status from the control unit described above and/or by sensing the radiation provided by the component. The radiation status of the radiation assembly, in other words, the overall or "combined" status, may then be identified in 202B.

The camera assembly is then controlled 204 to adjust image characteristics of the one or more images using the predetermined relationship determined at 306.

In summary, the present disclosure proposes controlling a camera assembly to adjust image characteristics, such as exposure and/or white balance, for image acquisition and processing to obtain images/video that can reliably reflect the appearance of food products.

Variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality.

The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

If the term "adapted" is used in the claims or specification, it should be noted that the term "adapted" is intended to be equivalent to the term "configured to".

Any reference signs in the claims shall not be construed as limiting the scope.

Claims

1. A food preparation assembly (100), the food preparation assembly comprising:

A food preparation chamber (102);

A camera assembly (104) for obtaining one or more images of an interior of the food preparation chamber, the camera assembly being controllable to adjust image characteristics of the one or more images;

A radiation assembly (106 a,106b,108a,108 b) controllable to adopt a radiation state in which the radiation assembly emits radiation comprising visible light inside the food preparation chamber, the radiation assembly being controllable to select the radiation state from a plurality of radiation states comprising a first radiation state and a second radiation state, the first radiation state causing a first optical condition in the food preparation chamber and the second radiation state causing a second optical condition different from the first optical condition in the food preparation chamber; and

One or more processors (112) configured to:

identifying a selected radiation state; and

Controlling the camera assembly to adjust the image characteristic using a predetermined relationship for adjusting the image characteristic in accordance with the identified selected radiation state, wherein the predetermined relationship is determined such that the image characteristic is adjusted to compensate for the difference between the first optical condition and the second optical condition.

2. The food preparation assembly (100) of claim 1, wherein the first and second radiation states are employed in a control sequence that controls the radiation states of the radiation assemblies (106 a,106b,108a,108 b) over time, the one or more processors (112) being configured to control the camera assembly to adjust the image characteristics to conform to the changes in the radiation states using the predetermined relationship.

3. The food preparation assembly (100) of claim 2, wherein the camera assembly (104) is configured to obtain a sequence of images of the interior of the food preparation chamber (102) during the change in radiation state.

4. A food preparation assembly (100) according to any of claims 1-3, wherein the radiation assembly (106 a,106b,108a,108 b) comprises at least one lamp (106 a,106 b) for illuminating the interior of the food preparation chamber (102), and wherein the radiation state comprises an illumination state of the at least one lamp.

5. The food preparation assembly (100) according to any one of claims 1 to 4, wherein the radiation assembly (106 a,106b,108a,108 b) comprises at least one heating element (108 a,108 b) arranged to radiate inside the food preparation chamber (102), and wherein the radiated state comprises a heated state of the at least one heating element.

6. The food preparation assembly (100) of any one of claims 1 to 5, comprising a control unit (110) having a control configuration arranged to control the radiation assembly (106 a,106b,108a,108 b) to adopt the radiation state, and wherein the one or more processors (112) are configured to identify the radiation state based on the control configuration.

7. The food preparation assembly (100) of any one of claims 1 to 6, comprising a sensor for sensing the radiation of the radiation assembly (106 a,106b,108a,108 b), and wherein the one or more processors (112) are configured to identify the radiation status based on the sensed radiation.

8. The food preparation assembly (100) of any one of claims 1 to 7, wherein the camera assembly (104) comprises an image capture module (114), and wherein the one or more processors (112) are configured to control at least one image capture setting of the image capture module to adjust the image characteristic in accordance with the identified radiation status.

9. The food preparation assembly (100) of claim 8, wherein the at least one image capture setting comprises an aperture size, a shutter speed, and/or a sensitivity setting.

10. The food preparation assembly (100) of any one of claims 1 to 9, comprising an image processing module (116), and wherein the one or more processors (112) are configured to control at least one image processing setting of the image processing module to adjust the image characteristic in accordance with the identified radiation status; optionally, wherein the image processing module is included in at least one of the camera module (104), a user device, and a cloud-based server.

11. The food preparation assembly (100) of claim 10, wherein the at least one treatment setting comprises a color balance setting; optionally, wherein the color balance setting is a white balance setting.

12. A method (200) of controlling a camera assembly adapted to obtain one or more images of an interior of a food preparation chamber of a food preparation assembly, the food preparation assembly further comprising a radiation assembly controllable to adopt a radiation state in which the radiation assembly emits radiation comprising visible light inside the food preparation chamber, the radiation assembly controllable to select the radiation state from a plurality of radiation states comprising a first radiation state and a second radiation state, the first radiation state causing a first optical condition in the food preparation chamber and the second radiation state causing a second optical condition different from the first optical condition in the food preparation chamber, the method comprising:

Identifying (202) the selected radiation state; and

-Controlling (204) the camera component to adjust image characteristics of the one or more images using a predetermined relationship for adjusting the image characteristics according to the identified selected radiation status, wherein the predetermined relationship is determined such that the image characteristics are adjusted to compensate for differences between the first optical condition and the second optical condition.

13. The method (200) of claim 12, wherein the first and second radiation states are employed in a control sequence that controls the radiation state of the radiation assembly over time, the controlling (204) the camera assembly adjusting the image characteristic using the predetermined relationship to conform to the change in the radiation state.

14. The method (200) of claim 12 or claim 13, the method comprising operating (206) the camera assembly to obtain one or more images of an interior of the food preparation chamber.

15. The method (200) of claim 14 when dependent on claim 13, wherein the operation (206) comprises obtaining a sequence of images of the interior of the food preparation chamber during the change in radiation state.

16. A computer program comprising computer program code configured to cause one or more processors comprised in a food preparation assembly to carry out the method according to any one of claims 12 to 15, the food preparation assembly further comprising a food preparation chamber, a camera assembly for obtaining one or more images of the interior of the food preparation chamber, and a radiation assembly controllable to adopt a radiation state in which the radiation assembly emits radiation comprising visible light inside the food preparation chamber.

17. A method (300) of determining a relationship for adjusting image characteristics of one or more images of a food preparation chamber interior of a food preparation assembly including a radiation assembly, the one or more images obtained using a camera assembly, the method comprising:

Identifying (302) a first radiation state and a second radiation state of the radiation assembly, in each of which radiation state radiation comprising visible light is emitted inside the food preparation chamber;

Measuring (304) an optical condition for each of the at least one radiation state, such that a first optical condition is measured for the first radiation state and a second optical condition is measured for the second radiation state; and

The relationship for adjusting the image characteristic according to the identified radiation state is determined (306) using the first and second optical conditions measured for each of the first and second radiation states, respectively, such that the image characteristic is adjusted to compensate for differences between the first and second optical conditions.