CN115066880A - Method for generating captured image and electronic device - Google Patents

Abstract

A method, comprising: capturing a first initial image within a first dynamic range, wherein the first initial image is captured at a different imaging setting; generating an intermediate image of a second dynamic range from the first initial image, wherein the second dynamic range is higher than the first dynamic range; detecting a light source region according to a light detection image to generate a light source map, wherein the light detection image is at least one image in the first initial image and is suitable for detecting the light source region; a captured image is generated from the intermediate image and the light source map, wherein the position of the light source region of the captured image is determined from the light source map, and the color of the light source region of the captured image is determined from the intermediate image.

Description

Method and electronic device for generating a captured image

technical field

The present application relates to a method of generating a captured image in an electronic device including a camera assembly and such electronic device.

Background technique

Electronic devices such as smartphones and tablet terminals are widely used in daily life. Today, many electronic devices are equipped with camera assemblies to capture images. Some electronic devices are portable and easy to carry, allowing their users to easily capture images anytime, anywhere using the camera assembly of the electronic device.

However, if the user of the electronic device captures the image at night, the captured image includes one or more regions of the light source that are overexposed or haloed. After an image is captured, image correction processing may be performed on overexposed light source regions and halo regions in the captured image.

However, even after image correction processing has been performed on the captured image, the quality of the light source area on the captured image is sometimes insufficient for the user, for example because its color is not suitable for the light source area or the bokeh effect introduced to the light source area on the captured image (Bokeh effect) is too strong. Therefore, there is a need for a technique to improve the quality of the light source area on the captured image.

SUMMARY OF THE INVENTION

The present application aims to solve at least one of the above-mentioned technical problems. Therefore, the present application needs to provide a method and electronic device for generating a captured image.

According to the present application, a method of generating a captured image in an electronic device including a camera assembly may include:

capturing, by the camera assembly, a plurality of first initial images in a first dynamic range, wherein the plurality of first initial images are captured under different imaging settings;

generating an intermediate image in a second dynamic range from the plurality of first initial images, wherein the second dynamic range is higher than the first dynamic range;

Detecting the light source area according to the light detection image to generate a light source map, wherein the light detection image is at least one image of the plurality of first initial images and is suitable for detecting the light source area;

The captured image is generated from the intermediate image and the light source map, wherein the position of the light source area of the captured image is determined from the light source map, and the color of the light source area of the captured image is determined from the intermediate image.

In some embodiments, in the method, capturing the plurality of first initial images may include capturing the plurality of first initial images at different exposure values.

In some embodiments, in the method, the light detection image may be one of the plurality of first initial images captured at the lowest exposure value.

In some embodiments, in the method, the minimum exposure value may be low enough to detect light from areas other than the light source area in the first initial image.

In some embodiments, in the method, the minimum exposure value may be a negative value.

In some embodiments, in the method, can also include:

capturing a second initial image via the camera assembly; and

The depth map is calculated from the plurality of first and second initial images.

In some embodiments, in the method, generating the captured image further comprises: performing a bokeh rendering process to introduce a bokeh effect into the captured image based on the depth map.

In some embodiments, in the method, the plurality of first initial images may be captured by a first main camera of the camera assembly, and the second initial images may be captured by a second main camera of the camera assembly.

In some embodiments, in the method, the generating the captured image may further comprise performing a segmentation process to introduce a bokeh effect into the captured image based on the intermediate image.

In some embodiments, in the method, the plurality of first initial images may be captured by a first primary camera in the back of the electronic device or may be captured by a secondary camera in the front of a camera assembly of the electronic device.

According to the present application, an electronic device may include:

a camera assembly capable of capturing images within a first dynamic range; and

Image processor, which is used to:

- capturing, by the camera assembly, a plurality of first initial images in a first dynamic range, wherein the plurality of first initial images are captured at different imaging settings;

- generating an intermediate image in a second dynamic range from the plurality of first initial images, wherein the second dynamic range is higher than the first dynamic range;

- detecting light source regions from a light detection image to generate a light source map, wherein the light detection image is at least one of a plurality of first initial images and is suitable for detecting light source regions;

- generating a captured image from the intermediate image and a light source map, wherein the position of the light source area of the captured image is determined from the light source map, and the color of the light source area of the captured image is determined from the intermediate image.

In some embodiments, in the electronic device, when capturing a plurality of first initial images, the plurality of first initial images may be captured at different exposure values.

In some embodiments, in the electronic device, the light detection image may be an image captured at the lowest exposure value of the plurality of first initial images.

In some embodiments, in the electronic device, the minimum exposure value may be low enough to detect light from areas other than the light source area in the first initial image.

In some embodiments, in the electronic device, the minimum exposure value may be a negative value.

In some embodiments, in the electronic device, the image processor can also be used to:

capturing a second initial image via the camera assembly; and

The depth map is calculated from the plurality of first and second initial images.

In some embodiments, in the electronic device, when the captured image is generated, a bokeh rendering process may also be performed to introduce a bokeh effect into the captured image according to the depth map.

In some embodiments, in the electronic device, the plurality of first initial images may be captured by a first main camera of the camera assembly, and the second initial images may be captured by a second main camera of the camera assembly.

In some embodiments, in the electronic device, when the captured image is generated, a bokeh effect may also be introduced into the captured image according to the intermediate image through a segmentation process.

In some embodiments, in the electronic device, the plurality of first initial images may be captured by a first primary camera in the back of the electronic device or may be captured by a secondary camera in the front of a camera assembly of the electronic device.

Description of drawings

These and/or other aspects and advantages of embodiments of the present application will become apparent and better understood from the following description with reference to the accompanying drawings, wherein:

1 shows a rear view of an electronic device according to a first embodiment of the present application;

Fig. 2 shows the front view of the electronic device according to the first embodiment of the present application;

3 shows a block diagram of an electronic device according to the first embodiment of the present application;

FIG. 4 shows a schematic flowchart of image capture processing performed by the electronic device according to the first embodiment of the present application;

FIG. 5 shows a rear view of an electronic device according to a second embodiment of the present application; and

FIG. 6 shows a schematic flowchart of an image capture process performed by an electronic device according to a second embodiment of the present application.

Detailed ways

Embodiments of the present application will be described in detail below, examples of which are illustrated in the accompanying drawings. Throughout the specification, the same or similar elements and elements having the same or similar functions are denoted by the same reference numerals. The embodiments described herein in conjunction with the accompanying drawings are illustrative and intended to explain the present application and should not be construed as limiting the present application.

[First Embodiment]

FIG. 1 shows a rear view of an electronic device according to a first embodiment of the present application. FIG. 2 shows a front view of an electronic device according to the first embodiment of the present application.

As shown in FIGS. 1 and 2 , the electronic device 10 may include a display 20 and a camera assembly 30 . In this embodiment, the camera assembly 30 includes a first main camera 32 , a second main camera 34 and a sub-camera 36 . The first main camera 32 and the second main camera 34 can capture images on the back of the electronic device 10 , while the secondary camera 36 can capture images on the front of the electronic device 10 . Therefore, the first main camera 32 and the second main camera 34 are referred to as outer cameras, and the secondary camera 36 is referred to as an inner camera.

The first main camera 32 and the second main camera 34 may have the same performance and/or characteristics, or the first main camera 32 and the second main camera 34 may have different performance and/or characteristics. For example, where the first main camera 32 and the second main camera 34 have different performance and/or characteristics from each other, the first main camera 32 may be equipped with a full color image sensor, while the second main camera 34 may be equipped with a black and white image sensor Image Sensor. Furthermore, the first main camera 32 may be a camera suitable for capturing still images, while the second main camera 34 may be a camera suitable for capturing moving images. Also, the first main camera 32 may be a camera equipped with a wide-angle lens, and the second main camera 34 may be a camera equipped with a telephoto lens.

In this embodiment, the performance of the sub-camera 36 is lower than that of the first main camera 32 and the second main camera 34 . However, the performance of the secondary camera 36 may also be the same as the performance of the first main camera 32 and the second main camera 34 .

Although the electronic device 10 according to the present embodiment has three cameras, the electronic device may have more than three cameras. For example, the electronic device 10 may have 3, 4, 5, etc. main cameras.

FIG. 3 shows a block diagram of an electronic device according to the first embodiment of the present application. As shown in FIG. 3 , in addition to the display 20 and the camera assembly 30 , the electronic device 10 may further include a main processor 40 , an image signal processor 42 , a memory 44 , a power supply circuit 46 and a communication circuit 48 . The display 20 , the camera assembly 30 , the main processor 40 , the image signal processor 42 , the memory 44 , the power supply circuit 46 and the communication circuit 48 are connected together by a bus 50 .

Main processor 40 executes one or more programs stored in memory 44 . The main processor 40 implements various applications and data processing of the electronic device 10 by executing programs. Main processor 40 may be one or more computer processors. The main processor 40 is not limited to one CPU core, and may have a plurality of CPU cores. The main processor 40 may be the main CPU of the electronic device 10 , an image processing unit (IPU) or a DSP provided with the camera assembly 30 .

Image signal processor 42 controls camera assembly 30 and processes images captured by camera assembly 30 . For example, the image signal processor 42 may perform demosaic processing, noise reduction processing, automatic exposure processing, automatic focus processing, automatic white balance processing, high dynamic range processing, etc. on the image captured by the camera assembly 30 .

In this embodiment, the main processor 40 and the image signal processor 42 cooperate with each other to acquire the image captured by the camera assembly 30 . That is, the main processor 40 and the image signal processor 42 are used to capture images through the camera assembly 30 and perform various image processing on the captured images.

The memory 44 stores programs to be executed by the main processor 40 and various data. For example, the data of the captured image is stored in the memory 44 .

Memory 44 may be high-speed RAM memory or may be non-volatile memory such as flash memory, disk memory, or the like.

The power circuit 46 may have a battery (not shown) such as a lithium-ion rechargeable battery and a battery management unit (BMU) for managing the battery.

Communication circuitry 48 is used to receive and transmit data to communicate with the Internet or other devices via wireless communication. Wireless communication can use any communication standard or protocol, including but not limited to: Global System for Mobile Communications (GSM), Code Division Multiple Access (CDMA), Long Term Evolution (LTE), Long Term Evolution Enhanced (LTE-Advanced), Fifth Generation Mobile Communication system (5G).

Communication circuitry 48 may include an antenna and radio frequency (RF) circuitry.

FIG. 4 shows a schematic flowchart of image capture processing performed by the electronic device according to the present embodiment. In the present embodiment, the image capturing process is performed by the main processor 40 in cooperation with the image signal processor 42 . Therefore, the main processor 40 and the image signal processor 42 constitute the image processor in this embodiment.

As shown in FIG. 4 , the electronic device 10 captures a plurality of first initial images within a first dynamic range through the first main camera 32 of the camera assembly 30 (step S10 ), and captures the first image through the second main camera 34 of the camera assembly 30 . A second initial image within a dynamic range (step S32).

In this embodiment, for example, the camera assembly 30 may have settings for auto focus, auto exposure, and auto white balance. In other words, the camera assemblies 30 may have settings of ISO sensitivity, settings of exposure time, settings of exposure value, and the like, respectively.

The imaging settings of the camera assembly 30 may be automatically set by the electronic device 10 . In other words, the user may select automatic settings for the imaging settings of the camera assembly 30 . Alternatively, the imaging settings of the camera assembly 30 may be preset by the user. In other words, the user can preset each item of imaging settings of the camera assembly 30 .

In addition, in this embodiment, when the electronic device 10 captures a plurality of first initial images in step S10, the electronic device 10 changes the exposure value of each first initial image to generate a second dynamic range in a subsequent process the intermediate image. In this embodiment, the first dynamic range corresponds to a low dynamic range, and the second dynamic range corresponds to a high dynamic range.

For example, in step S10, the electronic device 10 captures three first initial images, wherein the exposure values are set to -1, 0, and +1, respectively. That is, it is assumed that the electronic device 10 has set the exposure value of the camera assembly 30 to 0. To capture the other two first initial images, the exposure value is increased by 1 and decreased by 1, respectively.

However, the number of first initial images is optional. For example, the number of the first initial images captured in step S10 may be two, four, five, and so on.

When capturing the second initial image in step S12, the electronic device 10 captures one image of the second initial image to calculate the depth map in subsequent processing. In this embodiment, although the plurality of first initial images and the plurality of second initial images are captured within the first dynamic range, the dynamic range of the first initial image may be different from that of the second initial dynamic range . In other words, the first dynamic range is the dynamic range of the first main camera 32 and the second main camera 34 .

In this embodiment, the first initial image and the second initial image with the exposure value of 0 are captured simultaneously, because the exposure value at 0 is the imaging setting, which is set by the electronic device 10 . However, the capture time of the first initial image with the exposure value of 0 and the capture time of the second initial image may be different.

Next, the electronic device 10 executes high dynamic range generation processing (step S14). More specifically, in the high dynamic range generation process, the electronic device 10 generates an intermediate image in the second dynamic range from the plurality of first initial images.

In this embodiment, the electronic device 10 generates the intermediate images from three first initial images captured with exposure values of -1, 0, and 1, respectively. The high dynamic range intermediate image may be generated by combining and compositing multiple low dynamic range first initial images.

Next, as shown in FIG. 4 , the electronic device 10 calculates a depth map from the intermediate image generated in step S14 and the second initial image captured in step S12 (step S16 ).

More specifically, the position of the first main camera 32 is different from the position of the second main camera 34 . Therefore, the viewpoint of the intermediate image generated from the first initial image captured by the first main camera 32 is different from the viewpoint of the second initial image captured by the second main camera 34 . Using the disparity of the intermediate image and the second initial image, the electronic device 10 may generate a depth map indicating the distance between the electronic device 10 and the surface of the object in the intermediate image and the second main image.

For example, the depth map may be calculated by main processor 40 and/or image signal processor 42 . Additionally, the depth map may be computed inside the camera module of the camera assembly 30 . However, the depth map calculation circuit for calculating the depth map may be placed in the electronic device 10 .

Next, as shown in FIG. 4 , the electronic device 10 detects the light source area according to the light detection image, and generates a light source map by detecting the light source in the light detection image to indicate the position of the light source area in the intermediate image (step S18 ).

The light source map may be an image that indicates the center of the light source or may be a collection of one or more coordinates of the center of the light source on the captured image. The light source map may be temporarily stored in memory 44 .

The light detection image is at least one of the plurality of first initial images and is suitable for detecting light source regions. For example, in this case, one of the first initial images captured with an exposure value of -1 is a light detection image because the images captured with an exposure value of -1 are the first plurality of first images captured in step S10 The lowest exposure value in the initial image. In other words, an exposure value of -1 is sufficiently low that light from areas other than the light source area cannot be detected in the first initial image.

Generally, in order to detect the light source area in the light detection image, the exposure value (EV) is preferably a negative value. That is, the light detection image should be one of the first initial images captured at negative exposure values. Here, the exposure value can be composed of shutter speed, ISO sensitivity, and aperture.

In this embodiment, in order to generate the light source map, the electronic device 10 compares the brightness of a certain area in the light detection image with a threshold value, and if the brightness of the area is greater than the threshold value, the area is regarded as a light source area.

More specifically, the electronic device 10 compares the brightness of each region in the light detection image with the threshold. In other words, the electronic device 10 compares the brightness of each pixel in the light detection image with a threshold value to detect the light source area in the light detection image. The threshold may be stored in the memory 44 , set in a program executed by the main processor 40 , or set in the image signal processor 42 .

For example, in the case where each pixel in the light detection image includes a luminance composed of 256 grayscales, if the luminance value of the pixel is greater than 128, the electronic device 10 may determine that the pixel is in the light source area.

Incidentally, the threshold is not necessarily set to a specific value, but can also be set to a percentage. For example, with a maximum brightness of 100% and a minimum brightness of 0%, the threshold can be set to 50% of the brightness.

Thereafter, the electronic device 10 detects the center of the area whose brightness is greater than the threshold value and determines the center as the center of the light source to generate a light source map.

In addition, in this embodiment, although the electronic device 10 detects the light source region in the light detection image after calculating the depth map, the electronic device 10 may detect the light source region in the light detection image before calculating the depth map. In this case, step S18 is performed before step S16.

For example, the light source area may be detected by the main processor 40 and/or the image signal processor 42 from the light detection image. In addition, the light source area may be detected according to the light detection image captured by the camera assembly 30 .

Next, as shown in FIG. 4 , the electronic device 10 performs image generation processing to generate a captured image (step S20 ). In the present embodiment, the image generation process is performed based on at least the intermediate image generated in step S14, the light source map generated in step S18, and the depth map calculated in step S16. Of course, in the image generation process of step S20, other images and/or any information may be used to generate the captured image.

In the present embodiment, the image generation process includes at least a light source area adjustment process that appropriately renders the light source area onto the captured image. In the light source area adjustment process, the position of the light source area is determined according to the light source map, and the color of the light source area is determined according to the intermediate image.

Furthermore, the image generation process includes at least a depth map-based bokeh rendering process. That is, after the intermediate image is generated from the plurality of first initial images, a bokeh rendering process is applied to the intermediate image to defocus the background of the intermediate image according to the depth map. In other words, the background of the intermediate image is defocused to become blurred by the bokeh rendering process.

In general, the colors of the high dynamic range intermediate images are more natural and accurate than the colors of each of the first initial images and each of the second initial images. On the other hand, if the bokeh rendering process is applied to a high dynamic range intermediate image, it will also introduce a bokeh effect into the light source area, making the light source area too blurry. Therefore, in this embodiment, the position of the light source area is adjusted according to the light source map generated in step S18, and the color of the light source area is adjusted according to the color of the intermediate image.

When the image generation process in step S20 is completed, a captured image that the user wants to capture is generated. Of course, the image generation process for generating the captured image may include various processes other than the bokeh rendering process and the light source area adjustment process applied to the intermediate image. For example, the image generation processing may include demosaic processing, noise reduction processing, auto exposure processing, auto focus processing, automatic white balance processing, high dynamic range processing, etc. to generate the captured image.

For example, image generation processing may be performed by main processor 40 and/or image signal processor 42 . However, a bokeh rendering circuit for performing bokeh rendering processing may be placed in the electronic device 10 . Similarly, a light source area adjustment circuit for performing light source area adjustment processing may be placed in the electronic device 10 .

Next, as shown in FIG. 4, the electronic device 10 outputs the captured image (step S22). For example, the electronic device 10 may display the captured image on the display 20 . Additionally, electronic device 10 may store captured images in memory 44 .

After the electronic device 10 outputs the captured image in step S22, the image capture process of this embodiment is completed.

As described above, according to the electronic device 10 of the present embodiment, when the electronic device 10 generates the captured image, the position of the light source region on the captured image is determined according to the light source map and the color of the captured image, and is determined according to the high dynamic range intermediate image Capture the light source area on the image. As a result, the area of the light source on the captured image is more aesthetically pleasing and more natural. In this way, the user can be made satisfied with the quality of the captured image of the electronic device 10 according to the present embodiment.

Furthermore, since one of the plurality of first initial images can be used as the light detection image to detect the light source area, there is no need for the camera assembly 30 to capture additional images or additional mechanical elements. Therefore, the user's satisfaction with the captured image can be improved without any additional cost.

Although the electronic device 10 described above changes the exposure value for capturing each of the plurality of first initial images in step S10 in order to generate an intermediate image in the high dynamic range in step S14, the electronic device 10 may change the exposure value for capturing Imaging settings other than the exposure value of each of the plurality of first initial images. For example, the electronic device 10 may change ISO sensitivity, exposure time, etc. to capture each of the plurality of first initial images in step S10.

In any case, the most suitable image for detecting the light source area and generating the light source map in the first initial image may be used as the light detection image in step S18. In other words, since the electronic device 10 captures a plurality of first initial images in order to generate a high dynamic range intermediate image, an image most suitable for detecting the light source area may be selected as the light detection image in step S18.

[Second Embodiment]

Although the electronic device 10 has two main cameras on its back in the first embodiment of the present application, in the second embodiment of the present application, the electronic device 10 has one main camera on its back. Hereinafter, differences from the first embodiment will be explained.

FIG. 5 shows a rear view of the electronic device 10 according to the second embodiment of the present application. In addition, FIG. 5 corresponds to FIG. 1 in the first embodiment. The front view of the electronic device 10 according to the second embodiment is substantially the same as that of FIG. 2 in the first embodiment.

As shown in FIG. 5 , the electronic device 10 according to the present embodiment has a camera assembly 30 that includes a main camera 38 but does not include other main cameras. That is, the camera assembly 30 of the electronic device 10 according to the present embodiment has a main camera 38 on the back of the electronic device 10 to capture images on the back of the electronic device 10 .

As shown in FIG. 2 , in the second embodiment, the electronic device 10 has a sub-camera 36 as in the first embodiment. That is, the camera assembly 30 of the electronic device 10 according to the second embodiment further includes a built-in camera on the front of the electronic device 10 so that it captures images on the front.

FIG. 6 shows a schematic flowchart of an image capture process performed by an electronic device according to a second embodiment of the present application. In the present embodiment, the image capturing process is performed by the main processor 40 in cooperation with the image signal processor 42 . Therefore, the main processor 40 and the image signal processor 42 constitute the image processor in this embodiment.

As shown in FIG. 6 , the electronic device 10 captures a plurality of first initial images within a first dynamic range through the main camera 38 of the camera assembly 30 (step S30 ), wherein the first dynamic range is lower than the second dynamic range. Step S30 in the second embodiment is basically the same as step S10 in the first embodiment, except that the plurality of first initial images are captured by a single main camera 38 .

On the other hand, since the camera assembly 30 of the electronic device 10 of the second embodiment does not have the second main camera 34, the second initial image is not captured. In other words, step S12 of capturing the second initial image in the first embodiment is omitted in the second embodiment.

Next, as shown in FIG. 6 , the electronic device 10 performs a high dynamic range generation process to generate a high dynamic range intermediate image from a plurality of low dynamic range first initial images (step S14 ). Step S14 in the second embodiment is basically the same as the corresponding step in the first embodiment.

Next, as shown in FIG. 6 , the electronic device 10 detects the light source area according to the light detection image to generate a light source map by detecting the light source in the light detection image, thereby indicating the position of the light source area in the intermediate image (step S18 ) ). Step S18 in the second embodiment is basically the same as the corresponding step in the first embodiment.

Next, as shown in FIG. 6 , the electronic device 10 performs image generation processing to generate a captured image (step S32 ). In the present embodiment, the image generation process is performed based on at least the intermediate image generated in step S14 and the light source map generated in step S18. Of course, in the image generation process of step S32, other images and/or any information may be used to generate the captured image.

However, because the camera assembly 30 includes a single main camera 38, no depth map is used in the image generation process according to the present embodiment. That is, the step S16 of generating the depth map in the first embodiment is omitted in the second embodiment.

Therefore, the image generation processing according to the present embodiment includes segmentation processing in place of the bokeh rendering processing. In the segmentation process, the bokeh effect is introduced into the captured image without the need for the depth map calculated in step S16 of the first embodiment. For example, the bokeh effect is introduced into the captured image by analyzing and identifying the human face in the intermediate image, the human silhouette is segmented and the bokeh effect is introduced into the background of the human silhouette in the intermediate image.

Like the first embodiment, the image generation process according to the present embodiment includes the light source area adjustment process. That is, in this embodiment, the position of the light source region is determined according to the light source map, and the color of the light source region is determined according to the intermediate image.

Next, as shown in FIG. 4, the electronic device 10 outputs the captured image (step S22). Step S22 in the second embodiment is basically the same as the corresponding step in the first embodiment.

As described above, according to the electronic device 10 of the present embodiment, when the electronic device 10 generates a captured image, the position of the light source region on the captured image is determined according to the light source map, and the color of the light source on the captured image is determined according to the high dynamic range intermediate image . Therefore, the light source area on the captured image can be more aesthetically pleasing and more natural. In this way, the user can be made satisfied with the quality of the captured image captured by the electronic device 10 according to the present embodiment.

Furthermore, even if the electronic device 10 includes a single main camera 38, the electronic device 10 can perform image generation processing including light source area adjustment processing. Therefore, the present embodiment may be implemented with the camera assembly 30 and the single main camera 38 of the electronic device 10 .

Similarly, this embodiment can be implemented by the sub-camera 36 on the front of the electronic device 10 . In other words, if the sub-camera 36 captures a plurality of first initial images, the image capturing process in FIG. 6 may be performed according to the plurality of first initial images captured by the sub-camera 36 . Therefore, the quality of the captured image captured by the sub-camera 36 can also be improved.

In the description of the embodiments of the present application, it should be understood that such terms as "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front" ", "Rear", "Rear", "Left", "Right", "Vertical", "Horizontal", "Top", "Bottom", "Inner", "Outer", "Clockwise" and "Counterwise" Terms such as "hour hand" should be understood to refer to the direction or position shown in the drawings described or discussed. These relative terms are only used to simplify the description of the present application and do not indicate or imply that a referenced device or element must have a particular orientation, or be constructed or operated in a particular orientation. Therefore, these terms should not be construed as limitations on this application.

Furthermore, terms such as "first" and "second" are used herein for descriptive purposes and are not intended to indicate or imply relative importance or importance, or to imply the number of technical features indicated. Thus, features defined with "first" and "second" may include one or more of that feature. In the description of this application, "plurality" means two or more, unless otherwise specified.

In the description of the embodiments of the present application, unless otherwise stated or limited, terms such as "installation", "connection", and "coupling" are broad, for example, it may be a fixed connection, a detachable connection or an integral connection; it may also be a mechanical connection Or electrical connection; it can also be a direct connection or an indirect connection through an intermediate structure; it can also be an internal communication between two elements, which can be understood by those skilled in the art according to specific situations.

In the embodiments of the present application, unless otherwise stated or limited, the structure in which the first feature is "on" or "under" the second feature may include embodiments in which the first feature is in direct contact with the second feature, and may also include The first and second features are not in direct contact with each other, but are in contact through additional features formed between them. Furthermore, a first feature being "above," "over," or "top" a second feature may include that the first feature is "above," "over," or "top" the second feature straight or slanted, or just Indicates that the height of the first feature is higher than the height of the second feature; while the first feature is "below", "beneath" or "bottom" of the second feature may include that the first feature is directly or obliquely "below" the second feature ”, “below” or “bottom”, or simply mean that the height of the first feature is lower than the height of the second feature.

Various embodiments and examples have been provided in the above description to implement different structures of the present application. To simplify this application, certain elements and arrangements are described above. However, these elements and arrangements are only examples and are not intended to limit the application. Furthermore, reference numerals and/or reference letters may be repeated in different examples of the present application. This repetition is for the purpose of simplicity and clarity and does not refer to a relationship between different embodiments and/or arrangements. Additionally, examples of different processes and materials are provided in this application. However, those skilled in the art will understand that other processes and/or materials may also be used.

References throughout this specification to "one embodiment," "some embodiments," "one exemplary embodiment," "one example," "one specific example," or "some examples" refer to specific aspects described in connection with the embodiments or examples. A feature, structure, material or characteristic is included in at least one embodiment or example of the present application. Thus, appearances of the above phrases in this specification are not necessarily referring to the same embodiment or example of the present application. Furthermore, the particular features, structures, materials or characteristics may be combined in any suitable manner in one or more embodiments or examples.

Any process or method described in a flowchart or otherwise described herein can be understood to include one or more modules, segments or portions of code of executable instructions for implementing the specified logical functions or steps in the process, and the present The scope of the application for the preferred embodiment includes other implementations, those of ordinary skill in the art should understand that the functions of the implementations may be performed in a different order than shown or discussed, including in substantially the same order or in the reverse order.

Logic and/or steps otherwise described herein or shown in flowcharts, such as a specific sequence listing of executable instructions for implementing the logical functions, may be embodied in any computer-readable medium, such as For use by an instruction execution system, apparatus, or device (eg, a computer-based system, a system including a processor, or other system capable of retrieving and executing instructions from an instruction execution system, apparatus, and apparatus), or for an instruction execution system, Combination of devices and equipment. With respect to this specification, a "computer-readable medium" can be any means suitable for containing, storing, communicating, propagating, or transporting the program for use by or in combination with the instruction execution system, apparatus, or apparatus. More specific examples of computer readable media include, but are not limited to: electronic connections with one or more wires (electronic devices), portable computer housings (magnetic devices), random access memory (RAM), read only memory memory (ROM) ), erasable programmable read only memory (EPROM or flash memory), fiber optic equipment and portable compact disc read only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program can be printed, since, for example, the program is obtained electronically, if necessary, optically scanned on the paper or other suitable medium, and then printed in other suitable ways. Edit, decrypt, or process it, and store the program in computer memory.

It should be understood that various parts of the present application may be implemented by hardware, software, firmware or a combination thereof. In the above-described embodiments, various steps or methods may be implemented by software or firmware stored in memory and executed by an appropriate instruction execution system. For example, if implemented by hardware, also in another embodiment, these steps or methods may be implemented by one or a combination of the following techniques known in the art: a logic gate having a logic function for implementing a data signal Discrete logic circuits of circuits, application specific integrated circuits with appropriate combinations of combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.

Those skilled in the art should understand that all or part of the steps in the above-mentioned exemplary method of the present application can be implemented by programming related hardware. The program may be stored in a computer-readable storage medium, and when the program runs on a computer, includes a combination of one or more steps in the method embodiments of the present application.

In addition, each functional unit in the embodiments of the present application may be integrated into one processing module, or may exist independently physically, or two or more units may be integrated into one processing module. The integrated module can be implemented in the form of hardware or in the form of software functional modules. When the integrated module is implemented in the form of a software functional module and sold or used as a stand-alone product, the integrated module may be stored in a computer-readable storage medium.

The above-mentioned storage medium may be a read-only memory, a magnetic disk, a CD, or the like.

Although the embodiments of the present application have been shown and described, it should be understood by those skilled in the art that these embodiments are illustrative and not to be construed as limiting the present application, and that, without departing from the scope of the present application, the Variations, modifications, substitutions and variations are made to this application.

Claims (20)

1. A method of generating a captured image in an electronic device, the electronic device including a camera assembly, the method comprising:

capturing, by the camera assembly, a plurality of first initial images within a first dynamic range, wherein the plurality of first initial images are captured at different imaging settings;

generating an intermediate image of a second dynamic range from the plurality of first initial images, wherein the second dynamic range is higher than the first dynamic range;

detecting a light source region from a light detection image to generate a light source map, wherein the light detection image is at least one image of the plurality of first initial images and is adapted to detect the light source region;

generating the captured image from the intermediate image and the light source map, wherein the position of the light source region of the captured image is determined from the light source map, and the color of the light source region of the captured image is determined from the intermediate image.

2. The method of claim 1, wherein capturing the first plurality of initial images comprises: the plurality of first initial images are captured at different exposure values.

3. The method of claim 2, wherein the light detection image is one of the first plurality of initial images captured at a lowest exposure value.

4. A method according to claim 3, wherein the lowest exposure value is too low to be sufficient to detect light from regions outside the light source region in the first initial image.

5. The method of claim 4, wherein the lowest exposure value is a negative value.

6. The method of any of claims 1-5, further comprising:

capturing a second initial image by the camera assembly; and

a depth map is computed from the plurality of first initial images and the second initial image.

7. The method of claim 6, wherein the generating the captured image further comprises: performing a shot rendering process to introduce a shot effect to the captured image according to the depth map.

8. The method of claim 7, wherein the plurality of first initial images are captured by a first main camera of the camera assembly and the second initial images are captured by a second main camera of the camera assembly.

9. The method according to any one of claims 1-5, wherein the generating the captured image further comprises: a segmentation process is performed to introduce a shot effect to the captured image from the intermediate image.

10. The method of claim 9, wherein the first plurality of initial images are captured by a first primary camera in a back side of the electronic device or by a secondary camera in a front side of the camera assembly of the electronic device.

11. An electronic device, comprising:

a camera assembly capable of capturing images in a first dynamic range; and

an image processor configured to:

capturing, by the camera assembly, a plurality of first initial images within the first dynamic range, wherein the plurality of first initial images are captured at different imaging settings;

generating an intermediate image of a second dynamic range from the plurality of first initial images, wherein the second dynamic range is higher than the first dynamic range;

detecting a light source region from a light detection image to generate a light source map, wherein the light detection image is at least one image of the plurality of first initial images and is adapted to detect the light source region;

generating the captured image from the intermediate image and the light source map, wherein the position of the light source region of the captured image is determined from the light source map, and the color of the light source region of the captured image is determined from the intermediate image.

12. The electronic device of claim 11, wherein when capturing the plurality of first initial images, the plurality of first initial images are captured at different exposure values.

13. The electronic device of claim 12, wherein the light detection image is one of the plurality of first initial images captured at a lowest exposure value.

14. The electronic device of claim 13, wherein the lowest exposure value is low enough to be insufficient to detect light from regions other than the light source region in the first initial image.

15. The electronic device of claim 14, wherein the lowest exposure value is a negative value.

16. The electronic device of any one of claims 11-15, the image processor further configured to:

capturing a second initial image by the camera assembly; and

a depth map is computed from the plurality of first initial images and the second initial image.

17. The electronic device of claim 16, wherein when generating the captured image, further performs a shot rendering process to introduce a shot effect to the captured image according to the depth map.

18. The electronic device of claim 17, wherein the plurality of first initial images are captured by a first primary camera of the camera assembly and the second initial images are captured by a second primary camera of the camera assembly.

19. The electronic device according to any of claims 11-15, wherein a bokeh effect is also introduced to the captured image from the intermediate image by a segmentation process when generating the captured image.

20. The electronic device of claim 19, wherein the plurality of first initial images are captured by a first primary camera in a back side of the electronic device or by a secondary camera in a front side of the camera assembly of the electronic device.

Images