CN115426458A - Light source detection method and related equipment thereof - Google Patents

Abstract

The application provides a light source detection method and related equipment thereof, and relates to the field of image processing, wherein the light source detection method comprises the following steps: starting a camera application program of the electronic equipment; acquiring a first image by using a first camera; performing virtual focus processing on the second camera and/or reducing exposure value processing; acquiring a reference image by using the processed second camera; based on the reference image, a target light source included in the first image is determined. This application is through carrying out virtual burnt and/or reduce the exposure value to the second camera and handling, can assist the accurate detection light source of first camera with simple swift mode.

Description

Light source detection method and related equipment

technical field

The present application relates to the field of image processing, in particular to a light source detection method and related equipment.

Background technique

With the popularization of electronic devices with a shooting function in daily life, it has become a daily behavior of people to use electronic devices to take pictures.

When the scene to be photographed includes a light source, because the brightness generated by the light source is relatively high, which affects the light and shadow effect of the surrounding environment, the related technology usually first detects the light source according to the brightness threshold recognition method, for example, the brightness greater than the preset brightness threshold The local area is determined as the light source, and then the subsequent processing is performed.

However, the robustness of this detection method is not high, and some highly reflective objects and bright metals may be mistakenly identified as light sources. To solve this problem, a new detection method is urgently needed.

Contents of the invention

The present application provides a light source detection method and related equipment, which can assist the first camera to accurately detect the light source in a simple and quick manner by performing defocusing and/or reducing exposure value processing on the second camera.

In a first aspect, a light source detection method is provided, which is applied to an electronic device including a first camera and a second camera, and the first camera and the second camera are used to shoot the same scene; the method includes:

launch the camera application of the electronic device;

using the first camera to capture a first image;

Perform virtual focus processing on the second camera, and/or reduce exposure value processing;

Using the processed second camera to capture a reference image;

Target light sources included in the first image are determined based on the reference image.

In the light source detection method provided by the embodiment of the present application, when the first camera and the second camera are enabled to take pictures, the processed second camera captures the The area of the non-real light source in the reference image becomes blurred and black; then, combined with the reference image to assist the first image to filter out the highlighted areas produced by highly reflective objects, bright metal, noise, etc., and identify the real light source, thereby improving the accuracy of light source detection.

With reference to the first aspect, in an implementation manner of the first aspect, performing virtual focus processing on the second camera includes:

reducing the distance between the lens in the second camera and the image sensor to be smaller than the distance between the lens in the first camera and the image sensor when capturing the first image.

In this implementation manner, since virtual focus processing is performed on the second camera, most objects in the first reference image captured by the second camera may become blurred. For example, objects in the background that are far away from the second camera can be blurred. Since the bright metal and highly reflective objects are imaged by reflecting bright light, the energy of the reflected light is very low compared to the energy of the luminous light source. Therefore, these bright metal and highly reflective objects in the first reference image correspond to Imaging, after defocus processing, also becomes blurred and blends with the background. At this time, only the real luminous light source will not be blurred after the virtual focus processing due to its high energy, so it can stand out in the blurred background and become brighter relative to the surroundings.

With reference to the first aspect, in an implementation manner of the first aspect, the processing of reducing the exposure value of the second camera includes:

reducing the exposure value corresponding to the second camera to half of the exposure value corresponding to the first camera when capturing the first image.

In this implementation manner, reducing the exposure value will cause the second reference image captured by the second camera to become a relatively dark image. At this time, since the bright metal and highly reflective objects themselves are imaged by reflecting bright light, the energy of the reflected light is very low compared to the ability of the light source. Therefore, these bright metal and highly reflective objects are displayed in the second reference image The corresponding imaging, as the exposure value decreases, also becomes a gray or black area; the bright area generated by the noise will also become gray or black as the exposure value decreases; at this time, only the real light source due to The energy is high, and after the second camera reduces the exposure value, it will still be relatively bright, close to white.

With reference to the first aspect, in an implementation manner of the first aspect, the processing of reducing the exposure value of the second camera includes:

Keeping the sensitivity constant, reducing the exposure time corresponding to the second camera; or,

The exposure time is kept constant, and the sensitivity corresponding to the second camera is reduced.

In this implementation manner, the exposure value of the second camera can be reduced by reducing the exposure time or sensitivity. In addition, exposure time and sensitivity can be reduced at the same time.

With reference to the first aspect, in an implementation manner of the first aspect, before performing virtual focus processing on the second camera, and/or before reducing the exposure value, the method further includes:

using the second camera to capture a second image;

performing brightness threshold recognition and shape detection on the first image and the second image, determining and matching the area to be tested in the first image and the area to be tested in the second image;

Based on the reference image, determining a target light source included in the first image includes:

performing the brightness threshold identification and the shape detection on the reference image, and determining a target area in the reference image, where the target area has a corresponding relationship with an area to be tested in the second image;

Determining the area to be measured in the first image with the corresponding target area as the target light source.

In this implementation, the area to be tested in the first image is matched with the area to be tested in the second image, and because the target area in the second reference image has a corresponding relationship with the area to be tested in the second image, Therefore, the area to be tested in the first image can be associated with the target area in the second reference image. When there is a target area associated with the area to be tested in the first image in the second reference image, it means that the area to be tested indicates the real position of the light source; If there is an associated target area, it means that the area to be detected indicates a false light source position, such as a bright area formed by a highly reflective object, a bright metal, or noise. Thus, with the second reference image as a reference, the real and false light sources in the first image can be determined through simple comparison.

With reference to the first aspect, in an implementation manner of the first aspect, the second camera includes at least one second camera.

When there are multiple second cameras, virtual focus and/or exposure value reduction processing can be performed on the multiple second cameras, and combined with the target areas in the reference images captured by the multiple second cameras, the first image can be determined target light source. The more auxiliary cameras, the more accurate the detection.

In a second aspect, an electronic device is provided, and the electronic device includes: one or more processors, a memory, and a display screen; the memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code comprising computer instructions that are invoked by the one or more processors to cause the electronic device to perform:

launch the camera application of the electronic device;

using the first camera to capture a first image;

Perform virtual focus processing on the second camera, and/or reduce exposure value processing;

Using the processed second camera to capture a reference image;

Target light sources included in the first image are determined based on the reference image.

With reference to the second aspect, in an implementation manner of the second aspect, performing virtual focus processing on the second camera includes:

reducing the distance between the lens in the second camera and the image sensor to be smaller than the distance between the lens in the first camera and the image sensor when capturing the first image.

With reference to the second aspect, in an implementation manner of the second aspect, the processing of reducing the exposure value of the second camera includes:

reducing the exposure value corresponding to the second camera to half of the exposure value corresponding to the first camera when capturing the first image.

With reference to the second aspect, in an implementation manner of the second aspect, the processing of reducing the exposure value of the second camera includes:

Keeping the sensitivity constant, reducing the exposure time corresponding to the second camera; or,

The exposure time is kept constant, and the sensitivity corresponding to the second camera is reduced.

With reference to the second aspect, in an implementation manner of the second aspect, before performing virtual focus processing on the second camera, and/or before reducing the exposure value, the method further includes:

using the second camera to capture a second image;

performing brightness threshold recognition and shape detection on the first image and the second image, determining and matching the area to be tested in the first image and the area to be tested in the second image;

Based on the reference image, determining a target light source included in the first image includes:

performing the brightness threshold identification and the shape detection on the reference image, and determining a target area in the reference image, where the target area has a corresponding relationship with an area to be tested in the second image;

Determining the area to be measured in the first image with the corresponding target area as the target light source.

With reference to the second aspect, in an implementation manner of the second aspect, the second camera includes at least one second camera.

In a third aspect, a light source detection device is provided, including a unit for performing any light source detection method in the first aspect.

In a possible implementation, when the light source detection device is an electronic device, the processing unit may be a processor, and the input unit may be a communication interface; the electronic device may also include a memory, and the memory is used to store computer programs A code for causing the electronic device to execute any method in the first aspect when the processor executes the computer program code stored in the memory.

In a fourth aspect, a chip is provided, the chip is applied to an electronic device, and the chip includes one or more processors, and the processor is used to invoke computer instructions so that the electronic device performs any of the tasks in the first aspect. A light source detection method.

In a fifth aspect, a computer-readable storage medium is provided, the computer-readable storage medium stores computer program codes, and when the computer program codes are executed by an electronic device, the electronic device executes any of the items in the first aspect. A light source detection method.

In a sixth aspect, a computer program product is provided, the computer program product comprising: computer program code, when the computer program code is run by an electronic device, the electronic device is made to perform any light source detection in the first aspect method.

In the light source detection method provided in the embodiment of the present application, by enabling one first camera and at least one second camera, when shooting, the first camera is used to capture the first image, and the second camera is defocused and the exposure value is reduced. Then use the processed second camera to capture a second reference image; perform brightness threshold recognition and shape detection on the second reference image captured by the second camera; then, use the detection result to assist the first image to screen out highly reflective objects, The highlight area generated by highlighting metal, noise, etc., from which the real light source is identified as the target light source.

Since related electronic equipment generally includes multiple cameras, this application does not need to make structural modifications, and only needs to call multiple cameras, and the detection can be performed after simple virtual focus and/or exposure value reduction processing. The method is simple and the detection The efficiency is high, and the detection accuracy is also very high.

Description of drawings

Figure 1 is a schematic diagram of an application scenario applicable to this application;

Fig. 2 is a schematic flow chart of a light source detection method provided by an embodiment of the present application;

FIG. 3 is a schematic layout diagram of a camera provided in an embodiment of the present application;

FIG. 4 is a schematic diagram of an image during a light source detection process provided by an embodiment of the present application;

FIG. 5 is a schematic diagram of another light source detection process provided by the embodiment of the present application;

Fig. 6 is a schematic diagram of a display interface provided by an embodiment of the present application;

FIG. 7 is a schematic diagram of a hardware system applicable to the electronic device of the present application;

Fig. 8 is a schematic diagram of a software system applicable to the electronic device of the present application;

Fig. 9 is a schematic structural diagram of a light source detection device provided by the present application;

FIG. 10 is a schematic structural diagram of a chip provided by the present application.

detailed description

The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

First of all, some terms used in the embodiments of the present application are explained to facilitate the understanding of those skilled in the art.

1. The RGB (red, green, blue) color space, or RGB domain, refers to a color model related to the structure of the human visual system. According to the structure of the human eye, all colors are seen as different combinations of red, green and blue.

2. The YUV color space, or YUV domain, refers to a color coding method, Y represents brightness, and U and V represent chroma. The above-mentioned RGB color space focuses on the human eye's perception of color, while the YUV color space focuses on the sensitivity of vision to brightness. RGB color space and YUV color space can be converted to each other.

3. The brightness (lighting value, LV) value is used to estimate the ambient brightness. The specific calculation formula is as follows:

Among them, Exposure is the exposure time, Aperture is the aperture size, ISO is the sensitivity, and Luma is the average value of Y in the XYZ color space.

4. Autofocus (automatic focus, AF) means that the electronic device obtains the highest image frequency component by adjusting the position of the focusing lens to obtain a higher image contrast. Among them, automatic focusing is a process of continuous accumulation. The electronic device compares the contrast of images captured by the lens at different positions, so as to obtain the position of the lens when the contrast of the image is maximum, and then determine the focal length of the focus.

5. Focal length, the size of the focal length marks the size of the refractive power, the shorter the focal length, the greater the refractive power.

The focal length of the optical lens assembly determines the size of the image generated on the imaging plane of the object captured by the optical lens assembly. Assuming that the same subject is photographed at the same distance, the longer the focal length of the optical lens assembly, the greater the magnification of the image generated by the subject on the charge-coupled device (CCD).

6. The focus distance refers to the distance between the object and the imaging, which is the sum of the distance from the lens to the object and the distance from the lens to the photosensitive element.

7. Shooting parameters, including shutter speed, exposure time, aperture value (AV), exposure value (exposure value, EV) and sensitivity ISO. The following are introduced respectively.

The shutter is a device that controls the length of time that light enters the camera to determine the exposure time of the image. The longer the shutter is left open, the more light enters the camera and the longer the corresponding exposure time for the image. Conversely, the shorter the shutter remains open, the less light enters the camera and the shorter the exposure time for the image.

Exposure time refers to the time for the shutter to be open in order to project light onto the photosensitive surface of the photosensitive material of the camera. The exposure time is determined by the sensitivity of the photosensitive material and the illuminance on the photosensitive surface. The longer the exposure time, the more light enters the camera, and the shorter the exposure time, the less light enters the camera. Therefore, a long exposure time is required in a dark scene, and a short exposure time is required in a backlit scene.

The aperture value (f value) is the ratio of the focal length of the lens (lens) in the camera to the diameter of the lens. The larger the aperture value, the more light enters the camera. The smaller the aperture value, the less light enters the camera.

Exposure value is a value that combines the exposure time and aperture value to represent the light-transmitting ability of the camera lens. The exposure value can be defined as:

Among them, N is the aperture value; t is the exposure time in seconds.

ISO, a measure of how sensitive a film is to light, known as sensitivity or gain. For insensitive negatives, longer exposure times are required to achieve the same brightness as sensitive negatives. For sensitive negatives, shorter exposure times are required to achieve the same brightness as for insensitive negatives.

Among the shooting parameters, shutter, exposure time, aperture value, exposure value and ISO, electronic equipment can realize auto focus (auto focus, AF), auto exposure (automatic exposure, AE), auto white balance (auto white balance, AWB) through algorithms At least one of them to realize the automatic adjustment of these shooting parameters.

Exemplarily, the above exposure value may be any one of -24, -4, -3, -2, -1, 0, 1, 2, 3, 4, 24.

The exposure image corresponding to EV0 is used to indicate the exposure image captured by the determined exposure value 0 when the electronic device achieves exposure through an algorithm. The exposure image corresponding to EV-2 is used to indicate the exposure image captured by the determined exposure value -2 when the electronic device achieves exposure through an algorithm. The exposure image corresponding to EV1 is used to indicate the exposure image captured by the determined exposure value 1 when the electronic device achieves exposure through an algorithm. Others are analogized in turn, and will not be repeated here.

Among them, every increase of 1 in the exposure value will change the exposure by one stop, that is, doubling the exposure (referring to the integral of the illuminance received by a surface element on the surface of the object within time t), such as doubling the exposure time or aperture area. An increase in exposure value would then correspond to a slower shutter speed and smaller f-number. It can be seen from this that the exposure value of EV0 is increased by 2 relative to EV-2, which changes the exposure by two stops; similarly, the exposure value of EV1 is increased by 1 relative to EV0, which changes the exposure by one stop.

Here, when the exposure value EV is equal to 0, the exposure value is usually the best exposure value under the current lighting conditions. Correspondingly, the exposure image correspondingly acquired by the electronic device under the condition of EV0 is an optimal exposure image under the current lighting condition, and the optimal exposure image may also be referred to as a reference exposure image.

It should be understood that the "best" exposure image refers to an exposure image determined by a given electronic device through an algorithm. When the electronic device is different, the algorithm is different, or the current lighting conditions are different, the determined optimal exposure image is different.

The above is a brief introduction to the nouns involved in the embodiments of the present application, and details will not be repeated below.

The light source detection method provided in the embodiment of the present application may be applicable to various electronic devices.

In some embodiments of the present application, the electronic device may be various camera devices such as sports cameras, digital cameras, mobile phones, smart screens, tablet computers, wearable electronic devices, vehicle-mounted electronic devices, and augmented reality (augmented reality, AR) devices , virtual reality (virtual reality, VR) equipment, notebook computer, ultra-mobile personal computer (ultra-mobile personal computer, UMPC), netbook, personal digital assistant (personal digital assistant, PDA), projector, etc. The specific type of electronic device is not limited in any way.

In combination with electronic devices, FIG. 1 is a schematic diagram of an application scenario of a light source detection method provided in an embodiment of the present application.

Exemplarily, the light source detection method in the embodiment of the present application can be applied to the photographing field; for example, the light source detection method of the present application can be applied to photographing objects including the light source in a shooting environment with light sources. The shooting environment may be a shooting environment such as a natural exterior scene, a real scene exterior scene, a real scene interior scene, or a studio interior scene. Light sources may include natural light sources and artificial light sources, for example, the sun, a turned on electric light, a burning candle, etc. are all light sources. Wherein, the light source may include one light source, or may include multiple light sources.

For example, as shown in FIG. 1 , taking the electronic device 100 as an example of a mobile phone, the photographed objects include a glass door of a convenience store, a Christmas tree in front of the door, and a light source 101 (for example, a street lamp) that provides illumination next to it. The electronic device 100 runs the camera application program, and during the process of the electronic device 100 collecting images including the subject, the preview image of the subject including the light source 101 can be displayed on the display screen in real time; when the user is viewing the display screen of the electronic device 100 When previewing the image, if you want to shoot with the angle of view shown in Figure 1, you can click the shooting control displayed on the display screen. When the shooting control of the electronic device 100 is triggered, the electronic device 100 can capture an image of the subject including the light source 101 as shown in FIG. 1 .

As shown in FIG. 1 , when an electronic device photographs an object including the light source 101, the relatively high brightness generated by the light source affects the light and shadow effect of the surrounding environment, thereby affecting the image captured by the image sensor, which may cause the electronic device to capture The image appears blurred, partially overexposed and other problems. Therefore, related technologies usually first detect the position of the light source in the image based on a brightness threshold recognition method, such as determining a local area greater than a preset brightness threshold as the area where the light source is located, and then perform correction or other series of processing.

However, the detection method provided by the related technology is not robust, and the light source detection result is inaccurate. For the real luminous light source, the bright area formed by high reflection, and the bright area formed by noise, the relevant methods cannot correctly distinguish, and some highly reflective objects and bright areas formed by high-brightness metals will be mistakenly detected as light sources, which will cause Follow-up problems. To solve this problem, a new detection method is urgently needed.

In view of this, an embodiment of the present application provides a light source detection method. When the first camera and the second camera are enabled to take pictures, the second camera is processed by defocusing and/or reducing the exposure value, so that the processed The area of the non-real light source in the reference image collected by the second camera becomes blurred and blackened; then, combined with the reference image, the first image is assisted to filter out the highlighted areas generated by highly reflective objects, bright metal, noise, etc., The real light source is identified therefrom, thereby improving the accuracy of light source detection.

Exemplarily, when the camera application is in the preview state (for example, photo preview), the preview image displayed by the electronic device includes local blurring, overexposure, etc., and after the shooting control of the electronic device is triggered, the implementation of the present application can be executed. The light source detection method provided in the example; or, when the electronic device detects that the user clicks on a certain camera mode (for example, large aperture mode, night scene mode), according to the detection result combined with related image processing methods, a clearer and better contrast can be obtained , Light and shadow effects more natural images.

Optionally, if the electronic device has sufficient computing power, the light source detection method in the embodiment of the present application may also be applied to the field of video recording, video calling, or other image processing fields.

Exemplarily, video call scenarios may include but not limited to the following scenarios:

Video calls, video conferencing applications, long-term and short-term video applications, video live broadcast applications, video online course applications, application scenarios of smart portrait mirroring, system camera recording function recording video, video surveillance, or portrait shooting scenarios such as smart cat eyes, etc.

It should be understood that the foregoing is an illustration of an application scenario, and does not limit the application scenario of the present application in any way.

The light source detection method provided by the embodiment of the present application will be described in detail below with reference to FIG. 2 to FIG. 6 .

Fig. 2 is a schematic flowchart of a light source detection method provided by an embodiment of the present application. The method 200 can be executed by an electronic device including at least two cameras.

Exemplarily, FIG. 3 is a layout diagram of a camera on an electronic device according to an embodiment of the present application.

Taking the electronic device 100 as a mobile phone as an example, as shown in (a) in FIG. 3 , two cameras may be arranged on the back cover of the mobile phone, respectively located in two circular areas at the upper left corner of the back cover of the mobile phone. The two cameras may be a wide-angle camera 1931 and a telephoto camera 1932 respectively. Alternatively, as shown in (b) of FIG. 3 , four cameras may be arranged on the back cover of the mobile phone, and the four cameras are located in a circular area in the center above the back cover of the mobile phone. The four cameras may be, for example, a super wide-angle camera 1933 , a wide-angle camera 1931 , a telephoto camera 1932 and a multi-spectral camera 1934 .

It should be understood that the wide-angle camera 1931 is suitable for shooting close-up shots due to its small focusing distance, and, as the name suggests, the wide-angle camera is suitable for shooting scenes with a relatively large field of view. The field of view range corresponding to the super wide-angle camera 1933 is relatively larger than that of the wide-angle camera 1931, and the focusing distance is smaller, which is suitable for shooting scenes with a larger field of view. The field of view range corresponding to the telephoto camera 1932 is relatively smaller than that of the wide-angle camera 1931, and the focusing distance is relatively large, so it is suitable for shooting distant scenes.

Multispectral camera 1934, a camera including a multispectral sensor, wherein the multispectral sensor is another multispectral sensor with a wider spectral response range than the three primary colors (RGB) sensor. For example, the multi-spectral camera may be a red-green-blue-magenta-yellow (RGBCMY) sensor, and the color reproduction capability and signal-to-noise ratio of the multi-spectral sensor are improved compared with the RGB sensor. The field angle range corresponding to the multispectral camera 1934 may be consistent with the field angle range corresponding to the main camera.

Relatively speaking, the image obtained by the super wide-angle camera 1933 has richer details and higher definition, while the image obtained by the telephoto camera 1932 has less detail and relatively lower definition.

Of course, the above are only two examples. Three or more cameras can also be arranged on the back cover of the mobile phone. Modification, the embodiment of this application does not impose any limitation on this.

In this embodiment of the application, two or more cameras can be enabled, one of which can be called the first camera, and the other cameras can be called the second camera; the first camera is used for normal image collection, and the second camera is used for After processing, assist in identifying the light source in the image captured by the first camera. It can be understood that the first camera and the second camera are only an indication, and the indicated specific cameras can be replaced as required.

For example, when an electronic device includes two cameras, one of which is a wide-angle camera and the other is a super-wide-angle camera, the wide-angle camera can be called a first camera, and the super-wide-angle camera can be called a second camera. The ultra-wide-angle camera is used to assist in identifying light sources in images captured by the wide-angle camera.

With reference to the above examples, the light source detection method 200 provided in the embodiment of the present application may include the following S201 to S206, and S201 to S206 will be described in detail below.

S201. Start a camera application in the electronic device.

Exemplarily, the user can instruct the electronic device to start the camera application by clicking the icon of the "camera" application; or, when the electronic device is in a locked screen state, the user can slide right on the display screen of the electronic device The electronic device is instructed to launch a camera application. Alternatively, the electronic device is in a locked screen state, and the lock screen interface includes an icon of the camera application program, and the user instructs the electronic device to start the camera application program by clicking the icon of the camera application program. Alternatively, when the electronic device is running other applications, the application has the permission to call the camera application program; the user can instruct the electronic device to start the camera application program by clicking a corresponding control. For example, when the electronic device is running an instant messaging application, the user may instruct the electronic device to start the camera application by selecting the control of the camera function.

It should be understood that the above is an example of the operation of starting the camera application program; the camera application program can also be started by voice instruction operation, or other operation instruction electronic devices; this application does not make any limitation on this.

It should also be understood that launching the camera application may refer to running the camera application.

S202. Collect a first image by using the first camera.

Wherein, the first image is an image collected by the first camera for the subject, and the first camera may be any type of camera in an ultra-wide-angle camera, a wide-angle camera, a telephoto camera and a multispectral camera; the subject may include a light source, The first image includes one or more regions to be detected, and the regions to be detected can be determined through brightness threshold recognition and shape detection.

It should be understood that the first image may be an image captured by the first camera after the electronic device executes an algorithm related to the first camera, determines and adjusts the target distance and exposure value between the lens and the image sensor according to the algorithm. The first image is a clear image. Here, the target distance between the lens corresponding to the first camera and the image sensor refers to the distance between the lens of the first camera and the image sensor when capturing a clear image.

It should be understood that the one or more areas to be detected included in the first image are the positions of the light sources to be detected, which may be bright areas corresponding to real light sources, or bright areas formed by highly reflective objects or bright metals.

It should also be understood that the area to be detected may be determined after threshold recognition based on RGB pixel values corresponding to each pixel in the image; or, may be determined after threshold recognition based on brightness values. The shape of the area to be measured can be divided according to needs, which is not limited in this embodiment of the present application.

Exemplarily, the first image may be an image as shown in (a) in FIG. position of the light source.

Optionally, the first image may be an image located in the RAW domain, an image located in the RGB domain, or may also be an image located in the YUV domain.

S203. Perform virtual focus processing on the second camera, and use the virtual focus processed second camera to capture a first reference image.

Wherein, the first reference image is an image collected by the second camera based on the first exposure value, after performing virtual focus processing on the subject, and the second camera can also be any one of a super wide-angle camera, a telephoto camera and a multi-spectral camera type of camera; the type of the second camera may be different from that of the first camera, or may also be the same. The subject captured by the second camera is the same as the subject captured by the first camera.

Optionally, as an implementation manner, before performing virtual focus processing on the second camera, the electronic device may further capture a second image by using the second camera. The second image is an image collected by the second camera after the electronic device executes an algorithm related to the second camera, determines and adjusts the target distance and exposure value between the lens and the image sensor according to the algorithm. The second image is a clear image. Here, the target distance between the lens of the second camera and the image sensor refers to the distance between the lens of the second camera and the image sensor when capturing a clear image. The second image may include a plurality of regions to be detected, and the regions to be detected can be determined through brightness threshold recognition and shape detection; the positions of the regions to be detected in the second image and the regions to be detected in the first image have a matching relationship.

For example, the region to be measured with the center coordinates of (i1, j1) in the first image and the region to be measured with the center coordinates of (p2, q2) in the second image indicate the same light source in the subject, such as the same street light.

On this basis, performing virtual focus processing on the second camera refers to adjusting the determined target distance, so that the distance between the lens and the image sensor in the second camera is reduced, so that the second camera after the virtual focus processing The first reference image is blurred, with reduced sharpness relative to the second image. The first reference image is also blurred with reduced clarity relative to the first image.

It should be understood that, in addition to the second camera, the electronic device may also include a focus processing component and a focus motor. The focus processing component is used to collect the image formed on the image sensor in the second camera, and control the rotation of the focus motor according to the clarity of the collected image. The rotation of the focusing motor drives the lens in the second camera to move, thereby realizing the focusing function. For example, the focus motor may indicate an optical image stabilization (OIS) motor.

Here, when performing virtual focus processing, the focus processing component can control the rotation of the focus motor, and the rotation of the focus motor drives the lens in the second camera to move in a direction close to the image sensor, so that the distance between the lens in the second camera and the image sensor Decreases, the second camera gradually goes out of focus, and a blurred first reference image is collected.

Optionally, as another implementation, when performing virtual focus processing on the second camera, the target distance between the lens in the first camera and the image sensor can be obtained first, and then the distance between the lens and the image in the second camera can be set. The target distance between the sensors is a value smaller than the target distance between the lens in the first camera and the image sensor. Then, according to the set value, the lens in the second camera is adjusted, and then the adjusted second camera is used for acquisition, so that a blurred first reference image can be obtained.

Exemplarily, if the first camera is a wide-angle camera and the second camera is an ultra-wide-angle camera, it is determined through calculation that the target distance between the corresponding lens and the image sensor when the first camera captures the first image is V1, and the second camera The distance between the middle lens and the image sensor should be smaller than the target distance V1 of the first camera, so the distance between the middle lens and the image sensor of the second camera can be set as V2, and V2 is smaller than V1.

Further, if the minimum distance between the lens and the image sensor in the second camera is V3, the distance between the lens and the image sensor in the second camera can also be directly set to be V3. At this time, it is equivalent to that the focus motor does not need to rotate Driving the lens in the second camera to move, V3 is smaller than V1, and V3 is smaller than or equal to V2.

Optionally, the field angle range corresponding to the first camera is smaller than or equal to the field angle range corresponding to the second camera.

It should be understood that when the field angle range corresponding to the first camera is smaller than or equal to the field angle range corresponding to the second camera, when the image captured by the second camera is used to assist the first camera in identifying the real light source position, the All regions to be tested in the images collected by the first camera are identified.

For example, the field of view range corresponding to the first camera is equal to the field of view range corresponding to the second camera, the first image collected by the first camera may be as shown in (a) in Figure 4; the second camera performs virtual focus After processing, the collected first reference image may be the image shown in (b) of FIG. 4 .

If the field of view range corresponding to the first camera is larger than the field of view range corresponding to the second camera, the content of the image captured by the second camera is equivalent to a part of the content of the image captured by the first camera. When assisting the first camera to identify the real light source position, only part of the region to be tested in the image captured by the first camera can be identified. If you want to detect all the content of the first camera, you need to use the second camera to capture multiple first reference images, and the corresponding content of each first reference image is different, and multiple first reference images can be overwritten after deduplication to the field of view range of the first camera.

In the embodiment of the present application, since the virtual focus processing is performed on the second camera, most of the objects in the first reference image captured by the second camera can become blurred, for example, objects farther away from the second camera can be made The objects in the background become blurred; the bright metal and highly reflective objects are imaged by reflecting bright light, and the energy of the reflected light is very low compared to the energy of the light source. Therefore, these bright metal and highly reflective objects The corresponding imaging in the first reference image also becomes blurred after defocus processing, and is mixed with the background. At this time, only the real luminous light source will not be blurred after the virtual focus processing due to its high energy, so it can stand out in the blurred background and become brighter relative to the surroundings.

It should be noted that, since the second camera needs to be processed later, the first reference image may not be collected after only the virtual focus processing is performed on the second camera.

S204. Reduce the exposure value corresponding to the second camera, and use the second camera with the reduced exposure value to capture a second reference image.

Wherein, the second reference image is an image captured by the second camera based on the second exposure value of the subject; or, the second reference image is an image acquired by the second camera based on the second exposure value of the subject after virtual focus processing. The second exposure value is smaller than the first exposure value, and at the same time, the second exposure value is also smaller than the corresponding exposure value when the first camera captures the first image.

Here, the first exposure value may be the same as the corresponding exposure value when the first camera captures the first image, or the first exposure value may be smaller than the corresponding exposure value when the first camera captures the first image.

Optionally, in order to enhance the light-dark contrast effect, the reduced second exposure value may be half of the corresponding exposure value of the first camera.

For example, the exposure value corresponding to the second camera is half of the exposure value corresponding to the first camera. The first image captured by the first camera can be as shown in (a) in Figure 4; after the second camera performs virtual focus processing, the first reference image collected can be as shown in (b) in Figure 4 Image; after the exposure value corresponding to the second camera decreases, the second reference image that continues to be collected may be the image shown in (c) in FIG. 4 .

Optionally, since the exposure value is related to the exposure time and the sensitivity ISO, when reducing the exposure value corresponding to the second camera, the sensitivity ISO can be kept unchanged, and the exposure time can be reduced; or, the exposure time can also be maintained The same, but reduce the size of the ISO sensitivity.

In the embodiment of the present application, reducing the exposure value will make the second reference image captured by the second camera become a relatively dark image; at this time, since the bright metal and highly reflective objects themselves are imaged by reflecting the bright light, The energy of the reflected light is very low compared to the ability of the luminous light source. Therefore, the corresponding imaging of these bright metal and highly reflective objects in the second reference image also becomes a gray or black area as the exposure value decreases; noise The generated bright areas will also turn gray and black as the exposure value decreases; at this time, only the real light source will still be relatively bright and close to white due to its high energy after the second camera reduces the exposure value.

It should be understood that since the energies of the luminescent light sources may vary, after the exposure value is reduced, the image of the luminous light source with a relatively lower energy in the second reference image will also become grayer.

S205. Perform brightness threshold recognition and shape detection on the second reference image to determine a target area.

Exemplarily, if the second reference image is an image in the YUV domain, it can be judged for the Y value corresponding to the pixel in the second reference image, and it is determined that the pixel area connected by the pixel greater than the preset Y threshold is the preselected area; then Perform shape detection for the preselected area, and determine the shape corresponding to the preselected area, such as circle, rectangle, square, etc. The Y value represents the brightness level.

If the second reference image is not an image in the YUV domain, you can also use the color space conversion method to convert the second reference image into an image in the YUV domain, and then judge in conjunction with the preset Y threshold; the size of the preset Y threshold can be It can be set and modified as needed, and this embodiment of the present application does not impose any limitation on this.

It should be understood that the detection result determined for the second reference image may be a target area of a certain shape, and the Y value of the pixels in the target area is greater than a preset Y threshold. If one or more target areas are determined from the second reference image, the shapes corresponding to the multiple target areas may be the same or different.

Optionally, multiple second cameras with reduced exposure values can be used to capture multiple frames of second reference images, or second cameras with reduced exposure values can also be used to capture multiple frames of second reference images, and then the multi-frames Perform brightness threshold recognition and shape detection on the second reference image to determine the corresponding target areas; then, merge the target areas at similar positions in multiple frames of the second reference image to determine the corresponding intersection as the final target area .

It should be understood that the target area determined in the second reference image has a corresponding relationship with the area to be tested determined in the second image. For example, when the target area determined in the second reference image overlaps with the area to be measured determined in the second image, it is used to indicate the light source at the same position.

S206. Based on the detection result, identify the target light source in the first image.

Optionally, in combination with the target area in the second reference image, the intersection area of the area to be tested in the first image and the corresponding target area in the second reference image may be determined as the area where the target light source is located in the first image. If the area to be tested in the first image cannot find the corresponding target area in the second reference image and cannot be combined, it means that the area to be tested is a bright area formed by a non-real light source and cannot be used as the area where the target light source is located.

Here, it should be noted that, before performing S203, the content of the second image captured by the second camera may also be matched with the content of the first image captured by the first camera to generate a one-to-one correspondence; After the processing by the second camera, the corresponding target area corresponding to a certain area to be measured in the first image can be found from the collected second reference image by means of the corresponding relationship.

For example, the first image captured by the first camera may be shown in (a) in FIG. is the position of the light source to be detected.

The second reference image collected by the second camera after performing defocusing and reducing exposure value processing may be as shown in (c) in FIG. 4 . After performing brightness threshold recognition and shape detection on the second reference image, the resulting image can be shown as (b) in Figure 5, which can include three target areas, namely b1, b2 and b3, the target The shape of the area is circular. The three target areas are the real light sources assisted by the second camera.

Combining the detection results shown in (a) and (b) in Figure 5, it can be determined that the three regions to be tested in the first image are all real light source positions, and here the a1 in the second reference image and the area indicated by b1 in the second reference image are merged to obtain the intersection area a11 as the area where the first target light source is located; the area indicated by a2 in the first image and b2 in the second reference image can be obtained Merge to obtain the intersection area a12 as the area where the second target light source is located; similarly, merge the area indicated by a3 in the first image and b3 in the second reference image to obtain the intersection area a13 as the third target The area where the light source is located.

Here, it can also be determined that a region to be measured a4 in the first image has no matching target region in the second reference image, therefore, it can be determined that the region to be measured a4 is a non-real light source position.

In the light source detection method provided in the embodiment of the present application, by enabling one first camera and at least one second camera, when shooting, the first camera is used to capture the first image, and the second camera is defocused and the exposure value is reduced. Then use the processed second camera to capture a second reference image; perform brightness threshold recognition and shape detection on the second reference image captured by the second camera; then, use the detection result to assist the first image to screen out highly reflective objects, The highlight area generated by highlighting metal, noise, etc., from which the real light source is identified as the target light source.

Since related electronic devices generally include multiple cameras, this application does not need to make structural modifications, and only needs to call multiple cameras, and the detection can be carried out after simple virtual focus and exposure reduction processing. The method is simple and the detection efficiency is relatively high. High, the detection accuracy is also very high.

Optionally, one first camera and at least one second camera can also be enabled. When shooting, the first camera is used to capture the first image, and the second camera is subjected to virtual focus processing, and then the processed second camera is used to capture the image. The first reference image; perform brightness threshold recognition and shape detection on the first reference image collected by the second camera; then, use the detection result to assist the first image to filter out high-reflective objects, bright metal, noise, etc. Bright areas, from which the true light source is identified as the target light source.

Optionally, one first camera and at least one second camera can also be enabled. When shooting, the first camera is used to collect the first image, and the second camera is processed to reduce the exposure value, and then the processed second camera is used to Collect the second reference image; perform brightness threshold recognition and shape detection on the second reference image collected by the second camera; then, use the detection result to assist the first image to filter out highly reflective objects, bright metal, noise, etc. The highlighted area from which the real light source is identified as the target light source.

In addition, the first image may refer to a preview image in a preview interface of the electronic device; wherein, the preview interface may refer to a photo preview interface, or a video preview interface. The first image may also be a captured image in an album of the electronic device, or a video frame in a video.

When the first image is displayed, the detected position of the target light source may be displayed; of course, it may not be displayed, and this application does not impose any limitation on this.

The following describes a schematic diagram of an interface in an electronic device with reference to FIG. 6 .

In a possible implementation, the function of "light source detection" can be set to be enabled in the setting interface of the electronic device, and the function of "light source detection" can be automatically enabled after the application program for taking pictures in the electronic device is run. The light source detection method of the embodiment of the application.

In another possible implementation, the "light source detection" function can be enabled in the camera of the electronic device, and according to the setting, the "light source detection" function can be enabled when taking pictures, and the light source detection method of the embodiment of the present application can be executed .

In yet another possible implementation, the "light source detection" function can be added only in the "large aperture" mode of the camera of the electronic device. According to the addition, the "light source detection" function can be automatically turned on when the "large aperture" mode is selected. " function to execute the light source detection method of the embodiment of the present application.

Combining with the third implementation mode, taking the electronic device automatically enabling the "light source detection" function in the "large aperture" mode as an example, FIG. 6 is a schematic diagram of an interface of an electronic device provided by an embodiment of the present application.

As shown in (a) of FIG. 6 , it is a desktop 601 of an electronic device, and the desktop 601 includes an icon 602 corresponding to a camera application. The electronic device detects that the user clicks the icon 602 corresponding to the camera application on the desktop 601 , as shown in (b) of FIG. 6 . After the electronic device detects that the user clicks the icon 602 of the camera application on the desktop 601 , in response to the user's operation, the electronic device runs the camera application and displays a display interface 603 as shown in (c) in FIG. 6 , The display interface 603 includes a preview window, which can display a preview image captured by the first camera in the default photographing mode; the display interface 603 also includes other modes, such as a large aperture mode 604 and the like.

After the electronic device detects that the user has switched from the camera mode to the large aperture mode 604, it may execute the light source detection method provided in this application. For example, a first camera is used to capture a first image, and a second camera is enabled, and the second camera is processed to defocus and reduce the exposure value, and the processed second camera is used to capture a second reference image; for the second reference image Perform brightness threshold recognition and shape detection to determine the target area; then combine the target area to assist in determining the real light source in the first image as the target light source. Then, as shown in (d) of FIG. 6 , the first image is displayed in the preview window and the position of the light source in the first image may be prompted.

In this example, due to the strong light from the street light, when it shines on the glass door and the Christmas tree, the glass and metal decorations strongly reflect the light projected by the street light. If the relevant technology is used for detection, it may be wrong The fake street lamp reflected on the glass and the light reflected by the metal decoration are used as the light source; while using the light source detection method provided by the embodiment of the present application to detect, the second camera will perform defocusing and reduce the exposure value. During the process, the bright areas corresponding to the fake street lamps reflected on the glass and the light reflected by the metal decorations are blurred, darkened and then screened out, and the remaining bright areas are identified as the real light sources as street lamps.

It should be noted that the above description is an example of a display interface in an electronic device, and this application does not make any limitation thereto.

It should be understood that the above illustrations are intended to help those skilled in the art understand the embodiments of the present application, rather than to limit the embodiments of the present application to the illustrated specific values or specific scenarios. Those skilled in the art can obviously make various equivalent modifications or changes based on the above illustrations given, and such modifications or changes also fall within the scope of the embodiments of the present application.

The light source detection method, applicable scenarios and related display interfaces of the embodiments of the present application are described above with reference to FIG. 1 to FIG. 6 . The software system, hardware system, device and chip of the electronic equipment applicable to the present application will be described in detail below with reference to FIG. 7 to FIG. 10 . It should be understood that the software system, hardware system, device, and chip in the embodiments of the present application can execute the various methods in the foregoing embodiments of the present application, that is, the specific working processes of the following various products can refer to the corresponding processes in the foregoing method embodiments .

Fig. 7 shows a hardware system applicable to the electronic device of this application. The electronic device 100 may be used to implement the light source detection method described in the foregoing method embodiments.

The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, and an antenna 2 , mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and A subscriber identification module (subscriber identification module, SIM) card interface 195 and the like. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, bone conduction sensor 180M, etc.

It should be noted that the structure shown in FIG. 7 does not constitute a specific limitation on the electronic device 100 . In other embodiments of the present application, the electronic device 100 may include more or fewer components than those shown in FIG. 7 , or the electronic device 100 may include a combination of some of the components shown in FIG. 7 , or , the electronic device 100 may include subcomponents of some of the components shown in FIG. 7 . The components shown in FIG. 7 can be realized in hardware, software, or a combination of software and hardware.

Processor 110 may include one or more processing units. For example, the processor 110 may include at least one of the following processing units: an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor) , ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, neural network processor (neural-network processing unit, NPU). Wherein, different processing units may be independent devices or integrated devices.

The controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thus improving the efficiency of the system.

In some embodiments, processor 110 may include one or more interfaces. For example, the processor 110 may include at least one of the following interfaces: an inter-integrated circuit (inter-integrated circuit, I2C) interface, an inter-integrated circuit sound (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM ) interface, universal asynchronous receiver/transmitter (UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, SIM interface , USB interface.

Exemplarily, in the embodiment of the present application, the light source detection method provided in the present application may be executed in the processor 110 .

The connection relationship between the modules shown in FIG. 7 is only a schematic illustration, and does not constitute a limitation on the connection relationship between the modules of the electronic device 100 . Optionally, each module of the electronic device 100 may also adopt a combination of various connection modes in the foregoing embodiments.

The wireless communication function of the electronic device 100 may be realized by components such as the antenna 1 , the antenna 2 , the mobile communication module 150 , the wireless communication module 160 , a modem processor, and a baseband processor. Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 may be used to cover single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas.

The electronic device 100 can realize the display function through the GPU, the display screen 194 and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

Display 194 may be used to display images or video. The display screen 194 includes a display panel. The display panel can adopt liquid crystal display (liquid crystal display, LCD), organic light-emitting diode (organic light-emitting diode, OLED), active-matrix organic light-emitting diode (active-matrix organic light-emitting diode, AMOLED), flexible Light-emitting diode (flex light-emitting diode, FLED), mini light-emitting diode (mini light-emitting diode, Mini LED), micro light-emitting diode (micro light-emitting diode, Micro LED), micro OLED (Micro OLED) or quantum dots Diodes (quantum dotlight emitting diodes, QLEDs). In some embodiments, the electronic device 100 may include 1 or N display screens 194 , where N is a positive integer greater than 1.

The electronic device 100 can realize the shooting function through the ISP, the camera 193 , the video codec, the GPU, the display screen 194 , and the application processor.

Exemplarily, the ISP is used to process the data fed back by the camera 193 . For example, when taking a picture, open the shutter, the light is transmitted to the photosensitive element of the camera through the lens, and the light signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can optimize the algorithm of image noise, brightness and color, and ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP may be located in the camera 193 .

Exemplarily, the camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects it to the photosensitive element. The photosensitive element may be a charge coupled device (charge coupled device, CCD) or a complementary metal-oxide-semiconductor (complementary metal-oxide-semiconductor, CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard red green blue (red green blue, RGB), YUV and other image signals. In some embodiments, the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.

Exemplarily, the digital signal processor is used to process digital signals, and can process other digital signals in addition to digital image signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.

Exemplarily, a video codec is used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in various encoding formats, for example: moving picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3 and MPEG4.

Exemplarily, the pressure sensor 180A may be disposed on the display screen 194 . There are many types of pressure sensor 180A, for example, it may be a resistive pressure sensor, an inductive pressure sensor or a capacitive pressure sensor. The capacitive pressure sensor may include at least two parallel plates with conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes, and the electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the touch operation according to the pressure sensor 180A. The electronic device 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example: when the touch operation with the touch operation intensity less than the first pressure threshold acts on the short message application icon, execute the instruction of viewing the short message; when the touch operation with the intensity greater than or equal to the first pressure threshold acts on the short message application icon , to execute the instruction of creating a new short message.

Exemplarily, the ambient light sensor 180L is used to sense ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in the pocket, so as to prevent accidental touch.

Exemplarily, the fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement functions such as unlocking, accessing the application lock, taking pictures, and answering incoming calls.

Exemplarily, the touch sensor 180K is also called a touch device. The touch sensor 180K may be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, which is also called a touch screen. The touch sensor 180K is used to detect a touch operation on or near it. The touch sensor 180K may transmit the detected touch operation to the application processor to determine the touch event type. Visual output related to the touch operation can be provided through the display screen 194 . In some other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100 and disposed at a different position from the display screen 194 .

The hardware system of the electronic device 100 is described in detail above, and the software system of the electronic device 100 is introduced below. The software system may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application uses a layered architecture as an example to exemplarily describe the software system of the electronic device 100 .

As shown in FIG. 8 , the system architecture may include an application layer 210 , an application framework layer 220 , a hardware abstraction layer 230 , a driver layer 240 and a hardware layer 250 .

The application layer 210 may include a camera application.

Optionally, the application layer 210 may also include application programs such as gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.

The application framework layer 220 provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer; the application framework layer may include some predefined functions.

For example, the application framework layer 220 may include a camera access interface; the camera access interface may include camera management and camera equipment. Wherein, the camera management can be used to provide an access interface for managing the camera; the camera device can be used to provide an interface for accessing the camera.

The hardware abstraction layer 230 is used to abstract hardware. For example, the hardware abstraction layer can include the camera abstraction layer and other hardware device abstraction layers; the camera abstraction layer can include camera device 1, camera device 2, etc.; the camera hardware abstraction layer can be connected with the camera algorithm library, and the camera hardware abstraction layer can call Algorithms in the camera algorithm library.

Exemplarily, the camera algorithm library may include a light source detection method, which is used to execute the light source detection method provided in the embodiment of the present application when running the light source detection method.

The driver layer 240 is used to provide drivers for different hardware devices. For example, the driver layer may include camera device drivers.

The hardware layer 250 may include image sensors, multispectral sensors, image signal processors, and other hardware devices.

FIG. 9 is a schematic structural diagram of a light source detection device provided by an embodiment of the present application. The light source detection device 300 includes an acquisition unit 310 and a processing unit 320 .

The processing unit 320 is configured to start a camera application program of the electronic device;

The collection unit 310 is configured to use the first camera to collect a first image;

The processing unit 320 is configured to perform virtual focus processing on the second camera, and/or, reduce exposure value processing;

The collection unit 310 is configured to use the processed second camera to collect a reference image;

The processing unit 320 is further configured to determine the target light source included in the first image based on the reference image.

Optionally, as an embodiment, the processing unit 320 is further configured to:

reducing the distance between the lens in the second camera and the image sensor to be smaller than the distance between the lens in the first camera and the image sensor when capturing the first image.

Optionally, as an embodiment, the processing unit 320 is further configured to:

reducing the exposure value corresponding to the second camera to half of the exposure value corresponding to the first camera when capturing the first image.

Optionally, as an embodiment, the processing unit 320 is further configured to:

Keeping the sensitivity constant, reducing the exposure time corresponding to the second camera; or,

The exposure time is kept constant, and the sensitivity corresponding to the second camera is reduced.

Optionally, as an embodiment, the processing unit 320 is further configured to:

using the second camera to capture a second image;

performing brightness threshold recognition and shape detection on the first image and the second image, determining and matching the area to be tested in the first image and the area to be tested in the second image;

performing the brightness threshold identification and the shape detection on the reference image, and determining a target area in the reference image, where the target area has a corresponding relationship with an area to be tested in the second image;

Determining the area to be measured in the first image with the corresponding target area as the target light source.

Optionally, as an embodiment, the second camera includes at least one second camera.

It should be noted that the above-mentioned light source detection device 300 is embodied in the form of a functional unit. The term "unit" here may be implemented in the form of software and/or hardware, which is not specifically limited.

For example, a "unit" may be a software program, a hardware circuit or a combination of both to realize the above functions. The hardware circuitry may include application specific integrated circuits (ASICs), electronic circuits, processors (such as shared processors, dedicated processors, or group processors) for executing one or more software or firmware programs. etc.) and memory, incorporated logic, and/or other suitable components to support the described functionality.

Therefore, the units of each example described in the embodiments of the present application can be realized by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.

FIG. 10 shows a schematic structural diagram of an electronic device provided by the present application. The dotted line in FIG. 10 indicates that this unit or this module is optional, and the electronic device 400 can be used to implement the light source detection method described in the foregoing method embodiments.

The electronic device 400 includes one or more processors 401, and the one or more processors 402 can support the electronic device 400 to implement the method in the method embodiment. The processor 401 may be a general purpose processor or a special purpose processor. For example, the processor 401 may be a central processing unit (central processing unit, CPU), a digital signal processor (digital signal processor, DSP), an application specific integrated circuit (application specific integrated circuit, ASIC), a field programmable gate array (field programmable gate array, FPGA) or other programmable logic devices such as discrete gates, transistor logic devices, or discrete hardware components.

Optionally, the processor 401 may be used to control the electronic device 400, execute software programs, and process data of the software programs. The electronic device 400 may further include a communication unit 405, configured to implement signal input (reception) and output (send).

For example, the electronic device 400 can be a chip, and the communication unit 405 can be an input and/or output circuit of the chip, or the communication unit 405 can be a communication interface of the chip, and the chip can be used as a component of a terminal device or other electronic devices .

For another example, the electronic device 400 may be a terminal device, and the communication unit 405 may be a transceiver of the terminal device, or the communication unit 405 may be a transceiver circuit of the terminal device.

The electronic device 400 may include one or more memories 402, on which a program 404 is stored, and the program 404 may be run by the processor 401 to generate an instruction 403, so that the processor 401 performs the light source detection described in the above method embodiment according to the instruction 403 method.

Optionally, data may also be stored in the memory 402 .

Optionally, the processor 401 may also read data stored in the memory 402 , the data may be stored in the same storage address as the program 404 , and the data may also be stored in a different storage address from the program 404 .

Optionally, the processor 401 and the memory 402 may be set separately, or may be integrated together; for example, integrated on a system-on-chip (system on chip, SOC) of the terminal device.

Exemplarily, the memory 402 can be used to store the relevant program 404 of the light source detection method provided in the embodiment of the present application, and the processor 301 can be used to call the relevant program 404 of the light source detection method stored in the memory 402 during video processing, and execute The light source detection method of the embodiment of the present application.

The present application also provides a computer program product. When the computer program product is executed by the processor 401, the light source detection method described in any method embodiment in the present application is implemented.

The computer program product can be stored in the memory 402 , such as a program 404 , and the program 404 is finally converted into an executable target file that can be executed by the processor 401 through processes such as preprocessing, compiling, assembling and linking.

The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a computer, the light source detection method described in any method embodiment in the present application is implemented. The computer program may be a high-level language program or an executable object program.

Optionally, the computer-readable storage medium is, for example, the memory 402 . The memory 402 may be a volatile memory or a non-volatile memory, or, the memory 402 may include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory can be read-only memory (read-only memory, ROM), programmable read-only memory (programmable ROM, PROM), erasable programmable read-only memory (erasable PROM, EPROM), electrically programmable Erases programmable read-only memory (electrically EPROM, EEPROM) or flash memory. Volatile memory can be random access memory (RAM), which acts as external cache memory. By way of illustration and not limitation, many forms of RAM are available, such as static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous connection dynamic random access memory (synchlinkDRAM, SLDRAM) And direct memory bus random access memory (direct rambus RAM, DR RAM).

Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods may be implemented in other ways. For example, the embodiments of the electronic equipment described above are only illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be Incorporation may either be integrated into another system, or some features may be omitted, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

It should be understood that in various embodiments of the present application, the sequence numbers of the processes do not mean the order of execution, and the execution order of the processes should be determined by their functions and internal logic, rather than by the embodiments of the present application. The implementation process constitutes any limitation.

In addition, the term "and/or" in this article is only an association relationship describing associated objects, indicating that there may be three relationships, for example, A and/or B, which may mean: A exists alone, and A and B exist at the same time , there are three cases of B alone. In addition, the character "/" in this article generally indicates that the contextual objects are an "or" relationship.

If the functions described above are realized in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (read-only memory, ROM), random access memory (random access memory, RAM), magnetic disk or optical disk, and various media that can store program codes. .

The above is only a specific implementation of the application, but the scope of protection of the application is not limited thereto. Anyone familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the application. Should be covered within the protection scope of this application. Therefore, the protection scope of the present application should be based on the protection scope of the claims. In a word, the above description is only a preferred embodiment of the technical solution of the application, and is not intended to limit the protection scope of the application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application shall be included within the protection scope of this application.

Claims (9)

1. The light source detection method is characterized by being applied to electronic equipment comprising a first camera and a second camera, wherein the first camera and the second camera are used for shooting the same scene;

the method comprises the following steps:

starting a camera application of the electronic device;

acquiring a first image by using the first camera;

performing virtual focus processing on the second camera and/or reducing exposure value processing;

acquiring a reference image by using the processed second camera;

based on the reference image, a target light source included in the first image is determined.

2. The light source detection method according to claim 1, wherein performing the virtual focus processing on the second camera includes:

and reducing the distance between the lens in the second camera and the image sensor to be smaller than the distance between the lens in the first camera and the image sensor when the first image is acquired.

3. The light source detection method according to claim 1 or 2, wherein the exposure value reduction processing for the second camera includes:

and reducing the exposure value corresponding to the second camera to be half of the exposure value corresponding to the first camera when the first image is acquired.

4. The light source detection method according to claim 1 or 2, wherein the exposure value reduction processing for the second camera includes:

keeping the sensitivity unchanged, and reducing the exposure time corresponding to the second camera; or,

and keeping the exposure time unchanged, and reducing the corresponding sensitivity of the second camera.

5. The light source detection method according to claim 1 or 2, wherein before the virtual focus processing and/or the exposure value reduction processing is performed on the second camera, the method further comprises:

acquiring a second image by using the second camera;

performing brightness threshold recognition and shape detection on the first image and the second image, determining a region to be detected in the first image and a region to be detected in the second image, and matching;

determining a target light source included in the first image based on the reference image, including:

performing the brightness threshold recognition and the shape detection on the reference image, and determining a target area in the reference image, wherein the target area and the area to be detected in the second image have a corresponding relation;

and determining a region to be measured in the first image with the corresponding target region as a target light source.

6. The light source detection method of claim 1, wherein the second camera comprises at least one second camera.

7. An electronic device comprising a processor and a memory;

the memory for storing a computer program operable on the processor;

the processor for performing the light source detection method of any one of claims 1 to 6.

8. A chip, comprising: a processor for calling and running a computer program from a memory so that a device in which the chip is installed performs the light source detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program comprising program instructions that, when executed by a processor, cause the processor to perform the light source detection method according to any one of claims 1 to 6.

Images