WO2021057277A1

WO2021057277A1 - Photographing method in dark light and electronic device

Info

Publication number: WO2021057277A1
Application number: PCT/CN2020/107413
Authority: WO
Inventors: 杨邦慧; 李向东
Original assignee: 华为技术有限公司
Priority date: 2019-09-23
Filing date: 2020-08-06
Publication date: 2021-04-01
Also published as: CN110602403A

Abstract

Provided in the present application are a photographing method in dark light and an electronic device. According to the method, the current photography environment can be determined as a dark light environment according to brightness information of ambient light; when a user takes a portrait photograph in the dark light environment, the optimal photograph is selected from the existing photograph album by means of image recognition and processing technology, and the optimal facial detail information is acquired; fusion processing or replacement processing is performed on the optimal facial detail information in the existing photograph and the portrait photograph taken in the current dark light environment, such that the user can also clearly restore facial skin texture in the dark light environment, and take the most beautiful dark light portrait photograph; and combining with information such as the color temperature and color of the environment in the photograph currently taken, the synthesized photograph is integrally processed regarding the color temperature, color, white balance, etc., such that a photograph, which contains the optimal facial skin and is integrally harmonious and attractive, can also be acquired in the dark light environment, thereby improving the photography experience of the user.

Description

Method and electronic equipment for taking pictures under dark light

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on September 23, 2019, the application number is 201910899296.8, and the application name is "a method and electronic equipment for taking pictures under dark light", the entire content of which is incorporated by reference In this application.

Technical field

This application relates to the field of electronic technology, and in particular to a method and electronic equipment for taking pictures under dark light.

Background technique

At present, terminal devices such as smart phones have become necessities in people's daily life, and the camera function of terminal devices is widely and frequently used by users. In the process of taking pictures, the camera, as one of the most important key components on the smart phone, can collect the pictures or images that the user wants to take. In a well-lit environment, smartphones can basically take clear portrait shots; however, in low-light environments, it is difficult to clearly present the details of human faces, which makes it impossible to take clear portrait shots.

Summary of the invention

The present application provides a method and an electronic device for taking pictures in dark light, which can obtain photos containing the best facial skin quality and overall harmony and beauty in a dark light environment, thereby improving the user's shooting experience.

In a first aspect, a method for taking pictures in low light is provided, which is characterized in being applied to an electronic device, and the method includes: obtaining a first face photo in response to a user's photographing operation; when the electronic device is in a low light shooting environment When, according to the first face photo, determine the second face photo from at least one photo that has been saved in the electronic device, where the second face photo and the first face photo include facial feature information of the same user , The second face photo is the photo with the best face feature information in the at least one photo; when the face feature information of the first region of the second face photo is better than the corresponding first face photo When the face feature information of the first area is used, the face feature information of the first area of the second face photo is used to fuse the face feature information of the first area of the first face photo, or the first area of the first face photo is used. The face feature information of the first region of the two face photos replaces the face feature information of the first region of the first face photo to obtain a third face photo, and the first region is a part of the user's face Or all areas; save the third face photo to the local photo album of the electronic device.

It should be understood that the first face photo may be a selfie of the user or a photo taken by him, and the photo includes facial feature information. The electronic device may obtain the facial feature information included in the first facial photo, and according to the facial feature information of the first facial photo, determine multiple photos with the same facial feature information from the local album of the electronic device . Similar to the face recognition technology, the electronic device can select multiple photos that include the same facial feature information from a local album based on the collected facial feature information. In addition, the electronic device can identify the second face photo including clear facial skin quality and pore detail information from multiple photos. In other words, the second face photo is the photo with the best face detail information among the multiple photos. .

Through the above-mentioned method of taking pictures under low light, the electronic device can first determine that the current shooting environment is a low light environment according to the brightness information of the ambient light. When the user takes a portrait photo in a dark light environment, the best photo is selected from the existing album through image recognition and processing technology, and the best face detail information is obtained; then the best face in the existing photo The detail information and the current portrait photos taken in low light are fused or replaced, so that the user can clearly restore the skin texture of the face in the low light environment, and take the most beautiful low-light portrait photos.

With reference to the first aspect, in some implementations of the first aspect, the method further includes: when the face feature information of the first region of the first face photo is better than the first region corresponding to the second face photo When the face feature information of, it is determined that the first face photo is the third face photo and saved.

Specifically, in a possible implementation manner, comparing the face details of the currently taken photo (the first face photo) and the best photo (the second face photo) can be performed by comparing the sharpness of the photo. . For example, the image fine segmentation technology can be used to divide the face in the photo into different regions, and compare the face details of the different regions in the current photo and the best photo respectively.

Optionally, in the process of dividing the face, the face can be divided into 9 areas equally in the form of nine square grids; or, another possible way of dividing the face is divided according to the facial features of the face, that is, according to the forehead of the face , Cheeks, chin, eyes, nose, mouth and other areas are divided, which is not limited in this application.

It should be understood that the best detail information at the corresponding position of each region of the face can be obtained by comparison, and the detail information can include each pore, each skin texture, pixel information of each hair, and so on.

It should also be understood that the face division process is to divide the currently taken photo and the best photo separately through the same fine image segmentation technology. In other words, the current photo and the best photo are both averaged in the form of a nine-square grid. Divide into 9 regions and compare the detailed information of each region separately; or divide the face into 6 regions in the form of facial features for the currently taken photo and the best photo, and compare the detailed information of each region separately. In addition, the human face can also be divided into regions according to preset shapes, such as rectangular division, circular division, etc. The present application does not limit the form of human face division and the number of divided regions.

Combining the first aspect and the foregoing implementation manners, in some implementation manners of the first aspect, the facial feature information of the first area includes contrast information, average gradient information, and entropy information of pixels in the first area. At least one.

Optionally, in the process of evaluating the sharpness of the photo, the contrast value, the global average gradient value, and the global entropy value of the face photo can be combined to obtain a score for evaluating the overall sharpness of the picture. The score includes The evaluation results of the three dimensions of contrast, average gradient, and entropy can comprehensively reflect the clarity of the picture and improve the accuracy of the evaluation of picture clarity. This application does not limit the method of evaluating the clarity of the photo.

Combining the first aspect and the foregoing implementation manners, in some implementation manners of the first aspect, before saving the third face photo to the local photo album of the electronic device, the method further includes: obtaining information about the current low-light shooting environment; Perform an image processing operation on the third face photo according to the current information of the low-light shooting environment, and the image processing operation includes at least one processing mode of color temperature processing, color processing, and white balance processing.

Through the above technical solution, the best face detail information in the existing photos and the current portrait photos taken in low light are fused or replaced, and the texture of the face skin can be clearly restored in the low light environment, and the most beautiful dark photos can be taken. After the light portrait photo, combined with the color temperature, color and other information of the environment in the currently taken photo, the overall color temperature, color, white balance and other aspects of the synthesized photo are processed, so that in the dark environment, it can also obtain the most comprehensive information. The beautiful face and skin quality, and the overall harmonious and beautiful photos, improve the user experience.

With reference to the first aspect and the foregoing implementation manners, in some implementation manners of the first aspect, the method further includes: determining that the electronic device is in a low-light shooting environment.

Combining the first aspect and the foregoing implementation manners, in some implementation manners of the first aspect, determining that the electronic device is in a dark-light shooting environment includes: when the brightness of the ambient light collected by the ambient light brightness sensor of the electronic device is lower than a preset When the brightness threshold is set, it is determined that the electronic device is in a dark-light shooting environment; and/or when the electronic device turns on a flash or a soft light, it is determined that the electronic device is in a dark-light shooting environment.

The sensor of the mobile phone can collect the ambient light brightness information of the current shooting environment. When the brightness of the ambient light is less than the set brightness threshold, it is judged that the current shooting environment is a dark light shooting environment. When the brightness of the ambient light is greater than or equal to the set brightness When the brightness threshold is used, it is judged that the current shooting environment is a normal shooting environment. In this application, the normal shooting environment can be understood as an environment outside the low-light shooting environment.

When the ambient light brightness value acquired by the mobile phone is greater than or equal to the preset ambient light brightness threshold, and the mobile phone does not turn on the flash or soft light, the photo taken is the final photo captured and saved in the album. When the mobile phone judges that the current environment is in a low-light shooting environment according to the obtained ambient light brightness value less than the preset ambient light brightness threshold, or when the flash or soft light is turned on when the mobile phone takes pictures, it can be understood as the working mode of the flash It is an automatic mode. When the intensity of the ambient light is not within a certain preset range, the electronic device can automatically turn on the flash. In this scene, the method of taking pictures under dark light provided by this application can be used to obtain the most beautiful skin texture with clear human face. Portrait photos.

It should also be understood that when the flash or soft light is turned on on the mobile phone, the relationship between the ambient light brightness value and the preset ambient light brightness threshold may not be limited. For example, the user currently wants to take a picture that is clearer than the preview effect and manually turn it on In addition to scenes such as soft light or flashing light, you can use the method of taking pictures under dark light provided in this application to obtain the most beautiful portrait photos with clear human face skin texture. In the description of the embodiments of the present application, the above various possible scenes in which the photographing method of the present application is used to obtain the most beautiful portrait photos containing clear facial skin textures are collectively referred to as "photographing in the dark".

In a possible implementation manner, an interface for adjusting the brightness threshold of the ambient light can be provided to the user in the camera application, and the user can set or adjust the brightness threshold of the ambient light through the interface.

The above-described method of taking pictures under low light can perform a series of processing after the user has finished taking the picture, and has more processing time. For example, the corresponding processing can be performed when the system is idle, without occupying system resources.

In a second aspect, an electronic device is provided, including one or more processors; a memory; a plurality of application programs; and one or more programs, wherein the one or more programs are stored in the memory, when the one Or when multiple programs are executed by the processor, the electronic device is caused to perform the following steps: in response to the user's photographing operation, obtain a first face photo; when in a dark light shooting environment, according to the first face photo, A second face photo is determined from at least one saved photo, where the second face photo and the first face photo include facial feature information of the same user, and the second face photo is among the at least one photo The photo with the best facial feature information; when the facial feature information of the first region of the second face photo is better than the facial feature information of the first region corresponding to the first face photo, use the The face feature information of the first area of the second face photo is fused with the face feature information of the first area of the first face photo, or the face feature of the first area of the second face photo is used The information replaces the face feature information of the first area of the first face photo to obtain a third face photo, where the first area is part or all of the user’s face; save the third face photo to the local Photo album.

With reference to the second aspect, in some implementations of the second aspect, when the one or more programs are executed by the processor, the electronic device is caused to perform the following steps: When the face feature information is better than the face feature information of the first region corresponding to the second face photo, it is determined that the first face photo is a third face photo and saved.

Combining the second aspect and the foregoing implementation manners, in some implementation manners of the second aspect, the facial feature information of the first area includes contrast information, average gradient information, and entropy information of pixels in the first area. At least one.

Combining the second aspect and the foregoing implementation manners, in some implementation manners of the second aspect, before saving the third face photo to the local album, when the one or more programs are executed by the processor, the electronic device Perform the following steps: obtain information about the current low-light shooting environment; perform image processing operations on the third face photo according to the current low-light shooting environment information, and the image processing operations include color temperature processing, color processing, and white balance At least one of the processing methods.

Combining the second aspect and the foregoing implementation manners, in some implementation manners of the second aspect, when the one or more programs are executed by the processor, the electronic device is caused to perform the following steps: determining that it is in a low-light shooting environment.

Combining the second aspect and the foregoing implementation manners, in some implementation manners of the second aspect, when the one or more programs are executed by the processor, the electronic device is caused to perform the following steps: When the brightness of the ambient light collected by the sensor is lower than the preset brightness threshold, it is determined that it is in a low-light shooting environment; and/or the electronic device is determined to be in a low-light shooting environment when the flash or soft light is turned on.

In a third aspect, the present application provides a device included in an electronic device, and the device has the function of realizing the foregoing aspects and the behavior of the electronic device in the possible implementation manners of the foregoing aspects. The function can be realized by hardware, or the corresponding software can be executed by hardware. The hardware or software includes one or more modules or units corresponding to the above-mentioned functions. For example, a display module or unit, a detection module or unit, a processing module or unit, and so on.

In a fourth aspect, the present application provides an electronic device, including: a touch display screen, wherein the touch display screen includes a touch-sensitive surface and a display; a camera; one or more processors; a memory; a plurality of application programs; and one or Multiple computer programs. Among them, one or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instruction is executed by the electronic device, the electronic device is caused to execute the method of photographing under dark light in any possible implementation of any one of the foregoing aspects.

In a fifth aspect, the present application provides an electronic device including one or more processors and one or more memories. The one or more memories are coupled with one or more processors, and the one or more memories are used to store computer program codes. The computer program codes include computer instructions. When the one or more processors execute the computer instructions, the electronic device executes Any one of the above-mentioned aspects is a possible implementation of a method for taking pictures under dark light.

In a sixth aspect, the present application provides a computer storage medium, including computer instructions, which when the computer instructions run on an electronic device, cause the electronic device to perform any one of the possible methods for taking photos under dark light in any one of the foregoing aspects.

In a seventh aspect, the present application provides a computer program product, which when the computer program product runs on an electronic device, causes the electronic device to execute any one of the possible methods for taking photos in the dark in any one of the above aspects.

Description of the drawings

FIG. 1 is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application.

Fig. 2 is a schematic diagram of an example of the control structure of the photographing process of an electronic device.

FIG. 3 is a schematic diagram of an example of a graphical user interface for taking pictures in dark light according to an embodiment of the present application.

Fig. 4 is a schematic diagram of another example of a graphical user interface for taking pictures in dark light provided by an embodiment of the present application.

Fig. 5 is a schematic diagram of an example of face division provided by an embodiment of the present application.

FIG. 6 is a schematic flowchart of a method for taking pictures under dark light provided by an embodiment of the present application.

FIG. 7 is a schematic flowchart of a method for taking pictures under dark light provided by an embodiment of the present application.

FIG. 8 is a schematic diagram of a possible composition of an electronic device provided by an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, unless otherwise specified, "/" means or, for example, A/B can mean A or B; "and/or" in this document is only a description of related objects The association relationship of indicates that there can be three kinds of relationships, for example, A and/or B, which can indicate: A alone exists, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "plurality" refers to two or more than two.

Hereinafter, the terms “first” and “second” are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first" and "second" may explicitly or implicitly include one or more of these features.

The embodiment of the present application provides a method for taking pictures in dark light, which can be applied to electronic devices or a separate application program. The application program can implement the method of taking pictures under dark light conditions and restoring portrait details in this application. Specifically, the method for taking pictures in low light provided by the present application can present users with clear facial skin texture and other details through image recognition and processing technology, and solve the problem of serious loss of facial details when users take pictures or take selfies in low light conditions. The problem.

The method for taking pictures in dark light provided by the embodiments of the present application can be applied to electronic devices with camera functions such as mobile phones, tablet computers, and wearable devices. The embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

Exemplarily, FIG. 1 shows a schematic structural diagram of an electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, and an antenna 2. , Mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone jack 170D, sensor module 180, buttons 190, motor 191, indicator 192, camera 193, display screen 194, and Subscriber identification module (subscriber identification module, SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light Sensor 180L, bone conduction sensor 180M, etc.

It can be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or combine certain components, or split certain components, or arrange different components. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), and an image signal processor. (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU) Wait. Among them, the different processing units may be independent devices or integrated in one or more processors.

The controller may be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions.

A memory may also be provided in the processor 110 to store instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can store instructions or data that the processor 110 has just used or used cyclically. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. Repeated accesses are avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved.

For example, in this application, a firmware program (firmware) is stored in the memory, which is used to enable the controller or processor to implement the dark-light photography of this application through an interface or protocol and to clearly restore the facial skin details through image recognition and processing technology. .

In some embodiments, the processor 110 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, and a universal asynchronous transceiver (universal asynchronous) interface. receiver/transmitter, UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (SIM) interface, and / Or Universal Serial Bus (USB) interface, etc.

The I2C interface is a bidirectional synchronous serial bus, which includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple sets of I2C buses. The processor 110 may couple the touch sensor 180K, the charger, the flash, the camera 193, etc., respectively through different I2C bus interfaces. For example, the processor 110 may couple the touch sensor 180K through an I2C interface, so that the processor 110 and the touch sensor 180K communicate through an I2C bus interface to implement the touch function of the electronic device 100.

The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple sets of I2S buses. The processor 110 may be coupled with the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 may transmit audio signals to the wireless communication module 160 through an I2S interface, so as to realize the function of answering calls through a Bluetooth headset.

The PCM interface can also be used for audio communication to sample, quantize and encode analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface. In some embodiments, the audio module 170 may also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to realize the function of answering calls through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

The UART interface is a universal serial data bus used for asynchronous communication. The bus can be a two-way communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to realize the Bluetooth function. In some embodiments, the audio module 170 may transmit audio signals to the wireless communication module 160 through a UART interface, so as to realize the function of playing music through a Bluetooth headset.

The MIPI interface can be used to connect the processor 110 with the display screen 194, the camera 193 and other peripheral devices. The MIPI interface includes a camera serial interface (camera serial interface, CSI), a display serial interface (display serial interface, DSI), and so on. In some embodiments, the processor 110 and the camera 193 communicate through a CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate through a DSI interface to realize the display function of the electronic device 100.

The GPIO interface can be configured through software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface may be used to connect the processor 110 with the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, and so on. The GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, MIPI interface, etc.

The USB interface 130 is an interface that complies with the USB standard specification, and specifically may be a Mini USB interface, a Micro USB interface, a USB Type C interface, and so on. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones and play audio through the headphones. This interface can also be used to connect to other electronic devices, such as AR devices.

It can be understood that the interface connection relationship between the modules illustrated in the embodiment of the present application is merely a schematic description, and does not constitute a structural limitation of the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection modes in the foregoing embodiments, or a combination of multiple interface connection modes.

The charging management module 140 is used to receive charging input from the charger. Among them, the charger can be a wireless charger or a wired charger. The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charge management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 may also be provided in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may also be provided in the same device.

The wireless communication function of the electronic device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.

The antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. The mobile communication module 150 can provide a wireless communication solution including 2G/3G/4G/5G and the like applied to the electronic device 100.

The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The wireless communication module 160 can provide applications on the electronic device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), and global navigation satellites. System (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions.

The electronic device 100 implements a display function through a graphics processing unit (GPU), a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

The display screen 194 is used to display images, videos, and the like. The display screen 194 includes a display panel. The display panel can use liquid crystal display (LCD), organic light-emitting diode (OLED), active matrix organic light-emitting diode or active-matrix organic light-emitting diode (active-matrix organic light-emitting diode). AMOLED, flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or N display screens 194, and N is a positive integer greater than one.

The electronic device 100 may implement a shooting function through an image signal processor (ISP), a camera 193, a video codec, a GPU, a display screen 194, and an application processor. In this application, the camera 193 may be an optical zoom lens or the like, which is not limited in this application.

In some embodiments, the ISP may be set in the camera 193, which is not limited in this application.

The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and is projected to the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transfers the electrical signal to the ISP to convert it into a digital image signal. ISP outputs digital image signals to DSP for processing. DSP converts digital image signals into standard RGB, YUV and other formats of image signals. In some embodiments, the electronic device 100 may include one or N cameras 193, and N is a positive integer greater than one.

Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.

Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as: moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, and so on.

NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, for example, the transfer mode between human brain neurons, it can quickly process input information, and it can also continuously self-learn. Through the NPU, applications such as intelligent cognition of the electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, and so on.

The external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example, save music, video and other files in an external memory card.

The internal memory 121 may be used to store computer executable program code, where the executable program code includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121. The internal memory 121 may include a storage program area and a storage data area. Among them, the storage program area can store an operating system, at least one application program (such as a sound playback function, an image playback function, etc.) required by at least one function. The data storage area can store data (such as audio data, phone book, etc.) created during the use of the electronic device 100. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), and the like.

The electronic device 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. For example, music playback, recording, etc.

The audio module 170 is used to convert digital audio information into an analog audio signal for output, and is also used to convert an analog audio input into a digital audio signal. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be provided in the processor 110, or part of the functional modules of the audio module 170 may be provided in the processor 110.

The speaker 170A, also called "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music through the speaker 170A, or listen to a hands-free call.

The receiver 170B, also called "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a call or voice message, it can receive the voice by bringing the receiver 170B close to the human ear.

The microphone 170C, also called "microphone", "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can make a sound by approaching the microphone 170C through the human mouth, and input the sound signal into the microphone 170C. The electronic device 100 may be provided with at least one microphone 170C. In other embodiments, the electronic device 100 may be provided with two microphones 170C, which can implement noise reduction functions in addition to collecting sound signals. In other embodiments, the electronic device 100 may also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and realize directional recording functions.

The earphone interface 170D is used to connect wired earphones. The earphone interface 170D may be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, and a cellular telecommunications industry association (cellular telecommunications industry association of the USA, CTIA) standard interface.

The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A may be provided on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors and so on. The capacitive pressure sensor may include at least two parallel plates with conductive materials. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations that act on the same touch position but have different touch operation strengths may correspond to different operation instructions. For example, when a touch operation whose intensity of the touch operation is less than the first pressure threshold is applied to the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

The gyro sensor 180B may be used to determine the movement posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (ie, x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for image stabilization. Exemplarily, when the shutter is pressed, the gyro sensor 180B detects the shake angle of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to counteract the shake of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude based on the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

The magnetic sensor 180D includes a Hall sensor. The electronic device 100 may use the magnetic sensor 180D to detect the opening and closing of the flip holster. In some embodiments, when the electronic device 100 is a flip machine, the electronic device 100 can detect the opening and closing of the flip according to the magnetic sensor 180D. Furthermore, according to the detected opening and closing state of the holster or the opening and closing state of the flip cover, features such as automatic unlocking of the flip cover are set.

The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of electronic devices, and be used in applications such as horizontal and vertical screen switching, pedometers and so on.

Distance sensor 180F, used to measure distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 may use the distance sensor 180F to measure the distance to achieve fast focusing.

The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light to the outside through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode, and the pocket mode will automatically unlock and lock the screen.

The ambient light sensor 180L is used to sense the brightness of the ambient light. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived brightness of the ambient light. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in the pocket to prevent accidental touch.

In this application, when the user is taking a picture, the ambient light sensor 180L can collect the brightness information of the shooting environment, so that the electronic device can judge whether it is in the dark-light shooting environment according to the current ambient light brightness, and then use the method provided in this application to obtain Include photos with clear face details.

The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to realize fingerprint unlocking, access application locks, fingerprint photographs, fingerprint answering calls, and so on.

The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold value, the electronic device 100 reduces the performance of the processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 due to low temperature. In some other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

Touch sensor 180K, also called "touch panel". The touch sensor 180K may be disposed on the display screen 194, and the touch screen is composed of the touch sensor 180K and the display screen 194, which is also called a “touch screen”. The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. The visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100, which is different from the position of the display screen 194.

The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can obtain the vibration signal of the vibrating bone mass of the human voice. The bone conduction sensor 180M can also contact the human pulse and receive the blood pressure pulse signal. In some embodiments, the bone conduction sensor 180M may also be provided in the earphone, combined with the bone conduction earphone. The audio module 170 can parse the voice signal based on the vibration signal of the vibrating bone block of the voice obtained by the bone conduction sensor 180M, and realize the voice function. The application processor may analyze the heart rate information based on the blood pressure beating signal obtained by the bone conduction sensor 180M, and realize the heart rate detection function.

The button 190 includes a power-on button, a volume button, and so on. The button 190 may be a mechanical button. It can also be a touch button. The electronic device 100 may receive key input, and generate key signal input related to user settings and function control of the electronic device 100.

The motor 191 can generate vibration prompts. The motor 191 can be used for incoming call vibration notification, and can also be used for touch vibration feedback. For example, touch operations that act on different applications (such as photographing, audio playback, etc.) can correspond to different vibration feedback effects. Acting on touch operations in different areas of the display screen 194, the motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminding, receiving information, alarm clock, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

The indicator 192 may be an indicator light, which may be used to indicate the charging status, power change, or to indicate messages, missed calls, notifications, and so on.

The SIM card interface 195 is used to connect to the SIM card. The SIM card can be inserted into the SIM card interface 195 or pulled out from the SIM card interface 195 to achieve contact and separation with the electronic device 100. The electronic device 100 may support 1 or N SIM card interfaces, and N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. The same SIM card interface 195 can insert multiple cards at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 may also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as call and data communication. In some embodiments, the electronic device 100 adopts an eSIM, that is, an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

FIG. 2 is a schematic diagram of an example of the control structure of the photographing process of an electronic device. The control structure 200 includes an image signal processing (ISP) 201, a sensor 202, and an optical zoom lens 203. Wherein, the ISP 201 may correspond to the processor 110 in the electronic device 100 shown in FIG. 1, and may include a logic part and a firmware program (firmware) running on it. The sensor 202 may correspond to various sensors in the electronic device 100 shown in FIG. 1, such as an ambient light sensor 180L, a proximity light sensor 180G, etc.; the optical zoom lens 203 may correspond to the electronic device 100 shown in FIG. Camera 193.

Specifically, in this embodiment of the present application, the ISP 201 may be used to process the data fed back by the camera 193, and complete the digital image processing process through a series of digital image processing algorithms. As shown in Figure 2, when taking a picture, the shutter is opened, and the light is transmitted to the photosensitive element of the camera through the optical zoom lens 203. In other words, the optical zoom lens 203 can project the ambient light signal to the photosensitive area of the sensor 202, and the sensor 202 passes through the photoelectric Conversion, into an image visible to the naked eye. Then the internal original image (Bayer format) is sent to the ISP 201, and the ISP 201 is processed by an algorithm to output an image in the RGB space domain to the back-end acquisition unit.

In this process, ISP 201 controls the optical zoom lens 203, sensor 202, and ISP logic through the firmware running on it, and then completes functions such as automatic iris, automatic exposure, and automatic white balance to implement the embodiments of this application. Provided to restore the texture of the face skin in the dark, and take the clear and most beautiful portrait effect. In addition, ISP201 can also optimize the image noise, brightness, skin color, and optimize the exposure parameters, color temperature and other parameters of the shooting scene.

For ease of understanding, the following embodiments of the present application will take the electronic device (smartphone) with the structure shown in FIG. 1 and FIG. 2 as an example, combined with the accompanying drawings and application scenarios, to perform the method for taking pictures under dark light provided by the embodiments of the present application. Explain in detail.

At present, when the camera environment is dark, in order to improve the photo effect in the dark light, the existing smart phones are preset with front or rear flash or soft light. When the user takes a photo (or Selfie) in a dark light environment At the same time, by turning on the flash or soft light to fill in the light, and then increase the brightness of the shooting environment, so that you can take portraits. Some smart phones do not preset the front flash or soft light, but can fill in the light through the screen fill-in function. When the user takes a selfie in dim light, it will turn on the screen fill-in function, for example, by lighting the entire screen. Fill light, or light up a part of the screen continuously, fill the user's face with the screen brightness, and then when the user takes a photo (or self-portrait) in a dark light environment, a self-portrait with a clear face can be obtained. Although the above method can improve the overall brightness of the user during preview and filming, the loss of portrait details in the photos obtained by the user is still very serious, especially the loss of details such as facial skin quality.

In addition, users can use a third-party lighting device to fill light in a low-light environment. When a third-party lighting device is used, the ambient brightness can be greatly improved, thereby achieving a better portrait shooting effect. However, this method requires the user to carry a third-party lighting device with him, which is cumbersome to operate and is not portable enough to carry the lighting device, and the user experience is poor.

For another example, some smart phones have realized the super night scene function of ultra-long exposure and multi-frame synthesis, which can improve the overall brightness of photos taken in low light; or, through the combination of ultra-long exposure and flash or soft light, Improve the sharpness of photos taken in low light. Although the overall brightness of the final film can be improved, the loss of portrait details, especially facial skin quality, is still very serious.

The methods introduced above increase the brightness of the shooting environment and make the faces in the captured photos brighter, but there will still be the loss of facial skin quality, such as texture, pores, and other information, and the presentation of portrait details is still not clear enough.

Fig. 3 is a schematic diagram of an example of a graphical user interface (GUI) for taking pictures in dark light provided by an embodiment of the present application. This application will take a smartphone as an example to introduce in detail the method of taking pictures under dark light provided by this application. . Wherein, (a) in FIG. 3 shows that in the unlock mode of the mobile phone, the screen display system of the mobile phone displays the currently output interface content 301, which is the main interface of the mobile phone. The interface content 301 displays a variety of third-party applications (application, App), such as Alipay, photo album, music, settings, WeChat, card package, Weibo, camera and other applications. It should be understood that the interface content 301 may also include other more application programs, which is not limited in this application.

As shown in (a) of FIG. 3, the user clicks on the camera application, and in response to the user's click operation, the mobile phone enters the main interface 302 of the camera application as shown in (b). The main interface 302 of the camera application includes multiple controls, which are respectively used to implement various functions of the camera application, which will not be repeated here.

It should be understood that the method of taking photos or taking selfies in a dark light environment provided by this application requires a mobile phone to monitor the brightness of the ambient light. When it is determined that the current shooting environment is a dark light shooting environment, the method of this application is used to process the photos to Get the most beautiful portrait photo with clear skin texture.

Specifically, as shown in Table 1 below, the sensor of the mobile phone can collect the ambient light brightness information of the current shooting environment. When the brightness of the ambient light is less than the set brightness threshold, it is determined that the current shooting environment is a dark light shooting environment. When the brightness of the light is greater than or equal to the set brightness threshold, it is determined that the current shooting environment is a normal shooting environment. In this application, the normal shooting environment can be understood as an environment outside the low-light shooting environment.

Table 1

参数parameter	暗光拍摄环境Low light shooting environment	正常拍摄环境Normal shooting environment
环境光亮度(单位：勒克斯lux)Ambient brightness (unit: lux)	0-500-50	>50>50

For example, the user can set the brightness threshold of the ambient light through the setting control 10. As shown in (b) of FIG. 3, the user clicks the setting control 10, and in response to the user's click operation, the mobile phone enters the setting interface 303 shown in (c). On the setting interface 303, it includes general settings for the camera application's shooting resolution, geographic location information, automatic watermarking and other options, as well as the camera settings for the AI master photographer, reference line, shooting sound, smiley capture and other options during the shooting process , This application does not limit this. In this application, the interface provided for the user to adjust the brightness threshold of the ambient light can be set in the general setting menu of the camera.

As shown in (c) in FIG. 3, after the user clicks on the "ambient light brightness" option, the user enters the setting interface 304 of ambient light brightness as shown in (d) in FIG. In the setting interface 304 of ambient light brightness, "default mode" can be included. The default mode of ambient light brightness can be the ambient light brightness threshold that has been set when the phone leaves the factory. The threshold can be unmodifiable by the user, for example (d ) "Ambient light brightness threshold=50 lux" shown in the figure.

Optionally, the setting interface 304 of ambient light brightness may also include a "custom mode", through which the user can modify the ambient light brightness threshold.

Fig. 4 is a schematic diagram of another example of a graphical user interface for taking pictures in dark light provided by an embodiment of the present application. As shown in Figure 4 (a), when the user clicks on the "Custom Mode" option, an interface 306 as shown in Figure 4 (b) will pop up, which enumerates multiple custom strategies, For example, custom strategy 1, custom strategy 2, custom strategy 3, etc. Among them, the enumerated custom strategy can be the ambient light brightness threshold previously set by the user. For example, if a user sets the ambient light brightness threshold for the first time = 40 lux, the mobile phone will automatically save the set ambient light brightness threshold = 40 lux For custom strategy 1, this application does not limit the number of strategies enumerated in the custom mode.

Optionally, in addition to selecting a custom strategy that has been set before, the user can also execute the custom strategy shown in Figure 4 (b) to add the "+" option to enter (c) in Figure 4 ) The ambient light brightness threshold adjustment interface shown in the figure, drag the progress bar of the brightness range to set the ambient light brightness threshold, or manually enter the brightness threshold, click "Save" to automatically save and select the current custom strategy4.

In addition, the user can delete and modify each custom strategy. For example, as shown in Figure 4 (d), the user can click "Custom Strategy 3" to delete and modify the custom strategy. , This application does not limit this.

It should be understood that in the above-mentioned setting process of the ambient light brightness threshold, when the user does not click this interface to set the ambient light brightness threshold, the camera application defaults to the "default mode" if the user does not set it. Select the default ambient light brightness threshold, for example, the default ambient light brightness threshold=50lux. When the user selects the custom mode for setting, the camera then uses the newly set ambient light brightness threshold to determine whether the current shooting environment is a low-light shooting environment.

It should be understood that when the ambient light brightness value obtained by the mobile phone is greater than or equal to the preset ambient light brightness threshold, and the mobile phone does not turn on the flash or soft light, the photo taken is the finally obtained photo saved in the album. Of course, this algorithm can also be called to optimize face photos when the ambient light brightness value is greater than or equal to the preset ambient light brightness threshold to meet the needs of special scenes. For example, although the overall ambient light brightness is greater than the threshold, the light Scenes where the angle or layout or local light does not meet the shooting requirements, or scenes with special requirements for the skin texture of a certain part of the face. In this scenario, the user can manually operate, or input preset gestures, or trigger the execution of the optimization function through a menu or a specific button, so as to obtain a portrait photo containing a clear facial skin texture. When the mobile phone judges that the current environment is in a low-light shooting environment according to the obtained ambient light brightness value less than the preset ambient light brightness threshold, or when the flash or soft light is turned on when the mobile phone takes pictures, it can be understood as the working mode of the flash It is an automatic mode. When the intensity of the ambient light is not within a certain preset range, the electronic device can automatically turn on the flash. In this scene, the method of taking pictures under dark light provided by this application can be used to obtain the most beautiful skin texture with clear human face. Portrait photos.

In a possible implementation, when user A takes a photo in the dark, through image recognition technology, identify other existing photos of user A in the mobile phone album, and compare the current photo taken by user A Compare the face details with other photos of user A, and select the best photo. Then, through image processing technology, the selected best detail information is fused with the current photo, and finally the most beautiful portrait photo with clear facial skin texture is synthesized.

Optionally, the mobile phone can use image recognition technology to select the best photo containing clear texture, pores and other facial and skin quality information from the existing photos of user A in the album. There are no restrictions on the method of selecting the best photo among the photos. For example, the current photos of user A in the mobile phone album can be compared with the details of the face respectively, so as to obtain the best photos containing clear texture, pores and other facial skin quality information.

After the face details of the existing photos of the user A in the album are compared, and the best photo is determined, the face details of the currently taken photo and the best photo are compared.

In a possible implementation, comparing the face details of the currently taken photo and the best photo can be done by comparing the sharpness of the photo.

Fig. 5 is a schematic diagram of an example of face division provided by an embodiment of the present application. Specifically, the face in the photo can be divided into different regions through the image fine segmentation technology, and the face details of the different regions in the currently taken photo and the best photo can be compared respectively.

Optionally, in the process of dividing the face, as shown in (a) in FIG. 5, the face may be divided into 9 regions evenly in the form of a nine-square grid, for example, regions 41-49 in (a). Then compare separately to obtain the best detail information at the corresponding position of each region of the face. The detail information may include each pore, each skin texture, each hair pixel, and so on.

Or, another possible way of dividing the face is to divide the face according to the five senses, that is, to divide the face according to the forehead, cheeks, chin, eyes, nose, mouth and other areas. As shown in (b) in Figure 5, the face can be divided into 6 areas based on the user’s facial features, such as the eye area 49, nose area 50, left cheek area 51, and right cheek area 52 in (b). , The mouth area 53 and the chin area 54. Then compare the best detail information at the corresponding position of each region of the face. The detail information can include each pore, each skin texture, and pixel information of each hair.

It should be understood that the face division process is to divide the currently taken photo and the best photo separately through the same fine image segmentation technology. In other words, both the currently taken photo and the best photo are divided equally into faces in the form of nine square grids. 9 regions, compare the details of each region separately; or divide the face into 6 regions in the form of facial features for the currently taken photo and the best photo, and compare the details of each region separately. In addition, in processing the face division method introduced in FIG. 5, the face can also be divided into regions according to preset shapes, such as rectangular division, circular division, etc. The form of face division and the number of divided regions are different in this application. Make a limit.

In a possible implementation manner, the sharpness parameter of the corresponding position of the human face in the photo is calculated, and then the picture with the best sharpness of the position is obtained by comparison, and the detailed information of the best photo position is obtained and stored.

Optionally, in the process of evaluating the sharpness of the photo, the contrast value of the photo, the global average gradient value and the global entropy value can be combined to obtain a score for evaluating the overall sharpness of the picture, and the score includes the contrast. The evaluation results of the three dimensions of, average gradient and entropy can comprehensively reflect the clarity of the picture and improve the accuracy of the evaluation of picture clarity. This application does not limit the method of evaluating the clarity of the photo.

Specifically, in the embodiment of the present application, contrast information, average gradient information, and entropy value information are used as parameters for evaluating clarity. In the process of comparing detailed information of the same region of a human face, the five sense organs are divided by (b) in Figure 4 As an example, calculate the contrast, average gradient, and entropy of the corresponding position of the face in the photo to determine the best detail information of the best photo.

According to the best photo listed above and the contrast information and average gradient information of the 6 regions of the currently taken photo, from the best photo and the currently taken photo, select the best part of the face detail for each different region, and select The optimal part of the fusion process is performed with the corresponding area of the current photo. For example, for the nose area 50, the contrast of the nose area 50 of the best photo is better than the contrast of the nose area 50 of the current photo. That is, the mobile phone judges that the details of the best photo are better than the current photo. Therefore, the best photo is The nose area 50 and the nose area 50 of the currently taken photo are fused.

It should be understood that the contrast information, average gradient information, and entropy information listed above can be the average contrast information, average gradient information, and entropy information of all pixels in a certain area, or the average of partially sampled pixels. Contrast information, average gradient information, and entropy information are not limited in this application.

It should also be understood that in the process of comparing sharpness information, different numerical information can be comprehensively considered, or different numerical information can be set priority, for example, priority is given to the best photo and the average of pixels in a certain area of the currently taken photo. Contrast information, when the average gradient information is the same or the difference is not too close, then the average gradient information is compared, and when the contrast information and the average gradient information are the same or the difference is not too close, then the entropy information is compared, so To obtain the best detailed information, this application is not limited.

In a possible implementation, the best part of the best photo is fused or replaced with the corresponding area of the currently taken photo, which can be based on the currently taken photo, using the best detail in the corresponding area of the face Information is replaced and supplemented, combined with super-resolution technology, and the corresponding high-resolution images are reconstructed from low-resolution images, thereby synthesizing photos that can present the best facial skin quality and other details in the dark.

It should be understood that in the process of fusing or replacing the optimal part of the best photo with the corresponding area of the currently taken photo in this application, the fusion processing may be a corresponding fusion of a certain area of the human face, or a full-face area. Fusion of face details; the user can specify one or more regions to which the fusion process is applicable; the replacement process can only replace the face details of a certain area, or replace the face details of multiple areas, or all areas of the entire face All are replaced, and this application does not limit it.

For example, when it is determined that the detail information of the nose region 50 of the currently captured face photo is worse than the detail information of this region of the best photo, and the detail information of other regions are similar, only the nose region 50 can be replaced. Or, when it is determined that the detail information of all areas of the entire face of the currently captured face photo is worse than the detail information of the corresponding area of the best photo, all areas of the entire face may be replaced.

In another possible implementation manner, in the process of fusion processing, it is also possible to perform unified image processing on the entire replaced photo according to the environmental information in the currently taken photo, so as to obtain an overall harmonious and beautiful photo.

It should be understood that the environmental information can include information such as the color temperature and color of the environment in the currently taken photo. The mobile phone can process the overall color temperature, color, white balance and other aspects of the synthesized photo by obtaining the environmental information, so as to obtain the It presents the best facial skin quality under light, and the overall harmonious and beautiful photos.

Specifically, in combination with the system architecture diagrams shown in Figures 1 and 2, ISP 201 can complete the effect processing of digital images through a series of digital image processing algorithms, which can mainly include 3A algorithm processing, dead pixel correction processing, denoising processing, Strong light suppression processing, backlight compensation processing, color enhancement processing, lens shading correction processing, etc. As shown in the related description in Figure 2, ISP 201 includes the implementation of the above-described digital image effect processing through the firmware running on it, and then completes the functions of automatic iris, automatic exposure, automatic white balance, etc., which will not be repeated here.

Through the above-mentioned method of taking pictures under low light, the electronic device can determine that the current shooting environment is a low light environment according to the brightness information of the ambient light. When the user takes a portrait photo in a dark light environment, the best photo is selected from the existing album through image recognition and processing technology, and the best face detail information is obtained; then the best face in the existing photo The detail information and the current portrait photos taken in low light are fused or replaced, so that the user can clearly restore the texture of the face and skin in the dark environment, and take the most beautiful low-light portrait photos; at the same time, combine the current shots The color temperature, color, and other information of the environment in the photos, and the overall color temperature, color, white balance and other aspects of the synthesized photos are processed, so that in the dark environment, the best facial skin quality can be obtained, and the overall harmony and beauty Photos to improve the user experience.

In combination with the above-mentioned embodiments and related drawings, the embodiments of the present application provide a method for taking pictures under dark light. This method can be used in electronic devices with cameras (such as mobile phones, tablet computers, etc.) as shown in Figures 1 and 2 In the realization. FIG. 6 is a schematic flowchart of a method for taking pictures under dark light provided by an embodiment of the present application. As shown in FIG. 6, the method may include the following steps:

601: The electronic device enters a photo preview state.

It should be understood that the photo preview state here may be a preview state of the user taking a photo or taking a self-portrait.

602. Identify the brightness of the surrounding environment through the ambient light sensor of the electronic device, and record the brightness value of the ambient light.

The ambient light sensor 180L is used to sense the brightness of the ambient light. When the user is taking a photo, the ambient light sensor 180L can collect the brightness information of the shooting environment, so that the electronic device can determine whether it is in a dark-light shooting environment according to the current ambient light brightness, and thus use the method provided in this application to obtain the best human face. Photos of skin texture.

In addition, the ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The electronic device can automatically adjust the white balance according to the perceived brightness of the ambient light, so as to process the overall color temperature, color, white balance and other aspects of the synthesized photo, so that it can also obtain the best facial skin quality in the dark environment. , And overall harmonious and beautiful photos,

603. The user takes a photo.

For example, the user presses the camera control of the electronic device to complete a photo or self-portrait action.

604. The electronic device determines whether the recorded brightness value of the ambient light is greater than a preset brightness threshold, and the flash or soft light is not currently turned on.

605: When the electronic device judges that the current environment is in a low-light shooting environment according to the obtained ambient light brightness value being less than the preset ambient light brightness threshold, or when the flash or soft light is turned on when the mobile phone takes pictures, the image recognition technology is used to identify Compare other photos of the current user and select the best photo with clear facial skin texture.

606. After determining the best photo, use image processing technology to merge the best part of the best photo with the corresponding area of the currently taken photo.

It should be understood that after the best photo is determined, the electronic device can obtain the face detail information of the best photo, such as each pore, each skin texture, the contrast information of each hair, average Gradient information and entropy information, etc., so as to replace and supplement the corresponding area of the face with face detail information.

607: Acquire a final photo after image processing, and save it in a local photo album.

Optionally, in the fusion process of 606, it is also possible to perform unified image processing on the entire replaced photo according to the environmental information in the currently taken photo, so as to obtain an overall harmonious and beautiful photo. For example, the electronic device can perform color temperature processing, color processing, and white balance processing on the synthesized photo as a whole by acquiring environmental information, so as to obtain a photo that can present the best facial skin quality under dark light and is harmonious and beautiful overall.

Optionally, when the ambient light brightness value obtained by the mobile phone is greater than or equal to the preset ambient light brightness threshold, and the mobile phone does not turn on the flash or soft light, the photo taken by the user is the final photo obtained and saved in the album. There is no need to perform image processing on the captured photos, which is not limited in this application.

It should be understood that the above-described method of taking photos under low light can perform a series of processing after the user has completed the shooting, and has more processing time. For example, corresponding processing can be performed when the system is idle, and system resources may not be occupied.

Through the above-mentioned method flow of taking pictures under low light, the electronic device can first determine that the current shooting environment is a low light environment according to the brightness information of the ambient light. When the user takes a portrait photo in a dark light environment, the best photo is selected from the existing album through image recognition and processing technology, and the best face detail information is obtained; then the best face in the existing photo The detail information and the current portrait photos taken in low light are fused or replaced, so that the user can clearly restore the texture of the face and skin in the dark environment, and take the most beautiful low-light portrait photos; at the same time, combine the current shots The color temperature, color, and other information of the environment in the photos, and the overall color temperature, color, white balance and other aspects of the synthesized photos are processed, so that in the dark environment, the best facial skin quality can be obtained, and the overall harmony and beauty Photos to improve the user experience.

In combination with the above-mentioned embodiments and related drawings, the embodiments of the present application provide a video playback method, which can be implemented in an electronic device (such as a mobile phone, a tablet computer, etc.) having a touch screen and a camera as shown in FIG. 1. FIG. 7 is a schematic flowchart of a method for taking pictures under dark light provided by an embodiment of the present application. As shown in FIG. 7, the method may include the following steps:

701. Acquire a first face photo in response to a user's photographing operation.

702. When the electronic device is in a low-light shooting environment, determine a second face photo from at least one photo that has been saved in the electronic device according to the first face photo, wherein the second face photo The photo and the first face photo include facial feature information of the same user, and the second face photo is a photo with the best facial feature information among the at least one photo.

It should be understood that the first face photo may be a selfie of the user or a photo taken by him, and the photo includes facial feature information. The electronic device may obtain the facial feature information included in the first facial photo, and according to the facial feature information of the first facial photo, determine multiple photos with the same facial feature information from the local album of the electronic device . Similar to the face recognition technology, the electronic device can select multiple photos that include the same facial feature information from a local album based on the collected facial feature information.

In addition, the electronic device can identify the second face photo including clear facial skin quality and pore detail information from multiple photos. In other words, the second face photo is the photo with the best face detail information among the multiple photos. .

703. When the face feature information of the first region of the second face photo is better than the face feature information of the first region corresponding to the first face photo, use the second face photo The face feature information of the first region is fused with the face feature information of the first region of the first face photo, or the face feature information of the first region of the second face photo is used The information replaces the face feature information of the first region of the first face photo to obtain a third face photo, and the first region is a part or all of the user's face.

It should be understood that after the electronic device determines the currently acquired first face photo and the second face photo saved in the local album, it can compare the currently taken photo with the face detail information of the best photo through image processing technology. It is done by comparing the sharpness of the photos. For example, the image fine segmentation technology can be used to divide the face in the photo into different regions, and compare the face details of the different regions in the current photo and the best photo respectively.

Specifically, the sharpness parameter of the corresponding position of the human face in the photo is calculated, and then the picture with the best sharpness of the position is obtained by comparison, and the detailed information of the best photo position is obtained and stored.

The electronic device can merge the optimal part of the best photo (the second face photo) with the corresponding area of the currently taken photo (the first face photo), which can be based on the currently taken photo, corresponding to the face The region is replaced and supplemented with the best detail information, combined with super-resolution technology, and the corresponding high-resolution image is reconstructed from the low-resolution image, so as to synthesize photos that can show the best facial skin quality and other details in the dark.

It should be understood that in the process of fusing or replacing the optimal part of the best photo with the corresponding area of the currently taken photo in this application, the fusion processing may be a corresponding fusion of a certain area of the human face, or a full-face area. The fusion of human face details; the replacement process can only replace the face details of a certain area, or replace the face details of multiple areas, or replace all areas of the entire face, which is not limited in this application.

704. Save the third face photo to a local photo album of the electronic device.

After the above processing, the processed photo is finally obtained, that is, the third face photo, which is saved in the local photo album.

It can be understood that, in order to implement the above-mentioned functions, an electronic device includes hardware and/or software modules corresponding to each function. In combination with the algorithm steps of the examples described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed by hardware or computer software-driven hardware depends on the specific application and design constraint conditions of the technical solution. Those skilled in the art can use different methods for each specific application in combination with the embodiments to implement the described functions, but such implementation should not be considered as going beyond the scope of the present application.

In this embodiment, the electronic device may be divided into functional modules according to the foregoing method examples. For example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is illustrative, and is only a logical function division, and there may be other division methods in actual implementation.

In the case of dividing each functional module corresponding to each function, FIG. 8 shows a schematic diagram of a possible composition of the electronic device 800 involved in the foregoing embodiment. As shown in FIG. 8, the electronic device 800 may include: an acquiring unit 801, a processing unit 802, and a display unit 803.

Wherein, the acquiring unit 801 may be used to support the electronic device 800 to perform the above-mentioned step 601, etc., and/or used in other processes of the technology described herein.

The processing unit 802 may be used to support the electronic device 800 to perform the foregoing steps 602, 603, etc., and/or other processes used in the technology described herein.

The display unit 803 may be used to support the electronic device 800 to display final photos, etc., and/or other processes used in the technology described herein.

It should be noted that all relevant content of the steps involved in the foregoing method embodiments can be cited in the functional description of the corresponding functional module, and will not be repeated here.

The electronic device provided in this embodiment is used to perform the above-mentioned method for taking pictures under dark light, and therefore can achieve the same effect as the above-mentioned implementation method.

In the case of an integrated unit, the electronic device may include a processing module, a storage module, and a communication module. The processing module can be used to control and manage the actions of the electronic device. For example, it can be used to support the electronic device to execute the steps performed by the acquisition unit 801, the processing unit 802, and the display unit 803. The storage module can be used to support the electronic device to execute the storage program code and data. The communication module can be used to support the communication between electronic devices and other devices.

Among them, the processing module may be a processor or a controller. It can implement or execute various exemplary logical blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, and so on. The storage module may be a memory. The communication module may specifically be a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, and other devices that interact with other electronic devices.

In an embodiment, when the processing module is a processor and the storage module is a memory, the electronic device involved in this embodiment may be a device having the structure shown in FIG. 1.

This embodiment also provides a computer storage medium in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device is caused to execute the above-mentioned related method steps to realize the dark-light shooting in the above-mentioned embodiment. Methods.

This embodiment also provides a computer program product, which when the computer program product runs on a computer, causes the computer to execute the above-mentioned related steps, so as to realize the method of photographing under dark light in the above-mentioned embodiment.

In addition, the embodiments of the present application also provide a device. The device may specifically be a chip, component or module. The device may include a processor and a memory connected to each other. The memory is used to store computer execution instructions. When the device is running, The processor can execute the computer-executable instructions stored in the memory, so that the chip executes the method of photographing under dark light in the foregoing method embodiments.

Among them, the electronic device, computer storage medium, computer program product, or chip provided in this embodiment are all used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the corresponding method provided above. The beneficial effects of the method will not be repeated here.

Through the description of the above embodiments, those skilled in the art can understand that for the convenience and conciseness of the description, only the division of the above-mentioned functional modules is used as an example for illustration. In practical applications, the above-mentioned function can be assigned to different functions according to needs. The function module is completed, that is, the internal structure of the device is divided into different function modules to complete all or part of the functions described above.

In the several embodiments provided in this application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, for example, multiple units or components may be combined or It can be integrated into another device, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate parts may or may not be physically separate, and the parts displayed as units may be one physical unit or multiple physical units, that is, they may be located in one place, or they may be distributed to multiple different places. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application are essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of software products, which are stored in a storage medium. It includes several instructions to make a device (may be a single-chip microcomputer, a chip, etc.) or a processor (processor) execute all or part of the steps of the methods in the embodiments of the present application. The foregoing storage media include: U disk, mobile hard disk, read only memory (read only memory, ROM), random access memory (random access memory, RAM), magnetic disk or optical disk and other media that can store program codes.

The above content is only the specific implementation of this application, but the protection scope of this application is not limited to this. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application. Covered in the scope of protection of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

A method for taking pictures in low light, which is characterized in that it is applied to an electronic device, and the method includes:

In response to the user's photographing operation, obtain the first face photo;

When the electronic device is in a low-light shooting environment, according to the first face photo, a second face photo is determined from at least one photo that has been saved by the electronic device, wherein the second face photo and The first face photo includes face feature information of the same user, and the second face photo is a photo with the best face feature information among the at least one photo;

When the face feature information of the first region of the second face photo is better than the face feature information of the first region corresponding to the first face photo, all the features of the second face photo are used. The face feature information of the first region is fused with the face feature information of the first region of the first face photo, or replaced with the face feature information of the first region of the second face photo Obtaining face feature information of the first region of the first face photo to obtain a third face photo, where the first region is part or all of the user's face;

Saving the third face photo to the local photo album of the electronic device.
The method according to claim 1, wherein the method further comprises:

When the face feature information of the first region of the first face photo is better than the face feature information of the first region corresponding to the second face photo, the first face photo is saved.
The method according to claim 1 or 2, wherein the facial feature information of the first region includes at least one of contrast information, average gradient information, and entropy information of pixels in the first region .
The method according to any one of claims 1 to 3, wherein before the saving the third face photo to the local photo album of the electronic device, the method further comprises:

Obtain information about the current low-light shooting environment;

Perform an image processing operation on the third face photo according to the current information of the low-light shooting environment, and the image processing operation includes at least one of color temperature processing, color processing, and white balance processing.
The method according to any one of claims 1 to 4, wherein the method further comprises:

It is determined that the electronic device is in a dark light shooting environment.
The method according to claim 5, wherein the determining that the electronic device is in a low-light shooting environment comprises:

When the brightness of the ambient light collected by the ambient light brightness sensor of the electronic device is lower than the preset brightness threshold, it is determined that the electronic device is in a dark light shooting environment; and/or

When the flash or soft light is turned on by the electronic device, it is determined that the electronic device is in a low-light shooting environment.
An electronic device, characterized in that it comprises:

One or more processors; memory; multiple application programs; and one or more programs, wherein the one or more programs are stored in the memory, when the one or more programs are used by the processor When executed, the electronic device is caused to execute the following steps:

In response to determining that a first face photo is obtained, wherein the first face photo is obtained in response to a user's photographing operation; and it is determined that the first face photo is obtained in a low-light shooting environment, then according to the first face photo A face photo, a second face photo is determined from at least one saved photo, wherein the second face photo and the first face photo include facial feature information of the same user, and the second face photo The face photo is a photo with the best facial feature information among the at least one photo;

When the face feature information of the first region of the second face photo is better than the face feature information of the first region corresponding to the first face photo, all the features of the second face photo are used. The face feature information of the first region is fused with the face feature information of the first region of the first face photo, or replaced with the face feature information of the first region of the second face photo Obtaining face feature information of the first region of the first face photo to obtain a third face photo, where the first region is part or all of the user's face;

Instruct to save the third face photo to the local photo album.
The electronic device according to claim 7, wherein when the one or more programs are executed by the processor, the electronic device is caused to perform the following steps:

When the face feature information of the first region of the first face photo is better than the face feature information of the first region corresponding to the second face photo, instruct to save the first face photo.
The electronic device according to claim 7 or 8, wherein the facial feature information of the first area includes at least one of contrast information, average gradient information, and entropy information of pixels in the first area. Kind.
The electronic device according to any one of claims 7 to 9, wherein before the instruction to save the third face photo to a local album, when the one or more programs are executed by the processor When the time, the electronic device is caused to perform the following steps:

Obtain information about the current low-light shooting environment;

Perform an image processing operation on the third face photo according to the current information of the low-light shooting environment, and the image processing operation includes at least one of color temperature processing, color processing, and white balance processing.
The electronic device according to any one of claims 7 to 10, wherein:

The obtaining of determining to be in a low-light shooting environment specifically includes:

When the brightness of the ambient light collected by the ambient light brightness sensor of the electronic device is lower than the preset brightness threshold, it is determined that it is in a dark light shooting environment; and/or

When the flash or soft light is turned on, the electronic device is determined to be in a low-light shooting environment.
The electronic device according to any one of claims 7 to 11, wherein the electronic device is a chip.
A computer storage medium, characterized by comprising computer instructions, when the computer instructions are run on an electronic device, the electronic device is caused to perform the photographing in the dark according to any one of claims 1 to 6 method.
A computer program product, characterized in that, when the computer program product runs on a computer, the computer is caused to execute the method for photographing in dark light according to any one of claims 1 to 6.