WO2024040981A1 - 拍照方法及其相关设备 - Google Patents
拍照方法及其相关设备 Download PDFInfo
- Publication number
- WO2024040981A1 WO2024040981A1 PCT/CN2023/087489 CN2023087489W WO2024040981A1 WO 2024040981 A1 WO2024040981 A1 WO 2024040981A1 CN 2023087489 W CN2023087489 W CN 2023087489W WO 2024040981 A1 WO2024040981 A1 WO 2024040981A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- algorithm
- processing
- frames
- photographing
- level
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 202
- 238000012545 processing Methods 0.000 claims abstract description 591
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 577
- 230000015654 memory Effects 0.000 claims abstract description 336
- 230000008569 process Effects 0.000 claims description 103
- 238000005286 illumination Methods 0.000 claims description 99
- 238000001514 detection method Methods 0.000 claims description 72
- 230000004044 response Effects 0.000 claims description 53
- 230000007423 decrease Effects 0.000 claims description 20
- 238000004590 computer program Methods 0.000 claims description 14
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims description 12
- 230000007613 environmental effect Effects 0.000 claims description 7
- FFBHFFJDDLITSX-UHFFFAOYSA-N benzyl N-[2-hydroxy-4-(3-oxomorpholin-4-yl)phenyl]carbamate Chemical compound OC1=C(NC(=O)OCC2=CC=CC=C2)C=CC(=C1)N1CCOCC1=O FFBHFFJDDLITSX-UHFFFAOYSA-N 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 25
- 230000006870 function Effects 0.000 description 22
- 238000012937 correction Methods 0.000 description 14
- 238000007726 management method Methods 0.000 description 12
- 238000013507 mapping Methods 0.000 description 11
- 210000000988 bone and bone Anatomy 0.000 description 10
- 230000000694 effects Effects 0.000 description 9
- 230000009467 reduction Effects 0.000 description 8
- 239000003086 colorant Substances 0.000 description 6
- 230000035945 sensitivity Effects 0.000 description 6
- 230000008859 change Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000003384 imaging method Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 5
- 229920001621 AMOLED Polymers 0.000 description 4
- 238000009825 accumulation Methods 0.000 description 4
- 238000013500 data storage Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000006467 substitution reaction Methods 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 230000001133 acceleration Effects 0.000 description 3
- 230000006399 behavior Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 238000001914 filtration Methods 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 2
- 238000003705 background correction Methods 0.000 description 2
- 238000010009 beating Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000036772 blood pressure Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000013021 overheating Methods 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 238000003825 pressing Methods 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 230000001755 vocal effect Effects 0.000 description 2
- 241000023320 Luma <angiosperm> Species 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000002146 bilateral effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000007429 general method Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 230000003862 health status Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 239000010985 leather Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- OSWPMRLSEDHDFF-UHFFFAOYSA-N methyl salicylate Chemical compound COC(=O)C1=CC=CC=C1O OSWPMRLSEDHDFF-UHFFFAOYSA-N 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 239000002096 quantum dot Substances 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 229920006395 saturated elastomer Polymers 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000006641 stabilisation Effects 0.000 description 1
- 238000011105 stabilization Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/95—Computational photography systems, e.g. light-field imaging systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Definitions
- the present application relates to the field of image technology, specifically, to a photographing method and related equipment.
- This application provides a photographing method and related equipment. By distinguishing memory usage and adaptively selecting processing algorithms with different processing durations and/or processing frame numbers, the pressure on memory increments can be reduced and continuous processing can be achieved. Take a quick photo.
- a photographing method is provided, which is applied to an electronic device including a camera.
- the method includes:
- the x-level processing algorithm is determined to be the first photographing algorithm, and the number of x-level processing frames is determined to be the first frame number, and x is an integer greater than 1;
- the x-level processing algorithm is determined to be the second photographing algorithm, and the x-level processing frame number is determined to be the second frame. number;
- the x-level processing algorithm is determined to be the third photographing algorithm, and the x-level processing frame number is determined to be the third frame number, and the first memory threshold is less than the second memory threshold;
- the first photographing algorithm, the second photographing algorithm and the third photographing algorithm are the same, the first frame number, the second frame number and the third frame number gradually decrease;
- the first frame number, the second frame number and the third frame number are the same or slowing shrieking.
- the first operation is a click operation.
- the first operation may also include a voice instruction operation or other operation of instructing the electronic device to take pictures.
- Embodiments of the present application provide a photographing method.
- the electronic device uses a camera to collect an original image, and then determines the size of the memory occupancy and determines the size of the memory occupancy according to the difference in the size of the memory occupancy.
- adaptively select algorithms with different processing times for processing For example, when the memory footprint is small, an algorithm with a longer processing time is selected for processing, and when the memory footprint gradually increases, an algorithm with a shorter processing time is selected for processing. This can reduce the incremental pressure on the memory in the electronic device, improve the photo processing efficiency, respond to the user's photo needs in a timely manner, and achieve continuous and fast photo taking.
- the method further includes:
- a single frame algorithm is used to process 1 frame of the original image to obtain the corresponding captured image
- the processing time of the single frame algorithm is shorter than the processing time of the third photographing algorithm
- the processing time of the single frame algorithm is the same as the processing time of the third photography algorithm, or, The processing time of the single frame algorithm is shorter than the processing time of the third photographing algorithm.
- the device temperature refers to the temperature inside the electronic device.
- the device temperature when the device temperature is higher than the temperature threshold, it can be switched to a single-frame algorithm to process one frame of original image, which can minimize the workload of the electronic device, slow down the increase in heat, and reduce the increase in heat. It can reduce the amount of pressure, so that the electronic device can take more pictures before it cannot work, so as to meet the user's photography needs.
- the single-frame algorithm can be processed only in the image signal processor, the processing speed of the single-frame algorithm is relatively fast and the processing time is relatively short.
- the method further includes:
- the first-level processing algorithm is used to process the first-level processing frame number in the original image collected corresponding to the number of photos taken for the first time to obtain the corresponding captured image.
- this application can determine different scenes by performing environmental detection of the scene to be shot, and then select different algorithms based on the different subdivided scenes, and select different number of frames from the corresponding collected original images. To perform processing, it is possible to adaptively improve the quality and effect of the captured images corresponding to the first number of photos taken in each scenario.
- the environment detection at least includes illumination detection and dynamic range detection, and the environment detection results at least include illumination and dynamic range values;
- Determine the number of level 1 processing frames and level 1 processing algorithm corresponding to the first number of photos according to the environment detection results include:
- the illumination is less than the illumination threshold and the dynamic range value is greater than or equal to the dynamic range threshold, determine the number of level 1 processing frames to be a1 frames, and determine the level 1 processing algorithm to be the A1 algorithm;
- the illumination is less than the illumination threshold and the dynamic range value is less than the dynamic range threshold, determine the number of level 1 processing frames to be b1 frames, and determine the level 1 processing algorithm to be the B1 algorithm;
- the number of level 1 processing frames is determined to be c1 frames, and the level 1 processing algorithm is determined to be the C1 algorithm.
- the illumination is greater than or equal to the illumination threshold and the dynamic range value is less than the dynamic range threshold, determine the number of level 1 processing frames to be d1 frames, and determine the level 1 processing algorithm to be the D1 algorithm;
- a1, b1, c1 and d1 are all integers greater than or equal to 1.
- the a1 frame, b1 frame, c1 frame or d1 frame can be selected from all frames of the original image in the order of acquisition, or the a1 frame, b1 frame, c1 frame or d1 frame can be arbitrarily extracted from all the frames of the original image. d1 frame.
- the level 1 processing algorithm is used to process the number of level 1 processing frames in the original image collected corresponding to the number of photos taken for the first time to obtain the corresponding captured image.
- the A1 algorithm is used to process the a1 frame in the original image collected corresponding to the first number of shots to obtain the corresponding shot image;
- the B1 algorithm is used to process the b1 frame in the original image collected corresponding to the first number of photos to obtain the corresponding captured images;
- the C1 algorithm is used to perform the C1 frame in the original image collected corresponding to the first number of times of photography. Process to obtain the corresponding captured image;
- the D1 algorithm When the illumination is greater than or equal to the illumination threshold and the dynamic range value is less than the dynamic range threshold, use the D1 algorithm to process the d1 frame in the original image collected corresponding to the first number of shots, Get the corresponding captured image.
- the first number of pictures corresponds to the frame a1 in the original image collected, and/or the number of pictures taken for the first time corresponds to the frame c1 in the original image collected, It includes at least one of a long-exposure original image, a normal-exposure original image, and a short-exposure original image.
- the dynamic range and image details of the image can be adjusted so that the content of the obtained captured image is more realistic and the image quality is relatively better.
- the method further includes:
- the x-level processing algorithms corresponding to the number of photos taken waiting in the background are respectively used as the initial processing algorithms
- the target processing algorithm is used to process the number of x-level processing frames in the original image corresponding to the arbitrary number of photography times to obtain the corresponding captured image.
- the processing algorithm for the number of waiting pictures is adjusted accordingly, which can control the degree of memory accumulation and reduce memory. occupancy to achieve continuous and rapid photography.
- the first interface refers to a photo-taking interface
- the first control refers to a control for instructing to take photos
- the first interface refers to a video recording interface
- the first control refers to a control used to instruct a capture
- an image processing device which device includes a unit for performing each step in the above first aspect or any possible implementation of the first aspect.
- an electronic device including one or more processors and memories;
- the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors invoke the computer instructions to cause the The electronic device performs the method provided in the above first aspect or any possible implementation of the first aspect.
- a chip is provided.
- the chip is applied to an electronic device.
- the chip includes one or more processors.
- the processor is used to call computer instructions to cause the electronic device to execute the first aspect or the first aspect.
- a computer-readable storage medium stores computer program code.
- the computer program code When the computer program code is run by an electronic device, it causes the electronic device to execute the first aspect or the third aspect.
- a computer program product includes: computer program code.
- the electronic device causes the electronic device to execute the first aspect or any of the first aspects. Methods provided in Possible Implementations.
- Embodiments of the present application provide a photographing method and related equipment.
- the electronic device uses a camera to collect an original image.
- environment detection is performed on the scene to be shot, and then the number of first-level processing frames and the first-level processing algorithm corresponding to the first number of photos taken are determined based on the environment detection results.
- the memory usage is first detected, and the determined memory usage is compared with the preset memory threshold, and then, based on different Memory usage size, choose different processing frames and processing algorithms.
- Figure 1 is a schematic diagram of a scenario applicable to the embodiment of the present application.
- Figure 2 is a schematic diagram of another scenario applicable to the embodiment of the present application.
- Figure 3 is a schematic flow chart of a photographing method provided by an embodiment of the present application.
- Figure 4 is a schematic interface diagram of a click operation provided by an embodiment of the present application.
- Figure 5 is a schematic diagram of another click operation interface provided by an embodiment of the present application.
- Figure 6 is a schematic diagram of an image processing process provided by an embodiment of the present application.
- Figure 7 is a schematic flow chart of an environment detection provided by an embodiment of the present application.
- Figure 8 is a form provided by the embodiment of this application.
- Figure 9 is a schematic flowchart of determining the corresponding frame number and algorithm according to the memory usage provided by the embodiment of the present application.
- Figure 10 is a schematic diagram of memory usage growth provided by an embodiment of the present application.
- Figure 11 is a schematic flow chart of another photographing method provided by an embodiment of the present application.
- Figure 12 is a schematic flow chart of yet another photographing method provided by an embodiment of the present application.
- Figure 13 is a schematic flow chart of yet another photographing method provided by an embodiment of the present application.
- Figure 14 is another form provided by the embodiment of this application.
- Figure 15 is another form provided by the embodiment of this application.
- Figure 16 is another form provided by the embodiment of the present application.
- Figure 17 is a schematic diagram of the processing process corresponding to Figure 16;
- Figure 18 is a schematic diagram of a display interface of an electronic device provided by an embodiment of the present application.
- Figure 19 is a schematic diagram of a display interface of another electronic device provided by an embodiment of the present application.
- Figure 20 is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application.
- Figure 21 is a schematic diagram of the software structure of an electronic device provided by an embodiment of the present application.
- Figure 22 is a schematic structural diagram of an image processing device provided by an embodiment of the present application.
- Figure 23 is a schematic structural diagram of a chip provided by an embodiment of the present application.
- first and second are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of indicated technical features. Therefore, features defined as “first” and “second” may explicitly or implicitly include one or more of these features.
- RGB (red, green, blue) color space refers to a color model related to the structure of the human visual system. Think of all colors as different combinations of red, green, and blue, based on the structure of the human eye.
- YUV color space refers to a color encoding method. Y represents brightness, and U and V represent chroma. The above-mentioned RGB color space focuses on the human eye's perception of color, while the YUV color space focuses on the visual sensitivity to brightness. The RGB color space and the YUV color space can be converted into each other.
- Pixel value refers to a set of color components corresponding to each pixel in a color image located in the RGB color space.
- each pixel corresponds to a set of three primary color components, where the three primary color components are the red component R, the green component G, and the blue component B respectively.
- Bayer pattern color filter array When the image is converted from the actual scene into image data, the image sensor usually receives the red channel signal, the green channel signal and the blue channel signal respectively. information of three channel signals, and then synthesize the information of the three channel signals into a color image. However, in this scheme, three filters are required for each pixel position, which is expensive and difficult to produce. Therefore, it can be used on the image sensor. The surface is covered with a color filter array to obtain information from the three channel signals.
- Bayer format color filter array refers to filters arranged in a checkerboard format. For example, the minimum repeating unit in the Bayer format color filter array is: one filter to obtain the red channel signal, two filters to obtain the green channel signal, and one filter to obtain the blue channel signal, arranged in a 2 ⁇ 2 manner. cloth.
- Bayer image that is, the image output by the image sensor based on the Bayer format color filter array. Pixels of multiple colors in this image are arranged in a Bayer format. Among them, each pixel in the Bayer format image only corresponds to the channel signal of one color. For example, since human vision is more sensitive to green, it can be set that green pixels (pixels corresponding to green channel signals) account for 50% of all pixels, blue pixels (pixels corresponding to blue channel signals) and red pixels (pixels corresponding to the red channel signal) each account for 25% of all pixels. Among them, the smallest repeating unit of the Bayer format image is: one red pixel, two green pixels and one blue pixel arranged in a 2 ⁇ 2 manner. Bayer format images are images located in the RAW domain.
- Shooting parameters may include shutter, exposure time, aperture value (AV), exposure value (EV) and sensitivity ISO. They are introduced separately below.
- the shutter is a device that controls the length of time light enters the camera to determine the image exposure time. The longer the shutter remains open, the more light enters the camera and the corresponding exposure time of the image is longer. Conversely, the shorter the shutter remains open, the less light enters the camera and the corresponding exposure time of the image is shorter.
- Exposure time refers to the time the shutter has to be open in order to project light onto the photosensitive surface of the camera's photosensitive material.
- the exposure time is determined by the sensitivity of the photosensitive material and the illumination on the photosensitive surface. The longer the exposure time, the more light enters the camera, the shorter the exposure time, the less light enters the camera. Therefore, a long exposure time is required in dark light scenes, and a short exposure time is required in backlight scenes.
- the aperture value is the ratio of the focal length of the lens in the camera to the diameter of the lens. The larger the aperture value, the more light enters the camera. The smaller the aperture value, the less light enters the camera.
- Exposure value is a value that combines exposure time and aperture value to represent the light-passing ability of the camera lens. Exposure value can be defined as:
- N is the aperture value
- t is the exposure time, in seconds.
- ISO is a measure of how sensitive a film is to light, that is, sensitivity or gain.
- sensitivity or gain For insensitive films, it is necessary to A longer exposure time is required to achieve the same brightness as the sensitive film.
- shutter, exposure time, aperture value, exposure value and ISO electronic equipment can realize automatic focus (AF), automatic exposure (AE), automatic white balance (AWB) through algorithms ) to achieve automatic adjustment of these shooting parameters.
- AF automatic focus
- AE automatic exposure
- AVB automatic white balance
- Autofocus means that electronic equipment obtains the highest image frequency component by adjusting the position of the focusing lens to obtain higher image contrast.
- focusing is a continuous accumulation process.
- the electronic device compares the contrast of the images taken by the lens at different positions to obtain the position of the lens when the contrast of the image is maximum, and then determines the focal length of the focus.
- Automatic exposure is when electronic devices automatically set exposure values based on available light source conditions.
- the electronic device can automatically set the shutter speed and aperture value based on the exposure value of the currently captured image to achieve automatic setting of the exposure value.
- the color of the object will change due to the color of the projected light.
- the images collected by the electronic device under different light colors will have different color temperatures.
- White balance is closely related to ambient light. Regardless of the ambient light, the electronic device's camera can recognize white and use white as a basis to restore other colors.
- Automatic white balance enables electronic devices to adjust the fidelity of image color based on light source conditions. 3A stands for autofocus, autoexposure and autowhite balance.
- the exposure value can be any one of -24, -4, -3, -2, -1, 0, 1, 2, 3, 4, and 24.
- the exposure image corresponding to EV0 is used to indicate the exposure image captured by the determined exposure value 0 when the electronic device implements exposure through the algorithm.
- the exposure image corresponding to EV-2 is used to indicate the exposure image captured by the determined exposure value -2 when the electronic device implements exposure through the algorithm.
- the exposure image corresponding to EV1 is used to indicate the exposure image captured by the determined exposure value 1 when the electronic device implements exposure through the algorithm. Others are deduced in turn and will not be described again here.
- every increase in the exposure value by 1 will change the exposure by one level, that is, the exposure amount (referring to the integration of the illumination received by a certain surface element on the object surface during time t) will be doubled, such as doubling the exposure time or aperture area. Then, an increase in exposure value will correspond to a slower shutter speed and smaller f-number. It can be seen that compared to EV-2, the exposure value of EV0 increases by 2, changing the exposure by two stops; similarly, compared with EV0, the exposure value of EV1 increases by 1, changing the exposure by one stop.
- the exposure value EV when the exposure value EV is equal to 0, the exposure value is usually the best exposure value under the current lighting conditions.
- the exposure image correspondingly obtained by the electronic device under the condition of EV0 is the best exposure image under the current lighting conditions.
- the best exposure image may also be called a reference exposure image.
- the "optimal" exposure image refers to the exposure image determined by an algorithm for a given electronic device.
- the algorithm is different, or the current lighting conditions are different, the optimal exposure image determined is different.
- the electronic device When the electronic device performs photographing processing, the electronic device will perform the next photographing process only after the last photographing processing is completed. Based on this single-threaded processing process, if the user triggers multiple photo-taking commands in succession and the interval between triggering photos is less than the above 2.5s, the electronic device will not be able to respond in time because each photo-taking algorithm takes a long time to process the photo. , causing the number of photos waiting in the background to gradually increase. As the number of waiting photos increases, more and more data is accumulated and stored in the background to be processed, which continuously expands the memory footprint. When the remaining memory is not enough to support photo processing, the background will no longer respond to user triggers. photo command.
- embodiments of the present application provide a photographing method.
- the electronic device uses a camera to collect an original image, and then determines the size of the memory occupancy and determines the size of the memory occupancy according to the memory occupancy.
- adaptively select algorithms with different processing times for processing For example, when the memory footprint is small, an algorithm with a longer processing time is selected for processing, and when the memory footprint gradually increases, an algorithm with a shorter processing time is selected.
- Algorithms are used for processing, which can reduce the incremental pressure on memory in electronic devices, improve the efficiency of photo processing, respond to users' photo needs in a timely manner, and achieve continuous and rapid photo taking.
- the photographing method provided by the embodiment of the present application can be applied to various electronic devices.
- the electronic device may be various imaging devices such as action cameras and digital cameras, mobile phones, tablets, wearable devices, vehicle-mounted devices, augmented reality (AR)/virtual reality (VR), etc. reality, VR) equipment, notebook computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (personal digital assistants, PDA), etc., or other equipment or devices capable of image display,
- imaging devices such as action cameras and digital cameras, mobile phones, tablets, wearable devices, vehicle-mounted devices, augmented reality (AR)/virtual reality (VR), etc. reality, VR) equipment, notebook computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (personal digital assistants, PDA), etc., or other equipment or devices capable of image display,
- AR augmented reality
- VR virtual reality
- VR VR
- notebook computers notebook computers
- netbooks personal digital assistants
- PDA personal digital assistants, PDA
- PDA personal digital assistants, PDA
- PDA personal digital assistants
- a mobile phone may include a display, one or more cameras. Among them, the mobile phone uses one or more cameras to shoot in a scene to be photographed to obtain captured images with different characteristics.
- the display screen is used to display the captured images processed after shooting.
- the scene to be photographed refers to the scene that the user expects to be photographed. If the user uses the camera of the electronic device to point at a scene including a certain object, the scene including the certain object is the scene to be photographed. It should be understood that the scene to be photographed does not refer to a specific scene, but the scene that is aimed at in real time following the direction of the camera.
- Figure 1 shows a schematic diagram of a scenario to which the embodiment of the present application is applicable
- Figure 2 shows a schematic diagram of another scenario to which the embodiment of the present application is applicable.
- a camera application is installed on the electronic device 100 .
- a variety of applications are also installed, and the embodiments of this application do not impose any restrictions on this.
- the electronic device 100 displays a shooting interface as shown in (b) of FIG. 1 .
- the shooting interface includes multiple shooting modes of the camera application, such as large aperture mode 41, night scene mode 42. Portrait mode 43, photo mode 44, video mode 45, etc.
- the shooting interface also includes a first control.
- the first control is a shooting key 50 .
- the shooting key 50 is used to indicate the current shooting mode. For example, when the camera is turned on, the shooting key 50 indicates that the current shooting mode is the shooting mode 44 by default. .
- the shooting interface also includes a viewfinder window 60 , which can be used to display a preview image before taking a picture in real time.
- a second control is also displayed in the shooting interface, and the second control is zoom option 61.
- the user can select the currently required zoom factor in the zoom option 61, for example, 0.5x, 2x or 50x zoom factor, etc.
- the viewfinder window 60 can be displayed in real time. Preview image before taking a photo; then, in response to the user's multiple consecutive click operations on the shooting key 50, the electronic device can call the photo-taking method provided by this application, and adaptively select different photo-taking algorithms for processing according to the size of the memory, so that it can Reduce the pressure on memory increment, improve the efficiency of photo processing, respond to users' photo needs in a timely manner, achieve fast photo taking, and avoid the situation where the background cannot respond to photo commands in related technologies.
- Application scenario 2 Scene captured in video
- a camera application is installed on the electronic device 100 .
- the electronic device 100 displays a shooting interface as shown in (b) of FIG. 2 .
- the description of the shooting interface is the same as the description of (b) in FIG. 1 , and will not be described again here.
- the default shooting key 50 indicates that the current shooting mode is the shooting mode 44 .
- the shooting mode can be switched from the photographing mode 44 to the video recording mode 45 .
- the viewfinder window 60 included in the shooting interface can be used to display the preview image before recording in real time.
- the electronic device 100 may display a recording interface as shown in (d) of Figure 2.
- the video recording interface can display the currently shot video image (the man walking with a shopping bag as shown in the picture) and the shooting progress (00 as shown in the picture: 12), status icons (the shooting icon in front of the shooting progress in the figure is used to indicate the current shooting state), and a third control, such as the capture control 70.
- the shooting interface can also include other controls, such as zoom options. 61. End control 80, pause/continue control 90, etc., which are not limited in the embodiment of the present application.
- the pause/continue control 90 is used to display a pause icon during the video shooting process (refer to (d) in Figure 2), and is also used to pause the current video shooting process when the pause icon is clicked; and when the video shooting is paused
- the shooting icon is displayed when the user clicks the shooting icon to continue the current video shooting process (not shown in Figure 2).
- the end control 80 is used to end the current video shooting process.
- the capture control 70 is used to capture photos without pausing the current video capture process and without ending the current video capture process.
- the user when the user wants to capture a photo of the man while photographing a man walking with a shopping bag, the user can click the capture control 70 in the shooting interface to obtain Snap a photo.
- the captured photo is stored as an image in the gallery of the electronic device. If the user clicks the capture control 70 in the video recording interface multiple times in succession, the electronic device can call the photo-taking method provided by this application and adaptively select different photo-taking algorithms for processing according to the size of the memory, thereby reducing the pressure on the memory increment. , improve the efficiency of snapshot processing, and Respond to the user's photography needs in real time, achieve quick capture, and avoid the situation where the backend cannot respond to photography commands in related technologies.
- FIG 3 is a schematic flow chart of a photographing method provided by this application.
- the photographing method is applied to electronic devices including cameras, such as mobile phones.
- the steps of the method may include the following S11 to S19.
- the electronic device displays a first interface, and the first interface includes a first control.
- the first interface may be a shooting interface
- the first control is used to indicate the shooting key 50 in the shooting interface.
- the first interface may be a video recording interface
- the first control is used to indicate the capture control 70 in the video recording interface.
- the first interface can also be other interfaces, and the first control can accordingly be a control on other interfaces used to instruct taking pictures, and the embodiments of the present application do not impose any restrictions on this.
- the electronic device uses the camera to collect an original image.
- the first operation may be a clicking operation on the first control, or may be a voice instruction operation or other operation of instructing the electronic device to take a photo, and the embodiment of the present application does not impose any limitation on this.
- the click operation refers to the behavior of the user touching the first control for a short period of time and then leaving.
- Figure 4 shows a schematic interface diagram of a click operation.
- the user's finger presses the shooting key 50 and then lifts it up, which is recorded as a click operation.
- the time between pressing the shooting key 50 and lifting your finger away from the shooting key 50 is the duration of one click operation, which is usually very short.
- FIG. 5 shows another schematic interface diagram of a click operation. As shown in (a) and (b) in FIG. The time between pressing the finger on the capture control 70 and lifting the finger away from the capture control 70 is the duration of one click operation.
- the electronic device uses the camera to capture an original image.
- the electronic device can use the camera to collect multiple original images.
- the first click operation corresponds to the first number of pictures
- the second click operation corresponds to the second number of pictures
- the third click operation corresponds to the third time. The number of photos taken corresponds to each other, and so on, and will not be repeated again.
- the first preset interval duration is at least longer than the processing duration of one photo taking, and the specific duration of the first preset interval duration can be set and modified as needed, and the embodiments of the present application do not impose any restrictions on this.
- the electronic device uses the camera to collect an original image, and the number of times of taking pictures corresponds to the first time. Number of photos taken. The user then closes and reopens the camera and performs another click on the capture key 50 In response to the current click operation, the electronic device uses the camera to collect the original image again. At this time, since the camera is reopened, the number of photos taken should be re-recorded as the first number of photos taken.
- Figure 6 provides a schematic diagram of the image processing process.
- the electronic device uses the camera to collect an original image, and the number of times of taking pictures can be recorded as the first click operation K1. 1 number of photos taken.
- the user performs the next click operation K1' on the shooting button 50.
- the electronic device uses the camera to collect the original image again.
- the electronic device uses the camera to collect the original image again, because the interval between the click operation K2' and the click operation K1' is less than If the first preset interval is longer, the number of photos taken can be recorded as the second number of photos taken.
- the electronic device uses the camera to collect the original image again, because the interval between the click operation K3' and the click operation K2' is less than For the first preset interval time, the number of photos taken can be recorded as the third number of photos taken; and so on, which will not be described again here.
- one or more cameras can be used to collect original images, and each camera can collect one or more frames of original images. That is to say, one or more frames of original images can be collected at a time, and the specific number of frames can be set and modified as needed.
- the embodiments of the present application do not impose any restrictions on this.
- the original image may be an image located in the RAW domain.
- the original image when the camera is a black and white camera, the original image may be a grayscale image; when the camera is a multispectral camera, the original image may be a signal including multiple color channels.
- the format or characteristics of the original image change as the camera changes, and the embodiments of the present application do not impose any restrictions on this.
- the original images collected at one time may also include at least one of a long-exposure original image, a normal-exposure original image, and a short-exposure original image.
- the original image with long exposure refers to the image obtained after a longer exposure time when shooting
- the original image with short exposure refers to the image obtained after a shorter exposure time during shooting.
- “long” and “ “Short” is relative to the "normal” exposure time. When the exposure time corresponding to the normal exposure of the original image is different, the long exposure and short exposure will also change accordingly.
- the multi-frame original image includes a long-exposure original image, and/or a normal-exposure original image, and/or a short-exposure original image
- the number of frames of the long-exposure original image, the number of frames of the normal-exposure original image, and The number of frames of the short-exposure original image can be selected and modified as needed, and the embodiments of the present application do not impose any restrictions on this.
- the environment detection at least includes illuminance (lightness value, LV) detection and dynamic range detection.
- the environment detection can also include detection items, which can be set and modified as needed. The embodiments of this application do not impose any restrictions on this.
- illumination detection refers to estimating the ambient brightness in the scene to be photographed.
- Illumination refers to the degree to which an object is illuminated, specifically the energy of visible light received by the object per unit area, referred to as illuminance, and the unit is Lux. Streets in the dark, dimly lit indoors, etc. generally have low illumination. In contrast, playgrounds in the sun, stages under spotlights, etc. can be called medium to high illumination. If the illumination of the scene to be photographed is relatively low, when using electronic equipment to photograph the scene to be photographed, it will be blurry and the effect will be poor. If the illumination of the scene to be photographed is relatively high, it will be clearer and the shooting effect will be better when using electronic equipment to photograph the scene to be photographed.
- a photoelectric sensor can be provided in the electronic device, and the photoelectric sensor can be used to detect the illumination of the scene to be photographed.
- the illumination can be calculated using the following formula based on the exposure parameters of the camera, such as exposure time, sensitivity, aperture and other parameters, as well as the received response value:
- Exposure is the exposure time
- Aperture is the aperture size
- ISO is the sensitivity
- Luma is the average value of Y in the XYZ color space.
- the calculated illumination value is also greater.
- dynamic range detection refers to detecting the dynamic range value of the scene to be shot.
- the dynamic range value is used to represent the brightness range of the pixels in the image, that is, the number of gray levels in the image from the "brightest” pixels to the “darkest” pixels.
- the larger the dynamic range value of an image the richer the brightness levels it can represent, and the more realistic the visual effect of the image.
- Expressions for dynamic range values can be:
- dynamic range is the dynamic range value
- bright is the brightness of the "brightest” pixel
- dark is the brightness of the "darkest” pixel.
- the unit of dynamic range is stop.
- a low dynamic range scene refers to a scene in which the ambient light intensity is all low or all high, and the dynamic range is relatively narrow
- a high dynamic range scene refers to a part of the light intensity that is low, and the other part of the light intensity is high, and the dynamic range is relatively wide. scene.
- Figure 7 is a schematic flow chart of an environment detection provided by an embodiment of the present application.
- the above-mentioned S13 includes: when the first operation corresponds to the first number of photos, environmental detection is performed on the scene to be photographed.
- the environmental detection includes illumination detection. Therefore, accordingly, the environmental detection result includes the detected illumination; similarly, since environment detection also includes dynamic range detection, accordingly, the environment detection result also includes the detected dynamic range value.
- the methods of illumination detection and dynamic range detection can refer to the above description, and will not be described again here.
- S14 can include S141 to S146 below.
- the illumination threshold can be set and modified as needed, and the embodiments of the present application do not impose any restrictions on this.
- the dynamic range threshold can be set and modified as needed, and the embodiments of the present application do not impose any restrictions on this.
- a1 is all the frames or part of the frames in the original image collected corresponding to the first number of shots.
- b1 is all the frames or part of the frames in the original image collected corresponding to the first number of shots.
- c1 is all the frames or part of the frames in the original image collected corresponding to the first number of shots.
- d1 is all the frames or part of the frames in the original image collected corresponding to the first number of shots.
- A1, B1, C1 and D1 are used to refer to the names of algorithms.
- A1 algorithm, B1 algorithm, C1 algorithm and D1 algorithm can indicate the same algorithm or different algorithms. They can be set and modified as needed. , the embodiments of this application do not impose any restrictions on this.
- a1, b1, c1 and d1 are used to refer to the number of processing frames.
- a1, b1, c1 and d1 can indicate the same frame number or different frame numbers.
- the specific frame number can be set and modified as needed. The embodiments of this application do not impose any limitation on this.
- the a1 frame, b1 frame, c1 frame or d1 frame can be selected from all the frames of the original image according to the acquisition order, or the a1 frame, b1 frame, c1 frame or d1 frame can be arbitrarily extracted from all the frames of the original image. d1 frame for processing, or you can also select the images collected by one or several cameras by specifying the camera to process as a1 frame, b1 frame, c1 frame or d1 frame.
- the methods of selecting the a1 frame, b1 frame, c1 frame or d1 frame may be the same or different. The specific settings and modifications may be made as needed, and the embodiments of the present application do not impose any restrictions on this.
- the determined a1 frame or c1 frame may include at least one of a long-exposure original image, a normal-exposure original image, and a short-exposure original image, or may additionally Adding at least one of a long-exposure original image, a normal-exposure original image, and a short-exposure original image can be set and modified as needed, and the embodiments of the present application do not impose any restrictions on this.
- the dynamic range and image details of the image can be adjusted so that the content of the obtained captured image is more realistic and the image quality is relatively better.
- the determined A1 algorithm is used to process the a1 frame in the original image collected corresponding to the first number of shots, and the corresponding of captured images.
- the determined B1 algorithm is used to process the b1 frame in the original image collected corresponding to the first number of photos to obtain the corresponding captured image.
- the determined C1 algorithm is used to process the c1 frame in the original image collected corresponding to the first number of shots to obtain the corresponding captured image.
- the determined D1 algorithm is used to process the d1 frame in the original image collected corresponding to the first number of shots to obtain the corresponding captured image.
- the corresponding original image is collected once, and a total of 9 frames of original images are collected this time.
- the illumination is less than the illumination threshold and the dynamic range value is greater than or equal to the dynamic range threshold, use determine
- this application can determine different scenes by performing environmental detection on the scene to be shot, and then select different algorithms based on the different subdivided scenes, and select different number of frames from the corresponding collected original images for processing. , which can adaptively improve the quality and effect of the captured images corresponding to the first number of photos taken in each scenario.
- memory usage refers to the amount of data stored in memory. Because every time the camera collects a raw image or obtains a frame of captured image, it will increase part of the memory usage, that is, the memory usage will increase relatively.
- the memory usage will become larger and larger. In this way, when the memory usage reaches a certain level, the related technology may not be able to respond to the photo taking command. In order to avoid this situation, this application will monitor the memory usage in real time. For different memory usage, different processing algorithms and/or different processing frames can be set to reduce the pressure on memory increment. Among them, the memory usage can be obtained by obtaining data inside the electronic device.
- the x-level processing algorithm is used to refer to the algorithm used in the processing corresponding to the x-th number of photos taken
- the number of x-level processing frames is used to refer to the number of frames of the original image used in the processing corresponding to the x-th number of photos taken.
- the number of photos taken for the second time corresponds to the number of level 2 processing frames and the level 2 processing algorithm
- the number of photos taken for the third time corresponds to the number of level 3 processing frames and the level 3 processing algorithm
- subsequent processing frames subsequent processing frames
- FIG. 9 shows a schematic flow chart for determining the corresponding number of processing frames and processing algorithm according to the memory usage.
- the above-mentioned S17 may include the following S171 to S175.
- the first frame number refers to all frames or part of the original images collected corresponding to the x-th photographing number.
- the second number of frames refers to all the frames or part of the frames in the original image collected corresponding to the x-th photographing number.
- the third frame number refers to all the frames or some frames in the original image collected corresponding to the x-th photographing number.
- the first memory threshold is smaller than the second memory threshold, and the second memory threshold is smaller than the total amount of memory.
- the sizes of the first memory threshold and the second memory threshold can be set and modified as needed, and the embodiments of the present application do not impose any restrictions on this.
- first photographing algorithm the second photographing algorithm and the third photographing algorithm may be the same or different.
- the first photographing algorithm, the second photographing algorithm and the third photographing algorithm are the same, the first frame number, the second frame number and the third frame number gradually decrease.
- the processing time of the first photographing algorithm, the second photographing algorithm and the third photographing algorithm gradually decreases.
- the first photographing algorithm The frame number, second frame number, and third frame number can be the same size, or progressively smaller.
- the decrease amount of the second frame number relative to the first frame number and the decrease amount of the third frame number relative to the second frame number can be the same or different, and can be carried out as needed.
- the embodiments of this application do not impose any restrictions on this.
- the processing steps included in the first photographing algorithm, the second photographing algorithm and the third photographing algorithm are gradually reduced, or the complexity of the processing steps included is gradually reduced, so that the processing time is gradually reduced.
- the processing time reduced by the second photographing algorithm relative to the first photographing algorithm and the processing time reduced by the third photographing algorithm relative to the second photographing algorithm can be the same or different. Specifically, they can be set and modified as needed. This application implements There are no restrictions on this.
- the first frame number, the second frame number or the third frame number can be selected for processing from all the frames in the original image collected with the corresponding number of shots according to the acquisition sequence, or you can also select from all the frames in the original image.
- the first, second or third frame number can be arbitrarily extracted from the frame for processing, or you can also select a camera by specifying it.
- the images collected by one or several cameras are processed as images of the first frame number, the second frame number or the third frame number.
- the methods for selecting the first frame number, the second frame number, or the third frame number may be the same or different. Specifically, they may be set and modified as needed, and the embodiments of the present application do not impose any restrictions on this.
- the selected original images of the first number of frames may include at least one of a long-exposure original image, a normal-exposure original image, and a short-exposure original image
- the selected original images of the second number of frames may include long-exposure original images. At least one of the original image with exposure, the original image with normal exposure, and the original image with short exposure.
- the selected original image of the third frame number may also include the original image with long exposure, the original image with normal exposure, and the original image with normal exposure. Short exposure raw image.
- the number of long-exposure original images and short-exposure original images included in the first frame number, the second frame number and the third frame number It can be reduced gradually. In this way, when the memory usage increases, the maintenance of the dynamic range will be given up to ensure that the user's need for fast photography is achieved as much as possible.
- the number of x-level processing frames corresponding to the x-th number of photos taken can be the same as the number of processing frames corresponding to the first number of photos taken, or it can be smaller than the number of processing frames corresponding to the first number of photos taken;
- the x-level processing algorithm can be the same as the level-1 processing algorithm corresponding to the first number of photos, or it can be shorter than the processing time of the level-1 processing algorithm corresponding to the first number of photos.
- the first photographing algorithm, the second photographing algorithm and the third photographing algorithm can be the same as the level 1 processing algorithm or have a shorter processing time than the level 1 processing algorithm.
- the first frame number, the second frame number and the third frame number gradually decrease, then the first frame number can be the same as the level 1 processing frame number or less than the level 1 processing frame number, then the second level processing frame number can be The number of frames and the number of third frames are both less than the number of frames processed in level 1.
- the first photographing algorithm may be the same as the level 1 processing algorithm corresponding to the first photographing number, or the first photographing algorithm may be shorter than the first photographing algorithm.
- the processing time of the level 1 processing algorithm corresponding to the number of photos taken is shorter.
- the processing time of the second photo taking algorithm and the third photo taking algorithm is shorter than that of the level 1 processing algorithm.
- the first frame number, the second frame number and the third frame number are the same, the first frame number, the second frame number and the third frame number are the same as or reduced from the level 1 processing frame number.
- the first frame number, the second frame number and the third frame number gradually decrease, the first frame number is the same as or less than the level 1 processing frame number. Then, the second frame number and the third frame number are smaller than the level 1 processing frame number. The number is smaller.
- Figure 10 shows a schematic diagram of the growth of memory usage.
- the memory usage corresponding to the second number of photos is determined. If the memory usage corresponding to the second number of photos taken is less than the first memory threshold, the level 2 processing algorithm is determined to be the first photo taking algorithm, and the number of level 2 processing frames is determined to be the first number of frames.
- the level 3 processing algorithm is determined to be the second photo taking algorithm, and Determine the level 3 processing frame number as the second frame number.
- the level 5 processing algorithm is determined to be the third photo taking algorithm, and the number of level 5 processing frames is determined to be the third number of frames. The following is analogous and will not be repeated here.
- the processing algorithm is the same, so the number of processing frames is gradually reduced, or the number of processing frames is the same, Then the processing time of the processing algorithm will be reduced; or, the processing time of the processing algorithm and the number of frames processed will be reduced. At this time, the growth rate of memory usage will slow down rapidly, reducing the memory growth in electronic devices. amount of pressure.
- this application can adjust the processing in time according to the memory usage.
- the algorithm and processing frame number will slow down and level off when the memory is larger, so that electronic devices can take more photos with the same memory size.
- the amount of memory corresponding to the first time of taking pictures is not shown in Figure 10.
- the original images collected for the first time of taking pictures are processed according to the above-mentioned environment detection results.
- the method of processing based on the memory usage results is applicable to the second and subsequent times of taking pictures.
- the first number of photos can also be processed in the same way as the second and subsequent number of photos, and the embodiments of the present application do not limit this.
- x determines the memory usage corresponding to the first number of photos taken.
- the level 1 processing algorithm determines the first photo taking algorithm, and determine the number of level 1 processing frames as The first number of frames
- determine the level 1 processing algorithm determines the second photography algorithm, and determine the number of level 1 processing frames as the second number of frames
- the level 1 processing algorithm is determined to be the third photography algorithm, and the number of level 1 processing frames is determined to be the third number of frames
- the level 1 processing algorithm is determined using the determined level 1 processing algorithm.
- the frame number is processed to obtain the corresponding captured image.
- Captured images can be stored in the gallery or displayed.
- one frame of captured images in response to one click operation, one frame of captured images can be obtained, and in response to multiple click operations, multiple frames of captured images can be obtained; the specific number of captured images is related to the number of user operations.
- the embodiment of the present application is This does not impose any restrictions.
- Embodiments of the present application provide a photographing method.
- the electronic device uses a camera to collect an original image.
- environment detection is performed on the scene to be shot, and then the number of first-level processing frames and the first-level processing algorithm corresponding to the first number of photos taken are determined based on the environment detection results.
- the memory usage is first detected, and the determined memory usage is compared with the preset memory threshold, and then, based on different Memory usage size, choose different processing frames and processing algorithms.
- the number of photos taken corresponds to the quality and effect of the captured images.
- the temperature of the electronic equipment will gradually rise. If rapid photos are taken continuously, the temperature of the device may rise exponentially. In this way, in the event of overheating, the electronic device will not be able to operate normally, and it will also be unable to respond to camera commands.
- embodiments of the present application can also detect the device temperature of the electronic device before processing the collected data. In response to overheating, the processing algorithm and the number of processing frames can be switched to reduce the workload and heat loss. Incremental pressure.
- Figure 11 shows a schematic flow chart of another photographing method. As shown in Figure 11, based on Figure 3, the method may also include the following S20 and S21.
- the device temperature refers to the temperature inside the electronic device.
- the electronic equipment will generate a certain amount of heat, which will cause the temperature of the equipment to increase.
- the temperature of the equipment can be collected through a temperature sensor inside the electronic equipment.
- the temperature threshold can be set and modified as needed, and the embodiments of this application do not impose any restrictions on this.
- the device temperature can be obtained by obtaining data inside the electronic device.
- the single-frame algorithm refers to the algorithm for processing one frame of original image.
- the single-frame algorithm is different from the level 1 processing algorithm.
- the processing time of the single-frame algorithm is compared with the level 1 processing algorithm (A1 algorithm, B1 algorithm, C1 algorithm). or D1 algorithm) is shorter.
- the single-frame algorithm can include dead pixel correction (default pixel correction, DPC), RAW domain noise reduction (raw domain noise filter, RAWNF), black level correction (black level correction, BLC), lens shading correction (lens shading) correction (LSC), automatic white balance, demosaicing, color correction (color correction matrix, CCM), YUV domain noise reduction, tone mapping (tone mapping), gamma (Gamma) correction, color space conversion, color enhancement (color enhancement) , CE) and other processing steps, the processing speed is fast and the time is short.
- DPC dead pixel correction
- RAW domain noise reduction raw domain noise filter
- RAWNF black level correction
- black level correction black level correction
- BLC black level correction
- LSC lens shading correction
- LSC lens shading correction
- demosaicing demosaicing
- color correction color correction matrix, CCM
- YUV domain noise reduction tone mapping (tone mapping), gamma (Gamma) correction
- color space conversion color enhancement (color enhancement) , CE)
- Dead pixel correction dead pixels are white points in the output image in a completely dark environment, and black points in the output image in a bright environment. Under normal circumstances, the three primary color channel signals should have a linear response relationship with the ambient brightness. However, due to poor signal output from the image sensor, white or black spots may appear. For this, dead pixels can be automatically detected and repaired, or, The dead pixel linked list performs fixed position repair of bad pixels. Among them, a point refers to a pixel.
- Noise reduction refers to the process of reducing noise in images.
- General methods include mean filtering, Gaussian filtering, bilateral filtering, etc.
- RAW domain noise reduction refers to the process of reducing noise in RAW domain images.
- YUV domain noise reduction refers to the process of reducing noise in YUV domain images.
- Black level correction is due to the existence of dark current in the image sensor, which causes the pixels to have a certain output voltage when there is no light. Moreover, pixels at different positions may correspond to different output voltages. Therefore, it is necessary to correct the black level correction when there is no light. The output voltage corresponding to the time (i.e., black) pixel is corrected.
- Lens shading correction can solve the problem of shadows appearing around the lens due to uneven refraction of light by the lens.
- Automatic white balance is to eliminate the influence of light source on image sensor imaging, simulate the color constancy of human vision, and ensure that the white seen in any scene is truly white. Therefore, it is necessary to correct the color temperature and automatically adjust the white balance. to the appropriate location.
- demosaic Since each pixel in the RAW domain image only corresponds to the color information of one channel, surrounding pixel information can be used to estimate other colors. For example, through linear interpolation, each missing pixel can be determined. The color information of the other two channels is used to recover all channel information of all pixels in the image.
- the process of demosaicing is equivalent to converting the image from the RAW domain to the RGB domain.
- demosaicing can also be called color interpolation.
- Tone mapping refers to using a spatial invariant mapping function to map all pixels in the image. That is to say, when tone mapping performs dynamic range transformation on an image, each pixel of the image uses the same transformation function, which is a one-to-one mapping relationship.
- tone mapping can also be called dynamic range compression (DRC).
- Gamma correction refers to editing the gamma curve of an image, using a non-linear tonal editing method to detect the dark and light parts of the image, and increase the two proportionally, thereby Improve image contrast effect.
- Color space conversion refers to converting an image from the RGB domain to the YUV domain.
- the image obtained after processing in the previous step can be converted from the RGB domain to the YUV domain to reduce the amount of subsequent data storage and transmission and save bandwidth.
- Color enhancement makes the original unsaturated color information saturated and rich.
- color enhancement may also be called color processing.
- the single-frame algorithm can also delete or add some other steps, and the level 1 processing algorithm (A1 algorithm, B1 algorithm, C1 algorithm or D1 algorithm) can include other steps on the basis of the single-frame algorithm.
- the level 1 processing algorithm (A1 algorithm, B1 algorithm, C1 algorithm or D1 algorithm) may also include different steps from the single frame algorithm, and the embodiments of the present application do not impose any restrictions on this.
- the single-frame algorithm can be processed only in the image signal processor, the processing speed of the single-frame algorithm is relatively fast and the processing time is relatively short.
- the processing time of the single-frame algorithm is shorter than that of the third photographing algorithm. In this way, compared with the processing of the first photographing algorithm and the second photographing algorithm, The duration is also shorter.
- the single frame algorithm may be the same as the third photographing algorithm, or the single frame algorithm may have a relatively longer processing time than the third photographing algorithm. Shorter.
- the device temperature when the device temperature is higher than the temperature threshold, you can switch to a single-frame algorithm to process one frame of original image according to the above steps, minimizing the workload of the electronic device, slowing down the increase in heat, and reducing the increase in heat. It can reduce the amount of pressure, so that the electronic device can take more pictures before it cannot work, so as to meet the user's photography needs.
- the device temperature is less than the temperature threshold, you can continue to judge the memory usage of the electronic device according to the process described in S16 to S19 above. According to different memory usage, switch to different algorithms and different frame numbers to reduce The pressure of low memory increment allows more photos to be taken before the memory is full to meet the user's photography needs.
- the execution sequence shown in Figure 11 is only an example. It is also possible to detect the device temperature of the electronic device after collecting the original image once, and when determining that the first operation is the first number of photos, and then , when the device temperature of the electronic device is greater than or equal to the temperature threshold, a single frame algorithm is used to process 1 frame of the original image. When the device temperature of the electronic device is less than the temperature threshold, the environment of the scene to be shot is then detected. Alternatively, the device temperature detection and the environment detection of the scene to be photographed can also be performed at the same time, and the number of level 1 processing frames and level 1 processing algorithm corresponding to the first number of photos can be determined based on the device temperature detection results and the environment detection results. Or, you can also perform temperature detection for each number of times a photo is taken to determine whether it is less than the temperature threshold, and then perform subsequent related steps.
- the above method only sets one temperature threshold.
- This application can also set multiple or three or more temperature thresholds, which can be set as needed. For example, two temperature thresholds can be set, namely a first temperature threshold and a second temperature threshold. The first temperature threshold is smaller than the second temperature threshold. Through these two temperature thresholds, the temperature range can be divided into three intervals. scope. When the device temperature meets one of the intervals, the algorithm and frame number corresponding to the interval are used for processing to obtain the corresponding captured image. On this basis, different memory thresholds can also be set for different temperature ranges to perform different judgment processes.
- Figure 12 shows a schematic flow chart of yet another photographing method.
- the method provides a new process including S22 based on Figure 11, which will be introduced below.
- S22 may also include the following S221 to S231.
- the second photography algorithm can be determined to be the x-level processing algorithm, and the number of x-level processing frames can be determined to be the second frame number.
- S226 Use the second photographing algorithm to process the second frame number in the original image collected this time to obtain the corresponding photographed image.
- the device temperature is less than the second temperature threshold and the memory usage is less than the second memory threshold, it means that the device temperature is not particularly high and the memory is not full. You can also use an algorithm with a longer processing time or a larger number of frames.
- the original image continues to be processed, such as executing S225 and S226 above.
- temperature and memory conditions can be combined when dividing the temperature range into multiple interval ranges and dividing the memory size into multiple interval ranges.
- an algorithm with a long processing time is used to process multiple frames in the original image to ensure better image quality and effect; when the temperature is low, the memory usage is large or the temperature
- adaptive adjustments are made, and an algorithm with a relatively short processing time is used to process more frames in the original image to balance the relationship between image quality, temperature, and memory usage;
- the above method provided by this application can make a judgment based on the device temperature and memory usage of the electronic device when the user triggers a photo command, and then associate the determined processing algorithm and number of frames with the number of photos.
- the frequency of user triggers is not high and the background processing of the electronic device is relatively timely, the number of processing frames can be processed according to the determined processing algorithm to obtain the corresponding captured image.
- the determined processing algorithm and number of processing frames are associated with the number of photos taken, and then stored together with the collected original images and other data, and then wait for subsequent call and processing by the electronic device.
- embodiments of the present application provide a photographing method to switch the processing algorithm to adapt to the memory situation at the actual processing time.
- Figure 13 is another photographing process provided by an embodiment of the present application. As shown in Figure 13, the method 30 may include the following 31 to S39.
- the electronic device displays a first interface, and the first interface includes a first control.
- the electronic device uses the camera to collect an original image.
- the frequency may indicate the speed at which the user performs the first operation, such as the clicking speed of clicking the shooting button. For example, the interval between two consecutive photo commands issued by the user is detected. If it is continuously detected that the interval for 10 or 20 times is less than the second preset interval, it can be shown that the frequency meets the preset frequency condition.
- the second preset interval duration can be set as needed, for example, it can be 100m or 200ms. The second preset interval duration is shorter than the first preset interval duration.
- the preset frequency condition that is, the number of consecutive detection intervals of adjacent photographing times, and the second preset interval length can be set and modified as needed, and the embodiments of the present application do not impose any restrictions on this.
- the memory occupancy is determined according to The method shown in Figure 3 determines the number of x-level processing frames and the x-level processing algorithm corresponding to the number of photos taken.
- the memory usage is determined, and the number of x-level processing frames and the x-level processing algorithm corresponding to the number of photos taken are determined according to the method shown in Figure 3, the number of x-level processing frames
- the long-exposure original image and the short-exposure original image may not be included, and the x-level processing algorithm does not include the high dynamic range (high dynamic range, HDR) algorithm.
- the memory water level refers to the maximum usage of the memory.
- the size of the memory water level can be obtained by obtaining parameters provided by the manufacturer, or it may need to be set and modified. The embodiments of this application do not impose any restrictions on this.
- the difference between the memory water level and the memory usage is greater than the preset difference, it means that the memory usage is still far from reaching the memory water level, and there is still a lot of available storage space left; when the memory water level is different from the memory usage
- the difference in occupancy is less than the preset difference, it means that the memory occupancy is very close to reaching the memory water mark, and there is not much remaining available storage space.
- the size of the preset difference value can be set and modified as needed, and the embodiments of the present application do not impose any restrictions on this.
- Determining the number of pictures waiting in the background refers to determining the number of pictures waiting in the background.
- the number of pictures waiting in the background Numbers include 1, 2 or more. For example, it includes two, namely the number of photos taken at level 5 and the number of photos taken at level 6. Then, the 5-level processing algorithm corresponding to the 5th time of taking pictures is used as its corresponding initial processing algorithm, and the 6-level processing algorithm corresponding to the 6th time of taking pictures is used as its corresponding initial processing algorithm. Of course, there may be no waiting number of photos taken in the background. When there is no waiting number of photos taken, there is no need to perform subsequent steps.
- the processing algorithm needs to be switched, and when switching, the processing algorithm that needs to be replaced must first be clarified.
- the latest number of photos taken refers to the number of photos currently recorded in response to the user's first operation.
- the process of determining the processing algorithm and the number of frames corresponding to the latest number of photos taken can be determined using the method described in S17 above, which will not be described again here.
- the processing algorithm and the number of processing frames corresponding to the number of pictures waiting in the background are determined based on the previous memory usage, the previous memory usage is different from the current memory usage. Therefore, the currently determined Taking the memory usage into consideration, all the processing algorithms corresponding to the number of photos waiting in the background are replaced with the currently determined processing algorithms to adapt to the current memory conditions.
- the processing algorithm corresponding to the most recently determined number of photos taken can be compared with the initial processing algorithm corresponding to the waiting number of photos taken. If they are the same, there is no need to replace them; if they are different, then Make a substitution. Therefore, by comparing the processing algorithms and only making substitutions when the processing algorithms are different, invalid substitutions can be avoided and the workload can be reduced.
- the number of processing frames corresponding to the number of waiting photos can also be used as the initial number of processing frames, and the number of processing frames corresponding to the most recently determined number of photos taken can be used as the number of target processing frames, and then, the target number of processing frames can be used Replace the number of initial processing frames corresponding to the number of waiting photos.
- the target processing frame number is used to replace the initial processing frame number
- the value of the processing frame number is simply changed. Changes are made without replacing the corresponding original image.
- the number of photos to be waited for is the 10th number of photos.
- 9 frames of original images were collected.
- 5 of the original images were screened out as the number of frames to be processed, "5 frames” That is the number of initial processing frames. If it is later determined that the target number of frames to be processed is 1 frame, the initial number of frames to be processed "5 frames" can be replaced with "1 frame”.
- the original image corresponding to this 1 frame is 1 frame filtered out from the original 5 frames of original images. That’s it.
- the processing algorithm and/or the number of processing frames corresponding to the number of waiting photos can be replaced each time a new photo command is responded to, or in other words, the processing algorithm corresponding to a certain number of waiting photos can be replaced.
- the processing algorithm and/or the number of processing frames may be replaced multiple times before being used for processing to obtain the corresponding captured image.
- the processing algorithm corresponding to the number of photos taken waiting in the background is the processing algorithm determined based on the memory usage in response to the user's first operation on the first control, or the processing algorithm after the last replacement during the waiting process.
- Target processing algorithm is the processing algorithm determined based on the memory usage in response to the user's first operation on the first control, or the processing algorithm after the last replacement during the waiting process.
- the processing algorithm corresponding to the 20th time is used as the target processing algorithm, and the corresponding number of processing frames is used as the target number of frames; then, the processing algorithm can be used as the target processing algorithm.
- the processing algorithm corresponding to the multiple number of waiting photos is used as the initial processing algorithm, and the corresponding processing frame number is used as the initial processing frame number. Then the initial processing algorithm is replaced with the target processing algorithm, and the initial processing frame number is replaced with the target processing Number of frames.
- the latest corresponding number of photos is updated to the 21st number of photos, then the processing algorithm corresponding to the 21st number of photos is used as the target processing algorithm, and the corresponding number of processed frames is used as the target.
- the number of frames to be processed, and then the processing algorithm corresponding to the multiple waiting number of photos can be used as the initial processing algorithm, and the corresponding number of frames to be processed (the target processing algorithm corresponding to the 20th photo number)
- the target processing frame number is used as the initial processing frame number, and then the initial processing algorithm is replaced with the target processing algorithm for the 21st photo taking time, and the initial processing frame number is replaced with the target processing frame number for the 21st photo taking time. If the user continues to perform the first operation, the subsequent replacement process will be deduced in sequence, which will not be described again here.
- the corresponding target processing algorithm is used to process the number of x-level processing frames in the original image collected for the number of photography times to obtain the corresponding captured image.
- the processing continues according to the originally determined processing algorithm.
- the previously determined x-level processing algorithm is used to process the number of x-level processing frames to obtain the corresponding captured image.
- the above-mentioned frequency or difference conditions are met, there is no need to perform the above steps when there is no waiting number of photos taken in the background.
- Captured images can be stored in the gallery or displayed.
- the electronic device in response to the user's first operation on the first control, uses a camera to collect an original image, and when the frequency used to perform the first operation on the first control meets the preset frequency
- the number of pictures waiting in the background is determined and the processing algorithm corresponding to each waiting number of pictures is used as the corresponding initial processing algorithm.
- the processing algorithm corresponding to the most recently determined number of photographing times is used as the target processing algorithm, and the target processing algorithm is used to replace the initial processing algorithm corresponding to the waiting number of photographing times.
- the replaced target processing algorithm can be used to process the initial processing frame number in the original image collected for the number of photos to obtain the corresponding captured image. Since the processing algorithm corresponding to the most recently determined number of photos is based on It is determined based on the current memory situation, so the processing algorithm for the number of waiting photos is adjusted accordingly, which can control the degree of memory accumulation, reduce memory usage, and achieve continuous and rapid photos.
- the number of frames processed can also be replaced.
- the number of processing frames corresponding to the most recently determined number of photos taken is used as the target number of processing frames, and the target number of processing frames is used to convert the initial number of frames corresponding to the number of waiting photos into Process frame number for replacement.
- the number of processing frames corresponding to the number of waiting photos is reduced. In this way, when the number of waiting photos is called, the reduced number of target processing frames can be used to determine the captured image. Since the number of processing frames corresponding to the most recently determined number of photos taken is determined based on the current memory situation, the number of frames processed for the number of waiting photos is adjusted accordingly, which can also control the degree of memory accumulation and reduce memory usage.
- the corresponding processing algorithm and/or number of frames can be replaced multiple times as the latest determined number of photos is changed, and the latest one can be used.
- the target processing algorithm replaces its last target processing algorithm, and/or uses the latest target processing frame number to replace its last target processing frame number. Therefore, when the number of shots waiting in the background is actually processed, The latest target processing algorithm can be used for processing. In this way, after multiple replacements, the latest data can be used as the benchmark to reduce the memory footprint as much as possible and achieve more consecutive and rapid photos.
- this method performs two replacements, or even multiple replacements, and can update and optimize the processing algorithm and/or the number of processing frames in real time, making it more flexible to adapt to the current memory situation. , to ensure that photography can proceed normally.
- FIG. 14 is a table provided by an embodiment of the present application, illustrating the processing algorithm and the number of processing frames corresponding to each number of photos taken.
- the mobile phone displays a shooting interface, which includes a shooting button.
- a shooting button When the user takes a photo, he uses his finger to click the shooting button multiple times.
- the mobile phone When the mobile phone detects the user's first click operation, in response to the first click operation, the mobile phone uses the camera to collect an original image. For example, it collects 9 frames of original images in the RAW domain. The number of pictures taken can be recorded as the first click operation. 1 number of photos taken.
- the mobile phone can perform environment detection for the scene to be photographed, such as illumination detection and dynamic range detection. At this time, it is also determined according to the environment detection result whether the detected illumination is less than the illumination threshold, and whether the detected dynamic range value is less than the dynamic range threshold.
- the level 1 processing algorithm is the A1 algorithm.
- the number of first-level processing frames corresponding to the first number of photography times is, for example, 6 frames.
- the 6 frames refer to 6 frames of original images filtered out from the 9 frames of original images collected corresponding to the first number of photography times.
- the mobile phone When the mobile phone detects the user's second click operation, in response to the click operation, the mobile phone uses the camera to collect an original image, or 9 frames of original images, and the number of photos taken can be recorded as the second number of photos taken.
- the phone continues to measure the memory usage. It is determined that since the memory usage at this time is less than the first memory threshold, it can be determined that the level 2 processing algorithm corresponding to the second number of photos is the first photo algorithm, and the first photo algorithm is, for example, the first night scene algorithm. At the same time, it can be determined that the number of second-level processing frames corresponding to the second number of photos is 6 frames, and 6 frames is the first frame number.
- the 6 frames refer to the 6 frames of original images filtered out from the 9 frames of original images collected corresponding to the second photographing time.
- the mobile phone detects the user's third click operation, in response to the click operation, the mobile phone uses the camera to collect an original image, or 9 frames of original images, and the number of photos taken can be recorded as the third number of photos taken.
- the mobile phone determines the memory usage at this time.
- the memory usage at this time is greater than the first memory threshold and less than the second memory threshold. Therefore, It can be determined that the 3-level processing algorithm corresponding to the third photographing number is the second photographing algorithm, and the preset second photographing algorithm is the same as the first photographing algorithm, that is, the second photographing algorithm is the first night scene algorithm.
- the number of level 3 processing frames corresponding to the third number of photos is 5 frames, 5 frames is the second frame number, and the second frame number is smaller than the first frame number.
- the 5 frames refer to the 5 frames of original images filtered out from the 9 frames of original images collected corresponding to the third photographing time.
- the mobile phone detects the user's fourth click operation, in response to the click operation, the mobile phone uses the camera to collect an original image, or 9 frames of original images, and the number of photos taken can be recorded as the fourth number of photos taken.
- the corresponding level 4 processing algorithm is the third photographing algorithm, and the preset third photographing algorithm is the same as the second photographing algorithm and the first photographing algorithm, that is, the third photographing algorithm is the first night scene algorithm.
- the number of 4-level processing frames corresponding to the fourth photographing algorithm is 4 frames, 4 frames is the third frame number, and the third frame number is smaller than the second frame number.
- the 4 frames refer to the 4 frames of original images filtered out from the 9 frames of original images collected corresponding to the fourth photographing time.
- the first night scene algorithm can include all the steps included in the single frame algorithm, as well as a night scene algorithm module generated based on the Unet network model.
- the night scene algorithm module can fuse multiple frames of original images in the RAW domain into one frame of image in the RAW domain.
- the night scene algorithm module can also be generated based on other models, and the first night scene algorithm can also include other modules or steps, and the embodiments of the present application have no limitations on this.
- the reduction in the number of processed frames corresponding to the number of times each photo is taken can be the same or different. Specifically, it can be set and modified as needed. The embodiments of the present application do not impose any restrictions on this.
- the scene to be photographed is a scene with low brightness and low dynamic range, it means that the ambient light in the scene to be photographed is relatively dark. If a general photographing algorithm is used for processing, the obtained photographed image will be unclear, which will lead to Users cannot see the details in the captured images clearly. In this regard, in the embodiment of the present application, you can choose to use the first night scene algorithm for processing to increase the brightness of the image during the processing, so that the subsequently acquired captured image can present more details.
- the number of processing frames determined for the number of pictures corresponding to different memory ranges gradually decreases.
- the processing time of the processing algorithm will be shortened accordingly, and the growth rate of memory usage will tend to be flat.
- the number of processing frames corresponding to each number of photos taken is reduced, thereby reducing the processing time of the processing algorithm, improving processing efficiency, and at the same time Slow down the growth rate of memory usage and reduce the pressure on memory increment.
- Figure 15 is another table provided by the embodiment of the present application, used to illustrate the processing algorithm and the number of processing frames corresponding to each number of photos taken.
- the mobile phone displays a shooting interface, which includes a shooting button.
- a shooting button When the user takes a photo, he uses his finger to click the shooting button multiple times.
- the mobile phone When the mobile phone detects the user's first click operation, in response to the click operation, the mobile phone uses the camera to collect an original image. For example, it collects 6 frames of original images in the RAW domain. This number of photos can be recorded as the first number of photos.
- the mobile phone can perform environment detection on the scene to be photographed, for example, illumination detection and dynamic range detection. At this time, it is also determined according to the environment detection result whether the detected illumination is less than the illumination threshold, and whether the detected dynamic range value is less than the dynamic range threshold.
- the level 1 processing algorithm can be determined to be the D1 algorithm. It is the first multi-frame noise reduction (MFNR) algorithm. In addition, it can be determined that the number of first-level processing frames corresponding to the first number of photos taken is 6 frames, and the 6 frames refer to all 6 frames of original images collected corresponding to the first number of photos taken.
- MFNR multi-frame noise reduction
- the mobile phone When the mobile phone detects the user's second click operation, in response to the click operation, the mobile phone uses the camera to collect an original image, or 6 frames of original images, and the number of photos taken can be recorded as the second number of photos taken.
- the mobile phone can then detect the device temperature to determine whether the device temperature is less than the temperature threshold. If it is less than the temperature threshold, continue to determine the memory usage. And because the memory usage at this time is less than the first memory threshold, it can be determined that the level 2 processing algorithm corresponding to the second number of photos is the first photo algorithm, and the first photo algorithm is, for example, the second MFNR algorithm. At the same time, it can be determined that the number of second-level processing frames corresponding to the second number of photos is 6 frames, and 6 frames is the first frame number. The 6 frames refer to all 6 frames of original images collected corresponding to the second photographing time.
- the mobile phone detects the user's third click operation, in response to the click operation, the mobile phone uses the camera to collect an original image, or 6 frames of original images, and the number of photos taken can be recorded as the third number of photos taken.
- the mobile phone can then detect the device temperature to determine whether the device temperature is less than the temperature threshold. If it is less than the temperature threshold, it will continue to target the memory usage at this time. It is determined that the memory usage at this time is greater than the first memory threshold and less than the second memory threshold. From this, it can be determined that the level 3 processing algorithm corresponding to the third number of photos is the second photo algorithm, and the preset second photo algorithm Different from the first photographing algorithm, and the processing time of the second photographing algorithm is shorter than the processing time of the first photographing algorithm, the second photographing algorithm is, for example, a third MFNR algorithm.
- the number of level 3 processing frames corresponding to the third number of photos is 6 frames
- 6 frames is the second frame number
- the second frame number is the same as the first frame number.
- the 6 frames refer to all 6 frames of original images collected corresponding to the third photographing time.
- the mobile phone When the mobile phone detects the user's fourth click operation, in response to the click operation, the mobile phone uses the camera to collect an original image, or 6 frames of original images. The number of photos taken can be recorded as the fourth number of photos taken.
- the device temperature can be detected to determine whether the device temperature is less than the temperature threshold. If it is less than the temperature threshold, the phone continues to process the memory usage at this time. It is determined that the memory usage at this time is greater than the second memory threshold. From this, it can be determined that the 4-level processing algorithm corresponding to the fourth number of photos is the third photo algorithm.
- the preset third photo algorithm is the same as the second photo algorithm. The first photographing algorithm is different, and the processing time of the third photographing algorithm is shorter than that of the second photographing algorithm.
- the third photographing algorithm is, for example, the fourth MFNR algorithm.
- the number of 4-level processing frames corresponding to the fourth number of photos is 6 frames
- 6 frames is the third frame number
- the third frame number is the same as the second frame number and the first frame number.
- the 6 frames refer to all 6 frames of original images collected corresponding to the fourth photographing time.
- the mobile phone detects the user's fifth click operation, in response to the click operation, the mobile phone uses the camera to collect an original image, or 6 frames of original images, and the number of photos taken can be recorded as the fifth number of photos taken.
- the device temperature can be detected to determine whether the device temperature is less than the temperature threshold. If it is greater than the temperature threshold, there is no need to judge the memory usage and directly determine the time.
- the processing algorithm is a single-frame algorithm, and the number of frames processed is 1 frame. This 1 frame refers to 1 frame of the original image among the 6 frames of original images collected corresponding to the 5th photographing time.
- the first MFNR algorithm can be a process of first aligning multiple frames in the original image, and then processing the aligned original image using a wavelet fusion algorithm;
- the second MFNR algorithm can be an optical flow algorithm, and the third MFNR
- the algorithm and the fourth MFNR algorithm are other algorithms, and the embodiments of the present application do not have any restrictions on this.
- the scene to be photographed is a scene with high brightness and low dynamic range
- the MFNR algorithm can be selected for processing to reduce the noise of the captured images and improve the dynamic range of the captured images.
- the processing time of the processing algorithm determined by the number of pictures corresponding to different memory ranges gradually decreases.
- the number of frames selected for processing is the same, during subsequent processing, since the processing time of the processing algorithm gradually becomes shorter, the memory usage increases at a slower rate. Therefore, in the embodiment of the present application, in multiple Under the number of photos taken, as the number of photos taken increases, based on the same number of processing frames, the processing time or performance of the processing algorithm corresponding to each number of photos taken can be reduced, thereby improving processing efficiency and slowing down the memory usage. Growth speed, reducing the pressure on memory increment.
- one frame of original image can be processed according to the direct switching to a single-frame algorithm to reduce the workload, slow down the growth of heat, and reduce the incremental pressure of heat, thereby making the mobile phone more Achieve a certain number of photos to meet the user's photo needs.
- Figure 16 is another table provided by the embodiment of the present application, used to illustrate the processing algorithm and the number of processing frames corresponding to each number of photos taken.
- Figure 17 is a schematic diagram of the processing process corresponding to Figure 16.
- the mobile phone displays a recording interface, which includes a capture control.
- the user uses his finger to click the capture control multiple times while recording.
- the mobile phone uses the camera to collect an original image. For example, it collects 9 frames of original images.
- the 9 frames of original images include 6 frames of normal exposure original images. Image, 1 frame long exposure original image and 2 frames short exposure original image. This number of photos can be recorded as the first number of photos.
- the mobile phone Since this click operation corresponds to the first number of photos taken, the mobile phone performs environment detection on the scene to be photographed, for example, illumination detection and dynamic range detection. At this time, it is also determined according to the detection result whether the detected illumination is less than the illumination threshold, and whether the detected dynamic range value is less than the dynamic range threshold.
- the level 1 processing algorithm can be determined as C1 algorithm, C1 algorithm is the first HDR algorithm.
- the number of first-level processing frames corresponding to the first number of photos taken is 9 frames, and these 9 frames refer to all 9 frames of original images collected corresponding to the first number of photos taken.
- the first HDR algorithm may include an HDR algorithm module, and the electronic device may implement the processing process of the first HDR algorithm through the HDR algorithm module.
- the first HDR algorithm module is based on multi-exposure fusion processing and tone mapping model generation. It can fuse multiple frames of original images with different exposure values or different exposure levels into one frame of image in the RAW domain and perform mapping transformation processing on its color.
- the tone mapping model can be any one of Unet network model, Resnet network model and Hdrnet network model.
- the tone mapping model can also be other models, and the embodiments of the present application do not impose any limitation on this.
- the background begins to use the first HDR algorithm determined for the first number of photos to process the number of first-level processing frames, that is, all 9 frames of original images collected corresponding to the first number of photos Processing is performed, and this process can be called the first photographing process.
- the mobile phone uses the camera to collect an original image, which is still 9 frames of original images.
- the 9 frames of original images include 7 frames of normal exposure original images, 1 frame Long exposure original image and 1 frame short exposure original image. This number of photos can be recorded as the second number of photos.
- the phone can then determine whether the frequency of the user's click operations on the capture control meets the preset frequency conditions. If the frequency does not meet the preset frequency conditions, continue The memory occupancy is determined, and since the memory occupancy at this time is less than the first memory threshold, it can be determined that the 2-level processing algorithm corresponding to the second number of photos is the first photo algorithm, and the first photo algorithm is, for example, the second HDR algorithm, the second HDR algorithm is different from the first HDR algorithm. At the same time, it can be determined that the number of second-level processing frames corresponding to the second number of photos is 8 frames, and 8 frames is the first frame number.
- the 8 frames refer to 5 frames of normal-exposure original images, 1 frame of long-exposure original images, and 2 frames of short-exposure original images among the 9 frames of original images collected corresponding to the second photographing time.
- the 8 frames refer to the 5 normal-exposure original images among the 9 original images collected corresponding to the second photo-taking time, and the 1-frame long-exposure original image and 2 frames collected in the first photo-taking time.
- Short exposure raw image Alternatively, the 8 frames refer to the multiplexing of 1 frame of long-exposure original image and 2 frames of short-exposure original image collected in the first photographing time, and the collection time of the long-exposure original image or the short-exposure original image is closer 5 frames of normally exposed original image.
- the subsequent process from the third to the fifth photo taking times is similar to the above-mentioned determination process of the second photo taking times, and will not be described again here.
- the mobile phone uses the camera to collect One original image is still 9 frames of original images.
- the 9 frames of original images include 6 frames of normal exposure original images, 1 frame of long exposure original images, and 2 frames of short exposure original images. This number of photos can be recorded as the 6th time of photos.
- the mobile phone can then determine whether the frequency of the user's click operations on the capture control meets the preset frequency conditions. If the frequency does not meet the preset frequency conditions, continue Determine the memory usage at this time. The memory usage at this time is greater than the first memory threshold and less than the second memory threshold. From this, it can be determined that the 6-level processing algorithm corresponding to the 6th number of photos is the second photo algorithm. , the preset second photographing algorithm is different from the first photographing algorithm, and the processing time of the second photographing algorithm is shorter than the processing time of the first photographing algorithm.
- the second photographing algorithm is, for example, the third HDR algorithm, and the third HDR algorithm is the same as the first photographing algorithm.
- the first HDR algorithm and the second HDR algorithm are different. At the same time, it can be determined that the number of 6-level processing frames corresponding to the sixth number of photos is 7 frames, 7 frames is the second frame number, and the second frame number is smaller than the first frame number.
- the 7 frames refer to 4 frames of normal-exposure original images, 1 frame of long-exposure original images, and 2 frames of short-exposure original images among the 9 frames of original images collected corresponding to the sixth photographing time.
- the 7 frames refer to the 4 normal-exposure original images among the 9 original images collected corresponding to the sixth photo-taking time, and the multiplexing of the 1-frame long-exposure original image and 2 frames collected in the first photo-taking time. Short exposure raw image.
- the 7 frames refer to the multiplexing of 1 frame of long-exposure original image and 2 frames of short-exposure original image collected in the first photographing time, and the collection time of the long-exposure original image or the short-exposure original image is closer 5 frames of normally exposed original image.
- the data related to the sixth photo-taking number will be processed first according to the principle of time from the second to the sixth photo-taking number waiting in the background.
- the background began to use the determined third HDR algorithm to process 7 frames of original images, that is, 4 frames of normal exposure original images and 1 frame of long exposure original images collected corresponding to the 6th time of taking pictures.
- the image is processed with the original image of 2 frames of short exposure. This process can be called the second photo processing.
- the mobile phone uses the camera to collect one original image, or 9 frames of original images.
- the 9 frames of original images include 6 frames of normal exposure original images, 1 frame of long exposure original images, and 2 frames of short exposure original images. This number of photos can be recorded as the 7th number of photos.
- the mobile phone can then determine whether the frequency of the user's click operations on the capture control meets the preset frequency conditions. If the frequency meets the preset frequency conditions, it is necessary to Give up protecting dynamic range and switch to a non-HDR algorithm that only processes normally exposed raw images. For example, the memory usage at this time can be determined. If the memory usage at this time is greater than the first memory threshold and less than the second memory threshold, at this time, the 7-level processing algorithm corresponding to the 7th number of photos can be changed from the 7th level to the 7th level. The third HDR algorithm is switched to the third MFNR algorithm. At the same time, it can be determined that the number of 7-level processing frames corresponding to the 7th number of photos is 4 frames of normally exposed original images, and other long-exposure original images and short-exposure original images are removed.
- the frequency meets the preset frequency condition, it can also be determined that the number of photos waiting in the background at this time is the second to fifth photos. Based on this, the corresponding processing algorithms stored before the 2nd to 5th photo-taking times can be used as the initial processing algorithm, and the 7-level processing algorithm determined by the 7th photo-taking time can be used as the target processing algorithm, and then, using this target The processing algorithm replaces the initial processing algorithm corresponding to the number of waiting pictures.
- the 7-level processing algorithm determined by the 7th number of photos is the third MFNR algorithm.
- the third MFNR algorithm is the target processing algorithm.
- the second HDR algorithm originally corresponding to the second number of photos is the initial processing algorithm.
- the second level processing algorithm corresponding to the second number of photos is It will be changed to the third MFNR algorithm. The replacement process of the third to fifth photography times is similar and will not be described again here.
- the 7th photo-taking time can only be compared to all the original images collected and the determined 7-level
- the processing algorithm, 7-level processing frame number and other related data are stored and enter the background waiting state.
- the mobile phone uses the camera to collect an original image, which is still 9 frames of original images.
- the 9 frames of original images include 6 frames of normal exposure original images and 1 frame length.
- the mobile phone can then determine whether the frequency of the user's click operation on the capture control meets the preset frequency condition. If the frequency meets the preset frequency condition, it is necessary to Give up protecting dynamic range and switch to a non-HDR algorithm that only processes normally exposed raw images. For example, the memory usage at this time can be determined. If the memory usage at this time is greater than the second memory threshold, the 8-level processing algorithm corresponding to the 8th number of photos can be switched from the third HDR algorithm to Fourth MFNR algorithm. The fourth MFNR algorithm is different from the third MFNR algorithm. At the same time, it can be determined that the number of 8-level processing frames corresponding to the 8th number of photos is 3 frames of normally exposed original images, and other long-exposure original images and short-exposure original images are removed.
- the frequency meets the preset frequency condition, it can also be determined that the number of photos waiting in the background at this time is the second to fifth photos, and the seventh photo.
- the processing algorithm after the last replacement from the 2nd to 5th photo times can be used as the initial processing algorithm, and the 7-level processing algorithm determined from the 7th photo time can be used as the initial processing algorithm, and then the 7th level processing algorithm can be used as the initial processing algorithm.
- the 8-level processing algorithm determined by the number of 8 photos taken is used as the target processing algorithm. Then, the target processing algorithm is used to replace the initial processing algorithm corresponding to the number of waiting photos.
- the 8-level processing algorithm determined by the eighth number of photos is the fourth MFNR algorithm, so the fourth MFNR algorithm is the target processing algorithm.
- the third MFNR algorithm originally corresponding to the second number of photos was the initial processing algorithm.
- the second-level processing algorithm corresponding to the second number of photos was changed to the fourth MFNR algorithm.
- the process of replacing the number of photos taken for the fifth and seventh times is similar and will not be described again here.
- the 8th photo-taking time can only be compared to all the original images collected and the determined 8-level
- the processing algorithm, 8-level processing frame number and other related data are stored and enter the background waiting state.
- the processing algorithm corresponding to the number of pictures waiting in the background can continue to be processed based on the x-level processing algorithm corresponding to the most recently determined number of pictures taken. Algorithm replacement. When processing the number of photos waiting in the background, the processing can be performed according to the processing algorithm after the last replacement.
- this application combines various factors such as the environmental detection results in the scene to be photographed, the frequency of user operations, the memory usage, etc., to determine the processing algorithm for the number of photos taken and the number of frames processed under different situations. change.
- the above describes the photographing method provided by the embodiment of the present application in detail.
- the following describes how the user activates the photographing method provided by the embodiment of the present application in conjunction with the display interface of the electronic device.
- FIG. 18 is a schematic diagram of a display interface of an electronic device provided by an embodiment of the present application.
- the electronic device 100 displays a shooting interface as shown in (a) of FIG. 18 .
- a navigation bar is displayed at the top of the interface, and an icon indicating "AI Photography" is displayed in the middle position.
- the electronic device 100 In response to the user's click operation on "AI Photography", the electronic device 100 displays a prompt box as shown in (b) of FIG. 18 , and a prompt message that "AI Photography" is turned on is displayed on the interface. At this time, the electronic device 100 can activate the program related to the photographing method provided by the embodiment of the present application when photographing.
- the photographing method provided by the embodiment of the present application can also be enabled in other ways, or the photographing method can also be enabled during shooting.
- the photographing method provided by the embodiment of the present application is directly used in the process, and the embodiment of the present application does not impose any restrictions on this.
- FIG. 19 is a schematic diagram of a display interface of another electronic device provided by an embodiment of the present application.
- Figure 20 shows a hardware system suitable for the electronic device of the present application.
- the electronic device 100 may be a mobile phone, a smart screen, a tablet, a wearable electronic device, a vehicle-mounted electronic device, an augmented reality (AR) device, a virtual reality (VR) device, a notebook computer, or a super mobile personal computer ( Ultra-mobile personal computer (UMPC), netbook, personal digital assistant (personal digital assistant, PDA), projector, etc.
- the embodiment of the present application does not place any restrictions on the specific type of the electronic device 100.
- the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2 , mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone interface 170D, sensor module 180, button 190, Motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (subscriber identification module, SIM) card interface 195, etc.
- a processor 110 an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2 , mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone interface 170D, sensor module 180, button 190, Motor 191, indicator 192
- the sensor module 180 may include a pressure sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light. Sensor 180L, bone conduction sensor 180M, etc.
- the structure shown in FIG. 20 does not constitute a specific limitation on the electronic device 100.
- the electronic device 100 may include more or less components than those shown in FIG. 20 , or the electronic device 100 may include a combination of some of the components shown in FIG. 20 , or , the electronic device 100 may include sub-components of some of the components shown in FIG. 20 .
- the components shown in Figure 20 may be implemented in hardware, software, or a combination of software and hardware.
- Processor 110 may include one or more processing units.
- the processor 110 may include at least one of the following processing units: an application processor (application processor, AP), a modem processor, a graphics processing unit (GPU), an image signal processor (image signal processor) , ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, neural network processing unit (NPU).
- an application processor application processor, AP
- modem processor graphics processing unit
- GPU graphics processing unit
- image signal processor image signal processor
- ISP image signal processor
- controller video codec
- digital signal processor digital signal processor
- baseband processor baseband processor
- neural network processing unit NPU
- different processing units can be independent devices or integrated devices.
- the controller can generate operation control signals based on the instruction operation code and timing signals to complete the control of fetching and executing instructions.
- the processor 110 may also be provided with a memory for storing instructions and data.
- the memory in processor 110 is cache memory. This memory may hold instructions or data that have been recently used or recycled by processor 110 . If the processor 110 needs to use the instructions or data again, it can be called directly from the memory. Repeated access is avoided and the waiting time of the processor 110 is reduced, thus improving the efficiency of the system.
- the processor 110 may be configured to perform the photographing method of the embodiment of the present application; display a first interface, where the first interface includes a first control; and when a first operation on the first control is detected, respond to the first operation of the first control.
- the original image is collected once; for the x-th number of photos taken, the memory usage is determined; when the memory usage is less than the first memory threshold, the x-level processing algorithm is determined to be the first photo-taking algorithm, and the number of x-level processing frames is determined to be The number of the first frame, x is an integer greater than 1; when the memory usage is greater than or equal to the first memory threshold and less than the second memory threshold, determine the x-level processing algorithm as the second photography algorithm, and determine the number of x-level processing frames is the second number of frames; when the memory usage is greater than the second memory threshold, the x-level processing algorithm is determined to be the third photographing algorithm, and the number of x-level processing frames is determined to be the third number of frames, and the
- connection relationship between the modules shown in FIG. 20 is only a schematic illustration and does not constitute a limitation on the connection relationship between the modules of the electronic device 100.
- each module of the electronic device 100 may also adopt a combination of various connection methods in the above embodiments.
- the charging management module 140 is used to receive charging input from the charger.
- the charger can be a wireless charger or a wired charger.
- the power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110.
- the power management module 141 receives input from the battery 142 and/or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, the wireless communication module 160, and the like.
- the power management module 141 is also used to detect battery capacity, battery cycle times, battery health status (leakage, impedance) and other parameters.
- the electronic device 100 implements display functions through a GPU, a display screen 194, an application processor, and the like.
- the GPU is an image processing microprocessor and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering.
- Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.
- the display screen 194 is used to display images, videos, etc.
- Display 194 includes a display panel.
- the display panel can use a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (active-matrix organic light emitting diode).
- LCD liquid crystal display
- OLED organic light-emitting diode
- AMOLED organic light-emitting diode
- FLED flexible light-emitting diode
- Miniled MicroLed, Micro-oLed, quantum dot light emitting diode (QLED), etc.
- the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.
- the electronic device 100 can implement the shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like.
- the ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera sensor through the lens, the optical signal is converted into an electrical signal, and the camera sensor passes the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be provided in the camera 193.
- Camera 193 is used to capture still images or video. It can be triggered by application instructions to realize the camera function, such as capturing images of any scene.
- the camera can include imaging lenses, filters, image sensors and other components. The light emitted or reflected by the object enters the imaging lens, passes through the optical filter, and finally converges on the image sensor.
- the imaging lens is mainly used to collect and image the light emitted or reflected by all objects in the camera angle (which can also be called the scene to be shot, the target scene, or the scene image that the user expects to shoot);
- the filter is mainly used to To filter out excess light waves in the light (such as light waves other than visible light, such as infrared);
- the image sensor can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) ) phototransistor.
- CMOS complementary metal-oxide-semiconductor
- the image sensor is mainly used to photoelectrically convert the received optical signal into an electrical signal, and then transfer the electrical signal to the ISP to convert it into a digital image signal.
- ISP outputs digital image signals to DSP for processing.
- DSP converts digital image signals into standard RGB, YUV and other format image signals.
- the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.
- the camera 193 can be located in front of the electronic device 100 or at the back of the electronic device 100.
- the specific number and arrangement of the cameras can be set according to needs, and this application does not impose any restrictions.
- the electronic device 100 includes a front camera and a rear camera.
- the front camera or the rear camera may include one or more cameras.
- the camera method provided by the embodiment of the present application can be used.
- the camera is installed on an external accessory of the electronic device 100.
- the external accessory is rotatably connected to the frame of the mobile phone.
- the angle formed between the external accessory and the display screen 194 of the electronic device 100 is 0-360 degrees. any angle between.
- the electronic device 100 takes a selfie
- the external accessory drives the camera to rotate to a position facing the user.
- only some of the cameras can be set externally.
- the remaining cameras are arranged on the body of the electronic device 100, and the embodiment of the present application does not impose any restrictions on this.
- Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.
- Internal memory 121 may be used to store computer executable program code, which includes instructions.
- the internal memory 121 may include a program storage area and a data storage area.
- the stored program area can store an operating system, at least one application program required for a function (such as a sound playback function, an image playback function, etc.).
- the storage data area may store data created during use of the electronic device 100 (such as audio data, phone book, etc.).
- the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.
- the processor 110 executes various functional applications and data processing of the electronic device 100 by executing instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor.
- the internal memory 121 can also store the software code that provides the photographing method according to the embodiment of the present application.
- the processor 110 runs the software code, the process steps of the photographing method are executed to achieve rapid and continuous photography.
- the internal memory 121 can also store captured images.
- the external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100.
- the external memory card communicates with the processor 110 through the external memory interface 120 to implement the data storage function. For example, save files such as music on an external memory card.
- the software code of the photographing method provided by the embodiment of the present application can also be stored in an external memory.
- the processor 110 can run the software code through the external memory interface 120 to execute the process steps of the photographing method to obtain multiple frames of captured images.
- the captured image obtained by the electronic device 100 may also be stored in an external memory.
- the user can specify whether the image is stored in the internal memory 121 or in the external memory.
- the electronic device 100 is connected to an external memory, if the electronic device 100 captures an image, a prompt message can pop up to prompt the user whether to store the image in the external memory or the internal memory; of course, there can be other designated methods.
- the embodiments of the present application do not impose any limitation on this; alternatively, when the electronic device 100 detects that the memory amount of the internal memory 121 is less than the preset amount, the image can be automatically stored in the external memory.
- the electronic device 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone interface 170D, and the application processor. Such as music playback, recording, etc.
- the pressure sensor 180A is used to sense pressure signals and can convert the pressure signals into electrical signals.
- pressure sensor 180A may be disposed on display screen 194 .
- the gyro sensor 180B may be used to determine the motion posture of the electronic device 100 .
- the angular velocity of electronic device 100 about three axes ie, x, y, and z axes
- the gyro sensor 180B can be used for image stabilization.
- Air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.
- Magnetic sensor 180D includes a Hall sensor.
- the electronic device 100 may utilize the magnetic sensor 180D to detect opening and closing of the flip holster.
- the electronic device 100 may detect the opening and closing of the flip according to the magnetic sensor 180D. Then based on the detected opening and closing status of the leather case or the opening and closing status of the flip cover status, and set features such as automatic unlocking of the flip cover.
- the acceleration sensor 180E can detect the acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of electronic devices and be used in horizontal and vertical screen switching, pedometer and other applications.
- Distance sensor 180F for measuring distance.
- Electronic device 100 can measure distance via infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 may utilize the distance sensor 180F to measure distance to achieve fast focusing.
- Proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode.
- the light emitting diode may be an infrared light emitting diode.
- the electronic device 100 emits infrared light outwardly through the light emitting diode.
- Electronic device 100 uses photodiodes to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100 . When insufficient reflected light is detected, the electronic device 100 may determine that there is no object near the electronic device 100 .
- the electronic device 100 can use the proximity light sensor 180G to detect when the user holds the electronic device 100 close to the ear for talking, so as to automatically turn off the screen to save power.
- the proximity light sensor 180G can also be used in holster mode, and pocket mode automatically unlocks and locks the screen.
- the ambient light sensor 180L is used to sense ambient light brightness.
- the electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness.
- the ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures.
- the ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in the pocket to prevent accidental touching.
- Fingerprint sensor 180H is used to collect fingerprints.
- the electronic device 100 can use the collected fingerprint characteristics to achieve fingerprint unlocking, access to application locks, fingerprint photography, fingerprint answering of incoming calls, etc.
- Temperature sensor 180J is used to detect temperature.
- the electronic device 100 utilizes the temperature detected by the temperature sensor 180J to execute the temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of a processor located near the temperature sensor 180J in order to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to prevent the low temperature from causing the electronic device 100 to shut down abnormally. In some other embodiments, when the temperature is lower than another threshold, the electronic device 100 performs boosting on the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.
- Touch sensor 180K also known as "touch device”.
- the touch sensor 180K can be disposed on the display screen 194.
- the touch sensor 180K and the display screen 194 form a touch screen, which is also called a "touch screen”.
- the touch sensor 180K is used to detect a touch operation on or near the touch sensor 180K.
- the touch sensor can pass the detected touch operation to the application processor to determine the touch event type.
- Visual output related to the touch operation may be provided through display screen 194 .
- the touch sensor 180K may also be disposed on the surface of the electronic device 100 at a location different from that of the display screen 194 .
- Bone conduction sensor 180M can acquire vibration signals.
- the bone conduction sensor 180M can acquire the vibration signal of the vibrating bone mass of the human body's vocal part.
- the bone conduction sensor 180M can also contact the human body's pulse and receive blood pressure beating signals.
- the bone conduction sensor 180M can also be provided in an earphone and combined into a bone conduction earphone.
- the audio module 170 can analyze the voice signal based on the vibration signal of the vocal vibrating bone obtained by the bone conduction sensor 180M to implement the voice function.
- the application processor can analyze the heart rate information based on the blood pressure beating signal acquired by the bone conduction sensor 180M to implement the heart rate detection function.
- the buttons 190 include a power button, a volume button, etc.
- Key 190 may be a mechanical key. It can also be a touch button key.
- the electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100 .
- the motor 191 can generate vibration prompts.
- the motor 191 can be used for vibration prompts for incoming calls and can also be used for touch vibration feedback.
- touch operations for different applications can correspond to different vibration feedback effects.
- the indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, or may be used to indicate messages, missed calls, notifications, etc.
- the SIM card interface 195 is used to connect a SIM card.
- the SIM card can be connected to or separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195 .
- the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100 .
- the electronic device 100 may include more or fewer components than shown in the figures, or some components may be combined, some components may be separated, or some components may be arranged differently.
- the components illustrated may be implemented in hardware, software, or a combination of software and hardware.
- the hardware system of the electronic device 100 is described in detail above, and the software system of the electronic device 100 is introduced below.
- Figure 21 is a schematic diagram of a software system of an electronic device provided by an embodiment of the present application.
- the system architecture may include an application layer (APP) 210, an application framework layer 220, a hardware abstract layer (HAL) 230, a driver layer 240 and a hardware layer 250.
- APP application layer
- HAL hardware abstract layer
- the application layer 210 may include a camera application or other applications.
- Other applications include but are not limited to: camera, gallery and other applications.
- the application layer 210 is at the top of the entire framework and is responsible for interacting directly with the user. Once it receives the user's direct or indirect request, such as taking a photo, it will send the request to the application framework layer 220 through the interface and wait for the application framework layer 220 The processing result is transmitted back, where the result includes image data, camera parameters, etc.; and then the application layer 210 feeds back the result to the user.
- the application framework layer 220 can provide an application programming interface (API) and programming framework to applications in the application layer; the application framework layer can include some predefined functions.
- API application programming interface
- the application framework layer 220 may include a camera access interface; the camera access interface may include camera management and camera equipment; where camera management may be used to provide an access interface for managing cameras; and the camera device may be used to provide an interface for accessing cameras.
- Hardware abstraction layer 230 is used to abstract hardware.
- the hardware abstraction layer can include a camera hardware abstraction layer and other hardware device abstraction layers; the camera hardware abstraction layer can include camera device 1, camera device 2, etc.; the camera hardware abstraction layer can be connected to the camera algorithm library, and the camera hardware abstraction layer Algorithms in the camera algorithm library can be called.
- the perception engine used for various detections can be set in the hardware abstraction layer.
- the driver layer 240 is used to provide drivers for different hardware devices.
- the driver layer may include camera device drivers, digital signal processor drivers, and graphics processor drivers.
- the hardware layer 250 may include multiple image sensors, multiple image signal processors, digital signal processors, graphics processors, and other hardware devices.
- the hardware layer 250 includes a sensor and an image signal processor; the sensor may include sensor 1, sensor 2, depth sensor (time of flight, TOF), multispectral sensor, etc. image signal processor It may include image signal processor 1, image signal processor 2, etc.
- the connection between the application layer 210 and the application framework layer 220 above the hardware abstraction layer 230 and the driver layer 240 and the hardware layer 250 below can be realized.
- the camera hardware interface layer in the hardware abstraction layer 230 manufacturers can customize functions here according to needs. Compared with the hardware abstraction layer interface, the camera hardware interface layer is more efficient, flexible, and low-latency, and can also make more abundant calls to ISP and GPU to implement image processing.
- the image input to the hardware abstraction layer 230 may come from an image sensor or a stored picture.
- the scheduling layer in the hardware abstraction layer 230 includes general functional interfaces for implementing management and control.
- the camera service layer in the hardware abstraction layer 230 is used to access interfaces of ISP and other hardware.
- the following exemplifies the workflow of the software and hardware of the electronic device 100 in conjunction with capturing the photographing scene.
- the camera application in the application layer may be displayed on the screen of the electronic device 100 in the form of an icon.
- the electronic device 100 starts to run the camera application.
- the camera application runs on the electronic device 100, the camera application calls the interface corresponding to the camera application in the application framework layer 210, and then starts the camera driver by calling the hardware abstraction layer 230 to turn on the camera 193 on the electronic device 100.
- the camera The algorithm library begins to load the photographing method used in the embodiment of this application.
- the collected original images can be processed by the image signal processor and returned to the hardware abstraction layer, and processed using a certain processing algorithm called from the camera algorithm library to generate Capture an image and then save and/or transfer the captured image to the monitor for display.
- FIG. 22 is a schematic diagram of an image processing device 300 provided by an embodiment of the present application.
- the image processing device 300 includes a display unit 310 , an acquisition unit 320 and a processing unit 330 .
- the display unit 310 is used to display a first interface, and the first interface includes first controls.
- the acquisition unit 320 is used to detect the user's first operation on the first control.
- the processing unit 330 is configured to collect an original image once in response to the first operation.
- the processing unit 330 is also used to determine the memory occupancy for the x-th number of photos taken; when the memory occupancy is less than the first memory threshold, determine the x-level processing algorithm as the first photo-taking algorithm, and determine the x-level processing frame number as the first The number of frames, x is an integer greater than 1; when the memory usage is greater than or equal to the first memory threshold and less than the second memory threshold, determine the x-level processing algorithm as the second photography algorithm, and determine the x-level processing frame number as the Two frames; when the memory usage is greater than the second memory threshold, the x-level processing algorithm is determined to be the third photographing algorithm, and the x-level processing frame number is determined to be the third frame number, and the first memory threshold is smaller than the second memory threshold. ;Use the x-level processing algorithm to process the number of x-level processing frames in the original image to obtain the corresponding captured image; save the captured image.
- image processing device 300 is embodied in the form of functional units.
- unit here can be implemented in the form of software and/or hardware, and is not specifically limited.
- a "unit” may be a software program, a hardware circuit, or a combination of both that implements the above functions.
- the hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a dedicated processor, or a group processor) for executing one or more software or firmware programs. etc.) and memory, merged logic circuitry, and/or other suitable components to support the described functionality.
- ASIC application specific integrated circuit
- processor such as a shared processor, a dedicated processor, or a group processor for executing one or more software or firmware programs. etc.
- memory merged logic circuitry, and/or other suitable components to support the described functionality.
- the units of each example described in the embodiments of the present application can be implemented as electronic hardware or computers. Achieved by a combination of software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each specific application, but such implementations should not be considered beyond the scope of this application.
- Embodiments of the present application also provide a computer-readable storage medium, in which computer instructions are stored; when the computer-readable storage medium is run on the image processing device 300, the image processing device 300 causes the image processing device 300 to Perform the photo-taking method shown above.
- the computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transferred from a website, computer, server, or data center Transmission to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means.
- the computer-readable storage medium can be any available medium that can be accessed by a computer or include one or more data storage devices such as servers and data centers that can be integrated with the medium.
- the available media may be magnetic media (eg, floppy disk, hard disk, tape), optical media, or semiconductor media (eg, solid state disk (SSD)), etc.
- Embodiments of the present application also provide a computer program product containing computer instructions, which when run on the image processing device 300 enables the image processing device 300 to execute the photographing method shown above.
- Figure 23 is a schematic structural diagram of a chip provided by an embodiment of the present application.
- the chip shown in Figure 23 can be a general-purpose processor or a special-purpose processor.
- the chip includes a processor 401.
- the processor 401 is used to support the image processing device 300 in executing the technical solutions shown above.
- the chip also includes a transceiver 402, which is used to accept the control of the processor 401 and to support the image processing device 300 in executing the technical solution shown above.
- the chip shown in Figure 23 may also include: a storage medium 403.
- the chip shown in Figure 23 can be implemented using the following circuits or devices: one or more field programmable gate arrays (FPGA), programmable logic devices (PLD) , controller, state machine, gate logic, discrete hardware components, any other suitable circuit, or any combination of circuits capable of performing the various functions described throughout this application.
- FPGA field programmable gate arrays
- PLD programmable logic devices
- controller state machine
- gate logic discrete hardware components
- any other suitable circuit any combination of circuits capable of performing the various functions described throughout this application.
- the electronic equipment, image processing device 300, computer storage media, computer program products, and chips provided by the embodiments of the present application are all used to execute the methods provided above. Therefore, the beneficial effects they can achieve can be referred to the methods provided above. The beneficial effects corresponding to the method will not be repeated here.
- preset and predefined can be configured on the device (for example, package This can be achieved by pre-saving corresponding codes, tables or other methods that can be used to indicate relevant information (including electronic equipment). This application does not limit the specific implementation method.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
Abstract
本申请提供了一种拍照方法及其相关设备,涉及图像处理领域,该拍照方法包括:针对第x次拍照次数,确定内存占用量;当小于第一内存阈值时,确定x级处理算法为第一拍照算法,并确定x级处理帧数为第一帧数;当大于或等于第一内存阈值,而小于第二内存阈值时,确定x级处理算法为第二拍照算法,并确定x级处理帧数为第二帧数;当大于第二内存阈值时,确定x级处理算法为第三拍照算法,并确定x级处理帧数为第三帧数;利用x级处理算法对原始图像中的x级处理帧数进行处理,得到对应的拍摄图像;保存拍摄图像。本申请通过对内存占用量进行区分,适应性选择不同时长的处理算法和/或不同的处理帧数处理,可以减小内存增量的压力,实现连续快速拍照。
Description
本申请要求于2022年08月25日提交国家知识产权局、申请号为202211026658.0、申请名称为“拍照方法及其相关设备”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
本申请涉及图像技术领域,具体地,涉及一种拍照方法及其相关设备。
随着具有拍照功能的电子设备在生活中的普及,人们使用电子设备进行拍照已经成为了一种日常行为方式。
目前大多数电子设备在进行拍照处理时,为了达到更好的拍照画质效果,多采用的是多帧拍照算法。但是,通常算法处理的帧数越多,对应处理的时长就越长。在此情况下,若用户连续触发多次拍照,后台将出现拍照请求等待,等待处理的拍照次数所累积的待处理的图像将使得内存占用不断扩大,当剩余内存不足以支持拍照处理时,后台将不再响应用户的拍照命令,也即电子设备将无法继续进行拍照。对于这一问题,亟需一种新的方法进行解决。
发明内容
本申请提供了一种拍照方法及其相关设备,通过对内存占用量进行区分,适应性选择不同处理时长的处理算法和/或处理帧数处理,从而可以减小内存增量的压力,实现连续快速拍照。
第一方面,提供了一种拍照方法,应用于包括摄像头的电子设备,所述方法包括:
显示第一界面,所述第一界面包括第一控件;
当检测到对所述第一控件的第一操作时,响应于所述第一操作,采集一次原始图像;
针对第x次拍照次数,确定内存占用量;
当所述内存占用量小于第一内存阈值时,确定x级处理算法为第一拍照算法,并确定x级处理帧数为第一帧数,x为大于1的整数;
当所述内存占用量大于或等于所述第一内存阈值,而小于第二内存阈值时,确定所述x级处理算法为第二拍照算法,并确定所述x级处理帧数为第二帧数;
当所述内存占用量大于所述第二内存阈值时,确定所述x级处理算法为第三拍照算法,并确定所述x级处理帧数为第三帧数,所述第一内存阈值小于所述第二内存阈值;
利用所述x级处理算法对所述原始图像中的所述x级处理帧数进行处理,得到对应的拍摄图像;
保存所述拍摄图像;
其中,当所述第一拍照算法、所述第二拍照算法和所述第三拍照算法相同时,所述第一帧数、所述第二帧数和所述第三帧数逐渐减小;
当所述第一拍照算法、所述第二拍照算法和所述第三拍照算法的处理时长逐渐减小时,所述第一帧数、所述第二帧数和所述第三帧数相同或者逐渐减小。
可选地,第一操作为点击操作。第一操作还可以包括语音指示操作或者其他指示电子设备进行拍照的操作。
本申请实施例提供了一种拍照方法,响应于用户对第一控件的第一操作,电子设备采用摄像头采集一次原始图像,然后,通过确定内存占用量的大小,并根据内存占用量大小的不同,适应性地选择不同处理时长的算法进行处理,比如,在内存占用量较小时选择处理时间较长的算法进行处理,而在内存占用量逐渐增大时选择处理时间较短的算法进行处理,从而可以减小电子设备中的内存增量的压力,提高拍照处理效率,及时响应用户的拍照需求,实现连续快速的拍照。
在第一方面一种可能的实现方式中,所述方法还包括:
针对所述第x次拍照次数,当设备温度大于或等于温度阈值时,利用单帧算法对所述原始图像中的1帧进行处理,得到对应的拍摄图像;
其中,当所述第一拍照算法、所述第二拍照算法和所述第三拍照算法相同时,所述单帧算法的处理时长比所述第三拍照算法的处理时长更短;
当所述第一拍照算法、所述第二拍照算法和所述第三拍照算法的处理时长逐渐减小时,所述单帧算法的处理时长与所述第三拍照算法的处理时长相同,或者,所述单帧算法的处理时长比所述第三拍照算法的处理时长更短。
其中,设备温度指的是电子设备内部的温度。
在该实现方式中,当设备温度高于温度阈值时,可以切换成单帧算法对1帧原始图像进行处理,最大程度的减小电子设备的工作量,减缓热量的增长幅度,降低热量的增量压力,从而使得电子设备在无法工作之前,多实现一些拍照次数,满足用户的拍照需求。
应理解,由于单帧算法可以仅在图像信号处理器中运行处理,因此,单帧算法的处理速度相对比较快,处理时长相对也比较短。
在第一方面一种可能的实现方式中,所述方法还包括:
当所述第一操作对应第1次拍照次数时,针对待拍摄场景进行环境检测;
根据环境检测结果确定所述第1次拍照次数对应的1级处理帧数和1级处理算法;
利用所述1级处理算法,对所述第1次拍照次数对应采集的原始图像中的所述1级处理帧数进行处理,得到对应的拍摄图像。
在该实现方式中,本申请通过对待拍摄场景进行环境检测,可以确定出不同的场景,然后,基于细分出的不同场景,选择不同的算法,以及从对应采集的原始图像中选择不同帧数来进行处理,从而可以适应性提高在每种场景下,第1次拍照次数所对应得到的拍摄图像的质量和效果。
在第一方面一种可能的实现方式中,所述环境检测至少包括照度检测和动态范围检测,所述环境检测结果至少包括照度和动态范围值;
根据环境检测结果确定所述第1次拍照次数对应的1级处理帧数和1级处理算法,
包括:
当所述照度小于照度阈值,所述动态范围值大于或等于动态范围阈值时,确定所述1级处理帧数为a1帧,并确定所述1级处理算法为A1算法;
当所述照度小于所述照度阈值,所述动态范围值小于所述动态范围阈值时,确定所述1级处理帧数为b1帧,并确定所述1级处理算法为B1算法;
当所述照度大于或等于所述照度阈值,所述动态范围值大于或等于所述动态范围阈值时,确定所述1级处理帧数为c1帧,并确定所述1级处理算法为C1算法;
当所述照度大于或等于所述照度阈值,所述动态范围值小于所述动态范围阈值时,确定所述1级处理帧数为d1帧,并确定所述1级处理算法为D1算法;
其中,a1、b1、c1和d1均为大于或等于1的整数。
可选地,可以按照采集顺序从原始图像的全部帧中选择a1帧、b1帧、c1帧或d1帧,或者,也可以从原始图像的全部帧中任意抽取a1帧、b1帧、c1帧或d1帧。
在第一方面一种可能的实现方式中,利用所述1级处理算法,对所述第1次拍照次数对应采集的原始图像中的所述1级处理帧数进行处理,得到对应的拍摄图像,包括:
当所述照度小于照度阈值,所述动态范围值大于或等于动态范围阈值时,利用所述A1算法对所述第1次拍照次数对应采集的原始图像中的a1帧进行处理,得到对应的拍摄图像;
当所述照度小于所述照度阈值,所述动态范围值小于所述动态范围阈值时,利用所述B1算法对所述第1次拍照次数对应采集的原始图像中的b1帧进行处理,得到对应的拍摄图像;
当所述照度大于或等于所述照度阈值,所述动态范围值大于或等于所述动态范围阈值时,利用所述C1算法对所述第1次拍照次数对应采集的原始图像中的c1帧进行处理,得到对应的拍摄图像;
当所述照度大于或等于所述照度阈值,所述动态范围值小于所述动态范围阈值时,利用所述D1算法对所述第1次拍照次数对应采集的原始图像中的d1帧进行处理,得到对应的拍摄图像。
可选地,在动态范围值大于或等于动态范围阈值时,第1次拍照次数对应采集的原始图像中的a1帧,和/或,第1次拍照次数对应采集的原始图像中的c1帧,包括长曝光的原始图像、正常曝光的原始图像和短曝光的原始图像中的至少一种。
在该实现方式中,当处理的原始图像中包括长曝光、正常曝光和短曝光的原始图像时,可以调整图像的动态范围和图像细节,使得获得的拍摄图像呈现的内容更真实,图像质量相对更好。
在第一方面一种可能的实现方式中,在确定所述x级处理算法为第三拍照算法,并确定所述x级处理帧数为第三帧数之后,所述方法还包括:
确定对所述第一控件的第一操作的频率,以及内存水位线与所述内存占用量的差值;
当所述频率满足预设频率条件或所述差值小于预设差值时,确定后台等待的拍照次数;
将所述后台等待的拍照次数各自对应的x级处理算法分别作为初始处理算法;
将最近一次确定的拍照次数对应的x级处理算法作为目标处理算法;
利用所述目标处理算法替换所述后台等待的拍照次数各自的所述初始处理算法;
当后台等待的任意一次拍照次数被调用处理时,利用所述目标处理算法对所述任意一次拍照次数所对应的原始图像中的x级处理帧数进行处理,得到对应的拍摄图像。
在该实现方式中,由于最近一次确定出的拍照次数所对应的处理算法是根据当前的内存情况确定出来的,所以等待的拍照次数的处理算法随之调整,可控制内存堆积程度,减小内存占用量,实现连续快速拍照。
在第一方面一种可能的实现方式中,所述第一界面是指拍照界面,所述第一控件是指用于指示拍照的控件。
在第一方面一种可能的实现方式中,所述第一界面是指录像界面,所述第一控件是指用于指示抓拍的控件。
第二方面,提供了一种图像处理装置,该装置包括用于执行以上第一方面或第一方面的任意可能的实现方式中各个步骤的单元。
第三方面,提供一种电子设备,包括一个或多个处理器和存储器;
所述存储器与所述一个或多个处理器耦合,所述存储器用于存储计算机程序代码,所述计算机程序代码包括计算机指令,所述一个或多个处理器调用所述计算机指令以使得所述电子设备执行以上第一方面或第一方面的任意可能的实现方式中提供的方法。
第四方面,提供了一种芯片,所述芯片应用于电子设备,所述芯片包括一个或多个处理器,所述处理器用于调用计算机指令以使得所述电子设备执行第一方面或第一方面的任意可能的实现方式中提供的方法。
第五方面,提供了一种计算机可读存储介质,所述计算机可读存储介质存储有计算机程序代码,当所述计算机程序代码被电子设备运行时,使得所述电子设备执行第一方面或第一方面的任意可能的实现方式中提供的方法。
第六方面,提供了一种计算机程序产品,所述计算机程序产品包括:计算机程序代码,当所述计算机程序代码被电子设备运行时,使得所述电子设备执行第一方面或第一方面的任意可能的实现方式中提供的方法。
本申请实施例提供了一种拍照方法及其相关设备,在响应于用户对第一控件的第一操作时,电子设备采用摄像头采集一次原始图像。当第一操作对应第1次拍照次数时,针对待拍摄场景进行环境检测,然后,根据环境检测结果确定第1次拍照次数对应的1级处理帧数和1级处理算法。而当第一操作对应第2次拍照次数及后续其他次数的拍照次数时,先针对内存占用量进行检测,将确定出的内存占用量大小与预设的内存阈值进行比较,然后,根据不同的内存占用量大小,选择不同的处理帧数和处理算法。
在针对第1次拍照次数进行处理时,由于针对环境检测结果对场景进行了细分,根据不同场景选择了不同的算法和帧数,从而可以适应性提高在每种场景下,第1次拍照次数对应得到的拍摄图像的质量和效果。
而在针对第2次拍照次数及后续其他次数进行处理时,由于针对内存占用量进行细分,根据不同内存占用量选择了不同的算法和帧数,在内存占用量越大的情况下选
择处理时长越短的算法和/或减小处理帧数,从而减小了内存增量的压力,提高处理效率,实现快速拍照的需求。
图1是本申请实施例适用的一种场景示意图;
图2是本申请实施例适用的另一种场景示意图;
图3是本申请实施例提供的一种拍照方法的流程示意图;
图4是本申请实施例提供的一种点击操作的界面示意图;
图5是本申请实施例提供的另一种点击操作的界面示意图;
图6是本申请实施例提供的一种图像处理的进程示意图;
图7是本申请实施例提供的一种环境检测的流程示意图;
图8是本申请实施例提供的一种表格;
图9是本申请实施例提供的一种根据内存占用量确定相应帧数和算法的流程示意图;
图10是本申请实施例提供的一种内存占用量的增长示意图;
图11是本申请实施例提供的另一种拍照方法的流程示意图;
图12是本申请实施例提供的又一种拍照方法的流程示意图;
图13是本申请实施例提供的又一种拍照方法的流程示意图;
图14是本申请实施例提供的另一种表格;
图15是本申请实施例提供的又一种表格;
图16是本申请实施例提供的又一种表格;
图17是图16对应的处理进程示意图;
图18是本申请实施例提供的一种电子设备的显示界面的示意图;
图19是本申请实施例提供的另一种电子设备的显示界面的示意图;
图20是本申请实施例提供的一种电子设备的硬件结构示意图;
图21是本申请实施例提供的一种电子设备的软件结构示意图;
图22为本申请实施例提供的一种图像处理装置的结构示意图;
图23为本申请实施例提供的一种芯片的结构示意图。
下面将结合附图,对本申请中的技术方案进行描述。
在本申请实施例的描述中,除非另有说明,“/”表示或的意思,例如,A/B可以表示A或B;本文中的“和/或”仅仅是一种描述关联对象的关联关系,表示可以存在三种关系,例如,A和/或B,可以表示:单独存在A,同时存在A和B,单独存在B这三种情况。另外,在本申请实施例的描述中,“多个”是指两个或多于两个。
以下,术语“第一”、“第二”仅用于描述目的,而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此,限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个该特征。
首先,对本申请实施例中的部分用语进行解释说明,以便于本领域技术人员理解。
1、RGB(red,green,blue)颜色空间,指的是一种与人的视觉系统结构相关的颜色模型。根据人眼睛的结构,将所有颜色都当作是红色、绿色和蓝色的不同组合。
2、YUV颜色空间,指的是一种颜色编码方法,Y表示亮度,U和V表示的则是色度。上述RGB颜色空间着重于人眼对色彩的感应,YUV颜色空间则着重于视觉对亮度的敏感程度,RGB颜色空间和YUV颜色空间可以互相转换。
3、像素值,指的是位于RGB颜色空间的彩色图像中每个像素对应的一组颜色分量。例如,每个像素对应一组三基色分量,其中,三基色分量分别为红色分量R、绿色分量G和蓝色分量B。
4、拜耳格式(bayer pattern)彩色滤波阵列(color filter array,CFA),图像由实际的景物转换为图像数据时,通常是图像传感器分别接收红色通道信号、绿色通道信号和蓝色通道信号,三个通道信号的信息,然后将三个通道信号的信息合成彩色图像,但是,这种方案中每个像素位置处都对应需要三块滤镜,价格昂贵且不好制作,因此,可以在图像传感器表面覆盖一层彩色滤波阵列,以获取三个通道信号的信息。拜耳格式彩色滤波阵列指的是滤镜以棋盘格式进行排布。例如,该拜耳格式彩色滤波阵列中的最小重复单元为:一个获取红色通道信号的滤镜、两个获取绿色通道信号的滤镜、一个获取蓝色通道信号的滤镜以2×2的方式排布。
5、拜耳格式图像(bayer image),即基于拜耳格式彩色滤波阵列的图像传感器输出的图像。该图像中的多种颜色的像素以拜耳格式进行排布。其中,拜耳格式图像中的每个像素仅对应一种颜色的通道信号。示例性的,由于人的视觉对绿色较为敏感,所以,可以设定绿色像素(对应绿色通道信号的像素)占全部像素的50%,蓝色像素(对应蓝色通道信号的像素)和红色像素(对应红色通道信号的像素)各占全部像素的25%。其中,拜耳格式图像的最小重复单元为:一个红色像素、两个绿色像素和一个蓝色像素以2×2的方式排布。拜耳格式图像即为位于RAW域的图像。
6、拍摄参数
拍摄参数可包括快门、曝光时间、光圈值(aperture value,AV)、曝光值(exposure value,EV)和感光度ISO。以下分别进行介绍。
快门是控制光线进入摄像头时间长短,以决定图像曝光时间的装置。快门保持在开启状态的时间越长,进入摄像头的光线越多,图像对应的曝光时间越长。相反,快门保持在开启状态的时间越短,进入摄像头的光线越少,图像对应的曝光时间越短。
曝光时间是指为了将光投射到摄像头的感光材料的感光面上,快门所要打开的时间。曝光时间由感光材料的感光度和感光面上的照度确定。曝光时间越长,进入摄像头的光越多,曝光时间越短,进入摄像头的光越少。因此,暗光场景下需要长的曝光时间,逆光场景下需要短的曝光时间。
光圈值(f值),是摄像头中的镜头(lens)的焦距与镜头通光直径的比值。光圈值越大,进入摄像头的光线越多。光圈值越小,进入摄像头的光线越少。
曝光值,是曝光时间和光圈值组合起来表示摄像头的镜头通光能力的一个数值。曝光值可以定义为:
其中,N为光圈值;t为曝光时间,单位为秒。
ISO,用于衡量底片对于光的灵敏程度,即感光度或增益。对于不敏感的底片,需
要更长的曝光时间以达到跟敏感底片亮度相同的成像。对于敏感的底片,需要较短的曝光时间以达到与不敏感的底片亮度相同的成像。
拍摄参数中,快门、曝光时间、光圈值、曝光值和ISO,电子设备可通过算法实现自动对焦(auto focus,AF)、自动曝光(automatic exposure,AE)、自动白平衡(auto white balance,AWB)中的至少一项,以实现这些拍摄参数的自动调节。
自动对焦是指电子设备通过调整聚焦镜头的位置获得最高的图像频率成分,以得到更高的图像对比度。其中,对焦是一个不断积累的过程,电子设备比较镜头在不同位置下拍摄的图像的对比度,从而获得图像的对比度最大时镜头的位置,进而确定对焦的焦距。
自动曝光是指电子设备根据可用的光源条件自动设置曝光值。电子设备可根据当前所采集图像的曝光值,自动设定快门速度和光圈值,以实现自动设定曝光值。
物体颜色会因投射光线颜色产生改变,在不同光线颜色下电子设备采集出的图像会有不同的色温。白平衡与周围光线密切相关。无论环境光线如何,电子设备的摄像头能识别出白色,并以白色为基准还原其他颜色。自动白平衡可实现电子设备根据光源条件调整图像颜色的保真程度。3A即自动对焦、自动曝光和自动白平衡。
示例性的,曝光值的取值可以为-24、-4、-3、-2、-1、0、1、2、3、4、24中的任意一项。
EV0对应的曝光图像,用于指示电子设备通过算法实现曝光时,通过确定的曝光值0来捕获的曝光图像。EV-2对应的曝光图像,用于指示电子设备通过算法实现曝光时,通过确定的曝光值-2来捕获的曝光图像。EV1对应的曝光图像,用于指示电子设备通过算法实现曝光时,通过确定的曝光值1来捕获的曝光图像。其他依次类推,在此不再赘述。
其中,曝光值每增加1将改变一档曝光,也就是将曝光量(指物体表面某一面元接收的光照度在时间t内的积分)增加一倍,比如将曝光时间或光圈面积增加一倍。那么,曝光值的增加将对应于更慢的快门速度和更小的f值。由此可知,EV0相对于EV-2,曝光值增加了2,改变了两档曝光;同理,EV1相对于EV0,曝光值增加了1,改变了一档曝光。
此处,当曝光值EV等于0时,该曝光值通常为当前照明条件下的最佳曝光值。相应的,在EV0的条件下电子设备对应获取的曝光图像,为当前照明条件下的最佳曝光图像,该最佳曝光图像也可以称为参考曝光图像。
应理解,该“最佳”曝光图像指的是给定的电子设备通过算法确定的曝光图像,当电子设备不同、算法不同或当前照明条件不同时,确定的最佳曝光图像不同。
以上是对本申请实施例所涉及的名词的简单介绍,以下不再赘述。
随着具有拍照功能的电子设备在生活中的普及,人们使用电子设备进行拍照已经成为了一种日常行为方式。
大多数电子设备在进行拍照处理时,为了达到更好的拍照画质,多采用的是多帧拍照算法。但是,通常算法处理的帧数越多,对应处理的时长就越长。例如,处理的图像数量为9帧时,从用户按下电子设备显示界面上显示的拍摄键到图像处理完成后
供用户查阅的时长,也即算法端到端的时长约在2.5s左右;处理的图像数量大于9帧时,算法端到端的时长将大于2.5s。
而电子设备进行拍照处理时,当上一次拍照处理结束后,电子设备才会进行下一次的拍照处理。基于该单线程的处理进程,若用户连续触发了多次拍照命令,且触发拍照的间隔时间小于上述2.5s时,电子设备则会因为每次拍照算法进行拍照处理的时间较长,无法及时响应,导致后台等待的拍照次数逐渐增多。随着等待的拍照次数越来越多,后台累积存储的待处理的数据也越来越多,不断扩大了内存占用,在当剩余内存不足以支持拍照处理时,后台将不再响应用户所触发的拍照命令。
这样用户无论怎样点击拍摄键,都感觉拍摄键按不动,此时,电子设备相当于无法正常进行拍照,导致给用户带来了非常差的体验。
有鉴于此,本申请实施例提供了一种拍照方法,响应于用户对第一控件的第一操作,电子设备采用摄像头采集一次原始图像,然后,通过确定内存占用量的大小,并根据内存占用量大小的不同,适应性地选择不同处理时长的算法进行处理,比如,在内存占用量较小时选择处理时间较长的算法进行处理,而在内存占用量逐渐增大时选择处理时间较短的算法进行处理,从而可以减小电子设备中的内存增量的压力,提高拍照处理效率,及时响应用户的拍照需求,实现连续快速的拍照。
本申请实施例提供的拍照方法可以适用于各种电子设备。
在本申请的一些实施例中,该电子设备可以为运动相机、数码相机等各种摄像装置、手机、平板电脑、可穿戴设备、车载设备、增强现实(augmented reality,AR)/虚拟现实(virtual reality,VR)设备、笔记本电脑、超级移动个人计算机(ultra-mobile personal computer,UMPC)、上网本、个人数字助理(personal digital assistant,PDA)等,或者可以为其他能够进行图像显示的设备或装置,对于电子设备的具体类型,本申请实施例不作任何限制。
下文以电子设备为手机为例进行说明。手机可以包括显示屏、一个或多个摄像头。其中,手机在一个待拍摄场景下,采用一个或多个摄像头拍摄,以得到不同特点的拍摄图像。显示屏用于显示拍摄后处理得到的拍摄图像。
其中,待拍摄场景指的是用户所期望拍摄的场景。若用户使用电子设备的摄像头对准一包括某物体的场景,则该包括某物体的场景即为待拍摄场景。应理解,待拍摄场景并非特指某一个特定场景,而是跟随摄像头的指向所实时对准的场景。
基于此,结合图1和图2对本申请实施例提供的应用场景进行介绍。图1示出了本申请实施例适用的一种场景示意图,图2示出了本申请实施例适用的另一种场景示意图。
应用场景一:拍照场景
如图1中的(a)所示,该电子设备100上安装有相机应用。此外,还安装有多种应用,本申请实施例对此不进行任何限制。示例性的,响应于用户针对相机应用的点击操作,当电子设备100运行相机应用时,电子设备100显示如图1中的(b)所示的拍摄界面。
其中,该拍摄界面包括相机应用的多种拍摄模式,如,大光圈模式41、夜景模式
42、人像模式43、拍照模式44、录像模式45等。该拍摄界面还包括第一控件,第一控件为拍摄键50,拍摄键50用于指示当前的拍摄模式,例如,在打开相机的情况下,拍摄键50默认指示当前的拍摄模式为拍照模式44。
如图1中的(b)所示,该拍摄界面中还包括取景窗口60,取景窗口60可用于实时显示拍照前的预览图像。拍摄界面中还显示有第二控件,第二控件为变焦选项61。用户可以在变焦选项61中选择当前需要的变焦倍数,例如,0.5倍、2倍或50倍变焦倍数等。
结合图1中的(a)和(b)所示,在用户打开相机,当前拍摄模式默认为拍照模式的情况下,用户将手机上的摄像头对准待拍摄场景后,取景窗口60可以实时显示拍照前的预览图像;然后,响应于用户对拍摄键50的连续多次点击操作,电子设备可以调用本申请提供的拍照方法,根据内存的大小,适应性选择不同的拍照算法进行处理,从而可以减小内存增量的压力,提高拍照处理的效率,及时响应用户的拍照需求,实现快速拍照,避免出现相关技术中后台无法响应拍照命令的情况。
应用场景二:录像中抓拍的场景
如图2中的(a)所示,该电子设备100上安装有相机应用。示例性的,响应于用户针对相机应用的点击操作,当电子设备100运行相机应用时,电子设备显示如图2中的(b)所示的拍摄界面。
其中,针对该拍摄界面的描述与针对图1中的(b)的描述的相同,在此不再赘述。在打开相机的情况下,默认拍摄键50指示当前的拍摄模式为拍照模式44。示例性的,响应于用户向左的滑动操作,该拍摄模式可以从拍照模式44切换成录像模式45。
如图2中的(c)所示,当拍摄模式切换成录像模式45时,该拍摄界面中包括的取景窗口60可用于实时显示录像前的预览图像。示例性的,响应于用户针对拍摄键50的点击操作,当电子设备100开始进行录像时,电子设备可以显示如图2中的(d)所示的录像界面。
如图2中的(d)所示,该录像界面中可以显示当前拍摄的视频画面(如图中所示的提着购物袋走路的男士)、拍摄进度(如图中的所示的00:12)、状态图标(如图中拍摄进度前面的拍摄图标用于表示当前处于拍摄状态),以及第三控件,例如,抓拍控件70,当然,该拍摄界面还可以包括其他控件,例如:变焦选项61、结束控件80、暂停/继续控件90等,本申请实施例对此不进行限定。
其中,暂停/继续控件90用于在视频拍摄过程中显示暂停图标(参考图2中的(d)所示),还用于单击该暂停图标时暂停当前视频拍摄过程;以及在视频拍摄暂停时显示拍摄图标,用户点击该拍摄图标时继续当前视频拍摄过程(图2中未示出)。结束控件80用于结束当前视频拍摄过程。抓拍控件70用于在不暂停当前视频拍摄过程且不结束当前视频拍摄过程的情况下抓拍照片。
结合图2中的(c)和(d)所示,当用户在拍摄提着购物袋走路的男士的过程中,想要抓拍男士的照片时,用户可以点击拍摄界面中的抓拍控件70,获取抓拍照片。该抓拍照片作为图像存储在电子设备的图库中。若用户连续多次点击该录像界面中的抓拍控件70,电子设备可以调用本申请提供的拍照方法,根据内存的大小,适应性选择不同的拍照算法进行处理,从而可以减小内存增量的压力,提高抓拍处理的效率,及
时响应用户的拍照需求,实现快速抓拍,避免出现相关技术中后台无法响应拍照命令的情况。
上述为对应用场景的两种举例说明,并不对本申请的应用场景作任何限定。
下面结合图3至图19对本申请实施例提供的拍照方法进行详细描述。
图3是本申请提供的一种拍照方法的流程示意图。该拍照方法应用于包括摄像头的电子设备,例如手机。如图3所示,该方法的步骤可以包括以下S11至S19。
S11、电子设备显示第一界面,第一界面包括第一控件。
可选地,如图1中的(b)所示,第一界面可以为拍摄界面,第一控件用于指示拍摄界面中的拍摄键50。
可选地,如图2中的(d)所示,第一界面可以为录像界面,第一控件用于指示录像界面中的抓拍控件70。
当然,第一界面也可以其他界面,第一控件相应地的可以为其他界面上用于指示拍照的控件,本申请实施例对此不进行任何限制。
S12、当检测到用户对第一控件的第一操作时,响应于第一操作,电子设备利用摄像头采集一次原始图像。
可选地,第一操作可以是对第一控件的点击操作,还可以是语音指示操作或者其他指示电子设备进行拍照的操作,本申请实施例对此不进行任何限制。
其中,点击操作指的是用户在较短的时间触摸第一控件后离开的行为。
例如,图4示出了一种点击操作的界面示意图,如图4中的(a)和(b)所示,用户的手指按下拍摄键50后抬起记为一次点击操作,从手指按下拍摄键50到手指抬起离开拍摄键50之间的时间为一次点击操作的时长,通常该时长非常短。
又例如,图5示出了另一种点击操作的界面示意图,如图5中的(a)和(b)所示,用户的手指按下抓拍控件70后抬起记为一次点击操作,从手指按下抓拍控件70到手指抬起离开抓拍控件70之间的时间为一次点击操作的时长。
结合图5,示例性的,在打开相机进行录像的情况下,当用户对抓拍控件70进行一次点击操作时,响应于该次点击操作,电子设备采用摄像头采集一次原始图像。
当用户对抓拍控件70进行多次点击操作时,响应于该多次点击操作,电子设备可以采用摄像头采集多次原始图像。应理解,当用户对抓拍控件70进行多次点击操作时,第一次点击操作与第1次拍照次数对应,第二次点击操作与第2次拍照次数对应,第三次点击操作与第3次拍照次数对应,后续依次类推不再赘述。
此处,需要说明的是,当相机关闭后被重新打开时,拍照次数需要重新从1开始累计。或者,当相邻两次点击操作的间隔时长超过第一预设间隔时长且后台没有等待处理的数据时,拍照次数也需要重新从1开始累计。其中,第一预设间隔时长至少大于一次拍照的处理时长,第一预设间隔时长的具体时长可以根据需要进行设置和修改,本申请实施例对此不进行任何限制。
示例性的,当用户第一次打开相机,对拍摄键50进行第一次点击操作后,响应于该次点击操作,电子设备采用摄像头采集了一次原始图像,该次拍照次数对应为第1次拍照次数。然后,用户关闭又重新打开了相机,并对拍摄键50进行了另一次点击操
作,响应于当前的点击操作,电子设备采用摄像头又采集一次原始图像,此时,由于相机是重新开启的,所以,该次拍照次数应该重新记录为第1次拍照次数。
示例性的,图6提供了一种图像处理的进程示意图。
如图6所示,当用户打开相机应用,对拍摄键50进行第一次点击操作K1时,响应于该次点击操作K1,电子设备采用摄像头采集一次原始图像,该次拍照次数可记为第1次拍照次数。但是,隔了较长一段时间后,用户才对拍摄键50进行下一次点击操作K1',响应于该次点击操作K1',电子设备采用摄像头再采集一次原始图像,由于点击操作K1和点击操作K1'之间的间隔时间超过了第一预设间隔时长,且后台没有其他等待处理的数据,则对应拍照次数需从1重新开始累计,也即该次拍照次数重新记为第1次拍照次数;当用户对拍摄键后续再进行点击操作K2'时,响应于该次点击操作K2',电子设备采用摄像头又采集一次原始图像,由于点击操作K2'与点击操作K1'之间的间隔时间小于第一预设间隔时长,则该次拍照次数可以记为第2次拍照次数。当用户对拍摄键50后续再进行点击操作K3'时,响应于该次点击操作K3',电子设备采用摄像头又采集一次原始图像,由于点击操作K3'与点击操作K2'之间的间隔时间小于第一预设间隔时长,则该次拍照次数可以记为第3次拍照次数;后续依次类推,在此不再赘述。
需要说明的是,若相邻两次点击操作的间隔时长虽然超过了第一预设间隔时长,但是后台还有等待处理的数据时,说明用户之前进行了很多次点击操作,此时还有一些数据由于点击频率太高,后台处理的较慢没有来得及处理,此时,拍照次数不能重新从1开始累计,应保持原有的记录次数。
当电子设备包括多个摄像头时,响应于一次点击操作,可以采用一个或多个摄像头来采集原始图像,每个摄像头可以采集1帧或多帧原始图像。也就是说,每次可以采集1帧或多帧原始图像,具体帧数可以根据需要进行设置和修改,本申请实施例对此不进行任何限制。
还应理解,原始图像可以为位于RAW域的图像,但是,当摄像头为黑白摄像头时,原始图像可以为灰阶图像;当摄像头为多光谱摄像头时,原始图像可以为包括多个颜色通道信号的多光谱图像,原始图像的格式或称为特点随摄像头改变而改变,本申请实施例对此不进行任何限制。
此外,针对一次采集的原始图像,还可以包括长曝光的原始图像、正常曝光的原始图像和短曝光的原始图像中的至少一种。
其中,长曝光的原始图像指的是拍摄时经过较长时间曝光所得到的图像,短曝光的原始图像指的是拍摄时经过较短时间曝光所得到的图像,应理解,“长”和“短”都是相对“正常”曝光的时间而言的,当正常曝光的原始图像对应的曝光时间不同时,长曝光和短曝光也随之改变。
当多帧原始图像包括长曝光的原始图像,和/或正常曝光的原始图像,和/或短曝光的原始图像时,长曝光的原始图像的帧数、正常曝光的原始图像的帧数,以及短曝光的原始图像的帧数都可以根据需要进行选择和修改,本申请实施例对此不进行任何限制。
S13、当该第一操作对应第1次拍照次数时,针对待拍摄场景进行环境检测。
其中,环境检测至少包括照度(lightness value,LV)检测和动态范围检测,当然,环境检测还可以包括检测项目,具体可以根据需要进行设置和修改,本申请实施例对此不进行任何限制。
应理解,照度检测指的是对待拍摄场景中的环境亮度进行估计。照度指的是物体被照明的程度,具体指的是物体单位面积上所接受可见光的能量,简称照度,单位是勒克斯(Lux)。黑天的街道、灯光昏暗的室内等一般是低照度的,相对的,阳光下的操场、聚光灯下的舞台等可称为中高照度。如果待拍摄场景的照度比较低,则使用电子设备对待拍摄场景进行拍摄时,会模糊不清,效果比较差。如果待拍摄场景的照度比较高,则使用电子设备对待拍摄场景进行拍摄时,会比较清楚,拍摄效果好。
此处,例如,可以在电子设备中设置光电传感器,利用光电传感器检测待拍摄场景的照度。或者,还可以根据摄像头的曝光参数,比如曝光时间、感光度、光圈等参数,以及收到的响应值来通过以下公式计算得到照度:
其中,Exposure为曝光时间,Aperture为光圈大小,ISO为感光度,Luma为XYZ颜色空间中,Y的平均值。
在曝光参数相同的情况下,响应值越高,表明待拍摄场景的照度也越高,由此,计算得到的照度的值也越大。
应理解,动态范围(dynamic range)检测指的是对待拍摄场景的动态范围值进行检测。动态范围值,用于表示通过图像中像素的亮度范围,也就是图像从“最亮”像素到“最暗”像素之间灰度划分的等级数。一个图像的动态范围值越大,它所能表示的亮度层次越丰富,图像的视觉效果越逼真。动态范围值的表达式可以是:
其中,dynamic range为动态范围值,bright为“最亮”像素的亮度,dark为“最暗”像素的亮度。动态范围的单位为档(stop)。
在此,根据检测到的动态范围值的大小,可以判断待拍摄场景是低动态(low dynamic range,LDR)场景还是高动态范围(high dynamic range,HDR)场景。其中,低动态场景指的是环境光线强度全部较低或全部较高,动态范围比较窄的场景;高动态范围场景指的是一部分光线强度较低,另一部分光线强度较高,动态范围比较宽的场景。
S14、根据环境检测结果确定第1次拍照次数对应的1级处理帧数和1级处理算法。
结合所述的照度检测和动态范围检测方法,图7为本申请实施例提供的一种环境检测的流程示意图。
其中,由于上述S13包括:当该第一操作对应第1次拍照次数时,针对待拍摄场景进行环境检测,环境检测包括照度检测,因此,相应地,环境检测结果包括检测出的照度;同理,由于环境检测还包括动态范围检测,相应地,环境检测结果还包括检测出的动态范围值。其中,照度检测和动态范围检测的方法可以参考上述描述,在此不再赘述。
结合检测出的环境检测结果中的照度和动态范围值,如图7所示,上述S14可以包括
以下S141至S146。
S141、确定检测出的照度是否小于照度阈值。
其中,照度阈值可以根据需要进行设置和修改,本申请实施例对此不进行任何限制。
S142、确定检测出的动态范围值是否小于动态范围阈值。
其中,动态范围阈值可以根据需要进行设置和修改,本申请实施例对此不进行任何限制。
S143、如图8所示,当照度小于照度阈值,动态范围值大于或等于动态范围阈值时,或者说当待拍摄场景为低亮、高动态范围的场景时,确定第1次拍照次数对应的1级处理帧数为a1帧,并确定A1算法为1级处理算法。
其中,a1为第1次拍照次数所对应采集的原始图像中的全部帧或部分帧。
S144、如图8所示,当照度小于照度阈值,动态范围值小于动态范围阈值时,或者说当待拍摄场景为低亮、低动态范围的场景时,确定第1次拍照次数对应的1级处理帧数为b1帧,并确定B1算法为1级处理算法。
其中,b1为第1次拍照次数所对应采集的原始图像中的全部帧或部分帧。
S145、如图8所示,当照度大于或等于照度阈值,动态范围值也大于或等于动态范围阈值时,或者说待拍摄场景为高亮、高动态范围的场景时,确定第1次拍照次数对应的1级处理帧数为c1帧,并确定C1算法为1级处理算法。
其中,c1为第1次拍照次数所对应采集的原始图像中的全部帧或部分帧。
S146、如图8所示,当照度大于或等于照度阈值,动态范围值小于动态范围阈值时,或者说待拍摄场景为高亮、低动态范围的场景时,确定第1次拍照次数对应的1级处理帧数为d1帧,并确定D1算法为1级处理算法。
其中,d1为第1次拍照次数所对应采集的原始图像中的全部帧或部分帧。
应理解,A1、B1、C1和D1用于指代算法的名字,A1算法、B1算法、C1算法和D1算法可以指示相同的算法,也可以指示不同的算法,具体可以根据需要进行设置和修改,本申请实施例对此不进行任何限制。
同时,a1、b1、c1和d1用于指代处理帧数,a1、b1、c1和d1可以指示相同的帧数,也可以指示不同的帧数,具体帧数可以根据需要进行设置和修改,本申请实施例对此不进行任何限制。
其中,可以按照采集顺序从原始图像的全部帧中选择a1帧、b1帧、c1帧或d1帧进行处理,或者,也可以从原始图像的全部帧中任意抽取a1帧、b1帧、c1帧或d1帧进行处理,或者,还可以以指定摄像头的方式来选择一个或几个摄像头所采集的图像来作为a1帧、b1帧、c1帧或d1帧进行处理。当然,选择a1帧、b1帧、c1帧或d1帧的方式可以相同,也可以不同,具体可以根据需要进行设置和修改,本申请实施例对此不进行任何限制。
此外,针对待拍摄场景为高动态范围的场景,确定出的a1帧或c1帧,可以包括长曝光的原始图像、正常曝光的原始图像和短曝光的原始图像中的至少一种,或者可以额外增加长曝光的原始图像、正常曝光的原始图像和短曝光的原始图像中的至少一种,具体可以根据需要进行设置和修改,本申请实施例对此不进行任何限制。
应理解,处理的原始图像中包括长曝光、正常曝光和短曝光的原始图像时,可以调整图像的动态范围和图像细节,使得获得的拍摄图像呈现的内容更真实,图像质量相对更好。
需要说明的是,上述S141至S146的顺序仅为其中一种示例,在进行环境检测时,也可以先针对待拍摄场景进行照度检测,确定检测出的照度是否小于照度阈值,然后,再针对待拍摄场景进行动态范围检测,确定检测出的动态范围值是否小于动态范围阈值,或者,还可以将两者顺序进行调换。具体执行过程可以根据需要进行调整,此外,在此基础上还可以增加其他步骤或者删除一些步骤,本申请实施例对此不进行任何限制。
S15、如图3所示,利用1级处理算法对1级处理帧数进行处理,得到对应的拍摄图像。
结合上述S14中的示例,当照度小于照度阈值,动态范围值大于或等于动态范围阈值时,利用确定出的A1算法对第1次拍照次数对应采集的原始图像中的a1帧进行处理,得到对应的拍摄图像。
当照度小于照度阈值,动态范围小于动态范围阈值时,利用确定出的B1算法对第1次拍照次数对应采集的原始图像中的b1帧进行处理,得到对应的拍摄图像。
当照度大于或等于照度阈值,动态范围值也大于或等于动态范围阈值时,利用确定出的C1算法对第1次拍照次数对应采集的原始图像中的c1帧进行处理,得到对应的拍摄图像。
当照度大于或等于照度阈值,动态范围小于动态范围阈值时,利用确定出的D1算法对第1次拍照次数对应采集的原始图像中的d1帧进行处理,得到对应的拍摄图像。
例如,当第一操作为第1次拍照次数时,对应采集了一次原始图像,该次一共采集了9帧原始图像,当照度小于照度阈值,动态范围值大于或等于动态范围阈值时,利用确定出的A1算法可以对该9帧中的6帧(此时a1=6)进行处理,得到对应的拍摄图像。
当照度小于照度阈值,动态范围小于动态范围阈值时,利用确定出的B1算法可以对该9帧中的5帧(此时b1=5)进行处理,得到对应的拍摄图像。
当照度大于或等于照度阈值,动态范围值也大于或等于动态范围阈值时,利用确定出的C1算法可以对该9帧中的4帧(此时c1=4)进行处理,得到对应的拍摄图像。
当照度大于或等于照度阈值,动态范围小于动态范围阈值时,利用确定出的D1算法可以对该9帧中的3帧(此时d1=3)进行处理,得到对应的拍摄图像。
应理解,本申请通过对待拍摄场景进行环境检测,可以确定出不同的场景,然后,基于细分出的不同场景,选择不同的算法,以及从对应采集的原始图像中选择不同帧数来进行处理,从而可以适应性提高在每种场景下,第1次拍照次数所对应得到的拍摄图像的质量和效果。
S16、当该第一操作对应第x次拍照次数时,确定内存占用量,x为大于1的整数。
应理解,内存占用量指的是内存中已存储数据的用量。由于,摄像头每采集一次原始图像或得到一帧拍摄图像将增加部分内存占用,也即,内存占用量将相对增大。
那么,随着拍照次数的逐渐增多,内存占用量将会越来越大,这样,当内存占用量大到一定程度时,将可能出现相关技术中无法响应拍照命令的情况。为了避免该情况的出现,本申请将针对内存占用量进行实时监控,对不同的内存占用量,可以设置不同的处理算法和/或不同的处理帧数,以减小内存增量的压力。其中,内存占用量可以通过获取电子设备内部的数据得到。
S17、针对该内存占用量,确定该次拍照次数对应的x级处理帧数和x级处理算法。
其中,x级处理算法用于指代第x次拍照次数对应处理时所利用的算法,x级处理帧数用于指代第x次拍照次数对应处理时所利用的原始图像的帧数。例如,当x=2时,第2次拍照次数对应2级处理帧数和2级处理算法;当x=3,第3次拍照次数对应3级处理帧数和3级处理算法;后续处理帧数和处理算法的名称依次类推,不再赘述。
示例性的,图9示出了一种根据内存占用量确定相应处理帧数和处理算法的流程示意图。
如图9所示,上述S17可以包括以下S171至S175。
S171、针对第x次拍照次数,确定内存占用量是否小于第一内存阈值。
S172、当内存占用量小于第一内存阈值,确定x级处理算法为第一拍照算法,并确定x级处理帧数为第一帧数。
其中,第一帧数指的是第x次拍照次数对应采集的原始图像中的全部帧或部分帧。
S173、当内存占用量大于或等于第一内存阈值时,继续确定内存占用量是否小于第二内存阈值。
S174、当内存占用量小于第二内存阈值,确定x级处理算法为第二拍照算法,并确定x级处理帧数为第二帧数。
其中,第二帧数指的是第x次拍照次数对应采集的原始图像中的全部帧或部分帧。
S175、当内存占用量大于或等于第二内存阈值时,确定x级处理算法为第三拍照算法,并确定x级处理帧数为第三帧数。
其中,第三帧数指的是第x次拍照次数对应采集的原始图像中的全部帧或部分帧。
应理解,第一内存阈值小于第二内存阈值,第二内存阈值小于内存总量。此处,第一内存阈值和第二内存阈值的大小可以根据需要进行设置和修改,本申请实施例对此不进行任何限制。
应理解,第一拍照算法、第二拍照算法和第三拍照算法可以相同也可以不同。
作为一种示例,当第一拍照算法、第二拍照算法和第三拍照算法相同时,第一帧数、第二帧数和第三帧数逐渐减小。
作为另一种示例,当第一拍照算法、第二拍照算法和第三拍照算法不同时,第一拍照算法、第二拍照算法和第三拍照算法的处理时长逐渐减小,此时,第一帧数、第二帧数和第三帧数的大小可以相同,或者逐渐减小。
其中,在上述两种情况中,第二帧数相对于第一帧数的减小量与第三帧数相对于第二帧数的减小量可以相同,也可以不同,具体可以根据需要进行设置和修改,本申请实施例对此不进行任何限制。
在上述第二种情况中,第一拍照算法、第二拍照算法和第三拍照算法所包括的处理步骤逐渐减少,或者说包括的处理步骤的复杂程度逐渐降低,从而使得处理时长逐渐减小。第二拍照算法相对于第一拍照算法所减少的处理时长与第三拍照算法相对于第二拍照算法所减少的处理时长可以相同,也可以不同,具体可以根据需要进行设置和修改,本申请实施例对此不进行任何限制。
在本申请中,可以按照采集顺序从相应拍照次数采集的原始图像中的全部帧中选择第一帧数、第二帧数或第三帧数来进行处理,或者,也可以从原始图像的全部帧中任意抽取第一帧数、第二帧数或第三帧数来进行处理,或者,还可以以指定摄像头的方式来选择一
个或几个摄像头所采集的图像作为第一帧数、第二帧数或第三帧数的图像来进行处理。当然,选择第一帧数、第二帧数或第三帧数的方式可以相同,也可以不同,具体可以根据需要进行设置和修改,本申请实施例对此不进行任何限制。
此外,选择出的第一帧数的原始图像可以包括长曝光的原始图像、正常曝光的原始图像和短曝光的原始图像中的至少一种,选择出的第二帧数的原始图像可以包括长曝光的原始图像、正常曝光的原始图像和短曝光的原始图像中的至少一种,同理,选择出的第三帧数的原始图像也可以包括长曝光的原始图像、正常曝光的原始图像和短曝光的原始图像。其中,无论第一帧数、第二帧数和第三帧数是否相同,第一帧数、第二帧数和第三帧数所包括的长曝光的原始图像、短曝光的原始图像的数量可以逐渐减少。这样,在内存占用量增大时,将放弃动态范围的保持,尽可能地保证实现用户快速拍照的需求。
应理解,上述S171至S175中,仅设定了两个内存阈值,分别为第一内存阈值和第二内存阈值,通过该两个内存阈值,可以将内存大小划分成三个区间范围。当内存占用量满足其中一个区间时,相应地,利用该区间范围所对应的算法和帧数进行处理,从而得到对应的拍摄图像。除此之外,也可以设定一个内存阈值,将内存大小划分成两个区间范围;或者,还可以设置三个、四个或更多个内存阈值,将内存大小细分成多个区间范围,具体可以根据需要进行设置和修改,本申请实施例不进行任何限制。
此处,第x次拍照次数对应的x级处理帧数可以与第1次拍照次数对应的处理帧数相同,也可以比第1次拍照次数对应的处理帧数少;第x次拍照次数对应的x级处理算法可以与第1次拍照次数对应的1级处理算法相同,也可以比第1次拍照次数对应的1级处理算法的处理时长短一些。具体可以根据需要进行设置和修改,本申请实施例对此不进行任何限制。
当第一拍照算法、第二拍照算法和第三拍照算法相同时,第一拍照算法、第二拍照算法和第三拍照算法可以与1级处理算法相同或者比1级处理算法的处理时长少一些。此种情况下,第一帧数、第二帧数和第三帧数逐渐减小,则第一帧数可以与1级处理帧数相同或者比1级处理帧数少一些,那么,第二帧数和第三帧数都比1级处理帧数更少一些。
当第一拍照算法、第二拍照算法和第三拍照算法的处理时长逐渐减小时,第一拍照算法可以与第1次拍照次数对应的1级处理算法相同,或者第一拍照算法比第1次拍照次数对应的1级处理算法的处理时长少一些,那么,第二拍照算法和第三拍照算法比1级处理算法的处理时长更少一些。此种情况下,当第一帧数、第二帧数和第三帧数相同时,第一帧数、第二帧数和第三帧数与1级处理帧数相同或者减小。当第一帧数、第二帧数和第三帧数逐渐减小时,第一帧数与1级处理帧数相同或少一些,那么,第二帧数和第三帧数比1级处理帧数更少一些。
S18、利用x级处理算法对x级处理帧数进行处理,得到对应的拍摄图像。
图10示出了一种内存占用量的增长示意图。
如图10所示,结合上述S17中的示例,当x=2时,确定第2次拍照次数对应的内存占用量。若第2次拍照次数对应的内存占用量小于第一内存阈值,则确定2级处理算法为第一拍照算法,并确定2级处理帧数为第一帧数。
当x=3时,确定第3次拍照次数对应的内存占用量。若第3次拍照次数对应的内存占用量大于第一内存阈值,但小于第二内存阈值,则确定3级处理算法为第二拍照算法,并
确定3级处理帧数为第二帧数。
当x=4时,确定第4次拍照次数对应的内存占用量。若第4次拍照次数对应的内存占用量大于第一内存阈值,但小于第二内存阈值,则确定4级处理算法为第二拍照算法,并确定4级处理帧数为第二帧数。这样,第4次拍照次数的处理相当于与第3次拍照次数的处理相同。
当x=5时,确定第5次拍照次数对应的内存占用量。若第5次拍照次数对应的内存占用量大于第二内存阈值,则确定5级处理算法为第三拍照算法,并确定5级处理帧数为第三帧数。后续依次类推,在此不再赘述。
应理解,基于将内存大小划分成多个区间范围,当内存占用量越大,越满足更高的区间范围时,处理算法相同,则使处理帧数逐渐减小,或者,处理帧数相同,则使处理算法的处理时长减小;又或者,使处理算法的处理时长和处理的帧数都减小,此时,内存占用量的增长速度将会迅速减缓,减小了电子设备中内存增量的压力。
这样,相对于现有技术中每次拍照均以单一的处理算法和处理帧数进行处理的方式,或者说内存占用量以固定速度增长的方式来说,本申请可以根据内存占用量及时调整处理算法、处理帧数,在内存越大时,使其增长的速度下降,趋于平缓,从而使得电子设备在同等内存大小的条件下,可以实现更多次拍照。
还应理解,图10中未示出第1次拍照次数对应的内存量,此时第1次拍照次数采集的原始图像是按照上述环境检测结果进行处理的。根据内存占用量的结果进行处理的方法适用于第2次及其后续的拍照次数。当然,针对第1次拍照次数也可以选择如第2次及其后续的拍照次数相同的方法进行处理,本申请实施例对此不进行限制。
例如,x为1,确定第1次拍照次数所对应的内存占用量,当该内存占用量小于第一内存阈值时,确定1级处理算法为第一拍照算法,并确定1级处理帧数为第一帧数;当该内存占用量大于或等于第一内存阈值,而小于第二内存阈值时,确定1级处理算法为第二拍照算法,并确定1级处理帧数为第二帧数;当该内存占用量大于第二内存阈值时,确定1级处理算法为第三拍照算法,并确定1级处理帧数为第三帧数;然后,利用确定出的1级处理算法对1级处理帧数进行处理,得到对应的拍摄图像。
S19、保存拍摄图像。
拍摄图像可以存储于图库中或进行显示。
结合上述流程,其中,响应于一次点击操作,可以得到一帧拍摄图像,响应于多次点击操作,可以得到多帧拍摄图像;拍摄图像的具体数量与用户的操作数量相关,本申请实施例对此不进行任何限制。
本申请实施例提供了一种拍照方法,在响应于用户对第一控件的第一操作时,电子设备采用摄像头采集一次原始图像。当第一操作对应第1次拍照次数时,针对待拍摄场景进行环境检测,然后,根据环境检测结果确定第1次拍照次数对应的1级处理帧数和1级处理算法。而当第一操作对应第2次拍照次数及后续其他次数的拍照次数时,先针对内存占用量进行检测,将确定出的内存占用量大小与预设的内存阈值进行比较,然后,根据不同的内存占用量大小,选择不同的处理帧数和处理算法。
在针对第1次拍照次数进行处理时,由于针对环境检测结果对场景进行了细分,根据不同场景选择了不同的算法和帧数,从而可以适应性提高在每种场景下,第1次
拍照次数对应得到的拍摄图像的质量和效果。
而在针对第2次拍照次数及后续其他次数进行处理时,由于针对内存占用量进行细分,根据不同内存占用量选择了不同的算法和帧数,在内存占用量越大的情况下选择处理时长越短的算法和/或减小处理帧数,从而减小了内存增量的压力,提高处理效率,实现快速拍照的需求。
由于电子设备在进行拍照以及其他工作时,还会逐渐产生很高的热量,从而使得电子设备的设备温度逐渐上升。若连续进行快速拍照时,设备温度还有可能呈指数上升。这样,在过热的情况下,电子设备将无法正常运行,也会出现无法响应拍照命令的情况。
对此,本申请实施例还可以在对采集的数据进行处理之前,先对电子设备的设备温度进行检测,针对过热的情况,切换处理算法和处理帧数,以减小工作量,降低热量的增量压力。
图11示出了另一种拍照方法的流程示意图。如图11所示,该方法在图3的基础上,还可以包括以下S20和S21。
其中,S11至S19的内容可以参考上述针对图3的描述,在此不再赘述。
S20、在响应于第一操作,采集一次原始图像后,当该第一操作对应第2次拍照次数或后续其他拍照次数时,确定设备温度是否小于温度阈值。
其中,设备温度指的是电子设备内部的温度。通常随着电子设备进行工作,电子设备将会产生一定的热量,该热量将会造成设备温度的提高,针对该设备温度,可以通过电子设备内部的温度传感器来采集得到。
S21、当设备温度大于或等于温度阈值时,利用单帧算法对此次采集的原始图像中的1帧进行处理,得到对应的拍摄图像。然后,将得到的拍摄图像进行保存。
其中,温度阈值可以根据需要进行设置和修改,本申请实施例对此不进行任何限制。设备温度可以获取电子设备内部的数据得到。
应理解,单帧算法指的是针对1帧原始图像进行处理的算法,单帧算法与1级处理算法不同,单帧算法的处理时长相对于1级处理算法(A1算法、B1算法、C1算法或D1算法)更短。
此处,单帧算法可以包括坏点校正(default pixel correction,DPC)、RAW域降噪(raw域noise filter,RAWNF)、黑电平校正(black level correction,BLC)、镜头阴影校正(lens shading correction,LSC)、自动白平衡、去马赛克、颜色校正(color correction matrix,CCM)、YUV域降噪、色调映射(tone mapping)、伽马(Gamma)校正、色彩空间转换、色彩增强(color enhancement,CE)等处理步骤,处理速度快、耗时短。
坏点校正,坏点即为全黑环境下输出图像中的白点,高亮环境下输出图像中的黑点。一般情况下,三基色通道信号应与环境亮度呈线性响应关系,但是由于图像传感器输出的信号不良,就可能出现白点或黑点,对此,可以自动检测坏点并自动修复,或者,建立坏点像素链表进行固定位置的坏像素点修复。其中,一个点即指的是一个像素。
降噪,指的是减少图像中噪声的过程。一般方法有均值滤波、高斯滤波、双边滤波等。RAW域降噪,指的是减少RAW域图像中噪声的过程。YUV域降噪,指的是减小YUV域图像中噪声的过程。
黑电平校正,是由于图像传感器存在暗电流,导致在没有光线照射的时候,像素也对应有一定的输出电压,并且,不同位置处的像素可能对应不同的输出电压,因此,需要对没有光亮时(即,黑色)像素对应的输出电压进行校正。
镜头阴影校正,可以解决由于镜头对光线折射不均匀导致镜头周围出现阴影的情况。
自动白平衡,是为了消除光源对图像传感器成像的影响,模拟人类视觉的颜色恒常性,保证在任何场景下看到的白色是真正的白色,因此,需要对色温进行校正,自动将白平衡调到合适的位置。
去马赛克,由于RAW域的图像中的每个像素只对应一个通道的颜色信息,因此,可以利用周围像素信息来对其他颜色进行估算,例如,通过线性插值的方式,可以确定出每个像素缺失的另外两个通道的颜色信息,从而恢复出图像中所有像素的所有通道信息。去马赛克的过程,相当于将图像从RAW域转换至RGB域。此处,去马赛克也可以称为颜色插值。
颜色校正,由于摄像头获取的图像,与人们期望的颜色会存在一定差距,因此需要对颜色进行校正。又因为自动白平衡已经将白色校准了,因此,可以通过颜色校正来校准除白色以外的其他颜色。
色调映射,指的是用一个空间不变映射函数来映射图像中所有的像素点。也即色调映射在对图像进行动态范围变换时,图像的每个像素点使用同一个变换函数,是一对一的映射关系。此处,色调映射也可以称为动态范围压缩(dynamic range compression,DRC)。
伽马校正,指的是对图像的伽马曲线进行编辑,以对图像进行非线性色调编辑的方式,检出图像中的深色部分和浅色部分,并使两者按比例增大,从而提高图像对比度效果。
色彩空间转换指的是将图像从RGB域转YUV域。由此,可以将前一步骤处理后得到的图像由RGB域转至YUV域,以降低后续存储和传输的数据量,节省带宽。
色彩增强,以使得原有的不饱和的色彩信息变得饱和、丰富。此处,色彩增强也可以称为颜色处理。
上述仅为一种举例,单帧算法还可以删除或增加其他一些步骤,而1级处理算法(A1算法、B1算法、C1算法或D1算法)可以在单帧算法的基础上包括其他步骤,当然,1级处理算法(A1算法、B1算法、C1算法或D1算法)也可以与单帧算法包括的步骤不同,本申请实施例对此不进行任何限制。
应理解,由于单帧算法可以仅在图像信号处理器中运行处理,因此,单帧算法的处理速度相对比较快,处理时长相对也比较短。
在第一拍照算法、第二拍照算法和第三拍照算法相同的情况下,单帧算法相对于第三拍照算法的处理时长更短,这样,相对于第一拍照算法、第二拍照算法的处理时长也更短。
在第一拍照算法、第二拍照算法和第三拍照算法的处理时长逐渐递减的情况下,单帧算法可以与第三拍照算法相同,或者,单帧算法可以比第三拍照算法的处理时长相对更短。
结合上述,当设备温度高于温度阈值时,可以按照上述步骤切换成单帧算法对1帧原始图像进行处理,最大程度的减小电子设备的工作量,减缓热量的增长幅度,降低热量的增量压力,从而使得电子设备在无法工作之前,多实现一些拍照次数,满足用户的拍照需求。而当设备温度小于温度阈值时,则可以按照上述S16至S19描述的过程,继续对电子设备的内存占用量进行判断,根据不同的内存占用量,切换不同的算法和不同的帧数,降
低内存增量的压力,从而使得在内存存满之前,也能多实现一些拍照次数,满足用户的拍照需求。
需要说明的是,图11所示的执行顺序仅为一种示例,也可以在采集一次原始图像之后,在确定第一操作为第1次拍照次数时,对电子设备的设备温度进行检测,然后,当电子设备的设备温度大于或等于温度阈值时,利用单帧算法对原始图像中的1帧进行处理,当电子设备的设备温度小于温度阈值时,再对待拍摄场景进行环境检测。或者,还可以同时进行设备温度检测和待拍摄场景的环境检测,根据设备温度检测结果和环境检测结果一起来确定第1次拍照次数对应的1级处理帧数和1级处理算法。再或者,还可以针对每次拍照次数均进行温度检测,确定是否小于温度阈值,然后再执行后续相关步骤。
应理解,上述方法仅设定了一个温度阈值,本申请还可以设定多个或者三个、三个以上温度阈值,具体可以根据需要进行设置。示例性的,可以设定两个温度阈值,分别为第一温度阈值和第二温度阈值,第一温度阈值小于第二温度阈值,通过该两个温度阈值,可以将温度范围划分成三个区间范围。当设备温度满足其中一个区间时,相应地,利用该区间对应的算法和帧数进行处理,得到对应的拍摄图像。在此基础上,针对不同的温度区间,还可以设定不同的内存阈值,以执行不同的判断流程。
图12示出了又一种拍照方法的流程示意图。当设定了两个温度阈值,分别为第一温度阈值和第二温度阈值时,该方法在图11的基础上,重新提供了一种S22包括的流程,下面进行介绍。如图12所示,S22还可以包括以下S221至S231。
S221、在响应于第一操作,采集一次原始图像之后,当该第一操作对应第x次拍照次数(第2次拍照次数或后续其他次数的拍照次数)时,确定设备温度是否小于第一温度阈值。
S222、当设备温度小于第一温度阈值时,继续确定内存占用量是否小于第一内存阈值。
S223、根据S222,当设备温度小于第一温度阈值,内存占用量也小于第一内存阈值时,确定x级处理算法为第一拍照算法,并确定x级处理帧数为第一帧数。
S224、利用第一拍照算法对该次采集的原始图像中的第一帧数进行处理,得到对应的拍摄图像。
S225、根据S222,当设备温度小于第一温度阈值,但是内存占用量大于或等于第一内存阈值时,可以确定第二拍照算法为x级处理算法,并确定x级处理帧数为第二帧数。
S226、利用第二拍照算法对该次采集的原始图像中的第二帧数进行处理,得到对应的拍摄图像。
S227、在上述S221中,当设备温度大于或等于第一温度阈值时,继续确定设备温度是否小于第二温度阈值。
S228、当设备温度大于或等于第一温度阈值,而小于第二温度阈值时,可以继续确定内存占用量是否小于第二内存阈值。
此时,当设备温度小于第二温度阈值且内存占用量小于第二内存阈值时,说明设备温度不是特别高,内存也没有存满,还可以以处理时长较长算法或者以较多帧数的原始图像继续进行处理,比如执行上述S225和S226。
S229、根据S228,当设备温度大于或等于第一温度阈值,而小于第二温度阈值,但内存占用量大于或等于第二内存阈值时,可以确定第三拍照算法为x级处理算法,并确定x
级处理帧数为第三帧数。
S230、利用第三拍照算法对该次采集的原始图像中的第三帧数进行处理,得到对应的拍摄图像。
S231、根据S227,判断到设备温度大于或等于第一温度阈值,并且也大于第二温度阈值时,说明设备温度非常高,由此,此时可以利用单帧算法对该次采集的原始图像中的1帧进行处理,得到对应的拍摄图像。
应理解,将温度范围划分成多个区间范围,以及将内存大小划分成多个区间范围时,可以将温度和内存条件进行结合。这样,在温度较低,内存占用量较小时,利用处理时间长的算法对原始图像中的多帧进行处理,以保证图像质量和效果较好;在温度较低、内存占用量较多或者温度较高、内存占用量较低的情况下,适应性的进行调整,利用处理时长相对短一些的算法对原始图像中的较多帧进行处理,以平衡图像质量和温度、内存占用量的关系;而在温度较高、内存占用量较高时,切换成处理时长较短的算法对原始图像中的较少帧进行处理,以减小温度增量、内存增量的压力,在些许降低图像的质量的情况下保证拍照的正常进行;除此之外,在温度特别高的情况下,则不论内存的占用量为多少,直接切换最简单的算法对原始图像中的最少帧进行处理,此时虽然降低了图像质量,却在最大程度上降低了温度增量,来确保响应用户的拍照命令,实现用户的拍照需求。
应理解,上述仅为一种示例,还可以设定多个温度阈值和内存阈值以区分不同的情况,具体可以根据需要进行设置和修改,本申请实施例对此不进行任何限制。
本申请提供的上述方法可以在用户触发拍照命令时,根据当前触发时电子设备的设备温度、内存占用量进行判断,然后将确定出的处理算法和处理帧数与拍照次数关联起来。当用户触发的频率不高,电子设备后台处理的比较及时时,可以按照确定出的处理算法对处理帧数进行处理,以得到对应的拍摄图像。当电子设备后台无法及时处理时,将确定出的处理算法和处理帧数与拍照次数关联后,与采集的原始图像等数据一起存储起来,然后,等待电子设备后续进行调用和处理。
但是,当用户突然提高了触发拍照命令的频率时,电子设备获取的数据激增,造成数据严重拥堵,这种情况下,当某次拍照次数对应的数据后续被调用并进行处理时,由于处理时刻距离响应时刻已经过去了一些时间,在该段时间内,电子设备的内存占用量发生了很大变化,此时,原来确定出的处理算法将不再适用,需要对其进行适应性调整。
又或者,当调用某次拍照次数对应的数据进行处理时,内存占用量已接近内存上限,若此次的处理还按照先前确定好的处理算法进行处理时,将必然会造成无法响应的情况,此时,也需要及时对处理算法进行适应性调整,再次降低内存增量。
针对上述两种情况,本申请实施例又提供了一种拍照方法,以对处理算法进行切换,适应实际处理时刻的内存情况。
图13为本申请实施例提供的又一种拍照流程。如图13所示,该方法30可以包括以下31至S39。
S31、电子设备显示第一界面,第一界面包括第一控件。
S32、当检测到用户对第一控件的第一操作时,响应于第一操作,电子设备利用摄像头采集一次原始图像。
S33、当该第一操作对应第x次拍照次数时,确定内存占用量。
S34、针对该内存占用量,确定该次拍照次数对应的x级处理帧数和x级处理算法,x为大于1的整数。
其中,S31至S34的过程与上述图3所包括的S11、S12、S16和S17的过程相同,可以参考上述描述,在此不再赘述。
S35、确定用户对第一控件进行第一操作的频率是否满足预设频率条件。
其中,频率可以指示用户进行第一操作的速度,例如点击拍摄键的点击速度。例如,针对用户下发的相邻两次拍照命令的间隔时长进行检测,若连续检测到10次或者20次间隔时长小于第二预设间隔时长,则可以说明频率满足预设频率条件。第二预设间隔时长可以根据需要进行设置,例如可以为100m或200ms。第二预设间隔时长小于第一预设间隔时长。
应理解,当频率不满足预设频率条件时,说明用户触发拍照命令的次数比较缓和,用户进行正常的多次拍照;当频率不满足预设频率条件时,说明用户触发拍照命令的次数相对非常多,用户期望快速的进行多次拍照。预设频率条件,也即连续检测相邻拍照次数的间隔时长的次数,以及第二预设间隔时长均可以根据需要进行设置和修改,本申请实施例对此不进行任何限制。
可选地,在本申请实施例中,还可以先确定用户对第一控件进行第一操作的频率是否满足预设频率条件,然后,当满足预设频率条件时,再确定内存占用量,按照图3所示的方法确定该次拍照次数对应的x级处理帧数和x级处理算法。
作为一种示例,当频率满足预设频率条件,再确定内存占用量,按照图3所示的方法确定该次拍照次数对应的x级处理帧数和x级处理算法时,x级处理帧数可以不包括长曝光的原始图像和短曝光的原始图像,x级处理算法不包括高动态范围(high dynamic range,HDR)算法。
应理解,当频率满足预设频率条件时,说明此时用户突然快速下发了密集的拍照命令,此时,可以将x级处理帧数包括的长曝光的原始图像和短曝光的原始图像去掉,将x级处理算法切换成非HDR算法,这样,可以通过放弃保护动态范围的方式,切换成仅处理正常曝光的原始图像的非HDR算法上,来提高后续处理效率,达成用户的快速拍照需求。
S36、确定内存水位线与内存占用量的差值是否小于预设差值。
其中,内存水位线指的是内存的最大使用量,内存水位线的大小可以通过获取厂商提供的参数进行得知,或者,可以需要进行设置和修改,本申请实施例对此不进行任何限制。
应理解,当内存水位线与内存占用量的差值大于预设差值时,说明内存占用量距离达到内存水位线的水平还很远,剩余可用的存储空间还很多;当内存水位线与内存占用量的差值小于预设差值时,说明内存占用量距离达到内存水位线的水平已经很近,剩余可用的存储空间已经为数不多。预设差值的大小可以根据需要进行设置和修改,本申请实施例对此不进行任何限制。
S37、当频率满足预设频率条件或上述所述的内存水位线与内存占用量的差值小于预设差值时,确定后台等待的拍照次数,以及将各个等待的拍照次数所对应的x级处理算法作为初始处理算法。
确定后台等待的拍照次数指的是确定后台等待的有哪些拍照次数,后台等待的拍照次
数包括1个、两个或多个。比如,包括2个,分别是第5级拍照次数、第6级拍照次数。然后,将第5次拍照次数所对应的5级处理算法作为自身对应的初始处理算法,将第6次拍照次数所对应的6级处理算法作为自身对应的初始处理算法。当然,后台也可能没有等待的拍照次数,当没有等待的拍照次数时,无需进行后续步骤。
应理解,当频率满足预设频率条件时,说明用户提高了触发拍照命令的频率;当内存水位线与内存占用量的差值小于预设差值时,说明内存已经快要接近上限,在该两种情况下,需要进行处理算法的切换,而在切换时,先要明确需要被替换的处理算法。
上述仅为两种示例条件,也可以设置其他条件来启动替换的流程,或者,还可以取消上述条件,在针对后台等待的拍照次数进行处理时,均进行二次处理算法的替换。具体可以根据需要设置和调整,本申请对此不进行任何限制。
S38、将最近一次确定的拍照次数所对应的x级处理算法作为目标处理算法,利用目标处理算法将等待的拍照次数所对应的初始处理算法进行替换。
应理解,最近一次拍照次数指的是当前响应于用户的第一操作所记录的拍照次数。确定最近一次拍照次数所对应的处理算法和处理帧数的过程,可以利用上述S17中所述的方法进行确定,在此不再赘述。
由于后台等待的拍照次数所对应的处理算法和处理帧数是根据之前的内存占用量大小来确定得到的,之前的内存占用量与当前的内存占用量已不同,因此,可以按照当前确定出的内存占用量来作为考量,将后台等待的拍照次数所对应的处理算法全部替换成当前确定出的处理算法,以适应当前的内存状况。
可选地,可以在进行替换前,将最近一次确定出的拍照次数所对应的处理算法与等待的拍照次数所对应的初始处理算法进行对比,如果相同,则不需要进行替换;如果不同,再进行替换。由此,通过对处理算法进行对比,只针对处理算法不同的情况做替换,可以避免做无效替换,减小工作量。
可选地,还可以将等待的拍照次数所对应的处理帧数作为初始处理帧数,将最近一次确定出的拍照次数所对应的处理帧数作为目标处理帧数,然后,利用目标处理帧数将等待的拍照次数所对应的初始处理帧数进行替换。
应理解,按照当前确定出的内存占用量来作为考量时,可以将后台等待的拍照次数所对应的处理帧数全部替换成目标处理帧数。
在替换之前,还可以将最近一次确定出的拍照次数所对应的处理帧数与等待的拍照次数所对应的处理帧数进行对比,如果相同,则不需要进行替换;如果不同,再进行替换。由此,通过对处理帧数进行对比,只针对帧数不同的情况做替换,可以避免做无效替换,减小工作量。
其中,根据图3所述的内容可知,在第一次确定拍照次数对应的处理帧数时,处理帧数相同,或者,随着内存占用量越大,处理帧数逐渐减小。由此可知,在第二次确定拍照次数对应的处理帧数时,如果需要替换,则一定是排除了处理帧数相同的情况下,利用最近一次较小的处理帧数替换之前较大的处理帧数,相当于每次将等待的拍照次数所对应的处理帧数进行了减小。替换后的处理帧数越小,后续确定拍摄图像时工作效率越高,内存占用量增加的越少。
需要说明的是,利用目标处理帧数替换初始处理帧数时,只是将处理帧数的数值进行
了更改,并不替换所对应的原始图像。例如,等待的拍照次数为第10次拍照次数,在响应于第一操作时,采集了9帧原始图像,在确定处理帧数时筛选了其中5帧原始图像作为处理帧数,“5帧”即为初始处理帧数。后续若确定出目标处理帧数为1帧,则可以将初始处理帧数“5帧”替换为“1帧”,该1帧所对应的原始图像为原来5帧原始图像中筛选出的1帧即可。
还应理解,上述替换方法启动后,可以循环执行多次。换句话说,可以在后续每次新响应一次拍照命令时,对等待的拍照次数所对应的处理算法和/或处理帧数进行一次替换,又或者说,针对某一等待的拍照次数所对应的处理算法和/或处理帧数来说,在未用于处理来得到对应的拍摄图像之前,处理算法和/或处理帧数可以被替换多次。
由此,后台等待的拍照次数所对应的处理算法为响应于用户对第一控件的第一操作时,根据内存占用量所确定出的处理算法,或者,为等待过程中,最后一次替换后的目标处理算法。
例如,当最近一次确定出的拍照次数为第20次拍照次数时,则将第20次拍照次数所对应的处理算法作为目标处理算法,所对应的处理帧数作为目标帧数;然后,可以将等待的多个拍照次数所对应的处理算法作为初始处理算法,所对应的处理帧数作为初始处理帧数,再将该初始处理算法替换成该目标处理算法,初始处理帧数替换成该目标处理帧数。
响应于用户的又一次第一操作,最近一次所对应的拍照次数更新为第21次拍照次数,则将第21次拍照次数所对应的处理算法作为目标处理算法,所对应的处理帧数作为目标处理帧数,然后,可以将等待的多个拍照次数所对应的处理算法(第20次拍照次数对应的目标处理算法)作为初始处理算法,所对应的处理帧数(第20次拍照次数对应的目标处理帧数)作为初始处理帧数,再将该初始处理算法替换成此次第21次拍照次数的目标处理算法,初始处理帧数替换成此次第21次拍照次数的目标处理帧数。若用户继续执行第一操作,后续替换过程依次类推,在此不再赘述。
S39、当后台等待的任意一次拍照次数被调用处理时,利用对应的目标处理算法对该次拍照次数所采集的原始图像中的x级处理帧数进行处理,得到对应的拍摄图像。
除此之外,当频率不满足预设频率条件,或差值大于或等于预设差值时,则无需进行上述步骤,继续按照原来确定出的处理算法进行处理。例如,如S18所示,利用原先确定出的x级处理算法对x级处理帧数进行处理,得到对应的拍摄图像。又或者,虽然满足了上述所述的频率或差值的条件,但是后台没有等待的拍照次数时,也无需进行上述步骤。
S40、保存拍摄图像。
拍摄图像可以存储于图库中或进行显示。
本申请实施例提供的上述拍照方法,在响应于用户对第一控件的第一操作时,电子设备采用摄像头采集一次原始图像,当用于对第一控件进行第一操作的频率满足预设频率条件时或内存水位线与内存占用量的差值小于预设差值时,确定后台等待的拍照次数以及将各个等待的拍照次数所对应的处理算法作为各自对应的初始处理算法。然后,将最近一次确定出的拍照次数所对应的处理算法作为目标处理算法,利用目标处理算法将等待的拍照次数所对应的初始处理算法进行替换。这样,当等待的拍照次数后续被调用处理时,则可以利用替换后的目标处理算法对该次拍照次数所采集的原始图像中的初始处理帧数进行处理,从而得到对应的拍摄图像。由于最近一次确定出的拍照次数所对应的处理算法是根
据当前的内存情况确定出来的,所以等待的拍照次数的处理算法随之调整,可控制内存堆积程度,减小内存占用量,实现连续快速拍照。
此外,还可以对处理帧数进行替换。当随着拍照次数的增加,处理帧数是减少的时,将最近一次确定出的拍照次数所对应的处理帧数作为目标处理帧数,利用目标处理帧数将等待的拍照次数所对应的初始处理帧数进行替换。替换后,等待的拍照次数所对应的处理帧数是减小的,这样,当等待的拍照次数被调用处理时,则可以利用减小的目标处理帧数去确定拍摄图像。由于最近一次确定出的拍照次数所对应的处理帧数是根据当前的内存情况确定出来的,所以等待的拍照次数的处理帧数随之调整,也能控制内存堆积程度,减小内存占用量。
在此基础上,若后台等待的拍照次数等待处理的时间较长,则可以随着最近一次确定出的拍照次数的变更,多次替换自身对应的处理算法和/或处理帧数,利用最新一次的目标处理算法替换自身上一次的目标处理算法,和/或利用最新一次的目标处理帧数替换自身上一次的目标处理帧数,由此,在实际等到后台等待的拍照次数被进行处理时,可以利用最新一次的目标处理算法进行处理。这样经过多次替换,可以以最新的数据为基准,尽可能的减少内存占用量,实现更多次的连续快速拍照。
相对于之前仅确定一次处理算法和处理帧数来说,该方法进行二次替换,甚至多次替换后,可实时更新和优化处理算法和/或处理帧数,更灵活地适应当前的内存情况,保证拍照能够正常进行下去。
上述对本申请提供的拍照方法的流程进行了总体介绍,下面利用实施例进一步进行示意。
实施例1
图14为本申请实施例提供的一种表格,用于示意每个拍照次数所对应的处理算法和处理帧数。
结合图4所示,当用户打开相机应用,手机显示拍摄界面,拍摄界面包括拍摄键,用户进行拍照时,利用手指对拍摄键进行多次点击操作。
当手机检测到用户的第1次点击操作时,响应于该第1次点击操作,手机利用摄像头采集一次原始图像,例如采集了9帧位于RAW域的原始图像,该次拍照次数可记为第1次拍照次数。
由于该次点击操作对应第1次拍照次数,于是,手机可以针对待拍摄场景进行环境检测,例如进行照度检测和动态范围检测。此时,还根据环境检测结果确定检测出的照度是否小于照度阈值,确定检测出的动态范围值是否小于动态范围阈值。
当照度小于照度阈值,动态范围值小于动态范围阈值时,或者说当待拍摄场景为低亮、低动态范围的场景时,如图14所示,可确定1级处理算法为A1算法,A1算法例如为第一夜景算法。此外,可确定第1次拍照次数对应的1级处理帧数例如为6帧,该6帧指的是从第1次拍照次数对应采集的9帧原始图像中筛选出的6帧原始图像。
当手机检测到用户的第2次点击操作时,响应于该次点击操作,手机利用摄像头采集一次原始图像,还是9帧原始图像,该次拍照次数可记为第2次拍照次数。
由于该次拍照次数对应第2次拍照次数,x大于1,于是,手机继续针对内存占用量
进行确定,由于此时的内存占用量小于第一内存阈值,所以,可以确定第2次拍照次数对应的2级处理算法为第一拍照算法,第一拍照算法例如还为第一夜景算法。同时,可以确定第2次拍照次数对应的2级处理帧数为6帧,6帧为第一帧数。该6帧指的是从第2次拍照次数对应采集的9帧原始图像中筛选出的6帧原始图像。
当手机检测到用户的第3次点击操作时,响应于该次点击操作,手机利用摄像头采集一次原始图像,还是9帧原始图像,该次拍照次数可记为第3次拍照次数。
由于该次拍照次数对应第3次拍照次数,x大于1,于是,手机针对此时的内存占用量进行确定,此时的内存占用量大于第一内存阈值而小于第二内存阈值,由此,可以确定第3次拍照次数对应的3级处理算法为第二拍照算法,预设的第二拍照算法与第一拍照算法相同,即第二拍照算法为第一夜景算法。同时,可以确定第3次拍照次数对应的3级处理帧数为5帧,5帧为第二帧数,第二帧数小于第一帧数。该5帧指的是从第3次拍照次数对应采集的9帧原始图像中筛选出的5帧原始图像。
当手机检测到用户的第4次点击操作时,响应于该次点击操作,手机利用摄像头采集一次原始图像,还是9帧原始图像,该次拍照次数可记为第4次拍照次数。
由于该次拍照次数对应第4次拍照次数,x大于1,于是手机针对此时的内存占用量进行确定,此时的内存占用量大于第二内存阈值,由此,可以确定第4次拍照次数对应的4级处理算法为第三拍照算法,预设的第三拍照算法与第二拍照算法、第一拍照算法相同,即第三拍照算法为第一夜景算法。同时,可以确定第4次拍照算法对应的4级处理帧数为4帧,4帧为第三帧数,第三帧数小于第二帧数。该4帧指的是从第4次拍照次数对应采集的9帧原始图像中筛选出的4帧原始图像。
后续拍照次数的处理过程依次类推,在此不再赘述。
其中,例如,第一夜景算法可以包括单帧算法包括的所有步骤,以及基于Unet网络模型生成的夜景算法模块,夜景算法模块可以将多帧位于RAW域的原始图像融合成一帧位于RAW域的图像。当然,夜景算法模块也可以基于其他模型生成,第一夜景算法还可以包括其他模块或步骤,本申请实施例对此没有任何限制。
此外,每次拍照次数对应的处理帧数的减小量可以相同,也可以不同,具体可以根据需要进行设置和修改,本申请实施例对此不进行任何限制。
在上述实施例中,当待拍摄场景为低亮、低动态范围的场景时,说明待拍摄场景中的环境光线比较暗,若利用一般拍照算法进行处理,得到的拍摄图像会不清楚,进而导致用户看不清拍摄图像中的细节内容。对此,在本申请实施例中,可以选择利用第一夜景算法来进行处理,以提高处理过程中图像的亮度,使得后续获取的拍摄图像可以呈现更多细节。
在此基础上,基于将内存大小划分成多个区间范围,在多次拍照过程中,随着内存占用量的增大,针对对应不同内存范围的拍照次数所确定的处理帧数逐渐减小。此时,虽然选择的处理算法相同,但是在后续进行处理时,由于处理帧数的下降,使得处理算法的处理时长会相应变短,内存占用量的增长速度趋于平缓,因此,在本申请实施例中,在多次拍照次数下,随着拍照次数的增多,基于相同的算法,将每次拍照次数对应的处理帧数进行减小,从而降低处理算法的处理时长,提高处理效率,同时减缓内存占用量的增长速度,减小内存增量的压力。
实施例2
图15为本申请实施例提供的另一种表格,用于示意每个拍照次数所对应的处理算法和处理帧数。
结合图4所示,当用户打开相机应用,手机显示拍摄界面,拍摄界面包括拍摄键,用户进行拍照时,利用手指对拍摄键进行多次点击操作。
当手机检测到用户的第1次点击操作时,响应于该次点击操作,手机利用摄像头采集一次原始图像,例如,采集了6帧位于RAW域的原始图像。该次拍照次数可记为第1次拍照次数。
由于该次点击操作对应第1次拍照次数,于是,手机可以针对待拍摄场景进行环境检测,例如,进行照度检测和动态范围检测。此时,还根据环境检测结果确定检测出的照度是否小于照度阈值,确定检测出的动态范围值是否小于动态范围阈值。
当照度大于照度阈值,动态范围值小于动态范围阈值时,或者说当待拍摄场景为高亮、低动态范围的场景时,如图15所示,可确定1级处理算法为D1算法,D1算法为第一多帧降噪(multi frame noise reduction,MFNR)算法。此外,可以确定第1次拍照次数对应的1级处理帧数为6帧,该6帧指的是第1次拍照次数对应采集的全部6帧原始图像。
当手机检测到用户的第2次点击操作时,响应于该次点击操作,手机利用摄像头采集一次原始图像,还是6帧原始图像,该次拍照次数可记为第2次拍照次数。
由于该次拍照次数对应第2次拍照次数,x大于1,于是,手机接下来可以对设备温度进行检测,确定设备温度是否小于温度阈值,若小于温度阈值,则继续针对内存占用量进行确定,又由于此时的内存占用量小于第一内存阈值,所以,可以确定第2次拍照次数对应的2级处理算法为第一拍照算法,第一拍照算法例如为第二MFNR算法。同时,可以确定第2次拍照次数对应的2级处理帧数为6帧,6帧为第一帧数。该6帧指的是第2次拍照次数对应采集的全部6帧原始图像。
当手机检测到用户的第3次点击操作时,响应于该次点击操作,手机利用摄像头采集一次原始图像,还是6帧原始图像,该次拍照次数可记为第3次拍照次数。
由于该次拍照次数对应第3次拍照次数,x大于1,于是,手机接下来可以对设备温度进行检测,确定设备温度是否小于温度阈值,若小于温度阈值,则继续针对此时的内存占用量进行确定,此时的内存占用量大于第一内存阈值而小于第二内存阈值,由此,可以确定第3次拍照次数对应的3级处理算法为第二拍照算法,预设的第二拍照算法与第一拍照算法不同,且第二拍照算法的处理时长比第一拍照算法的处理时长短,第二拍照算法例如为第三MFNR算法。同时,可以确定第3次拍照次数对应的3级处理帧数为6帧,6帧为第二帧数,第二帧数与第一帧数相同。该6帧指的是第3次拍照次数对应采集的全部6帧原始图像。
当手机检测到用户的第4次点击操作时,响应于该次点击操作,手机利用摄像头采集一次原始图像,还是6帧原始图像,该次拍照次数可记为第4次拍照次数。
由于该次拍照次数对应第4次拍照次数,x大于1,于是可以对设备温度进行检测,确定设备温度是否小于温度阈值,若小于温度阈值,则继续手机针对此时的内存占用量进
行确定,此时的内存占用量大于第二内存阈值,由此,可以确定第4次拍照次数对应的4级处理算法为第三拍照算法,预设的第三拍照算法与第二拍照算法、第一拍照算法不同,且第三拍照算法的处理时长比第二拍照算法的处理时长短,第三拍照算法例如为第四MFNR算法。同时,可以确定第4次拍照次数对应的4级处理帧数为6帧,6帧为第三帧数,第三帧数与第二帧数、第一帧数相同。该6帧指的是第4次拍照次数对应采集的全部6帧原始图像。
当手机检测到用户的第5次点击操作时,响应于该次点击操作,手机利用摄像头采集一次原始图像,还是6帧原始图像,该次拍照次数可记为第5次拍照次数。
由于该次拍照次数对应第5次拍照次数,x大于1,于是可以对设备温度进行检测,确定设备温度是否小于温度阈值,若大于温度阈值,则无需对内存占用量进行判断,直接确定此时的处理算法为单帧算法,处理帧数为1帧,该1帧指的是第5次拍照次数对应采集的6帧原始图像中的1帧原始图像。
后续拍照次数的处理过程依次类推,在此不再赘述。
其中,例如,第一MFNR算法可以是先将原始图像中的多帧进行对齐,然后将对齐后的原始图像采用小波融合算法进行处理的过程;第二MFNR算法可以是光流算法,第三MFNR算法、第四MFNR算法为其他算法,本申请实施例对此没有任何限制。
在上述实施例中,当待拍摄场景为高亮、低动态范围的场景时,说明待拍摄场景中的环境光线较强,但动态范围较小,此时,若利用一般拍照算法进行处理,得到的拍摄图像噪声较大,动态范围较小。对此,在本申请实施例中,可以选择MFNR算法来进行处理,以降低拍摄图像的噪声,提高拍摄图像的动态范围。
在此基础上,基于将内存大小划分成多个区间范围,在多次拍照过程中,随着内存占用量的增大,对应不同内存范围的拍照次数所确定的处理算法的处理时长逐渐减小。此时,虽然选择的处理帧数相同,但是在后续进行处理时,由于处理算法的处理时长逐渐变短,使得内存占用量的增长速度趋于平缓,因此,在本申请实施例中,在多次拍照次数下,随着拍照次数的增多,基于相同的处理帧数,可以将每次拍照次数对应的处理算法的处理时长或者说性能减小,从而可以提高处理效率,同时减缓内存占用量的增长速度,减小内存增量的压力。
此外,当设备温度高于温度阈值时,可以按照直接切换成单帧算法对1帧原始图像进行处理,以减小工作量,减缓热量的增长幅度,降低热量的增量压力,从而使得手机多实现一些拍照次数,满足用户的拍照需求。
实施例3
图16为本申请实施例提供的又一种表格,用于示意每个拍照次数所对应的处理算法和处理帧数。图17为图16对应的处理进程示意图。
结合图5所示,当用户打开相机应用进行录像,手机显示录像界面,录像界面包括抓拍控件,用户在录像的同时利用手指对抓拍控件进行多次点击操作。
当手机检测到用户的第1次点击操作时,响应于该次点击操作,手机利用摄像头采集一次原始图像,例如,采集了9帧原始图像,该9帧原始图像包括6帧正常曝光的原始图
像、1帧长曝光的原始图像和2帧短曝光的原始图像。该次拍照次数可记为第1次拍照次数。
由于该次点击操作对应第1次拍照次数,于是,手机针对待拍摄场景进行环境检测,例如,进行照度检测和动态范围检测。此时,还根据检测结果确定检测出的照度是否小于照度阈值,确定过检测出的动态范围值是否小于动态范围阈值。
当照度大于照度阈值,动态范围值大于动态范围阈值时,或者说当待拍摄场景为高亮、高动态范围的场景时,如图16中的(a)所示,可确定1级处理算法为C1算法,C1算法为第一HDR算法。此外,可以确定第1次拍照次数对应的1级处理帧数为9帧,该9帧指的是第1次拍照次数对应采集的全部9帧原始图像。
其中,第一HDR算法可以包括HDR算法模块,该电子设备可以通过该HDR算法模块来实现第一HDR算法的处理过程。
例如,该第一HDR算法模块基于多曝光融合处理和色调映射模型生成,可以将多帧曝光值不同或者说曝光程度不同的原始图像融合成一帧位于RAW域的图像并对其颜色进行映射变换处理。其中,色调映射模型可以为Unet网络模型、Resnet网络模型和Hdrnet网络模型中的任意一种。当然,色调映射模型也可以为其他模型,本申请实施例对此不进行任何限制。
此时,结合图17所示,后台开始利用针对第1次拍照次数确定出的第一HDR算法对1级处理帧数进行处理,也即对第1次拍照次数对应采集的全部9帧原始图像进行处理,该过程可以称为第1次拍照处理。
当手机检测到用户的第2次点击操作时,响应于该次点击操作,手机利用摄像头采集一次原始图像,还是9帧原始图像,该9帧原始图像包括7帧正常曝光的原始图像、1帧长曝光的原始图像和1帧短曝光的原始图像。该次拍照次数可记为第2次拍照次数。
由于该次拍照次数对应第2次拍照次数,x大于1,于是,手机接下来可以确定用户对抓拍控件进行点击操作的频率是否满足预设频率条件,若频率不满足预设频率条件,则继续针对内存占用量进行确定,又由于此时的内存占用量小于第一内存阈值,所以,可以确定第2次拍照次数对应的2级处理算法为第一拍照算法,第一拍照算法例如为第二HDR算法,第二HDR算法与第一HDR算法不同。同时,可以确定第2次拍照次数对应的2级处理帧数为8帧,8帧为第一帧数。该8帧指的是第2次拍照次数对应采集的9帧原始图像中的5帧正常曝光的原始图像、1帧长曝光的原始图像和2帧短曝光的原始图像。或者,该8帧指的是第2次拍照次数对应采集的9帧原始图像中的5帧正常曝光的原始图像,以及复用第1次拍照次数采集的1帧长曝光的原始图像和2帧短曝光的原始图像。或者,该8帧指的是复用第1次拍照次数采集的1帧长曝光的原始图像和2帧短曝光的原始图像,以及与长曝光的原始图像或短曝光的原始图像采集时间较近的5帧正常曝光的原始图像。
需要说明的是,当用户点击拍照次数非常快速时,也即触发拍照命令的频率比较高时,图像之间的内容差异比较小,由此,此时可以选择部分复用或全部复用第1次拍照次数采集的原始图像。这样,电子设备不需要重新出帧,减小了工作量,也减小了内存占用量。
后续第3次拍照次数至第5次拍照次数的过程与上述第2次拍照次数的确定过程类似,在此不再赘述。
当手机检测到用户的第6次点击操作时,响应于该次点击操作,手机利用摄像头采集
一次原始图像,还是9帧原始图像,该9帧原始图像包括6帧正常曝光的原始图像、1帧长曝光的原始图像和2帧短曝光的原始图像。该次拍照次数可以记为第6次拍照次数。
由于该次拍照次数对应第6次拍照次数,x大于1,于是,手机接下来可以确定用户对抓拍控件进行点击操作的频率是否满足预设频率条件,若频率不满足预设频率条件,则继续针对此时的内存占用量进行确定,此时的内存占用量大于第一内存阈值,而小于第二内存阈值,由此,可以确定第6次拍照次数对应的6级处理算法为第二拍照算法,预设的第二拍照算法与第一拍照算法不同,且第二拍照算法的处理时长比第一拍照算法的处理时长短,第二拍照算法例如为第三HDR算法,第三HDR算法与第一HDR算法、第二HDR算法不同。同时,可以确定第6次拍照次数对应的6级处理帧数为7帧,7帧为第二帧数,第二帧数比第一帧数小。该7帧指的是第6次拍照次数对应采集的9帧原始图像中的4帧正常曝光的原始图像、1帧长曝光的原始图像和2帧短曝光的原始图像。或者,该7帧指的是第6次拍照次数对应采集的9帧原始图像中的4帧正常曝光的原始图像,以及复用第1次拍照次数采集的1帧长曝光的原始图像和2帧短曝光的原始图像。或者,该7帧指的是复用第1次拍照次数采集的1帧长曝光的原始图像和2帧短曝光的原始图像,以及与长曝光的原始图像或短曝光的原始图像采集时间较近的5帧正常曝光的原始图像。
结合图17所示,用户在进行第2次至第6次点击操作时,由于第1次拍照处理还在进行中,因此,只能将第2次至第6次拍照次数对应采集的所有原始图像,以及确定出的各级处理算法、各级处理帧数等相关数据存储起来,进入后台等待状态。
然后,当第1次拍照处理结束后,后台等待的第2次至第6次拍照次数按照时间由近即远的原则,先对第6次拍照次数相关的数据进行处理。后台针对第6次拍照次数,开始利用确定出的第三HDR算法对7帧原始图像进行处理,也即对第6次拍照次数对应采集的4帧正常曝光的原始图像、1帧长曝光的原始图像和2帧短曝光的原始图像进行处理,该过程可以称为第2次拍照处理。
接着,在对第6次拍照次数相关的数据进行处理的同时,当手机检测到用户的第7次点击操作时,响应于该次点击操作,手机利用摄像头采集一次原始图像,还是9帧原始图像,该9帧原始图像包括6帧正常曝光的原始图像、1帧长曝光的原始图像和2帧短曝光的原始图像。该次拍照次数可以记为第7次拍照次数。
由于该次拍照次数对应第7次拍照次数,x大于1,于是,手机接下来可以确定用户对抓拍控件进行点击操作的频率是否满足预设频率条件,若频率满足预设频率条件,此时需要放弃保护动态范围,切换成仅处理正常曝光的原始图像的非HDR算法。例如,可以针对此时的内存占用量进行确定,若此时的内存占用量大于第一内存阈值,而小于第二内存阈值,此时可以将第7次拍照次数对应的7级处理算法从第三HDR算法,切换成第三MFNR算法。同时,可以确定第7次拍照次数对应的7级处理帧数为4帧正常曝光的原始图像,去掉其他长曝光的原始图像和短曝光的原始图像。
另外,由于频率满足预设频率条件,还可以确定出此时后台等待的拍照次数为第2次拍照次数至第5次拍照次数。基于此,可以将第2次至第5次拍照次数之前存储的各自对应的处理算法作为初始处理算法,将第7次拍照次数确定出的7级处理算法作为目标处理算法,然后,利用该目标处理算法将等待的拍照次数所对应的初始处理算法进行替换。
如图16中的(b)所示,第7次拍照次数所确定出的7级处理算法为第三MFNR算法,
那么,此时第三MFNR算法即为目标处理算法。结合图16中的(a)所示,第2次拍照次数原来对应的第二HDR算法即为初始处理算法,利用目标处理算法替换初始处理算法后,第2次拍照次数对应的2级处理算法将变更为第三MFNR算法,第3次至第5次拍照次数的替换过程类似,在此不再赘述。
结合图17所示,用户在进行第7次点击操作时,由于第2次拍照处理还在进行中,因此,只能将第7次拍照次数对应采集的所有原始图像,以及确定出的7级处理算法、7级处理帧数等相关数据存储起来,进入后台等待状态。
当手机检测用户的第8次点击操作时,响应于该次点击操作,手机利用摄像头采集一次原始图像,还是9帧原始图像,该9帧原始图像包括6帧正常曝光的原始图像、1帧长曝光的原始图像和2帧短曝光的原始图像。该次拍照次数可以记为第8次拍照次数。
由于该次拍照次数对应第8次拍照次数,x大于1,于是,手机接下来可以确定用户对抓拍控件进行点击操作的频率是否满足预设频率条件,若频率满足预设频率条件,此时需要放弃保护动态范围,切换成仅处理正常曝光的原始图像的非HDR算法。例如,可以针对此时的内存占用量进行确定,若此时的内存占用量大于第二内存阈值,由此,可以将第8次拍照次数对应的8级处理算法从第三HDR算法,切换成第四MFNR算法,第四MFNR算法与第三MFNR算法不同。同时,可以确定第8次拍照次数对应的8级处理帧数为3帧正常曝光的原始图像,去掉其他长曝光的原始图像和短曝光的原始图像。
另外,由于频率满足预设频率条件,还可以确定出此时后台等待的拍照次数为第2次拍照次数至第5次拍照次数,以及第7次拍照次数。基于此,可以将第2次拍照次数至第5次拍照次数上一次替换后的处理算法作为初始处理算法,以及将第7次拍照次数确定出的7级处理算法作为初始处理算法,再将第8次拍照次数确定出的8级处理算法作为目标处理算法,然后,利用该目标处理算法将等待的拍照次数对应的初始处理算法进行替换。
结合图16中的(c)所示,第8次拍照次数所确定处的8级处理算法为第四MFNR算法,那么第四MFNR算法即为目标处理算法。第2次拍照次数原来对应的第三MFNR算法即为初始处理算法,利用目标处理算法替换初始处理算法后,第2次拍照次数对应的2级处理算法变更为第四MFNR算法,第3次至第5次、以及第7次拍照次数的替换过程类似,在此不再赘述。
结合图17所示,用户在进行第8次点击操作时,由于第2次拍照处理还在进行中,因此,只能将第8次拍照次数对应采集的所有原始图像,以及确定出的8级处理算法、8级处理帧数等相关数据存储起来,进入后台等待状态。
然后,用户在进行第9次点击操作至第20次点击操作时,后台等待的拍照次数所对应的处理算法可以继续根据最近一次确定出的拍照次数所对应的x级处理算法,来作为目标处理算法进行替换。在针对后台等待的拍照次数进行处理时,则可以根据最后一次替换后的处理算法来进行处理。
在上述实施例中,本申请通过结合待拍摄场景中的环境检测结果、用户操作的频率、内存占用量的情况等多种因素,针对不同的情况下的拍照次数的处理算法和处理帧数进行变更。
在此基础上,由于拍照次数较多时不能及时执行处理,本申请还将等待的拍照次数的处理算法及时替换成最近一次确定出的处理算法,这样,可以以最新的数据为基准进行处
理,尽可能的适应当前的内存情况,减小内存增量,实现更多次的连续快速拍照。
上述对本申请实施例提供的拍照方法进行了详细介绍,下面结合电子设备的显示界面介绍一下用户如何启用本申请实施例提供的拍照方法。
图18为本申请实施例提供的一种电子设备的显示界面的示意图。
示例性的,响应于用户的点击操作,当电子设备100运行相机应用时,电子设备100显示如图18中的(a)所示的拍摄界面。该界面的上方显示有导航栏,其中,最中间的位置处显示有用于指示“AI摄影”的图标。
响应于用户针对“AI摄影”的点击操作,电子设备100显示如图18中的(b)所示的提示框,在该界面上显示“AI摄影”已开启的提示信息。此时,电子设备100可以在拍摄时启用本申请实施例提供的拍照方法相关的程序。
应理解,上述仅为用户从电子设备的显示界面启用本申请实施例提供的拍照方法的一种示例,当然也可以通过其他方式来启用本申请实施例提供的拍照方法,或者,也可以在拍摄过程默认直接使用本申请实施例提供的拍照方法,本申请实施例对此不进行任何限制。
图19为本申请实施例提供的另一种电子设备的显示界面的示意图。
结合图18和图19所示,当“AI摄影”开启后,若用户打开相机应用后,对拍摄界面中的拍摄键快速连续进行了多次点击,在该点击的过程中由于后台利用本申请实施例提供的拍照方法进行处理时,在内存占用量增大的情况下,会使用处理时长更短的处理算法和/或更少的处理帧数来进行处理,因此,后续处理出的拍摄图像质量有一定下降。对此,为了提示用户画质有所变化,可以在拍摄界面中的预览图像上方显示“当前图像画质可能会降低”的提示语。
当然,上述仅为一种示例,也可以显示其他提示语来提示用户,或者,还可以不显示任何提示语,或者也可以以其他方式来提醒用户,本申请实施例对此不进行任何限制。
上文结合图1至图19详细描述了本申请实施例提供的拍照方法以及相关的显示界面;下面将结合图20至图23详细描述本申请实施例提供的电子设备、装置和芯片。图20示出了一种适用于本申请的电子设备的硬件系统。
电子设备100可以是手机、智慧屏、平板电脑、可穿戴电子设备、车载电子设备、增强现实(augmented reality,AR)设备、虚拟现实(virtual reality,VR)设备、笔记本电脑、超级移动个人计算机(ultra-mobile personal computer,UMPC)、上网本、个人数字助理(personal digital assistant,PDA)、投影仪等等,本申请实施例对电子设备100的具体类型不作任何限制。
电子设备100可以包括处理器110,外部存储器接口120,内部存储器121,通用串行总线(universal serial bus,USB)接口130,充电管理模块140,电源管理模块141,电池142,天线1,天线2,移动通信模块150,无线通信模块160,音频模块170,扬声器170A,受话器170B,麦克风170C,耳机接口170D,传感器模块180,按键190,
马达191,指示器192,摄像头193,显示屏194,以及用户标识模块(subscriber identification module,SIM)卡接口195等。其中传感器模块180可以包括压力传感器180A,陀螺仪传感器180B,气压传感器180C,磁传感器180D,加速度传感器180E,距离传感器180F,接近光传感器180G,指纹传感器180H,温度传感器180J,触摸传感器180K,环境光传感器180L,骨传导传感器180M等。
需要说明的是,图20所示的结构并不构成对电子设备100的具体限定。在本申请另一些实施例中,电子设备100可以包括比图20所示的部件更多或更少的部件,或者,电子设备100可以包括图20所示的部件中某些部件的组合,或者,电子设备100可以包括图20所示的部件中某些部件的子部件。图20所示的部件可以以硬件、软件、或软件和硬件的组合实现。
处理器110可以包括一个或多个处理单元。例如,处理器110可以包括以下处理单元中的至少一个:应用处理器(application processor,AP)、调制解调处理器、图形处理器(graphics processing unit,GPU)、图像信号处理器(image signal processor,ISP)、控制器、视频编解码器、数字信号处理器(digital signal processor,DSP)、基带处理器、神经网络处理器(neural-network processing unit,NPU)。其中,不同的处理单元可以是独立的器件,也可以是集成的器件。控制器可以根据指令操作码和时序信号,产生操作控制信号,完成取指令和执行指令的控制。
处理器110中还可以设置存储器,用于存储指令和数据。在一些实施例中,处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据,可从所述存储器中直接调用。避免了重复存取,减少了处理器110的等待时间,因而提高了系统的效率。
示例性地,处理器110可以用于执行本申请实施例的拍照方法;显示第一界面,所述第一界面包括第一控件;当检测到对第一控件的第一操作时,响应于第一操作,采集一次原始图像;针对第x次拍照次数,确定内存占用量;当内存占用量小于第一内存阈值时,确定x级处理算法为第一拍照算法,并确定x级处理帧数为第一帧数,x为大于1的整数;当内存占用量大于或等于第一内存阈值,而小于第二内存阈值时,确定x级处理算法为第二拍照算法,并确定x级处理帧数为第二帧数;当内存占用量大于所述第二内存阈值时,确定x级处理算法为第三拍照算法,并确定x级处理帧数为第三帧数,第一内存阈值小于第二内存阈值;利用x级处理算法对原始图像中的x级处理帧数进行处理,得到对应的拍摄图像;保存拍摄图像。
图20所示的各模块间的连接关系只是示意性说明,并不构成对电子设备100的各模块间的连接关系的限定。可选地,电子设备100的各模块也可以采用上述实施例中多种连接方式的组合。
充电管理模块140用于从充电器接收充电输入。其中,充电器可以是无线充电器,也可以是有线充电器。
电源管理模块141用于连接电池142,充电管理模块140与处理器110。电源管理模块141接收电池142和/或充电管理模块140的输入,为处理器110,内部存储器121,显示屏194,摄像头193,和无线通信模块160等供电。电源管理模块141还用于检测电池容量,电池循环次数,电池健康状态(漏电、阻抗)等参数。
电子设备100通过GPU,显示屏194,以及应用处理器等实现显示功能。GPU为图像处理的微处理器,连接显示屏194和应用处理器。GPU用于执行数学和几何计算,用于图形渲染。处理器110可包括一个或多个GPU,其执行程序指令以生成或改变显示信息。
显示屏194用于显示图像,视频等。显示屏194包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display,LCD),有机发光二极管(organic light-emitting diode,OLED),有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的,AMOLED),柔性发光二极管(flex light-emitting diode,FLED),Miniled,MicroLed,Micro-oLed,量子点发光二极管(quantum dot light emitting diodes,QLED)等。在一些实施例中,电子设备100可以包括1个或N个显示屏194,N为大于1的正整数。
电子设备100可以通过ISP,摄像头193,视频编解码器,GPU,显示屏194以及应用处理器等实现拍摄功能。
ISP用于处理摄像头193反馈的数据。例如,拍照时,打开快门,光线通过镜头被传递到摄像头感光元件上,光信号转换为电信号,摄像头感光元件将所述电信号传递给ISP处理,转化为肉眼可见的图像。ISP还可以对图像的噪点,亮度,肤色进行算法优化。ISP还可以对拍摄场景的曝光,色温等参数优化。在一些实施例中,ISP可以设置在摄像头193中。
摄像头193用于捕获静态图像或视频。可以通过应用程序指令触发开启,实现拍照功能,如拍摄获取任意场景的图像。摄像头可以包括成像镜头、滤光片、图像传感器等部件。物体发出或反射的光线进入成像镜头,通过滤光片,最终汇聚在图像传感器上。成像镜头主要是用于对拍照视角中的所有物体(也可称为待拍摄场景、目标场景,也可以理解为用户期待拍摄的场景图像)发出或反射的光汇聚成像;滤光片主要是用于将光线中的多余光波(例如除可见光外的光波,如红外)滤去;图像传感器可以是电荷耦合器件(charge coupled device,CCD)或互补金属氧化物半导体(complementary metal-oxide-semiconductor,CMOS)光电晶体管。图像传感器主要是用于对接收到的光信号进行光电转换,转换成电信号,之后将电信号传递给ISP转换成数字图像信号。ISP将数字图像信号输出到DSP加工处理。DSP将数字图像信号转换成标准的RGB,YUV等格式的图像信号。在一些实施例中,电子设备100可以包括1个或N个摄像头193,N为大于1的正整数。
其中,摄像头193可以位于电子设备100的前面,也可以位于电子设备100的背面,摄像头的具体个数以及排布方式可以根据需求设置,本申请不做任何限制。
示例性的,电子设备100包括前置摄像头和后置摄像头。例如,前置摄像头或者后置摄像头,均可以包括1个或多个摄像头。以电子设备100具有4个后置摄像头为例,这样,电子设备100启动4个后置摄像头进行拍摄时,可以使用本申请实施例提供的拍照方法。或者,摄像头设置于电子设备100的外置配件上,该外置配件可旋转的连接于手机的边框,该外置配件与电子设备100的显示屏194之间所形成的角度为0-360度之间的任意角度。比如,当电子设备100自拍时,外置配件带动摄像头旋转到朝向用户的位置。当然,手机具有多个摄像头时,也可以只有部分摄像头设置在外置
配件上,剩余的摄像头设置在电子设备100本体上,本申请实施例对此不进行任何限制。
数字信号处理器用于处理数字信号,除了可以处理数字图像信号,还可以处理其他数字信号。例如,当电子设备100在频点选择时,数字信号处理器用于对频点能量进行傅里叶变换等。
内部存储器121可以用于存储计算机可执行程序代码,所述可执行程序代码包括指令。内部存储器121可以包括存储程序区和存储数据区。其中,存储程序区可存储操作系统,至少一个功能所需的应用程序(比如声音播放功能,图像播放功能等)等。存储数据区可存储电子设备100使用过程中所创建的数据(比如音频数据,电话本等)等。此外,内部存储器121可以包括高速随机存取存储器,还可以包括非易失性存储器,例如至少一个磁盘存储器件,闪存器件,通用闪存存储器(universal flash storage,UFS)等。处理器110通过运行存储在内部存储器121的指令,和/或存储在设置于处理器中的存储器的指令,执行电子设备100的各种功能应用以及数据处理。
内部存储器121还可以存储本申请实施例提供拍照方法的软件代码,当处理器110运行所述软件代码时,执行拍照方法的流程步骤,实现快速连续拍照。
内部存储器121还可以存储拍摄得到的图像。
外部存储器接口120可以用于连接外部存储卡,例如Micro SD卡,实现扩展电子设备100的存储能力。外部存储卡通过外部存储器接口120与处理器110通信,实现数据存储功能。例如将音乐等文件保存在外部存储卡中。
当然,本申请实施例提供的拍照方法的软件代码也可以存储在外部存储器中,处理器110可以通过外部存储器接口120运行所述软件代码,执行拍照方法的流程步骤,得到多帧拍摄图像。电子设备100得到的拍摄图像也可以存储在外部存储器中。
应理解,用户可以指定将图像存储在内部存储器121还是外部存储器中。比如,电子设备100目标与外部存储器相连接时,若电子设备100拍摄得到1帧图像时,可以弹出提示信息,以提示用户将图像存储在外部存储器还是内部存储器;当然,还可以有其他指定方式,本申请实施例对此不进行任何限制;或者,电子设备100检测到内部存储器121的内存量小于预设量时,可以自动将图像存储在外部存储器中。
电子设备100可以通过音频模块170,扬声器170A,受话器170B,麦克风170C,耳机接口170D,以及应用处理器等实现音频功能。例如音乐播放,录音等。
压力传感器180A用于感受压力信号,可以将压力信号转换成电信号。在一些实施例中,压力传感器180A可以设置于显示屏194。
陀螺仪传感器180B可以用于确定电子设备100的运动姿态。在一些实施例中,可以通过陀螺仪传感器180B确定电子设备100围绕三个轴(即,x,y和z轴)的角速度。陀螺仪传感器180B可以用于拍摄防抖。
气压传感器180C用于测量气压。在一些实施例中,电子设备100通过气压传感器180C测得的气压值计算海拔高度,辅助定位和导航。
磁传感器180D包括霍尔传感器。电子设备100可以利用磁传感器180D检测翻盖皮套的开合。在一些实施例中,当电子设备100是翻盖机时,电子设备100可以根据磁传感器180D检测翻盖的开合。进而根据检测到的皮套的开合状态或翻盖的开合状
态,设置翻盖自动解锁等特性。
加速度传感器180E可检测电子设备100在各个方向上(一般为三轴)加速度的大小。当电子设备100静止时可检测出重力的大小及方向。还可以用于识别电子设备姿态,应用于横竖屏切换,计步器等应用。
距离传感器180F,用于测量距离。电子设备100可以通过红外或激光测量距离。在一些实施例中,拍摄场景,电子设备100可以利用距离传感器180F测距以实现快速对焦。
接近光传感器180G可以包括例如发光二极管(LED)和光检测器,例如光电二极管。发光二极管可以是红外发光二极管。电子设备100通过发光二极管向外发射红外光。电子设备100使用光电二极管检测来自附近物体的红外反射光。当检测到充分的反射光时,可以确定电子设备100附近有物体。当检测到不充分的反射光时,电子设备100可以确定电子设备100附近没有物体。电子设备100可以利用接近光传感器180G检测用户手持电子设备100贴近耳朵通话,以便自动熄灭屏幕达到省电的目的。接近光传感器180G也可用于皮套模式,口袋模式自动解锁与锁屏。
环境光传感器180L用于感知环境光亮度。电子设备100可以根据感知的环境光亮度自适应调节显示屏194亮度。环境光传感器180L也可用于拍照时自动调节白平衡。环境光传感器180L还可以与接近光传感器180G配合,检测电子设备100是否在口袋里,以防误触。
指纹传感器180H用于采集指纹。电子设备100可以利用采集的指纹特性实现指纹解锁,访问应用锁,指纹拍照,指纹接听来电等。
温度传感器180J用于检测温度。在一些实施例中,电子设备100利用温度传感器180J检测的温度,执行温度处理策略。例如,当温度传感器180J上报的温度超过阈值,电子设备100执行降低位于温度传感器180J附近的处理器的性能,以便降低功耗实施热保护。在另一些实施例中,当温度低于另一阈值时,电子设备100对电池142加热,以避免低温导致电子设备100异常关机。在其他一些实施例中,当温度低于又一阈值时,电子设备100对电池142的输出电压执行升压,以避免低温导致的异常关机。
触摸传感器180K,也称“触控器件”。触摸传感器180K可以设置于显示屏194,由触摸传感器180K与显示屏194组成触摸屏,也称“触控屏”。触摸传感器180K用于检测作用于其上或附近的触摸操作。触摸传感器可以将检测到的触摸操作传递给应用处理器,以确定触摸事件类型。可以通过显示屏194提供与触摸操作相关的视觉输出。在另一些实施例中,触摸传感器180K也可以设置于电子设备100的表面,与显示屏194所处的位置不同。
骨传导传感器180M可以获取振动信号。在一些实施例中,骨传导传感器180M可以获取人体声部振动骨块的振动信号。骨传导传感器180M也可以接触人体脉搏,接收血压跳动信号。在一些实施例中,骨传导传感器180M也可以设置于耳机中,结合成骨传导耳机。音频模块170可以基于所述骨传导传感器180M获取的声部振动骨块的振动信号,解析出语音信号,实现语音功能。应用处理器可以基于所述骨传导传感器180M获取的血压跳动信号解析心率信息,实现心率检测功能。
按键190包括开机键,音量键等。按键190可以是机械按键。也可以是触摸式按
键。电子设备100可以接收按键输入,产生与电子设备100的用户设置以及功能控制有关的键信号输入。
马达191可以产生振动提示。马达191可以用于来电振动提示,也可以用于触摸振动反馈。例如,作用于不同应用(例如拍照,音频播放等)的触摸操作,可以对应不同的振动反馈效果。
指示器192可以是指示灯,可以用于指示充电状态,电量变化,也可以用于指示消息,未接来电,通知等。
SIM卡接口195用于连接SIM卡。SIM卡可以通过插入SIM卡接口195,或从SIM卡接口195拔出,实现和电子设备100的接触和分离。
可以理解的是,本申请实施例示意的结构并不构成对电子设备100的具体限定。在本申请另一些实施例中,电子设备100可以包括比图示更多或更少的部件,或者组合某些部件,或者拆分某些部件,或者不同的部件布置。图示的部件可以以硬件,软件或软件和硬件的组合实现。
上文详细描述了电子设备100的硬件系统,下面介绍电子设备100的软件系统。
图21是本申请实施例提供的电子设备的软件系统的示意图。
如图21所示,系统架构中可以包括应用层(application,APP)210、应用框架层220、硬件抽象层(hardware abstract layer,HAL)230、驱动层240以及硬件层250。
应用层210可以包括相机应用程序或者其他应用程序,其他应用程序包括但不限于:相机、图库等应用程序。
应用层210处于整个框架的顶端,承担着与用户直接进行交互的责任,一旦接收到用户直接或间接的比如拍照的需求,便会通过接口将需求发送给应用框架层220,等待应用框架层220进行回传处理结果,其中,该结果包括图像数据以及相机参数等;然后应用层210再将该结果反馈给用户。
应用框架层220可以向应用层的应用程序提供应用程序编程接口(application programming interface,API)和编程框架;应用框架层可以包括一些预定义的函数。
例如,应用框架层220可以包括相机访问接口;相机访问接口中可以包括相机管理与相机设备;其中,相机管理可以用于提供管理相机的访问接口;相机设备可以用于提供访问相机的接口。
硬件抽象层230用于将硬件抽象化。比如,硬件抽象层可以包括相机硬件抽象层以及其他硬件设备抽象层;相机硬件抽象层中可以包括相机设备1、相机设备2等;相机硬件抽象层可以与相机算法库相连接,相机硬件抽象层可以调用相机算法库中的算法。
在本申请中,用于进行各种检测的感知引擎可以设置于硬件抽象层中。
驱动层240用于为不同硬件设备提供驱动。例如,驱动层可以包括相机设备驱动、数字信号处理器驱动和图形处理器驱动。
硬件层250可以包括多个图像传感器(sensor)、多个图像信号处理器、数字信号处理器、图形处理器以及其他硬件设备。
例如,硬件层250包括传感器和图像信号处理器;传感器中可以包括传感器1、传感器2、深度传感器(time of flight,TOF)、多光谱传感器等。图像信号处理器中
可以包括图像信号处理器1、图像信号处理器2等。
在本申请中,通过调用硬件抽象层230中的硬件抽象层接口,可以实现硬件抽象层230上方的应用程序层210、应用程序框架层220与下方的驱动层240、硬件层250的连接,实现摄像头数据传输及功能控制。
其中,在硬件抽象层230中的摄像头硬件接口层中,厂商可以根据需求在此定制功能。摄像头硬件接口层相比硬件抽象层接口,更加高效、灵活、低延迟,也能更加丰富的调用ISP和GPU,来实现图像处理。其中,输入硬件抽象层230中的图像可以来自图像传感器,也可以来自存储的图片。
硬件抽象层230中的调度层,包含了通用功能性接口,用于实现管理和控制。
硬件抽象层230中的摄像头服务层,用于访问ISP和其他硬件的接口。
下面结合捕获拍照场景,示例性说明电子设备100软件以及硬件的工作流程。
应用程序层中的相机应用可以以图标的方式显示在电子设备100的屏幕上。当相机应用的图标被用户点击以进行触发时,电子设备100开始运行相机应用。当相机应用运行在电子设备100上时,相机应用调用应用程序框架层210中的相机应用对应的接口,然后,通过调用硬件抽象层230启动摄像头驱动,开启电子设备100上的摄像头193,同时相机算法库开始加载本申请实施例所利用的拍照方法。
然后,通过图像传感器采集了一帧或多帧原始图像后,可将采集的原始图像经图像信号处理器处理后返回硬件抽象层,利用从相机算法库中调用的某一处理算法进行处理,生成拍摄图像,再将拍摄图像进行保存和/或传输至显示屏进行显示。
下面介绍本申请实施例提供的一种用于实现上述拍照方法的图像处理装置300。图22是本申请实施例提供的图像处理装置300的示意图。
如图22所示,图像处理装置300包括显示单元310、获取单元320和处理单元330。
其中,显示单元310用于显示第一界面,第一界面包括第一控件。
获取单元320用于检测用户对第一控件的第一操作。
处理单元330用于响应于第一操作,采集一次原始图像。
处理单元330还用于针对第x次拍照次数,确定内存占用量;当内存占用量小于第一内存阈值时,确定x级处理算法为第一拍照算法,并确定x级处理帧数为第一帧数,x为大于1的整数;当内存占用量大于或等于第一内存阈值,而小于第二内存阈值时,确定x级处理算法为第二拍照算法,并确定x级处理帧数为第二帧数;当内存占用量大于所述第二内存阈值时,确定x级处理算法为第三拍照算法,并确定x级处理帧数为第三帧数,第一内存阈值小于第二内存阈值;利用x级处理算法对原始图像中的x级处理帧数进行处理,得到对应的拍摄图像;保存拍摄图像。
需要说明的是,上述图像处理装置300以功能单元的形式体现。这里的术语“单元”可以通过软件和/或硬件形式实现,对此不作具体限定。
例如,“单元”可以是实现上述功能的软件程序、硬件电路或二者结合。所述硬件电路可能包括应用特有集成电路(application specific integrated circuit,ASIC)、电子电路、用于执行一个或多个软件或固件程序的处理器(例如共享处理器、专有处理器或组处理器等)和存储器、合并逻辑电路和/或其它支持所描述的功能的合适组件。
因此,在本申请的实施例中描述的各示例的单元,能够以电子硬件、或者计算机
软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本申请的范围。
本申请实施例还提供一种计算机可读存储介质,所述计算机可读存储介质中存储有计算机指令;当所述计算机可读存储介质在图像处理装置300上运行时,使得该图像处理装置300执行前述所示的拍照方法。
所述计算机指令可以存储在计算机可读存储介质中,或者从一个计算机可读存储介质向另一个计算机可读存储介质传输,例如,所述计算机指令可以从一个网站站点、计算机、服务器或者数据中心通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line,DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可以用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质(例如,软盘、硬盘、磁带),光介质、或者半导体介质(例如固态硬盘(solid state disk,SSD))等。
本申请实施例还提供了一种包含计算机指令的计算机程序产品,当其在图像处理装置300上运行时,使得图像处理装置300可以执行前述所示的拍照方法。
图23为本申请实施例提供的一种芯片的结构示意图。图23所示的芯片可以为通用处理器,也可以为专用处理器。该芯片包括处理器401。其中,处理器401用于支持图像处理装置300执行前述所示的技术方案。
可选的,该芯片还包括收发器402,收发器402用于接受处理器401的控制,用于支持图像处理装置300执行前述所示的技术方案。
可选的,图23所示的芯片还可以包括:存储介质403。
需要说明的是,图23所示的芯片可以使用下述电路或者器件来实现:一个或多个现场可编程门阵列(field programmable gate array,FPGA)、可编程逻辑器件(programmable logic device,PLD)、控制器、状态机、门逻辑、分立硬件部件、任何其他适合的电路、或者能够执行本申请通篇所描述的各种功能的电路的任意组合。
上述本申请实施例提供的电子设备、图像处理装置300、计算机存储介质、计算机程序产品、芯片均用于执行上文所提供的方法,因此,其所能达到的有益效果可参考上文所提供的方法对应的有益效果,在此不再赘述。
应理解,上述只是为了帮助本领域技术人员更好地理解本申请实施例,而非要限制本申请实施例的范围。本领域技术人员根据所给出的上述示例,显然可以进行各种等价的修改或变化,例如,上述检测方法的各个实施例中某些步骤可以是不必须的,或者可以新加入某些步骤等。或者上述任意两种或者任意多种实施例的组合。这样的修改、变化或者组合后的方案也落入本申请实施例的范围内。
还应理解,上文对本申请实施例的描述着重于强调各个实施例之间的不同之处,未提到的相同或相似之处可以互相参考,为了简洁,这里不再赘述。
还应理解,上述各过程的序号的大小并不意味着执行顺序的先后,各过程的执行顺序应以其功能和内在逻辑确定,而不应对本申请实施例的实施过程构成任何限定。
还应理解,本申请实施例中,“预先设定”、“预先定义”可以通过在设备(例如,包
括电子设备)中预先保存相应的代码、表格或其他可用于指示相关信息的方式来实现,本申请对于其具体的实现方式不做限定。
还应理解,本申请实施例中的方式、情况、类别以及实施例的划分仅是为了描述的方便,不应构成特别的限定,各种方式、类别、情况以及实施例中的特征在不矛盾的情况下可以相结合。
还应理解,在本申请的各个实施例中,如果没有特殊说明以及逻辑冲突,不同的实施例之间的术语和/或描述具有一致性、且可以相互引用,不同的实施例中的技术特征根据其内在的逻辑关系可以组合形成新的实施例。
最后应说明的是:以上所述,仅为本申请的具体实施方式,但本申请的保护范围并不局限于此,任何在本申请揭露的技术范围内的变化或替换,都应涵盖在本申请的保护范围之内。因此,本申请的保护范围应以所述权利要求的保护范围为准。
Claims (11)
- 一种拍照方法,其特征在于,应用于电子设备,所述方法包括:显示第一界面,所述第一界面包括第一控件;当检测到对所述第一控件的第一操作时,响应于所述第一操作,采集一次原始图像;针对第x次拍照次数,确定内存占用量;当所述内存占用量小于第一内存阈值时,确定x级处理算法为第一拍照算法,并确定x级处理帧数为第一帧数,x为大于1的整数;当所述内存占用量大于或等于所述第一内存阈值,而小于第二内存阈值时,确定所述x级处理算法为第二拍照算法,并确定所述x级处理帧数为第二帧数;当所述内存占用量大于所述第二内存阈值时,确定所述x级处理算法为第三拍照算法,并确定所述x级处理帧数为第三帧数,所述第一内存阈值小于所述第二内存阈值;利用所述x级处理算法对所述原始图像中的所述x级处理帧数进行处理,得到对应的拍摄图像;保存所述拍摄图像;其中,当所述第一拍照算法、所述第二拍照算法和所述第三拍照算法相同时,所述第一帧数、所述第二帧数和所述第三帧数逐渐减小;当所述第一拍照算法、所述第二拍照算法和所述第三拍照算法的处理时长逐渐减小时,所述第一帧数、所述第二帧数和所述第三帧数相同或者逐渐减小。
- 根据权利要求1所述的拍照方法,其特征在于,所述方法还包括:针对所述第x次拍照次数,当设备温度大于或等于温度阈值时,利用单帧算法对所述原始图像中的1帧进行处理,得到对应的拍摄图像;其中,当所述第一拍照算法、所述第二拍照算法和所述第三拍照算法相同时,所述单帧算法的处理时长比所述第三拍照算法的处理时长更短;当所述第一拍照算法、所述第二拍照算法和所述第三拍照算法的处理时长逐渐减小时,所述单帧算法的处理时长与所述第三拍照算法的处理时长相同,或者,所述单帧算法的处理时长比所述第三拍照算法的处理时长更短。
- 根据权利要求1或2所述的拍照方法,其特征在于,所述方法还包括:当所述第一操作对应第1次拍照次数时,针对待拍摄场景进行环境检测;根据环境检测结果确定所述第1次拍照次数对应的1级处理帧数和1级处理算法;利用所述1级处理算法,对所述第1次拍照次数对应采集的原始图像中的所述1级处理帧数进行处理,得到对应的拍摄图像。
- 根据权利要求3所述的拍照方法,其特征在于,所述环境检测至少包括照度检测和动态范围检测,所述环境检测结果至少包括照度和动态范围值;根据环境检测结果确定所述第1次拍照次数对应的1级处理帧数和1级处理算法,包括:当所述照度小于照度阈值,所述动态范围值大于或等于动态范围阈值时,确定所 述1级处理帧数为a1帧,并确定所述1级处理算法为A1算法;当所述照度小于所述照度阈值,所述动态范围值小于所述动态范围阈值时,确定所述1级处理帧数为b1帧,并确定所述1级处理算法为B1算法;当所述照度大于或等于所述照度阈值,所述动态范围值大于或等于所述动态范围阈值时,确定所述1级处理帧数为c1帧,并确定所述1级处理算法为C1算法;当所述照度大于或等于所述照度阈值,所述动态范围值小于所述动态范围阈值时,确定所述1级处理帧数为d1帧,并确定所述1级处理算法为D1算法;其中,a1、b1、c1和d1均为大于或等于1的整数。
- 根据权利要求4所述的拍照方法,其特征在于,利用所述1级处理算法,对所述第1次拍照次数对应采集的原始图像中的所述1级处理帧数进行处理,得到对应的拍摄图像,包括:当所述照度小于照度阈值,所述动态范围值大于或等于动态范围阈值时,利用所述A1算法对所述第1次拍照次数对应采集的原始图像中的a1帧进行处理,得到对应的拍摄图像;当所述照度小于所述照度阈值,所述动态范围值小于所述动态范围阈值时,利用所述B1算法对所述第1次拍照次数对应采集的原始图像中的b1帧进行处理,得到对应的拍摄图像;当所述照度大于或等于所述照度阈值,所述动态范围值大于或等于所述动态范围阈值时,利用所述C1算法对所述第1次拍照次数对应采集的原始图像中的c1帧进行处理,得到对应的拍摄图像;当所述照度大于或等于所述照度阈值,所述动态范围值小于所述动态范围阈值时,利用所述D1算法对所述第1次拍照次数对应采集的原始图像中的d1帧进行处理,得到对应的拍摄图像。
- 根据权利要求1至5中任一项所述的拍照方法,其特征在于,在确定所述x级处理算法为第三拍照算法,并确定所述x级处理帧数为第三帧数之后,所述方法还包括:确定对所述第一控件的第一操作的频率,以及内存水位线与所述内存占用量的差值;当所述频率满足预设频率条件或所述差值小于预设差值时,确定后台等待的拍照次数;将所述后台等待的拍照次数各自对应的x级处理算法分别作为初始处理算法;将最近一次确定的拍照次数对应的x级处理算法作为目标处理算法;利用所述目标处理算法替换所述后台等待的拍照次数各自的所述初始处理算法;当后台等待的任意一次拍照次数被调用处理时,利用所述目标处理算法对所述任意一次拍照次数所对应的原始图像中的x级处理帧数进行处理,得到对应的拍摄图像。
- 根据权利要求1至6中任一项所述的拍照方法,其特征在于,所述第一界面是指拍照界面,所述第一控件是指用于指示拍照的控件。
- 根据权利要求1至6中任一项所述的拍照方法,其特征在于,所述第一界面是指录像界面,所述第一控件是指用于指示抓拍的控件。
- 一种电子设备,其特征在于,包括:一个或多个处理器和存储器;所述存储器与所述一个或多个处理器耦合,所述存储器用于存储计算机程序代码,所述计算机程序代码包括计算机指令,所述一个或多个处理器调用所述计算机指令以使得所述电子设备执行如权利要求1至8中任一项所述方法。
- 一种芯片,其特征在于,所述芯片应用于电子设备,所述芯片包括一个或多个处理器,所述处理器用于调用计算机指令以使得所述电子设备执行如权利要求1至8中任一项所述的方法。
- 一种计算机可读存储介质,其特征在于,所述计算机可读存储介质存储了计算机程序,当所述计算机程序被处理器执行时,使得处理器执行权利要求1至8中任一项所述的方法。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211026658.0 | 2022-08-25 | ||
CN202211026658.0A CN116055897B (zh) | 2022-08-25 | 2022-08-25 | 拍照方法及其相关设备 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2024040981A1 true WO2024040981A1 (zh) | 2024-02-29 |
Family
ID=86114045
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2023/087489 WO2024040981A1 (zh) | 2022-08-25 | 2023-04-11 | 拍照方法及其相关设备 |
Country Status (2)
Country | Link |
---|---|
CN (2) | CN118175436A (zh) |
WO (1) | WO2024040981A1 (zh) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN118175436A (zh) * | 2022-08-25 | 2024-06-11 | 荣耀终端有限公司 | 图像处理方法及其相关设备 |
CN116708996B (zh) * | 2023-08-07 | 2023-11-17 | 荣耀终端有限公司 | 一种拍照方法、图像优化模型训练方法及电子设备 |
CN117479002A (zh) * | 2023-09-15 | 2024-01-30 | 荣耀终端有限公司 | 图像显示方法、电子设备及存储介质 |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160217328A1 (en) * | 2013-09-30 | 2016-07-28 | Danielle YANAI | Image and video processing and optimization |
CN108055556A (zh) * | 2017-11-16 | 2018-05-18 | 北京达佳互联信息技术有限公司 | 数据处理方法和装置 |
CN110955541A (zh) * | 2019-12-09 | 2020-04-03 | Oppo广东移动通信有限公司 | 数据处理方法及相关产品 |
CN113411504A (zh) * | 2021-08-18 | 2021-09-17 | 成都大熊猫繁育研究基地 | 一种野外红外相机的智能拍摄方法及系统 |
CN114650433A (zh) * | 2020-12-18 | 2022-06-21 | 北京达佳互联信息技术有限公司 | 缓存方法及缓存装置 |
CN114841853A (zh) * | 2022-04-18 | 2022-08-02 | 北京大甜绵白糖科技有限公司 | 图像处理方法、装置、设备及存储介质 |
CN116055897A (zh) * | 2022-08-25 | 2023-05-02 | 荣耀终端有限公司 | 拍照方法及其相关设备 |
Family Cites Families (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2008211367A (ja) * | 2007-02-23 | 2008-09-11 | Olympus Imaging Corp | 撮像装置 |
JP5948811B2 (ja) * | 2011-11-21 | 2016-07-06 | セイコーエプソン株式会社 | 制御装置、電気光学装置、電子機器および制御方法 |
JP6217169B2 (ja) * | 2013-06-21 | 2017-10-25 | 富士通株式会社 | 携帯端末装置、端末制御プログラム及び端末制御方法 |
JP2015154179A (ja) * | 2014-02-13 | 2015-08-24 | 株式会社リコー | 画像処理装置、撮像装置、画像処理プログラム、画像処理方法 |
JP2015179969A (ja) * | 2014-03-19 | 2015-10-08 | カシオ計算機株式会社 | 撮像装置、撮像方法及びプログラム |
CN105979142A (zh) * | 2016-06-03 | 2016-09-28 | 深圳市金立通信设备有限公司 | 一种温升控制方法及终端 |
KR102669687B1 (ko) * | 2016-12-05 | 2024-05-28 | 삼성전자주식회사 | 메모리의 온도에 따라 상이하게 동작하는 스토리지 장치 |
CN107016963A (zh) * | 2017-04-20 | 2017-08-04 | 京东方科技集团股份有限公司 | 电致发光显示面板的驱动方法、其驱动装置及显示装置 |
CN107277374A (zh) * | 2017-07-28 | 2017-10-20 | 盯盯拍(深圳)技术股份有限公司 | 摄像终端的控制方法以及摄像终端的控制装置 |
CN108289172A (zh) * | 2018-01-20 | 2018-07-17 | 深圳天珑无线科技有限公司 | 调节拍摄相关功能的方法、装置及移动终端 |
US11061740B2 (en) * | 2018-08-13 | 2021-07-13 | International Business Machines Corporation | Computer system workload manager |
CN110166708B (zh) * | 2019-06-13 | 2021-06-11 | Oppo广东移动通信有限公司 | 夜景图像处理方法、装置、电子设备以及存储介质 |
KR20210011166A (ko) * | 2019-07-22 | 2021-02-01 | 삼성전자주식회사 | 전자 장치 및 전자 장치에서 발열 기반의 데이터 처리량 제어 방법 |
CN110445988B (zh) * | 2019-08-05 | 2021-06-25 | Oppo广东移动通信有限公司 | 图像处理方法、装置、存储介质及电子设备 |
CN112532859B (zh) * | 2019-09-18 | 2022-05-31 | 华为技术有限公司 | 视频采集方法和电子设备 |
CN110659136B (zh) * | 2019-09-19 | 2022-07-15 | Oppo广东移动通信有限公司 | 限制帧率的方法、装置、终端及存储介质 |
CN110705497A (zh) * | 2019-10-11 | 2020-01-17 | Oppo广东移动通信有限公司 | 图像帧处理方法、装置、终端设备及计算机可读存储介质 |
CN110933294B (zh) * | 2019-11-04 | 2022-11-04 | 深圳传音控股股份有限公司 | 一种图像处理方法、终端及计算机存储介质 |
CN111416759A (zh) * | 2020-03-27 | 2020-07-14 | 海信集团有限公司 | 智能家居设备及控制方法 |
CN113918328A (zh) * | 2021-09-30 | 2022-01-11 | 天津南大通用数据技术股份有限公司 | 一种基于lru的数据缓冲区内存释放调度系统及方法 |
CN114286018A (zh) * | 2021-12-23 | 2022-04-05 | 蓝晟伦 | 视频图像处理方法、装置、计算机可读存储介质 |
-
2022
- 2022-08-25 CN CN202410171409.3A patent/CN118175436A/zh active Pending
- 2022-08-25 CN CN202211026658.0A patent/CN116055897B/zh active Active
-
2023
- 2023-04-11 WO PCT/CN2023/087489 patent/WO2024040981A1/zh unknown
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160217328A1 (en) * | 2013-09-30 | 2016-07-28 | Danielle YANAI | Image and video processing and optimization |
CN108055556A (zh) * | 2017-11-16 | 2018-05-18 | 北京达佳互联信息技术有限公司 | 数据处理方法和装置 |
CN110955541A (zh) * | 2019-12-09 | 2020-04-03 | Oppo广东移动通信有限公司 | 数据处理方法及相关产品 |
CN114650433A (zh) * | 2020-12-18 | 2022-06-21 | 北京达佳互联信息技术有限公司 | 缓存方法及缓存装置 |
CN113411504A (zh) * | 2021-08-18 | 2021-09-17 | 成都大熊猫繁育研究基地 | 一种野外红外相机的智能拍摄方法及系统 |
CN114841853A (zh) * | 2022-04-18 | 2022-08-02 | 北京大甜绵白糖科技有限公司 | 图像处理方法、装置、设备及存储介质 |
CN116055897A (zh) * | 2022-08-25 | 2023-05-02 | 荣耀终端有限公司 | 拍照方法及其相关设备 |
Also Published As
Publication number | Publication date |
---|---|
CN118175436A (zh) | 2024-06-11 |
CN116055897B (zh) | 2024-02-27 |
CN116055897A (zh) | 2023-05-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2021052232A1 (zh) | 一种延时摄影的拍摄方法及设备 | |
WO2022262260A1 (zh) | 一种拍摄方法及电子设备 | |
WO2024040981A1 (zh) | 拍照方法及其相关设备 | |
CN101610363B (zh) | 在数字图像处理装置中对图像背景进行模糊的设备和方法 | |
WO2023015981A1 (zh) | 图像处理方法及其相关设备 | |
WO2023015991A1 (zh) | 拍照方法、电子设备和计算机可读存储介质 | |
CN108419023A (zh) | 一种生成高动态范围图像的方法以及相关设备 | |
CN110198417A (zh) | 图像处理方法、装置、存储介质及电子设备 | |
CN115633252B (zh) | 拍摄方法及其相关设备 | |
WO2024045670A1 (zh) | 生成高动态范围视频的方法和电子设备 | |
WO2022083325A1 (zh) | 拍照预览方法、电子设备以及存储介质 | |
CN113630558B (zh) | 一种摄像曝光方法及电子设备 | |
WO2023015990A1 (zh) | 点光源图像检测方法和电子设备 | |
CN116095476B (zh) | 摄像头的切换方法、装置、电子设备及存储介质 | |
CN116320771B (zh) | 一种拍摄方法和电子设备 | |
CN114463191B (zh) | 一种图像处理方法及电子设备 | |
CN115604572B (zh) | 图像的获取方法、电子设备及计算机可读存储介质 | |
WO2023226612A1 (zh) | 一种曝光参数确定方法和装置 | |
WO2022267608A1 (zh) | 一种曝光强度调节方法及相关装置 | |
WO2024174625A1 (zh) | 图像处理方法和电子设备 | |
WO2024032033A1 (zh) | 视频处理的方法及电子设备 | |
WO2023160221A1 (zh) | 一种图像处理方法和电子设备 | |
WO2023131028A1 (zh) | 图像处理方法及其相关设备 | |
WO2022267763A1 (zh) | 一种拍摄方法及相关装置 | |
CN116055855B (zh) | 图像处理方法及其相关设备 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 23856076 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2023856076 Country of ref document: EP Effective date: 20240731 |