CN115705614A - Image processing method, image processing device, electronic equipment and storage medium - Google Patents
Image processing method, image processing device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115705614A CN115705614A CN202110897405.XA CN202110897405A CN115705614A CN 115705614 A CN115705614 A CN 115705614A CN 202110897405 A CN202110897405 A CN 202110897405A CN 115705614 A CN115705614 A CN 115705614A
- Authority
- CN
- China
- Prior art keywords
- image
- type
- image processing
- processing module
- format
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 title claims abstract description 325
- 238000003672 processing method Methods 0.000 title claims abstract description 39
- 238000000034 method Methods 0.000 claims abstract description 52
- 230000008569 process Effects 0.000 claims abstract description 42
- 230000000694 effects Effects 0.000 abstract description 15
- 238000012937 correction Methods 0.000 description 19
- 238000010586 diagram Methods 0.000 description 9
- 230000009467 reduction Effects 0.000 description 5
- 238000004590 computer program Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Abstract
The present disclosure relates to an image processing method, an image processing apparatus, an electronic device, and a storage medium, wherein the method is applied to an image signal processor, and includes: acquiring an image to be processed and the type of information contained in the image; determining an image processing module related to the type; and processing the image by adopting an image processing module related to the type to obtain a processed image. Therefore, the image signal processor processes the images containing different types of information only by adopting the image processing modules related to the corresponding types, and can realize targeted processing on the images containing different types of information so as to improve the processing effect of the images.
Description
Technical Field
The present disclosure relates to the field of image processing technologies, and in particular, to an image processing method and apparatus, an electronic device, and a storage medium.
Background
With the continuous development of terminal technology, more and more users use electronic devices to capture images. Conventionally, after a camera in an electronic device captures an Image, an Image Signal Processor (ISP) in a processing chip in the electronic device may process the captured Image to obtain an Image in a format capable of being displayed.
In the related art, the ISP in the processing chip is dedicated to processing an image containing certain types of information, and when an image containing other types of information is processed, there may be a case where an image processing error occurs or the image is not processed.
Disclosure of Invention
The present disclosure is directed to solving, at least to some extent, one of the technical problems in the related art.
Therefore, the following technical scheme is provided in the disclosure:
an embodiment of a first aspect of the present disclosure provides an image processing method, applied to an image signal processor, where the method includes:
acquiring an image to be processed and the type of information contained in the image; determining an image processing module associated with the type; and processing the image by adopting an image processing module related to the type to obtain a processed image.
In an embodiment of the present disclosure, the acquiring an image to be processed and a type of information included in the image includes:
acquiring the image and the format of the image;
determining a type of information contained in the image according to the format.
In an embodiment of the present disclosure, the determining a type of information contained in the image according to the format includes:
when the format is a first format, determining that the type comprises a brightness type and a color type;
when the format is a second format, determining that the type includes only a brightness type.
In one embodiment of the present disclosure, the determining an image processing module related to the type includes:
determining a configuration file corresponding to the type, wherein the configuration file comprises: indicia for each image processing module, the indicia including a first indicia characterizing the corresponding image processing module as being associated with the type and a second indicia characterizing the corresponding image processing module as being not associated with the type;
and determining the image processing module with the first mark in the configuration file as the image processing module related to the type.
In one embodiment of the present disclosure, when the types include a brightness type and a color type, the labels of all image processing modules in the image signal processor are a first label;
when the types include only a brightness type, the image brightness processing module in the image signal processor is marked as a first mark, and other modules except the image brightness processing module are marked as second marks.
In one embodiment of the present disclosure, the configuration file further includes: image processing parameters of the image processing module corresponding to the first mark;
the processing the image by using the image processing module related to the type to obtain a processed image includes:
acquiring image processing parameters in the configuration file for each image processing module related to the type;
and controlling the image processing module to process the image according to the image processing parameters to obtain a processed image.
An embodiment of a second aspect of the present disclosure provides an image processing apparatus, including:
the device comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring an image to be processed and the type of information contained in the image;
a determination module for determining an image processing module associated with the type;
and the processing module is used for processing the image by adopting the image processing module related to the type to obtain a processed image.
In one embodiment of the present disclosure, the obtaining module includes:
an acquisition unit configured to acquire the image and a format of the image;
a first determining unit configured to determine a type of information contained in the image according to the format.
In an embodiment of the present disclosure, the determining unit is specifically configured to:
when the format is a first format, determining that the type comprises a brightness type and a color type;
when the format is a second format, determining that the type includes only a brightness type.
In one embodiment of the present disclosure, the determining module includes:
a second determining unit, configured to determine a configuration file corresponding to the type, where the configuration file includes: indicia for each image processing module, the indicia including a first indicia characterizing the corresponding image processing module as being associated with the type and a second indicia characterizing the corresponding image processing module as being not associated with the type;
and the processing unit is used for determining the image processing module with the first mark in the configuration file as the image processing module related to the type.
In one embodiment of the present disclosure, when the types include a brightness type and a color type, the labels of all image processing modules in the image signal processor are a first label;
when the types include only a brightness type, the image brightness processing module in the image signal processor is marked as a first mark, and other modules except the image brightness processing module are marked as second marks.
In one embodiment of the present disclosure, the configuration file further includes: image processing parameters of the image processing module corresponding to the first mark;
the processing module is specifically configured to:
acquiring image processing parameters in the configuration file for each image processing module related to the type;
and controlling the image processing module to process the image according to the image processing parameters to obtain a processed image.
An embodiment of a third aspect of the present disclosure provides an electronic device, including: a processor; a memory for storing executable instructions of the processor; the processor is configured to call and execute the executable instructions stored in the memory to implement the image processing method set forth in the embodiment of the first aspect of the present disclosure.
A fourth aspect of the present disclosure provides a non-transitory computer-readable storage medium, where instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the image processing method provided in the first aspect of the present disclosure.
A fifth aspect of the present disclosure provides a computer program product, which when executed by a processor of an electronic device, enables the electronic device to execute the image processing method provided in the first aspect of the present disclosure.
According to the technical scheme, the image to be processed and the type of information contained in the image are obtained through the ISP, the image processing module related to the type is determined, and the image processing module related to the type is adopted to process the image to obtain the processed image. Therefore, the ISP only adopts the image processing module related to the corresponding type to process the image aiming at the image containing different types of information, and can realize the aim processing of the image containing different types of information so as to improve the processing effect of the image. In addition, the image is processed through the ISP without the CPU or the GPU, so that the resource of the CPU or the GPU can be avoided being occupied.
Additional aspects and advantages of the disclosure will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the disclosure.
Drawings
The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 is a flow diagram of an image processing method according to an embodiment of the present disclosure;
FIG. 2 is a flow diagram of an image processing method according to another embodiment of the present disclosure;
FIG. 3 is a schematic diagram of an image processing flow in an embodiment of the present disclosure;
FIG. 4 is a flow diagram of an image processing method according to yet another embodiment of the present disclosure;
FIG. 5 is a flow diagram of an image processing method according to yet another embodiment of the present disclosure;
fig. 6 is a block diagram of the structure of an image processing apparatus according to an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
Detailed Description
Reference will now be made in detail to the embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are illustrative and intended to explain the present disclosure, and should not be construed as limiting the present disclosure.
At present, an ISP in a processing chip in an electronic device is dedicated to image processing for an image containing certain type of information, that is, the ISP can only support a camera in one format, and for images collected by cameras in other formats, an error may be directly reported without processing the image. Alternatively, the ISP has the same processing flow for all images, and does not perform differential processing on images containing different types of information, which may cause an error in image processing.
For example, taking the electronic device as a robot dog for example, an ISP in the robot dog only supports a camera in bayer format, but does not support other formats, such as a camera in mono raw format. The ISP in the robot dog has a good processing effect on the image collected by the camera in the bayer format, and the ISP can report an error on the image collected by the camera in the mono raw format without processing the image collected by the camera in the mono raw format.
In the related art, the image may be processed by a Central Processing Unit (CPU) or a Graphics Processing Unit (GPU) in the Processing chip, so as to improve the image quality. That is, the CPU or GPU may design a new image processing algorithm for an image containing other types of information to improve the processing effect of the image.
In this way, more CPU and GPU resources are occupied.
An image processing method, an apparatus, an electronic device, and a storage medium of the embodiments of the present disclosure are described below with reference to the drawings.
Fig. 1 is a flowchart of an image processing method according to an embodiment of the present disclosure.
In the embodiment of the present disclosure, the image processing method may be applied to an ISP, wherein the ISP is provided in a processing chip of an electronic device.
The electronic device can be an intelligent robot, a robot dog or any terminal device provided with a camera and a processing chip. The terminal device can be a hardware device with various operating systems, touch screens and/or display screens, such as a mobile phone, a tablet computer, a personal digital assistant and a wearable device.
The processing chip may include an image processor ISP, a CPU, a GPU, and an image encoder, among others.
As shown in fig. 1, the image processing method may include the steps of:
In the embodiment of the present disclosure, the image to be processed may be an image acquired by a camera in the electronic device.
In the disclosed embodiment, the information contained in the image may include at least one of color information and luminance information.
In the disclosed embodiment, the type of information contained in the image is determined by the camera that captured the image, and may include a brightness type and/or a color type.
In the embodiment of the present disclosure, the ISP may acquire an image acquired by the camera, which is denoted as an image to be processed in this application, and acquire a type of information included in the image to be processed.
At step 102, an image processing module associated with the type is determined.
In the embodiment of the present disclosure, the image processing module refers to a relevant module in the ISP for processing information contained in the image, and the image processing module may include at least one of the following: an Exposure module (e.g., an Automatic Exposure (AE) module), a correction module (e.g., a gamma correction (gamma correction) module, a color correction (color correction) module), a noise reduction (noise) module, an interpolation (demosaic) module, an Automatic White Balance (AWB) module, an Automatic Focusing (AF) module, and the like.
In the embodiment of the present disclosure, after determining the type of information contained in the image, the ISP may determine an image processing module related to the type or determine an image processing module matching the type.
As an example, the ISP determines the type of information included in the image to include a brightness type, and the image processing module related to the brightness type may be an image processing module for processing brightness information, which is denoted as an image brightness processing module in the present disclosure, and the image brightness processing module may include at least one of an exposure module, a gamma correction module, and a noise reduction module.
As another example, the ISP determines the type of information contained in the image to include a color type, and the image processing module related to the color type may be an image processing module for processing color information, which is denoted as an image color processing module in the present disclosure, and the image color processing module may include at least one of an interpolation module, an automatic white balance module, a color correction module, an RGB color space to YUV color space conversion module, a chromaticity/saturation adjustment module, and the like.
As still another example, the ISP determines the type of information included in the image to include a color type and a brightness type, and the image processing module related to the type may be all image processing modules in the ISP.
And 103, processing the image by adopting an image processing module related to the type to obtain a processed image.
In the embodiment of the disclosure, the image processing modules in the ISP are classified according to different types, and the ISP only uses the image processing module related to or matched with the type to process the image for the image containing different types of information, so that the image containing different types of information can be processed in a targeted manner, and the image processing effect is improved. In addition, the image is processed through the ISP without the CPU or the GPU, so that the resource of the CPU or the GPU can be avoided being occupied.
As an application scenario, if information included in an image only includes color information, an image color processing module may be determined from all image processing modules in the ISP, and the image may be processed only by the image color processing module.
As another application scenario, if the information included in the image only includes luminance information, the image luminance processing module may be determined from all the image processing modules in the ISP, and the image is processed only by using the image luminance processing module.
As another application scenario, if the information included in the image includes both color information and luminance information, the image may be processed by using all image processing modules in the ISP.
According to the image processing method, the image to be processed and the type of information contained in the image are obtained through the ISP, the image processing module related to the type is determined, and the image processing module related to the type is adopted to process the image to obtain the processed image. Therefore, the ISP only adopts the image processing module related to the corresponding type to process the image aiming at the image containing different types of information, and can realize the aim processing of the image containing different types of information so as to improve the processing effect of the image. In addition, the image is processed through the ISP without the CPU or the GPU, so that the resource of the CPU or the GPU can be avoided being occupied.
In a possible implementation manner of the embodiment of the present disclosure, the type of the information included in the image is related to a format of a camera that collects the image, that is, in the present disclosure, the type of the information included in the image may be determined according to the format of the camera that collects the image. The above process is described in detail below with reference to fig. 2.
Fig. 2 is a flowchart of an image processing method according to another embodiment of the present disclosure.
As shown in fig. 2, the image processing method may include the steps of:
In the embodiment of the present disclosure, the format of the image is related to the format of the camera acquiring the image, and the format of the acquired image may be different for cameras with different formats. For example, the format of the image may include a first format and a second format.
Each pixel in the image in the first format has both luminance information and color information, for example, the first format may be a bayer format; each pixel in the image in the second format has only luminance information and no color information, e.g., the second format may be mono raw format. For example, a normal camera may output an image in a first format (e.g., bayer format), and a black and white camera may output an image in a second format (e.g., mono raw format).
In the embodiment of the present disclosure, the correspondence between different formats and types may be preset, so that after the format of the image is determined, the correspondence may be queried according to the format of the image, and the type corresponding to the format of the image is obtained as the type of information included in the image.
As an example, the correspondence between the first mode and the brightness type and the color type may be preset, and the correspondence between the second format and the brightness type may be preset. Assuming that the format of the image is the second format, by querying the correspondence relationship, it can be determined that the type corresponding to the second format is the brightness type, and thus the brightness type can be used as the type of information contained in the image.
At step 203, the image processing module associated with the type is determined.
And step 204, processing the image by adopting an image processing module related to the type to obtain a processed image.
In the related technology, the ISP hardware pipeline only supports a camera in a bayer format, but does not support processing of an image acquired by a camera in a mono raw format, and when the system starts the camera in the mono raw format, if the ISP hardware pipeline detects that the image acquired by the camera is in the mono raw format, an error is reported, and the image is not processed.
In the embodiment of the present disclosure, the mono raw format camera information may be fixedly configured into the bayer format camera information by modifying the drive configuration file. After the driver software reads and writes the driver configuration file, the mono raw format camera is considered to be a bayer format camera, so that no error is reported, and the mono raw format image is sent to an ISP hardware pipeline for processing.
When the image in mono raw format is sent to the ISP hardware pipeline for processing, because the ISP hardware pipeline processes the image in bayer format, the image in mono raw format is processed in a messy way, which causes the processed image to be abnormal, and the output image is not usable.
As an example, the ISP hardware pipeline may include image processing modules as shown in fig. 3, the ISP hardware pipeline includes not only image processing modules related to color types but also image processing modules related to brightness types, and images in mono raw format have no color components, and if all the image processing modules in the ISP hardware pipeline are adopted, processing the images in mono raw format may cause image processing errors.
In view of the above problems, in the present disclosure, an image quality debugging file (tuning file) of an ISP may be modified, and for an image in a bayer format, as in the prior art, all image processing modules in an ISP hardware pipeline are used to process the image, and for an image in a mono raw format, only an image processing module related to a brightness type in the ISP hardware pipeline may be used to process the image, instead of an image processing module related to a color type, so that the image in the mono raw format is processed by necessary image processing modules such as lens correction, dead pixel compensation, noise reduction, automatic exposure, edge enhancement, contrast adjustment, and gamma correction, and is not processed by the image processing module related to the color type, thereby avoiding an error condition of a processed image frame, and improving a processing effect of the image.
It should be noted that, the foregoing is exemplarily illustrated by using only one ISP hardware pipeline, and in practical application, a plurality of ISP hardware pipelines may be further used to process images acquired by a plurality of cameras, that is, the images acquired by a plurality of cameras may be processed by different ISP hardware pipelines at the same time, which is not limited in this disclosure.
According to the image processing method, the image to be processed and the format of the image are obtained, the type of information contained in the image is determined according to the format, and the reliability of the type determination result can be improved.
In a possible implementation manner of the embodiment of the present disclosure, each type has a corresponding configuration file, and the configuration file corresponding to each type includes an image processing module related to the corresponding type, so that in the present disclosure, the ISP may acquire the image processing module related to the type from the configuration file corresponding to the type of information included in the image to be processed. The above process is explained in detail with reference to fig. 4.
Fig. 4 is a flowchart of an image processing method according to yet another embodiment of the present disclosure.
As shown in fig. 4, the image processing method may include the steps of:
In this disclosure, step 401 may be implemented by any one of the embodiments of the present disclosure, and this is not limited in this disclosure and is not described again.
In the embodiment of the present disclosure, each type has a corresponding configuration file, and the configuration file corresponding to each type includes an image processing module related to the corresponding type and an image processing module unrelated to the corresponding type. Wherein the image processing module related to the type has a first label which characterizes that the corresponding image processing module is related to the type, and the image processing module not related to the type has a second label which characterizes that the corresponding image processing module is not related to the type.
For example, taking the first label as numeral 1 and the second label as numeral 0 as an example, in the configuration file corresponding to type 1, the image processing module related to type 1 has a label of "1", the image processing module not related to type 1 has a label of "0", and for example, in the configuration file corresponding to type 2, the image processing module related to type 2 has a label of "1", and the image processing module not related to type 2 has a label of "0".
It should be noted that, the first mark is only the number 1, and the second mark is the number 0, but the disclosure is not limited thereto, for example, the first mark and the second mark may also be other different numbers, or the first mark and the second mark may also be different characters, such as the first mark is the character a, and the second mark is the character B, which is not limited by the disclosure.
In the embodiment of the present disclosure, the corresponding relationship between each type and the configuration file may be preset, so that in the present disclosure, after the type of the information included in the image is determined, the corresponding relationship may be queried according to the type, and the configuration file corresponding to the type may be determined.
In step 403, the image processing module with the first mark in the configuration file is determined as the image processing module related to the type.
And step 404, processing the image by using an image processing module related to the type to obtain a processed image.
In the embodiment of the present disclosure, the image processing module having the first flag in the configuration file may be used as an image processing module related to the type of information included in the image, so that the image processing module related to the type may be used to process the image to obtain a processed image. That is, the image processing module with the second flag in the configuration file may be disabled, and the image processing module with the first flag may be used to process the image to obtain a processed image. For example, for an image in the second format, the image processing module with the second flag may be an image color processing module, the image processing module with the first flag may be an image brightness processing module, the image color processing module may be disabled, and the image brightness processing module is used to process the image to obtain a processed image.
It can be understood that, for an image in the second format (each pixel has only luminance information and no color information), if all image processing modules are used to process the image, for example, image color processing modules (an interpolation module, an automatic white balance module, a color correction module, etc.) are used to process the image, the image color processing modules may affect the luminance of each pixel while processing the color of each pixel in the image, because the luminance may be represented by the information of the color, etc., and thus the luminance of the image may be lower.
In the application, the image in the second format is processed only by the image brightness processing module, so that the condition that the output image is darker can be avoided, and the image processing effect is improved. In other words, according to the image processing method and device, the images containing different types of information are processed only by the image processing modules related to the corresponding types, so that the images containing different types of information can be processed in a targeted manner, and the image processing effect is improved.
According to the image processing method, the image to be processed and the type of information contained in the image are obtained through the ISP, the image processing module related to the type is determined, and the image processing module related to the type is adopted to process the image to obtain the processed image. Therefore, the ISP only adopts the image processing module related to the corresponding type to process the image aiming at the image containing different types of information, and can realize the aim processing of the image containing different types of information so as to improve the processing effect of the image. In addition, the image is processed through the ISP without the CPU or the GPU, so that the resource of the CPU or the GPU can be avoided being occupied.
In a possible implementation manner of the embodiment of the present disclosure, the configuration file may further include image processing parameters of the image processing modules, so that in order to improve an image processing effect, the image processing parameters corresponding to the image processing modules may be used to process the image, and a processed image is obtained. The above process is described in detail below with reference to fig. 5.
Fig. 5 is a flowchart of an image processing method according to still another embodiment of the present disclosure.
As shown in fig. 5, the image processing method may include the steps of:
In this disclosure, step 501 may be implemented by any method in each embodiment of the disclosure, and this is not limited in this disclosure and is not described again.
In this disclosure, step 502 may be implemented by any manner in each embodiment of the present disclosure, and this is not limited in this disclosure and is not described again.
In one possible implementation manner of the embodiment of the present disclosure, when the types of information included in the image include a brightness type and a color type, the labels of all the image processing modules in the ISP are the first label. When the type of information included in the image includes only a brightness type, the label of the image brightness processing module in the ISP is a first label, and the labels of the modules other than the image brightness processing module are second labels.
That is to say, the configuration file may include all image processing modules in the ISP, when the type of information included in the image includes a brightness type and a color type, the labels of all image processing modules in the configuration file corresponding to the type may be the first label, and when the type of information included in the image includes only a brightness type, only the label of the image brightness processing module in the configuration file corresponding to the type is the first label, and the labels of other modules except the image brightness processing module are the second label.
As an example, the image processing module in the ISP includes an exposure module (such as an automatic exposure AE module), a gamma correction module, a noise reduction dense module, an interpolation dominant module, an automatic white balance AWB module, a color correction module, and an auto focus AF module, where the image processing module related to the brightness type is referred to as an image brightness processing module, the image brightness processing module may include at least one of the AE module, the gamma correction module, and the dense module, the image processing module related to the color type is referred to as an image color processing module, and the image color processing module may include at least one of the dominant module, the AWB module, and the color correction module.
Assuming that the format of the image is mono raw format, in the configuration file corresponding to the type of the information contained in the image, the marks of the AE module, the gamma correction module and the dense module are first marks, and the marks of the demosaic module, the AWB module and the color correction module are second marks. And assuming that the format of the image is a bayer format, the marks of all the image processing modules in the configuration file corresponding to the type of the information contained in the image are the first marks.
It should be noted that, the image brightness processing module is only exemplified by three modules, namely, the AE module, the gamma correction module and the dense module, and in practical application, the image brightness processing module may further include other image processing modules related to brightness, which is not limited in this disclosure. Similarly, the image color processing module includes a demosaic module, an AWB module, and a color correction module, and when in actual application, the image color processing module may further include other image processing modules related to colors.
In the embodiment of the present disclosure, the configuration file corresponding to the type of the information contained in the image may further include image processing parameters of the image processing module corresponding to the first mark, for example, the image processing parameters of the AE module may include a specific exposure duration, for example, in a case where the image is liable to be underexposed in a backlight scene, a longer exposure duration may be set, and in a case where the image is liable to be overexposed in a frontlight scene, a shorter exposure duration may be set.
As a possible implementation manner, in order to improve the quality of the processed image, each image processing module may have multiple sets of image processing parameters, where each set of image processing parameters is related to a shooting scene (e.g., a backlight scene, a frontlight scene, etc.), and when the image processing module is subsequently used to process the image, the image processing parameters related to the current shooting scene may be used to process the image, so as to improve the processing effect of the image.
In the embodiment of the present disclosure, step 503 may be implemented by any one of the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this, and are not described again.
In the embodiment of the present disclosure, after determining the image processing module related to the type of the information contained in the image, the ISP may further acquire the image processing parameters of the image processing module related to the type from the configuration file, that is, acquire the image processing parameters of the image processing module having the first flag.
And 505, controlling the image processing module to process the image according to the image processing parameters to obtain a processed image.
In this embodiment of the present disclosure, the ISP may process the image according to the image processing parameters corresponding to each image processing module to obtain a processed image.
As an example, the ISP may be provided with a demosaic module (for performing interpolation calculation on an image to obtain an image in RGB format), a gamma correction module, and a dense module (for performing noise reduction processing on an image to obtain an image in YUV format for preview). Before or after the demosaic module, the ISP may further be provided with an AE module, an AWB module (black and white balance, avoid the image being reddish, etc.), and an AF module.
The ISP may specifically process the image in the first format, such as the image in the bayer format, and the image processing modules related to the type corresponding to the image in the first format may be all image processing modules in the ISP. For the image in the second format, such as mono raw format, since it only carries luminance information and not color information, when the image is processed by ISP, the image processing module related to color in ISP, which is referred to as image color processing module in this application, may be disabled, and only the image processing module related to luminance in ISP, which is referred to as image luminance processing module in this application, is used. The forbidden module directly outputs the input data without processing.
The ISP may have a plurality of interfaces, one of which may be connected to the camera in the second format to process the image captured by the camera in the second format, and the other of which may be connected to the camera in the first format to process the image captured by the camera in the first format by using all the image processing modules in the ISP.
In a possible implementation manner of the embodiment of the present disclosure, JPEG encoding may also be adopted to encode the processed image, so as to obtain an image in a JPEG format to be displayed.
According to the image processing method, the image to be processed and the type of information contained in the image are obtained through the ISP, the image processing module related to the type is determined, and the image processing module related to the type is adopted to process the image to obtain the processed image. Therefore, the ISP only adopts the image processing module related to the corresponding type to process the image aiming at the image containing different types of information, and can realize the aim processing of the image containing different types of information so as to improve the processing effect of the image. In addition, the image is processed through the ISP without the CPU or the GPU, and therefore the resource of the CPU or the GPU can be prevented from being occupied.
Corresponding to the image processing method provided in the embodiment of fig. 1 to 5, the present disclosure also provides an image processing apparatus, and since the image processing apparatus provided in the embodiment of the present disclosure corresponds to the image processing method provided in the embodiment of fig. 1 to 5, the embodiment of the image processing method is also applicable to the image processing apparatus provided in the embodiment of the present disclosure, and will not be described in detail in the embodiment of the present disclosure.
Fig. 6 is a block diagram of the structure of an image processing apparatus according to an embodiment of the present disclosure. Wherein the image processing apparatus may be configured in an ISP.
As shown in fig. 6, the image processing apparatus 600 includes: an obtaining module 610, a determining module 620, and a processing module 630, wherein:
an obtaining module 610, configured to obtain an image to be processed and a type of information included in the image.
A determining module 620 for determining the image processing module related to the type.
And a processing module 630, configured to process the image by using the image processing module related to the type to obtain a processed image.
In an embodiment of the present disclosure, the obtaining module 610 may include:
and the acquisition unit is used for acquiring the image and the format of the image.
A first determining unit for determining the type of information contained in the image according to the format
In an embodiment of the present disclosure, the determining unit is specifically configured to: when the format is a first format, determining the type to comprise a brightness type and a color type; when the format is the second format, the determination type includes only the brightness type.
In an embodiment of the present disclosure, the determining module 620 may include:
a second determining unit, configured to determine a configuration file corresponding to the type, where the configuration file includes: the labels of the image processing modules comprise a first label and a second label, the first label is used for representing that the corresponding image processing module is related to the type, and the second label is used for representing that the corresponding image processing module is not related to the type.
And the processing unit is used for determining the image processing module with the first mark in the configuration file as the image processing module related to the type.
In one embodiment of the present disclosure, when the types include a brightness type and a color type, the labels of all image processing modules in the image signal processor are a first label; when the type includes only the brightness type, the image brightness processing module in the image signal processor is marked as a first mark, and the other modules except the image brightness processing module are marked as second marks.
In one embodiment of the present disclosure, the configuration file further comprises: image processing parameters of the image processing module corresponding to the first mark; the processing module 630 is specifically configured to: acquiring image processing parameters in a configuration file for each image processing module related to the type; and controlling the image processing module to process the image according to the image processing parameters to obtain a processed image.
The image processing device of the embodiment of the disclosure acquires the image to be processed and the type of information contained in the image through the ISP, and determines the image processing module related to the type, so as to process the image by using the image processing module related to the type, thereby obtaining the processed image. Therefore, the ISP only adopts the image processing module related to the corresponding type to process the image aiming at the image containing different types of information, and can realize the aim processing of the image containing different types of information so as to improve the processing effect of the image. In addition, the image is processed through the ISP without the CPU or the GPU, so that the resource of the CPU or the GPU can be avoided being occupied.
Corresponding to the image processing method provided in the embodiments of fig. 1 to 5, the present disclosure also provides an electronic device, and since the electronic device provided in the embodiments of the present disclosure corresponds to the image processing method provided in the embodiments of fig. 1 to 5, the implementation manner of the image processing method is also applicable to the electronic device provided in the embodiments of the present disclosure, and is not described in detail in the embodiments of the present disclosure.
Fig. 7 is a block diagram of an electronic device according to an embodiment of the present disclosure.
As shown in fig. 7, the electronic device 700 includes: a memory 710 and a processor 720, and a bus 730 that connects the various components, including the memory 710 and the processor 720.
Wherein, the memory 710 is used for storing executable instructions of the processor 720; the processor 720 is configured to call up and execute the executable instructions stored in the memory 710 to implement the image processing method proposed by the above-mentioned embodiment of the present disclosure.
A program/utility 780 having a set (at least one) of program modules 770 may be stored, for example, in memory 710, such program modules 770 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 770 typically perform the functions and/or methods of the embodiments described in this disclosure.
The electronic device 700 may also communicate with one or more external devices 790 (e.g., keyboard, pointing device, display 791, etc.), with one or more devices that enable a user to interact with the electronic device 700, and/or with any devices (e.g., network card, modem, etc.) that enable the electronic device 700 to communicate with one or more other computing devices. Such communication may occur over input/output (I/O) interfaces 792. Also, the electronic device 700 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the internet) via the network adapter 793. As shown, the network adapter 793 communicates with the other modules of the electronic device 700 over a bus 730. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
The processor 720 executes various functional applications and data processing by executing programs stored in the memory 710.
In order to achieve the above embodiments, the present disclosure also proposes a non-transitory computer-readable storage medium, where instructions of the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the image processing method proposed in any one of the foregoing embodiments.
In order to implement the foregoing embodiments, embodiments of the present disclosure further provide a computer program product, where when executed by a processor of an electronic device, the computer program enables the electronic device to execute an image processing method provided in any one of the foregoing embodiments.
According to the technical scheme, the image to be processed and the type of information contained in the image are obtained through the ISP, the image processing module related to the type is determined, and the image processing module related to the type is adopted to process the image to obtain the processed image. Therefore, the ISP only adopts the image processing module related to the corresponding type to process the image aiming at the image containing different types of information, and can realize the aim processing of the image containing different types of information so as to improve the processing effect of the image. In addition, the image is processed through the ISP without the CPU or the GPU, and therefore the resource of the CPU or the GPU can be prevented from being occupied.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims. It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (14)
1. An image processing method applied to an image signal processor, the method comprising:
acquiring an image to be processed and the type of information contained in the image;
determining an image processing module associated with the type;
and processing the image by adopting an image processing module related to the type to obtain a processed image.
2. The method of claim 1, wherein the obtaining of the image to be processed and the type of information contained in the image comprises:
acquiring the image and the format of the image;
determining a type of information contained in the image according to the format.
3. The method of claim 2, wherein said determining a type of information contained in the image according to the format comprises:
when the format is a first format, determining that the type comprises a brightness type and a color type;
determining that the type includes only a brightness type when the format is a second format.
4. The method according to any one of claims 1 to 3, wherein said determining an image processing module associated with said type comprises:
determining a configuration file corresponding to the type, wherein the configuration file comprises: indicia for each image processing module, the indicia including a first indicia characterizing the corresponding image processing module as being associated with the type and a second indicia characterizing the corresponding image processing module as being not associated with the type;
and determining the image processing module with the first mark in the configuration file as the image processing module related to the type.
5. The method of claim 4, wherein when the types include a brightness type and a color type, the labels of all image processing modules in the image signal processor are a first label;
when the types include only a brightness type, the image brightness processing module in the image signal processor is marked as a first mark, and other modules except the image brightness processing module are marked as second marks.
6. The method of claim 4, wherein the configuration file further comprises: image processing parameters of the image processing module corresponding to the first mark;
the processing the image by using the image processing module related to the type to obtain a processed image includes:
acquiring image processing parameters in the configuration file for each image processing module related to the type;
and controlling the image processing module to process the image according to the image processing parameters to obtain a processed image.
7. An image processing apparatus characterized by comprising:
the device comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring an image to be processed and the type of information contained in the image;
a determination module for determining an image processing module associated with the type;
and the processing module is used for processing the image by adopting the image processing module related to the type to obtain a processed image.
8. The apparatus of claim 7, wherein the obtaining module comprises:
an acquisition unit configured to acquire the image and a format of the image;
a first determining unit configured to determine a type of information contained in the image according to the format.
9. The apparatus according to claim 8, wherein the determining unit is specifically configured to:
when the format is a first format, determining that the type comprises a brightness type and a color type;
determining that the type includes only a brightness type when the format is a second format.
10. The apparatus according to any one of claims 7 to 9, wherein the determining means comprises:
a second determining unit, configured to determine a configuration file corresponding to the type, where the configuration file includes: indicia for each image processing module, the indicia including a first indicia characterizing the corresponding image processing module as being associated with the type and a second indicia characterizing the corresponding image processing module as being not associated with the type;
and the processing unit is used for determining the image processing module with the first mark in the configuration file as the image processing module related to the type.
11. The apparatus of claim 10, wherein when the types include a brightness type and a color type, the labels of all image processing modules in the image signal processor are a first label;
when the types only comprise the brightness type, the mark of the image brightness processing module in the image signal processor is a first mark, and the marks of other modules except the image brightness processing module are a second mark.
12. The apparatus of claim 10, wherein the configuration file further comprises: image processing parameters of the image processing module corresponding to the first mark;
the processing module is specifically configured to:
acquiring image processing parameters in the configuration file for each image processing module related to the type;
and controlling the image processing module to process the image according to the image processing parameters to obtain a processed image.
13. An electronic device, comprising:
a processor;
a memory for storing executable instructions of the processor;
wherein the processor is configured to invoke and execute the memory-stored executable instructions to implement the image processing method of any of claims 1-6.
14. A non-transitory computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the image processing method of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110897405.XA CN115705614A (en) | 2021-08-05 | 2021-08-05 | Image processing method, image processing device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110897405.XA CN115705614A (en) | 2021-08-05 | 2021-08-05 | Image processing method, image processing device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115705614A true CN115705614A (en) | 2023-02-17 |
Family
ID=85178844
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110897405.XA Pending CN115705614A (en) | 2021-08-05 | 2021-08-05 | Image processing method, image processing device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115705614A (en) |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100060953A1 (en) * | 2008-09-11 | 2010-03-11 | Ricoh Company, Ltd. | Image processing apparatus and method and computer program product for controlling the same |
| US20130308006A1 (en) * | 2010-11-01 | 2013-11-21 | Nokia Corporation | Tuning of digital image quality |
| CN103458173A (en) * | 2012-06-01 | 2013-12-18 | 华为技术有限公司 | Device, method and terminal for processing image relevant data |
| CN103472971A (en) * | 2013-09-03 | 2013-12-25 | 小米科技有限责任公司 | Method, device and terminal equipment for setting photographing parameters |
| CN106506962A (en) * | 2016-11-29 | 2017-03-15 | 维沃移动通信有限公司 | A kind of image processing method and mobile terminal |
| CN107347137A (en) * | 2017-06-30 | 2017-11-14 | 维沃移动通信有限公司 | The processing method and mobile terminal of a kind of view data |
| CN108234933A (en) * | 2016-12-21 | 2018-06-29 | 上海杰图软件技术有限公司 | The method and system of real-time splicing panorama image based on multiway images signal processing |
| CN108605099A (en) * | 2016-10-17 | 2018-09-28 | 华为技术有限公司 | The method and terminal taken pictures for terminal |
| CN108632512A (en) * | 2018-05-17 | 2018-10-09 | Oppo广东移动通信有限公司 | Image processing method, device, electronic equipment and computer readable storage medium |
| CN108810406A (en) * | 2018-05-23 | 2018-11-13 | Oppo广东移动通信有限公司 | Portrait light efficiency processing method, device, terminal and computer readable storage medium |
| CN108900819A (en) * | 2018-08-20 | 2018-11-27 | Oppo广东移动通信有限公司 | Image processing method, device, storage medium and electronic equipment |
| WO2019006762A1 (en) * | 2017-07-07 | 2019-01-10 | 华为技术有限公司 | Image capturing apparatus and method |
| CN111343386A (en) * | 2018-12-19 | 2020-06-26 | 北京小米移动软件有限公司 | Image signal processing method and device, electronic device and storage medium |
| CN111614908A (en) * | 2020-05-29 | 2020-09-01 | Oppo广东移动通信有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN111696039A (en) * | 2020-05-28 | 2020-09-22 | Oppo广东移动通信有限公司 | Image processing method and device, storage medium and electronic equipment |
| CN111901520A (en) * | 2020-06-26 | 2020-11-06 | 深圳蚂里奥技术有限公司 | Scene self-adaptive system, method and terminal based on image processing |
-
2021
- 2021-08-05 CN CN202110897405.XA patent/CN115705614A/en active Pending
Patent Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100060953A1 (en) * | 2008-09-11 | 2010-03-11 | Ricoh Company, Ltd. | Image processing apparatus and method and computer program product for controlling the same |
| US20130308006A1 (en) * | 2010-11-01 | 2013-11-21 | Nokia Corporation | Tuning of digital image quality |
| CN103458173A (en) * | 2012-06-01 | 2013-12-18 | 华为技术有限公司 | Device, method and terminal for processing image relevant data |
| CN103472971A (en) * | 2013-09-03 | 2013-12-25 | 小米科技有限责任公司 | Method, device and terminal equipment for setting photographing parameters |
| CN108605099A (en) * | 2016-10-17 | 2018-09-28 | 华为技术有限公司 | The method and terminal taken pictures for terminal |
| CN106506962A (en) * | 2016-11-29 | 2017-03-15 | 维沃移动通信有限公司 | A kind of image processing method and mobile terminal |
| CN108234933A (en) * | 2016-12-21 | 2018-06-29 | 上海杰图软件技术有限公司 | The method and system of real-time splicing panorama image based on multiway images signal processing |
| CN107347137A (en) * | 2017-06-30 | 2017-11-14 | 维沃移动通信有限公司 | The processing method and mobile terminal of a kind of view data |
| CN109923850A (en) * | 2017-07-07 | 2019-06-21 | 华为技术有限公司 | A kind of image capture device and method |
| WO2019006762A1 (en) * | 2017-07-07 | 2019-01-10 | 华为技术有限公司 | Image capturing apparatus and method |
| CN108632512A (en) * | 2018-05-17 | 2018-10-09 | Oppo广东移动通信有限公司 | Image processing method, device, electronic equipment and computer readable storage medium |
| CN108810406A (en) * | 2018-05-23 | 2018-11-13 | Oppo广东移动通信有限公司 | Portrait light efficiency processing method, device, terminal and computer readable storage medium |
| CN108900819A (en) * | 2018-08-20 | 2018-11-27 | Oppo广东移动通信有限公司 | Image processing method, device, storage medium and electronic equipment |
| CN111343386A (en) * | 2018-12-19 | 2020-06-26 | 北京小米移动软件有限公司 | Image signal processing method and device, electronic device and storage medium |
| CN111696039A (en) * | 2020-05-28 | 2020-09-22 | Oppo广东移动通信有限公司 | Image processing method and device, storage medium and electronic equipment |
| CN111614908A (en) * | 2020-05-29 | 2020-09-01 | Oppo广东移动通信有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN111901520A (en) * | 2020-06-26 | 2020-11-06 | 深圳蚂里奥技术有限公司 | Scene self-adaptive system, method and terminal based on image processing |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11375128B2 (en) | Method for obtaining exposure compensation values of high dynamic range image, terminal device and non-transitory computer-readable storage medium | |
| US11373275B2 (en) | Method for generating high-resolution picture, computer device, and storage medium | |
| KR102149187B1 (en) | Electronic device and control method of the same | |
| WO2017215501A1 (en) | Method and device for image noise reduction processing and computer storage medium | |
| US10896634B2 (en) | Image signal processing apparatus and control method therefor | |
| JP6615917B2 (en) | Real-time video enhancement method, terminal, and non-transitory computer-readable storage medium | |
| US20170099435A1 (en) | Image Generation Method Based On Dual Camera Module And Dual Camera Apparatus | |
| CN111885312B (en) | HDR image imaging method, system, electronic device and storage medium | |
| CN110855958B (en) | Image adjusting method and device, electronic equipment and storage medium | |
| KR20200011000A (en) | Device and method for augmented reality preview and positional tracking | |
| CN108259754B (en) | Image processing method and device, computer readable storage medium and computer device | |
| CN110198419A (en) | Image processing method, device, storage medium and electronic equipment | |
| US10091415B2 (en) | Image processing apparatus, method for controlling image processing apparatus, image pickup apparatus, method for controlling image pickup apparatus, and recording medium | |
| CN112383772A (en) | Camera performance automatic test method and device, electronic equipment and storage medium | |
| WO2023010755A1 (en) | Hdr video conversion method and apparatus, and device and computer storage medium | |
| CN115705614A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN113315965B (en) | Optical anti-shake test method, device, equipment and medium | |
| TWI825410B (en) | Image processing methods, devices, photographic equipment and storage media | |
| WO2021259191A1 (en) | Video mirror processing method and apparatus, photographic device, and readable storage medium | |
| CN114554126B (en) | Baseboard management control chip, video data transmission method and server | |
| CN115118947B (en) | Image processing method and device, electronic equipment and storage medium | |
| WO2022227200A1 (en) | Method and apparatus for adjusting resolution during image capture, and smart device | |
| CN112185312B (en) | Image data processing method and device | |
| CN115529448B (en) | Image processing method and related device | |
| US20240029320A1 (en) | Image processing system and image object superimposition apparatus and method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |