CN114022367A

CN114022367A - Image quality adjusting method, device, electronic equipment and medium

Info

Publication number: CN114022367A
Application number: CN202111154585.9A
Authority: CN
Inventors: 董波; 石景怡; 丁悦; 姜宇航; 顾礼将
Original assignee: Dalian Thundersoft Co ltd
Current assignee: Dalian Thundersoft Co ltd
Priority date: 2021-09-29
Filing date: 2021-09-29
Publication date: 2022-02-08

Abstract

The embodiment of the invention provides an image quality adjusting method, an image quality adjusting device, electronic equipment and a readable storage medium, wherein the method comprises the following steps: determining the type of a chip platform currently used for image quality adjustment and determining image effect parameter information required to be achieved after image adjustment; selecting a target platform tuning flow corresponding to the chip platform type from preset platform tuning flows, and determining shooting reference parameter information corresponding to the chip platform type; and based on the image effect parameter information, performing image quality tuning according to the target platform tuning flow and the shooting reference parameter information. According to the embodiment of the invention, a user does not need to learn a knowledge system for image quality adjustment of any chip platform, and can obtain a corresponding image quality adjustment result by photographing and uploading the feedback information in the cloud adjustment system, so that the problem of high learning cost caused by chip platform differentiation can be greatly reduced, and the image quality adjustment efficiency is improved.

Description

Image quality adjusting method, device, electronic equipment and medium

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to an image quality tuning method, an image quality tuning apparatus, an electronic device, and a computer-readable storage medium.

Background

The vision technology of the intelligent Internet of things industry is rapidly developed, and the purposes of high information content and low hardware investment can be achieved by a camera, a look-around monitoring device and a vision device by means of a lens with a wide field angle. In 2021, the number of cameras required by the IOT (Internet of Things) industry would be in the billions.

Certain quality problems still exist with current camera products, including: modeling problems (distortion, chromatic aberration, blur, mottle, etc.), Sensor problems (noise, color, dead spots, etc.), reduction, mounting accuracy (viewing angle difference, viewing axis deviation, tilt deviation, etc.). The module and sensor problem can be corrected by adjusting the Image Quality Tuning (Image Quality Tuning), which means that the performance of the camera is optimized by adjusting the system software, hardware and optical parameters according to the application requirements of the camera. However, the tools, processes and module groups for adjusting the imaging quality are different for different processing chips, and have great difference. For example, under Haesi platform there is a separate module 3DNR for denoising, whereas high-pass platform does not have this module. The differences between the different platforms make the platforms relatively independent.

When the mutually independent differentiation platforms are subjected to Tuning work, workers need to learn Tuning processes of different platforms, and due to the fact that the potential possibility of Tuning under the platform is unknown, the situation of non-convergence caused by unknown targets can occur in the Tuning process, namely, relevant parameters are continuously adjusted, then, shooting and testing are continuously carried out, and finally, a subjective relatively approved result is obtained through multiple iterations and is used as the optimum Tuning state. The existing platform differentiation problem causes high learning cost of Tuning work and large manpower input.

Disclosure of Invention

In view of the above problems, embodiments of the present invention are proposed to provide an image quality tuning method and a corresponding image quality tuning apparatus, an electronic device, and a computer-readable storage medium that overcome or at least partially solve the above problems.

The embodiment of the invention discloses an image quality tuning method, which is applied to a cloud debugging system and comprises the following steps:

determining the type of a chip platform currently used for image quality adjustment and determining image effect parameter information required to be achieved after image adjustment;

selecting a target platform tuning flow corresponding to the chip platform type from preset platform tuning flows, and determining shooting reference parameter information corresponding to the chip platform type;

and based on the image effect parameter information, performing image quality tuning according to the target platform tuning flow and the shooting reference parameter information.

Optionally, the adjusting image quality according to the target platform adjusting process and the shooting reference parameter information based on the image effect parameter information includes:

determining imaging equipment currently used for adjusting and optimizing image quality, and determining comparison imaging equipment of the imaging equipment;

acquiring a shot image obtained by shooting an image according to a shooting flow and the shooting reference parameter information in the target platform tuning flow by using the comparison imaging equipment;

performing image quality adjustment and optimization on the shot image to obtain corresponding image quality optimization parameter information; the image quality optimization parameter information can enable the shot image subjected to image optimization processing to achieve an image effect corresponding to the image effect parameter information;

and performing parameter compiling on the image quality optimization parameter information based on the chip platform type.

Optionally, the adjusting and optimizing the image quality of the shot image to obtain corresponding image quality optimization parameter information includes:

carrying out image quality evaluation based on automatic segmentation on the shot image;

and after the evaluation is passed, carrying out image optimization processing based on a preset degradation model on the shot image to obtain corresponding image quality optimization parameter information.

Optionally, the preset platform tuning flow is generated as follows:

determining necessary parameter information which needs to be input for image tuning; the necessary parameter information is determined according to an operation instruction document of the chip platform; the necessary parameter information comprises at least one of execution sequence, image test card type and shooting notice of each tuning module;

generating an initial platform tuning flow aiming at the chip platform based on the necessary parameter information;

and acquiring image tuning project data corresponding to the chip platform, and adjusting the initial platform tuning flow according to the image tuning project data to generate the corresponding preset platform tuning flow.

Optionally, the adjusting the initial platform tuning flow according to the image tuning project data includes:

taking the item as a unit from the image tuning item data, taking the shooting chart and the shooting condition adopted by the newly added module in the platform tuning flow corresponding to each item as statistical objects, and calculating the distribution probability of each statistical object in all items respectively;

and if the distribution probability is greater than the preset confidence probability, determining the statistical object corresponding to the distribution probability, and updating the adjustment object corresponding to the statistical object in the initial platform adjustment process.

Optionally, the adjusting the initial platform tuning flow according to the image tuning project data further includes:

if the distribution probability is not greater than the preset confidence probability, determining a reference item matched with the current item from a plurality of items of the image tuning item data;

and adjusting the initial platform tuning flow according to the platform tuning flow of the reference item.

Optionally, the determining the shooting reference parameter information corresponding to the chip platform type includes:

determining a digital imaging combination type of the imaging device; the digital imaging combination type is composed of a chip platform type, an image signal processing unit type, an imaging unit type and a lens type;

and judging whether the number of the image tuning items corresponding to the digital imaging combination type is greater than a preset number threshold, and determining the shooting reference parameter information according to the judgment result.

Optionally, the determining the shooting reference parameter information according to the judgment result includes:

if the number is larger than the preset number threshold, acquiring a corresponding coding and decoding network model; the encoding and decoding network model is obtained by taking historical shooting parameter information of the image tuning item as output and taking historical image effect parameter information of the image tuning item as input training;

inputting the image effect parameter information into the coding and decoding network model, and outputting the corresponding shooting reference parameter information;

and if the number is not greater than the preset number threshold, adopting shooting parameter information in a development code packet corresponding to the digital imaging combination type as the shooting reference parameter information.

Optionally, the determining the contrast imaging device of the imaging device includes:

if the cloud debugging system has an image tuning item with the same type as the digital imaging combination type of the imaging equipment, taking the imaging equipment of the image tuning item as candidate comparison imaging equipment;

if the cloud debugging system does not have the image tuning item with the same digital imaging combination type as the imaging device, comparing the resolution of the imaging unit adopted by the current image tuning item with the resolution of the imaging unit adopted by the prior image tuning item, and taking the imaging device of the image tuning item with the least resolution difference as the candidate comparison imaging device;

determining an image quality evaluation result of the candidate comparison imaging equipment;

and sequencing the evaluation results, calculating the rank of the image quality loss of each candidate comparison imaging device, taking the candidate comparison imaging device with the last rank as the optimal comparison imaging device, and taking the candidate comparison imaging device in the middle of the rank as the reference comparison imaging device.

Optionally, before the image quality evaluation based on automatic segmentation is performed on the captured image, the method further includes:

judging whether shooting irregularity exists in the process of acquiring the shot image;

if yes, executing the operation of feeding back the shooting problem;

and if the image quality evaluation operation does not exist, performing image quality evaluation operation based on automatic segmentation on the shot image.

Optionally, the determining whether shooting non-specifications exist in the process of acquiring the shot image includes:

carrying out graying processing on the shot image to obtain a grayscale image corresponding to the shot image;

carrying out wide dynamic stretching on the gray level image to obtain an enhanced gray level image;

determining an image template, and determining an image area matched with the image template in the enhanced gray-scale image based on an MMSER method;

and judging whether shooting non-specifications exist in the process of acquiring the shot image or not based on the image area.

Optionally, the performing image quality evaluation based on automatic segmentation on the captured image includes:

segmenting the shot image based on a preset segmentation model to obtain a corresponding segmentation result; the preset segmentation model is obtained by adopting an objective image card scene, a subjective image scene and a corresponding label image to establish an image group and performing segmentation training on the image group;

correcting an image segmentation area of the shot image according to the segmentation result;

for the same group of shot images, calculating the shot images in the same area based on objective parameters, and comparing the calculation results with the objective parameters of the shot images shot by the corresponding optimal comparison imaging equipment one by one;

and evaluating the image quality according to the comparison result.

Optionally, the performing, on the captured image, an image optimization process based on a preset degradation model includes:

performing color optimization processing on the shot image;

carrying out wiener filtering optimization processing on the shot image;

and carrying out noise optimization processing on the shot image.

Optionally, the performing parameter compilation on the image quality optimization parameter information based on the chip platform type includes:

and performing parameter compiling on the image quality optimization parameter information in a column vector mode based on the chip platform type.

The embodiment of the invention also discloses an image quality adjusting and optimizing device, which is applied to a cloud adjusting and debugging system, and the device comprises:

the first determining module is used for determining the type of a chip platform which is currently used for carrying out image quality adjustment and determining image effect parameter information which needs to be achieved after image adjustment;

the second determination module is used for selecting a target platform tuning flow corresponding to the chip platform type from preset platform tuning flows and determining the shooting reference parameter information corresponding to the chip platform type;

and the adjusting and optimizing module is used for adjusting and optimizing the image quality according to the target platform adjusting and optimizing flow and the shooting reference parameter information based on the image effect parameter information.

Optionally, the tuning module includes:

the first determining submodule is used for determining the imaging equipment which is currently used for adjusting and optimizing the image quality and determining the comparison imaging equipment of the imaging equipment;

the acquisition sub-module is used for acquiring a shot image obtained by shooting the image according to the shooting flow and the shooting reference parameter information in the target platform tuning flow by adopting the comparison imaging equipment;

the adjusting and optimizing module is used for adjusting and optimizing the image quality of the shot image to obtain corresponding image quality optimization parameter information; the image quality optimization parameter information can enable the shot image subjected to image optimization processing to achieve an image effect corresponding to the image effect parameter information;

and the compiling submodule is used for carrying out parameter compiling on the image quality optimization parameter information based on the chip platform type.

Optionally, the tuning sub-module includes:

the evaluation unit is used for evaluating the image quality of the shot image based on automatic segmentation;

and the adjusting and optimizing unit is used for carrying out image optimization processing based on a preset degradation model on the shot image after the evaluation is passed, so as to obtain the corresponding image quality optimization parameter information.

Optionally, the preset platform tuning flow is generated as follows, and the apparatus further includes:

the third determining module is used for determining necessary parameter information which needs to be input for image tuning; the necessary parameter information is determined according to an operation instruction document of the chip platform; the necessary parameter information comprises at least one of execution sequence, image test card type and shooting notice of each tuning module;

the generating module is used for generating an initial platform tuning flow aiming at the chip platform based on the necessary parameter information;

and the adjusting module is used for acquiring image tuning project data corresponding to the chip platform and adjusting the initial platform tuning flow according to the image tuning project data so as to generate the corresponding preset platform tuning flow.

Optionally, the adjusting module includes:

the calculation submodule is used for calculating the distribution probability of each statistical object in all the projects by taking the project as a unit from the image tuning project data and taking the shooting chart and the shooting condition adopted by the newly added module in the platform tuning process corresponding to each project as the statistical object;

and the updating submodule is used for determining the statistical object corresponding to the distribution probability and updating the adjusting object corresponding to the statistical object in the initial platform tuning process if the distribution probability is greater than the preset confidence probability.

Optionally, the adjusting module further includes:

the selecting submodule is used for determining a reference item matched with the current item from a plurality of items of the image tuning item data if the distribution probability is not greater than the preset confidence probability;

and the adjusting submodule is used for adjusting the initial platform tuning flow according to the platform tuning flow of the reference item.

Optionally, the second determining module includes:

a second determination sub-module for determining a digital imaging combination type of the imaging device; the digital imaging combination type is composed of a chip platform type, an image signal processing unit type, an imaging unit type and a lens type;

and the judging submodule is used for judging whether the number of the image tuning items corresponding to the digital imaging combination type is larger than a preset number threshold value or not and determining the shooting reference parameter information according to the judgment result.

Optionally, the determining sub-module includes:

an obtaining unit, configured to obtain a corresponding coding and decoding network model if the number is greater than the preset number threshold; the encoding and decoding network model is obtained by taking historical shooting parameter information of the image tuning item as output and taking historical image effect parameter information of the image tuning item as input training;

the input/output unit is used for inputting the image effect parameter information into the coding/decoding network model and outputting the corresponding shooting reference parameter information;

and the first determining unit is used for adopting shooting parameter information in a development code packet corresponding to the digital imaging combination type as the shooting reference parameter information if the number is not greater than the preset number threshold.

Optionally, the first determining sub-module includes:

the second determining unit is used for taking the imaging equipment of the image tuning item as candidate comparison imaging equipment if the cloud debugging system has the image tuning item with the same digital imaging combination type as the imaging equipment;

a third determining unit, configured to, if the cloud debugging system does not have an image tuning item that is the same as the digital imaging combination type of the imaging device, compare a resolution of an imaging unit used in a current image tuning item with a resolution of an imaging unit used in a previous image tuning item, and use an imaging device of the image tuning item with the smallest resolution difference as the candidate comparison imaging device;

the fourth determining unit is used for determining the image quality evaluation result of the candidate comparison imaging device;

and the fifth determining unit is used for sequencing the evaluation results, calculating the rank of the image quality loss of each candidate comparison imaging device, taking the candidate comparison imaging device with the last rank as the optimal comparison imaging device, and taking the candidate comparison imaging device in the middle of the rank as the reference comparison imaging device.

Optionally, the tuning sub-module further includes:

the judging unit is used for judging whether shooting non-specifications exist in the process of acquiring the shot image;

the first execution unit is used for executing the operation of feeding back the shooting problem if the shooting problem exists;

and the second execution unit is used for executing image quality evaluation operation based on automatic segmentation on the shot image if the shot image does not exist.

Optionally, the determining unit includes:

the gray processing subunit is used for carrying out gray processing on the shot image to obtain a gray image corresponding to the shot image;

the stretching subunit is used for carrying out wide dynamic stretching on the gray level image to obtain an enhanced gray level image;

the determining subunit is used for determining an image template and determining an image area matched with the image template in the enhanced gray-scale image based on an MMSER method;

and the judging subunit is used for judging whether shooting irregularity exists in the process of acquiring the shot image or not based on the image area.

Optionally, the evaluation unit includes:

the segmentation subunit is used for segmenting the shot image based on a preset segmentation model to obtain a corresponding segmentation result; the preset segmentation model is obtained by adopting an objective image card scene, a subjective image scene and a corresponding label image to establish an image group and performing segmentation training on the image group;

the correction subunit is used for correcting the image segmentation area of the shot image according to the segmentation result;

the comparison subunit is used for calculating the shot images in the same area based on the objective parameters for the same group of shot images, and comparing the calculation results with the objective parameters of the shot images shot by the corresponding optimal comparison imaging equipment one by one;

and the evaluation subunit is used for carrying out image quality evaluation according to the comparison result.

Optionally, the tuning unit includes:

the first adjusting and optimizing subunit is used for carrying out color optimization processing on the shot image;

the second tuning subunit is used for carrying out wiener filtering optimization processing on the shot image;

and the third tuning subunit is used for carrying out noise optimization processing on the shot image.

Optionally, the compiling submodule includes:

and the compiling unit is used for carrying out parameter compiling on the image quality optimization parameter information in a column vector mode based on the chip platform type.

The embodiment of the invention also discloses an electronic device, which comprises: a processor, a memory and a computer program stored on the memory and capable of running on the processor, the computer program, when executed by the processor, implementing the steps of a method of image quality tuning as described above.

The embodiment of the invention also discloses a computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, and when the computer program is executed by a processor, the steps of the image quality tuning method are realized.

The embodiment of the invention has the following advantages:

in the embodiment of the invention, the cloud debugging system can determine the target platform tuning flow according to the type of the chip platform and carry out image quality tuning according to the target platform tuning flow. By adopting the method, the user does not need to learn a knowledge system for image quality adjustment of any chip platform, and can obtain a corresponding image quality adjustment result by photographing and uploading the feedback information in the cloud adjustment system, so that the problem of high learning cost caused by chip platform differentiation can be greatly reduced, and the image quality adjustment efficiency is improved.

Drawings

Fig. 1 is a flowchart illustrating steps of an image quality tuning method according to an embodiment of the present invention;

fig. 2 is a schematic architecture diagram of a cloud debugging system according to an embodiment of the present invention;

FIG. 3 is a flowchart illustrating steps of another method for adjusting image quality according to an embodiment of the present invention;

FIG. 4 is a subjective scene graph and corresponding labeled graph;

fig. 5 is a block diagram of an image quality adjusting apparatus according to an embodiment of the present invention.

Detailed Description

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, not all of them. All other embodiments that can be derived by one of ordinary skill in the art from the embodiments given herein are intended to be within the scope of the present invention.

Camera Tuning refers to a process of adjusting the imaging quality of a Camera by a chip platform tool. The Tuning chip platform is an integrated circuit assembly with an independent computing control unit (CPU), supports System On Chip (SOC) development, and can support software and hardware expansion (such as video card, storage, video capture, and the like). The contrast machine is a camera which takes images and the video quality is considered to reach the standard. A Tuning module refers to a relatively centralized collection of functions in an imaging processing unit (ISP) that support Tuning. The exposure table (ExposureTable) represents a correspondence relationship between a combination of the shutter speed and f-number of the camera and the actual automatic exposure state.

The vision technology of the intelligent Internet of things industry is rapidly developed, and the purposes of high information content and low hardware investment can be achieved by depending on wide-field-angle lenses and cameras and looking around monitoring and vision equipment. In 2021, the IOT industry will demand hundreds of millions of cameras.

Product quality problems greatly restrict the camera supply period, including: modeling problems (distortion, chromatic aberration, blur, mottle, etc.), Sensor problems (noise, color, dead spots, etc.), reduction, mounting accuracy (viewing angle difference, viewing axis deviation, tilt deviation, etc.). The module and sensor problem can be corrected by CameraTuning, but the tools, processes and modules for adjusting the imaging quality are different, even have larger differences, for different processing chips. For example: under the Haesi platform, there is a separate module 3DNR for denoising, while the high-pass platform does not exist. The differences between the different platforms make the platforms relatively independent.

When the chip platform with independent differentiation is subjected to Tuning work, CameraTuning staff need to learn Tuning processes of different platforms, and due to the fact that the potential possibility of Tuning under the platform is unknown, the situation of non-convergence caused by unknown targets can occur in the Tuning process, namely, relevant parameters are continuously adjusted, then, shooting test is continuously performed, and finally, a subjective relative approval result is achieved through multiple iterations to serve as the optimum Tuning state.

Still, the difference between different chip platforms is solved in a manual mode, and the process of Tuning implemented between different chip platforms is as follows:

1. platform tool usage learning;

2. experimental trials of platform tools;

3. problem summary and accumulation;

4. the real Tuning task implementation comprises the following steps:

a) compiling based on parameters of platform default imaging quality; b) shooting a special scene image based on a compiling result, and exporting data; the compiling result refers to the Tuning parameter of the adaptive current platform; c) importing data into a platform tool to perform initial Tuning; d) adjusting the Tuning parameter through the difference between the subjective quality and the objective quality of the image shot by the comparison machine; e) compiling parameters based on a platform compiler; f) repeating the processes b) to e) until the quality is subjectively considered to be substantially consistent.

5. And compiling the final Tuning parameters by the platform compiler to complete Tuning.

As can be seen from the above, for a typical Tuning task, the learning cost of the human power for the platform needs to be measured in months, and the whole operations of the shooting, Tuning and evaluation processes in the implementation process need to be measured in months, and if there are other special scenes, the human cost for learning and Tuning is higher.

The Tuning scheme for platform differentiation does not exist, but there are many optimization schemes for Tuning implementation processes, such as a quality automatic evaluation system and an automatic shooting system. Although these schemes can reduce the Tuning human input and improve the efficiency, the improvement of the efficiency has little effect on the learning cost caused by platform differentiation.

In conclusion, the learning cost caused by platform differentiation is too high, and the manpower input is too large. Due to empirical accumulation caused by insufficient matching degree of the operation specification document of the platform and the version of the platform tool, various problems of Tuning can not be processed, and the problem processing difficulty caused by difference between platforms is larger. The current instrument software problem that provides of platform is more, and the short time is difficult to rely on the platform firm to solve the problem, greatly influences the Tuning efficiency.

One of the core ideas of the embodiment of the invention is that a target platform tuning flow can be determined according to the type of a chip platform in a cloud debugging system, and image quality tuning is carried out according to the target platform tuning flow. By adopting the method, the user does not need to learn a knowledge system for image quality adjustment of any chip platform, and can obtain a corresponding image quality adjustment result by photographing and uploading the feedback information in the cloud adjustment system, so that the problem of high learning cost caused by chip platform differentiation can be greatly reduced, and the image quality adjustment efficiency is improved.

Referring to fig. 1, a flowchart illustrating steps of an image quality tuning method provided in an embodiment of the present invention is shown, and is applied to a cloud debugging system, where the method specifically includes the following steps:

step 101, determining the type of a chip platform currently used for image quality adjustment and determining image effect parameter information required to be achieved after image adjustment.

In the embodiment of the invention, image quality Tuning aiming at different chip platform types can be performed in the cloud debugging system, namely Camera Tuning is performed.

Fig. 2 is a schematic diagram of an architecture of a cloud debugging system according to an embodiment of the present invention. The cloud debugging system comprises 6 key function modules, including a platform differentiation compatible module, an integrated automatic Tuning tool module, an integrated automatic evaluation module, a Tuning data cloud compiling module, a Tuning data cloud storage, management and mining module, a basic interaction module and a communication module.

The type of the chip platform currently used for adjusting the image quality can be determined, and the image effect parameter information required to be achieved after the image is adjusted can be determined. In one example, after a user logs in a cloud debugging system, a chip platform type currently used for image quality tuning can be selected from a plurality of chip platform types provided by the cloud debugging system, and information of image effect parameters to be achieved can be input into the cloud debugging system. The image effect parameter information includes definition, noise level, exposure curve, and the like. The process of determining the chip platform type and the image effect parameter information can be carried out in an interactive and communication module.

And 102, selecting a target platform tuning flow corresponding to the chip platform type from preset platform tuning flows, and determining the shooting reference parameter information corresponding to the chip platform type.

In the embodiment of the invention, the chip platform type corresponds to the platform tuning flow one by one, and the cloud debugging system stores the corresponding relation between the chip platform type and the platform tuning flow in advance. In addition, the chip platform type corresponds to the shooting reference parameter information one by one, and the cloud debugging system also stores the corresponding relation between the chip platform type and the corresponding shooting reference parameter information in advance. After the chip platform type is determined, a target platform tuning flow corresponding to the selected chip platform type can be selected from preset platform tuning flows, and shooting reference parameter information corresponding to the chip platform type is determined. When the cloud debugging system provides the target platform tuning flow, the cloud debugging system can also provide an operation guidance instruction of the corresponding target platform tuning flow.

And 103, based on the image effect parameter information, performing image quality tuning according to the target platform tuning flow and the shooting reference parameter information.

In the embodiment of the present invention, based on the image effect parameter information, image quality tuning may be performed according to the target platform tuning flow and the shooting reference parameter information.

In summary, in the embodiment of the present invention, the cloud debugging system may determine the target platform tuning process according to the chip platform type, and perform image quality tuning according to the target platform tuning process. By adopting the method, the user does not need to learn a knowledge system for image quality adjustment of any chip platform, and can obtain a corresponding image quality adjustment result by photographing and uploading the feedback information in the cloud adjustment system, so that the problem of high learning cost caused by chip platform differentiation can be greatly reduced, and the image quality adjustment efficiency is improved.

Referring to fig. 3, a flowchart illustrating steps of another image quality tuning method provided in the embodiment of the present invention is shown, and is applied to a cloud debugging system, where the method specifically includes the following steps:

step 301, determining the type of the chip platform currently used for image quality tuning, and determining the image effect parameter information required to be achieved after image tuning.

In the embodiment of the invention, the image quality optimization aiming at different chip platform types can be carried out in the cloud debugging system, the chip platform type used for carrying out the image quality optimization at present can be determined in response to the user operation, and the image effect parameter information required to be achieved after the image adjustment is determined.

Step 302, selecting a target platform tuning process corresponding to the chip platform type from preset platform tuning processes, and determining the shooting reference parameter information corresponding to the chip platform type.

In the embodiment of the invention, the cloud debugging system is pre-stored with the corresponding relation between the chip platform type and the platform tuning flow, and the cloud debugging system is also pre-stored with the corresponding relation between the chip platform type and the corresponding shooting reference parameter information. The target platform tuning procedure corresponding to the selected chip platform type can be selected from the preset platform tuning procedures, and shooting reference parameter information corresponding to the chip platform type is determined.

In an alternative embodiment, the preset platform tuning flow may be generated as follows:

determining necessary parameter information which needs to be input for image tuning; generating an initial platform tuning flow aiming at the chip platform based on the necessary parameter information; and acquiring image tuning project data corresponding to the chip platform, and adjusting the initial platform tuning flow according to the image tuning project data to generate the corresponding preset platform tuning flow.

The necessary parameter information is determined according to an operation instruction document of the chip platform; the necessary parameter information includes at least one of an execution order of each tuning module, a kind of the image test card, and a shooting notice.

In the embodiment of the invention, the corresponding initial platform tuning flow for tuning the image can be determined based on the operation instruction documents which are provided by different chip platforms and are related to the tuning of the image, wherein the initial platform tuning flow comprises necessary parameter information which is required to be input for tuning the image. In one example, the necessary parameter information may include an execution order of the respective tuning modules, a kind of the image test card, a shooting notice, and the like.

The execution sequence of each Tuning module refers to the calling sequence of the Tuning module supporting the ISP by any platform, different Tuning modules are interdependent, image quality Tuning is performed according to a certain sequence, otherwise, Tuning results are diverged, and good Tuning results cannot be obtained. For example, a general Tuning sequence is: automatic exposure- > black level- > Shading- > rough denoising- > Gamma adjustment- > color adjustment- > sharpening adjustment- > other special module adjustment.

The image test card type is a shooting card type, and typical cards for Tuning include: gray board, 24 color card, ISO12233 chart, transmission 20 th order, 36 th order chart, etc.

The shooting notice may be imaging matters that need to be set when shooting a certain image card for a certain module in the shooting sequence during shooting, such as an exposure value, an EV value, an aperture value, and the like; the requirements of shooting a picture card, such as occupied area of a picture, shooting inclination and the like; in addition, the color temperature, the light intensity and the like of a shooting light source which needs to be noticed can be also adopted; finally, a captured image format, a saving method, and the like may also be possible.

In the embodiment of the invention, the initial platform tuning flow of the corresponding chip platform can be adjusted by accumulating the project data of the image tuning projects of the same or different types of chip platforms.

Adjusting the initial platform tuning flow according to the image tuning project data, comprising:

taking the item as a unit from the image tuning item data, taking the shooting chart and the shooting condition adopted by the newly added module in the platform tuning flow corresponding to each item as statistical objects, and calculating the distribution probability of each statistical object in all items respectively; and if the distribution probability is greater than the preset confidence probability, determining the statistical object corresponding to the distribution probability, and updating the adjustment object corresponding to the statistical object in the initial platform adjustment process. If the distribution probability is not greater than the preset confidence probability, determining a reference item matched with the current item from a plurality of items of the image tuning item data; and adjusting the initial platform tuning flow according to the platform tuning flow of the reference item.

In the embodiment of the present invention, the initial platform tuning process is increased, and the adjusting content may include: and the newly added module is used for shooting the picture card and shooting conditions. Wherein, the newly added module is a module additionally added in the image tuning process; shooting a graphic card, and meeting the requirements of the product/platform for image tuning, such as the requirement on the color temperature of the shooting graphic card; the photographing condition may be a lighting condition, a recommended exposure table, or the like.

And counting newly added modules, shooting image cards and shooting conditions in the platform tuning process of each item from the accumulated image tuning item data, and calculating the distribution probability of the statistical object in all the items respectively, wherein if the distribution probability is greater than the confidence probability (such as 80%), the adjusting object corresponding to the statistical object can be updated in the initial platform tuning process corresponding to the chip platform. For example, the 20-order transmission grayscale graphic card under the color temperature conditions of 1000lux and 5000k cannot be shot in the initial platform tuning process, but in the actually implemented data statistics, 80% of the items adopt the shooting method, and then the corresponding shooting graphic card scene can be suggested to be added in the current initial platform tuning process. In practical applications, even though the chip platforms are the same, the product directions of the applications are different, for example, the indoor monitoring camera does not need to consider the outdoor special weather condition, but the outdoor monitoring camera with the same model and the same platform is relatively complicated. In addition, the combination of CCD + lens may also cause differences. The difference between the actual flow and the implementation flow just indicates that the provided default flow may be incomplete, so the initial platform tuning flow can be automatically strengthened by using the image tuning project data.

For the case that the distribution probability is not greater than the confidence probability, a reference item matched with the current item can be determined from a plurality of items of the image tuning item data, and the key index difference between the two items is compared, a specific comparison method can adopt an Euclidean distance (less than a 3-dimensional index item) or a correlation coefficient (a high-dimensional index item, which is often a quality curve), if the difference is small, for example, the distance is less than 0.0001 or the absolute value of the correlation coefficient is more than 0.8, the initial platform tuning flow can be adjusted according to the platform tuning flow of the reference item, for example, a corresponding tuning module is added; otherwise, the association degree is considered to be low, and the tuning flow of the initial platform does not need to be adjusted, for example, the tuning module does not need to be added.

With respect to step 302, the following steps may be performed:

and a sub-step S11 of determining a digital imaging combination type of the imaging device.

And a substep S12 of determining whether the number of image tuning items corresponding to the digital imaging combination type is greater than a preset number threshold, and determining the photographing reference parameter information according to the determination result.

In the embodiment of the present invention, the shooting reference parameter information corresponding to the type of the chip platform is determined, a digital imaging combination type of the imaging device may be determined first, and the shooting reference parameter information may be determined according to the digital imaging combination type. Wherein the photographing reference parameter information may include an exposure table. The digital imaging combination type is composed of a chip platform type, an image signal processing unit type (ISP), an imaging unit type (Sensor) and a lens type.

The cloud debugging system stores image tuning items processed in advance, can determine the number of the image tuning items with the same digital imaging combination type as the current image tuning items, judges whether the number is larger than a preset number threshold value or not, and determines shooting reference parameter information of the current image tuning items according to a judgment result.

For sub-step S12, the following steps may be performed:

if the number is larger than the preset number threshold, acquiring a corresponding coding and decoding network model; inputting the image effect parameter information into the coding and decoding network model, and outputting the corresponding shooting reference parameter information; and if the number is not greater than the preset number threshold, adopting shooting parameter information in a development code packet corresponding to the digital imaging combination type as the shooting reference parameter information.

The coding and decoding network model is obtained by taking historical shooting parameter information of the image tuning item as output and taking historical image effect parameter information of the image tuning item as input training.

In the embodiment of the present invention, for the combination of the chip platform and the Sensor (digital imaging combination): the chip platform, the ISP processor, the Sensor (imaging unit) and the lens can inquire whether shooting reference parameter information corresponding to the digital imaging combination exists in a database of the cloud debugging system.

If the number of the items exists, the number of the currently accumulated items with the same digital imaging combination type can be determined, when the number of the items is large enough (for example, more than 30 items), the historical shooting parameter information finally tuned in the accumulated items is used as output, the historical image effect parameter information is used as input, a coding and decoding network model can be obtained through training by constructing a coding and decoding network, then the image effect parameter information of the current item is used as input, and shooting reference parameter information can be obtained by using the coding and decoding model. When the number of accumulated items is insufficient, it can be considered that the current data accumulation is insufficient to provide a quantization condition, and the shooting reference parameter in the development code package is used.

If the type of the development code package does not exist, the code under the chip platform type in the cloud debugging system is considered to be incomplete, and a notification can be sent to a user to upload the development code package of the type (generally provided by a platform developer).

And compiling the updated shooting reference parameter information through a cloud compiling module in the cloud debugging system to generate a compiling result for shooting. It should be noted that the cloud compiling module has a plurality of compilers, the compilers correspond to chip platform types, and the compilers and the camera development code packages can be uploaded to the cloud debugging system together in advance.

Step 303, determining the imaging device currently used for adjusting the image quality, and determining a comparison imaging device of the imaging device.

In the embodiment of the invention, the imaging equipment currently used for image quality optimization can be determined, and screening of the comparison imaging equipment is carried out based on web crawlers and data mining.

With respect to step 303, the following steps may be performed:

and a substep S21, if there is an image tuning item in the cloud debugging system that is the same as the digital imaging combination type of the imaging device, taking the imaging device of the image tuning item as a candidate comparison imaging device.

And a substep S22, if the cloud debugging system does not have the image tuning item with the same type as the digital imaging combination type of the imaging device, comparing the resolution of the imaging unit used by the current image tuning item with the resolution of the imaging unit used by the previous image tuning item, and taking the imaging device of the image tuning item with the least resolution difference as the candidate comparison imaging device.

And a substep S23 of determining the image quality evaluation result of the candidate comparison imaging device.

And a substep S24, sorting the evaluation results, calculating the rank of the image quality loss of each candidate comparison imaging device, taking the candidate comparison imaging device with the last rank as the optimal comparison imaging device, and taking the candidate comparison imaging device with the middle rank as the reference comparison imaging device.

When the cloud debugging system stores the Tuning project of the same digital imaging combination, the imaging device of the project can be used as a candidate comparison imaging device. Furthermore, in an alternative embodiment, it may be determined that all of the Tuning projects having the same digital imaging combination have: and (3) sequencing the final evaluation results of the imaging devices and the image quality evaluation results of the related comparison imaging devices according to the evaluation results of geometric distortion rate, noise, detail loss, color cast and the like, then calculating the comprehensive ranking of the imaging quality loss of each imaging device, wherein the calculation method is shown as formula 1, finally selecting the imaging device with the last comprehensive ranking as a golden comparison imaging device, comparing the imaging devices with the ranking names in the middle as a reference, and finishing the screening work of the comparison imaging devices.

Wherein score represents the composite score; α represents a geometric distortion rate; σ represents noise; η represents loss of detail; delta C_meanIndicating an average color shift.

When the cloud debugging system does not have the same Tuning project of the digital imaging combination, the project closest to the Sensor resolution or the frame rate can be selected from all the projects, specifically, the difference between the resolution and the frame rate is calculated, and the imaging device involved in the project with the difference value of 0 is taken as a candidate comparison imaging device. Searching image quality evaluation results of candidate comparison imaging devices, sorting the evaluation results according to geometric distortion rate, noise, detail loss, average color cast and the like, then calculating the comprehensive ranking of the imaging quality loss of each imaging device, wherein the calculation method is shown as formula 1, finally selecting the imaging device with the last comprehensive ranking as the golden comparison imaging device to form the shooting device with the highest imaging quality, and taking the ranking with the middle ranking as the reference comparison imaging device to form the shooting device with the average imaging quality, thereby completing the screening work of the comparison imaging devices.

In another optional embodiment, when the storage of the Tuning item of the digital imaging combination in the cloud debugging system is empty, an imaging terminal device consistent with the resolution of the imaging device to be tuned, including but not limited to a mobile phone, an IOT dedicated device, a customized product, and the like, may be searched through a web crawler, and captured device information is analyzed by matching the sensor information of the current imaging device with a keyword in the captured device information, and a second-order norm is calculated for attribute values having the same keyword, where the keyword is the resolution, and the resolution of the terminal device is h₁×w₁Resolution of the current imaging apparatus to be tuned is h₂×w₂Then the second order norm with the keyword being the resolution is

Neglecting non-numerical attribute calculation, then calculating second-order norm values and average values of all numerical types and keyword matching, performing mean ascending sorting on the captured terminal equipment and imaging equipment to be subjected to Tuning based on the average values, and selecting terminal equipment with T (T is recommended to be more than 4) before the average value ranking and complete attribute information (price, manufacturer, size, specification and the like) as candidate comparison imaging equipment; based on the detailed information of the T-money terminal equipment, sorting from small to large according to the mining information quantity, selecting the terminal equipment with the information quantity ranked as the middle position as a reference comparison imaging equipment, and using the terminal equipment with the most information quantity as a golden comparison imaging equipment.

And 304, acquiring a shot image obtained by shooting the image according to the shooting flow and the shooting reference parameter information in the target platform tuning flow by using the comparison imaging equipment.

In the embodiment of the invention, after the comparison imaging device is determined, the comparison imaging device can be adopted to shoot images according to the shooting flow and the shooting reference parameter information in the target platform tuning flow.

The user can shoot images according to the provided comparison imaging device and the shooting flow. In an example, the model of the imaging device may be compared in a cloud debugging system, a selection suggestion of the imaging device may be provided for a user, then a shooting process is provided in an auxiliary manner, a worker needs to perform shooting according to the shooting process in a matching manner, and data naming is performed on a corresponding shot image and related data according to shooting process requirements. The cloud debugging system acquires a shot image uploaded by a user.

And 305, performing image quality adjustment on the shot image to obtain corresponding image quality optimization parameter information.

The image quality optimization parameter information can enable the shot image subjected to the image optimization processing to achieve the image effect corresponding to the image effect parameter information.

In the embodiment of the invention, the image quality of the obtained shot image is adjusted and optimized to obtain the image quality optimization parameter information meeting the image effect requirement corresponding to the image effect parameter information.

With respect to step 305, the following steps may be performed:

and a substep S31 of performing image quality evaluation based on automatic segmentation on the captured image.

After the shot image is identified, image quality evaluation can be carried out.

In an optional embodiment, the target platform tuning process has a constraint condition on the uploaded shot image, and needs to first determine whether the uploaded shot image belongs to abnormal data, and needs to perform corresponding interactive correction if the uploaded shot image belongs to the abnormal data.

In one example, the determination may be made as to the naming format of the captured image. The method can check the shot images uploaded according to the shooting process in a one-to-one correspondence mode according to the data naming rules, determine whether shooting image names inconsistent with the shooting process requirements exist, and if the shooting image names exist, the shot images are considered to be irregular, and irregular data quantity F can be recorded. In the case where there is no irregular captured image, F is 0.

Further, the content of the captured image may also be determined. Before the image quality evaluation based on automatic segmentation is performed on the shot image, the method further comprises the following steps:

judging whether shooting irregularity exists in the process of acquiring the shot image; if yes, executing the operation of feeding back the shooting problem; and if the image quality evaluation operation does not exist, performing image quality evaluation operation based on automatic segmentation on the shot image.

The judging whether shooting non-specifications exist in the process of acquiring the shot image comprises the following steps:

carrying out graying processing on the shot image to obtain a grayscale image corresponding to the shot image; carrying out wide dynamic stretching on the gray level image to obtain an enhanced gray level image; determining an image template, and determining an image area matched with the image template in the enhanced gray-scale image based on an MMSER method; and judging whether shooting non-specifications exist in the process of acquiring the shot image or not based on the image area.

In the embodiment of the invention, the gray processing can be carried out on the shot image to obtain a gray image f corresponding to the scene of the shot image; the method for carrying out wide dynamic stretching on the gray image comprises the following specific steps: firstly, counting a normalized gray level histogram of a shot image, then calculating the integral of the statistical histogram to obtain an integral histogram, and finding a gray level value g of which the numerical value is closest to a lower gray level limit (for example, the integral value is 0.05) and an upper gray level limit (for example, 0.95)_minAnd g_maxFor y ∈ [1, h)₂]Line x ∈ [1, w ]₂]The gray level f (x, y) at the column is adjusted according to equation 2 to obtain an enhanced gray level f' (x, y).

Wherein, g_minRepresenting a lower limit at which the integral histogram value is closest to the gray scale; g_maxThe resolution of the imaging device currently used for image quality optimization is h, which represents the upper limit of the integral histogram value closest to the gray scale₂×w₂，y∈[1，h₂]Representing a row; x is an element of [1, w ]₂]Representing a column; the gray level f (x, y) is within y ∈ [1, h ]₂]Line, x ∈ [1, w ]₂]Gray scale values at the columns; the enhanced gray scale is f' (x, y).

Determining an image area S matched with the template in the enhanced gray level image based on an MMSER method by taking a standard shooting image card image as an image template; suppose that the image area S occupies the area S of the entire captured image_t＝h₂×w₂The ratio is too different from the requirement in the shooting process, such as S/S_tIf the judgment result is more than 0.2, the shooting is not standardized, and F is F +1, so that the judgment is finished; otherwise, based on the matched convex hull coordinates of the image region, calculating the circumscribed rectangle of the matched image region, and assuming that the area of the circumscribed rectangle is S_qIf S/S_qIf the ratio of (a) is too small, for example, less than 0.9, it can be considered that the shooting is not standardized, and F is F +1, so that the determination is completed; otherwise the shooting specification can be considered. And judging whether the F is larger than 0, if so, feeding back to the user, and prompting which shot images have problems, and what specific problems are, such as shooting non-specification caused by substandard area ratio, and the like.

After the shot image is determined to meet the requirements, quality evaluation can be performed on the shot image. For sub-step S31, the following steps may be performed:

segmenting the shot image based on a preset segmentation model to obtain a corresponding segmentation result; correcting an image segmentation area of the shot image according to the segmentation result; for the same group of shot images, calculating the shot images in the same area based on objective parameters, and comparing the calculation results with the objective parameters of the shot images shot by the corresponding optimal comparison imaging equipment one by one; and evaluating the image quality according to the comparison result.

The preset segmentation model is obtained by adopting an objective image card scene, a subjective image scene and a corresponding label image to establish an image group and performing segmentation training on the image group.

In the embodiment of the present invention, a segmentation model may be established, and the captured image may be segmented based on the segmentation model, and the segmentation result may be corrected. For the same group of shot images, objective parameters of the shot images in the same area can be calculated, and the objective parameters are used for comparing with the objective parameters of the shot images shot by the optimal comparison imaging equipment to evaluate the image quality.

The specific mode can be as follows:

1. first, a plurality of sets of objective chart scenes and subjective scene images and corresponding labeled graphs are prepared, and as shown in fig. 4, the objective chart scenes and the subjective scene images are a subjective scene graph and corresponding labeled graphs (in fig. 4, (a) is a subjective scene graph, and (b) is a corresponding labeled graph). Typical image cards such as 24 color cards, SFRPlus image cards, dot diagrams and the like must be contained in the objective image card scene; the subjective image scene covers different color temperatures and illumination conditions as much as possible, for example, the color temperature range is 2300K-10000K, and the illumination is 51 ux-10000 lux; the size of the marking map is consistent with that of the original image, the gray level value of each pixel represents a classification label at the same position of the original image, and labels in the same area or attribute are consistent, for example, the gray level of a blue sky in a subjective scene image defined in the marking map is 1, and the gray level of a road scene defined in the marking map is 2; the image group composed of the original image and the corresponding label image is enough, such as more than 10000 groups, and the difference between scenes is as large as possible, such as picture card, color temperature and lighting condition can not be completely consistent.

2. The image group is trained based on a pixel2pixel segmentation method, considering that a high-resolution image is difficult to train, and the current deep learning segmentation method generally processes the high-resolution image by using a down-sampling method, so that the precision is insufficient, therefore, for input image group data, while down-sampling is performed, a plurality of groups (for example, more than 10 groups) of low-resolution image groups are generated for the high-resolution image group by using a random interception method, and the specific method is as follows:

(1) suppose the image width is W and the height is H, at [1, W]And [1, H]Randomly choosing the starting point as the starting point position (x) of the upper left corner of the randomly intercepted image according to a uniformly distributed mode within the integer range of (2)_s，y_s)；

(2) Suppose that the training network input layer width is w_inHeight of h_inIn a

Or

Randomly selecting a cutting proportion omega of the width or the height according to a uniform distribution mode, calculating according to the width, and then calculating the cutting width

And corresponding height

(3) Judging whether the cutting width and the cutting height exceed the boundary of the image or not on the basis of fixed starting point, namely whether x is satisfied or not_s+w_c-1. gtoreq.W, or y_s+h_cH is more than or equal to-1; if so, the adjustment mode is x_s＝W-w_c+1, or y_s＝H-h_c+1；

(4) And (4) repeating the steps (1) to (3) until the requirement of the number of enough groups is met, such as 10 groups.

3. Training the generated image group to obtain a corresponding segmentation model; generally, in the normal training mode, the loss of training is gradually reduced;

4. under the condition of providing comparison imaging equipment images, segmenting the shot images based on a trained segmentation model, and simultaneously comparing the results of partial segmentation with the results of overall segmentation, and under the condition of repeated positions but containing various classification results, modifying the image area segmentation of the original image by using a voting method;

5. for any group of images, calculating multi-objective parameters of the images in the same area, wherein the objective parameters comprise: frequency domain information entropy, smoothness, noise level, distribution in Lab space and the like;

6. and calculating respective two norms of objective parameters of the image shot by the imaging equipment according to the counted result, namely comparing the objective parameters with corresponding golden one by one to serve as quality differences of all items, and then calculating the average value of the quality differences of all the items, wherein the smaller the value is, the closer the objective parameter value representing the imaging equipment which is currently subjected to image quality optimization is to the golden comparison imaging equipment, and otherwise, the larger the quality difference is.

And a substep S32, after the evaluation is passed, performing image optimization processing based on a preset degradation model on the shot image to obtain corresponding image quality optimization parameter information.

In the embodiment of the invention, after the evaluation is passed, the image quality optimization processing based on the preset degradation model can be carried out on the shot image.

For sub-step S32, the following steps may be performed:

performing color optimization processing on the shot image; carrying out wiener filtering optimization processing on the shot image; and carrying out noise optimization processing on the shot image.

For performing the color optimization process, the method may include:

for the captured image, the light signal L (x, y), x ∈ [1, H ], y ∈ [1, W ], at the coordinate position (x, y), the image generated by the imaging device after capturing the signal is f (x, y, c), c ∈ {1, 2, 3} (the general image is divided into R, G, B three channels, and 1, 2, 3 is adopted for representation). The whole imaging model can refer to equation 3:

wherein g (x, y, c) represents the gray value before color adjustment, K (x, y, c) is 0, 1 represents the photoelectric conversion coefficient table instead of gamma or lut transformation, psf is a low-pass filter and represents the detail loss function, T (·) represents the distortion function, n (x, y, c) represents random noise, b represents fixed bias, and M represents the color conversion matrix.

The method comprises the steps of firstly solving T (·), specifically providing a result of a geometric distortion evaluation chart obtained by comparing imaging equipment with current imaging equipment by golden according to shooting requirements, calculating an optical distortion equation of a standard chart, and removing geometric distortion according to parameters of the equation;

then solving a photoelectric conversion coefficient table, wherein the specific mode is that different scene images shot by the imaging equipment and the current imaging equipment are compared through golden, histogram matching of different channels is carried out, an adjusting value corresponding to each gray scale is obtained, namely under the condition that any gray scale is provided, the adjusting value can be found through a histogram matching result, then the gray scale T [ L (x, y) ] is input, and a matching transformation coefficient matched with the gray scale T [ L (x, y, c) ] is found to be used as K (x, y, c);

solving a color conversion matrix, wherein the specific mode is that the transformation matrix solving is carried out on the scene images shot by the imaging device and the current imaging device through golden comparison:

(1) arranging all image gray scales of the same scene image shot by the two imaging devices into a 3-row H multiplied by W matrix according to the column direction, wherein the gray ratio of the color matrix of the imaging devices is C_rThe color matrix of the present image forming apparatus is C_tConstructing an objective function as shown in formula 4:

wherein, R (-) represents the correlation function, λ ∈ 0.5, and 1 represents the constraint value.

(2) Respectively to | | MC_t-C_rTransformation element m of | |_i(i∈[1，9]) Calculating partial derivatives, wherein each partial derivative result is 0 to obtain a 9-element linear equation set, and solving can be completed to obtain a conversion matrix;

(3) for the solved matrix, | R (MC) is calculated_t，C_t) L, suppose R (MC)_t，C_t) If the value is more than or equal to lambda, the conversion matrix is considered to be solved; otherwise slackenedEquation 4| | MC_t-C_rI is MC_t-C_r||≥Δr，Δr＝||MC_r-C_rL; Δ r refers to the transformation variance constraint;

(4) and (3) repeating the steps (2) and (3) until the solution of the transformation matrix is completed or a certain number of iterations is reached, stopping for example when more than 100 iterations are performed, and taking the last transformation matrix as a final result.

For performing wiener filter optimization, a blind loss function can be used, including:

by using a blind loss function estimation method, the transfer function psf of the system of the current imaging device with different scales (namely different image card distances and special requirement scenes) is estimated, and then, the shot image can be sharpened by using wiener filtering.

For performing the noise optimization process, the method may include:

based on a special shooting chart and wavelet transformation principle, the noise level of the current imaging equipment and the noise level of the image shot by the imaging equipment are obtained by comparing the current imaging equipment and the golden under different scenes, and the noise levels are assumed to be sigma and sigma respectively_rThen, by constructing gaussian low-pass filters of different scales (generally using original size, 1/4 size and 1/16 size maps), for a certain scale:

(1) based on a specified noise level interval σ, with [0.5 σ, 1.5 σ ]]For example, the onset noise σ_s0.5 σ, with a fixed step t (e.g., t 0.05 σ);

(2) constructing a filter window size of w according to equation 5_lpThe Gaussian low-pass filter ensures that the window is odd;

wherein the filter window size is w_lpThe initial noise σ_s0.5 sigma, value b₁Is a number of 0 or 1, and,

representing rounding to 0 and mod (·) representing complementation.

(3) Filtering and denoising a current shot image of the imaging equipment based on the provided filter to obtain a denoised image f_nEstimating f based on wavelet transform_nNoise level σ of_nSuppose | σ_n-σ_rI is larger, such as: i sigma_n-σ_rIf the value is greater than 0.1, the filter is considered to be not suitable, and the current filtering difference value sigma is recorded at the same time_n-σ_rI and sigma_sPerforming step (4); otherwise, executing the step (5);

(4)σ_s＝σ_s+ t, judging σ_sIf the difference value is larger than the upper limit of the interval, such as 1.5 sigma, if so, selecting the | sigma with the minimum difference value in all records_n-σ_rSigma corresponding to |_sIf the value is the optimal value, if not, performing the step (2);

(5) considering that the image effect requirement corresponding to the image effect parameter information is currently achieved, selecting the current filtering parameter sigma_sCorresponding to the size w of the filter window_lpThe current best denoising filter parameter is obtained.

And step 306, performing parameter compiling on the image quality optimization parameter information based on the chip platform type.

In the embodiment of the invention, after the image optimization processing is carried out on the shot image, the image quality optimization parameter information meeting the image effect requirement corresponding to the image effect parameter information can be obtained. The image quality optimization parameter information may include the color conversion matrix, the system transfer function, the optimal denoising filter parameter, and the like. After the image quality optimization parameter information is determined, the image quality optimization parameter information can be compiled based on the chip platform type.

With respect to step 306, the following steps may be performed:

and a substep S41, performing parameter compilation on the image quality optimization parameter information in a column vector mode based on the chip platform type.

In the embodiment of the invention, the image quality optimization parameter information can be compiled by adopting a column vector and a self-coding model. Specifically, the following manner may be adopted:

1. for the image quality optimization parameter information of any chip platform, based on the shot image data shot according to the requirements and the original compiling parameters (such as the parameters after typical configuration parameter adjustment or default values, and curve adjustment or default parameters) accumulated during image shooting, the image quality optimization parameter information of different platforms is arranged into column vectors, such as N in total related parameters of the module 1₁The parameters are arranged according to the column direction, and then the related parameters of the module 2 are N₂Arranging the parameters of all modules in module 1, … …, and storing the parameters of each chip platform in a development code package; each item of each platform can generate a column vector related to the item, and assuming that the column vector length of any platform is D and P items exist under the same platform, the column vectors of all platforms can form a matrix y with D rows and P columns_outThe vector sizes of different platforms are different;

2. based on the method in the degradation model, T (-) in the same platform and corresponding items is searched, a histogram matching transformation curve, namely the corresponding reference value of K (x, y, c), a color conversion matrix M, a system transfer function psf and a filter window size w are searched_lpThe gaussian low-pass filter lp, all data obtained by using the degradation model are arranged according to a column vector mode, which may specifically be arranged in the following order: t (·) distortion parameters (at most 8) account for 9 rows, complement 0 is insufficient, then gamma/lut curves account for 3 × 256 length, the color channels are arranged according to red, green and blue, the color conversion matrix accounts for 9 rows, the data volume is consistent with that of the conversion matrix, system transfer functions account for 121 rows according to the column direction, complement 0 with insufficient transfer function size is taken, filter parameters of different scales (taking 3 scales as an example) are taken, and filters of each scale account for 121 rows and complement 0 with insufficient; thus, the Tuning reference vector data of the platform and the project is obtained, and the vector length is 1270 dimension;

3. sorting vector data in all projects on the same platform according to rows to form input x_in(generally, the number of inputs per platform is as high as possible, e.g., greater than 10 sets, and when the number is small, it is considered to provide multiple sets of Tunin at the demand sideg parameters and scene results), assuming a dimension of the contract degradation model as D_e(e.g., 1270 dimension) and there are P items under the same platform, then x_inIs D in the dimension_eRow P and column;

4. with x_inTo input, y_outFor output, a self-coding model is constructed, self-coding training is carried out, and a self-coding conversion network of a corresponding platform is obtained; it can be considered that the parameters of platform differentiation can be eliminated by a uniform degradation model and adapted by a self-coding network;

5. based on the image quality optimization parameter information, arranging the image quality optimization parameter information into the input x of the current platform according to the mode of the step 3_sampleAnd carrying out output decoding based on the coding training model of the platform to obtain a decoding parameter y_sampleI.e. the original compilation parameters of the platform.

6. Based on y_sampleModifying the compiling parameters of the corresponding platform and calling compilers of different platform + Sensor combinations (digital imaging combinations) to complete parameter compiling in the arrangement mode of the step 1;

7. the user subjectively judges whether fine adjustment is needed or not through actually shooting the image: if fine adjustment is needed, replacing the relevant shooting scene image of the imaging equipment by a result image which is subjectively approved through adjustment and comparison until subjective approval is achieved; and if fine adjustment is not needed, the image quality adjustment of the cloud end is finished.

In summary, in the embodiment of the present invention, the cloud debugging system may determine the target platform tuning process according to the chip platform type, and perform image quality tuning according to the target platform tuning process. By adopting the method, the user does not need to learn a knowledge system for image quality adjustment of any chip platform, and can obtain a corresponding image quality adjustment result by photographing and uploading the feedback information in the cloud adjustment system, so that the problem of high learning cost caused by chip platform differentiation can be greatly reduced, and the image quality adjustment efficiency is improved. In the process of image quality Tuning of the cloud debugging system, a user does not need to learn any knowledge system of the chip platform Tuning, only needs to shoot and upload according to the requirements of the platform Tuning flow, can obtain Tuning results, and then finishes fine Tuning of the results by changing subjective tendency. According to the scheme, the evaluation, Tuning and shooting integrated system is constructed, the learning cost caused by platform differentiation can be eliminated by means of a unified degradation model, and the Tuning efficiency is improved. This scheme is through the mode of quantization accumulation Tuning project data, based on excavate and the reptile technique, can provide the various statistical data that chip platform side can't provide, and it is more reasonable to let the Tuning process, makes the Tuning process can rely on by law. Because the AI mode is adopted mostly to the chip platform, after accumulating certain quantity of project data, need not rely on platform Tuning instrument, can accomplish and shoot inspection, Tuning and evaluation, greatly improve Tuning's stability and referential.

It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.

Referring to fig. 5, a block diagram of a structure of an image quality tuning apparatus provided in an embodiment of the present invention is shown, and is applied to a cloud debugging system, and specifically includes the following modules:

a first determining module 501, configured to determine a type of a chip platform currently used for performing image quality tuning, and determine image effect parameter information required to be achieved after image tuning;

a second determining module 502, configured to select a target platform tuning procedure corresponding to the chip platform type from preset platform tuning procedures, and determine the shooting reference parameter information corresponding to the chip platform type;

and an adjusting and optimizing module 503, configured to perform image quality adjustment and optimization according to the target platform adjusting and optimizing process and the shooting reference parameter information based on the image effect parameter information.

In an embodiment of the present invention, the tuning module includes:

In an embodiment of the present invention, the tuning and optimizing module includes:

In this embodiment of the present invention, the preset platform tuning flow is generated as follows, and the apparatus further includes:

In an embodiment of the present invention, the adjusting module includes:

In an embodiment of the present invention, the adjusting module further includes:

In an embodiment of the present invention, the second determining module includes:

In an embodiment of the present invention, the determining sub-module includes:

In an embodiment of the present invention, the first determining sub-module includes:

In this embodiment of the present invention, the tuning and optimizing module further includes:

In an embodiment of the present invention, the determining unit includes:

In an embodiment of the present invention, the evaluation unit includes:

In an embodiment of the present invention, the tuning unit includes:

In an embodiment of the present invention, the compiling submodule includes:

For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.

An embodiment of the present invention further provides an electronic device, including: the image quality tuning method comprises a processor, a memory and a computer program which is stored in the memory and can run on the processor, wherein when the computer program is executed by the processor, each process of the image quality tuning method embodiment is realized, the same technical effect can be achieved, and the details are not repeated here to avoid repetition.

The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements each process of the above-mentioned embodiment of the image quality tuning method, and can achieve the same technical effect, and is not described here again to avoid repetition.

The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.

Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.

The foregoing has described in detail an image quality tuning method, an image quality tuning device, an electronic device, and a computer-readable storage medium, which are provided by the present invention, and the present invention has been described in detail by applying specific examples to explain the principles and embodiments of the present invention, where the descriptions of the above examples are only used to help understand the method and the core ideas of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims

1. An image quality tuning method is applied to a cloud debugging system, and the method comprises the following steps:

2. The method according to claim 1, wherein the performing image quality tuning according to the target platform tuning procedure and the shooting reference parameter information based on the image effect parameter information comprises:

3. The method according to claim 2, wherein the performing image quality optimization on the captured image to obtain corresponding image quality optimization parameter information comprises:

4. The method of claim 1, wherein the preset platform tuning procedure is generated as follows:

5. The method of claim 4, wherein said adjusting said initial platform tuning procedure according to said image tuning project data comprises:

6. The method of claim 5, wherein said adjusting said initial platform tuning procedure according to said image tuning project data further comprises:

7. The method according to claim 1, wherein the determining the photographing reference parameter information corresponding to the chip platform type comprises:

8. The method according to claim 7, wherein the determining the photographing reference parameter information according to the determination result includes:

9. The method of claim 2, wherein determining the alignment imaging device of the imaging device comprises:

10. The method according to claim 3, wherein before the image quality evaluation based on automatic segmentation of the captured image, the method further comprises:

if yes, executing the operation of feeding back the shooting problem;

11. The method of claim 10, wherein the determining whether a photographic irregularity exists during the capturing of the photographic image comprises:

12. The method according to claim 3, wherein the automatically segmenting based image quality evaluation of the captured image comprises:

and evaluating the image quality according to the comparison result.

13. The method according to claim 3, wherein the image optimization processing based on a preset degradation model for the shot image comprises:

performing color optimization processing on the shot image;

carrying out wiener filtering optimization processing on the shot image;

and carrying out noise optimization processing on the shot image.

14. The method of claim 2, wherein the parameter compiling the image quality optimization parameter information based on the chip platform type comprises:

15. An image quality tuning apparatus, applied to a cloud debugging system, the apparatus comprising:

16. An electronic device, comprising: processor, memory and computer program stored on the memory and capable of running on the processor, which computer program, when executed by the processor, carries out the steps of a method of image quality tuning as claimed in any one of claims 1 to 14.

17. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of a method for image quality tuning as claimed in any one of claims 1 to 14.