CN116363100A

CN116363100A - Image quality evaluation method, device, equipment and storage medium

Info

Publication number: CN116363100A
Application number: CN202310339951.0A
Authority: CN
Inventors: 荆帅
Original assignee: Neusoft Reach Automotive Technology Shanghai Co Ltd
Current assignee: Neusoft Reach Automotive Technology Shanghai Co Ltd
Priority date: 2023-03-31
Filing date: 2023-03-31
Publication date: 2023-06-30

Abstract

The application provides an image quality evaluation method, an image quality evaluation device and a storage medium, wherein the image quality evaluation method comprises the following steps: obtaining at least one image to be evaluated, determining a first evaluation index of the image to be evaluated according to the image characteristics of the image to be evaluated, taking the image to be evaluated with the first evaluation index being greater than or equal to a first preset threshold value as a target image, then processing the target image to obtain processing results of different target objects in the target image, wherein the processing results at least comprise area of areas corresponding to the target objects which cannot be identified, and finally calculating a second evaluation index of the target image according to the area of the areas, wherein the second evaluation index is used for representing the quality evaluation result of the target image. To improve the accuracy of the image quality assessment.

Description

Image quality evaluation method, device, equipment and storage medium

Technical Field

The embodiment of the application relates to the technical field of image processing, in particular to an image quality evaluation method, an image quality evaluation device, image quality evaluation equipment and a storage medium.

Background

The image quality evaluation method can be classified into full reference image quality evaluation, half reference image quality image evaluation, and No reference image quality evaluation (No-Reference Image Quality Assessment, NR-IQA). In an actual business scenario, an undistorted reference image cannot be generally obtained, so no reference image quality evaluation is of interest. But because there is no image available for reference and the image content is quite rich, there is a high challenge in no reference image quality evaluation.

The image is used as one of visual perception information, and the image quality determines the implementation strategy of downstream perception fusion. For example, in an autopilot business scenario, for complex and changeable road environments and weather conditions, image quality is often poor due to dirt, fog, reflection, backlight or abnormal internal communication of a vehicle-mounted camera, so that visual observation information of autopilot is unclear, for example, the image is blurred and distorted, so that useful information is difficult to extract from the visual observation information, the image quality cannot be accurately evaluated, and then fusion results of subsequent visual observation information are affected.

Disclosure of Invention

The application provides an image quality evaluation method, an image quality evaluation device and a storage medium, so as to improve the accuracy of image quality evaluation.

In a first aspect, there is provided an image quality evaluation method including: acquiring at least one image to be evaluated; determining a first evaluation index of the image to be evaluated according to the image characteristics of the image to be evaluated, and taking the image to be evaluated, of which the first evaluation index is greater than or equal to a first preset threshold value, as a target image; processing the target image to obtain processing results of different target objects in the target image, wherein the processing results at least comprise area corresponding to the target objects which cannot be identified; and calculating a second evaluation index of the target image according to the area, wherein the second evaluation index is used for representing the quality evaluation result of the target image.

In a second aspect, there is provided an image quality evaluation apparatus comprising: the device comprises an acquisition unit, a first determination unit, a processing unit and a calculation unit, wherein the acquisition unit is used for acquiring at least one image to be evaluated; the first determining unit is used for determining a first evaluation index of the image to be evaluated according to the image characteristics of the image to be evaluated, and taking the image to be evaluated, of which the first evaluation index is greater than or equal to a first preset threshold value, as a target image; the processing unit is used for processing the target image to obtain processing results of different target objects in the target image, wherein the processing results at least comprise area corresponding to the target objects which cannot be identified; and the calculating unit is used for calculating a second evaluation index of the target image according to the area of the region, wherein the second evaluation index is used for representing the quality evaluation result of the target image.

In a third aspect, there is provided an electronic device comprising: a processor and a memory for storing a computer program, the processor being for invoking and running the computer program stored in the memory for performing the method as in the first aspect or in various implementations thereof.

In a fourth aspect, a computer-readable storage medium is provided for storing a computer program for causing a computer to perform the method as in the first aspect or in various implementations thereof.

In a fifth aspect, a computer program product is provided comprising computer program instructions for causing a computer to perform the method as in the first aspect or in various implementations thereof.

In a sixth aspect, a computer program is provided, the computer program causing a computer to perform the method as in the first aspect or in various implementations thereof.

According to the technical scheme, at least one image to be evaluated is obtained, then a first evaluation index of the image to be evaluated is determined according to the image characteristics of the image to be evaluated, of which the first evaluation index is larger than or equal to a first preset threshold value, is used as a target image, then the target image is processed to obtain processing results of different target objects in the target image, the processing results at least comprise area of areas corresponding to the unidentifiable target objects, and finally a second evaluation index of the target image is calculated according to the area of the areas and is used for representing quality evaluation results of the target image. In the process, the first evaluation index is used for initially screening the image to be evaluated to filter out the image to be evaluated with obvious distortion or blurring, then the second evaluation index is used for evaluating the quality of the filtered target image, and finally the fusion strategy of the target image in visual perception fusion is determined according to the evaluation result, so that the problem that the follow-up fusion result is affected due to unclear visual observation information is solved, the accuracy of image quality evaluation is improved, and the reliability of the image quality in the follow-up information fusion process is ensured.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is an application scenario diagram provided in an embodiment of the present application;

fig. 2 is a flowchart of an image quality evaluation method according to an embodiment of the present application;

FIG. 3 is a flowchart of another image quality evaluation method according to an embodiment of the present application;

fig. 4 is a schematic diagram of an image quality evaluation method according to an embodiment of the present application;

FIG. 5 is a schematic diagram of another image quality evaluation method according to an embodiment of the present application;

FIG. 6 is a schematic diagram of still another image quality evaluation method according to an embodiment of the present application;

FIG. 7 is a schematic diagram of still another image quality evaluation method according to an embodiment of the present application;

FIG. 8 is a schematic diagram of still another image quality evaluation method according to an embodiment of the present application;

fig. 9 is a schematic diagram of an image quality evaluation apparatus 900 according to an embodiment of the present application;

Fig. 10 is a schematic block diagram of an electronic device 1000 provided in an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present application based on the embodiments herein.

It should be noted that the terms "first," "second," and the like in the description and claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that embodiments of the present application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or server that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

As described above, in the prior art, for complex and changeable road environments and weather conditions, the image quality is often poor due to dirt, fog, reflection, backlight or abnormal internal communication of the vehicle-mounted camera, so that the visual observation information of the automatic driving is unclear, for example, the image is blurred and distorted, so that the useful information is difficult to extract from the visual observation information, the image quality cannot be accurately evaluated, and the fusion result of the subsequent visual observation information is affected.

In order to solve the technical problems, the invention concept of the application is as follows: the method comprises the steps of firstly carrying out preliminary screening on an image to be evaluated by using a first evaluation index to filter out an obviously distorted or blurred image to be evaluated, then carrying out evaluation on the quality of a filtered target image by using a second evaluation index, and finally determining a fusion strategy of the target image in visual perception fusion according to an evaluation result, thereby solving the problem that visual observation information is unclear and affects a subsequent fusion result, improving the accuracy of image quality evaluation, and ensuring the reliability of image quality in a subsequent information fusion process.

It should be understood that the technical solution of the present application may be applied to the following scenarios, but is not limited to:

In some implementations, fig. 1 is an application scenario diagram provided in an embodiment of the present application, where, as shown in fig. 1, an electronic device 110 and a network device 120 may be included in the application scenario. The electronic device 110 may establish a connection with the network device 120 through a wired network or a wireless network.

By way of example, the electronic device 110 may be, but is not limited to, a desktop computer, a notebook computer, a tablet computer, and the like. The network device 120 may be a terminal device or a server, but is not limited thereto. In one embodiment of the present application, the electronic device 110 may send a request message to the network device 120, where the request message may be used to request to obtain at least one image to be evaluated, and further, the electronic device 110 may receive a response message sent by the network device 120, where the response message includes at least one image to be evaluated.

Furthermore, fig. 1 illustrates one electronic device and one network device, and may actually include other numbers of electronic devices and network devices, which is not limited in this application.

In other realizations, the technical solutions of the present application may also be executed by the electronic device 110, or the technical solutions of the present application may also be executed by the network device 120, which is not limited in this application.

After the application scenario of the embodiment of the present application is introduced, the following details of the technical solution of the present application will be described:

fig. 2 is a flowchart of an image quality evaluation method according to an embodiment of the present application, which may be performed by the electronic device 110 shown in fig. 1, but is not limited thereto. As shown in fig. 2, the method may include the steps of:

s210: at least one image to be evaluated is acquired.

In this step, the image to be evaluated is an original image output by the vision sensor for quality evaluation, and the image to be evaluated may be multiple images, for example, in the automatic driving field, the automatic driving automobile acquires the information of the vehicle and the external environment by using its own sensing system, analyzes the information by the computing system, makes a decision, and controls the execution system to implement the operation of the vehicle acceleration, deceleration or steering lamp, thereby implementing self-service running without intervention of the driver. The image to be evaluated corresponds to an original image output by a visual sensor in the automatic driving automobile, and in some realizable modes, external visual observation information can be captured through the visual sensor in the automatic driving automobile, and then at least one image to be evaluated is obtained according to the visual sensing information.

The vehicle-mounted camera is used as a visual sensor in an automatic driving vehicle, the visual sensor can collect an external image as visual observation information in the driving process of the automatic driving vehicle and transmit the collected image to the electronic equipment, when the electronic equipment receives one frame of image, the frame of image is used as an image to be evaluated, the vehicle-mounted sensor can continuously collect multiple frames of images along with the movement of the automatic driving vehicle, and accordingly, the electronic equipment receives the multiple frames of images and further obtains at least one image to be evaluated.

It can be understood that the visual observation information collected by the vehicle-mounted camera can also be in a video form, and then the visual observation information in the video form is split into a plurality of frames of images by an image extraction mode, so as to obtain at least one image to be evaluated, and specifically, the extraction quantity per unit time can be set in the image extraction process, for example, 1 second for 1 sheet, 1 second for 5 sheets, 1 second for 10 sheets, and the like.

Further, since the image to be evaluated is an original image output by the vehicle-mounted camera, the image to be evaluated can be a clear undistorted image, or can be a distorted image with a local unclear region caused by bad weather or dirt and reflection of the camera.

S220: according to the image characteristics of the image to be evaluated, a first evaluation index of the image to be evaluated is determined, and the image to be evaluated, of which the first evaluation index is larger than or equal to a first preset threshold value, is taken as a target image.

In this step, the first evaluation index may be used to perform preliminary screening on the image to be evaluated, so as to filter out the image to be evaluated with obvious distortion or blurring, and the screened image to be evaluated is used as the target image for performing subsequent evaluation. According to the method, the device and the system, the images with better quality are screened out of the multiple images to be evaluated based on the first evaluation index and used for subsequent processing, so that poor images can be prevented from interfering with the fusion result, processing time and resource load of electronic equipment due to the fact that too many invalid information are processed can be avoided, and adverse effects on the fusion result due to unclear visual observation information can be reduced.

The first evaluation index may be a specific value, and correspondingly, the first preset threshold may also be a value, and may be preset, and the embodiment of the present application is not limited to the first preset threshold, for example, it may be 1.0. That is, when the first evaluation index is greater than or equal to 1.0, it is indicated that the image to be evaluated can be used as a target image for subsequent processing, whereas when the first evaluation index is less than 1.0, it is indicated that the image to be evaluated has significant blurring or distortion, and no reuse is recommended.

S230: and processing the target image to obtain processing results of different target objects in the target image, wherein the processing results at least comprise area corresponding to the target objects which cannot be identified.

It should be noted that, each target image may include a plurality of different target objects, for example, in an automatic driving scene, the target objects may be people, vehicles, backgrounds, road signs, traffic lights, and other regular objects in the automatic driving scene, and by processing the target objects in the target image, processing results of the different target objects in the target image may be identifiable target objects and corresponding area or unrecognizable target objects and corresponding area. It can be understood that the area corresponding to the target object that cannot be identified is the area corresponding to the unclear or blurred area in the target image, for example, the area of the target object that cannot be identified due to dirt, fog, light reflection, etc. of the camera.

In one possible implementation manner, the target object in the target image may be identified by the existing image processing method, then the area of the area corresponding to the identifiable target object may be calculated based on the image segmentation algorithm, so as to obtain the area of the area corresponding to the identifiable target object, since the area of the area corresponding to the identifiable target object is usually low in pixels, in order to further ensure the accuracy of the area, after the area of the area corresponding to the identifiable target object is removed from the target image, the average value of pixels may be calculated for the remaining area in the target image, and if the average value of pixels corresponding to the remaining area is smaller than the set value, the area of the area corresponding to the remaining area is regarded as the area of the area corresponding to the identifiable target object.

S240: and calculating a second evaluation index of the target image according to the area of the region corresponding to the target object which cannot be identified, wherein the second evaluation index is used for representing the quality evaluation result of the target image.

In this step, the larger the area of the region corresponding to the target object cannot be recognized, the larger the unclear region contained in the target image is, that is, the limited useful information which can be extracted in the target image is, otherwise, the fewer the unclear region contained in the target image is, the more useful information can be extracted.

According to the method, the device and the system, the first evaluation index is used for initially screening the image to be evaluated to filter out the image to be evaluated with obvious distortion or blurring, then the second evaluation index is used for evaluating the quality of the filtered target image, and finally the fusion strategy of the target image in visual perception fusion is determined according to the evaluation result, so that the problem that the follow-up fusion result is affected due to unclear visual observation information is solved, the accuracy of image quality evaluation is improved, and the reliability of the image quality in the follow-up information fusion process is guaranteed.

In order to provide an evaluation criterion of an image to be evaluated, and filter the image to be evaluated to obtain a target image, the embodiment of the application also provides another image quality evaluation method. Based on fig. 2, as shown in fig. 3, determining a first evaluation index of the image to be evaluated according to the image characteristics of the image to be evaluated in S220 includes:

S310: and extracting features of the image to be evaluated to obtain image feature information.

It will be appreciated that the image features may be any of color features, texture features, shape features and spatial relationship features of the image, and that the image should have some physical quantity that is measurable by the image features when evaluating the image quality. The image characteristic information is obtained by carrying out characteristic extraction on the image to be evaluated, and the image characteristic information forms an evaluation parameter of a first evaluation index.

For example, if the image to be evaluated is a color image, when the image feature is a color feature, the corresponding image feature information may be three chromaticity values of red, green and blue of a certain pixel point in the image.

S320: based on the image characteristic information, calculating the information entropy, normalized relative gray distance and average contrast of the image to be evaluated.

In the visual perception fusion scene, whether the subjective value of the image is dependent on the information content of the image, and the corresponding quantization index is information entropy (Information Entropy, IE), that is, the more abundant the gray level information is in the image, the better the image quality is. Based on this, the present embodiment adopts the information entropy as one of the evaluation parameters of the first evaluation index. The information entropy may be calculated by any method in the prior art, for example, if the image to be evaluated is a color image, the image to be evaluated is first converted into a gray image, and then the information entropy of the image is calculated based on the probability of the number of pixels on the corresponding gray level.

Further, the image effect is not good because the human eye subjectively considers that the gray value is concentrated near 0 or 255. The quality of the image is also related to the proper brightness (i.e. gray value). In this embodiment, an Average Gray (AG) may be used as an evaluation index of the brightness level of the image to be evaluated, for example, the number of pixels in the image line and the gray of the pixels in the image line to be evaluated may be obtained first, then the Average gray may be calculated based on any Average gray calculation method in the prior art, and similarly, the optimal cognitive gray (Optimum Perception Gray, OPG) may be calculated according to an arithmetic sequence with a tolerance of 1, and then the normalized relative gray distance (Normalized Gray Distance, NGD) may be calculated using the Average gray and the optimal cognitive gray. Wherein the normalized relative gray distance is used to describe a deviation value of the relative best perceived gray of the average gray. The greater the normalized relative gray-scale distance deviation 1, the poorer the image quality is considered.

Further, since the basis of visual perception is an image, the contrast reflects important information of the image. Based on this, the present embodiment adopts Average Contrast (AC) as another evaluation parameter of the first evaluation index. The average contrast may be calculated by any method in the prior art, and for example, the target and background gray levels may be acquired first, and then the average contrast may be calculated based on any method in the prior art.

S330: and determining a first evaluation index of the image to be evaluated according to the information entropy, the normalized relative gray distance and the average contrast of the image to be evaluated.

For an image to be evaluated, the information entropy IE of the image to be evaluated is calculated by using probabilities of pixel numbers on different gray levels, the normalized relative gray scale distance NGD of the image to be evaluated is calculated by using the brightness water and the optimal cognitive gray scale corresponding to the image, the average contrast AC of the image to be evaluated is calculated by using the target object and the background gray scale in the image, and the first evaluation index IQI (Image Quality Indicator) of the image to be evaluated is determined according to the information entropy IE, the normalized relative gray scale distance NGD and the average contrast AC of the image to be evaluated, wherein the specific expression formula is as follows:

it should be noted that, the content of the image feature information is not limited, and other parameter indexes for measuring the image quality, such as image resolution, image brightness, etc., may be added in the process of determining the first evaluation index of the image to be evaluated.

Further, the image quality can be initially measured according to a first evaluation index of the image to be evaluated, if the first evaluation index is larger than or equal to a first preset threshold value, the image quality to be evaluated is clear as a whole and can be used for subsequent processing, otherwise, if the first evaluation index is smaller than the first preset threshold value, the image to be evaluated is obviously blurred or distorted and is not suitable for the subsequent information fusion process, on one hand, poor-quality image interference information fusion results are avoided, and on the other hand, excessive invalid information processing time and resource load are avoided.

Fig. 4 is a flowchart of still another image quality evaluation method according to an embodiment of the present application. Based on fig. 2, as shown in fig. 4, S230 includes:

s410: acquiring a target image to be processed;

s420: identifying a target image through a target identification model obtained through pre-training to obtain a classification label corresponding to each target object;

s430: and counting the area of the target object corresponding to the unrecognizable label in the target image by utilizing the target segmentation processing module obtained by pre-training.

In some implementations, the target image to be processed includes a plurality of target objects, the target image to be processed can be extracted from the target image through a pre-trained target recognition model, the target image to be processed is input into the target recognition model by using a pre-set classification label, and a position area of each target object in the target image and a classification label of the target object are correspondingly output, wherein the classification label at least includes an identifiable label and an unrecognizable label. The identifiable tag can be a class tag such as a road vehicle, a red road lamp, a lane line, a road sign, a pedestrian, road side facilities, a bicycle, a tripod warning sign, a portal frame and the like; and local unclear areas caused by dirt, fog, light reflection and backlight of the camera can be uniformly set as the labels which cannot be identified.

The target image to be processed is input into a target recognition model, so that the target image to be processed comprises areas capable of recognizing the target object, such as a background, a sidewalk, a vehicle, a person, a lawn and the like, and the target image to be processed comprises areas incapable of recognizing the target object, such as a cluster fog, a reflection and the like, the areas capable of recognizing the target object in the target image are marked as the corresponding types of identifiable tags, and the areas incapable of recognizing the target object in the target image are marked as the unrecognizable tags.

In some possible implementations, the target recognition model may be trained before the target image is specifically recognized to obtain a classification label corresponding to each target object.

The method comprises the steps of obtaining image sample sets containing different target objects in advance, classifying and marking the target objects in the image sample sets to obtain classification labels, establishing mapping relations between the different target objects in the image sample sets and the classification labels, and storing the mapping relations so as to explain input and output results of a target recognition model and ensure accuracy and normalization of data. Then, according to the mapping relation, identifying the target objects in the image sample set to obtain the actual target objects and the classification labels of the actual target objects contained in the image sample set, inputting the image sample set carrying the classification labels into the target identification model to obtain the predicted target objects and the classification labels of the predicted target objects contained in the image sample set, calculating the loss values between the predicted target objects and the classification labels of the predicted target objects contained in the image sample set between the actual target objects and the classification labels of the actual target objects through a loss function, and updating each parameter of the target identification model according to the loss values after obtaining the loss values. When the loss value reaches a set value, training of the target recognition model is completed, or when the training frequency of the target recognition model reaches a set iteration frequency, training of the target recognition model is completed, and the training process of the target recognition model is not limited.

It should be noted that, the present application does not limit the form of the image sample set when the target recognition model is input, and the form of the classification label corresponding to each target object in the output image of the target recognition model.

In some implementations, the target segmentation processing module may count the area of the target object corresponding to the unrecognizable tag in the target image by using an area accumulation deduplication method, specifically, may combine the areas of the target object corresponding to the unrecognizable tag in the target image, then determine whether the combined areas have overlapping areas, if so, remove the overlapping areas in the combined areas, and then count the area of the area after deduplication, so as to obtain the area of the target object corresponding to the unrecognizable tag in the target image.

Fig. 5 is a flowchart of still another image quality evaluation method according to an embodiment of the present application. Based on fig. 2, as shown in fig. 5, S240 includes:

s510: calculating the area ratio of the area corresponding to the unrecognizable target object to the area of the target image;

s520: and calculating a second evaluation index of the target image based on the mapping relation between the area ratio and the second evaluation index.

In a specific application scenario, the area ratio of the area of the unrecognizable target object to the area of the target image may be set to be α, theoretically α∈0,1, but since the target image is obtained through primary quality screening, the image with a larger α value is already removed in the primary quality screening process, so at least α∈ 0,0.9 may be considered. In practical application, the larger α is, the smaller the probability of occurrence of the case, and the larger the probability of occurrence of the case of smaller α is.

In this embodiment, the second evaluation index of the target image may be calculated based on a mapping relationship between the area occupation ratio and the second evaluation index, and the mapping relationship may be:

CL＝(1-α) ²

wherein CL is a second evaluation index of the target image, and α is the area ratio of the area corresponding to the unidentifiable target object to the target image.

It can be understood that the second evaluation index of the target image can be understood as the confidence of the target image, the larger the area ratio α is, the lower the second evaluation index CL is, which indicates that the image quality is worse, and finally, the fusion strategy of the target image in the visual perception fusion can be determined according to the result of the second evaluation index CL.

Fig. 6 is a flowchart of still another image quality evaluation method according to an embodiment of the present application. Based on fig. 2, as shown in fig. 6, the method further includes:

S610: and determining a fusion strategy of the target image in visual perception fusion according to the second evaluation index of the target image.

It can be understood that the second evaluation index of the target image is taken as the quality evaluation result of the target image, and can reflect the extractable information amount in the target image, and the accuracy of the fusion result is determined by the information amount contained in the target image, so that the target images with different quality evaluation results are suitable for different fusion strategies. For the target image with lower quality evaluation result, the overall image quality is not very high, at the moment, the target image is unsuitable to be used as input information with higher priority in visual perception fusion or unsuitable to be used as input information in visual perception fusion, the priority of the target image in visual perception fusion can be specifically adjusted according to actual scenes, and for the target image with higher quality evaluation result, the overall image quality is relatively clear, at the moment, the target image is suitable to be used as input information in visual perception fusion, and the target image can be specifically used as input information to be subjected to multi-mode fusion with information output by other sensors.

In some implementations, if the second evaluation index of the target image is greater than the second preset threshold, the target image is used as the image to be fused in the visual perception fusion, and the second evaluation index is used as the weighting coefficient of the image to be fused to participate in the visual perception fusion. The second preset threshold may be preset, and embodiments of the present application are not limited to this second preset threshold, and for example, it may be 0.6.

The second evaluation index of the target image is larger than 0.6, the target image is taken as an image to be fused in visual perception fusion, at this time, the visual perception fusion process involves information to be fused of 3 types of modes, the weighting coefficient of the image to be fused to participate in the visual perception fusion is set to be 0.4, the weighting coefficient of the information to be fused of A type mode to participate in the visual perception fusion is set to be 0.3, the weighting coefficient of the information to be fused of B type mode to participate in the visual perception fusion is set to be 0.3, and the image to be fused and the information to be fused of other modes are further jointly participated in the visual perception fusion according to the weighting coefficient.

It should be noted that, the weighting coefficients corresponding to the to-be-fused image and the to-be-fused information of other modes in the visual perception fusion process are not limited, and the weighting coefficients can be specifically set according to actual application scenes.

In an actual application scene, n modes of information x to be fused are used _n The weighting coefficient corresponding to the information to be fused of each mode is expressed as w _n The output information after visual perception fusion at this time can be expressed as y=w ₁ *x ₁ +w ₂ *x ₂ +…+w _n *x _n 。

In some implementations, if the second evaluation index of the target image is less than or equal to the second preset threshold, the target image is taken as the image to be fused in the visual perception fusion and the second evaluation index is taken as the priority coefficient to participate in the visual perception fusion.

Specifically, the priority coefficient may be determined according to environmental factors in an actual scene, such as weather factors, road factors, and air factors, and for example, in a rainy scene, an image output by a visual sensor may be subjected to weather to generate a large number of unclear areas, other radar, millimeter wave, and other modal outputs may be set to be preferentially used as information to be fused in visual perception fusion, and in a sunny scene, a target image may be set to be preferentially used as the image to be fused in visual perception fusion.

In an actual application scenario, as shown in fig. 8, the visual perception fusion process may include the following module information: the vehicle-mounted camera outputs an image to be evaluated to the image quality evaluation module, the image quality evaluation module performs preliminary quality evaluation on the image to be evaluated, screens out available images and outputs the available images to the image target detection module, the image target detection module can identify target objects in the available images, classifies and marks the target objects and then inputs the target objects to the target segmentation processing module, and the target segmentation processing module calculates the area ratio of the unrecognizable class labels in the available images according to class labels in the available images, obtains the confidence of the available images and inputs the available images to the perception fusion module according to the confidence.

In combination with module information involved in the visual perception fusion process in fig. 7, an exemplary visual perception fusion process may be as shown in fig. 8, after the vehicle-mounted camera outputs an image to be evaluated, the image quality evaluation module transmits a new frame of image to be evaluated, calculates a first evaluation index of the image to be evaluated according to image characteristics of the image to be evaluated, judges whether the first evaluation index is less than or equal to 1.0, and if yes, re-acquires the image to be evaluated output by the vehicle-mounted camera, otherwise, takes the image to be evaluated as a target image, uses the image target detection module to segment the target image, sets a classification label for each target object obtained by segmentation, calculates an area ratio of a target object corresponding to the classification label which cannot be identified in the image to be evaluated, calculates a second evaluation index of the target image according to the area ratio, and uses the perception fusion module to fuse the target image as the image to be fused with other modal information according to the second evaluation index.

Fig. 9 is a schematic diagram of an image quality evaluation apparatus 900 according to an embodiment of the present application. As shown in fig. 9, the apparatus 900 includes:

a first acquiring unit 901, configured to acquire at least one image to be evaluated;

A determining unit 902, configured to determine a first evaluation index of an image to be evaluated according to an image feature of the image to be evaluated, and take the image to be evaluated, where the first evaluation index is greater than or equal to a first preset threshold, as a target image;

the processing unit 903 is configured to process the target image to obtain processing results of different target objects in the target image, where the processing results at least include an area corresponding to the target object that cannot be identified;

a calculating unit 904, configured to calculate a second evaluation index of the target image according to the area of the region, where the second evaluation index is used to characterize a quality evaluation result of the target image.

In some implementations, the first determining unit 902 is specifically configured to: extracting features of an image to be evaluated to obtain image feature information; calculating information entropy, normalized relative gray distance and average contrast of the image to be evaluated based on the image characteristic information; and determining a first evaluation index of the image to be evaluated according to the information entropy, the normalized relative gray distance and the average contrast of the image to be evaluated.

In some implementations, the processing unit 903 is specifically configured to: acquiring a target image to be processed, wherein the target image comprises a plurality of target objects; identifying a target image through a target identification model obtained through pre-training to obtain a classification label corresponding to each target object, wherein the classification label at least comprises an identifiable label and an unrecognizable label; and counting the area of the target object corresponding to the unrecognizable label in the target image by utilizing the target segmentation processing module obtained by pre-training.

In some implementations, the computing unit 904 is specifically configured to: calculating the area ratio of the area of the region to the area of the target image; and calculating a second evaluation index of the target image based on the mapping relation between the area ratio and the second evaluation index.

In some implementations, the apparatus 900 further includes: and a second determining unit 905, where the second determining unit 905 is configured to determine, according to a second evaluation index of the target image, a fusion policy of the target image in visual perception fusion.

In some implementations, the second determining unit 905 is specifically configured to take the target image as the image to be fused in the visual perception fusion and take the second evaluation index as the weighting coefficient of the image to be fused to participate in the visual perception fusion if the second evaluation index of the target image is greater than the second preset threshold.

In some implementations, the second determining unit 905 is specifically further configured to take the target image as the image to be fused in the visual perception fusion and take the second evaluation index as the priority coefficient to participate in the visual perception fusion if the second evaluation index of the target image is less than or equal to the second preset threshold.

It should be understood that the apparatus embodiment and the image quality evaluation method embodiment may correspond to each other, and similar descriptions may refer to the image quality evaluation method embodiment. To avoid repetition, no further description is provided here. Specifically, the apparatus 900 shown in fig. 9 may perform the above-mentioned image quality evaluation method embodiment, and the foregoing and other operations and/or functions of each module in the apparatus 900 are respectively for implementing the corresponding flow in the above-mentioned image quality evaluation method, which is not repeated herein for brevity.

The apparatus 900 of the embodiments of the present application is described above from the perspective of functional modules in connection with the accompanying drawings. It should be understood that the functional module may be implemented in hardware, or may be implemented by instructions in software, or may be implemented by a combination of hardware and software modules. Specifically, each step of the image quality evaluation method embodiment in the embodiment of the present application may be completed by an integrated logic circuit of hardware in a processor and/or an instruction in a software form, and the steps of the image quality evaluation method disclosed in connection with the embodiment of the present application may be directly embodied and executed by a hardware decoding processor or may be completed by a combination of hardware and software modules in the decoding processor. Alternatively, the software modules may be located in a well-established storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, and the like. The storage medium is located in the memory, and the processor reads the information in the memory, and the steps in the embodiment of the image quality evaluation method are completed by combining the hardware of the processor.

As shown in fig. 10, the electronic device 1000 may include:

A memory 1010 and a processor 1020, the memory 1010 being for storing a computer program and for transmitting the program code to the processor 1020. In other words, the processor 1020 may call and run a computer program from the memory 1010 to implement the methods in embodiments of the present application.

For example, the processor 1020 may be configured to perform the above-described method embodiments according to instructions in the computer program.

In some embodiments of the present application, the processor 1020 may include, but is not limited to:

a general purpose processor, digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like.

In some embodiments of the present application, the memory 1010 includes, but is not limited to:

volatile memory and/or nonvolatile memory. The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable EPROM (EEPROM), or a flash Memory. The volatile memory may be random access memory (Random Access Memory, RAM) which acts as an external cache. By way of example, and not limitation, many forms of RAM are available, such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (Double Data Rate SDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), and Direct memory bus RAM (DR RAM).

In some embodiments of the present application, the computer program may be partitioned into one or more modules that are stored in the memory 1010 and executed by the processor 1020 to perform the methods provided herein. The one or more modules may be a series of computer program instruction segments capable of performing the specified functions, which are used to describe the execution of the computer program in the electronic device.

As shown in fig. 10, the electronic device may further include:

a transceiver 1030, the transceiver 1030 being connectable to the processor 1020 or the memory 1010.

The processor 1020 may control the transceiver 1030 to communicate with other devices, and in particular, may send information or data to other devices or receive information or data sent by other devices. The transceiver 1030 may include a transmitter and a receiver. The transceiver 1030 may further include an antenna, the number of which may be one or more.

It will be appreciated that the various components in the electronic device are connected by a bus system that includes, in addition to a data bus, a power bus, a control bus, and a status signal bus.

The present application also provides a computer storage medium having stored thereon a computer program which, when executed by a computer, enables the computer to perform the method of the above-described method embodiments. Alternatively, embodiments of the present application also provide a computer program product comprising instructions which, when executed by a computer, cause the computer to perform the method of the method embodiments described above.

When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces, in whole or in part, a flow or function consistent with embodiments of the present application. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., a floppy Disk, a hard Disk, a magnetic tape), an optical medium (e.g., a digital video disc (Digital Video Disc, DVD)), or a semiconductor medium (e.g., a Solid State Disk (SSD)), or the like.

Those of ordinary skill in the art will appreciate that the various illustrative modules and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, and for example, the division of the modules is merely a logical function division, and there may be additional divisions when actually implemented, for example, multiple modules or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or modules, which may be in electrical, mechanical, or other forms.

The modules illustrated as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules, i.e., may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. For example, functional modules in the embodiments of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module.

The foregoing is merely a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes or substitutions are covered in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An image quality evaluation method, comprising:

acquiring at least one image to be evaluated;

determining a first evaluation index of the image to be evaluated according to the image characteristics of the image to be evaluated, and taking the image to be evaluated, of which the first evaluation index is larger than or equal to a first preset threshold value, as a target image;

Processing the target image to obtain processing results of different target objects in the target image, wherein the processing results at least comprise area corresponding to the target objects which cannot be identified;

and calculating a second evaluation index of the target image according to the area of the region, wherein the second evaluation index is used for representing a quality evaluation result of the target image.

2. The method according to claim 1, wherein determining the first evaluation index of the image to be evaluated according to the image characteristics of the image to be evaluated comprises:

extracting features of the image to be evaluated to obtain image feature information;

calculating information entropy, normalized relative gray distance and average contrast of the image to be evaluated based on the image characteristic information;

and determining a first evaluation index of the image to be evaluated according to the information entropy, the normalized relative gray distance and the average contrast of the image to be evaluated.

3. The method according to claim 1, wherein the processing the target image to obtain a processing result of different target objects in the target image, where the processing result includes at least an area corresponding to an unrecognizable target object, includes:

Acquiring a target image to be processed, wherein the target image comprises a plurality of target objects;

identifying the target image through a target identification model obtained through pre-training to obtain a classification label corresponding to each target object, wherein the classification label at least comprises an identifiable label and an unrecognizable label;

and counting the area of the target object corresponding to the unrecognizable label in the target image by using a target segmentation processing module obtained through pre-training.

4. The method according to claim 1, wherein calculating a second evaluation index of the target image according to the area of the region, the second evaluation index being used for characterizing a quality evaluation result of the target image, comprises:

calculating the area ratio of the area of the region to the area of the target image;

and calculating a second evaluation index of the target image based on the mapping relation between the area occupation ratio and the second evaluation index.

5. The method according to claim 1, wherein the method further comprises:

and determining a fusion strategy of the target image in visual perception fusion according to the second evaluation index of the target image.

6. The method of claim 5, wherein determining a fusion policy of the target image in visual perception fusion based on the second evaluation index of the target image comprises:

And if the second evaluation index of the target image is larger than a second preset threshold value, taking the target image as an image to be fused in visual perception fusion, and taking the second evaluation index as a weighting coefficient of the image to be fused so as to participate in the visual perception fusion.

7. The method of claim 6, wherein the method further comprises:

and if the second evaluation index of the target image is smaller than or equal to a second preset threshold value, taking the target image as an image to be fused in visual perception fusion and taking the second evaluation index as a priority coefficient to participate in the visual perception fusion.

8. An image quality evaluation device, comprising:

the acquisition unit is used for acquiring at least one image to be evaluated;

a first determining unit, configured to determine a first evaluation index of the image to be evaluated according to an image feature of the image to be evaluated, and take the image to be evaluated, where the first evaluation index is greater than or equal to a first preset threshold, as a target image;

the processing unit is used for processing the target image to obtain processing results of different target objects in the target image, wherein the processing results at least comprise area corresponding to the target objects which cannot be identified;

And the calculating unit is used for calculating a second evaluation index of the target image according to the area of the region, wherein the second evaluation index is used for representing a quality evaluation result of the target image.

9. An electronic device, comprising:

a processor and a memory for storing a computer program, the processor being for invoking and running the computer program stored in the memory to perform the method of any of claims 1-7.

10. A computer readable storage medium storing a computer program for causing a computer to perform the method of any one of claims 1-7.