CN116915971A - Quality evaluation method, device, equipment and medium for hardware-in-loop video data - Google Patents
Quality evaluation method, device, equipment and medium for hardware-in-loop video data Download PDFInfo
- Publication number
- CN116915971A CN116915971A CN202310718576.0A CN202310718576A CN116915971A CN 116915971 A CN116915971 A CN 116915971A CN 202310718576 A CN202310718576 A CN 202310718576A CN 116915971 A CN116915971 A CN 116915971A
- Authority
- CN
- China
- Prior art keywords
- video image
- image
- domain
- video
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 84
- 238000013441 quality evaluation Methods 0.000 title claims abstract description 24
- 238000013507 mapping Methods 0.000 claims description 22
- 238000004422 calculation algorithm Methods 0.000 claims description 15
- 238000002347 injection Methods 0.000 claims description 15
- 239000007924 injection Substances 0.000 claims description 15
- 230000008447 perception Effects 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 11
- 238000001303 quality assessment method Methods 0.000 claims description 9
- 238000001514 detection method Methods 0.000 claims description 5
- 238000011156 evaluation Methods 0.000 abstract description 21
- 230000000694 effects Effects 0.000 abstract description 10
- 238000012360 testing method Methods 0.000 description 10
- 238000006243 chemical reaction Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 7
- 238000004088 simulation Methods 0.000 description 7
- 230000005540 biological transmission Effects 0.000 description 6
- 230000009467 reduction Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 239000000243 solution Substances 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N17/00—Diagnosis, testing or measuring for television systems or their details
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C5/00—Registering or indicating the working of vehicles
- G07C5/08—Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
- G07C5/0808—Diagnosing performance data
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C5/00—Registering or indicating the working of vehicles
- G07C5/08—Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
- G07C5/0841—Registering performance data
- G07C5/085—Registering performance data using electronic data carriers
- G07C5/0866—Registering performance data using electronic data carriers the electronic data carrier being a digital video recorder in combination with video camera
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention provides a quality evaluation method, a device, equipment and a medium of hardware-in-loop video data, wherein the method comprises the following steps: by collecting a plurality of video data under different environments and different scenes, the video data comprises compressed storage video images and lossless storage video images; injecting the compressed storage video image into an intelligent driving domain controller, and restoring the compressed storage video image into a raw domain video image by using an image interpolation method; and comparing the raw domain video image with the lossless storage video image by taking the lossless storage video image as a reference so as to evaluate the data quality of the restored raw domain video image. The invention realizes pixel expansion based on the video image compressed in the real scene by adopting an image interpolation method, restores to a raw domain video image, evaluates the quality of the data in an image space domain and a frequency domain, solves the problem that the evaluation of the restored video image is easily influenced by factors such as subjective evaluation, display equipment effect and the like, unifies the evaluation standard and accelerates the evaluation speed.
Description
Technical Field
The invention relates to the technical field of vehicle simulation test, in particular to a quality evaluation method, device, equipment and medium of hardware-in-the-loop video data.
Background
The HIL (hardware in the loop ) closed loop is to simulate the running state of the tested electric control unit by running a simulation model through a real-time processor, simulate the input state of the tested electric control unit through an I/O interface, a bus interface and the like, and finally realize the closed loop test by automatically judging whether the output state of the tested electric control unit is a design result.
The safe intelligent driving system needs to be subjected to road test of billions kilometers, the real vehicle training test cost is high, time and labor are consumed, the rapid, efficient and low-cost bench and simulation test are adopted by various companies, and the problem to be solved is solved in a method for realizing closed loop test of intelligent driving domain hardware in the loop, wherein the method is also one scheme for hardware in the loop of historical acquisition data playback. The present video storage scheme compresses and stores high-definition video data, limited by the present storage technology and hardware cost. In order to realize closed-loop simulation of the domain controller, interpolation reduction needs to be carried out on the compressed video, whether the video after interpolation truly restores an original image or not is judged by subjective human eyes, time and effort are consumed at present, and the evaluation result has larger differences in individual feeling, display equipment effect and the like.
Disclosure of Invention
Aiming at the problems in the prior art, the hardware-in-loop video data quality evaluation method, device, equipment and medium provided by the invention restore the injected compressed storage video image in the loop simulation test of the intelligent driving hardware of the vehicle, solve the problem that the restored video image is easily influenced by factors such as human subjective evaluation factors and display equipment effects to cause larger evaluation difference, unify evaluation criteria and accelerate the evaluation speed.
The invention provides a quality evaluation method of hardware-in-loop video data, which comprises the following steps:
collecting a plurality of video data under different environments and different scenes, wherein the video data comprises compressed storage video images and lossless storage video images;
injecting the compressed storage video image into an intelligent driving domain controller, and restoring the compressed storage video image into a raw domain video image by using an image interpolation method;
and comparing the raw domain video image with the lossless storage video image by taking the lossless storage video image as a reference so as to evaluate the data quality of the restored raw domain video image.
According to the method for evaluating the quality of hardware-in-loop video data provided by the invention, the step of comparing the raw domain video image with the lossless storage video image comprises the following steps: performing feature contrast on the restored raw domain video image and the lossless storage video image in an image space domain, wherein the feature contrast comprises the following steps of:
converting the raw domain video image and the lossless storage video image into RGB domain images respectively;
determining the color mapping difference value of each channel of the image based on the pixel points with the same coordinate position in the RGB domain image;
determining the center deviation and standard deviation of the color mapping deviation of the total pixel points based on the color mapping difference value;
and evaluating the color gamut accuracy of the raw domain video image based on the center deviation and the standard deviation.
According to the method for evaluating the quality of hardware-in-loop video data provided by the invention, the step of comparing the restored raw domain video image with the lossless storage video image in the image space domain further comprises the following steps:
converting the RGB domain images corresponding to the raw domain video image and the lossless storage video image into YUV domain images respectively;
determining a gray level difference value of an image edge part based on pixel points with the same coordinate position in the YUV domain image;
and evaluating the spatial definition of the raw domain video image based on the gray level difference value of the image edge part.
According to the method for evaluating the quality of hardware-in-loop video data provided by the invention, the step of comparing the raw domain video image with the lossless storage video image further comprises the following steps: performing feature contrast on the restored video image and the lossless storage video image in an image frequency domain, wherein the method specifically comprises the following steps:
converting YUV domain images corresponding to the raw domain video image and the lossless storage video image from a space domain to a frequency domain respectively;
and acquiring high-frequency components of the image frequency chart and comparing and analyzing to evaluate the high-frequency characteristics of the raw domain video image.
According to the quality evaluation method of hardware-in-the-loop video data provided by the invention, the step of comparing the raw domain video image with the lossless storage video image by taking the lossless storage video image as a reference to evaluate the data quality of the restored raw domain video image further comprises the following steps: and comparing and identifying the raw domain video image with the lossless storage video image through a preset perception algorithm, and determining the target detection rate and the data precision of the raw domain video image so as to verify the data quality of the injected raw domain video image.
According to the method for evaluating the quality of hardware-in-the-loop video data provided by the invention, the step of injecting the compressed storage video data into the intelligent driving domain controller comprises the following steps:
converting the compressed storage video image into HDMI image data frame by frame and sending the HDMI image data to a video injection board card;
converting the HDMI image data into high definition image data by an image interpolation method;
the video injection board converts the high-definition image data into camera format data and injects the camera format data into the intelligent driving domain controller of the hardware-in-the-loop;
and the intelligent driving domain controller restores the camera format data into the raw domain video image.
According to the method for evaluating the quality of hardware-in-the-loop video data provided by the invention, the step of injecting the compressed storage video image into the intelligent driving domain controller further comprises the following steps:
transmitting the camera format data to the intelligent driving domain controller in a serial manner;
the intelligent driving domain controller deserializes the camera format data transmitted in series to restore to the raw domain video image.
The invention also provides a quality evaluation device of the hardware-in-loop video data, which comprises:
the video data acquisition module is used for acquiring a plurality of video data under different environments and different scenes, wherein the video data comprises compressed storage video images and lossless storage video images;
the video image restoration module is used for injecting the compressed storage video image into the intelligent driving domain controller and restoring the compressed storage video image into a raw domain video image by using an image interpolation method;
and the data quality evaluation module is used for comparing the raw domain video image with the lossless storage video image by taking the lossless storage video image as a reference so as to evaluate the data quality of the restored raw domain video image.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor realizes the hardware-in-loop video data quality evaluation method according to any one of the above when executing the program.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method of quality assessment of hardware-in-loop video data as described in any of the above.
The quality evaluation method, the device, the equipment and the medium for the hardware-in-loop video data provided by the invention are characterized in that by collecting a plurality of video data in different environments and different scenes, the video data comprise compressed storage video images and lossless storage video images; injecting the compressed storage video image into an intelligent driving domain controller, and restoring the compressed storage video image into a raw domain video image by using an image interpolation method; and comparing the raw domain video image with the lossless storage video image by taking the lossless storage video image as a reference so as to evaluate the data quality of the restored raw domain video image. The invention realizes pixel expansion based on the video image compressed in the real scene by adopting an image interpolation method, restores to a raw domain video image, evaluates the quality of video data in an image space domain and a frequency domain, solves the problem that the restored video image is easily influenced by factors such as human subjective evaluation factors, display equipment effects and the like, causes great evaluation difference, unifies evaluation standards and accelerates the evaluation speed.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a method for evaluating quality of hardware-in-loop video data according to the present invention;
FIG. 2 is a data acquisition flow chart of a hardware-in-the-loop video data quality evaluation method provided by the invention;
FIG. 3 is a video data injection flow chart of a hardware-in-the-loop video data quality evaluation method provided by the invention;
FIG. 4 is a schematic diagram of video image output provided by the present invention;
FIG. 5 is a video image comparison flow chart of a hardware-in-the-loop video data quality evaluation method provided by the invention;
FIG. 6 is a schematic diagram of a hardware-in-the-loop video quality assessment device according to the present invention;
fig. 7 is a schematic structural diagram of an electronic device provided by the present invention.
Reference numerals:
21: a video data acquisition module; 22: a video image restoration module; 23: and a data quality evaluation module.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be clearly and completely described in the following description with reference to specific embodiments of the present invention and the accompanying drawings. It will be apparent that the described embodiments are only some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be noted that those skilled in the art explicitly and implicitly understand that the described embodiments of the invention can be combined with other embodiments without conflict. Unless defined otherwise, technical or scientific terms used herein should be given the ordinary meaning as understood by one of ordinary skill in the art to which this invention belongs. The terms "a," "an," "the," and similar referents in the context of the invention are not to be construed as limiting the quantity, but rather as singular or plural. The terms "comprising," "including," "having," and any variations thereof, are intended to cover a non-exclusive inclusion; the terms "first," "second," "third," and the like, as used herein, are merely distinguishing between similar objects and not representing a particular ordering of objects.
The hardware-in-the-loop simulation test system comprises a video injection module, wherein the video injection module is connected with an upper computer and is used for receiving information of a real road scene, obtaining a source video according to the information of the real road scene and then sending the source video to a tested electronic control unit, such as an intelligent driving domain controller. The hardware of the domain controller mainly judges the video injection effect through human eyes, experimenters with abundant color observation experience watch and score the content of the injected video one by one, time and labor are consumed, and individual subjective differences are large. Therefore, the invention provides a quality evaluation method of hardware-in-loop video data, which aims to solve the evaluation difference problem which varies from person to person.
Example 1
Referring to fig. 1, the present embodiment provides a quality evaluation method for hardware-in-loop video data, including:
step S1: collecting a plurality of video data under different environments and different scenes, wherein the video data comprises compressed storage video images and lossless storage video images;
specifically, as shown in fig. 2, a certain amount of video data in different environments and different scenes is collected through an on-board camera device, the video data comprise conventional compressed stored video data and lossless stored video data (raw data), the video of the lossless stored video data is used as a reference standard library video, after the images of the compressed stored video data are restored later, the video images of the compressed stored video data and the video images of the compressed stored video data are compared, and quality evaluation is carried out on the restored data.
Step S2: injecting the compressed storage video image into an intelligent driving domain controller, and restoring the compressed storage video image into a raw domain video image by using an image interpolation method;
in this embodiment, step S2 includes:
step S201: converting the compressed storage video image into HDMI image data frame by frame and transmitting the HDMI image data to a video injection board card;
step S202: converting the HDMI image data into high definition image data by an image interpolation method;
step S203: the video injection board card converts the high-definition image data into camera format data and injects the camera format data into the intelligent driving domain controller of the hardware-in-the-loop;
step S204: the intelligent driving domain controller restores the camera format data to a raw domain video image.
Specifically, as shown in fig. 3, the video data stored in a conventional compression manner is played frame by frame and sent to a video injection board through a video data protocol such as HDMI, and the video injection board is used to collect HDMI image data, and to implement pixel expansion (a method of increasing the size of an image pixel without generating a pixel, calculating the color of a lost pixel by a mathematical formula, such as nearest neighbor interpolation, bilinear interpolation, cubic spline interpolation, etc.), so as to obtain high-definition image data. The video injection board converts the high-definition image data from DSI (display serial interface ) to camera format data of CSI (camera serial interface ), and performs tone mapping (tone mapping) of each channel color, where tone mapping is used to map the color from an original tone (usually high dynamic range, HDR) to a target tone (usually low dynamic range, LDR), and the mapping result is displayed through a medium, so as to achieve the effect of restoring the original scene as much as possible under the visual characteristics of human eyes. Therefore, the conventional compressed stored video data is restored to lossless stored video data which is close to the original image through the steps as far as possible, so that the influence on related training caused by unrealistic restoration of the injected video in the loop training stage of the perception algorithm hardware is reduced as far as possible.
In step S203, the camera format data is transmitted to the intelligent driving domain controller in a serial manner, which specifically includes: the converted CSI camera format data is transmitted in a serial mode through a corresponding serial chip, the corresponding information data is transmitted in a serial mode, and the video image transmission is completed through a corresponding GMSL cable (GMSL, gigabit Multimedia Serial Links, chinese name is gigabit multimedia serial link, which is a high-speed serial interface suitable for transmission of audio, video and control signals) and is injected into an intelligent driving domain controller of hardware in the loop.
In step S204, the intelligent driving domain controller deserializes the serially transmitted camera format data to restore to a raw domain video image, which specifically includes: and the intelligent driving domain controller restores the serial data stream into a raw domain video image through the corresponding deserializing chip.
The serializer/deserializer technique used in steps S203 and S204 is abbreviated as SerDes. The parallel data stream is first converted into a serial data stream by a serializer, then transmitted at a higher frequency, and then the received serial data stream is converted into a parallel data stream by a deserializer. The technology fully utilizes the channel capacity of the transmission medium, reduces the number of required transmission channels and device pins, and improves the transmission speed of signals, thereby greatly reducing the communication cost.
Step S3: and comparing the raw domain video image with the lossless storage video image by taking the lossless storage video image as a reference so as to evaluate the data quality of the restored raw domain video image.
Image enhancement is the use of a series of techniques to improve the visual effect of an image or to transform an image into a form that is more suitable for analysis and processing by a person or machine. For example, a series of techniques are used to selectively highlight certain information of interest while suppressing some unwanted information, thereby improving the value of the image. The image enhancement method starts from an enhancement action domain and can be divided into two types of spatial domain enhancement and frequency domain enhancement.
Spatial domain (spatial domain): the image plane is directly an image which can be directly recognized by a person.
In this embodiment, in step S3, feature comparison is performed on the restored video image and the lossless storage video image in the image space domain, including:
step S301: converting the raw domain video image and the lossless storage video image into RGB domain images respectively;
step S302: determining the color mapping difference value of each channel of the image based on the pixel points with the same coordinate position in the RGB domain image;
step S303: determining the center deviation and standard deviation of the color mapping deviation of the total pixel points based on the color mapping difference value;
step S304: and evaluating the color gamut accuracy of the raw domain video image based on the center deviation and the standard deviation.
Specifically, referring to the video image output principle shown in fig. 4, the raw domain video image and the lossless storage video image are respectively subjected to automatic white balance, color correction, gamma correction, and the like of the ISP to complete conversion from the raw domain to the RGB domain. Referring to the video image comparison flow shown in fig. 5, in the image RGB domain, the restored raw domain video image is compared with the stored lossless storage video image, and the difference value of three channels is calculated by performing R, G, B difference calculation on the pixel points with the same coordinate positions in the RGB domain image, so as to calculate the color mapping difference value of each channel. Based on the color mapping difference value of each channel, calculating the center deviation and standard deviation of the color mapping deviation by counting the deviation magnitude and deviation distribution condition of the color mapping of the total pixel points. The closer the center deviation of the color map is to zero and the smaller the standard deviation, the smaller the deviation of the overall color map characterizing the restored raw domain video image. Therefore, the difference value of each pixel channel is adopted in the RGB domain of the image so as to evaluate the restored raw domain video image from the color gamut accuracy of the image space domain, and the color gamut accuracy of the injected video is effectively ensured.
In this embodiment, in step S3, feature contrast is performed on the restored video image and the lossless storage video image in the image space domain, and the method further includes:
step S305: converting RGB domain images corresponding to the raw domain video image and the lossless storage video image into YUV domain images respectively;
step S306: determining a gray level difference value of an image edge part based on pixel points with the same coordinate position in the YUV domain image;
step S307: and evaluating the spatial definition of the raw domain video image based on the gray level difference value of the image edge part.
Specifically, the red domain video image and the RGB domain image corresponding to the lossless storage video image are converted into YUV format (YUV format means pixel format in which luminance parameter and chrominance parameter are expressed separately, and color model is derived from RGB model) through YUV noise reduction and enhancement of ICP (Intra Coded Pictures, intra-frame coding diagram), wherein "Y" represents luminance, that is, gray value, "U" and "V" represent chrominance, and the color of the designated pixel is used to describe image color and saturation. The YUV image denoising link can effectively increase the signal-to-noise ratio of image signals again before final imaging of the image, improve the image quality and directly influence the visual effect of the subsequent image. The information processed by the link includes image brightness and chromaticity. The RGB domain image is converted into YUV domain image to obtain the pixel format of brightness and chromaticity of the image. The definition of the image is calculated based on the gradient value of the gray level of each pixel point at the same coordinate position in the YUV domain image, specifically, the gray level value of the upper, lower, left and right fields of each pixel is obtained to be a horizontal convolution kernel and a vertical convolution kernel in the YUV domain image through a Sobel operator, and the horizontal gray level difference value and the vertical gray level difference value are calculated respectively. The average gray difference value of the image after the edge is extracted by the Sobel operator is smaller, the higher the image quality of the restored raw domain video image is represented, and the clearer the image is. Therefore, the gray difference value of the image edge part is obtained in the YUV domain of the image, so that the restored raw domain video image is evaluated from the spatial definition of the image spatial domain, and the spatial definition of the injected video is effectively ensured.
Frequency domain (frequency domain): the characteristic of the image is described by taking the space frequency as an independent variable, the spatial variation of the pixel value of one image can be decomposed into linear superposition of Jian Zhen functions with different amplitudes, space frequencies and phases, the composition and distribution of various space frequency components in the image are called image frequency spectrum, and the image is obtained by carrying out frequency domain transformation such as Fourier transformation on the space domain image.
In this embodiment, in step S3, feature comparison is performed on the restored video image and the lossless storage video image in the image frequency domain, including:
step S308: converting YUV domain images corresponding to the raw domain video image and the lossless storage video image from a space domain to a frequency domain respectively;
step S309: and acquiring high-frequency components of the image frequency chart and comparing and analyzing to evaluate the high-frequency characteristics of the raw domain video image.
Specifically, YUV domain images corresponding to the raw domain video image and the lossless storage video image are respectively processed through two-dimensional discrete fourier transform to complete the conversion from the spatial domain to the frequency domain. In the image frequency domain, the low frequency signal represents the area of the image where the brightness and gray level change are small, and is a description of the main area of the image. The high-frequency signal represents a region with large image brightness and gray level variation, and represents the image contour and detail. Through the contrast analysis of the high-frequency components of the image, the smaller the difference of the high-frequency components is, the clearer the image detail reduction of the video image representing the raw domain is, and the richer the image is. Therefore, the conversion of the image from the space domain to the frequency domain is completed by adopting a two-dimensional discrete Fourier transform method, the difference size and the distribution of the high-frequency signals are analyzed in frequency, and the texture characteristics of the image are effectively ensured by evaluating the high-frequency characteristics of the image.
Further, by calculating the deviation of different color mapping and the gradient value deviation of gray scale on the spatial domain and analyzing the deviation of high-frequency signals on the frequency domain for the restored raw video image and the stored lossless storage video image, the ratio between each deviation value and the original characteristic value of the lossless storage video image is further calculated to obtain the difference ratio, and if the difference ratio is within 10%, the restored raw video image meets the data restoration requirement.
Therefore, after the compressed storage video image is restored, the quality evaluation is carried out on the spatial domain and the frequency domain characteristics of the image from the pixel level, and the injection precision of the domain controller hardware in-loop test data is effectively improved.
In this embodiment, after step S3, the method further includes: and comparing and identifying the raw domain video image with the lossless storage video image through a preset perception algorithm, and determining the target detection rate and the data precision of the raw domain video image so as to verify the data quality of the injected raw domain video image.
Specifically, the image processed in the YUV domain is finally saved and used for recognition by a perception algorithm, and the preset perception algorithm is a recognition algorithm for recognizing traffic environments of vehicles and pedestrians in the video image. The video injected into the domain controller is mainly used for target recognition by a perception algorithm, and the machine vision and the artificial vision used by the perception algorithm have certain difference, so that the image quality judged by human eyes cannot well meet the restoration of the machine vision to the image. Therefore, the lossless storage video image is taken as a reference, the image of the raw domain is subjected to comparison and identification through a perception algorithm, when the target detection rate and the data precision of the image of the raw domain are not lower than 95%, the target identification result is consistent, the grading result of the algorithm is basically consistent, and the data reduction requirement is met for the reduction of the compressed storage video image. Therefore, the identification is carried out through the corresponding perception algorithm, whether the video data meets the data restoration requirement is judged, and the quality of the injected video data can be verified rapidly and effectively, so that the problem that the quality of the injected video influences the target identification when the perception algorithm hardware is used for loop training is solved.
In summary, the hardware-in-loop video data quality evaluation method provided by the embodiment is used for the vehicle intelligent driving hardware-in-loop simulation test, based on the video image compressed in the real scene, the image interpolation method is adopted to perform pixel expansion reduction on the injected compressed storage video image so as to restore the video image into a raw domain video image, and the video data quality is evaluated in the image space domain and the frequency domain, so that the problem that the restored video image is easily influenced by factors such as human subjective evaluation factors and display equipment effects, so that evaluation difference is large is solved, the evaluation standard is unified, and the evaluation speed is accelerated.
Example two
Based on the same inventive concept as the method of the first embodiment, referring to fig. 6, the present embodiment provides a quality evaluation device of hardware-in-loop video data, including:
a video data acquisition module 21, configured to acquire a plurality of video data in different environments and different scenes, where the video data includes a compressed storage video image and a lossless storage video image;
the video image restoration module 22 is configured to inject the compressed stored video image into the intelligent driving domain controller, and restore the compressed stored video image into a raw domain video image by using an image interpolation method;
the data quality evaluation module 23 is configured to compare the raw domain video image with the lossless storage video image based on the lossless storage video image, so as to evaluate the data quality of the restored raw domain video image.
In this embodiment, the video image restoration module 22 includes: the first conversion unit is used for converting the compressed storage video image into HDMI image data frame by frame and sending the HDMI image data to the video injection board card; a second conversion unit for converting the HDMI image data into high definition image data by an image interpolation method; a third conversion unit for converting the high definition image data into camera format data; the serial transmission unit is used for transmitting the camera format data to the intelligent driving domain controller in a serial mode; the video data restoring unit is used for de-serializing the camera format data transmitted in series so as to restore the camera format data into a raw domain video image.
The data quality evaluation module 23 includes: the spatial domain feature comparison unit is used for performing feature comparison on the restored video image and the lossless storage video image in the image spatial domain; and the frequency domain feature comparison unit is used for performing feature comparison on the restored video image and the lossless storage video image in the image frequency domain.
Specifically, the spatial domain feature contrast unit includes: an RGB domain image conversion subunit, configured to convert the raw domain video image and the lossless storage video image into RGB domain images respectively; the channel color mapping difference value determining subunit is used for determining the color mapping difference value of each channel of the image based on the pixel points with the same coordinate position in the RGB domain image; a color mapping center deviation and standard deviation determining subunit, configured to determine a center deviation and a standard deviation of the color mapping deviation of the total pixel point based on the color mapping difference; and the first evaluation subunit is used for evaluating the color gamut accuracy of the raw domain video image based on the center deviation and the standard deviation.
The spatial domain feature contrast unit further includes: the YUV domain image conversion subunit is used for respectively converting RGB domain images corresponding to the raw domain video image and the lossless storage video image into YUV domain images; an image edge gray level difference value determining subunit, configured to determine a gray level difference value of an image edge portion based on pixel points in the same coordinate position in the YUV domain image; and the second evaluation subunit is used for evaluating the spatial definition of the raw domain video image based on the gray level difference value of the image edge part.
The frequency domain feature comparison unit includes: the frequency domain conversion subunit is used for respectively converting YUV domain images corresponding to the raw domain video image and the lossless storage video image from a space domain to a frequency domain; and the third evaluation subunit is used for acquiring the high-frequency component of the image frequency chart and performing contrast analysis so as to evaluate the high-frequency characteristic of the raw domain video image.
The apparatus further comprises: and the perception algorithm verification module is used for comparing and identifying the raw domain video image and the lossless storage video image through a preset perception algorithm, and determining the target detection rate and the data precision of the raw domain video image so as to verify the data quality of the injected raw domain video image.
The implementation process of the functions and actions of each module in the above device is specifically detailed in the implementation process of the corresponding steps in the above method, so relevant parts only need to be referred to in the description of the method embodiments, and are not repeated here.
The above-described embodiment of the apparatus is merely illustrative, for example, the division of the modules is merely a logic function division, and there may be another division manner in actual implementation, and each functional module in the embodiment may be all integrated in one processor, or each module may be separately used as one device, or two or more modules may be integrated in one device; the functional modules in the embodiments may be implemented in the form of hardware or in the form of hardware and software functional units.
Example III
Referring to fig. 7, the present embodiment provides an electronic apparatus including: processor 310 (processor), communication interface 320 (Communications Interface), memory 330 (memory) and communication bus 340, wherein processor 310, communication interface 320, memory 330 complete communication with each other through communication bus 340. Processor 310 may invoke logic instructions in memory 330, processor 310 performing a hardware-in-the-loop video data quality assessment method as described in embodiment one, the method comprising:
collecting a plurality of video data under different environments and different scenes, wherein the video data comprises compressed storage video images and lossless storage video images;
injecting the compressed storage video image into an intelligent driving domain controller, and restoring the compressed storage video image into a raw domain video image by using an image interpolation method;
and comparing the raw domain video image with the lossless storage video image by taking the lossless storage video image as a reference so as to evaluate the data quality of the restored raw domain video image.
Further, the logic instructions in the memory 330 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution in the form of a software product stored in a storage medium, comprising several instructions to cause a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, the computer program product comprising a computer program, the computer program being storable on a non-transitory computer readable storage medium, the computer program, when executed by a processor, being capable of performing a method of quality assessment of hardware-in-loop video data as described in embodiment one, the method comprising:
collecting a plurality of video data under different environments and different scenes, wherein the video data comprises compressed storage video images and lossless storage video images;
injecting the compressed storage video image into an intelligent driving domain controller, and restoring the compressed storage video image into a raw domain video image by using an image interpolation method;
and comparing the raw domain video image with the lossless storage video image by taking the lossless storage video image as a reference so as to evaluate the data quality of the restored raw domain video image.
Example IV
The present embodiment provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the hardware-in-loop video data quality assessment method according to the first embodiment, the method comprising:
collecting a plurality of video data under different environments and different scenes, wherein the video data comprises compressed storage video images and lossless storage video images;
injecting the compressed storage video image into an intelligent driving domain controller, and restoring the compressed storage video image into a raw domain video image by using an image interpolation method;
and comparing the raw domain video image with the lossless storage video image by taking the lossless storage video image as a reference so as to evaluate the data quality of the restored raw domain video image.
The embodiments of the present invention are described in a progressive manner, and the same and similar parts of the embodiments are all referred to each other, and each embodiment is mainly described in the differences from the other embodiments. In particular, for the apparatus and medium embodiments, the description is relatively simple, as it is substantially similar to the method embodiments, with reference to the section of the method embodiments being relevant.
The devices and media provided in the embodiments of the present invention are in one-to-one correspondence with the methods, so that the devices and media also have similar beneficial technical effects as the corresponding methods, and since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media are not repeated here.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process article or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process article or method. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process method article or method comprising the element.
The foregoing is merely exemplary of the present invention and is not intended to limit the present invention. While the invention has been described in detail in the foregoing general description and with reference to specific embodiments thereof, it will be apparent to one skilled in the art that modifications and improvements can be made thereto. Accordingly, such modifications or improvements may be made without departing from the spirit of the invention and are intended to be within the scope of the invention as claimed.
Claims (10)
1. A method for quality assessment of hardware-in-the-loop video data, comprising:
collecting a plurality of video data under different environments and different scenes, wherein the video data comprises compressed storage video images and lossless storage video images;
injecting the compressed storage video image into an intelligent driving domain controller, and restoring the compressed storage video image into a raw domain video image by using an image interpolation method;
and comparing the raw domain video image with the lossless storage video image by taking the lossless storage video image as a reference so as to evaluate the data quality of the restored raw domain video image.
2. The method of claim 1, wherein the step of comparing the raw domain video image with the lossless stored video image comprises: performing feature contrast on the restored raw domain video image and the lossless storage video image in an image space domain, wherein the feature contrast comprises the following steps of:
converting the raw domain video image and the lossless storage video image into RGB domain images respectively;
determining the color mapping difference value of each channel of the image based on the pixel points with the same coordinate position in the RGB domain image;
determining the center deviation and standard deviation of the color mapping deviation of the total pixel points based on the color mapping difference value;
and evaluating the color gamut accuracy of the raw domain video image based on the center deviation and the standard deviation.
3. The method for evaluating quality of hardware-in-the-loop video data according to claim 2, wherein the step of comparing the restored raw domain video image with the lossless stored video image in an image space domain further comprises:
converting the RGB domain images corresponding to the raw domain video image and the lossless storage video image into YUV domain images respectively;
determining a gray level difference value of an image edge part based on pixel points with the same coordinate position in the YUV domain image;
and evaluating the spatial definition of the raw domain video image based on the gray level difference value of the image edge part.
4. The method for evaluating the quality of hardware-in-the-loop video data according to claim 1, wherein the step of comparing the raw domain video image with the lossless stored video image further comprises: performing feature contrast on the restored video image and the lossless storage video image in an image frequency domain, wherein the method specifically comprises the following steps:
converting YUV domain images corresponding to the raw domain video image and the lossless storage video image from a space domain to a frequency domain respectively;
and acquiring high-frequency components of the image frequency chart and comparing and analyzing to evaluate the high-frequency characteristics of the raw domain video image.
5. The method for evaluating quality of hardware-in-the-loop video data according to any one of claims 1 to 4, wherein after the step of comparing the raw domain video image with the lossless storage video image based on the lossless storage video image to evaluate the data quality of the restored raw domain video image, further comprising: and comparing and identifying the raw domain video image with the lossless storage video image through a preset perception algorithm, and determining the target detection rate and the data precision of the raw domain video image so as to verify the data quality of the injected raw domain video image.
6. The method of claim 1, wherein the step of injecting the compressed stored video data into an intelligent driving domain controller comprises:
converting the compressed storage video image into HDMI image data frame by frame and sending the HDMI image data to a video injection board card;
converting the HDMI image data into high definition image data by an image interpolation method;
the video injection board converts the high-definition image data into camera format data and injects the camera format data into the intelligent driving domain controller of the hardware-in-the-loop;
and the intelligent driving domain controller restores the camera format data into the raw domain video image.
7. The method of claim 6, wherein the step of injecting the compressed stored video image into an intelligent driving domain controller further comprises:
transmitting the camera format data to the intelligent driving domain controller in a serial manner;
the intelligent driving domain controller deserializes the camera format data transmitted in series to restore to the raw domain video image.
8. A quality assessment apparatus for hardware-in-the-loop video data, comprising:
the video data acquisition module is used for acquiring a plurality of video data under different environments and different scenes, wherein the video data comprises compressed storage video images and lossless storage video images;
the video image restoration module is used for injecting the compressed storage video image into the intelligent driving domain controller and restoring the compressed storage video image into a raw domain video image by using an image interpolation method;
and the data quality evaluation module is used for comparing the raw domain video image with the lossless storage video image by taking the lossless storage video image as a reference so as to evaluate the data quality of the restored raw domain video image.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements a method for quality assessment of hardware-in-loop video data as claimed in any one of claims 1 to 7 when the program is executed by the processor.
10. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the method of quality assessment of hardware-in-loop video data according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310718576.0A CN116915971A (en) | 2023-06-16 | 2023-06-16 | Quality evaluation method, device, equipment and medium for hardware-in-loop video data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310718576.0A CN116915971A (en) | 2023-06-16 | 2023-06-16 | Quality evaluation method, device, equipment and medium for hardware-in-loop video data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116915971A true CN116915971A (en) | 2023-10-20 |
Family
ID=88350135
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310718576.0A Pending CN116915971A (en) | 2023-06-16 | 2023-06-16 | Quality evaluation method, device, equipment and medium for hardware-in-loop video data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116915971A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117408872A (en) * | 2023-12-14 | 2024-01-16 | 深圳市艾利光科技有限公司 | Color image data conversion method, device, equipment and storage medium |
-
2023
- 2023-06-16 CN CN202310718576.0A patent/CN116915971A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117408872A (en) * | 2023-12-14 | 2024-01-16 | 深圳市艾利光科技有限公司 | Color image data conversion method, device, equipment and storage medium |
| CN117408872B (en) * | 2023-12-14 | 2024-04-05 | 深圳市艾利光科技有限公司 | Color image data conversion method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113518185B (en) | Video conversion processing method and device, computer readable medium and electronic equipment | |
| CN100389437C (en) | Method and apparatus for modeling film grain patterns in the frequency domain | |
| CN108074241B (en) | Quality scoring method and device for target image, terminal and storage medium | |
| CN111489346B (en) | Full-reference image quality evaluation method and system | |
| CN110944200B (en) | Method for evaluating immersive video transcoding scheme | |
| CN111696078B (en) | Ultra-high definition video detection method and system | |
| CN108933935A (en) | Detection method, device, storage medium and the computer equipment of video communication system | |
| CN107801093B (en) | Video rendering method and device, computer equipment and readable storage medium | |
| CN101146226A (en) | A highly-clear video image quality evaluation method and device based on self-adapted ST area | |
| CN116915971A (en) | Quality evaluation method, device, equipment and medium for hardware-in-loop video data | |
| Chen et al. | Blind quality index for tone-mapped images based on luminance partition | |
| CN103024328B (en) | A kind of method improving screenshot quality of digital video recorder | |
| CN113297937B (en) | Image processing method, device, equipment and medium | |
| CN103780895A (en) | Stereoscopic video quality evaluation method | |
| CN110717892A (en) | Tone mapping image quality evaluation method | |
| CN113115112B (en) | Verification platform and verification method | |
| CN112818958A (en) | Action recognition method, device and storage medium | |
| CN111754412B (en) | Method and device for constructing data pair and terminal equipment | |
| US8750648B2 (en) | Image processing apparatus and method of processing image | |
| WO2023282425A3 (en) | Electronic device, system, and method for intelligent horizontal-vertical image transform | |
| TWI497981B (en) | Method and apparatus for testing video signals | |
| CN114067006A (en) | Screen content image quality evaluation method based on discrete cosine transform | |
| CN113378923A (en) | Image generation device acquisition method and image generation device | |
| CN108171704B (en) | No-reference image quality evaluation method based on excitation response | |
| CN113542864A (en) | Video flower screen area detection method, device, equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |