CN116012365A - Method for determining display faults of intelligent cabins and fault detection device - Google Patents

Abstract

The present invention relates to the field of display fault detection technology, and more particularly, to a method for determining a display fault of an intelligent cockpit, a fault detection apparatus, an intelligent cockpit test stand, and a computer storage medium. The method comprises the following steps: A. receiving a real-time image acquired by a camera, wherein the real-time image comprises raw display images of one or more displays within a smart cockpit test rig; B. simultaneously starting a plurality of image processing algorithms through multithreading to perform fault identification on each original display image so as to determine whether each display has a display fault and the type of the display fault, wherein each image processing algorithm in the plurality of image processing algorithms is used for identifying different types of display faults; and C, determining whether to store the real-time image based on a fault identification result.

Description

Method and fault detection device for determining display fault of smart cockpit

Technical Field

The present invention relates to the technical field of display fault detection, and more particularly to a method for determining a display fault of an intelligent cockpit, a fault detection device, an intelligent cockpit test bench and a computer storage medium.

Background Art

With the development of automotive smart cockpit technology, smart cockpits are gradually becoming the carrier of user experience. Since the functions of smart cockpits are large and complex, it is usually necessary to control the rendering of multiple screens in the car at the same time, so there is a possibility of display failures (for example, black screens). According to the probability of occurrence, display failures can be divided into inevitable problems and occasional problems. Among them, occasional problems have uncertain timing, are easy to miss the first scene after they occur, and even after they are repaired, it is difficult to accurately judge the effectiveness of the repair. Therefore, it is necessary to verify the display failure through automated stress testing. However, the currently disclosed automated stress testing methods are difficult to cover a variety of display problems while ensuring real-time performance.

Summary of the invention

In order to solve or at least alleviate one or more of the above problems, the following technical solutions are provided. Embodiments of the present invention provide a method for determining display faults of a smart cockpit, a fault detection device, a smart cockpit test bench, and a computer storage medium, which can process images captured by a camera in real time during automated testing of the smart cockpit to accurately locate a variety of display faults, and retain the first scene in time, providing important data analysis support for further diagnosis.

According to a first aspect of the present invention, a method for determining a display fault in a smart cockpit is provided, comprising: A. receiving a real-time image captured by a camera, wherein the real-time image comprises an original display image of one or more displays in a smart cockpit test bench; B. simultaneously starting multiple image processing algorithms through multi-threaded calls to perform fault identification on each original display image to determine whether there is a display fault on each display and the type of display fault, wherein each of the multiple image processing algorithms is used to identify different types of display faults; and C. determining whether to store the real-time image based on the fault identification result.

As an alternative or supplement to the above scheme, in a method according to an embodiment of the present invention, step A further includes one or more of the following items: waking up the smart cockpit test bench in response to receiving a wake-up signal; loading a reference image as a reference benchmark for the one or more displays; denoising the real-time image using a Gaussian low-pass filtering algorithm; and extracting the original display image of one or more displays from the real-time image using an image capture algorithm.

As an alternative or supplement to the above scheme, in a method according to an embodiment of the present invention, the multiple image processing algorithms include one or more of the following: a first image processing algorithm for identifying black screen type faults, a second image processing algorithm for identifying snow screen type faults, a third image processing algorithm for identifying screen freezing type faults, and a fourth image processing algorithm for identifying screen jitter type faults.

As an alternative or supplement to the above scheme, in a method according to an embodiment of the present invention, the multiple image processing algorithms include a first image processing algorithm for identifying black screen type faults, and step B includes: performing grayscale processing on the original display image using a grayscale processing function; binarizing the grayscale processed image using a binarization function; counting the number of non-zero pixels in the binarized image using a non-zero counting function; comparing the number of non-zero pixels with a first threshold, and determining whether the display corresponding to the original display image has a black screen type fault based on the comparison result.

As an alternative or supplement to the above scheme, in a method according to an embodiment of the present invention, the multiple image processing algorithms include a second image processing algorithm for identifying snow screen type faults, and step B includes: using a feature extraction function to perform feature point detection on the original display image and the benchmark image used as a reference benchmark and calculate their feature descriptors; using an image matching function to match the feature descriptors of the original display image with the feature descriptors of the benchmark image to generate matched feature point pairs; calculating the number of feature point pairs in the matched feature point pairs whose feature point similarity is greater than or equal to a second threshold; comparing the number with a third threshold, and determining whether the display corresponding to the original display image has a snow screen type fault based on the comparison result.

As an alternative or supplement to the above scheme, in a method according to one embodiment of the present invention, step A includes changing the CAN bus signal input to the smart cockpit test bench after loading a benchmark image as a reference benchmark, and receiving the real-time image captured by the camera after a first time period, and the multiple image processing algorithms include a third image processing algorithm for identifying screen jamming type faults.

As an alternative or supplement to the above scheme, in a method according to an embodiment of the present invention, step B includes: based on the number of feature point pairs between the original display image and the reference image whose similarity is greater than or equal to a fourth threshold, judging whether the display corresponding to the original display image has a risk of jamming; if it is determined that there is a risk of jamming, grayscale processing and binarization processing are performed on the original display image and the reference image, and the number of non-zero pixels in the grayscale and binarization processed images is counted; if the absolute value of the difference between the number of non-zero pixels of the original display image and the reference image is greater than or equal to a fifth threshold, determining that the display has a screen jamming type fault.

As an alternative or supplement to the above scheme, in a method according to an embodiment of the present invention, the multiple image processing algorithms include a fourth image processing algorithm for identifying screen jitter type faults, and step B includes: storing the original display image acquired during the second time period and performing frame sorting on it; performing jitter comparison on the original display images with adjacent frame numbers, and obtaining the total number of frames of the jittered images; comparing the total number of frames with a sixth threshold, and determining whether the display corresponding to the original display image has a screen jitter type fault based on the comparison result.

As an alternative or supplement to the above solution, in a method according to an embodiment of the present invention, step B further includes: monitoring the running status of each image processing thread in real time; and adjusting the priority of the multiple image processing threads based on the running status.

As an alternative or supplement to the above scheme, in a method according to an embodiment of the present invention, step C includes: if it is determined that one or more of the displays have a display fault, storing the real-time image; if it is determined that the display does not have a display fault, controlling the smart cockpit test bench to power off, and sending a wake-up signal to the smart cockpit test bench after a preset time interval.

According to a second aspect of the present invention, there is provided a fault detection device, comprising: a memory configured to store instructions; and a processor configured to execute the instructions to perform any one of the methods according to the first aspect of the present invention.

According to a third aspect of the present invention, there is provided an intelligent cockpit test bench, comprising: a camera for collecting original display images of one or more displays in the intelligent cockpit test bench; and any one of the fault detection devices described in the second aspect of the present invention.

According to a fourth aspect of the present invention, there is provided a computer storage medium, the computer storage medium comprising instructions, the instructions executing any one of the methods according to the first aspect of the present invention when run.

According to one or more embodiments of the present invention, the solution for determining the display fault of the smart cockpit collects the original display images of each display in real time and identifies the fault of each original display image by starting multiple image processing algorithms, so that while ensuring the real-time performance of image recognition and processing, various display faults can be accurately located and the first scene can be retained in time, thereby providing important data analysis support for further diagnosis. In addition, according to one or more embodiments of the present invention, the solution for determining the display fault of the smart cockpit starts multiple image processing algorithms simultaneously through multi-threaded calls, avoiding the inefficiency of single-threaded processing, and can identify and locate various display faults in a targeted manner, thereby improving the recognition accuracy and repair efficiency of the display fault of the smart cockpit.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and/or other aspects and advantages of the present invention will become clearer and easier to understand through the following description of various aspects in conjunction with the accompanying drawings, in which the same or similar units are represented by the same reference numerals. In the accompanying drawings:

FIG. 1 is a schematic flow chart of a method 10 for determining a display failure of a smart cockpit according to one or more embodiments of the present invention; and

FIG. 2 is a schematic flow chart of a method 20 for determining a display fault of a smart cockpit according to one or more embodiments of the present invention.

DETAILED DESCRIPTION

The description of the following specific embodiments is merely exemplary in nature and is not intended to limit the disclosed technology or the application and use of the disclosed technology. In addition, it is not intended to be bound by any express or implied theory presented in the aforementioned technical field, background technology or the following specific embodiments.

In the following detailed description of the embodiments, many specific details are set forth in order to provide a more thorough understanding of the disclosed technology. However, it is apparent to one of ordinary skill in the art that the disclosed technology can be practiced without these specific details. In other instances, well-known features are not described in detail to avoid unnecessarily complicating the description.

Terms such as "comprising" and "including" indicate that in addition to the units and steps directly and explicitly stated in the specification, the technical solution of the present invention does not exclude the situation of having other units and steps that are not directly or explicitly stated. Terms such as "first" and "second" do not indicate the order of units in terms of time, space, size, etc., but are only used to distinguish the units. In this specification, the term "vehicle" or other similar terms include general motor vehicles, such as passenger cars (including sports utility vehicles, buses, trucks, etc.), various commercial vehicles, etc., and include hybrid vehicles, electric vehicles, plug-in hybrid electric vehicles, etc. A hybrid vehicle is a vehicle with two or more power sources, such as gasoline-powered and electric vehicles.

Hereinafter, exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

FIG1 is a schematic flow chart of a method 10 for determining a display fault of a smart cockpit according to one or more embodiments of the present invention. It should be noted that the step names mentioned above (and those mentioned below) are only used to distinguish between the steps and facilitate the reference of the steps, and do not represent the order relationship between the steps. The flowchart including the attached figure is only an example of executing the present method. In the absence of obvious conflicts, the steps can be executed in various orders or simultaneously.

As shown in FIG. 1 , in step S110 , a real-time image captured by a camera is received, wherein the real-time image includes an original display image of one or more displays in the smart cockpit test bench.

As the functions of the smart cockpit become richer, one or more displays may be arranged in the smart cockpit, such as the dashboard in front of the driver, the head-up display (HUD) and the center console screen, and the entertainment screen in front of the co-pilot and the back row. The smart cockpit host is connected to various buses and information transmission components on the vehicle, completes the reception, transmission and preprocessing of various communication signals (for example, controller area network (CAN) bus signals), and renders the display images of one or more displays based on the above communication signals. It is understandable that one or more displays are correspondingly arranged in the test bench for functional testing of the smart cockpit, and a camera (for example, an industrial camera) is also provided to capture the original display images of one or more displays. It should be understood that in the embodiments of the present invention, cameras, video cameras, cameras, etc. all represent devices that can obtain images or images within the coverage range, and their meanings are similar and interchangeable, and the present invention does not limit this.

Exemplarily, method 10 is implemented based on OpenCV. OpenCV is an open source software library that can run on a variety of operating systems and supports Python language interface. It has the advantages of lightweight and efficient functions and provides support for a large number of image recognition library functions. The intelligent cockpit test bench uses Python scripts to automatically control the operation, and Python implements the call to the library function through the compatible interface cv2 of OpenCV. Exemplarily, the following core library functions can be called through OpenCV, for example, Gaussian filtering function gaussian(), image capture function VideoCapture(), image loading function imread(), grayscale processing function cvColor(), binarization function threshold(), non-zero count function countNonZero(), object creation function xfeatures2d.SIFT_create(), feature extraction function detectAndCompute(), image matching function FlannBasedMatcher() and knnMatch(), etc.

Optionally, in method 10, a power-on cycle interval is set for the smart cockpit test bench, for example, a wake-up signal is sent to the smart cockpit test bench every 5 minutes. In response to receiving the wake-up signal, the smart cockpit test bench turns on one or more displays and cameras in the smart cockpit test bench. Optionally, after the smart cockpit test bench is powered on, an image capture algorithm is used (for example, by calling the image capture function VideoCapture() in the OpenCV library function) to capture the original display image of one or more displays from the real-time image, and/or a reference image is loaded as a reference for one or more displays (for example, by calling the image capture function VideoCapture() in the OpenCV library function).

Optionally, after receiving the real-time image captured by the camera, the real-time image can also be denoised using a Gaussian low-pass filtering algorithm. Exemplarily, the real-time image is first converted from the spatial domain to the frequency domain by a two-dimensional discrete Fourier transform F(u, v) as shown in the following formula to obtain pixels (u, v) on the two-dimensional spectrum:

Wherein, x and y are the horizontal and vertical coordinates on the real-time image, respectively; M and N are the height and width of the real-time image; and f(x, y) is the grayscale value at the coordinate point (x, y) on the real-time image.

Next, a low-pass filtering operation is performed on the two-dimensional spectrum using a Gaussian low-pass filter H(u, v) as shown in the following equation:

Where D(u, v) is the distance from the pixel (u, v) to the center of the spectrum on the two-dimensional spectrum.

Finally, a two-dimensional discrete Fourier inverse transform is performed on the filtered spectrum to obtain the filtered image f(x, y) as shown below:

As shown in FIG. 1 , in step S120 , multiple image processing algorithms are simultaneously started through multi-threaded calls to perform fault identification on each original display image to determine whether there is a display fault on each display and the type of display fault, wherein each of the multiple image processing algorithms is used to identify different types of display faults.

Multithreaded calls refer to the technology of implementing concurrent execution of multiple threads from software or hardware through program calls. Devices with multithreading capabilities can execute multiple threads at the same time due to hardware support, thereby improving overall processing performance. By simultaneously starting multiple image processing algorithms through multithreaded calls, it is possible to identify and locate multiple display faults in a targeted manner while improving fault identification accuracy and repair efficiency, for example, identifying which display has which type of display fault.

Optionally, the plurality of image processing algorithms include one or more of the following: a first image processing algorithm for identifying black screen faults, a second image processing algorithm for identifying snow screen faults, a third image processing algorithm for identifying screen stuck faults, and a fourth image processing algorithm for identifying screen shaking faults. That is, for the original display image shown by each display, black screen faults, snow screen faults, screen stuck faults, and screen shaking faults can be identified and located at the same time through multi-threaded calls.

Optionally, in an embodiment where the plurality of image processing algorithms include a first image processing algorithm for identifying a black screen fault, step S120 includes: grayscale processing the original display image using a grayscale processing function; binarization processing the grayscale processed image using a binarization function; counting the number of non-zero pixels in the binarized image using a non-zero counting function; comparing the number of non-zero pixels with a first threshold, and determining whether the display corresponding to the original display image has a black screen fault based on the comparison result. Exemplarily, grayscale processing and binarization threshold processing are performed on the original display image of each display by calling the grayscale processing function cvColor() and the binarization function threshold() in the OpenCV library function, for example, if the grayscale value of the pixel is greater than or equal to 127, the pixel is processed as a white point (that is, the grayscale value of the pixel is set to 255), and if the grayscale value of the pixel is less than 127, the pixel is processed as a black point (that is, the grayscale value of the pixel is set to 0). Next, the number of non-zero pixels (i.e., white dots) in the binary image is counted by calling the non-zero counting function countNonZero() in the OpenCV library function. If the total number of non-zero pixels is greater than or equal to the preset first threshold, it is determined that the original display image is normally lit (i.e., the display does not have a black screen failure). Otherwise, it is determined that the original display image is not normally lit (i.e., the display has a black screen failure). Optionally, although most laboratories where the smart cockpit test bench is located use light sources with stable brightness, there may still be certain brightness fluctuations at different time periods in a day. Therefore, the brightness of the laboratory environment can be calibrated in advance to determine the first threshold for different time periods.

Optionally, in an embodiment where the plurality of image processing algorithms include a second image processing algorithm for identifying snow screen faults, step S120 includes: using a feature extraction function to perform feature point detection on the original display image and the reference image as a reference respectively and calculating their feature descriptors; using an image matching function to match the feature descriptors of the original display image and the feature descriptors of the reference image to generate matched feature point pairs; calculating the number of feature point pairs whose feature point similarity is greater than or equal to a second threshold in the matched feature point pairs; comparing the number with a third threshold, and determining whether the display corresponding to the original display image has a snow screen fault based on the comparison result. Exemplarily, an object is created by calling the object creation function xfeatures2d.SIFT_create() in the OpenCV library function, and feature points (e.g., contour corners, vertices) of the original display image and the reference image are extracted respectively by calling the feature extraction function detectAndCompute() and calculating their feature descriptors. A feature descriptor is a representation of an image (e.g., it can represent a local image gradient calculated at a selected ratio in an area around each feature point), and the commonality of the two images can be found by comparing the feature descriptors of the two images. Then, the feature descriptors of the two images are matched by calling the image matching functions FlannBasedMatcher() and knnMatch() in the OpenCV library function to obtain matched feature point pairs. Next, similarity screening is performed in the matched feature point pairs, that is, feature point pairs with similarity greater than or equal to the second threshold are retained and their number is counted. If the number is greater than or equal to the preset third threshold, it is determined that the display does not have a snow screen fault, otherwise, it is determined that the display has a snow screen fault.

Optionally, in an embodiment where the plurality of image processing algorithms include a second image processing algorithm for identifying snow screen type faults, step S110 includes: after loading a reference image as a reference benchmark, changing the CAN bus signal input to the smart cockpit test bench, and receiving a real-time image captured by the camera after a first time period. Exemplarily, the normally displayed reference image is first loaded by calling the image loading function imread() in the OpenCV library function, and then the changes in the CAN bus signal, such as vehicle speed, gear position, etc., are input to the test bench, and the original display image of the display is captured and loaded after the first time period. The two pictures can also be partially cut out (for example, by calling the image capture function VideoCapture()) to extract the part of interest.

In addition, optionally, in an embodiment where the plurality of image processing algorithms include a third image processing algorithm for identifying a screen stuck fault, step S120 includes: judging whether the display corresponding to the original display image has a stuck risk based on the number of feature point pairs whose similarity is greater than or equal to a fourth threshold value in the feature point pairs of the original display image and the reference image; if it is determined that there is a stuck risk, grayscale processing and binarization processing are performed on the original display image and the reference image, and the number of non-zero pixels in the grayscale and binarization processed images is counted; if the absolute value of the difference between the number of non-zero pixels of the original display image and the reference image is greater than or equal to a fifth threshold value, it is determined that the display has a screen stuck fault. Exemplarily, feature point detection, feature descriptor calculation, feature point pair matching, and similarity screening operations are performed on the reference image and the original display image respectively acquired before and after the CAN bus signal input to the intelligent cockpit test bench is changed, and the above operations have the same or similar steps as feature point detection, feature descriptor calculation, feature point pair matching, and similarity screening in the identification of snow screen faults, which are not repeated here. Then, the number counted in the similarity screening is compared with the fourth threshold value. If the number is greater than or equal to the preset fourth threshold value, it is determined that the display has a risk of jamming. Otherwise, it is determined that the display does not have a risk of jamming (that is, no screen jamming fault occurs). If it is determined that the display has a risk of jamming, in order to avoid misjudgment of the scene, next, the grayscale processing function cvColor() and the binarization function threshold() in the OpenCV library function are called to perform grayscale processing and binarization threshold processing on the original display image and the reference image, and the number of non-zero pixels in the two binarized images is counted by calling the non-zero counting function countNonZero() in the OpenCV library function. If the absolute value of the difference between the number of non-zero pixels in the original display image and the reference image is greater than or equal to the fifth threshold value, it is determined that the display has a screen jamming fault. Otherwise, it is determined that the display does not have a screen jamming fault.

Optionally, in an embodiment where the plurality of image processing algorithms include a fourth image processing algorithm for identifying screen jitter type faults, step S120 includes: storing the original display image acquired during the second time period and performing frame sorting on it; performing jitter comparison on the original display image with adjacent frame numbers, and obtaining the total number of frames of the jitter image; comparing the total number of frames with the sixth threshold, and determining whether the display corresponding to the original display image has a screen jitter type fault based on the comparison result. Exemplarily, in an embodiment where the real-time video is captured by a camera, the video acquired during the second time period after the camera is powered on can be directly stored and frame sorted. It should be noted that it is necessary to ensure that there is no change in the CAN bus signal input during the second time period, and thus no display update occurs. Next, for each frame image, the frame image is compared with the adjacent frame image (for example, the previous frame, the next frame) for jitter, and the total number of frames of the jitter image (for example, the jitter value between adjacent frames is large) is counted. If the total number of frames is greater than or equal to the sixth threshold, it is determined that the display has a screen jitter type fault, otherwise it is determined that there is no screen jitter type fault.

Optionally, step S120 further includes: monitoring the running state of each image processing thread in real time; and adjusting the priority of multiple image processing threads based on the running state. It is understandable that in order to prevent thread conflicts, the priority of multiple threads can be pre-set, for example, a first thread for executing a first image processing algorithm, a second thread for executing a second image processing algorithm, a third thread for executing a third image processing algorithm, and a fourth thread for executing a fourth image processing algorithm. Exemplarily, the priority can be set according to the severity of each display fault, for example, the priority of the first thread for a black screen fault is higher than that of other threads. However, thread starvation is prone to occur under priority scheduling, that is, before executing a thread with a lower priority, there is always a thread with a higher priority waiting to be executed, so this low-priority thread is never executed. In order to avoid thread starvation, thread priority can be dynamically set, that is, based on the real-time running state of each thread, its priority is dynamically adjusted, for example, according to the frequency of the thread entering the waiting state to adjust its priority.

In step S130, based on the fault identification result, it is determined whether to store the real-time image. Optionally, if it is determined that one or more of the displays have a display fault, the real-time image is stored; if it is determined that the display does not have a display fault, the smart cockpit test bench is controlled to power off, and a wake-up signal for the smart cockpit test bench is sent after a preset time interval. Exemplarily, if it is determined in step S120 that any type of display fault (such as a black screen fault, a snow screen fault, a screen stuck fault, a screen jitter fault) occurs on a display, in order to retain the first scene to provide data analysis support for further diagnosis, the real-time image collected by the camera during the power-on cycle is stored. If no display has a display fault, a sleep signal is sent to the smart cockpit test bench, and a wake-up signal is sent to the smart cockpit test bench after a preset power-on cycle interval (for example, 5 minutes) to restart the execution of steps S110-S130.

According to one or more embodiments of the present invention, the method 10 collects the original display images of each display in real time and identifies the faults of each original display image by starting multiple image processing algorithms, so that it can accurately locate various display faults and retain the first scene in time while ensuring the real-time performance of image recognition and processing, thereby providing important data analysis support for further diagnosis. In addition, the method 10 simultaneously starts multiple image processing algorithms through multi-threaded calls, avoiding the low efficiency of single-threaded processing, and can identify and locate various display faults in a targeted manner, thereby improving the recognition accuracy and repair efficiency of smart cockpit display faults.

In order to more clearly illustrate the principles of the present application, FIG2 shows in a more complete form a method 20 for determining a display failure of a smart cockpit. It should be understood that the example of FIG2 should not be regarded as constituting additional limitations on other examples in this document (for example, the embodiment corresponding to FIG1 ).

In step S210, in response to receiving a wake-up signal, the smart cockpit test bench is awakened, and then one or more displays and cameras in the smart cockpit test bench are turned on. In step S220, a reference image is loaded as a reference benchmark for one or more displays. In step S230, the main thread is started, a real-time image captured by the camera is received (wherein the real-time image includes the original display image of one or more displays in the smart cockpit test bench) and the original display image of one or more displays is intercepted from the real-time image using an image capture algorithm. In step S240, the original display image of one or more displays is loaded. In step S250, the real-time image is denoised using a Gaussian low-pass filtering algorithm. In step S260, multiple sub-threads are started at the same time, wherein the first sub-thread includes executing a first image processing algorithm for identifying black screen faults, the second sub-thread includes executing a second image processing algorithm for identifying snow screen faults, the third sub-thread includes executing a third image processing algorithm for identifying screen stuck faults, the fourth sub-thread includes executing a fourth image processing algorithm for identifying screen jitter faults, and the fifth sub-thread includes real-time monitoring of the running state of each image processing thread and adjusting the priority of multiple image processing threads based on the running state. In step S270, it is determined whether the display has a display fault based on the execution results of the first sub-thread to the fourth sub-thread. If so, the method 20 proceeds to step S280, otherwise the method 20 proceeds to step S290. In step S280, the program stops the loop and stores the real-time image so that the test bench maintains the fault site. In step S290, the timer is started to send a wake-up signal to the smart cockpit test bench after a preset power-on cycle interval to restart the execution of steps S210-S290.

According to another aspect of the present invention, a fault detection device is provided, comprising: a memory configured to store instructions; and a processor configured to execute the instructions to perform the method 10 shown in FIG. 1 or the method 20 shown in FIG. 2 .

According to another aspect of the present invention, there is provided a smart cockpit test bench, comprising: a camera for collecting original display images of one or more displays in the smart cockpit test bench; and a fault detection device according to one aspect of the present invention.

In addition, the present invention can also be implemented as a computer storage medium, in which a computer performs method 10 shown in FIG1 or method 20 shown in FIG2 is stored. Here, as the computer storage medium, various computer storage media such as disks (e.g., magnetic disks, optical disks, etc.), cards (e.g., memory cards, optical cards, etc.), semiconductor memories (e.g., ROMs, nonvolatile memories, etc.), and tapes (e.g., magnetic tapes, cassette tapes, etc.) can be used.

In applicable situations, hardware, software or a combination of hardware and software can be used to realize the various embodiments provided by the present invention. Moreover, in applicable situations, without departing from the scope of the present invention, the various hardware components and/or software components set forth herein can be combined into composite components comprising software, hardware and/or both. In applicable situations, without departing from the scope of the present invention, the various hardware components and/or software components set forth herein can be divided into subcomponents comprising software, hardware or both. In addition, in applicable situations, it is contemplated that software components can be implemented as hardware components, and vice versa.

Software according to the present invention (such as program code and/or data) can be stored on one or more computer storage media. It is also contemplated that the software identified herein can be implemented using one or more general or special computers and/or computer systems that are networked and/or otherwise. The embodiments and examples proposed herein are provided to best illustrate the embodiments according to the present invention and its specific applications, and thus enable those skilled in the art to implement and use the present invention. However, those skilled in the art will appreciate that the above description and examples are provided only for ease of explanation and example. The proposed description is not intended to cover all aspects of the present invention or to limit the present invention to the disclosed precise form.

Claims (13)

1. A method for determining a display failure of a smart cockpit, characterized in that the method comprises the following steps: A. receiving a real-time image captured by a camera, wherein the real-time image includes an original display image of one or more displays in the smart cockpit test bench; B. performing fault identification on each original display image by simultaneously starting multiple image processing algorithms through multi-threaded calls to determine whether each display has a display fault and the type of the display fault, wherein each of the multiple image processing algorithms is used to identify different types of display faults; and C. Based on the fault identification result, determine whether to store the real-time image. 2. The method according to claim 1, wherein step A further comprises one or more of the following: In response to receiving a wake-up signal, waking up the smart cockpit test bench; loading a reference image as a reference for the one or more displays; Performing denoising on the real-time image using a Gaussian low-pass filtering algorithm; and An image capture algorithm is used to extract original display images of one or more displays from the real-time image. 3. The method according to claim 1, wherein the multiple image processing algorithms include one or more of the following: a first image processing algorithm for identifying black screen type faults, a second image processing algorithm for identifying snow screen type faults, a third image processing algorithm for identifying screen freezing type faults, and a fourth image processing algorithm for identifying screen shaking type faults. 4. The method according to claim 1, wherein the plurality of image processing algorithms include a first image processing algorithm for identifying black screen type faults, and step B includes: Performing grayscale processing on the original display image using a grayscale processing function; The grayscale processed image is binarized using a binarization function; The number of non-zero pixels in the binary image is counted using a non-zero counting function; The number of the non-zero pixels is compared with a first threshold, and based on the comparison result, it is determined whether a display corresponding to the original display image has a black screen fault. 5. The method according to claim 1, wherein the plurality of image processing algorithms include a second image processing algorithm for identifying snow screen type faults, and step B comprises: Using a feature extraction function, the original display image and the reference image as a reference benchmark are respectively subjected to feature point detection and feature descriptors are calculated; Matching the feature descriptor of the original display image with the feature descriptor of the reference image using a picture matching function to generate matched feature point pairs; Calculate the number of feature point pairs whose feature point similarity is greater than or equal to a second threshold value among the matched feature point pairs; The number is compared with a third threshold, and based on the comparison result, it is determined whether the display corresponding to the original display image has a snow screen type fault. 6. The method according to claim 1, wherein step A includes changing the CAN bus signal input to the smart cockpit test bench after loading a benchmark image as a reference benchmark, and receiving the real-time image captured by the camera after a first time period, and the multiple image processing algorithms include a third image processing algorithm for identifying screen stuck type faults. 7. The method according to claim 6, wherein step B comprises: Based on the number of feature point pairs whose similarities between the feature point pairs of the original display image and the reference image are greater than or equal to a fourth threshold, determining whether a display corresponding to the original display image has a risk of getting stuck; If it is determined that there is a risk of jamming, grayscale processing and binarization processing are performed on the original display image and the reference image, and the number of non-zero pixels in the grayscale and binarization processed images is counted; If the absolute value of the difference between the number of non-zero pixels of the original display image and the number of non-zero pixels of the reference image is greater than or equal to a fifth threshold, it is determined that a screen lag fault exists in the display. 8. The method according to claim 1, wherein the plurality of image processing algorithms include a fourth image processing algorithm for identifying screen jitter type faults, and step B includes: storing and frame-sorting raw display images acquired during a second time period; Performing jitter comparison on original display images with adjacent frame numbers, and obtaining the total number of frames of the jittered images; The total number of frames is compared with a sixth threshold, and based on the comparison result, it is determined whether a display corresponding to the original display image has a screen jitter fault. 9. The method according to claim 1, wherein step B further comprises: Monitor the running status of each image processing thread in real time; and The priorities of the plurality of image processing threads are adjusted based on the running status. 10. The method according to claim 1, wherein step C comprises: If it is determined that one or more of the displays have a display failure, storing the real-time image; If it is determined that there is no display fault on the display, the smart cockpit test bench is controlled to be powered off, and a wake-up signal for the smart cockpit test bench is sent after a preset time interval. 11. A fault detection device, comprising: a memory configured to store instructions; and A processor configured to execute the instructions to perform the method according to any one of claims 1-10. 12. An intelligent cockpit test bench, characterized by comprising: A camera, used to capture raw display images of one or more displays in the smart cockpit test bench; and The fault detection device as claimed in claim 11. 13. A computer storage medium, characterized in that the computer storage medium comprises instructions, and the instructions execute the method according to any one of claims 1 to 10 when executed.

Images