CN116012365A - Method for determining display faults of intelligent cabins and fault detection device - Google Patents
Method for determining display faults of intelligent cabins and fault detection device Download PDFInfo
- Publication number
- CN116012365A CN116012365A CN202310102059.0A CN202310102059A CN116012365A CN 116012365 A CN116012365 A CN 116012365A CN 202310102059 A CN202310102059 A CN 202310102059A CN 116012365 A CN116012365 A CN 116012365A
- Authority
- CN
- China
- Prior art keywords
- image
- display
- fault
- image processing
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000001514 detection method Methods 0.000 title claims abstract description 18
- 238000012545 processing Methods 0.000 claims abstract description 105
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 74
- 238000012360 testing method Methods 0.000 claims abstract description 45
- 230000006870 function Effects 0.000 claims description 47
- 241000533950 Leucojum Species 0.000 claims description 16
- 230000015654 memory Effects 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 4
- 238000012544 monitoring process Methods 0.000 claims description 4
- 230000004044 response Effects 0.000 claims description 4
- 238000005516 engineering process Methods 0.000 abstract description 6
- 230000008439 repair process Effects 0.000 description 5
- 238000001228 spectrum Methods 0.000 description 5
- 238000007405 data analysis Methods 0.000 description 4
- 238000012216 screening Methods 0.000 description 4
- 238000003745 diagnosis Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 235000003642 hunger Nutrition 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000037351 starvation Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 239000000969 carrier Substances 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Landscapes
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
Abstract
The present invention relates to the field of display fault detection technology, and more particularly, to a method for determining a display fault of an intelligent cockpit, a fault detection apparatus, an intelligent cockpit test stand, and a computer storage medium. The method comprises the following steps: A. receiving a real-time image acquired by a camera, wherein the real-time image comprises raw display images of one or more displays within a smart cockpit test rig; B. simultaneously starting a plurality of image processing algorithms through multithreading to perform fault identification on each original display image so as to determine whether each display has a display fault and the type of the display fault, wherein each image processing algorithm in the plurality of image processing algorithms is used for identifying different types of display faults; and C, determining whether to store the real-time image based on a fault identification result.
Description
Technical Field
The present invention relates to the field of display fault detection technology, and more particularly, to a method for determining a display fault of an intelligent cockpit, a fault detection apparatus, an intelligent cockpit test stand, and a computer storage medium.
Background
With the development of automobile intelligent cabin technology, intelligent cabins are gradually becoming carriers of user experience. Because of the large and complex functionality of the intelligent cockpit, it is often necessary to control multiple screen rendering within the vehicle simultaneously, and there is a possibility of display failure (e.g., a black screen) occurring. According to the occurrence probability, the display faults can be divided into indispensable problems and sporadic problems, wherein the sporadic problems are difficult to accurately judge the effectiveness of repair even after repair due to the fact that the occurrence time is variable, the first site is easy to miss after occurrence, and therefore the display faults need to be verified through an automatic pressure test. However, the automatic pressure testing method disclosed at present is difficult to cover various display problems on the premise of ensuring real-time performance.
Disclosure of Invention
To solve or at least alleviate one or more of the above problems, the following solutions are provided. The embodiment of the invention provides a method for determining display faults of an intelligent cabin, a fault detection device, an intelligent cabin test bench and a computer storage medium, which can process images acquired by a camera in real time in an automatic test for the intelligent cabin so as to obtain accurate positioning of various display faults, and reserve a first site in time, thereby providing important data analysis support for further diagnosis.
According to a first aspect of the invention, there is provided a method for determining a display failure of an intelligent cockpit, comprising: A. receiving a real-time image acquired by a camera, wherein the real-time image comprises raw display images of one or more displays within a smart cockpit test rig; B. simultaneously starting a plurality of image processing algorithms through multithreading to perform fault identification on each original display image so as to determine whether each display has a display fault and the type of the display fault, wherein each image processing algorithm in the plurality of image processing algorithms is used for identifying different types of display faults; and C, determining whether to store the real-time image based on a fault identification result.
Alternatively or additionally to the above, in a method according to an embodiment of the invention, step a further comprises one or more of the following: in response to receiving a wake-up signal, wake-up the intelligent cockpit test bench; loading a reference image as a reference datum for the one or more displays; denoising the real-time image by using a Gaussian low-pass filtering algorithm; and intercepting original display images of one or more displays from the real-time images by using an image capturing algorithm.
Alternatively or additionally to the above, in a method according to an embodiment of the invention, the plurality of image processing algorithms comprises one or more of: the method comprises a first image processing algorithm for identifying black screen type faults, a second image processing algorithm for identifying snowflake screen type faults, a third image processing algorithm for identifying screen clamping type faults and a fourth image processing algorithm for identifying screen dithering type faults.
Alternatively or additionally to the above, in a method according to an embodiment of the present invention, the plurality of image processing algorithms includes a first image processing algorithm for identifying a black screen type fault, and step B includes: carrying out gray processing on the original display image by using a gray processing function; performing binarization processing on the gray-scale processed image by using a binarization function; counting the number of non-zero pixels in the binarized image by using a non-zero count function; and comparing the number of the non-zero pixels with a first threshold value, and determining whether a black screen type fault exists in a display corresponding to the original display image based on a comparison result.
Alternatively or additionally to the above, in a method according to an embodiment of the invention, the plurality of image processing algorithms comprises a second image processing algorithm for identifying a snowflake screen type fault, and step B comprises: respectively carrying out feature point detection and calculating a feature descriptor of the original display image and a reference image serving as a reference standard by utilizing a feature extraction function; matching the feature descriptors of the original display image and the feature descriptors of the reference image by using a picture matching function to generate matched feature point pairs; calculating the number of the feature point pairs with the feature point similarity of the matched feature point pairs being greater than or equal to a second threshold value; and comparing the number with a third threshold value, and determining whether a snowflake screen type fault exists in a display corresponding to the original display image based on a comparison result.
Alternatively or additionally to the above, in a method according to an embodiment of the invention, step a comprises changing a CAN bus signal input to the intelligent cockpit test rig after loading a reference image as a reference datum and receiving the real-time image acquired by the camera after a first period of time, and the plurality of image processing algorithms comprises a third image processing algorithm for identifying a screen-stuck-at-type fault.
Alternatively or additionally to the above, in a method according to an embodiment of the invention, step B comprises: judging whether a display corresponding to the original display image has a clamping stagnation risk or not based on the number of characteristic point pairs with the similarity of the characteristic point pairs of the original display image and the reference image being larger than or equal to a fourth threshold value; if the risk of jamming is judged, gray scale processing and binarization processing are carried out on the original display image and the reference image, and the number of non-zero pixels in the image subjected to gray scale and binarization processing is counted; and if the absolute value of the difference value between the number of the non-zero pixels of the original display image and the reference image is greater than or equal to a fifth threshold value, determining that the display has a screen clamping fault.
Alternatively or additionally to the above, in a method according to an embodiment of the present invention, the plurality of image processing algorithms includes a fourth image processing algorithm for identifying a screen shake type failure, and step B includes: storing and frame ordering the original display images acquired during the second period of time; performing jitter comparison on an original display image with adjacent frame numbers, and acquiring the total frame number of the jittered image; and comparing the total frame number with a sixth threshold value, and determining whether a display corresponding to the original display image has a screen shake type fault or not based on a comparison result.
Alternatively or additionally to the above, in a method according to an embodiment of the invention, step B further comprises: monitoring the running state of each image processing thread in real time; and adjusting priorities of the plurality of image processing threads based on the run state.
Alternatively or additionally to the above, in a method according to an embodiment of the invention, step C comprises: if it is determined that one or more of the displays has a display failure, storing the real-time image; and if the display is determined to have no display fault, controlling the intelligent cabin test bench to be powered down, and sending a wake-up signal aiming at the intelligent cabin test bench after a preset time interval.
According to a second aspect of the present invention, there is provided a fault detection device comprising: a memory configured to store instructions; and a processor configured to execute the instructions to perform any one of the methods according to the first aspect of the invention.
According to a third aspect of the present invention there is provided an intelligent cockpit testing bench comprising: a camera for acquiring raw display images of one or more displays within the intelligent cockpit testing rig; and any one of the fault detection devices according to the second aspect of the present invention.
According to a fourth aspect of the present invention there is provided a computer storage medium comprising instructions which, when executed, perform any of the methods according to the first aspect of the present invention.
The scheme for determining the display faults of the intelligent cabin according to one or more embodiments of the invention enables accurate positioning of multiple display faults and timely reservation of a first site while ensuring the instantaneity of image recognition and processing by acquiring original display images of the displays in real time and performing fault recognition on each original display image by starting a plurality of image processing algorithms, thereby providing important data analysis support for further diagnosis. In addition, according to the scheme for determining the display faults of the intelligent cabin, the multiple image processing algorithms are simultaneously started through multi-thread calling, so that the problem of inefficiency of single-thread processing is avoided, multiple display faults can be identified and positioned in a targeted manner, and therefore the identification accuracy and the repair efficiency of the display faults of the intelligent cabin are improved.
Drawings
The foregoing and/or other aspects and advantages of the present invention will become more apparent and more readily appreciated from the following description of the various aspects taken in conjunction with the accompanying drawings in which like or similar elements are designated with the same reference numerals. In the drawings:
FIG. 1 is a schematic flow diagram of a method 10 for determining a display failure of a smart cockpit in accordance with one or more embodiments of the present invention; and
fig. 2 is a schematic flow diagram of a method 20 for determining a display failure of a smart cockpit in accordance with one or more embodiments of the present invention.
Detailed Description
The following description of the specific embodiments is merely exemplary in nature and is in no way intended to limit the disclosed technology or the application and uses of the disclosed technology. Furthermore, there is no intention to be bound by any expressed or implied theory presented in the preceding technical field, background, or the following detailed description.
In the following detailed description of embodiments, numerous specific details are set forth in order to provide a more thorough understanding of the disclosed technology. It will be apparent, however, to one skilled in the art that the disclosed techniques may be practiced without these specific details. In other instances, well-known features have not been described in detail so as not to unnecessarily complicate the description.
Terms such as "comprising" and "including" mean that in addition to having elements and steps that are directly and explicitly recited in the description, the inventive aspects also do not exclude the presence of other elements and steps not directly or explicitly recited. The terms such as "first" and "second" do not denote the order of units in terms of time, space, size, etc. but rather are merely used to distinguish one unit from another. In the present specification, the term "vehicle" or other similar terms include general motor vehicles such as passenger vehicles (including sport utility vehicles, buses, trucks, etc.), various commercial vehicles, and the like, and include hybrid vehicles, electric vehicles, plug-in hybrid electric vehicles, and the like. A hybrid vehicle is a vehicle having two or more power sources, such as a gasoline powered and an electric vehicle.
Hereinafter, various exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings.
Fig. 1 is a schematic flow diagram of a method 10 for determining a display failure of a smart cockpit in accordance with one or more embodiments of the present invention. It should be noted that the above-mentioned (and the following further mentioned) step names are only used for distinguishing between steps and facilitating the reference of steps, and do not represent a sequential relationship between steps, and the flowcharts including the figures are only examples of performing the method. The steps may be performed in various orders or concurrently without significant conflict.
As shown in fig. 1, in step S110, a real-time image acquired by a camera is received, wherein the real-time image includes raw display images of one or more displays within a smart car test rack.
With the richness of the intelligent cabin functionality, one or more displays may be disposed within the intelligent cabin, such as dashboards, heads-up displays (HUDs), and center stack screens in front of the driver, as well as entertainment screens in front of the copilot and the back row. The intelligent cabin host is connected with various buses and information transmission components on the vehicle to receive, send and preprocess various communication signals (such as Controller Area Network (CAN) bus signals) and render display images of one or more displays based on the communication signals. It will be appreciated that one or more displays are provided within the test rig for functional testing of the intelligent cabin, respectively, and that a camera (e.g. an industrial camera) is also provided which captures the raw display images of the one or more displays. It should be understood that in the embodiments of the present invention, cameras, video cameras, etc. represent devices that can acquire images or videos within a coverage area, which are similar in meaning and interchangeable, and the present invention is not limited in this regard.
Illustratively, the method 10 is based on an OpenCV implementation. OpenCV is an open-source software library, which can run on multiple operating systems and support Python language interfaces, and has the advantages of functional lightweight and efficient, and provides support for a large number of image recognition library functions. The intelligent cabin test bench uses a Python script to automatically control operation, and the Python realizes the call to the library function through a compatible interface cv2 of the OpenCV. By way of example, calls to core library functions such as gaussian filter function gaussian (), image capture function VideoCapture (), image load function imread (), gray processing function cvColor (), binarization function threshold (), non-zero count function countnon zero (), object creation function xfeature2d.
Optionally, in the method 10, a power-on cycle interval is set for the intelligent cockpit test rack, for example, a wake-up signal is sent to the intelligent cockpit test rack every 5 minutes. The intelligent cockpit testing stand, in response to receiving the wake-up signal, then turns on one or more displays and cameras within the intelligent cockpit testing stand. Optionally, after the intelligent cockpit test bench is powered up, the original display image of the one or more displays is truncated from the real-time image using an image capture algorithm (e.g., by calling the image capture function video capture ()) in the OpenCV library function, and/or a reference image is loaded as a reference for the one or more displays (e.g., by calling the image capture function video capture ()) in the OpenCV library function).
Optionally, after receiving the real-time image acquired by the camera, the real-time image may also be denoised using a gaussian low pass filtering algorithm. Illustratively, a real-time image is first transferred from the spatial domain to the frequency domain by a two-dimensional discrete fourier transform F (u, v) shown in the following equation to obtain a pixel (u, v) on a two-dimensional spectrum:
wherein x and y are respectively the abscissa and ordinate on the real-time image, M and N are the height and width of the real-time image, and f (x, y) is the gray value at the coordinate point (x, y) on the real-time image.
Next, the two-dimensional spectrum is subjected to a low-pass filtering operation using a gaussian low-pass filter H (u, v) shown in the following equation:
where D (u, v) is the distance of the pixel (u, v) from the center of the spectrum over the two-dimensional spectrum.
Finally, the filtered spectrum is subjected to a two-dimensional inverse discrete fourier transform to obtain a filtered image f (x, y) as shown in the following equation:
as shown in fig. 1, in step S120, fault recognition is performed on each original display image by simultaneously turning on a plurality of image processing algorithms through a multi-threaded call to determine whether each display has a display fault and a type of the display fault, wherein each image processing algorithm of the plurality of image processing algorithms is used to recognize a different type of display fault.
Multithreaded calls refer to techniques for implementing concurrent execution of multiple threads from software or hardware by program calls. The device with multithreading capability can execute multiple threads at the same time due to hardware support, thereby improving the overall processing performance. By simultaneously starting a plurality of image processing algorithms through multithread calling, the fault identification accuracy and the repair efficiency can be improved, and meanwhile, various display faults can be identified and positioned in a targeted manner, for example, which display has the display faults of which type.
Optionally, the plurality of image processing algorithms includes one or more of: the method comprises a first image processing algorithm for identifying black screen type faults, a second image processing algorithm for identifying snowflake screen type faults, a third image processing algorithm for identifying screen clamping type faults and a fourth image processing algorithm for identifying screen dithering type faults. That is, the original display image shown for each display can be identified and located simultaneously for black screen type faults, snowflake screen type faults, screen clamping type faults, and screen dithering type faults by multi-threaded calls.
Optionally, in an embodiment in which the plurality of image processing algorithms includes a first image processing algorithm for identifying a black screen type fault, step S120 includes: carrying out gray processing on the original display image by using a gray processing function; performing binarization processing on the gray-scale processed image by using a binarization function; counting the number of non-zero pixels in the binarized image by using a non-zero count function; the number of non-zero pixels is compared with a first threshold value, and whether a black screen type fault exists in a display corresponding to an original display image is determined based on a comparison result. Illustratively, the gray processing and the binarization thresholding are performed on the original display image of each display by calling the gray processing function cvColor () and the binarization function threshold () in the OpenCV library function, for example, if the gray value of a pixel is greater than or equal to 127, the pixel is processed by white point (i.e., the gray value of the pixel is set to 255), and if the gray value of the pixel is less than 127, the pixel is processed by black point (i.e., the gray value of the pixel is set to 0). Next, the number of non-zero pixels (i.e., white points) in the binarized image is counted by calling a non-zero count function countNonZero () in the OpenCV library function, if the total number of non-zero pixels is greater than or equal to a preset first threshold value, the original display image is judged to be normally lighted (i.e., the display does not have a black screen type fault), otherwise, the original display image is judged to not be normally lighted (i.e., the display has a black screen type fault). Alternatively, although the laboratory where the intelligent cabin test bench is located mostly adopts a light source with stable brightness, a certain brightness fluctuation may still exist in different time periods in a day, so the brightness of the laboratory environment can be calibrated in advance to determine the first threshold value for different time periods.
Optionally, in an embodiment in which the plurality of image processing algorithms includes a second image processing algorithm for identifying a snowflake screen class fault, step S120 includes: respectively carrying out feature point detection and calculating feature descriptors of the original display image and a reference image serving as a reference standard by utilizing a feature extraction function; matching the feature descriptors of the original display image and the feature descriptors of the reference image by using a picture matching function to generate matched feature point pairs; calculating the number of feature point pairs with the similarity of the feature points in the matched feature point pairs being greater than or equal to a second threshold value; and comparing the number with a third threshold value, and determining whether a snowflake screen type fault exists in a display corresponding to the original display image based on the comparison result. Illustratively, an object is created by calling an object creation function xfeature2d.sift_create () in the OpenCV library function, and feature points (e.g., contour break angles, vertices) of the original display image and the reference image are extracted and feature descriptors thereof are calculated, respectively, by calling a feature extraction function detectandcputj. A feature descriptor is a representation of an image (e.g., it may represent local image gradients computed at a selected scale in a region around each feature point) that can be found in common for two pictures by comparing the feature descriptors of the two pictures. Then, feature descriptors of the two images are matched by calling picture matching functions FlannBasedMatcher () and knnMatch () in the OpenCV library function, so that matched feature point pairs are obtained. And then, screening the similarity among the matched characteristic point pairs, namely, reserving the characteristic point pairs with the similarity larger than or equal to a second threshold value, counting the number of the characteristic point pairs, judging that the display does not have snowflake screen faults if the number of the characteristic point pairs is larger than or equal to a preset third threshold value, otherwise, judging that the display has snowflake screen faults.
Optionally, in an embodiment in which the plurality of image processing algorithms includes a second image processing algorithm for identifying a snowflake screen class fault, step S110 includes: the CAN bus signal input to the intelligent cockpit test bench is changed after loading a reference image as a reference standard, and a real-time image acquired by a camera is received after a first period of time. Illustratively, the normal displayed reference image is loaded by calling an image loading function imread () in the OpenCV library function, then the change of the CAN bus signal, such as the vehicle speed, the gear, etc., is input to the test bench, and the original display image of the display is captured and loaded after the first period of time, and the 2 pictures CAN be partially scratched (for example, by calling an image capturing function VideoCapture ()) to extract the interesting part.
Further, optionally, in an embodiment in which the plurality of image processing algorithms includes a third image processing algorithm for identifying a screen-clamping-type fault, step S120 includes: judging whether a display corresponding to the original display image has a clamping stagnation risk or not based on the number of the characteristic point pairs with the similarity larger than or equal to a fourth threshold value; if the risk of jamming is judged, gray scale processing and binarization processing are carried out on the original display image and the reference image, and the number of non-zero pixels in the image subjected to the gray scale and binarization processing is counted; and if the absolute value of the difference value between the number of the non-zero pixels of the original display image and the reference image is greater than or equal to a fifth threshold value, determining that the display has a screen clamping fault. The feature point detection, the feature descriptor calculation, the feature point pair matching, and the similarity screening are sequentially performed on the reference image and the original display image, which are respectively acquired before and after the CAN bus signal input to the intelligent cabin test bench is changed, and the feature point detection, the feature descriptor calculation, the feature point pair matching, and the similarity screening are the same as or similar to those in the snowflake screen fault identification, and are not repeated herein. And then comparing the counted number in the similarity screening with a fourth threshold value, if the counted number is larger than or equal to the preset fourth threshold value, judging that the display has a jamming risk, otherwise, judging that the display has no jamming risk (namely, no screen jamming fault occurs). If the display is judged to have the jamming risk, in order to avoid misjudging a scene, gray processing and binarization threshold processing are carried out on an original display image and a reference image by calling a gray processing function cvColor () and a binarization function threshold () in an OpenCV library function, the number of non-zero pixels in the two images subjected to binarization processing is counted by calling a non-zero counting function countNonzero () in the OpenCV library function, and if the absolute value of the difference value between the number of the non-zero pixels of the original display image and the number of the non-zero pixels of the reference image is larger than or equal to a fifth threshold, the display is judged to have the screen jamming fault, otherwise, the display is judged to not have the screen jamming fault.
Optionally, in an embodiment in which the plurality of image processing algorithms includes a fourth image processing algorithm for identifying a screen shake type fault, step S120 includes: storing and frame ordering the original display images acquired during the second period of time; performing jitter comparison on an original display image with adjacent frame numbers, and acquiring the total frame number of the jittered image; and comparing the total frame number with a sixth threshold value, and determining whether a display corresponding to the original display image has a screen shake type fault or not based on a comparison result. For example, in embodiments where real-time video is captured by the camera, video acquired during a second period of time after camera power up may be stored directly and frame ordered, it being noted that it is necessary to ensure that there is no change in CAN bus signal input during this second period of time and that no display update has occurred. Next, for each frame image, the frame image is compared with the adjacent frame image (e.g., the previous frame, the next frame) in jitter, and the total number of frames of the jittered image (e.g., the jitter value between the adjacent frames is large) is counted. If the total frame number is larger than or equal to a sixth threshold value, judging that the display has the screen shake type fault, otherwise, judging that the display does not have the screen shake type fault.
Optionally, step S120 further includes: monitoring the running state of each image processing thread in real time; and adjusting priorities of the plurality of image processing threads based on the run state. It will be appreciated that in order to prevent thread conflicts, the priority of the multiple threads may be preset, for example, a first thread for executing a first image processing algorithm, a second thread for executing a second image processing algorithm, a third thread for executing a third image processing algorithm, and a fourth thread for executing a fourth image processing algorithm. For example, the priority may be set according to the severity of each display failure, e.g., the first thread for a black-screen type failure may be higher priority than the other threads. However, thread starvation is a phenomenon that occurs easily under priority scheduling, that is, a thread having a higher priority than a thread having a lower priority always waits for execution before executing the thread, so that this thread having a lower priority is not always executed. To avoid starvation of threads, thread priority dynamic settings may be employed, i.e., the priority of each thread may be dynamically adjusted based on its real-time running state, e.g., according to how frequently the thread enters a wait state.
In step S130, it is determined whether to store the real-time image based on the failure recognition result. Optionally, if it is determined that one or more of the displays has a display failure, storing the real-time image; and if the display is determined to have no display fault, controlling the intelligent cabin test bench to be powered down, and sending a wake-up signal for the intelligent cabin test bench after a preset time interval. Illustratively, if it is determined in step S120 that any kind of display failure (such as a black screen type failure, a snowflake screen type failure, a screen stuck type failure, a screen shake type failure) has occurred in the display, in order to preserve the first scene to provide data analysis support for further diagnostics, the real-time images acquired by the camera during this power-up cycle are stored. If there is no display failure, a sleep signal is sent to the intelligent cockpit test bench, and a wake-up signal is sent to the intelligent cockpit test bench after a preset power-up cycle interval (e.g., 5 minutes) to resume execution of steps S110-S130.
The method 10 according to one or more embodiments of the present invention enables accurate localization of multiple display faults and timely reservation of a first site while ensuring the real-time of image recognition and processing by capturing the original display images of the respective displays in real time and performing fault recognition on each of the original display images by turning on multiple image processing algorithms, thereby providing important data analysis support for further diagnosis. In addition, the method 10 simultaneously starts a plurality of image processing algorithms through multi-thread calling, avoids the problem of low efficiency of single-thread processing, and can specifically identify and position various display faults, thereby improving the identification accuracy and the repair efficiency of the display faults of the intelligent cabin.
For a clearer illustration of the principles of the present application, fig. 2 illustrates, in a more complete form, a method 20 for determining a display failure of a smart cockpit, it being understood that the example of fig. 2 should not be considered as an additional limitation to other examples herein (e.g., the corresponding embodiment of fig. 1).
In step S210, in response to receiving the wake-up signal, the intelligent cockpit test rack is woken up, and then one or more displays and cameras within the intelligent cockpit test rack are turned on. In step S220, a reference image is loaded as a reference for one or more displays. In step S230, the main thread is started, a real-time image acquired by the camera (wherein the real-time image includes raw display images of one or more displays within the intelligent cockpit test stand) is received and the raw display images of the one or more displays are truncated from the real-time image using an image capture algorithm. In step S240, the original display image of the one or more displays is loaded. In step S250, the real-time image is subjected to denoising processing using a gaussian low pass filter algorithm. In step S260, a plurality of sub-threads are simultaneously started, wherein the first sub-thread includes executing a first image processing algorithm for identifying a black screen type fault, the second sub-thread includes executing a second image processing algorithm for identifying a snowflake screen type fault, the third sub-thread includes executing a third image processing algorithm for identifying a screen stuck type fault, the fourth sub-thread includes executing a fourth image processing algorithm for identifying a screen shake type fault, the fifth sub-thread includes monitoring an operation state of each image processing thread in real time, and adjusting priorities of the plurality of image processing threads based on the operation state. In step S270, it is determined whether or not there is a display failure on the display based on the execution results of the first to fourth sub-threads, if so, the method 20 proceeds to step S280, otherwise, the method 20 proceeds to step S290. In step S280, the program stops the loop and stores the real-time image so that the test bench remains in the faulty spot. In step S290, a timer is started to send a wake-up signal to the intelligent cockpit test bench after a preset power-up cycle interval to resume execution of steps S210-S290.
According to another aspect of the present invention, there is provided a fault detection device including: a memory configured to store instructions; and a processor configured to execute the instructions to perform the method 10 as shown in fig. 1 or the method 20 as shown in fig. 2.
According to yet another aspect of the present invention, there is provided an intelligent cockpit testing bench including: a camera for acquiring raw display images of one or more displays within the intelligent cockpit testing rig; and a fault detection device according to one aspect of the present invention.
In addition, the present invention may also be embodied as a computer storage medium having stored therein a program for causing a computer to perform the method 10 shown in fig. 1 or the method 20 shown in fig. 2. Here, as the computer storage medium, various types of computer storage media such as disks (e.g., magnetic disks, optical disks, etc.), cards (e.g., memory cards, optical cards, etc.), semiconductor memories (e.g., ROM, nonvolatile memory, etc.), tapes (e.g., magnetic tape, magnetic cassette, etc.), and the like can be employed.
Where applicable, hardware, software, or a combination of hardware and software may be used to implement the various embodiments provided by the present invention. Moreover, where applicable, the various hardware components and/or software components set forth herein may be combined into composite components comprising software, hardware, and/or both without departing from the scope of the present invention. Where applicable, the various hardware components and/or software components set forth herein can be separated into sub-components comprising software, hardware, or both without departing from the scope of the present invention. Further, where applicable, it is contemplated that software components may be implemented as hardware components, and vice versa.
Software in accordance with the present invention, such as program code and/or data, may be stored on one or more computer storage media. It is also contemplated that the software identified herein may be implemented using one or more general-purpose or special-purpose computers and/or computer systems that are networked and/or otherwise. The embodiments and examples set forth herein are presented to best explain the embodiments consistent with the invention and its particular application and to thereby enable those skilled in the art to make and use the invention. However, those skilled in the art will recognize that the foregoing description and examples have been presented for the purpose of illustration and example only. The description as set forth is not intended to cover various aspects of the invention or to limit the invention to the precise form disclosed.
Claims (13)
1. A method for determining a display failure of a smart cockpit, the method comprising the steps of:
A. receiving a real-time image acquired by a camera, wherein the real-time image comprises raw display images of one or more displays within a smart cockpit test rig;
B. simultaneously starting a plurality of image processing algorithms through multithreading to perform fault identification on each original display image so as to determine whether each display has a display fault and the type of the display fault, wherein each image processing algorithm in the plurality of image processing algorithms is used for identifying different types of display faults; and
C. based on the fault recognition result, it is determined whether to store the real-time image.
2. The method of claim 1, wherein step a further comprises one or more of:
in response to receiving a wake-up signal, wake-up the intelligent cockpit test bench;
loading a reference image as a reference datum for the one or more displays;
denoising the real-time image by using a Gaussian low-pass filtering algorithm; and
an original display image of one or more displays is truncated from the real-time image using an image capture algorithm.
3. The method of claim 1, wherein the plurality of image processing algorithms comprises one or more of: the method comprises a first image processing algorithm for identifying black screen type faults, a second image processing algorithm for identifying snowflake screen type faults, a third image processing algorithm for identifying screen clamping type faults and a fourth image processing algorithm for identifying screen dithering type faults.
4. The method of claim 1, wherein the plurality of image processing algorithms includes a first image processing algorithm for identifying a black screen type fault, and step B includes:
carrying out gray processing on the original display image by using a gray processing function;
performing binarization processing on the gray-scale processed image by using a binarization function;
counting the number of non-zero pixels in the binarized image by using a non-zero count function;
and comparing the number of the non-zero pixels with a first threshold value, and determining whether a black screen type fault exists in a display corresponding to the original display image based on a comparison result.
5. The method of claim 1, wherein the plurality of image processing algorithms includes a second image processing algorithm for identifying a snowflake screen type fault, and step B includes:
respectively carrying out feature point detection and calculating a feature descriptor of the original display image and a reference image serving as a reference standard by utilizing a feature extraction function;
matching the feature descriptors of the original display image and the feature descriptors of the reference image by using a picture matching function to generate matched feature point pairs;
calculating the number of the feature point pairs with the feature point similarity of the matched feature point pairs being greater than or equal to a second threshold value;
and comparing the number with a third threshold value, and determining whether a snowflake screen type fault exists in a display corresponding to the original display image based on a comparison result.
6. The method of claim 1, wherein step a comprises changing CAN bus signals input to the intelligent cockpit test bench after loading a reference image as a reference datum and receiving the real-time image acquired by a camera after a first period of time, and the plurality of image processing algorithms comprises a third image processing algorithm for identifying screen-stuck-at faults.
7. The method of claim 6, wherein step B comprises:
judging whether a display corresponding to the original display image has a clamping stagnation risk or not based on the number of characteristic point pairs with the similarity of the characteristic point pairs of the original display image and the reference image being larger than or equal to a fourth threshold value;
if the risk of jamming is judged, gray scale processing and binarization processing are carried out on the original display image and the reference image, and the number of non-zero pixels in the image subjected to gray scale and binarization processing is counted;
and if the absolute value of the difference value between the number of the non-zero pixels of the original display image and the reference image is greater than or equal to a fifth threshold value, determining that the display has a screen clamping fault.
8. The method of claim 1, wherein the plurality of image processing algorithms includes a fourth image processing algorithm for identifying a screen shake type fault, and step B includes:
storing and frame ordering the original display images acquired during the second period of time;
performing jitter comparison on an original display image with adjacent frame numbers, and acquiring the total frame number of the jittered image;
and comparing the total frame number with a sixth threshold value, and determining whether a display corresponding to the original display image has a screen shake type fault or not based on a comparison result.
9. The method of claim 1, wherein step B further comprises:
monitoring the running state of each image processing thread in real time; and
and adjusting the priority of the plurality of image processing threads based on the running state.
10. The method of claim 1, wherein step C comprises:
if it is determined that one or more of the displays has a display failure, storing the real-time image;
and if the display is determined to have no display fault, controlling the intelligent cabin test bench to be powered down, and sending a wake-up signal aiming at the intelligent cabin test bench after a preset time interval.
11. A fault detection device, comprising:
a memory configured to store instructions; and
a processor configured to execute the instructions to perform the method of any one of claims 1-10.
12. An intelligent cockpit testing bench, comprising:
a camera for acquiring raw display images of one or more displays within the intelligent cockpit testing rig; and
the fault detection device of claim 11.
13. A computer storage medium comprising instructions which, when executed, perform the method of any one of claims 1-10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310102059.0A CN116012365A (en) | 2023-01-19 | 2023-01-19 | Method for determining display faults of intelligent cabins and fault detection device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310102059.0A CN116012365A (en) | 2023-01-19 | 2023-01-19 | Method for determining display faults of intelligent cabins and fault detection device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116012365A true CN116012365A (en) | 2023-04-25 |
Family
ID=86030011
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310102059.0A Pending CN116012365A (en) | 2023-01-19 | 2023-01-19 | Method for determining display faults of intelligent cabins and fault detection device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116012365A (en) |
-
2023
- 2023-01-19 CN CN202310102059.0A patent/CN116012365A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20200125885A1 (en) | Vehicle insurance image processing method, apparatus, server, and system | |
| CN110766679A (en) | Lens contamination detection method and device and terminal equipment | |
| CN109840883B (en) | Method and device for training object recognition neural network and computing equipment | |
| CN110443212B (en) | Positive sample acquisition method, device, equipment and storage medium for target detection | |
| CN115829921B (en) | Method, apparatus and computer readable storage medium for detecting cell defects | |
| CN113240673B (en) | Defect detection method, defect detection device, electronic equipment and storage medium | |
| JP2009048629A (en) | Detecting method | |
| CN108107611B (en) | Self-adaptive defect detection method and device and electronic equipment | |
| CN109784322B (en) | Method, equipment and medium for identifying vin code based on image processing | |
| CN115470109A (en) | Automatic testing method and device for automobile instrument | |
| CN114298985B (en) | Defect detection method, device, equipment and storage medium | |
| CN113065454B (en) | High-altitude parabolic target identification and comparison method and device | |
| CN113139419B (en) | Unmanned aerial vehicle detection method and device | |
| CN111369492A (en) | Display screen detection method, detection device and detection system | |
| CN116012365A (en) | Method for determining display faults of intelligent cabins and fault detection device | |
| CN114821194B (en) | Equipment running state identification method and device | |
| CN114764779A (en) | Computing device and defect detection method for near-eye display device | |
| CN111416974A (en) | Camera-based television screen acquisition method and system and intelligent terminal | |
| CN115512302A (en) | Vehicle detection method and system based on improved YOLOX-s model | |
| CN111400534B (en) | Cover determination method and device for image data and computer storage medium | |
| US11676265B2 (en) | Method and image processing device for mura detection on display | |
| CN116071246A (en) | Image processing method, image processing apparatus, and storage medium | |
| CN114973394A (en) | Gesture motion recognition method and device, electronic equipment and computer storage medium | |
| CN111986144A (en) | Image blur judgment method and device, terminal equipment and medium | |
| CN112989866B (en) | Object recognition method, device, electronic equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |