WO2021174407A1

WO2021174407A1 - Image display monitoring method, apparatus and device

Info

Publication number: WO2021174407A1
Application number: PCT/CN2020/077523
Authority: WO
Inventors: 李华; 薛林; 文长春
Original assignee: 华为技术有限公司
Priority date: 2020-03-03
Filing date: 2020-03-03
Publication date: 2021-09-10
Also published as: CN113614700A

Abstract

Disclosed are an image display monitoring method, apparatus and device. The image display monitoring device may comprise a first display controller and a neural network processor, wherein the first display controller may generate a first image by using first data, the first image is used for display on a display device, while in fact, the first image may have a problem of generating an error, thereby causing the first image to not correctly reflect the features of the first data. Therefore, in embodiments of the present application, at least a part of the first image may be obtained by using the neural network processor, and whether at least a part of the first image is displayed correctly may be determined by using a neural network model, so as to obtain a monitoring result of at least a part of the first image. Since in the embodiments of the present application, at least a part of the first image is monitored by using the neural network model, the flexibility is higher, and the reliability and pertinence of the obtained monitoring result are also higher, thereby improving the accuracy of image display.

Description

Method, device and equipment for image display monitoring

Technical field

This application relates to the field of image display technology, and in particular to a method, device and equipment for image display monitoring.

Background technique

At present, the image to be displayed can be generated by the display controller, and the display device can display the image, so that the user can obtain the required information. However, the to-be-displayed image generated by the display controller may have an error, so that the display device displays the wrong image, allowing the user to obtain the wrong information.

Therefore, it is necessary to monitor the image to be displayed generated by the display controller. At present, a monitoring chip is connected between the display controller and the display device. The monitoring chip is a logic calculation unit of hardware, which can perform calculations on pixel data, and can include red, green, and blue (red- Green-blue, RGB) values are weighted and calculated or averaged, or the cyclic redundancy check (CRC) value of the pixel data set is calculated, or the contrast between the light-emitting part and the background part of the image in the area is calculated, for example RGB difference and so on. In this way, when the comparison result between the calculated value and the preset threshold value does not satisfy the preset condition, it can be considered that the image to be displayed generated by the display controller is incorrect.

However, in this way, once the function of the hardware circuit is deployed, it cannot be artificially modified, nor can it be artificially added to a new monitoring algorithm, so it is not flexible enough, the features that can be monitored are not comprehensive enough, and the accuracy of monitoring is not high.

Summary of the invention

The technical problem to be solved by this application is to provide a method, device and equipment for image display monitoring, which can use a neural network model to monitor the generated first image and improve the flexibility and accuracy of image display.

In the first aspect, an embodiment of the present application provides an image display monitoring device, including: a first display controller and a neural network processor; wherein, the first display controller is used to generate a first image using first data, and the generated first image An image is used for display on a display device, but in fact, the first image may have a problem of generating errors, resulting in the first image not being able to correctly reflect the characteristics of the first data. Therefore, the neural network processor in the embodiment of the present application is used to obtain at least a part of the first image, and use the neural network model to determine whether at least a part of the first image is displayed correctly, so as to obtain a monitoring result of at least a part of the first image. The application embodiment uses a neural network model to monitor at least a part of the first image, and the related model can be freely changed, configured or upgraded. Therefore, compared with the detection circuit of dedicated hardware, the flexibility is higher, and the obtained monitoring result is better. Reliability and pertinence are also higher, which in turn improves the accuracy of image display.

In some possible implementation manners, the neural network processor is further configured to: when the monitoring result is that at least a part of the first image is incorrectly displayed, send an image generation instruction to the second display controller; the The second display controller is configured to generate a second image in response to the image generation instruction, and the second image is used to replace the first image for display on the display device.

In the embodiment of the present application, when the neural network processor determines that at least a part of the first image is incorrectly displayed, the second display controller can also be used to generate the second image, so that the second image can be used to replace the first image on the display device. The above display can also display the correct image without displaying errors, which further improves the display accuracy, improves user experience and system security.

In some possible implementation manners, the safety integrity level of the second display controller is higher than the safety integrity level of the first display controller.

In the embodiment of the present application, the second display controller can be used to generate the second image, so that the second image replaces the first image to be displayed on the display device. Therefore, the safety integrity level of the second display controller can be higher than the first image. A safety integrity level of the display controller, so that the accuracy of the second image generated by the second display controller is higher than that of the first image, so that the accuracy of the image display can be improved.

In some possible implementation manners, the first display controller includes at least one of a first image processor GPU or a first display subsystem DSS, and the second display controller includes a second image processor GPU, At least one of the second display subsystem DSS or the microcontroller unit MCU.

In the embodiment of the present application, both the first display controller and the second display controller may include GPU and/or DSS, so that the generation process of the first image and the second image will be more complete, and the generated images will be more satisfactory. The needs of users.

In some possible implementation manners, at least a part of the first image includes an indicator icon; the device further includes: an MCU, configured to determine that when the monitoring result is that at least a part of the first image is displayed correctly Describe whether the indicator icon is an indicator icon that needs to be displayed.

In the embodiment of the present application, the correct display of the indicator icon is more important to the functional safety of the system, so at least a part of the first image may be an indicator icon, and the monitoring of the indicator icon is beneficial to improve the functional safety of the system. In this way, when the monitoring result is that at least a part of the first image is displayed correctly, the MCU can further determine whether the correctly displayed indicator icon is the indicator icon that needs to be displayed, thereby further improving the accuracy of the image display.

In some possible implementation manners, at least a part of the first image includes an indicator icon; the neural network processor is further configured to: when the monitoring result is that at least a part of the first image is displayed correctly, use neural network The network model and the instruction information from the MCU determine whether the instruction icon is an instruction icon that needs to be displayed.

In the embodiment of the present application, when the monitoring result is that at least a part of the first image is displayed correctly, the neural network processor can use the neural network model and the instruction information from the MCU to determine whether the indicator icon is an indicator icon that needs to be displayed. So as to further improve the accuracy of image display.

In some possible implementation manners, the image display monitoring device further includes: a main processor, configured to generate the first data; the first display controller, specifically configured to obtain the first data from the main processor data.

In the embodiment of the present application, the first display controller may obtain the first data from the main processor, so that the first image can be generated based on the data obtained from the main processor to display the information that the main processor needs to display, and improve user experience.

In some possible implementation manners, the main processor is further configured to: obtain an instruction from the MCU, and generate the first data in response to the instruction.

In the embodiment of the present application, the main processor may generate the first data based on instructions obtained from the MCU, which may improve the reliability of the first data.

An embodiment of the present application also provides a method for image display monitoring, including: generating a first image using first data through a first display controller, the first image being used for display on a display device; At least a part; using a neural network model to determine whether at least a part of the first image is displayed correctly, so as to obtain a monitoring result of at least a part of the first image.

In some possible implementation manners, the method further includes: when the monitoring result is that at least a part of the first image is incorrectly displayed, sending an image generation instruction to a second display controller; The controller generates a second image in response to the image generation instruction, and the second image is used to replace the first image for display on the display device.

In some possible implementation manners, at least a part of the first image includes an indicator icon, and the method further includes:

When the monitoring result is that at least a part of the first image is displayed correctly, it is determined whether the indicator icon is an indicator icon that needs to be displayed.

An embodiment of the present application also provides an image display monitoring device, including: a display control module, configured to generate a first image using first data through a first display controller, and the first image is used for display on a display device; The processing module is configured to obtain at least a part of the first image; use a neural network model to determine whether at least a part of the first image is displayed correctly, so as to obtain a monitoring result of at least a part of the first image.

In some possible implementation manners, the processing module is further configured to: when the monitoring result is that at least a part of the first image is incorrectly displayed, send an image generation instruction to the second display controller; the display control The module is also used to generate a second image in response to the image generation instruction through the second display controller, and the second image is used to replace the first image for display on the display device.

In some possible implementation manners, at least a part of the first image includes an indicator icon, and the processing module is further configured to: when the monitoring result is that at least a part of the first image is displayed correctly, determine that the Whether the indicator icon is an indicator icon that needs to be displayed.

The embodiment of the present application also provides a neural network processor, including: a processor and a memory; the memory is used to store computer programs or instructions; the processor is used to execute the computer programs in the memory Or an instruction to implement the image display monitoring method provided in the embodiment of the present application.

The embodiment of the present application also provides a computer-readable storage medium, including a computer program or instruction, which, when running on a computer, enables the computer to implement the image display monitoring method provided by the embodiment of the present application.

The embodiments of the application provide a method, device, and equipment for image display monitoring. The image display monitoring equipment may include a first display controller and a neural network processor. The first display controller may generate a first image using first data. , The first image is used for display on the display device, but in fact, the first image may have a problem of generating errors, resulting in the first image not being able to correctly reflect the characteristics of the first data. Therefore, in the embodiments of the present application, a neural network processor may be used to obtain at least a part of the first image, and a neural network model may be used to determine whether at least a part of the first image is displayed correctly, so as to obtain a monitoring result of at least a part of the first image. The embodiment of the application uses a neural network model to monitor at least a part of the first image, and the related model can be freely changed, configured or upgraded online. Therefore, it is more flexible than the detection circuit of dedicated hardware, and the obtained monitoring results are more flexible. Reliability and pertinence are also higher, which in turn improves the accuracy of image display.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the description of the embodiments. Obviously, the drawings in the following description are only some of the implementations recorded in the present application. For example, for those of ordinary skill in the art, other drawings can be obtained based on these drawings.

Figure 1 is a schematic diagram of indicator icons in the monitoring process of electrical and electronic equipment in the automotive field;

2 is a schematic diagram of a system framework for image monitoring provided by an embodiment of the application;

FIG. 3 is a schematic diagram of another image display system provided by an embodiment of the application;

FIG. 4 is a schematic diagram of another image monitoring system provided by an embodiment of the application;

FIG. 5 is a flowchart of an image monitoring method provided by an embodiment of the application;

FIG. 6 is a structural block diagram of an image display monitoring device provided by an embodiment of the application.

Detailed ways

The embodiments of the present application provide a method, device, and equipment for image display monitoring, which can use a neural network model to monitor the generated first image, thereby improving the flexibility and accuracy of image display.

Before introducing the method provided by the embodiment of the present application, the following points are explained first. The terms "first", "second", "third" or "fourth" (if any) in the description and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and do not need to be used To describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments described herein can be implemented in a sequence other than the content illustrated or described herein. In addition, the terms "including" and "having" and any variations of them are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those clearly listed. Those steps or units may include other steps or units that are not clearly listed or are inherent to these processes, methods, products, or equipment.

"At least one" means one or more, and "plurality" means two or more. "And/or" describes the association relationship of the associated objects, indicating that there can be three relationships, for example, A and/or B, which can mean: A alone exists, A and B exist at the same time, and B exists alone, where A, B can be singular or plural. The character "/" generally indicates that the associated objects before and after are in an "or" relationship. "The following at least one item (a)" or similar expressions refers to any combination of these items, including any combination of a single item (a) or a plurality of items (a). For example, at least one of a, b, and c can mean: a, or b, or c, or a and b, or a and c, or b and c, or a, b and c, where a, b, c can be single or multiple.

In the embodiment of the present invention, the image to be displayed can be generated by the display controller based on the display data, and then the display device can display the image, so that the user can obtain the required information through the displayed image. For example, in the industrial, automotive, and medical fields, in order to improve the function safety of the system, electronic and electrical equipment are often required to have a series of real-time self-detection, post-fault detection alarms, and post-fault detection processing. Therefore, it can be used as a warning to the user in the event of a fault, or it can still have a certain function and operability in the case of a part of the circuit failure, so as to avoid harm to personal safety. Among them, the IEC 61508 standard describes the functional safety design of industrial control electrical and electronic equipment, and the ISO 26262 standard describes the functional safety design of electrical and electronic equipment in the automotive field. Take the automotive safety integrity level (ASIL) as an example. ASIL refers to the level of functional safety requirements for the module system obtained by module suppliers and integrators according to the risk and hazard analysis process, which can be divided into QM, There are 5 levels A, B, C and D, from level QM to ASIL-D, requiring the system to have higher and higher fault self-detection and self-processing capabilities.

Among them, alarming after an abnormality occurs in the system is an effective means to improve the functional safety of the system. Specifically, an indicator icon can be generated based on the abnormal information, and the indicator icon is displayed, and the user obtains the specific content of the abnormal information through the indicator icon. Specifically, in the process of monitoring electronic and electrical equipment by the power control center in the industrial control scenario, or in the monitoring process of electronic and electrical equipment in the automotive field, or in the monitoring process of electronic and electrical equipment in the medical field, if there is a problem with the monitored equipment If the working status is abnormal, the indicator icon can be generated according to the abnormal information, and the indicator icon can be displayed, so that the user can obtain the abnormal information of the monitored device in time, so that the corresponding operation can be made in response to the abnormal information, thereby improving the system Functional safety.

In the automotive field, the abnormal information of the car is indicated by the indicator icon, including the fault display when the vehicle cannot be started or the vehicle fails. The corresponding ASIL is usually B level. Refer to Figure 1, which is a schematic diagram of indicator icons in the monitoring process of electrical and electronic equipment in the automotive field. The indicator icons can include: (1) cruise control indicator light, (2) power indicator light, (3) brake protection Anti-lock braking system (ABS) indicator light, (4) economy mode indicator light, (5) parking indicator light, (6) engine self-check light, (7) high beam auxiliary function indicator light, ( 8) Coolant temperature indicator, (9) Tire pressure monitoring system (TPMS) indicator, (10) Reversing radar warning indicator, (11) High beam indicator, (12) Fuel indicator.

The on and off states of these indicator icons can reflect whether the vehicle has a fault corresponding to the indicator icon. It should be noted that when the indicator icon is not illuminated, it can be considered that the indicator icon does not exist in the image to be displayed. For example, the battery indicator light is an indicator light showing the working status of the battery. If the generator or circuit fails, the battery indicator light does not light up or stays on for a long time, so that the driver can check whether the battery indicator light is included in the displayed image and whether the battery indicator light is The status of the generator or circuit can be obtained by blinking.

In other words, in the industrial, automotive, and medical fields, whether the image to be displayed generated by the display controller is correct, that is, whether the indicator icon in the image to be displayed is displayed correctly, directly determines the image displayed by the user Whether the abnormal information of the vehicle can be obtained, so the accuracy of the image display is particularly important, especially in the case of personal safety, if the image to be displayed is incorrect, the user will not be able to know the malfunction of the monitored device in time In order to make a reasonable operation, this kind of error will cause a certain threat to the human body, and the functional safety of the system will also be reduced. For example, when the generator of the vehicle is faulty, the abnormal information of the vehicle is that the generator is faulty. However, the icon indicating the abnormality of the generator is not normally displayed in the image to be displayed. For example, the icon is blocked or displayed incompletely, and the driver Based on the incorrectly displayed icon, the generator failure information may not be accurately obtained, which is dangerous for the driver.

The solution described in the background art is not flexible enough, and the features of monitoring are not comprehensive enough, which leads to the limitation of the accuracy of monitoring. In particular, with the advancement of automotive electronics, intelligence, networking, and home furnishing, the dashboards of some new concept models have evolved from the traditional motor control pointer and level signal control warning indicator solution to a "one piece". In this case, all the information of the “pointer” and “warning light” are displayed on the LCD screen. At the same time, the LCD screen can also display gorgeous background and ambient light, and real-time navigation information. , Real-time media playback information, real-time weather and other network information, enhanced visual night vision cameras, etc., image monitoring using hardware circuits has been disturbed to a certain extent, the accuracy is further reduced, and it cannot meet actual needs.

Based on the above technical problems, embodiments of the present application provide a method, device, and equipment for image display monitoring. The image display monitoring equipment may include a first display controller and a neural network processor. The first display controller may use the first The data generates a first image, and the first image is used for display on a display device. In fact, the first image may have a problem of generating errors, which causes the first image to not correctly reflect the characteristics of the first data. Therefore, in the embodiments of the present application, a neural network processor may be used to obtain at least a part of the first image, and a neural network model may be used to determine whether at least a part of the first image is displayed correctly, so as to obtain a monitoring result of at least a part of the first image. The embodiment of the application uses a neural network model to monitor at least a part of the first image, and the related model can be freely changed, configured or upgraded online. Therefore, it is more flexible than the detection circuit of dedicated hardware, and the obtained monitoring results are more flexible. Reliability and pertinence are also higher, which in turn improves the accuracy of image display.

Referring to FIG. 2, it is a schematic diagram of an image monitoring system framework provided by an embodiment of this application. The system may include an image display monitoring device and a display device 200. The image display monitoring device may include a first display controller 101 and a neural network processor 102.

The first display controller 101 may include a display subsystem (DSS) or a display driver, and may be a high-end media processor with a complex drawing function. The first display controller 101 may also include an image processor (graphics processing unit, GPU). Of course, the first display controller 101 may also include both a DSS and a GPU. For example, the first display controller 101 may be a GPU with embedded DSS functions, or an integrated component including a GPU and a DSS.

Among them, the GPU can be used for image rendering and drawing. DSS can be used to perform layer overlay processing, and send the image formed after the layer overlay to the display device 200 for display. Optionally, DSS can also be used to perform processing such as image inversion, method, or reduction. Not limited. The layer overlay processing includes, but is not limited to, overlaying the image drawn by the GPU with other images, such as overlaying with a background image or a window. The background image can include gorgeous background and ambient light, real-time navigation information, real-time media playback information, real-time weather and other network information, and night vision camera images that enhance vision. These contents can be used as background images of current indicator icons to enhance users The visual effects or the expansion of the display content of the display device to meet the needs of users.

In the embodiment of the present application, the first display controller 101 may generate the first image based on the first data. Specifically, when the first display controller 101 generates the first image, it may be formed by layered drawing and graphics superimposition. The first image, for example, multiple indicator icons can be located on different layers, and the first display controller 101 can draw multiple indicator icons and background images located on different layers from the indicator icons to obtain multi-layer graphics, and then use the graphics Superimpose to generate the first image.

Since the display of the first image is of great significance for improving the functional safety of the system, the functional safety enhancement design of the first display controller 101 can be carried out to improve the functional safety of the first display controller 101. These functions The security enhancement design includes but is not limited to: setting dual-core backup, enabling the memory connected to the first display controller 101 to support error detection and error correction verification codes, setting the signal detection strategy for the first display controller 101, and the first display controller 101 performs self-detection and provides hardware logic guarantee for the first display controller 101 to power down the first display controller 101 or reset the device to indicate that the icon is lit by default, and so on.

The first data used to generate the first image is data indicating image features in the first image. For example, it may be a detection result generated based on the detection data obtained by detecting the device to be detected, and the detection result may include an abnormal result and/ Or normal results. In the field of vehicles, the equipment to be tested can be electrical and electronic equipment on automobiles, such as engines, tires or reversing radars. In the field of industrial control or medical treatment, the equipment to be tested can be other electrical and electronic equipment. I will not give an example one by one. When the first image includes the indicator icon, the first data includes data indicating the status information of the indicator icon in the first image, and the status information of the indicator icon may include on or off. Generally speaking, if the first data indicates the first image If the status of the indicator icon in the first image is on, the first image generated based on the first data includes a lit indicator icon, and if the first data indicates that the indicator icon in the first image is off, then it’s generated based on the first data The first image of does not include a lit indicator icon.

As shown in Figure 1, the indicator icons in the monitoring process of electrical and electronic equipment in the automotive field can include: (1) cruise control indicator light, (2) power indicator light, (3) ABS indicator light, (4) economy mode Indicator light, (5) parking indicator light, (6) engine self-check light, (7) high beam auxiliary function indicator light, (8) coolant temperature indicator light, (9) TPMS indicator light, (10) reversing radar Alarm indicator, (11) high beam indicator, (12) fuel indicator. The on and off states of these indicator icons reflect whether the equipment to be tested corresponding to the indicator icon in the vehicle is faulty. For example, the battery indicator is an indicator that shows the working status of the battery. If the battery indicator is off or keeps on, it means the generator or circuit failure.

For each first image, the corresponding first data is determined, so the on-off state of the indicator icon therein is also determined, and the on-off state of the indicator icon is exactly corresponding to the first data in content . In actual operation, the first display controller 101 may sequentially generate a plurality of first images based on the image display order, so as to display the dynamic effect from a macro perspective when the plurality of first images are displayed in sequence, for example, to realize the indicator icon in the first image. The long light or flashing effect.

In the embodiment of the present application, the first display controller 101 may be connected to the main processor 300, and obtain first data from the main processor 300, so as to generate the first image according to the first data. The main processor 300 may generate the first data, for example, may generate the first data according to the acquired detection data or instructions. Specifically, the main processor 300 may include a central processing unit (CPU), a field programmable gate array (FPGA), a network processor (NP), and a digital signal processing circuit (digital signal processing circuit). signal processor, DSP), programmable logic device (PLD), microcontroller unit (MCU), application specific integrated circuit (ASIC), or system on chip (SoC) Wait.

Generally speaking, since the first image is generated based on the first data, the first image can reflect the content of the first data, that is, the indicator icon in the first image and the state of the indicator icon in the first data The data should be consistent. At this time, it can be considered that the generated first image is correct. In this way, the display device 200 can display the first image, and the user can obtain the first data corresponding to the first image more vividly according to the first image. Content.

However, the first image may have problems of generating errors, for example, the indicator icons in the generated first image are incomplete, covered, and color wrong. This is because in actual operation, based on cost considerations, the safety integrity level of the first display controller 101 is often not very high. Taking the vehicle field as an example, the ASIL of the first display controller 101 often only meets the quality of QM Therefore, the additional workload of the first display controller 101 is generally lower and the cost is also lower. Therefore, the first display controller 101 may make an error when generating the first image. If the wrong first image is displayed, the user may not be able to obtain accurate information contained in the first data in time.

Therefore, in the embodiment of the present application, the neural network processor 102 can be used to monitor the display of the first image, and the neural network processor 102 can obtain at least a part of the first image, and use the neural network model to determine at least a part of the first image. Whether the display is correct to obtain the monitoring result of at least a part of the first image.

The neural network processor 102 may be a processing device with a neural network model computing capability, such as a neural-network process unit (NPU), an artificial intelligence (AI) processor, or a Bionic device Etc., the model operation performed by it may be a neural network operation. For example, a neural network model that has been pre-trained can be embedded in the neural network processor 102 to realize the model operation. For example, the neural network processor 102 can be used as an independent chip, such as a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or Other types of neural processing units.

In the embodiment of the present application, the neural network processor 102 may monitor at least a part of the first image, so as to obtain a monitoring result of at least a part of the first image. Specifically, the neural network processor 102 may input at least a part of the first image into the neural network model to obtain a judgment result of the correctness of at least a part of the first image as a monitoring result of at least a part of the first image. A process can be considered as matching at least a part of the first image with the preset image to obtain the matching degree. If the preset image is a correct image, the higher the matching degree, the higher the probability that at least a part of the first image is correct. If the preset image is an incorrect image, the higher the matching degree, the higher the probability that at least a part of the first image is incorrect.

Of course, in order to obtain the judgment result, a first threshold can be set for the correct probability of at least a part of the first image. When the correct probability is higher than the first threshold, at least a part of the first image can be considered to be correct, otherwise it can be considered At least a part of the first image is wrong. For example, it can be obtained that the probability of at least part of the first image being correct is 20%, the probability of error is 80%, and the first threshold is 80%. At this time, it can be considered that at least part of the monitored first image is wrong, that is, the first An image is generated incorrectly.

Specifically, the neural network processor 102 may obtain at least a part of the first image from the memory 104, and the memory 104 may be connected to the first display controller 101 and the neural network processor 102 respectively, so that the first display controller 101 generates the first image. After the image is imaged, the first image may be stored in the memory 104, and the neural network processor 102 may then obtain at least a part of the first image from the memory 104. The first image may be stored in the image buffer area of the memory 104. After the first display controller 101 generates the first image, it may send the storage location of at least a part of the first image to the neural network processor 102 for the neural network The processor 102 acquires at least a part of the first image according to the storage location of the at least part of the first image. Of course, the memory 104 can also be connected to the display device 200, so that the display device 200 can acquire the first image in the image buffer area, and then display the first image; of course, the memory 104 can also not be directly connected to the display device 200 to display The device 200 may obtain the first image through the first display controller 101, and then display the first image.

Based on the above description, in the first image, the correct display of the indicator icon is more important to the functional safety of the system. Therefore, in the embodiment of the present application, at least a part of the first image may be an indicator icon, and the monitoring of the indicator icon is Conducive to improving the functional safety of the system. In other words, the neural network processor 102 may not monitor all image features in the first image, but monitor the indicator icons in the first image, which reduces the computational workload of the neural network processor 102. For example, at least a part of the first image is an indication area image including an indication icon, where the indication area graphic may be a rectangle or other shapes.

During specific implementation, a preset storage area may be set for at least a part of the first image in the memory 104, so that after the first display controller 101 generates the first image, the image data of at least a part of the first image may be stored in the preset storage area. Assuming it is in the storage area, the neural network processor 102 can also read this part of the data from the preset storage area. Specifically, when the first image is stored in a certain data structure, at least a part of the image data of the first image can be mapped to the physical address of the storage area, that is, to the preset storage area. In this way, the neural network processor 102 The pixel data of the preset storage area is read to obtain the image data of at least a part of the first image.

It should be noted that when at least a part of the first image has a fixed display position in the first image, the preset storage area mapped to at least a part of the first image may be determined during the design stage; When there is no fixed display position in the first image, but dynamic changes according to the actual situation, the neural network processor 102 can obtain the physical address of the preset storage area mapped to at least a part of the first image, thereby obtaining the data of this part, Specifically, the neural network processor 102 can obtain the physical address of the preset storage area from the main processor 300 or the MCU 400 with a security authentication function connected to the neural network processor 102, and the acquisition method can be through various inter-core communications. Methods, such as interrupts and specific message mechanisms.

In the embodiment of the present application, since the generation of the first image has a certain periodicity, for example, a video image of 50 frames per second (FPS) will only output one frame in about 20 ms. Therefore, the neural network processor 102 can output one frame according to the first image. The generation period of an image determines the period for acquiring the first image. For example, the neural network processor 102 may acquire the first image every 20 ms. In this way, the neural network processor 102 in the embodiment of the present application is not modeled by the neural network. Exclusively, it may also have the ability to perform other data processing. For example, the neural network processor 102 may also have an audio recognition model, etc., and these different models are not affected by each other.

The neural network model in the neural network processor 102 can be trained in advance by using historical display information and historical images, so that the neural network model has the ability to recognize images. Among them, the neural network model can be trained during the design phase, or It can be obtained by training before monitoring the first image later. Among them, historical images may include true (Truth) historical images and/or false (False) historical images, where incorrect historical images may be occluded, displayed incompletely, color errors, and shape errors, etc., where the occlusion is, for example, The indicator icon in the historical image is obscured by the upper-layer graphics, and the color is wrong. For example, the indicator icon in the historical image is lit incorrectly. For example, the color of the indicator icon in the historical image should be red, but actually green.

Since the neural network processor 102 can only acquire at least a part of the image data of the first image, the historical image can only include the image data of the part of the image corresponding to at least a part of the first image, so as to reduce the amount of calculation for data training and The accuracy of image recognition. For example, at least a part of the first image includes the indicator icon in the first image, the historical image may also include only the correct indicator icon and/or the wrong indicator icon, so that the trained neural network model has the ability to recognize the indicator icon .

In the embodiment of this application, the first image can be superimposed with background images during the generation process, such as superimposing brilliant background and ambient light, real-time navigation information, real-time media playback information, real-time weather and other network information to enhance visual night vision cameras. The screen, etc., rather than just the indicator icon on the black background in Figure 1, to enhance the user’s visual effect or expand the display content of the display device to meet the user’s needs. Therefore, in order to further improve the image processing capabilities of the neural network model, this application implements In an example, the historical image may be formed by superimposing the indicator icon and the historical background image, and the historical background image may be at least one of the above backgrounds, so that a historical image with at least one background can be obtained as training data. In this way, even if the background image is superimposed on the first image, it will not affect the recognition of the first image by the neural network model. Therefore, in the process of generating the first image in actual operation, more categories of background images can be superimposed without the need A black rectangular frame is set on the outside of the indicator icon, which is conducive to the diversification and beauty of the first image. Among them, the superposition of the indicator icon and the historical background image can be superimposed by DSS, which will not be described in detail here.

After monitoring the latter part of the first image, if the neural network model is used to determine that at least a part of the first image is incorrectly displayed, it means that there is an error in the generation process of the first image. At this time, if the first image is displayed, it will cause The displayed image cannot truly reflect the information of the first data, which reduces the functional safety of the system. Therefore, the neural network processor 102 can send the judgment result to the display device 200, so that the display device 200 stops displaying the first image, and the neural network processor 102 can also send to the MCU 400 or the main processor 300 connected to the neural network processor. According to the judgment result, the MCU 400 or the main processor 300 can generate a stop display instruction to prevent the display device 200 from displaying the first image. In addition, the MCU 400 or the main processor 300 can also generate alarms such as displaying alarm icons and displaying alarm voices. Information, thereby reminding the user that the display is malfunctioning, so that the user does not need to rely on the display content of the display device, avoids being misled by the wrong display content, and improves the display security to a certain extent.

Of course, in order to obtain accurate display content, the neural network processor 102 may also send an image generation instruction to the second display controller. The image generation instruction is used to instruct the second display controller to generate a second image, and the second image is used to replace the second display controller. An image is displayed by the display device, the second data used to generate the second image is generated by the MCU 400 connected to the neural network processor 102, and the second data and the first data are obtained based on the same detection data, that is, the first The first data and the second data have the same data source. When the first image is stored in the image buffer area in the memory 104, the generated second image can replace the first image and be stored in the image buffer area in the memory 104. In this way, the second display controller can be used to regenerate the image corresponding to the information to be displayed, so that the display device 200 can acquire and display the second image in the image buffer area, which prevents the display device 200 from directly displaying the incorrect first image. The problem caused, therefore, the display accuracy of the image is improved.

The second display controller 103 can be the above MCU 400 with functional safety certification capabilities or a high-end media processor with complex graphics functions. The two can have different image generation capabilities. Refer to Figures 3 and 4 for this The application embodiments provide schematic diagrams of two other image display monitoring devices. Among them, the high-end media processor as the second display controller 103 may include DSS or GPU. Of course, the second display controller 103 may also include both GPU and DSS. In order to distinguish between the first display controller 101 and the second display controller 101, The second display controller 103 may use the GPU in the first display controller 101 as the first GPU, the DSS as the first DSS, the GPU in the second display controller 103 as the second GPU, and the DSS as the second DSS. Wherein, the second display controller 103 can be connected to the memory 104, so that the second display controller 103 can store the generated second image in the image buffer area.

In the embodiment of the present application, the main processor 300 connected to the first display controller 101 and the MCU 400 connected to the neural network processor 102 can obtain the detection data at the same time, for example, through other buses or modules inside the vehicle. The main processor 300 can obtain the first data based on the detection data, thereby generating the first image, and the MCU 400 can implement the backup of the detection data and then use the backup detection data to generate the second image, thereby improving the accuracy of image generation.

In fact, the neural network processor 102 and the second display controller 103 are used to monitor the first image output by the first display controller 101 and generate the second image, so the second display controller 103 and the neural network The network processor 102 often needs to have a higher safety integrity level than the main processor 300 and the first display controller 101. For example, in the vehicle field, according to functional safety standards such as ISO 26262, the neural network processor 102 and the second display controller 103 need to implement the ASIL-B quality standard, while the first display controller 101 and the main processor 300 Only the QM quality standard needs to be implemented, which can improve the overall safety integrity of the system without increasing the cost of the main processor 300 and the first display controller 101. Therefore, in the embodiment of the present application, the MCU 400 may also obtain the detection data, and then obtain instructions based on the detection data, and send the obtained instructions to the main processor 300, and the main processor 300 responds to the received instructions from the MCU 400 The first data is generated, and then the first display controller 101 obtains the first data through the main processor 300.

When the safety integrity level of the second display controller 103 is higher than the safety integrity level of the first display controller 101, the reliability of the second image generated by the second display controller 103 is higher than that of the first display controller 101. The reliability of the first image, the possibility of errors in the second image is less than the possibility of errors in the first image, so displaying the second image is more reliable than displaying the first image. Of course, after storing the second image in the image buffer area, the neural network processor 102 can also be used to monitor the second image, and the monitoring process can refer to the monitoring process of the first image.

For ease of understanding, the image display monitoring device provided in the embodiments of the present application will be exemplarily described below in conjunction with specific scenarios.

As shown in Fig. 3, NPU can be used as neural network processor 102, MCU 400 can be used as second display controller 102 at the same time, CPU can be used as main processor 300, so the system includes CPU 300, first display controller 101, NPU 102. MCU 400, memory 104, and display device 200, where the arrow direction can reflect the flow of information between each device, that is, the first display controller 101 can obtain the first data from the CPU 300, and use the first data to generate the first data. The image is stored in the image buffer area in the memory 104, and the NPU 102 can obtain at least a part of the first image from the image buffer area in the memory 104, thereby using the neural network model therein to monitor at least a part of the first image. When it is determined that at least a part of the first image is incorrect, the NPU 102 can send an image generation instruction to the MCU 400, which can instruct the MCU 400 to generate a second image based on the image generation instruction, and store the generated second image in the image in the memory 104 In the buffer area, the display device 200 can display the second image.

As shown in FIG. 4, the NPU can be used as the neural network processor 102, and the CPU can be used as the main processor 300. Therefore, the system includes a CPU 300, a first display controller 101, an NPU 102, an MCU 400, a second display controller 103, The memory 104 and the display device 200, where the arrow direction can reflect the flow of information between each device, that is, the first display controller 101 can obtain the first data from the CPU 300, use the first data to generate the first image and store it in the memory 104 The NPU 102 can obtain at least a part of the first image from the image buffer area in the memory 104, thereby using the neural network model therein to monitor at least a part of the first image. When part of it is incorrect, the NPU 102 can send an image generation instruction to the second display controller 103 through the MCU 400, which can instruct the second display controller 103 to generate a second image based on the image generation instruction, and store the generated second image in In the image buffer area in the memory 104, the display device 200 can display the second image.

The above are only exemplary descriptions, and those skilled in the art can freely combine other forms of image monitoring systems according to the functions of each module, and no examples are given here.

In the embodiment of the present application, the first display controller 101, the neural network processor 102, and the second display controller 103 may be respectively provided in multiple chips. For example, the first display controller 101 may be integrated with the main processor 300. On the same chip, the neural network processor 102 and the MCU 400 can be integrated on the same chip; or the first display controller 101 can be integrated with the neural network processor 102 on the same chip; or the first display controller 101 can also be integrated with the first display controller 101. The two display controllers 103 are integrated on the same chip; or the first display controller 101 can also be integrated with the neural network processor 102 and the second display controller 103 on the same chip; or the neural network processor 102 can be integrated with the main processor The device 300 and the MCU 400 are integrated on the same chip, and the first display controller 101 and the second display controller 103 are integrated on the same chip. It is understandable that the more functional modules integrated on the same chip, the more conducive to reducing the overall hardware size, and thus the more conducive to reducing costs.

Based on the above description, the neural network processor 102 can monitor at least a part of the first image generated by the first display controller. The monitoring content is mainly whether the first image is displayed correctly, such as whether the display is complete, whether it is blocked, and whether it exists. Color error, etc. In actual operation, at least a part of the first image may have another problem, that is, the displayed icon is not the desired icon. For example, the first data indicates that a certain indicator icon is displayed, and the first image The indicator icon is not included in the indicator, but another indicator icon is generated. The neural network processor 102 in the previous embodiment cannot monitor whether the indicator icon is an icon that needs to be displayed. In this case, it is also easy to cause the first The image is displayed incorrectly, which affects the safety and integrity of the system.

Therefore, as a possible implementation, the neural network processor 102 in the embodiment of the present application may also obtain indication information from the MCU 400 in advance, and then monitor the indication icon in the first image based on the indication information and the neural network model. When the monitoring result is that at least a part of the first image is displayed correctly, it is further determined whether the correctly displayed indicator icon is the indicator icon that needs to be displayed, so as to further improve the accuracy of the image display. That is, the indication information may indicate the indication icon in the first image that needs to be displayed, so that the indication icon in the first image can be further monitored. Among them, the indication information and the first data are obtained based on the same detection data. The difference is that the first data is obtained by the main processor 300, while the MCU 400 obtains the indication information, because the MCU 400 is often more powerful than the main processor 300. A higher safety integrity level, so the indication information has higher reliability than the first data.

As another possible implementation manner, the neural network processor 102 in the embodiment of the present application may only monitor the content of the indicator icon, and the MCU 400 determines whether the indicator icon is an indicator icon that needs to be displayed. . Specifically, when the monitoring result of the neural network processor 102 is that at least a part of the first image is displayed correctly, the MCU 400 may determine whether the correctly displayed indicator icon is the indicator icon that needs to be displayed based on the obtained detection data, and the manner of determination It can be a neural network model or other image recognition methods, again without limitation.

The embodiment of the present application provides an image display monitoring device, which may include a first display controller and a neural network processor, wherein the first display controller may use the first data to generate a first image, and the first image is used in the display device The above display, but in fact, the first image may have a problem of generating errors, causing the first image to not correctly reflect the characteristics of the first data. Therefore, in the embodiments of the present application, a neural network processor may be used to obtain at least a part of the first image, and a neural network model may be used to determine whether at least a part of the first image is displayed correctly, so as to obtain a monitoring result of at least a part of the first image. The embodiment of the application uses a neural network model to monitor at least a part of the first image, and the related model can be freely changed, configured or upgraded online. Therefore, it is more flexible than the detection circuit of dedicated hardware, and the obtained monitoring results are more flexible. Reliability and pertinence are also higher, which in turn improves the accuracy of image display.

For ease of understanding, an image monitoring method provided in an embodiment of the present application will be specifically exemplified below. Referring to FIG. 5, the method may include the following steps:

S101: Generate a first image by using the first data by the first display controller. In the embodiment of the present application, the first display controller 101 may generate a first image based on the first data, where the first data used to generate the first image is data indicating image characteristics in the first image, for example, it may be based on The detection result generated from the detection data obtained by the detection device to be detected, the detection result may include an abnormal result and/or a normal result. When the first image includes the indicator icon, the first data includes data indicating the status information of the indicator icon in the first image, and the status information of the indicator icon may include on or off. Generally speaking, if the first data indicates the first image If the status of the indicator icon in the first image is on, the first image generated based on the first data includes a lit indicator icon, and if the first data indicates that the indicator icon in the first image is off, then it’s generated based on the first data The first image of does not include a lit indicator icon. For the manner in which the first display controller generates the first image based on the first data, reference may be made to the description of the foregoing system embodiment, which will not be repeated here.

S102. Acquire at least a part of the first image. In the embodiment of the present application, the neural network processor 102 may monitor at least a part of the first image, so as to obtain a monitoring result of at least a part of the first image. Specifically, the neural network processor 102 may input at least a part of the first image into the neural network model to obtain a judgment result of the correctness of at least a part of the first image as a monitoring result of at least a part of the first image. A process can be considered as matching at least a part of the first image with the preset image to obtain the matching degree. If the preset image is a correct image, the higher the matching degree, the higher the probability that at least a part of the first image is correct. If the preset image is an incorrect image, the higher the matching degree, the higher the probability that at least a part of the first image is incorrect.

In the embodiment of the present application, at least a part of the first image may be an indicator icon, and the monitoring of the indicator icon is beneficial to improve the functional safety of the system. Therefore, at least a part of the first image is an indicator area image including the indicator icon. , Where the indication area graphics can be rectangular or other shapes. For the manner of acquiring at least a part of the first image, reference may be made to the description of the foregoing system embodiment, which will not be repeated here.

S103: Use a neural network model to determine whether at least a part of the first image is displayed correctly, so as to obtain a monitoring result of at least a part of the first image. Generally speaking, since the first image is generated based on the first data, the first image can reflect the content of the first data, that is, the indicator icon in the first image and the state of the indicator icon in the first data The data should be consistent. At this time, it can be considered that the generated first image is correct. In this way, the display device 200 can display the first image, and the user can obtain the first data corresponding to the first image more vividly according to the first image. Content.

However, the first image may have problems of generating errors, for example, the indicator icons in the generated first image are incomplete, covered, and color wrong. Therefore, when the first display controller 101 generates the first image, an error may occur. If the wrong first image is displayed, the user may not be able to obtain accurate information contained in the first data in time. Therefore, in the embodiment of the present application, the neural network processor 102 can be used to monitor the display of the first image, and the neural network processor 102 can obtain at least a part of the first image, and use the neural network model to determine at least a part of the first image. Whether the display is correct to obtain the monitoring result of at least a part of the first image.

In the embodiment of the present application, the neural network processor 102 may monitor at least a part of the first image, so as to obtain a monitoring result of at least a part of the first image. Specifically, the neural network processor 102 may input at least a part of the first image into the neural network model to obtain a judgment result of the correctness of at least a part of the first image as a monitoring result of at least a part of the first image. A process can be considered as matching at least a part of the first image with the preset image to obtain the matching degree. If the preset image is a correct image, the higher the matching degree, the higher the probability that at least a part of the first image is correct. If the preset image is an incorrect image, the higher the matching degree, the higher the probability that at least a part of the first image is incorrect. For the training process of the neural network model in the neural network processor 102, reference may be made to the description in the foregoing system embodiment, which will not be repeated here.

Of course, in order to obtain accurate display content, the neural network processor 102 may also send an image generation instruction to the second display controller 103. The image generation instruction is used to instruct the second display controller 103 to generate a second image. Instead of the first image being displayed by the display device, the second data used to generate the second image is generated by the MCU 400 connected to the neural network processor 102, and the second data and the first data are obtained based on the same detection data, that is to say , The first data and the second data have the same data source. When the first image is stored in the image buffer area in the memory 104, the generated second image can replace the first image and be stored in the image buffer area in the memory 104. In this way, the second display controller can be used to regenerate the image corresponding to the information to be displayed, so that the display device 200 can acquire and display the second image in the image buffer area, which prevents the display device 200 from directly displaying the incorrect first image. The problem caused, therefore, the display accuracy of the image is improved.

As a possible implementation, the neural network processor 102 in the embodiment of the present application may also obtain indication information from the MCU 400 in advance, and then monitor the indication icon in the first image based on the indication information and the neural network model, so as to When the monitoring result is that at least a part of the first image is displayed correctly, it is further determined whether the correctly displayed indicator icon is the indicator icon that needs to be displayed. That is, the indication information may indicate the indication icon in the first image that needs to be displayed, so that the indication icon in the first image can be further monitored. Among them, the indication information and the first data are obtained based on the same detection data. The difference is that the first data is obtained by the main processor 300, while the MCU 400 obtains the indication information, because the MCU 400 is often more powerful than the main processor 300. A higher safety integrity level, so the indication information has higher reliability than the first data.

The embodiment of the application provides an image display monitoring method. The first display controller can use the first data to generate a first image. The first image is used for display on the display device. In fact, the first image may have a generation error. The problem caused the first image to not correctly reflect the characteristics of the first data. Therefore, in the embodiments of the present application, a neural network processor may be used to obtain at least a part of the first image, and a neural network model may be used to determine whether at least a part of the first image is displayed correctly, so as to obtain a monitoring result of at least a part of the first image. The embodiment of the application uses a neural network model to monitor at least a part of the first image, and the related model can be freely changed, configured or upgraded online. Therefore, it is more flexible than the detection circuit of dedicated hardware, and the obtained monitoring results are more flexible. Reliability and pertinence are also higher, which in turn improves the accuracy of image display.

Above, the image display monitoring method provided by the embodiment of the present application has been described in detail with reference to FIG. 5. Hereinafter, the image display monitoring device provided by the embodiment of the present application will be described in detail with reference to FIG. 6. It should be understood that the description of the device embodiment and the description of the method embodiment correspond to each other. Therefore, for the content that is not described in detail, please refer to the above method embodiment. For the sake of brevity, it will not be repeated here.

The display control module 110 is configured to generate a first image using the first data through the first display controller, and the first image is used for display on a display device; the processing module 120 is configured to obtain at least a part of the first image; The neural network model judges whether at least a part of the first image is displayed correctly, so as to obtain a monitoring result of at least a part of the first image.

The image display monitoring device may implement steps or processes executed by the processing device in the method corresponding to the embodiment of the present application, and the device may include a unit for executing the method executed by the processing device in the method in FIG. 5. In addition, each unit in the device and other operations and/or functions described above are used to implement the corresponding process of the method in FIG. 5.

It should be understood that the specific process for each unit to execute the foregoing corresponding steps has been described in detail in the foregoing method embodiment, and is not repeated here for brevity. Any of the above modules can be implemented by software, hardware or a combination of the two. The hardware may include various electronic circuits such as digital logic circuits or analog circuits, which are not limited in this embodiment. If the module is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solution of the present application essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , Including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), or magnetic disks or optical disks and other media that can store program codes. .

According to the image display monitoring method provided by the embodiment of the present application, the embodiment of the present application also provides a neural network processor, including a processor and a memory; the memory is used for storing computer programs or instructions; the processor is used for By executing the computer program or the instruction in the memory, the image display monitoring method of any one of the embodiments shown in FIG. 5 is implemented.

According to the image display monitoring method provided by the embodiments of the present application, the present application also provides a computer program product containing a computer program or instruction. The computer program product includes: computer program code, when the computer program code runs on a computer, The computer implements the image display monitoring method of any one of the embodiments shown in FIG. 5.

According to the image display monitoring method provided by the embodiments of the present application, the present application also provides a computer-readable medium that stores a program code, and when the program code runs on a computer, the computer realizes the method shown in FIG. 5 The image display monitoring method of any one of the embodiments is shown.

Those skilled in the art can clearly understand that, for the convenience and conciseness of the description, the specific working process of the above-described system, device, and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, device, and method can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or It can be integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

As mentioned above, the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions recorded in the embodiments are modified, or some of the technical features are equivalently replaced; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

An image display monitoring device, which is characterized in that it comprises:

A first display controller, configured to generate a first image using the first data, the first image being used for display on a display device;

A neural network processor is configured to obtain at least a part of the first image, and use a neural network model to determine whether at least a part of the first image is displayed correctly, so as to obtain a monitoring result of at least a part of the first image.
The device according to claim 1, wherein the neural network processor is further configured to: when the monitoring result is that at least a part of the first image is incorrectly displayed, send the image to the second display controller Generation instruction; the second display controller is used to generate a second image in response to the image generation instruction, and the second image is used to replace the first image for display on the display device.
The device according to claim 2, wherein the safety integrity level of the second display controller is higher than the safety integrity level of the first display controller.
The device according to claim 2 or 3, wherein the first display controller includes at least one of a first image processor GPU or a first display subsystem DSS, and the second display controller includes At least one of the second image processor GPU, the second display subsystem DSS, or the microcontroller MCU.
The device according to any one of claims 1-4, wherein at least a part of the first image comprises an indicator icon;

The device further includes: an MCU, configured to determine whether the indicator icon is an indicator icon that needs to be displayed when the monitoring result is that at least a part of the first image is displayed correctly.
The device according to any one of claims 1-4, wherein at least a part of the first image comprises an indicator icon;

The neural network processor is further configured to: when the monitoring result is that at least a part of the first image is displayed correctly, use the neural network model and the instruction information from the MCU to determine whether the instruction icon needs to be displayed Indicator icon.
The device according to any one of claims 1-6, further comprising:

The main processor is used to generate the first data;

The first display controller is specifically configured to obtain the first data from the main processor.
The device according to claim 7, wherein:

The main processor is further configured to obtain instructions from the MCU, and generate the first data in response to the instructions.
A method for image display monitoring, which is characterized in that it comprises:

Generating a first image by using the first data by the first display controller, the first image being used for display on a display device;

Acquiring at least a part of the first image;

A neural network model is used to determine whether at least a part of the first image is displayed correctly, so as to obtain a monitoring result of at least a part of the first image.
The method according to claim 9, wherein the method further comprises:

When the monitoring result is that at least a part of the first image is incorrectly displayed, sending an image generation instruction to the second display controller;

A second image is generated by the second display controller in response to the image generation instruction, and the second image is used to replace the first image for display on the display device.
The method according to claim 10, wherein the safety integrity level of the second display controller is higher than the safety integrity level of the first display controller.
The method according to any one of claims 9-11, wherein at least a part of the first image includes an indicator icon, and the method further comprises:

When the monitoring result is that at least a part of the first image is displayed correctly, it is determined whether the indicator icon is an indicator icon that needs to be displayed.
An image display monitoring device, characterized in that it comprises:

A display control module, configured to generate a first image by using the first data through the first display controller, and the first image is used for display on a display device;

The processing module is configured to obtain at least a part of the first image; use a neural network model to determine whether at least a part of the first image is displayed correctly, so as to obtain a monitoring result of at least a part of the first image.
A computer-readable storage medium, including a computer program or instruction, which when running on a computer, enables the computer to implement the image display monitoring method according to any one of claims 9-12.