CN111917974B

CN111917974B - FPGA-based video processing system, method, device and medium

Info

Publication number: CN111917974B
Application number: CN202010586794.XA
Authority: CN
Inventors: 姜凯; 刘强; 王子彤; 金长新
Original assignee: Shandong Inspur Scientific Research Institute Co Ltd
Current assignee: Shandong Inspur Scientific Research Institute Co Ltd
Priority date: 2020-06-24
Filing date: 2020-06-24
Publication date: 2022-04-15
Anticipated expiration: 2040-06-24
Also published as: CN111917974A

Abstract

The application discloses a video processing system, method, device and medium based on FPGA, the system includes: the sensor unit is used for acquiring corresponding sensor data; the video acquisition unit is used for acquiring video data; the FPGA unit is used for controlling the video acquisition unit to acquire video when receiving the sensor data; and performing image processing on the video data; when the existence of the target is determined, sending video data and/or an image processing result to a server; and the server is used for carrying out corresponding processing. When determining the preset target, a preliminary detection is first performed by the sensor unit. If the target is detected by the preliminary detection, the video acquisition unit is controlled to carry out video acquisition, so that the consumption of resources can be effectively saved. And the collected video is subjected to primary image processing, whether a target exists in the video is determined, if yes, the video data are sent to the server, computing and storage resources of the cloud can be effectively reduced, and working efficiency of the cloud is improved.

Description

FPGA-based video processing system, method, device and medium

Technical Field

The present application relates to the field of video processing, and in particular, to a system, method, device, and medium for video processing based on an FPGA.

Background

The Field Programmable Gate Array (FPGA) is a product of further development on the basis of Programmable devices such as PAL and GAL, which not only solves the deficiency of the customized circuit, but also overcomes the disadvantage of limited Gate circuit number of the original Programmable device.

In the prior art, an FPGA and a corresponding video capture device are generally used as front-end devices in each application to perform operations such as video capture. However, the prior art still has the following disadvantages: when the front-end equipment collects videos, the collected videos are processed and stored, and a large amount of cloud computing and storage resources are occupied.

Disclosure of Invention

In order to solve the above problem, the present application provides a video processing system based on an FPGA, including: the sensor unit is used for acquiring corresponding sensor data; the video acquisition unit is used for acquiring video data; the FPGA unit is connected with the sensor unit and the video acquisition unit and is used for controlling the video acquisition unit to acquire video when receiving the sensor data, wherein the sensor data represents that a preset target is detected; performing image processing on the video data acquired by the video acquisition unit to determine whether the target exists in the video data; when the target is determined to exist, sending the video data and/or the image processing result to a server; and the server is used for receiving the video data and/or the image processing result sent by the FPGA unit so as to perform corresponding processing.

In one example, the FPGA unit includes a processing system PS terminal and a programmable logic PL terminal: the PS end is connected with the sensor unit and used for controlling the video acquisition unit to acquire video and controlling the PL end to start working when receiving the sensor data; the PL terminal is connected with the video acquisition unit and used for carrying out image processing on the video data acquired by the video acquisition unit so as to determine whether the target exists in the video data and feeding back an image processing result and/or the video data to the PS terminal; and the PS terminal is also used for sending the image processing result and/or the video data to a server when the target is determined to exist according to the image processing result.

In one example, the PS terminal includes: the control module is connected with the sensor unit and used for carrying out corresponding data processing and control; the PS terminal memory module is connected with the control module and is used for providing operation and storage spaces of an operating system and an application program; the DMA module is connected with the control module and is used for data transmission; and the network controller is connected with the control module and is used for network communication.

In one example, the PL side includes: the ISP module is connected with the video acquisition unit and is used for performing original processing on the video data; the front-end video control logic is connected with the ISP module and is used for controlling a processing strategy in the image processing process; video processing logic, connected to the front-end video control logic, for performing target detection in an image of the video data; an AXI bus connected with the front-end video control logic and used for information transmission; and the PL terminal memory module is connected with the front-end video control logic and is used for providing an algorithm model, model parameters and a cache space of the video data.

On the other hand, the present application further provides an FPGA-based video processing method, which is applied to the FPGA-based video processing system according to any of the above examples, and the method includes: the method comprises the steps that an FPGA unit receives sensor data sent by a sensor unit, wherein the sensor data represent that the sensor unit detects a preset target; controlling a video acquisition unit to acquire video and receiving video data sent by the video acquisition unit; performing image processing on the video data to determine that the target exists; and sending the image processing result and/or the video data to a server.

In one example, the FPGA unit includes a PS terminal and a PL terminal, the PS terminal is connected with the sensor unit, and the PL terminal is connected with the video acquisition unit; control video acquisition unit and carry out video acquisition, and receive the video data that video acquisition unit sent, include: controlling a video acquisition unit to acquire video through the PS terminal and controlling the PL terminal to start working; receiving video data sent by the video acquisition unit through the PL terminal; performing image processing on the video data to determine that the target exists, including; performing image processing on the video data through the PL end to determine that the target exists; and feeding back an image processing result and/or the video data to the PS end through the PL end.

In one example, before the FPGA unit receives the sensor data transmitted by the sensor unit through the PS terminal, the method further includes: setting the PL end to be in a non-working state, and acquiring corresponding sensor data through a sensor unit; after the FPGA unit receives the sensor data sent by the sensor unit, the method further includes: and interrupting the connection between the sensor unit and the external interrupt, and awakening the PS terminal in a standby mode based on the external interrupt.

In one example, the video data is image processed, and when it is determined that the target does not exist, the method further comprises: and feeding back an image processing result to the PS end through the PL end so as to enable the PS end to enter a standby mode and control the PL end to stop working.

On the other hand, the present application further provides an FPGA-based video processing apparatus, which is applied to the FPGA-based video processing system according to any of the above examples, and the apparatus includes: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to: the method comprises the steps that an FPGA unit receives sensor data sent by a sensor unit, wherein the sensor data represent that the sensor unit detects a preset target; controlling a video acquisition unit to acquire video and receiving video data sent by the video acquisition unit; performing image processing on the video data to determine that the target exists; and sending the image processing result and/or the video data to a server.

In another aspect, the present application further provides a non-volatile computer storage medium for FPGA-based video processing, storing computer-executable instructions, for use in the FPGA-based video processing system according to any of the above examples, where the computer-executable instructions are configured to: the method comprises the steps that an FPGA unit receives sensor data sent by a sensor unit, wherein the sensor data represent that the sensor unit detects a preset target; controlling a video acquisition unit to acquire video and receiving video data sent by the video acquisition unit; performing image processing on the video data to determine that the target exists; and sending the image processing result and/or the video data to a server.

The video processing system based on the FPGA can bring the following beneficial effects:

when determining the preset target, a preliminary detection is first performed by the sensor unit. If the target is detected by the preliminary detection, the video acquisition unit is controlled to carry out video acquisition, so that the consumption of resources can be effectively saved. And the collected video is subjected to primary image processing, whether a target exists in the video is determined, if yes, the video data are sent to the server, computing and storage resources of the cloud can be effectively reduced, and working efficiency of the cloud is improved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

fig. 1 is a schematic flowchart of a video processing method based on an FPGA in an embodiment of the present application;

FIG. 2 is a schematic diagram of an FPGA-based video processing device according to an embodiment of the present application;

FIG. 3 is a block diagram of an FPGA-based video processing system according to an embodiment of the present application;

fig. 4 is a detailed structural diagram of the FPGA-based video processing system in the embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.

As shown in fig. 3, an embodiment of the present application provides an FPGA-based video processing system, which includes: sensor unit, video acquisition unit and remote server. Of course, the system also includes a power supply, which is not described herein.

The sensor unit is mainly used for acquiring corresponding sensor data, and can adopt corresponding sensors based on different scenes, different working requirements and different target detection in the actual working process, for example, an infrared sensor, a heat sensor and the like can be used for target detection. One or more sensors may be provided in the sensor unit, or one or more sensors may be provided, which is not limited herein. In addition, the sensor here may include a sensor having a corresponding processing and calculating function, and may also include a sensor having only a detecting function, which is not limited herein.

The video acquisition unit is mainly used for acquiring video data, and specifically may be a device having a camera shooting function, such as a camera, and the like, which is not described herein again.

And the FPGA unit is connected with the sensor unit and the video acquisition unit and is mainly used for processing and sending data and controlling the sensor unit and the video acquisition unit.

Specifically, when the sensor data sent by the sensor unit is received, if the sensor data indicates that a preset target has been detected, the video capture unit may be controlled to start video capture at this time. Wherein, when the sensors in the sensor unit have corresponding calculation processing functions, the sensor data can be received at the moment, that is, the preset target is detected. If the sensor only has a detection function, the data sent by the sensor unit can be received in real time, and the FPGA unit carries out processing and analysis to determine whether a preset target is detected. Here, when it is determined that the target is detected by the data transmitted by the sensor, the data may be referred to as sensor data.

After the target is determined to be detected through the sensor data, the video acquisition unit can be controlled to start working. After the video acquisition unit works, corresponding video data can be acquired, and the acquired video data is sent to the FPGA unit in real time. Since the target detected by the sensor is only a rough detection process, there is a possibility of erroneous judgment, and video data including the target may not be collected at the present stage. Thus, the FPGA unit can now perform image processing on the received video data to determine whether a target is present in the video data.

Then, if it is determined that the target exists in the video data, the video data and/or the image processing result may be transmitted to the server. The image processing result here may include video data processed by structuring, for example, a structured label of the video data after the structuring, a position of the target in the video, and the like.

And the server is used for receiving the video data and/or the image processing result sent by the FPGA unit and carrying out corresponding processing based on the video data and/or the image processing result. Different processes are performed based on different programs and different requirements, and are not limited.

In one embodiment, as shown in fig. 4, the FPGA includes a Processing System (PS) terminal and a Programmable Logic terminal (PL), which may be a heterogeneous architecture computing System composed of a PS terminal and a PL terminal. The PS terminal is mainly used for processing system operation, application program operation, image processing result caching and the like, and the PL terminal is mainly used for video caching, target detection algorithm model, parameter caching and the like.

Specifically, in the daily work process, when the sensor data is not received, the PL end may suspend working and enter a sleep state, while the PS end may only enable a part of functions to receive the sensor data and perform corresponding processing, and the entire video processing system is in a low-power consumption standby mode. The PS end is connected with the sensor unit, and can control the video acquisition unit to acquire video and control the PL end to start working when receiving sensor data.

The PL end is connected with the video acquisition unit, when the video acquisition unit starts to work to acquire videos, the collected video data can be sent to the PL end, the PL end can cache the video data, and image processing is carried out on each frame of image in the video data to determine whether a target exists in the video data. If the target exists, the image processing result and/or the video data can be fed back to the PS terminal and sent to the server by the PS terminal. If the target does not exist, the image processing result can be fed back to the PS end only, the PS end does not need to send the image processing result to the server, the PL end can be controlled to enter the dormant state again, the PS end is controlled to start partial functions, and the whole video processing system enters the low-power-consumption standby module again.

Further, as shown in fig. 4, the PS terminal includes: the system comprises a control module, a PS (packet switched) end memory module, a DMA (direct memory access) module and a network controller. The control module is connected with the sensor unit and is mainly used for carrying out corresponding data processing and control. For example, it can be used to receive sensor data, perform analysis processing on the sensor data, control the operating state of the PL side, and the like. And the PS terminal memory module is connected with the control module and is mainly used for providing running and storage spaces of an operating system and an application program. The DMA module is generally called Direct Memory Access (DMA), is connected to the control module, and is mainly used for high-speed data transmission. The network controller is mainly used for network communication between the PS terminal and the server.

The PL side comprises an ISP module, a front-end video control logic, a video processing logic, an AXI bus and a PL side memory module. The ISP module is called Image Signal Processing (ISP), and is connected to the video capture unit, and is mainly used for performing original Processing on the video data captured by the video capture unit, such as correcting and compensating the video data. The front-end video control logic is connected with the ISP module, and is mainly used for controlling a processing strategy in the image processing process, for example, performing image processing on each frame of data in video data, or performing image processing on frames. The video processing logic is connected with the front-end video control logic and is used for carrying out image processing, namely target detection, on the selected image in the video data. The video processing logic may include convolution, pooling, Relu, and FIFO portions, and target detection is performed by training corresponding image processing models, which is not described herein again. The AXI bus whole-course Advanced eXtensible Interface is connected with the front-end video control logic and is mainly used for information transmission. And the PL terminal memory module is connected with the front-end video control logic and is mainly used for providing an algorithm model, model parameters and a cache space of video data.

The embodiment of the present application further provides a video processing method based on an FPGA, which is applied to the video processing system in any one of the above embodiments, and as shown in fig. 1, the method includes:

s101, receiving sensor data sent by a sensor unit by an FPGA unit, wherein the sensor data represent that the sensor unit detects a preset target.

When the video processing system works, the sensor unit works to perform primary detection on a preset target. When a preset target is detected, corresponding sensor data indicating that the preset target has been detected may be sent to the FPGA unit. The configuration of the sensor unit, and several cases with or without calculation processing capability have been explained in the above embodiments, and are not described herein again.

Specifically, when the FPGA unit includes a PS terminal and a PL terminal, the PS terminal may receive sensor data sent by the sensor unit, and the PS terminal performs corresponding data processing or data analysis. In the daily working process, the PL terminal may be set to be in an off state, and the PS terminal may be set to be in a low power consumption standby mode. For example, when the control module in the PS side is a multi-core processor, only one core may be in an operating state. After receiving the sensor data, the PS terminal may interrupt the connection between the sensor unit and the PS terminal, and wake up the PS terminal in the standby mode by means of external interrupt, so as to perform subsequent operations. When the target is not retrieved in daily life, the whole system is in a low-power consumption state, and only when the sensor data is received and the target is detected with a high probability, the whole system works, so that the consumption of resources can be greatly saved.

S102, controlling a video acquisition unit to acquire video and receiving video data sent by the video acquisition unit.

After the FPGA unit receives the sensor data, the FPGA unit can control the video acquisition unit to acquire videos, and after the video acquisition unit acquires the videos, the FPGA unit receives the video data fed back by the video acquisition unit.

Specifically, when the FPGA unit includes a PS terminal and a PL terminal, the PS terminal may control the video acquisition unit to perform video acquisition, and control the PL terminal to start working, and then the PL terminal receives video data sent by the video acquisition module.

S103, carrying out image processing on the video data and determining that the target exists.

And S104, sending the image processing result and/or the video data to a server.

The received video data is then image processed and it is then determined whether a target is present. If a target is present, the image processing results and/or video data may be sent to a server, which may perform further processing or applications. If the data does not exist, the data does not need to be sent to the server, and the storage space of the server is saved.

Specifically, when the FPGA unit includes a PS terminal and a PL terminal, image processing may be performed by the PL terminal to determine whether a preset target exists in the video data. If the image processing result and/or the video data exist, the PL terminal feeds back the image processing result and/or the video data to the PS terminal, and the PS terminal sends the image processing result and/or the video data to the server. If the sensor data does not exist, the PL end only needs to feed back the image processing result to the PS end, the PS end does not need to feed back data to the server, then the PL end can be controlled to stop working, the PS end is controlled to enter a standby mode, and the whole video processing system enters a low-power-consumption mode to save resources until the sensor data is received again. Of course, when performing image processing on the video data, the image processing may be performed sequentially for each frame of image in the video data until it is determined that no target exists, and details are not repeated again.

As shown in fig. 2, an embodiment of the present application further provides an FPGA-based video processing apparatus, which is applied to an FPGA-based video processing system according to any of the above embodiments, where the apparatus includes:

at least one processor; and the number of the first and second groups,

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to:

the method comprises the steps that an FPGA unit receives sensor data sent by a sensor unit, wherein the sensor data represent that the sensor unit detects a preset target;

controlling a video acquisition unit to acquire video and receiving video data sent by the video acquisition unit;

performing image processing on the video data to determine that the target exists;

and sending the image processing result and/or the video data to a server.

The embodiment of the present application further provides a non-volatile computer storage medium for video processing based on FPGA, which stores computer executable instructions, and is applied to the video processing system based on FPGA according to any one of the above embodiments, where the computer executable instructions are set as:

and sending the image processing result and/or the video data to a server.

The embodiments in the present application are described in a progressive manner, and the same and similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the device and media embodiments, the description is relatively simple as it is substantially similar to the method embodiments, and reference may be made to some descriptions of the method embodiments for relevant points.

The device and the medium provided by the embodiment of the application correspond to the method one to one, so the device and the medium also have the similar beneficial technical effects as the corresponding method, and the beneficial technical effects of the method are explained in detail above, so the beneficial technical effects of the device and the medium are not repeated herein.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. An FPGA-based video processing system, comprising:

the sensor unit is used for acquiring corresponding sensor data;

the video acquisition unit is used for acquiring video data;

the FPGA unit is connected with the sensor unit and the video acquisition unit and is used for controlling the video acquisition unit to acquire video when receiving the sensor data, wherein the sensor data represents that a preset target is detected; and are

Performing image processing on the video data acquired by the video acquisition unit to determine whether the target exists in the video data; and are

When the target is determined to exist, sending the video data and/or the image processing result to a server;

the server is used for receiving the video data and/or the image processing result sent by the FPGA unit so as to perform corresponding processing;

the FPGA unit comprises a processing system PS end and a programmable logic PL end:

the PL terminal is in a dormant state when the sensor data is not received;

the PS terminal is connected with the sensor unit, only enables partial functions and is used for receiving the sensor data; the PL terminal is used for controlling the video acquisition unit to acquire video and controlling the PL terminal to start working when receiving the sensor data;

the FPGA unit is used for carrying out image processing on the video data through the PL end to determine that the target exists;

feeding back an image processing result and/or the video data to the PS terminal through the PL terminal;

before receiving sensor data sent by a sensor unit through the PS end, setting the PL end to be in a non-working state, and acquiring corresponding sensor data through the sensor unit;

after the FPGA unit receives the sensor data sent by the sensor unit, the connection between the external interrupt and the sensor unit is interrupted, and the PS end in the standby mode is awakened based on the external interrupt;

and carrying out image processing on the video data, and feeding back an image processing result to the PS end through the PL end when the target does not exist, so that the PS end enters a standby mode and is controlled to stop working.

2. The system according to claim 1, wherein the PL side is connected to the video capture unit, and configured to perform image processing on the video data captured by the video capture unit to determine whether the target exists in the video data, and feed back an image processing result and/or the video data to the PS side;

and the PS terminal is also used for sending the image processing result and/or the video data to a server when the target is determined to exist according to the image processing result.

3. The system of claim 2, wherein the PS end comprises:

the control module is connected with the sensor unit and used for carrying out corresponding data processing and control;

the PS terminal memory module is connected with the control module and is used for providing operation and storage spaces of an operating system and an application program;

the DMA module is connected with the control module and is used for data transmission;

and the network controller is connected with the control module and is used for network communication.

4. The system of claim 2, wherein the PL side comprises:

the ISP module is connected with the video acquisition unit and is used for performing original processing on the video data;

the front-end video control logic is connected with the ISP module and is used for controlling a processing strategy in the image processing process;

video processing logic, connected to the front-end video control logic, for performing target detection in an image of the video data;

an AXI bus connected with the front-end video control logic and used for information transmission;

and the PL terminal memory module is connected with the front-end video control logic and is used for providing an algorithm model, model parameters and a cache space of the video data.

5. An FPGA-based video processing method applied to an FPGA-based video processing system according to any one of claims 1 to 4, the method comprising:

sending the image processing result and/or the video data to a server;

the FPGA unit comprises a PS end and a PL end, the PS end is connected with the sensor unit, and the PL end is connected with the video acquisition unit; the PL terminal is in a dormant state when the sensor data is not received; the PS terminal only enables partial functions and is used for receiving the sensor data;

control video acquisition unit and carry out video acquisition, and receive the video data that video acquisition unit sent, include:

controlling a video acquisition unit to acquire video through the PS terminal and controlling the PL terminal to start working;

receiving video data sent by the video acquisition unit through the PL terminal;

performing image processing on the video data to determine that the target exists, including;

performing image processing on the video data through the PL end to determine that the target exists;

before the FPGA unit receives the sensor data sent by the sensor unit through the PS terminal, the method further includes:

setting the PL end to be in a non-working state, and acquiring corresponding sensor data through a sensor unit;

after the FPGA unit receives the sensor data sent by the sensor unit, the method further includes:

connecting an external interrupt with the sensor unit, and waking up the PS terminal in a standby mode based on the external interrupt;

performing image processing on the video data, and when determining that the target does not exist, the method further comprises:

and feeding back an image processing result to the PS end through the PL end so as to enable the PS end to enter a standby mode and control the PL end to stop working.

6. An FPGA-based video processing apparatus for use in an FPGA-based video processing system according to any one of claims 1-4, said apparatus comprising:

at least one processor; and the number of the first and second groups,

sending the image processing result and/or the video data to a server;

after receiving the sensor data sent by the sensor unit, the FPGA unit further comprises:

performing image processing on the video data, and when determining that the target does not exist, further comprising:

7. A non-transitory computer storage medium for FPGA-based video processing, storing computer-executable instructions, for use in an FPGA-based video processing system according to any one of claims 1-4, the computer-executable instructions being configured to:

sending the image processing result and/or the video data to a server;