CN112702540B - Embedded image acquisition processing method and system based on FPGA - Google Patents
Embedded image acquisition processing method and system based on FPGA Download PDFInfo
- Publication number
- CN112702540B CN112702540B CN202011525127.7A CN202011525127A CN112702540B CN 112702540 B CN112702540 B CN 112702540B CN 202011525127 A CN202011525127 A CN 202011525127A CN 112702540 B CN112702540 B CN 112702540B
- Authority
- CN
- China
- Prior art keywords
- data
- image data
- image
- full
- sampling data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/268—Signal distribution or switching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/22—Adaptations for optical transmission
Abstract
The invention discloses an embedded image acquisition processing method and system based on FPGA, wherein the method comprises the steps of randomly selecting image data of at least one digital camera from a plurality of digital cameras before the format conversion of the image data of the digital cameras, and using the image data as single sampling data, and recording the original image data and the coding identification of the image data by the single sampling data; extracting transmitted image data corresponding to the single sampling data and sending the image data to a processing unit, wherein the coding identification of the image data is used as a basis for extracting the image; acquiring single sampling data from a single-camera random storage unit, acquiring transmitted image data corresponding to the single sampling data from an image extraction unit, and detecting whether the single sampling data is interfered and the degree of the interference after transmission; and when the single sampling data or the full sampling data is interfered after transmission and the interference degree reaches a first-class threshold value, adjusting the working time sequence of the high-speed serial deserializer and the photoelectric transceiver.
Description
Technical Field
The invention relates to an embedded image acquisition processing method and system based on an FPGA.
Background
In the related prior art, a method for transmitting images of a plurality of digital cameras based on an FPGA is disclosed in chinese patent CN200910067516.7, which uses the FPGA as a controller to realize lumped long-distance transmission of image information captured by the digital cameras according to the following steps: a. respectively converting the LVDS-format image data sent by each digital camera into TTL-format image data, and respectively inputting the TTL-format image data into a designated storage area; b. when the byte number stored in the storage area reaches a preset value, sending a request signal; c. the system sets the flag bit of the corresponding channel according to the received request signal, packs the image data in the corresponding storage area according to the principle of 'first-in first-out', converts the image data into parallel data and sends the parallel data into a high-speed serial deserializer; d. the high-speed serial deserializer encodes the image data, converts the parallel data into high-speed serial data and sends the high-speed serial data to the photoelectric transceiver; e. the photoelectric transceiver converts the serial electric signal sent by the high-speed serializer/deserializer into an optical signal and transmits the optical signal for a long distance through an optical fiber; f. and after the transmission of one frame of image data is finished, resetting the corresponding channel zone bit and starting to process the next frame of image data. The technology can integrate and convert the LVDS format image data shot by dozens of digital cameras working simultaneously, and transmit the image data for a long distance in an optical fiber communication mode. The method has the advantages that long-distance transmission is realized by converting image signals (LVDS mode) output by a plurality of digital cameras into optical fiber channels, data relay is not performed, the number of transmission cables is reduced, the complexity of the system and the error rate of data transmission are reduced, and the maintainability of the system is improved. An FPGA is adopted as a system controller, and an on-chip integrated SERDES (serial deserializer) module and block RAM (random access memory) resource are utilized, so that an externally-extended high-speed SERDES chip and an FIFO (first-in first-out) chip are omitted, the system design is simplified, and the expandability of the system is improved; as shown in FIG. 1, the system corresponding to the image transmission method mainly comprises a plurality of digital cameras (1-N), a data format conversion unit, a storage area, a high-speed serializer/deserializer, a photoelectric transceiver, an optical fiber and a receiving end device which are electrically connected in sequence, the key points are a data format conversion unit, a storage area and a high-speed serial deserializer, wherein the data format conversion unit is used for respectively converting image data in LVDS format into image data in TTL format and respectively inputting the image data into the appointed storage area, the storage area supports the packing processing of the image data in the corresponding storage area under the request command, and the image data is converted into parallel data to be sent into a high-speed serial deserializer, the high-speed serializer-deserializer can encode image data, convert parallel data into high-speed serial data and send the high-speed serial data into the photoelectric transceiver, and the photoelectric transceiver is connected with the optical fiber and used for completing signal conversion and transmission processes.
In the image transmission technology based on the FPGA, there is a main problem that, in the prior art (method), a system simultaneously accommodates image data of tens of cameras, and performs multi-flow and multi-type operations such as data format conversion, buffering, parallel-serial conversion and the like before a transmission process, in the transmission process, a multiplexing signal of image data of multiple cameras may be simultaneously transmitted on a channel, so that mutual influence (interference) easily occurs between different camera image data; especially, in a specific application scenario, for example, when the shooting positions of tens of cameras are relatively close and are all used for image acquisition, the image source data acquired by adjacent cameras are relatively similar, and the positions, time sequences and encoding contents of code streams in corresponding transmission are also relatively close (the repetition rate of encoding contents is relatively high), in this case, the problem of interference between image data of different cameras is more serious.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides an embedded image acquisition processing method and system based on an FPGA.
In order to realize the purpose of the invention, the invention adopts the following technical scheme:
the embedded image acquisition processing method based on the FPGA comprises the following steps that a plurality of digital cameras (1-N) acquire image signals and then transmit the image signals to a data format conversion unit; the data format conversion unit converts the data format and temporarily buffers the image data in a storage area; the high-speed serial deserializer encodes the image data, converts the parallel data into high-speed serial data and sends the high-speed serial data to the photoelectric transceiver; the photoelectric transceiver completes the conversion of the image data signal and sends the converted image data signal to the optical fiber; the optical fiber completes image data signal transmission; the method is characterized by also comprising the steps that the optical fiber transmits the image data signal to receiving end equipment, and the receiving end equipment finishes receiving and processing the image data signal;
the single-camera random storage unit randomly selects image data of at least one digital camera from the plurality of digital cameras before format conversion of the image data of the digital cameras and uses the image data as single sampling data, and the single sampling data records original image data and coding identification of the image data;
the image extraction unit is used for extracting the transmitted image data corresponding to the single sampling data and sending the image data to the processing unit, and the coding identification of the image data is used as the basis for extracting the image;
the processing unit is used for acquiring single sampling data from the single-camera random storage unit, acquiring transmitted image data corresponding to the single sampling data from the image extraction unit, and detecting whether the single sampling data is interfered and the interference degree after transmission;
when the single sampling data or the full sampling data is interfered after transmission and the interference degree reaches a first-class threshold value, the control unit adjusts the working time sequence of the high-speed serial deserializer and the photoelectric transceiver.
Further, the embedded image acquisition processing method based on FPGA is characterized by also comprising the steps of,
before and after the 'single-camera random storage unit selects the image data of at least one digital camera from a plurality of digital cameras randomly as single sampling data' before the format conversion of the image data of the digital cameras, the full-camera random storage unit selects the image data randomly from all the digital cameras before the format conversion of the image data of the digital cameras and ensures that the image data of each digital camera is included, the image data is used as full sampling data, and the full sampling data records the original image data of each digital camera and the coding identification of the image data;
the image extraction unit extracts the transmitted image data corresponding to the full sampling data and sends the image data to the processing unit at the same time or before and after the time sequence of the image extraction unit, and the coding identification of the image data is used as the basis for extracting the image;
in a processing unit, acquiring single sample data from a single-camera random storage unit, acquiring transmitted image data corresponding to the single sample data from an image extraction unit, and then detecting whether the single sample data is interfered and the interference degree after transmission or not or before and after a time sequence, acquiring full sample data from a full-camera random storage unit, acquiring transmitted image data corresponding to the full sample data from the image extraction unit, and then detecting whether the full sample data is interfered and the interference degree after transmission;
when the single sampling data is interfered after transmission and the interfered degree reaches a first class threshold value, the control unit adjusts the working time sequence of the high-speed serial deserializer and the photoelectric transceiver.
Further, the embedded image acquisition processing method based on the FPGA further comprises the step that the control unit closes the work of the high-speed serial deserializer and the photoelectric transceiver when single-sampling data or full-sampling data are interfered after transmission and the interfered degree reaches a second threshold value.
The embedded image acquisition processing system based on the FPGA comprises a plurality of digital cameras (1-N), a data format conversion unit, a storage area, a high-speed serial deserializer, a photoelectric transceiver, an optical fiber and receiving end equipment which are electrically connected in sequence, wherein the digital cameras (1-N) are used for acquiring image signals; the data format conversion unit is used for converting data formats; the storage area is used for temporarily caching image data; the high-speed serial deserializer is used for coding image data, converting parallel data into high-speed serial data and transmitting the high-speed serial data to the photoelectric transceiver; the photoelectric transceiver is used for converting image data signals and sending the converted image data signals to the optical fiber; the optical fiber is used for transmitting image data signals; the receiving end equipment is used for receiving and processing image data signals and is characterized by further comprising a single-camera random storage unit and a full-camera random storage unit which are electrically connected with a plurality of digital cameras (1-N), a processing unit which is electrically connected with the single-camera random storage unit and the full-camera random storage unit, an image extraction unit and a control unit which are electrically connected with the processing unit, wherein the image extraction unit is also electrically connected with the receiving end equipment, and the control unit is also electrically connected with a high-speed serial deserializer and a photoelectric transceiver;
the single-camera random storage unit is used for randomly selecting image data of at least one digital camera from a plurality of digital cameras before format conversion of the image data of the digital cameras and using the image data as single sampling data, and the single sampling data records original image data and coding identification of the image data;
the all-camera random storage unit is used for randomly selecting image data from all digital cameras before format conversion of the image data of the digital cameras and ensuring that the image data of each digital camera is included, the image data is used as full sampling data, and the full sampling data records original image data of each digital camera and coding identification of the image data;
the image extraction unit is used for extracting the transmitted image data corresponding to the single sampling data and sending the image data to the processing unit, the image extraction unit is also used for extracting the transmitted image data corresponding to the full sampling data and sending the image data to the processing unit, and the coding identification of the image data is used as the basis for extracting the image;
the processing unit is used for acquiring single sampling data from the single-camera random storage unit, acquiring transmitted image data corresponding to the single sampling data from the image extraction unit, and then detecting whether the single sampling data is interfered and the interference degree after transmission; the control unit is used for controlling the working time sequence of the high-speed serializer and deserializer and the working time sequence of the photoelectric transceiver, and is also used for adjusting the working time sequence of the high-speed serializer and deserializer when single-sampling data or full-sampling data are interfered after being transmitted and the interfered degree reaches a first threshold.
Further, detecting whether the single sample data is interfered after transmission and the interference degree is specific, detecting a code stream change rate before and after transmission of the single sample data or detecting a change rate of display image pixels before and after transmission of the single sample data in the same data format.
Further, the processing unit is further configured to obtain the full-sampling data from the full-camera random access memory unit, obtain the transmitted image data corresponding to the full-sampling data from the image extraction unit, and then detect whether the full-sampling data is interfered and the degree of the interference after transmission.
Further, the detecting whether the full-sampling data is interfered after transmission and the degree of interference specifically includes:
classifying the full sampling data acquired from the full camera random storage unit and the corresponding images of the full sampling data acquired from the image extraction unit after transmission into a plurality of single images according to different cameras;
constructing a CNN image processing model, and extracting the characteristics among a plurality of single images after the full sampling data corresponds to image classification through the CNN image processing model;
extracting the characteristics among a plurality of single images after the corresponding image classification corresponding to the transmitted full sampling data through a CNN image processing model;
constructing a feature vector for the features among a plurality of single images after the images corresponding to the full sampling data are classified, performing point multiplication on the feature vectors among different single images, and performing joint learning and feature extraction to obtain first feature data;
correspondingly, feature vectors are also constructed for the features between a plurality of single images after the image classification corresponding to the transmitted full sampling data, then the feature vectors between different single images are subjected to point multiplication, and then joint learning and feature extraction are carried out to obtain second feature data;
whether the full sampling data is interfered after transmission and the interference degree are detected by comparing the first characteristic data with the second characteristic data.
Further, after the full-sampling data corresponds to the image classification, feature vectors of the features of a plurality of single images are constructed, then the feature vectors of different single images are subjected to point multiplication, then the point multiplication, the joint learning and the feature extraction are carried out, and then the feature vectors are input into a softmax function classifier which is trained to carry out classification operation to obtain first classification data; establishing a feature vector for the feature structure among a plurality of single images after the image classification corresponding to the transmitted full sampling data, performing point multiplication on the feature vectors among different single images, performing joint learning, and performing feature extraction, and then inputting the feature vectors into a softmax function classifier which is trained to perform classification operation to obtain second classification data; whether the full-sampling data is interfered after transmission and the interference degree are detected by comparing the first classified data and the second classified data.
Furthermore, the processing unit is also used for closing the work of the high-speed serializer-deserializer and the photoelectric transceiver when the single-sampling data or the full-sampling data is interfered after transmission and the interference degree reaches a second threshold value.
Furthermore, the processing unit is configured on the FPGA circuit.
The invention has the advantages that the single-camera random storage unit randomly selects the image data of at least one digital camera from a plurality of digital cameras before the format conversion of the image data of the digital cameras as single sampling data, then the processing unit detects whether the single sampling data is interfered and the interference degree according to the acquisition of the single sampling data and the acquisition of the image data after the transmission corresponding to the single sampling data from the image extraction unit, and then the control unit adjusts the working time sequence of the high-speed serializer/deserializer and the photoelectric transceiver after judging whether the single sampling data is interfered and the interference degree reaches a first threshold after the transmission, so that the interference degree of the data after the transmission can be directly reduced or even eliminated by reducing the working efficiency of the high-speed serializer/deserializer and the photoelectric transceiver when the data is interfered and the interference degree is higher after the transmission, thereby solving the problems in the background art and even the problems of the specific application scenarios in the background art. In addition, through the all-camera random storage unit, before the format conversion of the image data of the digital camera, the image data is randomly selected from all the digital cameras and the image data of each digital camera is ensured to be included, the image data is used as full sampling data, the full sampling data records the original image data of each digital camera and the coding identification of the image data, then the processing unit acquires the image data corresponding to the full sampling data from the image extraction unit according to the acquired full sampling data, detects whether the full sampling data is interfered and the interference degree after transmission, further adjusts the working time sequence of the high-speed serial deserializer and the photoelectric transceiver after judging whether the full sampling data is interfered and the interference degree reaches a first threshold after transmission, and thus the working efficiency of the high-speed serial deserializer and the photoelectric transceiver can be reduced in the case that the data is interfered and the interference degree is higher after transmission The interference degree of the data after transmission is directly reduced, even the interference is eliminated, so that the problems in the background technology and even the problems of specific application scenes in the background technology are solved. The full sampling data ensures the image data of each digital camera and records the original image data of each digital camera and the coding identification of the image data, so that the full sampling data has the global characteristic in the judgment, the image processing quality can be integrally reflected and the working efficiency of the high-speed serial deserializer and the photoelectric transceiver can be controlled, or more specifically, the control of the image processing through the single sampling data is only the control of the local characteristic, the control of the image processing through the full sampling data is the control of the global characteristic, and the quality of the image acquisition and processing can be more effectively improved through the combination of the local characteristic and the global characteristic.
Description of the drawings:
fig. 1 is a block diagram of a system corresponding to an image acquisition processing method in 200910067516.7 in the prior art;
FIG. 2 is a block diagram of the FPGA-based embedded image acquisition processing system of the present application;
fig. 3 is a flowchart illustrating the timing steps of the FPGA-based embedded image acquisition processing method according to the present application.
Detailed Description
In a specific implementation, as shown in fig. 3, the embedded image acquisition processing method based on FPGA in the present application includes the steps of t1 acquiring image signals by a plurality of digital cameras (1-N) and then transmitting the image signals to a data format conversion unit; t2 conversion of data format conversion unit data format and temporary buffering of image data in a storage area; t3 high-speed deserializer encodes the image data, converts the parallel data into high-speed serial data, and sends the high-speed serial data to the photoelectric transceiver; t4 photoelectric transceiver completes the conversion of image data signal and sends the converted image data signal to optical fiber; t5, the optical fiber completes image data signal transmission, the optical fiber transmits the image data signal to the receiving end equipment, and the receiving end equipment completes image data signal receiving and processing; t6 single camera random memory unit, selecting at least one digital camera image data from several digital cameras before format conversion of digital camera image data as single sample data, recording original image data and image data coding mark;
the t7 image extraction unit extracts the transmitted image data corresponding to the single sampling data and sends the image data to the processing unit, and the coding identification of the image data is used as the basis for extracting the image;
a t8 processing unit, which acquires single sample data from the single-camera random storage unit and acquires transmitted image data corresponding to the single sample data from the image extraction unit, and then detects whether the single sample data is interfered and the interference degree after transmission;
t9 when the single sample data is interfered and the interference degree reaches the first threshold, the control unit adjusts the working time sequence of the high-speed serial deserializer and the photoelectric transceiver.
In one embodiment with working time sequence, the embedded image acquisition processing method based on the FPGA comprises the following steps,
t1 a plurality of digital cameras (1-N) collect image signals and then transmit the image signals to the data format conversion unit; t6 single-camera random storage unit, selecting at least one image data of digital camera from several digital cameras before format conversion of image data of digital camera, and using the selected image data as single sample data, the single sample data recording original image data and coding identification of image data;
t2 conversion of data format conversion unit data format and temporary buffering of image data in a storage area; t3 high-speed deserializer encodes the image data, converts the parallel data into high-speed serial data, and sends the high-speed serial data to the photoelectric transceiver; t4 photoelectric transceiver completes the conversion of image data signal and sends the converted image data signal to optical fiber; t5, the optical fiber completes image data signal transmission, the optical fiber transmits the image data signal to the receiving end equipment, and the receiving end equipment completes image data signal receiving and processing; the t7 image extraction unit extracts the transmitted image data corresponding to the single sampling data and sends the image data to the processing unit, and the coding identification of the image data is used as the basis for extracting the image;
a t8 processing unit, which acquires single sample data from the single-camera random storage unit and acquires transmitted image data corresponding to the single sample data from the image extraction unit, and then detects whether the single sample data is interfered and the interference degree after transmission;
t9 when the single sample data is interfered after transmission and the interference degree reaches the first threshold, the control unit adjusts the working time sequence of the high-speed serial deserializer and the photoelectric transceiver; then, the t3 high-speed serial deserializer encodes the image data, converts the parallel data into high-speed serial data and sends the high-speed serial data to the photoelectric transceiver; t4 photoelectric transceiver completes the conversion of image data signal and sends the converted image data signal to optical fiber; t5 the optical fiber completes the image data signal transmission, the optical fiber transmits the image data signal to the receiving end device, and the receiving end device completes the image data signal receiving and processing.
In the implementation, the single-camera random storage unit randomly selects the image data of at least one digital camera from a plurality of digital cameras before the format conversion of the image data of the digital cameras is carried out and uses the image data as single sampling data, then the processing unit detects whether the single sampling data is interfered and the interference degree after the transmission according to the acquisition of the single sampling data and the acquisition of the image data after the transmission corresponding to the single sampling data from the image extraction unit, and then the control unit adjusts the working time sequence of the high-speed serializer deserializer and the photoelectric transceiver after judging whether the single sampling data is interfered and the interference degree reaches a first threshold value after the transmission, so that the interference degree of the data after the transmission can be directly reduced or even eliminated by reducing the working efficiency of the high-speed serializer deserializer and the photoelectric transceiver when the data is interfered and the interference degree is higher after the transmission, thereby solving the problems in the background art and even the problems of the specific application scenarios in the background art.
In a further embodiment, the method further comprises the steps of, before and after "the single-camera random storage unit, selecting image data of at least one digital camera from the plurality of digital cameras at random as single sampling data" before format conversion of the image data of the digital cameras, selecting image data randomly from all the digital cameras before format conversion of the image data of the digital cameras and ensuring that the image data of each digital camera is included as full sampling data, and recording original image data of each digital camera and an encoding identifier of the image data by the full sampling data;
before or after the time sequence of the ' image extraction unit ' extracting the transmitted image data corresponding to the single sampling data and sending the image data to the processing unit ', the image extraction unit extracting the transmitted image data corresponding to the full sampling data and sending the image data to the processing unit, wherein the coding identification of the image data is used as the basis for extracting the image;
in a processing unit, acquiring single sample data from a single-camera random storage unit, acquiring transmitted image data corresponding to the single sample data from an image extraction unit, and then detecting whether the single sample data is interfered and the interference degree after transmission or not or before and after a time sequence, acquiring full sample data from a full-camera random storage unit, acquiring transmitted image data corresponding to the full sample data from the image extraction unit, and then detecting whether the full sample data is interfered and the interference degree after transmission;
when the single sampling data or the full sampling data is interfered after transmission and the interference degree reaches a first-class threshold value, the control unit adjusts the working time sequence of the high-speed serial deserializer and the photoelectric transceiver.
In the implementation, on the basis of the implementation effect, the full-camera random storage unit randomly selects image data from all the digital cameras before format conversion of the image data of the digital cameras and ensures that the image data of each digital camera is included, the image data is used as full sampling data, the full sampling data records original image data of each digital camera and the coding identification of the image data, then the processing unit acquires the transmitted image data corresponding to the full sampling data from the image extraction unit according to the acquired full sampling data, detects whether the full sampling data is interfered and the interference degree after transmission, and then adjusts the working timing sequence of the high-speed serial deserializer and the photoelectric transceiver by the control unit after judging whether the full sampling data is interfered and the interference degree reaches a first-type threshold value after transmission, so that when the data is interfered and the interference degree is high after transmission, the high-speed serial deserializer and the photoelectric transceiver can be reduced The working efficiency of the photoelectric transceiver and the photoelectric transceiver directly reduces the interference degree of the data after transmission and even eliminates the interference, thereby solving the problems in the background technology and even solving the problems of specific application scenes in the background technology. In addition, the full sampling data ensures the image data of each digital camera and records the original image data of each digital camera and the coding identification of the image data, so that the full sampling data has the global characteristic in the judgment, the image processing quality and the working efficiency of the high-speed serial deserializer and the photoelectric transceiver can be integrally reflected, or more specifically, the control of the image processing through the single sampling data is only the control of the local characteristic, the control of the image processing through the full sampling data is the control of the global characteristic, and the quality of the image acquisition processing can be more effectively improved through the combination of the local characteristic and the global characteristic.
In a further embodiment, the method further comprises the step that the control unit turns off the operation of the high-speed serializer and deserializer and the photoelectric transceiver when the single-sample data or the full-sample data is interfered after transmission and the interference degree reaches a second type threshold value. In this implementation, the standard of the second type of threshold is generally higher than the standard of the first type of threshold, so that when single-sample data or full-sample data is interfered after transmission and the interference degree is higher, the work of the high-speed serializer and the photoelectric transceiver can be closed, and the image acquisition and processing with lower quality can be avoided.
The embodiment of the embedded image acquisition processing system based on the FPGA in the application, as shown in FIG. 2, comprises a plurality of digital cameras (1-N), a data format conversion unit, a storage area, a high-speed serial deserializer, a photoelectric transceiver, an optical fiber and receiving end equipment which are electrically connected in sequence, wherein the digital cameras (1-N) are used for acquiring image signals; the data format conversion unit is used for converting data formats; the storage area is used for temporarily caching image data; the high-speed serializer-deserializer is used for coding image data, converting parallel data into high-speed serial data and sending the high-speed serial data into the photoelectric transceiver; the photoelectric transceiver is used for converting image data signals and sending the converted image data signals to the optical fiber; the optical fiber is used for image data signal transmission; the receiving end equipment is used for receiving and processing image data signals, and also comprises a single-camera random storage unit and a full-camera random storage unit which are electrically connected with a plurality of digital cameras (1-N), a processing unit which is electrically connected with the single-camera random storage unit and the full-camera random storage unit, an image extraction unit and a control unit which are electrically connected with the processing unit, wherein the image extraction unit is also electrically connected with the receiving end equipment, and the control unit is also electrically connected with a high-speed serial deserializer and a photoelectric transceiver;
the single-camera random storage unit is used for randomly selecting image data of at least one digital camera from a plurality of digital cameras before format conversion of the image data of the digital cameras and using the image data as single sampling data, and the single sampling data records original image data and coding identification of the image data;
the all-camera random storage unit is used for randomly selecting image data from all digital cameras before format conversion of the image data of the digital cameras and ensuring that the image data of each digital camera is included, the image data is used as full sampling data, and the full sampling data records original image data of each digital camera and coding identification of the image data;
the image extraction unit is used for extracting the transmitted image data corresponding to the single sampling data and sending the image data to the processing unit, the image extraction unit is also used for extracting the transmitted image data corresponding to the full sampling data and sending the image data to the processing unit, and the coding identification of the image data is used as the basis for extracting the image;
the processing unit is used for acquiring single sampling data from the single-camera random storage unit, acquiring transmitted image data corresponding to the single sampling data from the image extraction unit, and then detecting whether the single sampling data is interfered and the degree of the interference after transmission; the control unit is used for controlling the working time sequence of the high-speed serializer de-serializer and the photoelectric transceiver and adjusting the working time sequence of the high-speed serializer de-serializer and the photoelectric transceiver when single-sampling data or full-sampling data are interfered after being transmitted and the interfered degree reaches a first-class threshold value.
The processing unit is configured on an FPGA circuit, and the FPGA circuit is a product further developed on the basis of programmable devices such as PAL, GAL, CPLD and the like. The FPGA circuit is used as a semi-custom circuit in the field of Application Specific Integrated Circuits (ASICs), not only overcomes the defects of custom circuits, but also overcomes the defect that the number of gate circuits of the original programmable device is limited.
In a more specific implementation, the detecting whether the single sample data is interfered after transmission and the degree of the interference are specific, the rate of change of the code stream before and after transmission of the single sample data or the rate of change of the pixels of the display image before and after transmission of the single sample data is detected under the same data format.
In a more specific implementation, the processing unit is further configured to obtain the full-sample data from the full-camera random access memory unit, obtain the transmitted image data corresponding to the full-sample data from the image extraction unit, and then detect whether the full-sample data is interfered and the degree of the interference after transmission.
In a more specific implementation, the detecting whether the full-sampling data is interfered after transmission and the degree of interference specifically includes: classifying the full sampling data acquired from the full camera random storage unit and the corresponding images of the full sampling data acquired from the image extraction unit after transmission into a plurality of single images according to different cameras;
constructing a CNN image processing model, and extracting the characteristics among a plurality of single images after the full sampling data corresponds to image classification through the CNN image processing model;
extracting the characteristics among a plurality of single images after the corresponding image classification corresponding to the transmitted full sampling data through a CNN image processing model;
constructing a feature vector for the features among a plurality of single images after the corresponding image classification of the full sampling data, performing point multiplication on the feature vectors among different single images, and performing joint learning and feature extraction to obtain first feature data;
correspondingly, feature vectors are also constructed for the features between a plurality of single images after the image classification corresponding to the transmitted full sampling data, then the feature vectors between different single images are subjected to point multiplication, and then joint learning and feature extraction are carried out to obtain second feature data;
whether the full sampling data is interfered after transmission and the interference degree are detected by comparing the first characteristic data with the second characteristic data. In implementation, the CNN image processing model is mainly a convolutional layer and a pooling layer in the convolutional neural network, so that the feature extraction capability of the convolutional neural network on multiple images can be exerted, the control of image processing through full sampling data is supported, the control of global characteristics is matched, and the quality of image acquisition and processing is improved more effectively.
In a more specific implementation, feature vectors are constructed for features among a plurality of single images after the images corresponding to the full-sampling data are classified, then point multiplication is carried out on the feature vectors among different single images, then joint learning is carried out, the feature vectors are extracted, and then the feature vectors are input into a softmax function classifier which is trained to carry out classification operation to obtain first classification data; establishing a feature vector for the feature structure among a plurality of single images after the image classification corresponding to the transmitted full sampling data, performing point multiplication on the feature vectors among different single images, performing joint learning, and performing feature extraction, and then inputting the feature vectors into a softmax function classifier which is trained to perform classification operation to obtain second classification data; whether the full-sampling data is interfered after transmission and the interference degree are detected by comparing the first classified data and the second classified data.
In a more specific implementation, the processing unit is further configured to turn off the operations of the high-speed serializer and deserializer and the optoelectronic transceiver when the single-sample data or the full-sample data is interfered after transmission and the interference degree reaches a second type threshold.
Claims (9)
1. The embedded image acquisition processing method based on the FPGA comprises the following steps that a plurality of digital cameras (1-N) acquire image signals and then transmit the image signals to a data format conversion unit; the data format conversion unit converts the data format and temporarily buffers the image data in a storage area; the high-speed serial deserializer encodes the image data, converts the parallel data into high-speed serial data and sends the high-speed serial data to the photoelectric transceiver; the photoelectric transceiver completes the conversion of the image data signal and sends the converted image data signal to the optical fiber; the optical fiber completes image data signal transmission; it is characterized by also comprising the following steps of,
the optical fiber transmits the image data signal to receiving end equipment, and the receiving end equipment receives and processes the image data signal;
the single-camera random storage unit randomly selects image data of at least one digital camera from the plurality of digital cameras before format conversion of the image data of the digital cameras and uses the image data as single sampling data, and the single sampling data records original image data and coding identification of the image data;
the image extraction unit is used for extracting the transmitted image data corresponding to the single sampling data and sending the image data to the processing unit, and the coding identification of the image data is used as the basis for extracting the image;
the processing unit is used for acquiring single sampling data from the single-camera random storage unit, acquiring transmitted image data corresponding to the single sampling data from the image extraction unit, and then detecting whether the single sampling data is interfered and the interference degree after transmission;
the method also comprises the step of carrying out the following steps,
before and after the single-camera random storage unit selects the image data of at least one digital camera from a plurality of digital cameras at random as single sampling data before the format conversion of the image data of the digital cameras, a full-camera random storage unit randomly selects the image data from all the digital cameras before the format conversion of the image data of the digital cameras and ensures that the image data of each digital camera is included as full sampling data, and the full sampling data records the original image data of each digital camera and the coding identification of the image data;
before or after the time sequence of the ' image extraction unit ' extracting the transmitted image data corresponding to the single sampling data and sending the image data to the processing unit ', the image extraction unit extracting the transmitted image data corresponding to the full sampling data and sending the image data to the processing unit, wherein the coding identification of the image data is used as the basis for extracting the image;
the processing unit acquires full-sampling data from the full-camera random storage unit and acquires transmitted image data corresponding to the full-sampling data from the image extraction unit at the same time or before and after a time sequence, and then detects whether the full-sampling data is interfered and the interference degree after transmission;
when the single sampling data and the full sampling data are interfered after transmission and the interference degree reaches a first-class threshold value, the control unit adjusts the working time sequence of the high-speed serial deserializer and the photoelectric transceiver.
2. The FPGA-based embedded image acquisition processing method of claim 1, further comprising the step of,
and the control unit closes the work of the high-speed serial deserializer and the photoelectric transceiver when the single-sample data or the full-sample data is interfered after transmission and the interference degree reaches a second-class threshold value.
3. The embedded image acquisition processing system based on the FPGA comprises a plurality of digital cameras (1-N), a data format conversion unit, a storage area, a high-speed serial deserializer, a photoelectric transceiver, an optical fiber and receiving end equipment which are electrically connected in sequence, wherein the digital cameras (1-N) are used for acquiring image signals; the data format conversion unit is used for converting data formats; the storage area is used for temporarily caching image data; the high-speed serializer-deserializer is used for coding image data, converting parallel data into high-speed serial data and sending the high-speed serial data into the photoelectric transceiver; the photoelectric transceiver is used for converting image data signals and sending the converted image data signals to the optical fiber; the optical fiber is used for image data signal transmission; the receiving end equipment is used for receiving and processing image data signals and is characterized by further comprising a single-camera random storage unit and a full-camera random storage unit which are electrically connected with a plurality of digital cameras (1-N), a processing unit which is electrically connected with the single-camera random storage unit and the full-camera random storage unit, an image extraction unit and a control unit which are electrically connected with the processing unit, wherein the image extraction unit is further electrically connected with the receiving end equipment, and the control unit is further electrically connected with a high-speed serial deserializer and a photoelectric transceiver;
the single-camera random storage unit is used for randomly selecting image data of at least one digital camera from a plurality of digital cameras before format conversion of the image data of the digital cameras and using the image data as single sampling data, and the single sampling data records original image data and coding identification of the image data;
the all-camera random storage unit is used for randomly selecting image data from all digital cameras before format conversion of the image data of the digital cameras and ensuring that the image data of each digital camera is included, the image data is used as full sampling data, and the full sampling data records original image data of each digital camera and coding identification of the image data;
the image extraction unit is used for extracting the transmitted image data corresponding to the single sampling data and sending the image data to the processing unit, the image extraction unit is also used for extracting the transmitted image data corresponding to the full sampling data and sending the image data to the processing unit, and the coding identification of the image data is used as the basis for extracting the image;
the processing unit is used for acquiring single sampling data from the single-camera random storage unit, acquiring transmitted image data corresponding to the single sampling data from the image extraction unit, and then detecting whether the single sampling data is interfered and the interference degree after transmission;
the control unit is used for controlling the working time sequence of the high-speed serializer and deserializer and the working time sequence of the photoelectric transceiver, and is also used for adjusting the working time sequences of the high-speed serializer and the photoelectric transceiver when single-sampling data and full-sampling data are interfered after being transmitted and the interfered degree reaches a first-class threshold value.
4. The FPGA-based embedded image acquisition processing system of claim 3, wherein the detection of whether the single sample data is interfered after transmission and the degree of the interference is specific is to detect a code stream change rate before and after transmission of the single sample data or a change rate of display image pixels before and after transmission of the single sample data in the same data format.
5. The FPGA-based embedded image acquisition processing system of claim 3, wherein the processing unit is further configured to obtain the full-sample data from the full-camera random access memory unit and obtain the transmitted image data corresponding to the full-sample data from the image extraction unit, and then detect whether the full-sample data is interfered and the degree of the interference after transmission.
6. The FPGA-based embedded image acquisition processing system of claim 5, wherein the detecting whether the full-sample data is interfered after transmission and the degree of interference specifically comprises:
classifying the full sampling data acquired from the full camera random storage unit and the corresponding images of the full sampling data acquired from the image extraction unit after transmission into a plurality of single images according to different cameras;
constructing a CNN image processing model, and extracting the characteristics among a plurality of single images after the full sampling data corresponds to image classification through the CNN image processing model;
extracting the characteristics among a plurality of single images after the corresponding image classification corresponding to the transmitted full sampling data through a CNN image processing model;
constructing a feature vector for the features among a plurality of single images after the corresponding image classification of the full sampling data, performing point multiplication on the feature vectors among different single images, and performing joint learning and feature extraction to obtain first feature data;
correspondingly, feature vectors are also constructed for the features between a plurality of single images after the image classification corresponding to the transmitted full sampling data, then the feature vectors between different single images are subjected to point multiplication, and then joint learning and feature extraction are carried out to obtain second feature data;
whether the full sampling data is interfered after transmission and the interference degree are detected by comparing the first characteristic data with the second characteristic data.
7. The FPGA-based embedded image acquisition processing system of claim 6, wherein the first classification data is obtained by constructing a feature vector for the features between a plurality of single images after the full-sampling data corresponds to image classification, performing point multiplication on the feature vectors between different single images, performing joint learning, performing feature extraction, and inputting the feature vectors into a trained softmax function classifier; establishing a feature vector for the feature structure among a plurality of single images after the image classification corresponding to the transmitted full sampling data, performing point multiplication on the feature vectors among different single images, performing joint learning, and performing feature extraction, and then inputting the feature vectors into a softmax function classifier which is trained to perform classification operation to obtain second classification data; whether the full-sampling data is interfered after transmission and the interference degree are detected by comparing the first classified data and the second classified data.
8. The FPGA-based embedded image acquisition processing system of claim 3, wherein the processing unit is further configured to turn off the operations of the high-speed serializer and the optoelectronic transceiver when the single-sample data or the full-sample data is interfered after transmission and the interference degree reaches a second-type threshold.
9. The FPGA-based embedded image acquisition processing system of claim 3, wherein the processing unit is configured on an FPGA circuit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011525127.7A CN112702540B (en) | 2020-12-22 | 2020-12-22 | Embedded image acquisition processing method and system based on FPGA |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011525127.7A CN112702540B (en) | 2020-12-22 | 2020-12-22 | Embedded image acquisition processing method and system based on FPGA |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112702540A CN112702540A (en) | 2021-04-23 |
| CN112702540B true CN112702540B (en) | 2022-08-26 |
Family
ID=75510003
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011525127.7A Active CN112702540B (en) | 2020-12-22 | 2020-12-22 | Embedded image acquisition processing method and system based on FPGA |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112702540B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113409920B (en) * | 2021-08-18 | 2021-12-21 | 明品云(北京)数据科技有限公司 | Data transmission management method and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101662671A (en) * | 2009-09-10 | 2010-03-03 | 中国科学院长春光学精密机械与物理研究所 | Method based on FPGA for transmitting images of multiple digital cameras |
| GB201818882D0 (en) * | 2018-11-20 | 2019-01-02 | Imagination Tech Ltd | GNSS Receiver |
| CN109526035A (en) * | 2018-10-24 | 2019-03-26 | 维沃移动通信有限公司 | A kind of communication means and mobile terminal |
| CN110739996A (en) * | 2019-11-15 | 2020-01-31 | 安徽中骄智能科技有限公司 | LED-based visible light communication transmission equipment |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104618697B (en) * | 2015-01-28 | 2017-11-24 | 中国科学院光电技术研究所 | A kind of full layout type Camera link based on FPGA turn optical fiber realtime graphic optical transmitter and receiver |
| JP2017028373A (en) * | 2015-07-16 | 2017-02-02 | 富士通株式会社 | Radio communication device |
-
2020
- 2020-12-22 CN CN202011525127.7A patent/CN112702540B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101662671A (en) * | 2009-09-10 | 2010-03-03 | 中国科学院长春光学精密机械与物理研究所 | Method based on FPGA for transmitting images of multiple digital cameras |
| CN109526035A (en) * | 2018-10-24 | 2019-03-26 | 维沃移动通信有限公司 | A kind of communication means and mobile terminal |
| GB201818882D0 (en) * | 2018-11-20 | 2019-01-02 | Imagination Tech Ltd | GNSS Receiver |
| CN110739996A (en) * | 2019-11-15 | 2020-01-31 | 安徽中骄智能科技有限公司 | LED-based visible light communication transmission equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112702540A (en) | 2021-04-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101662671B (en) | Method based on FPGA for transmitting images of multiple digital cameras | |
| CN107071520B (en) | Method for realizing CoaXPres high-speed image interface protocol IP | |
| CN112702540B (en) | Embedded image acquisition processing method and system based on FPGA | |
| CN111757128B (en) | Video coding system | |
| CN110896431B (en) | Non-compression high-definition video transmission method and system | |
| CN102014277A (en) | Video monitoring system, video monitoring method and spherical camera | |
| KR20160134623A (en) | An IP Transmitting System For CCTV Video Signal | |
| CN103167322A (en) | Ethernet-based image transmitting/receiving system | |
| KR20020047069A (en) | Device and Method for transmitting picture data | |
| CN109495707B (en) | High-speed video acquisition and transmission method | |
| US20140022383A1 (en) | Surveillance system, image compression serializer and image decompression deserializer | |
| CN114363626A (en) | Satellite-borne ultrahigh-speed image compression system | |
| CN103705260A (en) | Data transmitting system of digital medical imaging device based on optical fiber communications | |
| CN105554592A (en) | Method and system for collecting and transmitting high frame rate video image | |
| CN105100813A (en) | Video image preprocessing method and apparatus | |
| CN103974018A (en) | Method for converting Camera Link into SD/HD-SDI on basis of FPGA | |
| JP6840049B2 (en) | Video / packet converter, packet / packet converter and program | |
| CN100571385C (en) | Chip chamber data and message same frame transmission system and method based on the chip video mouth | |
| CN219577134U (en) | Industrial vision detecting system | |
| CN102340656A (en) | Monitoring terminal, monitoring system, monitoring data transmission method and receiving method | |
| CN107249113B (en) | Signal extension method and system | |
| CN201667698U (en) | High-definition video monitoring system based on IP network | |
| CN218825618U (en) | Video data processing device and vehicle | |
| CN205092920U (en) | High resolution video transmission system | |
| CN104349016A (en) | High-definition image data transmission method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220808 Address after: Room K2-4, 5th Floor, Building 2, International Enterprise Center, No. 1, Guanshan 2nd Road, Donghu New Technology Development Zone, Wuhan City, Hubei Province 430000 (Free Trade Zone Wuhan Area) Applicant after: Wuhan Cool Nuo Technology Co., Ltd. Address before: 311831 no.360 Doumen, Luoshan new village, Huandong street, Zhuji City, Shaoxing City, Zhejiang Province Applicant before: Chen Wangfang |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |