CN112235578A - Multi-mode high-speed hyperspectral image parallel acquisition and processing method - Google Patents
Multi-mode high-speed hyperspectral image parallel acquisition and processing method Download PDFInfo
- Publication number
- CN112235578A CN112235578A CN202010954433.6A CN202010954433A CN112235578A CN 112235578 A CN112235578 A CN 112235578A CN 202010954433 A CN202010954433 A CN 202010954433A CN 112235578 A CN112235578 A CN 112235578A
- Authority
- CN
- China
- Prior art keywords
- code stream
- stream data
- data
- image
- group
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/436—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements
Landscapes
- Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention relates to a parallel image acquisition and processing method, in particular to a parallel multi-mode high-speed hyperspectral image acquisition and processing method, which solves the problems that the existing image acquisition system has single function and limited transmission rate, can not simultaneously process eight kinds of image data, including original image data, compressed data, half-view field and full-view field combination, which correspond to visible light and infrared light, and is difficult to meet the ground test requirement of space load. The acquisition processing method comprises the following steps: step 1: collecting data; if the collected image is the complete original data of the frame of image, executing the step 5; if a group of corresponding left view field image compressed code stream data and right view field image compressed code stream data are acquired, respectively and independently executing the step 2 on the compressed code stream data of the two view field images; step 2: searching frames; and step 3: transferring; and 4, step 4: decompressing; and 5: splicing; step 6: storing and displaying images; and 7: and (5) circularly executing the step 1 to the step 6 until the step is finished.
Description
Technical Field
The invention relates to a parallel image acquisition and processing method, in particular to a parallel multi-mode high-speed hyperspectral image acquisition and processing method.
Background
The image acquisition system is used for ensuring that the image data stream is reliably transmitted, displayed and stored according to a prearranged path. When the aerospace load is tested on the ground, the image acquisition system is required to meet the requirements of acquiring and processing original and compressed data under 1040-1500 fps variable high-speed frame frequency of a satellite-borne visible spectrum camera and an infrared spectrum camera.
However, the existing image acquisition system has single function and limited transmission rate. For example, the currently very popular high-speed Camera interface Camera link uses the LVDS receiving chip DS90CR288A of the national semiconductor company, which supports only 2.38Gbit/s data transmission at the highest. In addition, at present, no data report exists, wherein the image acquisition system can simultaneously process eight modes of image data, including image original data, compressed data, half-view field and full-view field combinations, which correspond to visible light and infrared light respectively.
Disclosure of Invention
The invention aims to provide a multi-mode high-speed hyperspectral image parallel acquisition processing method, which solves the technical problems that the existing image acquisition system is single in function and limited in transmission speed, can not simultaneously process eight modes of image original data, compressed data, half-field-of-view and full-field-of-view combined image data respectively corresponding to visible light and infrared light, and is difficult to meet the requirements of the image acquisition system in aerospace load ground test.
The invention adopts the technical scheme that a multi-mode high-speed hyperspectral image parallel acquisition processing method is characterized by comprising the following steps of:
step 1: data acquisition
Adopting a Medium Camera link acquisition card to acquire image data;
judging whether the collected image data is complete original image data of one frame or a group of corresponding left view field image compressed code stream data and right view field image compressed code stream data with fixed byte length;
if the collected image data is the complete original data of one frame of image, executing the step 5;
if the collected image data is a group of corresponding left view field image compressed code stream data and right view field image compressed code stream data with fixed byte length, respectively and independently executing the step 2 on the collected compressed code stream data of the two view field images;
step 2: frame search
Judging whether the residual bytes of the previous group of compressed code stream data are stored;
if the residual bytes of the previous group of compressed code stream data are stored, merging and searching the stored residual bytes of the previous group of compressed code stream data and the group of compressed code stream data in sequence according to the flag bit of the complete frame of image compressed code stream data; if the residual bytes of the previous group of compressed code stream data are not stored, carrying out frame search on the group of compressed code stream data according to the flag bit of the complete frame of image compressed code stream data;
storing the error data found in the search; step 3 is executed to the searched complete one-frame image compression code stream data; if the group of compressed code stream data has residual bytes, storing the residual bytes for next group searching and calling;
and step 3: transfer of data
Carrying out frame continuity interpretation on the complete frame image compression code stream data searched in the step 2, and if the complete frame image compression code stream data is continuous, executing a step 4; if not, after storing the error information, executing step 4;
and 4, step 4: decompression
Copying data, decompressing to obtain corresponding field image decompressed code stream data corresponding to the complete frame image compressed code stream data;
and 5: splicing
For the complete original data of the frame of image acquired in the step 1, respectively taking a line of original data of the image from the original data of the image acquired by each acquisition card, arranging the original data of the image in order according to the normal sequence of pixels, splicing to form complete full-field data, circularly reading all lines of the complete original data of the frame of image, and splicing to obtain the full-field data of the complete frame of image;
or for the left view field image decompressed code stream data and the right view field image decompressed code stream data of the complete frame image obtained after the decompression in the step 4, taking a line of decompressed code stream data from the left view field image decompressed code stream data and the right view field image decompressed code stream data respectively, arranging the lines in order according to the normal sequence of the pixels, splicing to form complete full view field data, then circularly reading all lines of the complete frame image decompressed code stream data, and splicing to obtain the full view field data of the complete frame image;
step 6: storing and displaying images;
storing and displaying the image corresponding to the full-field data of the complete frame image spliced in the step 5;
and 7: and (6) circularly executing the steps 1 to 6 until the acquisition processing is finished.
Further, in order to meet the requirement of on-satellite transmission, in step 1, the collected compressed code stream data is standard JPEG2000 compressed code stream data.
Further, in order to deal with the fault tolerance mechanism required by various data possibly encountered on the satellite and the processing mechanism aiming at special data, step 2.1: judging whether the residual bytes of the previous group of compressed code stream data are stored;
if the residual bytes of the last group of compressed code stream data are stored, executing the step 2.2;
if the residual bytes of the previous group of compressed code stream data are not stored, skipping the step 2.2 and executing the step 2.3;
step 2.2: merging and searching the stored residual bytes of the previous group of compressed code stream data and the group of compressed code stream data in sequence, performing corresponding processing according to the search result, and then executing the step 2.3; the corresponding processing of the different search results is as follows:
if the last group of compressed code stream data is found to end with FF and the group of compressed code stream data begins with D9, executing step 3 on the complete one-frame image compressed code stream data formed by the remaining bytes of the last group of compressed code stream data and D9 beginning with the group of compressed code stream data;
if the beginning of the group of compressed code stream data is searched for by 3C4F, the residual bytes of the previous group of compressed code stream data are error data, and the error data are stored;
if the search result is not the two cases, searching the first 3C4F line by line in the group of compressed code stream data, executing the step 3 on the complete one-frame image compressed code stream data formed by the remaining bytes of the previous group of compressed code stream data and the bytes before the searched first 3C4F in the group of compressed code stream data, and recording the line number of the line where the searched first 3C4F is located;
step 2.3: setting initial search line number
For the case where the remaining bytes of the previous set of compressed bitstream data are stored, according to the search result of step 2.2, if the previous set of compressed bitstream data ends with FF and the current set of compressed bitstream data begins with D9, or the current set of compressed bitstream data begins with 3C4F, the line number corresponding to the second line of the current set of compressed bitstream data is set as the initial search line number; if the search result is not the two cases, adding 1 to the row number of the searched row where the first 3C4F is located recorded in step 2.2, and setting the row number as the initial search row number;
setting the line number corresponding to the first line of the group of compressed code stream data as the initial search line number for the condition that the residual bytes of the group of compressed code stream data are not stored;
step 2.4: searching line by line from the corresponding line of the initial search line number set in the step 2.3 to the end of the last line of the last row of the compressed code stream data of the group, and performing corresponding processing;
step 2.4.1: judging whether the end of the penultimate line is reached;
if the end of the penultimate line is not reached, executing the step 2.4.2;
if the end of the penultimate line has been reached, go to step 2.5;
step 2.4.2: judging whether 3C4F is searched before the searching line which is being searched in the group of compressed code stream data;
if the 3C4F is not searched, searching the first 3C4F in the group of compressed code stream data line by line, and executing the step 2.4.3 after the first 3C4F is searched;
if the 3C4F is searched, directly executing the step 2.4.3;
step 2.4.3: searching the next 3C4F in the group of compressed code stream data line by line, and judging whether the next 3C4F is searched;
if the next 3C4F is searched, performing step 3 on the complete one-frame image compression code stream data formed by the bytes from the previous 3C4F adjacent to the searched next 3C4F to the previous 3C4F, and then returning to step 2.4.1;
if the last row of the group of compressed code stream data is searched and the next 3C4F is not searched, directly executing the step 2.5;
step 2.5: searching the last line of the group of compressed code stream data and carrying out corresponding processing;
step 2.5.1: judging whether the last line of the group of compressed code stream data starts with 3C 4F;
if the last line starts with 3C4F, step 3 is performed on the complete one-frame image compressed coded stream data composed of the bytes before the previous 3C4F adjacent to the 3C4F at the beginning of the last line and the 3C4F at the beginning of the last line, and the last line bytes are recorded as the remaining bytes of the group of compressed coded stream data, and then step 2.6 is performed;
if the last row does not begin with 3C4F, then step 2.5.2 is performed;
step 2.5.2: judging whether a code stream ending identifier is searched in the last line;
if the code stream ending identifier is searched, executing the step 3 on the complete frame of image compression code stream data where the searched code stream ending identifier is located, and recording the residual bytes of the group of compression code stream data as zero;
if the code stream ending identifier is not searched, recording the last 3C4F searched in the group of compressed code stream data and the bytes after the last 3C4F as the residual bytes of the group of compressed code stream data, and executing the step 2.6;
step 2.6: if the group of compressed code stream data has residual bytes, setting and storing the residual bytes of the group of compressed code stream data according to the residual bytes of the group of compressed code stream data recorded in the step 2.5 for the next group to search and call.
Further, in order to reduce data errors, for the complete original data of one frame of image acquired in step 1, after step 5 is completed, step 5 feeds back a completed signal to step 1, and after step 1 receives the completed signal, step 5 is executed for the complete original data of the next frame of image acquired in step 1;
for a group of compressed code stream data collected in the step 1, after finishing each of the four steps of the step 2, the step 3, the step 4 and the step 5, feeding back a finished signal of the step to the previous step, and after receiving the finished signal in the previous step, executing the next step corresponding to the data corresponding to the next group of compressed code stream data collected in the step 1. If the steps do not communicate with each other, the data processing does not adopt a synchronization mechanism, for example, a new frame is collected in the process of splicing the frame, so that the shared buffer area is updated, and data errors are caused.
Further, in order to increase the rate of splicing, so that the rate of the whole acquisition processing process is higher, in step 5, the splicing adopts an Openmp compiling instruction for acceleration processing.
Further, in order to ensure the smoothness of transmission, the step 3 further includes a step of calculating a real-time frame rate and an average frame rate.
Further, in order to increase the speed of the acquisition processing process, in step 1, the acquisition card adopts a KBN-PCE-CL4-F acquisition card of BitFlow company.
Further, the storage in step 2, step 3 and step 6 is implemented by a disk array composed of a plurality of PCI-E solid state disks.
The invention has the beneficial effects that:
(1) by adopting the multi-mode high-speed hyperspectral image parallel acquisition processing method, various mode data such as visible light, infrared light and the like in original and compressed formats of the image transmitted by the satellite equipment through the data transmission interface can be processed; the method adopts a Medium Camera link acquisition card to acquire, so that the acquisition rate is greatly improved; the real-time decompression of the complete one-frame image compression code stream data with indefinite length can be realized through the steps of frame searching, transferring and decompressing; the collected image original data or the left view field image decompressed code stream data and the right view field image decompressed code stream data obtained after decompression can be spliced into full view field data through the splicing step; and storing and displaying the image corresponding to the full-field-of-view data obtained by splicing; therefore, the invention solves the technical problems that the existing image acquisition system has single function and limited transmission rate, can not simultaneously process eight modes of image original data, compressed data, combined image data of half-view field and full-view field corresponding to visible light and infrared light respectively, and is difficult to meet the requirements of the image acquisition system in the aerospace load ground test. The method can meet the requirements of collecting and processing the original and compressed data under 1040-1500 fps variable high-speed frame rate of a satellite-borne visible spectrum camera and an infrared spectrum camera.
(2) The multi-mode high-speed hyperspectral image parallel acquisition and processing method is suitable for a ground test system of a satellite-borne hyperspectral imager, and by adopting the method, the real-time acquisition and processing capacity reaches 13Gbps, the processing and storage capacity reaches 16Gbps, and the whole acquisition and processing flow is less than 0.2 s.
(3) The invention provides a multi-mode high-speed hyperspectral image parallel acquisition processing method, and provides a preferable specific frame search scheme.
(4) According to the multi-mode high-speed hyperspectral image parallel acquisition and processing method, the steps are preferably communicated with each other, and a synchronization mechanism is adopted for data processing, so that data errors can be effectively prevented.
(5) According to the multi-mode high-speed hyperspectral image parallel acquisition processing method, the Openmp compiling instruction is preferably adopted for accelerated processing in splicing, so that the splicing speed can be increased, and the speed of the whole acquisition processing process is higher.
Drawings
FIG. 1 is a flow chart of the method of the present invention for acquiring and processing raw image data;
FIG. 2 is a flow chart of the method of the present invention for collecting and processing compressed code stream data of an image;
FIG. 3 is a schematic diagram of a synchronization mechanism when the method of the present invention is used to collect and process the original image data;
FIG. 4 is a schematic diagram of a synchronization mechanism when the method of the present invention is used to collect and process data of image compression code stream;
FIG. 5 is an overall flow chart of frame search in an embodiment of the present invention;
fig. 6 is a flowchart illustrating merging and searching the stored remaining bytes of the previous group of compressed bitstream data and the present group of compressed bitstream data in sequence when the remaining bytes of the previous group of compressed bitstream data are stored in the frame search according to the embodiment of the present invention, and performing corresponding processing according to the search result;
fig. 7 is a flowchart of searching line by line from the corresponding line of the set start search line number to the end of the last but one line of the group of compressed code stream data and performing corresponding processing in the frame search according to the embodiment of the present invention;
fig. 8 is a flowchart of searching the last line of the group of compressed bitstream data in the frame search according to the embodiment of the present invention, and performing corresponding processing.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
The invention discloses a multi-mode high-speed hyperspectral image parallel acquisition and processing method which comprises the following steps:
step 1: data acquisition
Adopting a Medium Camera link acquisition card to acquire image data;
judging whether the collected image data is complete original image data of one frame or a group of corresponding left view field image compressed code stream data and right view field image compressed code stream data with fixed byte length;
if the collected image data is the complete original data of one frame of image, executing the step 5;
if the collected image data is a group of corresponding left view field image compressed code stream data and right view field image compressed code stream data with fixed byte length, respectively and independently executing the step 2 on the collected compressed code stream data of the two view field images; the flow chart of the method for acquiring and processing the original data of the image is shown in figure 1, and four Medium Camera link acquisition cards are adopted when the original data of the image is acquired; the flow chart of the method for collecting and processing the image compression code stream data is shown in figure 2; when compressed code stream data is collected, two Medium Camera link acquisition cards are adopted;
step 2: frame search
Judging whether the residual bytes of the previous group of compressed code stream data are stored;
if the residual bytes of the previous group of compressed code stream data are stored, merging and searching the stored residual bytes of the previous group of compressed code stream data and the group of compressed code stream data in sequence according to the flag bit of the complete frame of image compressed code stream data; if the residual bytes of the previous group of compressed code stream data are not stored, carrying out frame search on the group of compressed code stream data according to the flag bit of the complete frame of image compressed code stream data;
storing the error data found in the search; step 3 is executed to the searched complete one-frame image compression code stream data; if the group of compressed code stream data has residual bytes, storing the residual bytes for next group searching and calling;
and step 3: transfer of data
Carrying out frame continuity interpretation on the complete frame image compression code stream data searched in the step 2, and if the complete frame image compression code stream data is continuous, executing a step 4; if not, after storing the error information, executing step 4;
and 4, step 4: decompression
Copying data, decompressing to obtain corresponding field image decompressed code stream data corresponding to the complete frame image compressed code stream data;
and 5: splicing
For the complete original data of the frame of image acquired in the step 1, respectively taking a line of original data of the image from the original data of the image acquired by each acquisition card, arranging the original data of the image in order according to the normal sequence of pixels, splicing to form complete full-field data, circularly reading all lines of the complete original data of the frame of image, and splicing to obtain the full-field data of the complete frame of image;
or for the left view field image decompressed code stream data and the right view field image decompressed code stream data of the complete frame image obtained after the decompression in the step 4, taking a line of decompressed code stream data from the left view field image decompressed code stream data and the right view field image decompressed code stream data respectively, arranging the lines in order according to the normal sequence of the pixels, splicing to form complete full view field data, then circularly reading all lines of the complete frame image decompressed code stream data, and splicing to obtain the full view field data of the complete frame image;
step 6: storing and displaying images;
storing and displaying the image corresponding to the full-field data of the complete frame image spliced in the step 5;
and 7: and (6) circularly executing the steps 1 to 6 until the acquisition processing is finished.
FIG. 1 is a flow chart of the method of the present invention for acquiring and processing raw image data. Fig. 2 is a flow chart of collecting and processing image compression code stream data by using the method of the invention.
The invention processes high-speed high-spectrum data, collects the data from the front end to the back end for storage and display, and improves the processing capability through parallel and multi-thread technology. In the embodiment of the invention, during the acquisition in the step 1, a KBN-PCE-CL4-F acquisition card of the American BitFlow company is adopted as a Medium Camera link (an industrial high-speed serial port data and connection standard aiming at simplifying an image acquisition interface and facilitating the connection of a high-speed digital Camera and a data acquisition card). In the standard protocol Camera Link, the interface has three configurations, Base, Medium and Full. In the invention, the highest speed is 3.6Gbps under 75MHz of each Medium Camera link acquisition card. The storage in the step 2, the step 3, and the step 6 is implemented by a disk array (RAID) composed of a plurality of PCI-E (peripheral component interconnect express) solid-state disks.
For high-speed data streams, each step is processed at high speed, which is not a short board in real-time streams. In the present invention, it is preferable to perform an accelerated process on the splicing process. For different detectors on the satellite, the image element output sequence may be from left to right and from top to bottom, and sometimes may be from right to left and from bottom to top, and the image inversion processing needs to be performed on different image blocks, so that the required time is long. Taking the splicing of the image raw data during the acquisition and processing of the method of the invention as an example, the splicing thread respectively takes a line of image raw data from four acquisition threads, the image elements are arranged orderly according to the normal sequence, the complete full-field data is formed by splicing, and then all lines of the complete frame of image raw data are read circularly. The splicing is very convenient by adopting Openmp compiling instructions to accelerate the processing. For example, a compilation instruction such as "# pragma omp parallel for private (i) schedule (dynamic) num _ reads (6)" is adopted, and 6 dynamic allocation threads are used for simultaneously processing multiple rows of reads and writes, so that 6 times of acceleration is realized. Where i is the row number, a private copy of each thread's own variables.
In the invention, in order to guarantee high-speed (1500 frames/s) data flow and prevent data errors, the steps preferably communicate with each other, and a synchronization mechanism is adopted for data processing. If a synchronization mechanism is not used, for example, during the process of splicing the current frame, a new frame is acquired, and thus the shared buffer is updated, resulting in data errors. FIG. 3 is a schematic diagram of a synchronization mechanism when the method of the present invention is used to collect and process the raw image data. As can be seen from fig. 3, when the method of the present invention is used to collect and process the original image data, after step 5 is completed, step 5 feeds back a completed signal to step 1 for the complete original image data of one frame collected in step 1, and after the completed signal is received in step 1, step 5 is executed for the complete original image data of the next frame collected in step 1. Fig. 4 is a schematic diagram of a synchronization mechanism when the method of the present invention is used to collect and process data of image compression code stream. As can be seen from fig. 4, when the method of the present invention is used to collect and process the image compressed code stream data, for a group of compressed code stream data collected in step 1, after each of the four steps of step 2, step 3, step 4 and step 5 is completed, a completed signal of the step is fed back to the previous step, and after the previous step receives the completed signal, the corresponding next step is executed on the data corresponding to the next group of compressed code stream data collected in step 1. In the embodiment of the invention, WaitForMultipleObjects are adopted to wait for a plurality of semaphores, and after processing of a certain thread is finished, the releaseStophon releases the semaphores so as to rerun the related blocked thread.
Referring to fig. 2, when the method of the present invention is used to collect and process data of image compression code stream, the frame search thread is a major feature of the method of the present invention. The byte length of the compressed code stream data acquired by the acquisition card each time is a fixed value, but the byte length of the complete frame of image compressed code stream data is constantly changed, and the byte length of the compressed code stream data acquired by the acquisition card each time is often larger than the byte length of the complete frame of image compressed code stream data, so that a part of byte data is often left during each frame search, and the frame search needs to be carried out together when the next frame of image is searched. In the present invention, preferably, when frame search is performed, the frame search is performed according to the flowcharts shown in fig. 5, fig. 6, fig. 7 and fig. 8, so that a fault tolerance mechanism required for various data that may be encountered on the satellite and a processing mechanism for special data can be dealt with. This is because the standard JPEG2000 compressed codestream is transmitted on the satellite. For standard JPEG2000 compressed bitstream data, each frame has a start flag and an end flag of the bitstream, where the start flag is "FF 4F FF 51" and the end flag is "FF D9". The auxiliary data is preceded by the code stream data, beginning with "3C 4F", and the frame count is located between 3C4F and the start flag. Therefore, in step 1, the collected compressed bitstream data is standard JPEG2000 compressed bitstream data. The acquisition card acquires compressed code stream data with the size of 1024 multiplied by 128 bytes each time. The start of frame identification "3C 4F" is at the beginning of each row for 128 rows. The frame search thread retrieves the entire one-frame image compression code stream data according to the flag bits. When performing frame search according to the flows shown in fig. 5, 6, 7, and 8, the frame search in step 2 specifically includes the following steps:
referring to fig. 5, step 2.1: judging whether the residual bytes of the previous group of compressed code stream data are stored;
if the residual bytes of the last group of compressed code stream data are stored, executing the step 2.2;
if the residual bytes of the previous group of compressed code stream data are not stored, skipping the step 2.2 and executing the step 2.3;
step 2.2: merging and searching the stored residual bytes of the previous group of compressed code stream data and the group of compressed code stream data in sequence, performing corresponding processing according to the search result, and then executing the step 2.3; referring to fig. 6, the corresponding processing of the different search results is as follows:
if the last group of compressed code stream data is found to end with FF and the group of compressed code stream data begins with D9, executing step 3 on the complete one-frame image compressed code stream data formed by the remaining bytes of the last group of compressed code stream data and D9 beginning with the group of compressed code stream data;
if the beginning of the group of compressed code stream data is searched for by 3C4F, the residual bytes of the previous group of compressed code stream data are error data, and the error data are stored;
if the search result is not the two cases, searching the first 3C4F line by line in the group of compressed code stream data, executing the step 3 on the complete one-frame image compressed code stream data formed by the remaining bytes of the previous group of compressed code stream data and the bytes before the searched first 3C4F in the group of compressed code stream data, and recording the line number of the line where the searched first 3C4F is located;
step 2.3: setting initial search line number
For the case where the remaining bytes of the previous set of compressed bitstream data are stored, according to the search result of step 2.2, if the previous set of compressed bitstream data ends with FF and the current set of compressed bitstream data begins with D9, or the current set of compressed bitstream data begins with 3C4F, the line number corresponding to the second line of the current set of compressed bitstream data is set as the initial search line number; if the search result is not the two cases, adding 1 to the row number of the searched row where the first 3C4F is located recorded in step 2.2, and setting the row number as the initial search row number;
setting the line number corresponding to the first line of the group of compressed code stream data as the initial search line number for the condition that the residual bytes of the group of compressed code stream data are not stored;
step 2.4: searching line by line from the line corresponding to the initial search line number set in the step 2.3 to the end of the last but one line of the compressed code stream data of the group, and performing corresponding processing, referring to fig. 7;
step 2.4.1: judging whether the end of the penultimate line is reached;
if the end of the penultimate line is not reached, executing the step 2.4.2;
if the end of the penultimate line has been reached, go to step 2.5;
step 2.4.2: judging whether 3C4F is searched before the searching line which is being searched in the group of compressed code stream data;
if the 3C4F is not searched, searching the first 3C4F in the group of compressed code stream data line by line, and executing the step 2.4.3 after the first 3C4F is searched;
if the 3C4F is searched, directly executing the step 2.4.3;
step 2.4.3: searching the next 3C4F in the group of compressed code stream data line by line, and judging whether the next 3C4F is searched;
if the next 3C4F is searched, performing step 3 on the complete one-frame image compression code stream data formed by the bytes from the previous 3C4F adjacent to the searched next 3C4F to the previous 3C4F, and then returning to step 2.4.1;
if the last row of the group of compressed code stream data is searched and the next 3C4F is not searched, directly executing the step 2.5;
step 2.5: searching the last line of the group of compressed code stream data, and performing corresponding processing, see fig. 8;
step 2.5.1: judging whether the last line of the group of compressed code stream data starts with 3C 4F;
if the last line starts with 3C4F, step 3 is performed on the complete one-frame image compressed coded stream data composed of the bytes before the previous 3C4F adjacent to the 3C4F at the beginning of the last line and the 3C4F at the beginning of the last line, and the last line bytes are recorded as the remaining bytes of the group of compressed coded stream data, and then step 2.6 is performed;
if the last row does not begin with 3C4F, then step 2.5.2 is performed;
step 2.5.2: judging whether a code stream ending identifier is searched in the last line;
if the code stream ending identifier is searched, executing the step 3 on the complete frame of image compression code stream data where the searched code stream ending identifier is located, and recording the residual bytes of the group of compression code stream data as zero;
if the code stream ending identifier is not searched, recording the last 3C4F searched in the group of compressed code stream data and the bytes after the last 3C4F as the residual bytes of the group of compressed code stream data, and executing the step 2.6;
step 2.6: if the group of compressed code stream data has residual bytes, setting and storing the residual bytes of the group of compressed code stream data according to the residual bytes of the group of compressed code stream data recorded in the step 2.5 for the next group to search and call.
The transfer thread in the invention judges the continuity of the frame of the complete frame of image compression code stream data searched in the step 2, if the complete frame of image compression code stream data is continuous, the image compression code stream data is decompressed, and if the complete frame of image compression code stream data is discontinuous, the error information is stored and then the image compression code stream data is decompressed. In the embodiment of the present invention, preferably, the transferring in step 3 further includes a step of calculating a real-time frame rate and an average frame rate, so as to ensure the smoothness of transmission.
The decompression thread of the invention copies data of the complete frame of image compressed code stream data transferred by the transfer thread, decompresses the data and obtains corresponding field image decompressed code stream data corresponding to the complete frame of image compressed code stream data.
The storage thread and the fast sight thread are used for storing and displaying the image corresponding to the full-view data of the complete frame of image obtained by splicing the splicing thread.
The method is suitable for a satellite-borne hyperspectral imager test system, has strong adaptability to satellite multimode (original and compressed) data, has the real-time acquisition processing capacity of 13Gbps and the processing and storage capacity of 16Gbps, and greatly improves the capacity compared with the traditional acquisition processing method. Meanwhile, due to reasonable synchronization mechanism and acceleration measure among a plurality of threads, experimental results show that the delay of the whole acquisition and processing flow is less than 0.2 s. The method can meet the requirements of collecting and processing the original and compressed data under 1040-1500 fps variable high-speed frame rate of a satellite-borne visible spectrum camera and an infrared spectrum camera.
Claims (8)
1. A multi-mode high-speed hyperspectral image parallel acquisition processing method is characterized by comprising the following steps:
step 1: data acquisition
Adopting a Medium Camera link acquisition card to acquire image data;
judging whether the collected image data is complete original image data of one frame or a group of corresponding left view field image compressed code stream data and right view field image compressed code stream data with fixed byte length;
if the collected image data is the complete original data of one frame of image, executing the step 5;
if the collected image data is a group of corresponding left view field image compressed code stream data and right view field image compressed code stream data with fixed byte length, respectively and independently executing the step 2 on the collected compressed code stream data of the two view field images;
step 2: frame search
Judging whether the residual bytes of the previous group of compressed code stream data are stored;
if the residual bytes of the previous group of compressed code stream data are stored, merging and searching the stored residual bytes of the previous group of compressed code stream data and the group of compressed code stream data in sequence according to the flag bit of the complete frame of image compressed code stream data; if the residual bytes of the previous group of compressed code stream data are not stored, carrying out frame search on the group of compressed code stream data according to the flag bit of the complete frame of image compressed code stream data;
storing the error data found in the search; step 3 is executed to the searched complete one-frame image compression code stream data; if the group of compressed code stream data has residual bytes, storing the residual bytes for next group searching and calling;
and step 3: transfer of data
Carrying out frame continuity interpretation on the complete frame image compression code stream data searched in the step 2, and if the complete frame image compression code stream data is continuous, executing a step 4; if not, after storing the error information, executing step 4;
and 4, step 4: decompression
Copying data, decompressing to obtain corresponding field image decompressed code stream data corresponding to the complete frame image compressed code stream data;
and 5: splicing
For the complete original data of the frame of image acquired in the step 1, respectively taking a line of original data of the image from the original data of the image acquired by each acquisition card, arranging the original data of the image in order according to the normal sequence of pixels, splicing to form complete full-field data, circularly reading all lines of the complete original data of the frame of image, and splicing to obtain the full-field data of the complete frame of image;
or for the left view field image decompressed code stream data and the right view field image decompressed code stream data of the complete frame image obtained after the decompression in the step 4, taking a line of decompressed code stream data from the left view field image decompressed code stream data and the right view field image decompressed code stream data respectively, arranging the lines in order according to the normal sequence of the pixels, splicing to form complete full view field data, then circularly reading all lines of the complete frame image decompressed code stream data, and splicing to obtain the full view field data of the complete frame image;
step 6: storing and displaying images;
storing and displaying the image corresponding to the full-field data of the complete frame image spliced in the step 5;
and 7: and (6) circularly executing the steps 1 to 6 until the acquisition processing is finished.
2. The multi-mode high-speed hyperspectral image parallel acquisition and processing method according to claim 1, characterized in that: in step 1, the collected compressed code stream data is standard JPEG2000 compressed code stream data.
3. The multi-mode high-speed hyperspectral image parallel acquisition and processing method according to claim 2, wherein the step 2 specifically comprises the following steps:
step 2.1: judging whether the residual bytes of the previous group of compressed code stream data are stored;
if the residual bytes of the last group of compressed code stream data are stored, executing the step 2.2;
if the residual bytes of the previous group of compressed code stream data are not stored, skipping the step 2.2 and executing the step 2.3;
step 2.2: merging and searching the stored residual bytes of the previous group of compressed code stream data and the group of compressed code stream data in sequence, performing corresponding processing according to the search result, and then executing the step 2.3; the corresponding processing of the different search results is as follows:
if the last group of compressed code stream data is found to end with FF and the group of compressed code stream data begins with D9, executing step 3 on the complete one-frame image compressed code stream data formed by the remaining bytes of the last group of compressed code stream data and D9 beginning with the group of compressed code stream data;
if the beginning of the group of compressed code stream data is searched for by 3C4F, the residual bytes of the previous group of compressed code stream data are error data, and the error data are stored;
if the search result is not the two cases, searching the first 3C4F line by line in the group of compressed code stream data, executing the step 3 on the complete one-frame image compressed code stream data formed by the remaining bytes of the previous group of compressed code stream data and the bytes before the searched first 3C4F in the group of compressed code stream data, and recording the line number of the line where the searched first 3C4F is located;
step 2.3: setting initial search line number
For the case where the remaining bytes of the previous set of compressed bitstream data are stored, according to the search result of step 2.2, if the previous set of compressed bitstream data ends with FF and the current set of compressed bitstream data begins with D9, or the current set of compressed bitstream data begins with 3C4F, the line number corresponding to the second line of the current set of compressed bitstream data is set as the initial search line number; if the search result is not the two cases, adding 1 to the row number of the searched row where the first 3C4F is located recorded in step 2.2, and setting the row number as the initial search row number;
setting the line number corresponding to the first line of the group of compressed code stream data as the initial search line number for the condition that the residual bytes of the group of compressed code stream data are not stored;
step 2.4: searching line by line from the corresponding line of the initial search line number set in the step 2.3 to the end of the last line of the last row of the compressed code stream data of the group, and performing corresponding processing;
step 2.4.1: judging whether the end of the penultimate line is reached;
if the end of the penultimate line is not reached, executing the step 2.4.2;
if the end of the penultimate line has been reached, go to step 2.5;
step 2.4.2: judging whether 3C4F is searched before the searching line which is being searched in the group of compressed code stream data;
if the 3C4F is not searched, searching the first 3C4F in the group of compressed code stream data line by line, and executing the step 2.4.3 after the first 3C4F is searched;
if the 3C4F is searched, directly executing the step 2.4.3;
step 2.4.3: searching the next 3C4F in the group of compressed code stream data line by line, and judging whether the next 3C4F is searched;
if the next 3C4F is searched, performing step 3 on the complete one-frame image compression code stream data formed by the bytes from the previous 3C4F adjacent to the searched next 3C4F to the previous 3C4F, and then returning to step 2.4.1;
if the last row of the group of compressed code stream data is searched and the next 3C4F is not searched, directly executing the step 2.5;
step 2.5: searching the last line of the group of compressed code stream data and carrying out corresponding processing;
step 2.5.1: judging whether the last line of the group of compressed code stream data starts with 3C 4F;
if the last line starts with 3C4F, step 3 is performed on the complete one-frame image compressed coded stream data composed of the bytes before the previous 3C4F adjacent to the 3C4F at the beginning of the last line and the 3C4F at the beginning of the last line, and the last line bytes are recorded as the remaining bytes of the group of compressed coded stream data, and then step 2.6 is performed;
if the last row does not begin with 3C4F, then step 2.5.2 is performed;
step 2.5.2: judging whether a code stream ending identifier is searched in the last line;
if the code stream ending identifier is searched, executing the step 3 on the complete frame of image compression code stream data where the searched code stream ending identifier is located, and recording the residual bytes of the group of compression code stream data as zero;
if the code stream ending identifier is not searched, recording the last 3C4F searched in the group of compressed code stream data and the bytes after the last 3C4F as the residual bytes of the group of compressed code stream data, and executing the step 2.6;
step 2.6: if the group of compressed code stream data has residual bytes, setting and storing the residual bytes of the group of compressed code stream data according to the residual bytes of the group of compressed code stream data recorded in the step 2.5 for the next group to search and call.
4. The multi-mode high-speed hyperspectral image parallel acquisition and processing method according to any one of claims 1 to 3, characterized in that:
for the complete original data of one frame of image collected in the step 1, after the step 5 is completed, the step 5 feeds back a completed signal to the step 1, and after the step 1 receives the completed signal, the step 5 is executed for the complete original data of the next frame of image collected in the step 1;
for a group of compressed code stream data collected in the step 1, after finishing each of the four steps of the step 2, the step 3, the step 4 and the step 5, feeding back a finished signal of the step to the previous step, and after receiving the finished signal in the previous step, executing the next step corresponding to the data corresponding to the next group of compressed code stream data collected in the step 1.
5. The multi-mode high-speed hyperspectral image parallel acquisition and processing method according to claim 4, characterized in that: in step 5, the splicing adopts an Openmp compiling instruction for acceleration processing.
6. The multi-mode high-speed hyperspectral image parallel acquisition and processing method according to claim 5, characterized in that: said step 3 further comprises the step of calculating the real-time frame rate and the average frame rate.
7. The multi-mode high-speed hyperspectral image parallel acquisition and processing method according to claim 6, characterized in that: in the step 1, the acquisition card adopts a KBN-PCE-CL4-F acquisition card of BitFlow company.
8. The multi-mode high-speed hyperspectral image parallel acquisition and processing method according to claim 7, characterized in that: and 2, the storage in the step 3 and the step 6 is realized by adopting a disk array consisting of a plurality of PCI-E solid-state disks.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010954433.6A CN112235578B (en) | 2020-09-11 | 2020-09-11 | Multi-mode high-speed hyperspectral image parallel acquisition and processing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010954433.6A CN112235578B (en) | 2020-09-11 | 2020-09-11 | Multi-mode high-speed hyperspectral image parallel acquisition and processing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112235578A true CN112235578A (en) | 2021-01-15 |
| CN112235578B CN112235578B (en) | 2021-06-18 |
Family
ID=74116990
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010954433.6A Active CN112235578B (en) | 2020-09-11 | 2020-09-11 | Multi-mode high-speed hyperspectral image parallel acquisition and processing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112235578B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1972448A (en) * | 2006-12-12 | 2007-05-30 | 北京中星微电子有限公司 | A method of fault-tolerant for file playing |
| CN101605012A (en) * | 2009-07-02 | 2009-12-16 | 中兴通讯股份有限公司南京分公司 | A kind of method and device of realizing the frame header of synchronous digital system location |
| US20110069761A1 (en) * | 2009-09-18 | 2011-03-24 | Kabushiki Kaisha Toshiba | Parallel encoding device, recording medium, and imaging data encoding method |
| CN102142917A (en) * | 2011-03-28 | 2011-08-03 | 华为技术有限公司 | Data frame positioning method and device |
| CN107094257A (en) * | 2017-05-14 | 2017-08-25 | 华中科技大学 | A kind of system to satellite image data real-time decompression |
| CN110719486A (en) * | 2019-10-22 | 2020-01-21 | 深圳市优立信科技有限公司 | Video decoding equipment based on self-adaptive video decoding method |
-
2020
- 2020-09-11 CN CN202010954433.6A patent/CN112235578B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1972448A (en) * | 2006-12-12 | 2007-05-30 | 北京中星微电子有限公司 | A method of fault-tolerant for file playing |
| CN101605012A (en) * | 2009-07-02 | 2009-12-16 | 中兴通讯股份有限公司南京分公司 | A kind of method and device of realizing the frame header of synchronous digital system location |
| US20110069761A1 (en) * | 2009-09-18 | 2011-03-24 | Kabushiki Kaisha Toshiba | Parallel encoding device, recording medium, and imaging data encoding method |
| CN102142917A (en) * | 2011-03-28 | 2011-08-03 | 华为技术有限公司 | Data frame positioning method and device |
| CN107094257A (en) * | 2017-05-14 | 2017-08-25 | 华中科技大学 | A kind of system to satellite image data real-time decompression |
| CN110719486A (en) * | 2019-10-22 | 2020-01-21 | 深圳市优立信科技有限公司 | Video decoding equipment based on self-adaptive video decoding method |
Non-Patent Citations (1)
| Title |
|---|
| 李晓娟;刘学斌;胡炳樑;程海波: "基于FPGA的CCD相机高速数据记录系统", 《科学技术与工程》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112235578B (en) | 2021-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| RU2479937C2 (en) | Information processing apparatus and method | |
| US20200154162A1 (en) | Transmission And Detection Of Multi-Channel Signals In Reduced Channel Format | |
| CN101536491B (en) | Imaging device and imaging method | |
| US8098941B2 (en) | Method and apparatus for parallelization of image compression encoders | |
| DE19811292B4 (en) | MPEG-II system with PES decoder | |
| EP2134092B1 (en) | Information processing apparatus and method, and program | |
| KR100928324B1 (en) | Operation method of frame buffer memory for recovering compressed video and decoding device suitable for this | |
| US20110025869A1 (en) | Method and apparatus for generating compressed file, camera module associated therewith, and terminal including the same | |
| CN101690173B (en) | Image processing apparatus and method for displaying captured image without time delay | |
| US20070098272A1 (en) | Reduced buffer size for JPEG encoding | |
| US20070279422A1 (en) | Processor system including processors and data transfer method thereof | |
| EP0868087A2 (en) | Multiplexing apparatus and method, transmitting apparatus and method, and recording medium | |
| US20070046683A1 (en) | Image processor, imaging device, and image processing system | |
| CN113271423A (en) | Multi-format video interface conversion device and method based on FPGA | |
| CN112235578B (en) | Multi-mode high-speed hyperspectral image parallel acquisition and processing method | |
| KR20050113500A (en) | Compression and decompression device of graphic data and therefor method | |
| US6275535B1 (en) | Method and device for decoding an image compressed in particular according to the MPEG standards, especially a bidirectional image | |
| US20100328514A1 (en) | Image processing device, imaging apparatus, and thumbnail image displaying method | |
| US9066111B2 (en) | Image encoder and method for encoding images | |
| KR100296684B1 (en) | Decoding method and decoder for decoding audio / video compression code data | |
| JP2012175303A (en) | Imaging device, imaging method, and program | |
| USRE41179E1 (en) | Device for extracting parameters for decoding a video data flow coded according to an MPEG standard | |
| US7460718B2 (en) | Conversion device for performing a raster scan conversion between a JPEG decoder and an image memory | |
| US20120106861A1 (en) | Image compression method | |
| WO2019164034A1 (en) | Reconfigurable imaging system for ultra-high resolution multi-channel image processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |