WO2015101847A1 - Reflective wave device for simultaneous event detection and signal reconstruction using compressive measurements - Google Patents

Reflective wave device for simultaneous event detection and signal reconstruction using compressive measurements Download PDF

Info

Publication number
WO2015101847A1
WO2015101847A1 PCT/IB2014/066032 IB2014066032W WO2015101847A1 WO 2015101847 A1 WO2015101847 A1 WO 2015101847A1 IB 2014066032 W IB2014066032 W IB 2014066032W WO 2015101847 A1 WO2015101847 A1 WO 2015101847A1
Authority
WO
WIPO (PCT)
Prior art keywords
event
frame
detection
reconstruction
image
Prior art date
Application number
PCT/IB2014/066032
Other languages
French (fr)
Inventor
Asha VIJAYAKUMAR
Original Assignee
Airbus Group Singapore Pte. Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Airbus Group Singapore Pte. Ltd. filed Critical Airbus Group Singapore Pte. Ltd.
Priority to SG11201603729UA priority Critical patent/SG11201603729UA/en
Publication of WO2015101847A1 publication Critical patent/WO2015101847A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H03M7/3059—Digital compression and data reduction techniques where the original information is represented by a subset or similar information, e.g. lossy compression
    • H03M7/3062—Compressive sampling or sensing

Definitions

  • Sonar underwater imaging has diversified since it was first developed and now has a wide range of applications .
  • small-scale devices which may be remote controlled or even autonomous robots, can be deployed for underwater surveillance.
  • the range of applications includes detection of hazardous materials in a harbour, mine sweeping, geophysical surveys, surveillance of naval traffic and others.
  • image-processing units that are efficient with regard to processing and memory requirements and at the same time reasonably fast.
  • Compressive sensing which is also known as compressed sensing, compressive sampling, or sparse sampling, is a signal processing technique for efficiently acquiring and reconstructing a signal by finding solutions to underdetermined linear systems.
  • sparsity of the signal refers to the degree to which the signal is represented by just a few coefficients of a given set of base functions, the other coefficients being zero, close to zero or at least much smaller.
  • Most of the known compressive sensing systems are of two kinds: imaging and subsequent reconstruction from compressive measurements or detection of an event directly from the compressive measurements.
  • the US 2013/0128042 discloses an event detection method using a compressive-sensing hyperspectral imaging architecture. Therein, an incident light beam is split up into its spectral components. An image portion or its time sequence is examined in the selected spectral range for a pre-determined trigger condition such as exceedance of a pre-determined magnitude or of a predetermined rate of change.
  • an event detection is per- formed in the compressed sensed samples .
  • a meth- od and device according to the present specification allows an event detection to be performed simultaneously during the reconstruction of a compressed sensed image.
  • Baraniuk G. Baraniuk, “Compressive sensing for background subtraction” in Proc. European Conf. Comp . Vision (ECCV) , Berlin, Germany, 2008.
  • An image reconstruction system provides the ability for simultaneous event detection, corresponding scene reconstruction and whole image reconstruction irrespective of an event in real time.
  • a system and method is disclosed that can be used for simultaneous reconstruction of the scene and detection of event in real time using suitable optimization and reconstruction algorithms, respectively.
  • a simultaneous execution of two processes is understood in a general sense that the two processes comprise common steps and/or comprise steps that are executed in parallel for both processes .
  • a method for event detection and reconstruction of image raw data in a reflective wave device is disclosed, in particular for a simultaneous or real-time event detection which is performed during the data acquisition and reconstruction of image raw data.
  • a reflective wave device refers to a device for generating and receiving sound waves or electromagnetic waves such as a sonar, a seismic device, or any form of radar.
  • Image raw data is acquired from a sensor for sensing reflected sound waves or electromagnetic waves and a compressed representation of the image raw data is with compressive sensing.
  • Compressive sensing which is explained in further detail in the specification text below, involves a sparse representation of the image raw data in a suitable set of base functions and a random projection. The random projection reduces the amount of data, wherein the randomness of the random projection helps to avoid image artefacts that are due to a systematic bias.
  • the compressed representation is processed frame-by-frame, wherein the frames may represent in particular rows or columns of a two-dimensional image, or also equal sized blocks of data of the two-dimensional image.
  • the two-dimensional dimension may also be provided by a slice of a three- dimensional image and the physical units may for example be space units, time units or units resulting from a transformation of the data, such as a Fourier transformation, a Wavelet transformation, a Radon transformation or others.
  • the frame-by-frame processing of the compressed representation a detection of events is performed which is based on a current frame of the compressed representation.
  • the frame-by-frame processing comprises a frame-by-frame image reconstruction.
  • a pre-determined number of frames that precede the current frame is received.
  • "preceding” is defined according to an order in which the frames are processed .
  • An event is reconstructed based on a detection subset, wherein the detection subset is derived from the current frame and the pre-determined number of retrieved frames.
  • the detections subset can also be identical with the current frame and the pre-determined number of retrieved frames .
  • the reconstruction of the event comprises reconstruction of event data such as event type, event time, event location, rate of change etc.
  • a reconstruction of an event scene refers to a more detailed reconstruction of a spatial arrangement of the event. This may, among others, involve orientation, spatial extent, size and other features defining a spatial arrangement.
  • the processing unit can trigger an adequate reaction. According to the present specification it is not necessary to defer the event detection until a complete image has been reconstructed but, rather, the event detection is performed during the process of image reconstruction.
  • the detection subset is stored in memory slots of a buffer "Bl".
  • the size of the buffer "Bl” is larger than the amount required to store the detection subset by a buffering size.
  • the buffering size depends on the time required to transmit the detection subset to a processing unit.
  • the detection subset for reconstructing the event corresponds to (R-t) slots that represent (R-t) frames and the buffering size corresponds to t slots representing t frames .
  • the detection subset is copied to a second buffer "B2" and the content of the second buffer "B2" is sent to a scene reconstruction unit, if an event is detected in the current frame.
  • a data message is generated which comprises event data of the detected event, such as event time, location, type, intensity.
  • event data such as event time, location, type, intensity.
  • This can, in turn trigger an output of a signal that is indicative of the detected event, such as a light signal or an acoustic warning or a display of event details on a screen, LCD display or the like.
  • the complete reconstructed image or part of it may be output.
  • an event scene is reconstructed, which is in particular based on the detection subset and which is indicate of the spatial arrangement of the event and/or of the change thereof.
  • the reconstruction of the event scene may comprise, for example, the spatial orientation and extension of the event, a direction of movement of the event, a rate of change of the event etc .
  • an image in which the event scene is marked is reconstructed, wherein the image represents the raw data.
  • the marked event scene is based on the detection subset.
  • the method may comprise a detection whether an event is present in the current frame using a part P of se lected measurements or coefficients for the current frame.
  • the selection of only a part P of the selected measurememts or coefficients further reduces the amount of data.
  • the compressive sensing comprises a random projection that is chosen such that 50% or less of the input data is used, which means that the complete input set of data need not to be used.
  • the random projection can be performed using a random projection matrix with a ratio of rows to columns, or columns to rows, as the case may be, which is 50% or less.
  • the derivation of the compressed representation may comprise an acquisition of a two-dimensional image and partitioning of the two-dimensional image in one- dimensional frames. For a predetermined number p of the one- dimensional frames repeating the steps of generating a random projection of a frame and of representing the random projection by M measurements or coefficients of a set of base functions to generate a sparse vector of base function coefficients are repeated.
  • a part P of the M measurements or coefficients may be selected for use in event detection. For example, if the event occupies only a small region of the frame and the base functions are concentrated around a region, only coefficients of base functions contribute which cover the event region. It is detected if an event has occurred according to a predetermined event criterion based on the P selected measurements or coefficients.
  • the event criterion may in particular comprise comparison with a threshold or a threshold event.
  • a two-dimensional image according to the present specification may represent a two-dimensional distribution of sensor signals, such as sonar signals.
  • the two- dimensional distribution may be spatial but it may also be spatiotemporal distribution.
  • the dimensions of the image may also result from a transformation of the original image. For example, a Fourier transformation with respect to two space dimensions will result in a two-dimensional image in which the axes represent frequencies.
  • the present application discloses a reflective wave device comprising a sensor for acquisition of image raw data for sensing the reflected acoustic or electromagnetic waves and a computation unit with a processor, a memory and an output unit.
  • the computation unit comprising a software program which provides for deriving a compressed representation of the image raw data with compressive sensing, frame-by-frame processing of the compressed representation.
  • a detection of an event is performed based on a current frame of the compressed representation.
  • the frame-by-frame processing comprises a frame-by-frame image reconstruction .
  • a pre-determined number of frames that precede the current frame is retrieved, wherein the numbering of the frames is defined according to the order in which the frames are processed. This order may correspond to the natural order or rows or the natural order of columns in which the image data is acquired.
  • An event is reconstructed based on a detection subset, wherein the detection subset is derived from - or is identical with - the current frame and the pre-determined number of retrieved frame .
  • the reconstructed event is compared with a pre-determined threshold event and a flag signal is output if the reconstructed event matches the threshold event.
  • the present specification disclose a reflective wave device in which the abovementioned functions are realized partially or entirely by specific dedicated hardware components, such as application specific integrated circuits or electric circuitry. Part of the functions may also be realized by software modules.
  • the software modules and the specific hardware components are referred as “units” or “sub-units " .
  • the reflective wave device comprises a sensor for acquisition of image raw data, which is operable for sensing the reflected waves.
  • the sensor may be part of an image acquisition device for acquiring a two-dimensional image.
  • the reflective wave device comprises a compressive sensing unit for deriving a compressed representation from the image raw data.
  • the compressive sensing unit may in turn comprise a first sub-unit for selecting a current frame from the image raw data and for generating a random projection of the frame and a second sub-unit for representing the frame by M measurements or coefficients of a pre-determined set of base functions and a third sub-unit for selecting a subset of P measurements or coefficients from the M measurements or coefficients.
  • the reflective wave device comprises a detection unit with a sub-unit for detecting an event based on the current frame.
  • the detection unit is furthermore operative to retrieve a pre-determined number of frames which precede the current frame according to the order in which the frames are processed, to reconstruct an event based on a detection subset, wherein the detection subset is derived from, or is identical to, the current frame and the pre-determined number of retrieved frames, to compare the reconstructed event with a pre-determined event criterion, such as a threshold event, and to output a flag signal if the reconstructed event matches the event criterion.
  • a pre-determined event criterion such as a threshold event
  • the reflective wave device comprises a buffering unit with a buffer "Bl", a size of the buffer "Bl” being larger than the amount required to store the detection subset by a buffering size, wherein the buffering size depends on a time required to transmit the detection subset to a processing unit.
  • the reflective wave ⁇ vice comprises a second buffer "B2".
  • a size of the second buffer is sufficient for storing the detection subset .
  • the current specification disclose a mobile underwater sonar device, which may be self-propelled or moved by an external device such as a string.
  • the sonar device comprises one of the abovementioned reflective wave devices.
  • the reflective wave device is provided at, or in other words in- side or onto, a body of the mobile underwater sonar device .
  • the sensor for acquiring the image raw data is provided by one or more acoustic sensors that are arranged laterally, which is starboard or larboard, to a direction of travel of the mobile underwater sonar device.
  • the current specification discloses a ship with one of the abovementioned reflective wave devices.
  • the re- flective wave device is provided at, or in other words in or onto, a body of the ship.
  • the sensor for acquiring the image raw data is provided by one or more acoustic sensors that are arranged laterally, which is starboard or larboard, to a di- rection of travel of the ship .
  • the current specification discloses an unmanned aerial vehicle with one of the abovementioned reflective wave devices.
  • the reflective wave device is provided at, or in other words in or onto, a body of the unmanned aerial vehicle.
  • the sensor for acquiring the image raw data is provided by one or more radar sensors that are arranged laterally to a direction of travel of the unmanned aerial vehicle, which may be at the bottom or slightly sideways but also at other locations .
  • the present specification discloses an aircraft with one of the abovementioned reflective wave .
  • the reflective wave device is provided at, or in other words in or on an outer surface of, a body of the aircraft.
  • the sensor for acquiring the image raw data is provided by one or more radar sensors that are arranged laterally to a direction of travel of the aircraft, which may be at the bottom or slightly sideways but also at other locations .
  • Fig. 1 shows a diagram of a data reconstruction and event detection system
  • Fig. 2 shows a method for data reconstruction and event detection using the system of Fig. 1,
  • Fig. 3 shows a row of an original signal
  • Fig. 4 shows a randomly sampled sparse signal
  • Fig. 5 shows a signal that is reconstructed from the signal of Fig. 4,
  • Fig. 6 shows an original image
  • Fig. 7 shows an image that is reconstructed from the
  • Fig. 8 shows a raw data image
  • Fig. 9 shows a row-wise event detection derived from the image of Fig .8 .
  • Fig. 10 shows a row of an original signal
  • Fig. 11 shows a randomly sampled signal from the signal of
  • Fig. 12 shows a row-wise detection result
  • Fig. 13 shows a row of an original signal
  • Fig. 14 shows a detection probability as a function of a number of measurements
  • Fig. 15 shows detection probabilities as a function of a number of measurements for an algorithm with and without reconstruction
  • Fig. 16 shows a sonar device comprising the detection system of Fig. 1
  • Fig. 17 shows a sectional enlargement of Fig. 1 that illustrates a buffer management according to the specification .
  • various basis functions may be used in order to achieve a sparsely populated representation of the image data in terms of basis function coefficients.
  • any of the known compressive sensing reconstruction algorithms may be used in a device or in a method according to the present specification .
  • FIG. 1 shows a system 1 for simultaneous image reconstruction and event detection.
  • block 10 represents the N x N sonar processed image 10 from the raw data.
  • Block 20 represents a compressive sensing unit 20 (CSU) for generating a compressive sensed input signal from the N x N raw image.
  • Block 30 represents a detection unit 30 where the compressively sensed samples are used for detection.
  • Block 40 represents a scene recovery unit (SRU) 40, which takes the frame information from the detection unit and passes the information on to a control unit 50 for reconstruction of the event scene and/or the complete image.
  • Block 50 represents the control unit (CU) 50 for passing control signals to the buffering unit 60 and to a reconstruction unit 70.
  • Block 60 represents the buffering unit 60, which serves as a temporary storage of the compressively sensed data for reconstruction purposes .
  • Block 70 represents the reconstruction unit 70.
  • the reconstruction unit 70 comprises two processing units which are not shown in Fig. 1.
  • the processing units may be provided by special purpose hardware such as a graphics processing unit (GPU) or a field programmable gate array (FPGA) .
  • the processing units schedule the task of reconstruction of the scene corresponding to the samples in the buffering unit 60 and also schedule the reconstruction of the event scene simultaneously.
  • Block 80 represents a user interface unit 80 which receives inputs that are specific to the application for example-the scenes to be recovered, the number of frames spanned by the target.
  • the user interface unit 80 may be provided by a LCD screen and a keypad, by an input device that is connectable to an image acquisition device by cable or by wireless connection.
  • a sonar is shown in Fig. 16.
  • Fig. 2 shows a flow diagram o a method for image reconstruc tion and event detection usin the system 1 of Fig. 1.
  • Fig. is now explained in further d tail in the context of sonar signal processing.
  • an N x N sonar image is acquired from raw data of a sonar. Specifically, the N x N sonar image is obtained by translating the raw acoustic measurements obtained from back scattered responses into images.
  • the images are processed row-wise or column wise depending on the swath of the sensor array. The rows or the columns form one- dimensional subsets of the two-dimensional N x N image. Each row or column of size N is referred to as a frame.
  • the frame number p is initialized to 1.
  • the acoustic signal acquisition is done in such a way that the sparsity of the N x N image is exploited at various levels .
  • randomness is introduced at various levels, for example by suitable placements of the transmitters and the receivers in the number of pulses emitted by the transmitter.
  • the introduction of randomness includes the use of random linear sensor arrays, of sparse sampled irregular arrays or of a steerable electronic array of antennas for signal acquisition.
  • a first sub-unit of the compressive sensing unit (CSU) 20 uses a random matrix for each frame to capture M measurements, where M is much less than the total number N of samples in a row or in a column.
  • the number M of measurements depends on the sparsity of the signal.
  • the number of captured samples M is stored in the buffering unit 60 and is stored along with the frame number p.
  • the M measurements are used for event detection in step 22 and the following steps and, in a step 61, the M measurements are buffered and passed on to a reconstruction unit 70 in step 71.
  • a second sub-unit of the CSU 20 uses a portion P of the M input samples for detection of an event in the frame.
  • the number of samples P is much less than the number M of captured samples.
  • the number P depends on the kind of input signal and is generally less than the number M of input samples.
  • the event could be a target in the context of sonar signals, such as a sea mine.
  • steps 31, 32 and 33 the detection unit 30 uses a suitable detection algorithm in the compressed domain to detect an event.
  • An event is detected in steps 31 and 32, if more than f consecutive frames pass the detection algorithm, wherein step 31 is carried out by a first sub-unit and step 32 is carried out by a second sub-unit of the detection unit 30.
  • the number f is provided in step 81 as a user defined input of the user interface 80.
  • step 32 When a target is detected in step 32, the last frame infor- mation of the set of f frames that satisfies the detection criteria is passed on to the scene recovery unit 40.
  • step 33 it is decided if a pre-determined number of frames has been reached. If yes, the next N x N image is acquired in step 35 and the algorithm loops back to step 13. Otherwise, the frame counter p is incremented by one and the algorithm loops back to step 21.
  • step 41 a first sub-unit of the SRU 40 checks if a scene recovery is required, which depends on the application and is provided by the user input interface.
  • steps 42 and 51 a second sub unit of the SRU 40 passes on the frame information p to the control unit 50. After passing the scene information, the flow continues with the step 33 of checking the end of frames. This synchronization condition is indicated in Fig. 2 by the two circled letter symbols " ⁇ '".
  • control unit 50 controls the buffer unit 60 and the reconstruction unit 70. Furthermore, the control unit 50 keeps track of the frame number p, gets the scene information from the scene recovery unit 40 and passes the scene information on to the buffering unit 60 for processing. Furthermore, the control unit 50 decides when the buffer of the buffering unit 60 is to be flushed and when the reconstruction unit 70 is to be activated.
  • the buffering unit 60 comprises a buffer with two storage spaces, Bl and B2.
  • the storage space Bl can store R frames length M while the storage space B2 can store T frames of length M, wherein R is greater than T .
  • T is chosen greater than f, the number of rows or columns spanned by the target i.e., the size should be greater than f.
  • the size of the storage space Bl should be greater than the size of the sto age space B2 so that no frames are lost in the process of buffering and at the same time no frames with overlap are getting reconstructed.
  • the buffer Bl comprises t extra slots for storing the incoming frames when the reconstruction of the (R - t) frames is carried out.
  • a "slot" refers to a memory space, which is sufficient to take up the relevant information of one frame. This may be, by way of example, the memory space required to store M measurements or coefficients of a suitably chosen set of base functions.
  • the total size of the t extra slots depends on the time taken for passing on the frames to the RU 70.
  • the size of R, T, t, and thereby of the corresponding memory space, depends on the signal.
  • the CU 50 After (R - t) frames of the storage space Bl are passed on to the RU 70, the CU 50 checks for the frame number of the frame that is currently getting processed in the detection unit 30. If, for example, the i-th frame is being processed, then all frames before the (i - f)th frame are cleared from the storage space Bl. The CU 50 also keeps track of the last frame number that was passed on to the reconstruction unit 70 for reconstruction .
  • d is the last frame number that was passed on to the RU 70.
  • the CU 50 passes on the next (R - t) slots to the RU 70 it will check the starting frame number, h, of the next set of (R - t) slots.
  • the CU 50 signals the RU 70 to reconstruct the event scene from the set of frames starting from the h-th or from the (d + 1 ) th frame number, whichever is the lowest .
  • [k-f, k] frames are accessed from storage space Bl . These frames are copied to the storage space B2. The frames from the storage space B2 are passed on to the reconstruction unit 70 for the event scene recovery.
  • the CU 50 resets the storage space B2.
  • the event scene is recovered separately as described above.
  • the event scene is highlighted in the reconstructed image by passing on the detected frames information, thus not requiring a separate buffer B2.
  • the reconstruction unit 70 provides three options for output:
  • the reconstruction unit 70 comprises two processing units ( FPGA, GPU etc.) of which one unit is used for reconstruction of the scene of the event from the frames in the storage space B2 and the other unit is used for reconstruction of the image scene part by part as a set of (R - t) frames from the storage space Bl .
  • a small buffer in the reconstruction unit 70 is used to store the intermediate reconstructed images .
  • the consecutive sets of (R - t) frames, which cover the whole image, are later combined to form the whole image. The latter step is carried out in the case when a complete reconstruction of the whole image scene needs to be provided simultaneously.
  • the reconstruction unit 70 uses one of many available compressive reconstruction algorithms. Suitable compressive reconstruction units can be found in "Wright, Nowak, Figueiredo" and "Bara- niuk” .
  • an implementation of the algorithm may be GPU based.
  • the number of processing units in the reconstruction unit need not be limited to two, but can have any number.
  • the user interface 80 provides information specific to the signal and the scene, like the number of frames f spanned by the target, the M/N ratio, the P/M ratio, the sparsity of the signal, or the size of the buffers t, R and T.
  • one processing unit of the two processing units in the reconstruction unit 70 processes the set of the f frames in parallel.
  • the average of the first ten sparse vectors is assumed as a background training vector.
  • the i-th sparse vector is subtracted from the background sparse vector.
  • K samples of the subtracted vector are taken, with K being much smaller than the length T of the vector.
  • An 1 1 norm reconstruction of the vector gives the location of anomaly values in the vector.
  • the 1 1 norm may be implemented by computing to the sum of the moduli for a given sequence of vector values.
  • the background reduction is one of several techniques that can be used for event detection.
  • the anomaly values are located in consecutive locations in the vector.
  • a similar process is done for all the f frame vectors thereby providing a localization of the event.
  • all the f frames are stacked up there will be a consecutive set of values in the row and the column thus defining a region.
  • the exact set of points constituting the region of the target is marked.
  • the proposed system is capable of delivering a complete image with the target region defined and/or the target region alone.
  • Figs. 3 - 15 show an evaluation of one of the side-scan sonar images acquired with a Klein 2000 side-scan sonar device of Klein Associates, Inc., of Nashua, New Hampshire, at a sound frequency of 500 kHz.
  • Each image line or frame consists of both the starboard and portside images, wherein starboard and portside, which is also known as "larboard", refer to the right and left side in the direction of travel, respectively.
  • Some images contain one underwater mine each and some do not contain any mine.
  • a careful analysis of the data set shows that the distortions introduced by yaw, pitch and roll movements of the sonar device in the water are relatively small and need not be addressed in this particular case.
  • the magnitudes of the coefficients are provided in arbitrary units (a.u. ) .
  • the nature of the side-scan sonar is such that the raw data exhibits across-track geometric distortion (slant-range distortion) .
  • slant-range correction ground- range projection
  • the sonar imagery is resampled to 0.1 m x 0.1 m per pixel.
  • a pixel is also referred to as a "bin”.
  • a resulting size of the sonar image is 1034 x 1491.
  • the sonar image is processed rowwise. These captured measurements are passed on to the buffer .
  • the fraction M/N which is 0.52 in the given example, is typically in the range [0.5, 0.8] .
  • the sonar signal is assumed to be sparse in a given wavelet basis. The data is processed row-wise in accordance with the sensing approach.
  • Fig. 3 shows an original side scan signal, wherein a signal intensity in arbitrary units (a.u.) is plotted as a function of the measurement bins.
  • Fig. 4 shows a randomly sample sparse signal that is derived from the original signal by applying a sparsity inducing matrix, also known as transformation matrix, and the random projection matrix ⁇ to the original signal.
  • Fig. 5 shows a side scan sonar row vector recovered from the Daubechies (D10) wavelet. The row-wise processing can be extended for the entire sonar image.
  • D10 Daubechies
  • Each of the input compressed sensed vectors is copied to the buffering unit 60, which comprises two memory slots Bl and B2.
  • the memory slot Bl can hold up to 20 frames whereas the memory slot B2 can hold up to 10 frames.
  • the size of the memory slot B2 is decided as twice the number f frames spanned by the target, which in this case is 5.
  • the size of the memory slot Bl is twice the size of the memory slot B2.
  • control unit After the memory slot Bl is filled with up to 10 frames the control unit signals the buffer to copy those 10 frames to the reconstruction unit 70.
  • the control unit 50 checks for the current frame that is being processed. If the 20-th frame is getting processed, the CU 50 sends control signals to delete all the frames before the 15-th frame (20-5) .
  • the buffer keeps track of the last frame number that was passed on to the reconstruction unit 70. If the current frame minus the number of target frames is more than the last frame number passed on to the reconstruction unit 70, then the last frame number is retained.
  • the size of the memory slot B2 is maintained in such a way that even if there are two consecutive set of mine frames they can be detected.
  • Each of the sparse vectors s is reconstructed using the SpaR SA approach according to "Wright, Nowak, Figueiredo".
  • a reconstructed sonar image according to SpaRSA is as shown in Fig. 7. If the 50-th frame is detected to contain an underwa ter mine then the CU 50 copies the 6 frames with frame numbers 50 - 5 to 50 on to B2, which is passed on to the recon- struction unit 70. This procedure is repeated for the rest of the image .
  • Fig. 6 shows an original image taken at a portside of the s nar scanner with a prominent feature 111 and Fig. 7 shows a reconstructed image with a prominent feature 112.
  • Figs. 12 and 13 The raw data of Fig. 12 shows a prominent feature 115, which is reflected in the row detection event 116 the detection diagram of Fig. 13.
  • the row detection event 116 occurs approximately at row 800. This corresponds to the extension of the prominent feature 115 between rows 800 and 900.
  • the input image was slant range corrected and the number of samples in the row was reduced to 500.
  • Each row of the original data is compressively sensed by a random matrix ⁇ (50 x 500) as shown in Figs. 8 to 11.
  • ⁇ 50 x 500
  • 50 random measurements out of 500 are captured.
  • These measurements could also be a subset P of the M measurements which are initially captured, as illustrated by the reduction step 23 of Fig. 2.
  • the number of samples P is much less than M and the range of P/M is typically [0.1, 0.5].
  • compressive background subtraction is carried out for each row of the sonar image.
  • the row which crosses the set threshold is marked as the row containing the target.
  • the events are marked by the event detection result 114 in Fig 9, which corresponds to the prominent feature 113 of Fig. 8, and by the event detection result 116 of Fig. 13, which corresponds to the prominent feature 115 of Fig. 12.
  • the background training was done using the first 10 rows of the image.
  • the DU 30 gathers information on the first 10 samples stored in the buffer at the start.
  • the impact of the variation of the number of measurements on the target detection is as shown in Fig. 14.
  • the probability of detection exponentially rises with increase in the number of the measurements.
  • the number of measurements reaches a regime after which the increase in the detection improvement is very marginal. It is to be noted that already for M equal to 0.15 N the probability of detection reaches 1.
  • the number of measurements required for good detection after reconstruction is more than the number of measurements required when detection is performed directly on the measurements . This is because for a good reconstruction of a sonar image M is lower bounded by the sparsity.
  • Fig. 16 shows a sonar data acquisition system 120 according to the present application.
  • the sonar data acquisition system of Fig. 16 comprises a pulse generator 121, a sender/receiver 122 which comprises a sound coupling device 125 such as a loudspeaker or a piezocrystal , an A/D converter 123 and a signal processor 124.
  • the signal processor 124 comprises the image reconstruction and event detection system 1 of Fig. 1. In Fig. 1, the uncompressed signal is symbolized by the letter "y".
  • Fig. 17 shows a sectional enlargement of Fig. 1 that illustrates a buffer management according to the specification.
  • the buffering unit comprises a first buffer 65, labelled as "Bl", and a second buffer 66, labelled as "B2".
  • the first buffer 65 has sufficient memory for storing R frames and the second buffer 66 has sufficient memory for storing T frames, wherein R is greater than T.
  • Data flows from the compressive sensing unit 20 to the first buffer 65, from the first buffer 65 to the second buffer 66, from the first buffer 65 to the reconstruction unit 70 and from the second buffer 66 to the reconstruction unit 70 are indicated by respective arrow symbols.
  • a method for event detection and reconstruction of image raw data in a reflective wave device comprising:
  • the frame-by-frame processing comprises a frame-by-frame image reconstruction
  • Method according to item 1, comprising:
  • the size of the buffer being larger than the amount required to store the detection/reconstruction subset by a buffering size, wherein the buffering size depends on the time required to transmit the detection/reconstruction subset to a processing unit.
  • Method according to item 2 comprising:
  • Method comprising, if an event is detected, reconstructing an event scene .
  • the compressive sensing comprises a random projection of the input set that is chosen such that the complete input set of data need not to be used.
  • a reflective wave device comprising:
  • a sensor for acquisition of image raw data a computation unit with a processor, a memory and an output unit, the computation unit comprising a software program which provides the following method steps:
  • the frame-by-frame processing comprises a frame-by-frame image reconstruction
  • a reflective wave device comprising:
  • a detection unit for detecting an event based on the current frame, for reconstructing an event based on a detection subset, wherein the detection subset is derived from the current frame and the pre-determined number of retrieved frames, for comparing the reconstructed event with a pre-determined event criterion and for out- putting a flag signal if the reconstructed event matches the event criterion.
  • a reflective wave device according to item 9 or item 10, further comprising
  • a buffering unit with a buffer "Bl” a size of the buffer "Bl” being larger than the amount required to store the detection subset by a buffering size, wherein the buffering size depends on a time required to transmit the detection subset to a processing unit.
  • a reflective wave device comprising a second buffer "B2", a size of the second buffer being sufficient for storing the detection subset.
  • a mobile underwater sonar device with a reflective wave device according to one of the items 9 to 12, the reflective wave device being provided at a body of the mobile underwater sonar device, wherein the sensor for acquiring the image raw data is provided by one or more acoustic sensors that are arranged laterally to a direction of travel of the mobile underwater sonar device.
  • a ship with a reflective wave device according to one of the items 9 to 12, the reflective wave device being provided at a body of the ship, wherein the sensor for acquiring the image raw data is provided by one or more acoustic sensors that are arranged laterally to a direction of travel of the ship.
  • An unmanned aerial vehicle with a reflective wave device according to one of the items 9 to 12, the reflective wave device being provided at a body of the unmanned aerial vehicle, wherein the sensor for acquiring the im- age raw data is provided by one or more radar sensors that are arranged laterally to a direction of travel of the unmanned aerial vehicle
  • the reflective wave device be ing provided at a body of the aircraft, wherein the sen sor for acquiring the image raw data is provided by one or more radar sensors that are arranged laterally to a direction of travel of the aircraft.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Measurement Of Velocity Or Position Using Acoustic Or Ultrasonic Waves (AREA)

Abstract

The present specification discloses a method for event detection and reconstruction of image raw data from a reflective wave device and the corresponding device. Image raw data is acquired with a sensor and a compressed representation of the image raw data is derived with compressive sensing. The com- pressed representation is processed frame-by-frame and, during the frame-by-frame processing of the compressed representation, a detection of events is performed based on the cur- rent frame of the compressed representation, wherein the frame-by-frame processing comprises a frame-by-frame image reconstruction. If an event is detected in the current frame, a pre-determined number of frames that precede the current frame is received. An event is reconstructed based on a detection subset, wherein the detection subset is derived from the current frame and the pre-determined number of retrieved frames.

Description

TITLE
REFLECTIVE WAVE DEVICE FOR SIMULTANEOUS EVENT DETECTION AND SIGNAL RECONSTRUCTION USING COMPRESSIVE MEASUREMENTS
BACKGROUND
Sonar underwater imaging has diversified since it was first developed and now has a wide range of applications . With present day technology small-scale devices, which may be remote controlled or even autonomous robots, can be deployed for underwater surveillance. The range of applications includes detection of hazardous materials in a harbour, mine sweeping, geophysical surveys, surveillance of naval traffic and others. In particular, for small-scale devices it is desirable to provide image-processing units that are efficient with regard to processing and memory requirements and at the same time reasonably fast.
Compressive sensing, which is also known as compressed sensing, compressive sampling, or sparse sampling, is a signal processing technique for efficiently acquiring and reconstructing a signal by finding solutions to underdetermined linear systems. By utilizing knowledge about a signal's spar- sity, the signal may be reconstructed with fewer samples than the Nyquist-Shannon theorem would require. Herein, sparsity of the signal refers to the degree to which the signal is represented by just a few coefficients of a given set of base functions, the other coefficients being zero, close to zero or at least much smaller. Most of the known compressive sensing systems are of two kinds: imaging and subsequent reconstruction from compressive measurements or detection of an event directly from the compressive measurements. For example, the US 2013/0128042 discloses an event detection method using a compressive-sensing hyperspectral imaging architecture. Therein, an incident light beam is split up into its spectral components. An image portion or its time sequence is examined in the selected spectral range for a pre-determined trigger condition such as exceedance of a pre-determined magnitude or of a predetermined rate of change.
According to the US 2013/01280 42, an event detection is per- formed in the compressed sensed samples . By contrast, a meth- od and device according to the present specification allows an event detection to be performed simultaneously during the reconstruction of a compressed sensed image.
For easier reading of the present specification, references are listed below and they are referred in the specification text using the names of the authors as abbreviation.
"Wright, Nowak, Figueiredo": S. J.Wright, R. Nowak, and M. A. T. Figueiredo, "Sparse reconstruction by Separable Approximation," IEEE Transactions on Signal Processing, vol. 57, no. 7, pp. 2479-2493, July 2009.
"Kim, Koh, Lustig, Boyd, Grinvesky": S. Kim, K.Koh, M. Lust- ig, S. Boyd, and D. Grinvesky, "An interior-point method for large-scale 11-regularized least squares, " IEEE Journal on Selected Topics in Signal Processing , vol. 1, no. 4, pp. 606-617, Dec. 2007. "Cevher, Sankaranarayanan, Duarte, Dikpal": Volkan Cevher and Aswin Sankaranarayanan and Marco F. Duarte and Dikpal Reddy and Richard
"Baraniuk": G. Baraniuk, "Compressive sensing for background subtraction" in Proc. European Conf. Comp . Vision (ECCV) , Berlin, Germany, 2008.
BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS
An image reconstruction system according to the present specification provides the ability for simultaneous event detection, corresponding scene reconstruction and whole image reconstruction irrespective of an event in real time. In the present specification a system and method is disclosed that can be used for simultaneous reconstruction of the scene and detection of event in real time using suitable optimization and reconstruction algorithms, respectively. Herein, a simultaneous execution of two processes is understood in a general sense that the two processes comprise common steps and/or comprise steps that are executed in parallel for both processes .
According to the present specification, a method for event detection and reconstruction of image raw data in a reflective wave device is disclosed, in particular for a simultaneous or real-time event detection which is performed during the data acquisition and reconstruction of image raw data. Herein, a reflective wave device refers to a device for generating and receiving sound waves or electromagnetic waves such as a sonar, a seismic device, or any form of radar. Image raw data is acquired from a sensor for sensing reflected sound waves or electromagnetic waves and a compressed representation of the image raw data is with compressive sensing. Compressive sensing, which is explained in further detail in the specification text below, involves a sparse representation of the image raw data in a suitable set of base functions and a random projection. The random projection reduces the amount of data, wherein the randomness of the random projection helps to avoid image artefacts that are due to a systematic bias.
The compressed representation is processed frame-by-frame, wherein the frames may represent in particular rows or columns of a two-dimensional image, or also equal sized blocks of data of the two-dimensional image. The two-dimensional dimension may also be provided by a slice of a three- dimensional image and the physical units may for example be space units, time units or units resulting from a transformation of the data, such as a Fourier transformation, a Wavelet transformation, a Radon transformation or others.
During the frame-by-frame processing of the compressed representation a detection of events is performed which is based on a current frame of the compressed representation. Herein, the frame-by-frame processing comprises a frame-by-frame image reconstruction.
If an event is detected in the current frame of the compressed representation, a pre-determined number of frames that precede the current frame is received. Herein, "preceding" is defined according to an order in which the frames are processed . An event is reconstructed based on a detection subset, wherein the detection subset is derived from the current frame and the pre-determined number of retrieved frames. In particular, the detections subset can also be identical with the current frame and the pre-determined number of retrieved frames . The reconstruction of the event comprises reconstruction of event data such as event type, event time, event location, rate of change etc. By contrast, a reconstruction of an event scene refers to a more detailed reconstruction of a spatial arrangement of the event. This may, among others, involve orientation, spatial extent, size and other features defining a spatial arrangement.
As soon as an event is detected in a frame that is currently being processed, the processing unit can trigger an adequate reaction. According to the present specification it is not necessary to defer the event detection until a complete image has been reconstructed but, rather, the event detection is performed during the process of image reconstruction.
According to a further embodiment, the detection subset is stored in memory slots of a buffer "Bl". The size of the buffer "Bl" is larger than the amount required to store the detection subset by a buffering size. The buffering size depends on the time required to transmit the detection subset to a processing unit. In the context of the present specification, the detection subset for reconstructing the event corresponds to (R-t) slots that represent (R-t) frames and the buffering size corresponds to t slots representing t frames .
According to yet another embodiment, the detection subset is copied to a second buffer "B2" and the content of the second buffer "B2" is sent to a scene reconstruction unit, if an event is detected in the current frame.
According to the present specification, there are various possibilities in which a detected event may be highlighted or signalled .
According to one embodiment, in the case that an event is detected, a data message is generated which comprises event data of the detected event, such as event time, location, type, intensity. This can, in turn trigger an output of a signal that is indicative of the detected event, such as a light signal or an acoustic warning or a display of event details on a screen, LCD display or the like. In addition, the complete reconstructed image or part of it may be output.
According to another embodiment, if an event is detected, an event scene is reconstructed, which is in particular based on the detection subset and which is indicate of the spatial arrangement of the event and/or of the change thereof. The reconstruction of the event scene may comprise, for example, the spatial orientation and extension of the event, a direction of movement of the event, a rate of change of the event etc .
According to yet another embodiment, if an event is detected an image in which the event scene is marked is reconstructed, wherein the image represents the raw data. The marked event scene is based on the detection subset.
Furthermore, the method may comprise a detection whether an event is present in the current frame using a part P of se lected measurements or coefficients for the current frame. The selection of only a part P of the selected measurememts or coefficients further reduces the amount of data.
According to a further embodiment, the compressive sensing comprises a random projection that is chosen such that 50% or less of the input data is used, which means that the complete input set of data need not to be used. In particular, the random projection can be performed using a random projection matrix with a ratio of rows to columns, or columns to rows, as the case may be, which is 50% or less.
This will in general result in a sub-Nyquist sampling, wherein the Nyquist frequency is defined by a desired level of resolution. The random selection avoids artefacts in the reconstructed image due to a systematic bias.
In particular, the derivation of the compressed representation may comprise an acquisition of a two-dimensional image and partitioning of the two-dimensional image in one- dimensional frames. For a predetermined number p of the one- dimensional frames repeating the steps of generating a random projection of a frame and of representing the random projection by M measurements or coefficients of a set of base functions to generate a sparse vector of base function coefficients are repeated.
For further reducing the amount of data a part P of the M measurements or coefficients may be selected for use in event detection. For example, if the event occupies only a small region of the frame and the base functions are concentrated around a region, only coefficients of base functions contribute which cover the event region. It is detected if an event has occurred according to a predetermined event criterion based on the P selected measurements or coefficients. The event criterion may in particular comprise comparison with a threshold or a threshold event.
A two-dimensional image according to the present specification may represent a two-dimensional distribution of sensor signals, such as sonar signals. The two- dimensional distribution may be spatial but it may also be spatiotemporal distribution. In general, the dimensions of the image may also result from a transformation of the original image. For example, a Fourier transformation with respect to two space dimensions will result in a two-dimensional image in which the axes represent frequencies.
In a further aspect, the present application discloses a reflective wave device comprising a sensor for acquisition of image raw data for sensing the reflected acoustic or electromagnetic waves and a computation unit with a processor, a memory and an output unit. The computation unit comprising a software program which provides for deriving a compressed representation of the image raw data with compressive sensing, frame-by-frame processing of the compressed representation. During the frame-by-frame processing of the compressed representation, a detection of an event is performed based on a current frame of the compressed representation. Herein, the frame-by-frame processing comprises a frame-by-frame image reconstruction .
If an event is detected in the current frame of the compressed representation, a pre-determined number of frames that precede the current frame, is retrieved, wherein the numbering of the frames is defined according to the order in which the frames are processed. This order may correspond to the natural order or rows or the natural order of columns in which the image data is acquired.
An event is reconstructed based on a detection subset, wherein the detection subset is derived from - or is identical with - the current frame and the pre-determined number of retrieved frame . The reconstructed event is compared with a pre-determined threshold event and a flag signal is output if the reconstructed event matches the threshold event.
In a further aspect, the present specification disclose a reflective wave device in which the abovementioned functions are realized partially or entirely by specific dedicated hardware components, such as application specific integrated circuits or electric circuitry. Part of the functions may also be realized by software modules. The software modules and the specific hardware components are referred as "units" or "sub-units " .
The reflective wave device according to this aspect comprises a sensor for acquisition of image raw data, which is operable for sensing the reflected waves. In particular, the sensor may be part of an image acquisition device for acquiring a two-dimensional image.
Furthermore, the reflective wave device comprises a compressive sensing unit for deriving a compressed representation from the image raw data. The compressive sensing unit may in turn comprise a first sub-unit for selecting a current frame from the image raw data and for generating a random projection of the frame and a second sub-unit for representing the frame by M measurements or coefficients of a pre-determined set of base functions and a third sub-unit for selecting a subset of P measurements or coefficients from the M measurements or coefficients.
Moreover, the reflective wave device comprises a detection unit with a sub-unit for detecting an event based on the current frame. The detection unit is furthermore operative to retrieve a pre-determined number of frames which precede the current frame according to the order in which the frames are processed, to reconstruct an event based on a detection subset, wherein the detection subset is derived from, or is identical to, the current frame and the pre-determined number of retrieved frames, to compare the reconstructed event with a pre-determined event criterion, such as a threshold event, and to output a flag signal if the reconstructed event matches the event criterion.
According to a further embodiment, the reflective wave device comprises a buffering unit with a buffer "Bl", a size of the buffer "Bl" being larger than the amount required to store the detection subset by a buffering size, wherein the buffering size depends on a time required to transmit the detection subset to a processing unit.
According to yet another embodiment, the reflective wave < vice comprises a second buffer "B2". A size of the second buffer is sufficient for storing the detection subset .
Furthermore, the current specification disclose a mobile underwater sonar device, which may be self-propelled or moved by an external device such as a string. The sonar device comprises one of the abovementioned reflective wave devices. The reflective wave device is provided at, or in other words in- side or onto, a body of the mobile underwater sonar device .
The sensor for acquiring the image raw data is provided by one or more acoustic sensors that are arranged laterally, which is starboard or larboard, to a direction of travel of the mobile underwater sonar device.
Furthermore, the current specification discloses a ship with one of the abovementioned reflective wave devices. The re- flective wave device is provided at, or in other words in or onto, a body of the ship. The sensor for acquiring the image raw data is provided by one or more acoustic sensors that are arranged laterally, which is starboard or larboard, to a di- rection of travel of the ship .
Moreover, the current specification discloses an unmanned aerial vehicle with one of the abovementioned reflective wave devices. The reflective wave device is provided at, or in other words in or onto, a body of the unmanned aerial vehicle. The sensor for acquiring the image raw data is provided by one or more radar sensors that are arranged laterally to a direction of travel of the unmanned aerial vehicle, which may be at the bottom or slightly sideways but also at other locations .
Moreover, the present specification discloses an aircraft with one of the abovementioned reflective wave . The reflective wave device is provided at, or in other words in or on an outer surface of, a body of the aircraft. The sensor for acquiring the image raw data is provided by one or more radar sensors that are arranged laterally to a direction of travel of the aircraft, which may be at the bottom or slightly sideways but also at other locations .
The object of the present specification is now explained with reference to the following Figures in which
Fig. 1 shows a diagram of a data reconstruction and event detection system,
Fig. 2 shows a method for data reconstruction and event detection using the system of Fig. 1,
Fig. 3 shows a row of an original signal,
Fig. 4 shows a randomly sampled sparse signal,
Fig. 5 shows a signal that is reconstructed from the signal of Fig. 4,
Fig. 6 shows an original image,
Fig. 7 shows an image that is reconstructed from the
sparse image according to the SpaRSa algorithm,
Fig. 8 shows a raw data image,
Fig. 9 shows a row-wise event detection derived from the image of Fig .8 ,
Fig. 10 shows a row of an original signal,
Fig. 11 shows a randomly sampled signal from the signal of
Fig. 10,
Fig. 12 shows a row-wise detection result,
Fig. 13 shows a row of an original signal,
Fig. 14 shows a detection probability as a function of a number of measurements,
Fig. 15 shows detection probabilities as a function of a number of measurements for an algorithm with and without reconstruction,
Fig. 16 shows a sonar device comprising the detection system of Fig. 1, and Fig. 17 shows a sectional enlargement of Fig. 1 that illustrates a buffer management according to the specification .
DETAILED DESCRIPTION ion, details
the present
d in the art,
bodiments may be practised without such details.
For example, various basis functions may be used in order to achieve a sparsely populated representation of the image data in terms of basis function coefficients. In particular, one may choose a set of wavelet functions that differs from the Daubechies D10 wavelets of the present specification, for example one may choose the basis function from the D2, D , D8 or from the D12, D14, D20 Daubechies wavelets.
Instead of an algorithm that is based on the 1 1 norm, other convex optimization or greedy algorithms can be used to find an approximation of the original image, see for example "Wright, Nowak, Figueiredo". In general, any of the known compressive sensing reconstruction algorithms may be used in a device or in a method according to the present specification .
While the subject of the current specification is explained with respect to a sonar, it may also be used for a radar using electromagnetic waves or even for medical image acquisition devices, such as nuclear magnetic resonance imaging devices . Fig. 1 shows a system 1 for simultaneous image reconstruction and event detection. Therein, block 10 represents the N x N sonar processed image 10 from the raw data. Block 20 represents a compressive sensing unit 20 (CSU) for generating a compressive sensed input signal from the N x N raw image. Block 30 represents a detection unit 30 where the compressively sensed samples are used for detection. Block 40 represents a scene recovery unit (SRU) 40, which takes the frame information from the detection unit and passes the information on to a control unit 50 for reconstruction of the event scene and/or the complete image. Block 50 represents the control unit (CU) 50 for passing control signals to the buffering unit 60 and to a reconstruction unit 70. Block 60 represents the buffering unit 60, which serves as a temporary storage of the compressively sensed data for reconstruction purposes .
Block 70 represents the reconstruction unit 70. The reconstruction unit 70 comprises two processing units which are not shown in Fig. 1. The processing units may be provided by special purpose hardware such as a graphics processing unit (GPU) or a field programmable gate array (FPGA) . During operation, the processing units schedule the task of reconstruction of the scene corresponding to the samples in the buffering unit 60 and also schedule the reconstruction of the event scene simultaneously. Block 80 represents a user interface unit 80 which receives inputs that are specific to the application for example-the scenes to be recovered, the number of frames spanned by the target. By way of example, the user interface unit 80 may be provided by a LCD screen and a keypad, by an input device that is connectable to an image acquisition device by cable or by wireless connection. As an example for a signal acquisition device, a sonar is shown in Fig. 16.
Fig. 2 shows a flow diagram o a method for image reconstruc tion and event detection usin the system 1 of Fig. 1. Fig. is now explained in further d tail in the context of sonar signal processing.
In a step 11, an N x N sonar image is acquired from raw data of a sonar. Specifically, the N x N sonar image is obtained by translating the raw acoustic measurements obtained from back scattered responses into images. In a step 12, the images are processed row-wise or column wise depending on the swath of the sensor array. The rows or the columns form one- dimensional subsets of the two-dimensional N x N image. Each row or column of size N is referred to as a frame. In a step 13, the frame number p is initialized to 1.
In one specific embodiment, the acoustic signal acquisition is done in such a way that the sparsity of the N x N image is exploited at various levels . To this end, randomness is introduced at various levels, for example by suitable placements of the transmitters and the receivers in the number of pulses emitted by the transmitter. The introduction of randomness includes the use of random linear sensor arrays, of sparse sampled irregular arrays or of a steerable electronic array of antennas for signal acquisition.
In a step 22, a first sub-unit of the compressive sensing unit (CSU) 20 uses a random matrix for each frame to capture M measurements, where M is much less than the total number N of samples in a row or in a column. The number M of measurements depends on the sparsity of the signal. The number of captured samples M is stored in the buffering unit 60 and is stored along with the frame number p.
According to the present specification, the M measurements are used for event detection in step 22 and the following steps and, in a step 61, the M measurements are buffered and passed on to a reconstruction unit 70 in step 71.
In a step 23, a second sub-unit of the CSU 20 uses a portion P of the M input samples for detection of an event in the frame. The number of samples P is much less than the number M of captured samples. The number P depends on the kind of input signal and is generally less than the number M of input samples. The event could be a target in the context of sonar signals, such as a sea mine.
In steps 31, 32 and 33 the detection unit 30 uses a suitable detection algorithm in the compressed domain to detect an event. An event is detected in steps 31 and 32, if more than f consecutive frames pass the detection algorithm, wherein step 31 is carried out by a first sub-unit and step 32 is carried out by a second sub-unit of the detection unit 30. The number f is provided in step 81 as a user defined input of the user interface 80.
When a target is detected in step 32, the last frame infor- mation of the set of f frames that satisfies the detection criteria is passed on to the scene recovery unit 40.
In step 33 it is decided if a pre-determined number of frames has been reached. If yes, the next N x N image is acquired in step 35 and the algorithm loops back to step 13. Otherwise, the frame counter p is incremented by one and the algorithm loops back to step 21.
In step 41 a first sub-unit of the SRU 40 checks if a scene recovery is required, which depends on the application and is provided by the user input interface. In steps 42 and 51, a second sub unit of the SRU 40 passes on the frame information p to the control unit 50. After passing the scene information, the flow continues with the step 33 of checking the end of frames. This synchronization condition is indicated in Fig. 2 by the two circled letter symbols "Α'".
During operation, the control unit 50 controls the buffer unit 60 and the reconstruction unit 70. Furthermore, the control unit 50 keeps track of the frame number p, gets the scene information from the scene recovery unit 40 and passes the scene information on to the buffering unit 60 for processing. Furthermore, the control unit 50 decides when the buffer of the buffering unit 60 is to be flushed and when the reconstruction unit 70 is to be activated.
The buffering unit 60 comprises a buffer with two storage spaces, Bl and B2. The storage space Bl can store R frames length M while the storage space B2 can store T frames of length M, wherein R is greater than T . T is chosen greater than f, the number of rows or columns spanned by the target i.e., the size should be greater than f. The size of the storage space Bl should be greater than the size of the sto age space B2 so that no frames are lost in the process of buffering and at the same time no frames with overlap are getting reconstructed. The buffer Bl comprises t extra slots for storing the incoming frames when the reconstruction of the (R - t) frames is carried out. Herein, a "slot" refers to a memory space, which is sufficient to take up the relevant information of one frame. This may be, by way of example, the memory space required to store M measurements or coefficients of a suitably chosen set of base functions.
The total size of the t extra slots depends on the time taken for passing on the frames to the RU 70. The size of R, T, t, and thereby of the corresponding memory space, depends on the signal. After the first (R - t) slots of storage space Bl are filled with the frames, the control unit 50 signals to pass a copy of the set of frames to the reconstruction unit 70 for image recovery in a step 71. The incoming frames are then stored in the next t free slots. The control unit 50 also checks on the value of p in step 33 of Fig. 2.
After (R - t) frames of the storage space Bl are passed on to the RU 70, the CU 50 checks for the frame number of the frame that is currently getting processed in the detection unit 30. If, for example, the i-th frame is being processed, then all frames before the (i - f)th frame are cleared from the storage space Bl. The CU 50 also keeps track of the last frame number that was passed on to the reconstruction unit 70 for reconstruction .
Supposed d is the last frame number that was passed on to the RU 70. When the CU 50 passes on the next (R - t) slots to the RU 70 it will check the starting frame number, h, of the next set of (R - t) slots. The CU 50 signals the RU 70 to reconstruct the event scene from the set of frames starting from the h-th or from the (d + 1 ) th frame number, whichever is the lowest .
If an event is detected at the k-th frame, then [k-f, k] frames are accessed from storage space Bl . These frames are copied to the storage space B2. The frames from the storage space B2 are passed on to the reconstruction unit 70 for the event scene recovery.
Once the set of target frames are passed to the storage space B2, the CU 50 resets the storage space B2. According to one embodiment, the event scene is recovered separately as described above. According to another embodiment, the event scene is highlighted in the reconstructed image by passing on the detected frames information, thus not requiring a separate buffer B2.
The reconstruction unit 70 provides three options for output:
(i) the reconstructed image itself
(ii) the event scene alone
(iii) the reconstructed image with the event scene marked on it .
The reconstruction unit 70 comprises two processing units ( FPGA, GPU etc.) of which one unit is used for reconstruction of the scene of the event from the frames in the storage space B2 and the other unit is used for reconstruction of the image scene part by part as a set of (R - t) frames from the storage space Bl .
A small buffer in the reconstruction unit 70 is used to store the intermediate reconstructed images . The consecutive sets of (R - t) frames, which cover the whole image, are later combined to form the whole image. The latter step is carried out in the case when a complete reconstruction of the whole image scene needs to be provided simultaneously. The reconstruction unit 70 uses one of many available compressive reconstruction algorithms. Suitable compressive reconstruction units can be found in "Wright, Nowak, Figueiredo" and "Bara- niuk" .
For enabling faster processing, an implementation of the algorithm may be GPU based. The number of processing units in the reconstruction unit need not be limited to two, but can have any number. The user interface 80 provides information specific to the signal and the scene, like the number of frames f spanned by the target, the M/N ratio, the P/M ratio, the sparsity of the signal, or the size of the buffers t, R and T.
For the localization of the event scene one processing unit of the two processing units in the reconstruction unit 70 processes the set of the f frames in parallel. In the reconstruction unit 70, the average of the first ten sparse vectors is assumed as a background training vector. The i-th sparse vector is subtracted from the background sparse vector. K samples of the subtracted vector are taken, with K being much smaller than the length T of the vector. An 1 1 norm reconstruction of the vector gives the location of anomaly values in the vector. The 1 1 norm may be implemented by computing to the sum of the moduli for a given sequence of vector values. The background reduction is one of several techniques that can be used for event detection.
Typically, the anomaly values are located in consecutive locations in the vector. A similar process is done for all the f frame vectors thereby providing a localization of the event. When all the f frames are stacked up there will be a consecutive set of values in the row and the column thus defining a region. The exact set of points constituting the region of the target is marked. Thus, the proposed system is capable of delivering a complete image with the target region defined and/or the target region alone.
Experimental result and analysis
Figs. 3 - 15 show an evaluation of one of the side-scan sonar images acquired with a Klein 2000 side-scan sonar device of Klein Associates, Inc., of Nashua, New Hampshire, at a sound frequency of 500 kHz. Each image line or frame consists of both the starboard and portside images, wherein starboard and portside, which is also known as "larboard", refer to the right and left side in the direction of travel, respectively.
Some images contain one underwater mine each and some do not contain any mine. A careful analysis of the data set shows that the distortions introduced by yaw, pitch and roll movements of the sonar device in the water are relatively small and need not be addressed in this particular case. The magnitudes of the coefficients are provided in arbitrary units (a.u. ) .
The nature of the side-scan sonar is such that the raw data exhibits across-track geometric distortion (slant-range distortion) . Hence, in order to have an accurate determination of the location of a target, slant-range correction (ground- range projection) is required. In the examples shown, the sonar imagery is resampled to 0.1 m x 0.1 m per pixel. In this context, a pixel is also referred to as a "bin". A resulting size of the sonar image is 1034 x 1491. According to the example of Figs. 3 to 15, the sonar image is processed rowwise. These captured measurements are passed on to the buffer .
A random projection matrix Φ of dimension M x N with M = 776 and N = 1491 is used to capture the input vector measurements y in steps 21 and 22 of Fig. 2. The fraction M/N, which is 0.52 in the given example, is typically in the range [0.5, 0.8] . The sonar signal is assumed to be sparse in a given wavelet basis. The data is processed row-wise in accordance with the sensing approach.
Fig. 3 shows an original side scan signal, wherein a signal intensity in arbitrary units (a.u.) is plotted as a function of the measurement bins. Fig. 4 shows a randomly sample sparse signal that is derived from the original signal by applying a sparsity inducing matrix, also known as transformation matrix, and the random projection matrix Φ to the original signal. Fig. 5 shows a side scan sonar row vector recovered from the Daubechies (D10) wavelet. The row-wise processing can be extended for the entire sonar image.
Each of the input compressed sensed vectors is copied to the buffering unit 60, which comprises two memory slots Bl and B2. The memory slot Bl can hold up to 20 frames whereas the memory slot B2 can hold up to 10 frames. The size of the memory slot B2 is decided as twice the number f frames spanned by the target, which in this case is 5. The size of the memory slot Bl is twice the size of the memory slot B2.
After the memory slot Bl is filled with up to 10 frames the control unit signals the buffer to copy those 10 frames to the reconstruction unit 70. The control unit 50 then checks for the current frame that is being processed. If the 20-th frame is getting processed, the CU 50 sends control signals to delete all the frames before the 15-th frame (20-5) . The buffer keeps track of the last frame number that was passed on to the reconstruction unit 70. If the current frame minus the number of target frames is more than the last frame number passed on to the reconstruction unit 70, then the last frame number is retained.
For example, if the last frame number that was passed on to the reconstruction unit 70 is 20 and the frame being processed by the detection unit 30 is 27 then the CU 50 checks if 22 (= 27-5) is greater than 20 and retains the smallest o both numbers i.e., 20. In this way, it is ensured that no frames are lost in reconstruction and no overlapping happens
The next time after 10 slots of the memory slot Bl gets filled, the CU 50 copies the next 10 slots, starting from th last frame number, to the reconstruction unit 70. In between if a target is detected at the 18-th frame, then frames in the range [13 (=18-5), 18] are copied to the memory slot B2. The CU 50 then copies these frames to RU 70 and resets the memory slot B2. The size of the memory slot B2 is maintained in such a way that even if there are two consecutive set of mine frames they can be detected.
Each of the sparse vectors s is reconstructed using the SpaR SA approach according to "Wright, Nowak, Figueiredo". A reconstructed sonar image according to SpaRSA is as shown in Fig. 7. If the 50-th frame is detected to contain an underwa ter mine then the CU 50 copies the 6 frames with frame numbers 50 - 5 to 50 on to B2, which is passed on to the recon- struction unit 70. This procedure is repeated for the rest of the image .
Fig. 6 shows an original image taken at a portside of the s nar scanner with a prominent feature 111 and Fig. 7 shows a reconstructed image with a prominent feature 112. The recon structed image of Fig. 7 was reconstructed using random sam pling of 49% of the data, corresponding to a ratio M/N = 0. of the dimensions of the random projection matrix Φ.
Compressive background subtraction according to "Cevher, San- karanarayanan, Duarte, Dikpal" was done on the sonar image from images of a database acquired by the "Klein 2000" sonar side scan system. The row-wise detection results for a sonar image having homogenous background are shown in Figs. 12 and 13. The raw data of Fig. 12 shows a prominent feature 115, which is reflected in the row detection event 116 the detection diagram of Fig. 13. The row detection event 116 occurs approximately at row 800. This corresponds to the extension of the prominent feature 115 between rows 800 and 900.
For the event detection, the input image was slant range corrected and the number of samples in the row was reduced to 500. Each row of the original data is compressively sensed by a random matrix Φ (50 x 500) as shown in Figs. 8 to 11. Thus, 50 random measurements out of 500 are captured. These measurements could also be a subset P of the M measurements which are initially captured, as illustrated by the reduction step 23 of Fig. 2. The number of samples P is much less than M and the range of P/M is typically [0.1, 0.5].
Then, compressive background subtraction is carried out for each row of the sonar image. The row which crosses the set threshold is marked as the row containing the target. The events are marked by the event detection result 114 in Fig 9, which corresponds to the prominent feature 113 of Fig. 8, and by the event detection result 116 of Fig. 13, which corresponds to the prominent feature 115 of Fig. 12.
The background training was done using the first 10 rows of the image. The DU 30 gathers information on the first 10 samples stored in the buffer at the start.
The impact of the variation of the number of measurements on the target detection is as shown in Fig. 14. The probability of detection exponentially rises with increase in the number of the measurements. The number of measurements reaches a regime after which the increase in the detection improvement is very marginal. It is to be noted that already for M equal to 0.15 N the probability of detection reaches 1.
Further, it can be inferred from Fig. 15 that the number of measurements required for good detection after reconstruction is more than the number of measurements required when detection is performed directly on the measurements . This is because for a good reconstruction of a sonar image M is lower bounded by the sparsity.
Fig. 16 shows a sonar data acquisition system 120 according to the present application. The sonar data acquisition system of Fig. 16 comprises a pulse generator 121, a sender/receiver 122 which comprises a sound coupling device 125 such as a loudspeaker or a piezocrystal , an A/D converter 123 and a signal processor 124. The signal processor 124 comprises the image reconstruction and event detection system 1 of Fig. 1. In Fig. 1, the uncompressed signal is symbolized by the letter "y".
Fig. 17 shows a sectional enlargement of Fig. 1 that illustrates a buffer management according to the specification. According to the embodiment of Fig. 17, the buffering unit comprises a first buffer 65, labelled as "Bl", and a second buffer 66, labelled as "B2". The first buffer 65 has sufficient memory for storing R frames and the second buffer 66 has sufficient memory for storing T frames, wherein R is greater than T. Data flows from the compressive sensing unit 20 to the first buffer 65, from the first buffer 65 to the second buffer 66, from the first buffer 65 to the reconstruction unit 70 and from the second buffer 66 to the reconstruction unit 70 are indicated by respective arrow symbols.
Although the above description contains much specificity, these should not be construed as limiting the scope of the embodiments but merely providing illustration of the foreseeable embodiments . Especially the above stated advantages of the embodiments should not be construed as limiting the scope of the embodiments but merely to explain possible achievements if the described embodiments are put into practise. Thus, the scope of the embodiments should be determined by the claims and their equivalents, rather than by the examples given .
The embodiments can also be described with the following lists of elements being organized into items. The respective combinations of features which are disclosed in the item list are regarded as independent subject matter, respectively, that can also be combined with other features of the application . A method for event detection and reconstruction of image raw data in a reflective wave device, the method comprising :
- acquisition of image raw data of a sensor,
- derivation of a compressed representation of the image raw data with compressive sensing,
- frame-by-frame processing of the compressed representation,
- during the frame-by-frame processing of the compressed representation, wherein the frame-by-frame processing comprises a frame-by-frame image reconstruction,
- performing a detection of events based on a current frame of the compressed representation,
wherein, if an event is detected in the current frame:
- retrieving a pre-determined number of frames, the predetermined number of frames preceding the current frame,
- reconstructing an event based on a detection/reconstruction subset, wherein the detection subset is derived from the current frame and the pre-determined number of retrieved frames .
Method according to item 1, comprising:
- storing the detection/reconstruction subset in a buffer "Bl", the size of the buffer being larger than the amount required to store the detection/reconstruction subset by a buffering size, wherein the buffering size depends on the time required to transmit the detection/reconstruction subset to a processing unit.
Method according to item 2, comprising:
if an event is detected: - copying the detection/reconstruction to a second buffer "B2",
- sending the content of the second buffer "B2" to a scene reconstruction unit.
Method according to one of the preceding items comprising :
if an event is detected, generating a data message, the data message comprising event data of the detected event .
Method according to one of the preceding items comprising, if an event is detected, reconstructing an event scene .
Method according to one of the preceding items comprising :
if an event is detected, reconstructing an image in which the event scene is marked.
The method of one of the preceding iterns, comprising - detecting whether an event is present in the current frame using a part P of selected coefficients or meas- urement subset for the current frame
The method according to one of the preceding items, wherein the compressive sensing comprises a random projection of the input set that is chosen such that the complete input set of data need not to be used.
A reflective wave device comprising:
- a sensor for acquisition of image raw data, a computation unit with a processor, a memory and an output unit, the computation unit comprising a software program which provides the following method steps:
- deriving a compressed representation of the image raw data with compressive sensing,
- frame-by-frame processing of the compressed representation,
- during the frame-by-frame processing of the compressed representation, wherein the frame-by-frame processing comprises a frame-by-frame image reconstruction,
- performing a detection of events based on a current frame of the compressed representation,
wherein, if an event is detected in the current frame:
- retrieving a pre-determined number of frames, the pre- determined number of frames preceding the current frame,
- reconstructing an event based on a detection subset, wherein the detection subset is derived from the current frame and the pre-determined number of retrieved frames,
- comparing the reconstructed event with a pre- determined threshold event,
-outputting a flag signal if the reconstructed event matches the threshold event. A reflective wave device comprising:
- a sensor for acquisition of image raw data,
- a compressive sensing unit for deriving a compressed representation from the image raw data,
- a detection unit for detecting an event based on the current frame, for reconstructing an event based on a detection subset, wherein the detection subset is derived from the current frame and the pre-determined number of retrieved frames, for comparing the reconstructed event with a pre-determined event criterion and for out- putting a flag signal if the reconstructed event matches the event criterion.
A reflective wave device according to item 9 or item 10, further comprising
- a buffering unit with a buffer "Bl", a size of the buffer "Bl" being larger than the amount required to store the detection subset by a buffering size, wherein the buffering size depends on a time required to transmit the detection subset to a processing unit.
A reflective wave device according to item 11, the reflective wave device comprising a second buffer "B2", a size of the second buffer being sufficient for storing the detection subset.
A mobile underwater sonar device with a reflective wave device according to one of the items 9 to 12, the reflective wave device being provided at a body of the mobile underwater sonar device, wherein the sensor for acquiring the image raw data is provided by one or more acoustic sensors that are arranged laterally to a direction of travel of the mobile underwater sonar device.
A ship with a reflective wave device according to one of the items 9 to 12, the reflective wave device being provided at a body of the ship, wherein the sensor for acquiring the image raw data is provided by one or more acoustic sensors that are arranged laterally to a direction of travel of the ship.
An unmanned aerial vehicle with a reflective wave device according to one of the items 9 to 12, the reflective wave device being provided at a body of the unmanned aerial vehicle, wherein the sensor for acquiring the im- age raw data is provided by one or more radar sensors that are arranged laterally to a direction of travel of the unmanned aerial vehicle
An aircraft with a reflective wave device according to one of the items 9 to 12, the reflective wave device be ing provided at a body of the aircraft, wherein the sen sor for acquiring the image raw data is provided by one or more radar sensors that are arranged laterally to a direction of travel of the aircraft.

Claims

1. A method for event detection and reconstruction of image raw data in a reflective wave device, the method comprising :
- acquisition of image raw data of a sensor,
- derivation of a compressed representation of the image raw data with compressive sensing,
- frame-by-frame processing of the compressed representation,
- during the frame-by-frame processing of the compressed representation, wherein the frame-by-frame processing comprises a frame-by-frame image reconstruction,
- performing a detection of events based on a current frame of the compressed representation,
wherein, if an event is detected in the current frame:
- retrieving a pre-determined number of frames, the predetermined number of frames preceding the current frame,
- reconstructing an event based on a detection/reconstruction subset, wherein the detection subset is derived from the current frame and the pre-determined number of retrieved frames .
2. Method according to claim 1, comprising:
- storing the detection/reconstruction subset in a buffer "Bl", the size of the buffer being larger than the amount required to store the detection/reconstruction subset by a buffering size, wherein the buffering size depends on the time required to transmit the detection/reconstruction subset to a processing unit.
3. Method according to claim 2, comprising:
if an event is detected: - copying the detection/reconstruction to a second buffer "B2",
- sending the content of the second buffer "B2" to a scene reconstruction unit.
4. Method according to claim 1 comprising:
if an event is detected, generating a data message, the data message comprising event data of the detected event .
Method according to claim 1 comprising, if an event detected, reconstructing an event scene.
6. Method according to claim 1 comprising:
if an event is detected, reconstructing an image in which the event scene is marked.
The method of claim 1, comprising
- detecting whether an event is present in the current frame using a part P of selected coefficients or measurement subset for the current frame .
The method according to claim 1, wherein the compressive sensing comprises a random projection of the input set that is chosen such that the complete input set of data need not to be used.
9. A reflective wave device comprising:
- a sensor for acquisition of image raw data,
a computation unit with a processor, a memory and an output unit, the computation unit comprising a software program which provides the following method steps: - deriving a compressed representation of the image raw data with compressive sensing,
- frame-by-frame processing of the compressed representation,
- during the frame-by-frame processing of the compressed representation, wherein the frame-by-frame processing comprises a frame-by-frame image reconstruction,
- performing a detection of events based on a current frame of the compressed representation,
wherein, if an event is detected in the current frame:
- retrieving a pre-determined number of frames, the predetermined number of frames preceding the current frame,
- reconstructing an event based on a detection subset, wherein the detection subset is derived from the current frame and the pre-determined number of retrieved frames,
- comparing the reconstructed event with a predetermined threshold event,
-outputting a flag signal if the reconstructed event matches the threshold event.
A reflective wave device comprising:
- a sensor for acquisition of image raw data,
- a compressive sensing unit for deriving a compressed representation from the image raw data,
- a detection unit for detecting an event based on the current frame, for reconstructing an event based on a detection subset, wherein the detection subset is derived from the current frame and the pre-determined number of retrieved frames, for comparing the reconstructed event with a pre-determined event criterion and for out- putting a flag signal if the reconstructed event matches the event criterion. A reflective wave device according to claim 9, further comprising
- a buffering unit with a buffer "Bl", a size of the buffer "Bl" being larger than the amount required to store the detection subset by a buffering size, wherein the buffering size depends on a time required to transmit the detection subset to a processing unit.
A reflective wave device according to claim 11, the reflective wave device comprising a second buffer "B2", a size of the second buffer being sufficient for storing the detection subset.
A mobile underwater sonar device with a reflective wave device according to claim 9, the reflective wave device being provided at a body of the mobile underwater sonar device, wherein the sensor for acquiring the image raw data is provided by one or more acoustic sensors that are arranged laterally to a direction of travel of the mobile underwater sonar device.
A ship with a reflective wave device according to claim 9, the reflective wave device being provided at a body of the ship, wherein the sensor for acquiring the image raw data is provided by one or more acoustic sensors that are arranged laterally to a direction of travel of the ship .
An unmanned aerial vehicle with a reflective wave device according to claim 9, the reflective wave device being provided at a body of the unmanned aerial vehicle, wherein the sensor for acquiring the image raw data is provided by one or more radar sensors that are arranged laterally to a direction of travel of the unmanned aerial vehicle.
An aircraft with a reflective wave device according to claim 9, the reflective wave device being provided at a body of the aircraft, wherein the sensor for acquiring the image raw data is provided by one or more radar sen sors that are arranged laterally to a direction of trav el of the aircraft.
PCT/IB2014/066032 2013-12-30 2014-11-14 Reflective wave device for simultaneous event detection and signal reconstruction using compressive measurements WO2015101847A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
SG11201603729UA SG11201603729UA (en) 2013-12-30 2014-11-14 Reflective wave device for simultaneous event detection and signal reconstruction using compressive measurements

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
IB2013061406 2013-12-30
IBPCT/IB2013/061406 2013-12-30

Publications (1)

Publication Number Publication Date
WO2015101847A1 true WO2015101847A1 (en) 2015-07-09

Family

ID=53493335

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2014/066032 WO2015101847A1 (en) 2013-12-30 2014-11-14 Reflective wave device for simultaneous event detection and signal reconstruction using compressive measurements

Country Status (2)

Country Link
SG (1) SG11201603729UA (en)
WO (1) WO2015101847A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111881735A (en) * 2020-06-17 2020-11-03 武汉光庭信息技术股份有限公司 Event classification extraction method and device for automatic driving video data

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6045508A (en) * 1997-02-27 2000-04-04 Acuson Corporation Ultrasonic probe, system and method for two-dimensional imaging or three-dimensional reconstruction

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6045508A (en) * 1997-02-27 2000-04-04 Acuson Corporation Ultrasonic probe, system and method for two-dimensional imaging or three-dimensional reconstruction

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
NUNO GRACIAS ET AL.: "Underwater Mosaicing and Trajectory Reconstruction using Global Alignment", OCEANS, 2001. MTS/ IEEE CONFERENCE AND EXHIBITION, vol. 4, 5 November 2001 (2001-11-05), pages 2557 - 2563, Retrieved from the Internet <URL:http://ieeexplore.ieee.org/xpls/a_s_all.jsp?arnumber=968403> *
SHAHRIAR NEGAHDARIPOUR ET AL.: "Opti-Acoustic Stereo Imaging: On System Calibration and 3-D Target Reconstruction", IEEE TRANSACTIONS ON IMAGE PROCESSING, vol. 18, no. 6, June 2009 (2009-06-01), Retrieved from the Internet <URL:http://ieeexplore.ieee.org/xpl/articleDetai.s.jsp?tp=&arnumber=4815402> *
Y.-J. ZHAO ET AL.: "Ultrasonic signal compressive detection using improved random equivalent sampling", SCIENCE, MEASUREMENT & TECHNO, vol. 6, no. ISSUE., July 2012 (2012-07-01), pages 261 - 266, ISSN: 1751-8822, Retrieved from the Internet <URL:http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7270250> *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111881735A (en) * 2020-06-17 2020-11-03 武汉光庭信息技术股份有限公司 Event classification extraction method and device for automatic driving video data
CN111881735B (en) * 2020-06-17 2022-07-29 武汉光庭信息技术股份有限公司 Event classification extraction method and device for automatic driving video data

Also Published As

Publication number Publication date
SG11201603729UA (en) 2016-07-28

Similar Documents

Publication Publication Date Title
US10094941B2 (en) System and method for acquisition and processing of seismic data using compressive sensing
Hahn et al. Compressive sensing and adaptive direct sampling in hyperspectral imaging
Bi et al. $ L_ {1} $-regularization-based SAR imaging and CFAR detection via complex approximated message passing
EP0562004B1 (en) Multi-dimensional data processing and display
Gao et al. Five-dimensional seismic reconstruction using parallel square matrix factorization
JP4946547B2 (en) Polarimetry SAR image processing method, polarimetry SAR apparatus, and image processing apparatus
US7990311B2 (en) Adaptive clutter filter for maritime surface search radar
Cobra et al. Geometric distortions in side-scan sonar images: a procedure for their estimation and correction
Sung et al. Image-based super resolution of underwater sonar images using generative adversarial network
Hunter et al. Sonar target enhancement by shrinkage of incoherent wavelet coefficients
CN114708257A (en) SAR moving ship target detection method and device
El-Ashkar et al. Compressed sensing for SAR image reconstruction
Lanz et al. The InflateSAR campaign: evaluating SAR identification capabilities of distressed refugee boats
WO2015101847A1 (en) Reflective wave device for simultaneous event detection and signal reconstruction using compressive measurements
KR102173012B1 (en) Apparatus, method, computer-readable storage medium and computer program for generating image of synthetic aperture radar based on compressed sensing
Williams et al. Multi-look processing of high-resolution SAS data for improved target detection performance
CN106911893A (en) A kind of single pixel calculates imaging method
Leier et al. Aperture undersampling using compressive sensing for synthetic aperture stripmap imaging
Mason et al. Passive synthetic aperture radar imaging based on low-rank matrix recovery
Larcom et al. Foveated image formation through compressive sensing
JP2007139449A (en) Magnetism detection device of fixed type
CN112147592A (en) Multi-view target identification method and device, control equipment and storage medium
Murtada et al. Efficient radar imaging using partially synchronized distributed sensors
Li et al. Two-dimensional random sparse sampling for high resolution SAR imaging based on compressed sensing
Liu et al. SAR imaging of dominant scatterers using cascading StOMP

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14876405

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14876405

Country of ref document: EP

Kind code of ref document: A1