CN115047448A - Indoor target rapid detection method and system based on acoustic-electromagnetic intermodulation - Google Patents
Indoor target rapid detection method and system based on acoustic-electromagnetic intermodulation Download PDFInfo
- Publication number
- CN115047448A CN115047448A CN202210385718.1A CN202210385718A CN115047448A CN 115047448 A CN115047448 A CN 115047448A CN 202210385718 A CN202210385718 A CN 202210385718A CN 115047448 A CN115047448 A CN 115047448A
- Authority
- CN
- China
- Prior art keywords
- acoustic
- electromagnetic
- fusion
- amplitude
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 81
- 230000004927 fusion Effects 0.000 claims abstract description 91
- 238000000034 method Methods 0.000 claims abstract description 43
- 230000005284 excitation Effects 0.000 claims abstract description 35
- 238000010801 machine learning Methods 0.000 claims abstract description 26
- 230000000694 effects Effects 0.000 claims description 34
- 238000002592 echocardiography Methods 0.000 claims description 26
- 238000000605 extraction Methods 0.000 claims description 9
- 230000035515 penetration Effects 0.000 claims description 8
- 238000001228 spectrum Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 4
- 238000007781 pre-processing Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 14
- 239000000523 sample Substances 0.000 description 11
- 230000003993 interaction Effects 0.000 description 7
- 238000004088 simulation Methods 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000000149 penetrating effect Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000002238 attenuated effect Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000013145 classification model Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000007500 overflow downdraw method Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000001902 propagating effect Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 230000003595 spectral effect Effects 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 238000012952 Resampling Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000005672 electromagnetic field Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000005305 interferometry Methods 0.000 description 1
- 238000007477 logistic regression Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012847 principal component analysis method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 229920006395 saturated elastomer Polymers 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
- G01S13/887—Radar or analogous systems specially adapted for specific applications for detection of concealed objects, e.g. contraband or weapons
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
- G01S13/04—Systems determining presence of a target
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/87—Combinations of radar systems, e.g. primary radar and secondary radar
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S15/00—Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
- G01S15/02—Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems using reflection of acoustic waves
- G01S15/04—Systems determining presence of a target
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- General Physics & Mathematics (AREA)
- Electromagnetism (AREA)
- Acoustics & Sound (AREA)
- Measurement Of Velocity Or Position Using Acoustic Or Ultrasonic Waves (AREA)
Abstract
The invention provides a method and a system for quickly detecting an indoor target based on acoustic-electromagnetic intermodulation, which comprises the steps of generating an acoustic excitation signal by using an acoustic wave signal, and simultaneously performing acoustic-electromagnetic intermodulation by using an electromagnetic wave signal to generate an acoustic-electromagnetic echo; receiving the acoustic electromagnetic echo of each target in the plurality of indoor targets, and extracting Doppler characteristics and amplitude characteristics of each acoustic electromagnetic echo in different frequency bands; performing feature fusion on the features positioned in different frequency bands by using a splicing method to obtain a multi-band Doppler fusion feature and a multi-band amplitude fusion feature of each target; and constructing a plurality of machine learning models based on different classifiers according to the multi-band Doppler fusion characteristics and the multi-band amplitude fusion characteristics, obtaining corresponding classification results, performing decision fusion, and judging the type of the target corresponding to each acoustic electromagnetic echo. The invention improves the efficiency and the accuracy of target detection and solves the problem of single detection.
Description
Technical Field
The invention relates to the technical field of target detection, in particular to an indoor target rapid detection method and system based on acoustic-electromagnetic intermodulation.
Background
In actions such as disaster rescue, rescue workers often need to go deep into unfamiliar buildings, and the lack of building internal information can bring great threats to smooth development of actions and safety of the workers. Therefore, the penetration detection in the building is researched, and the method has important practical significance and research value.
In recent years, penetration detection schemes have improved over the past due to advances in implementation algorithms and receiver performance. Under the condition of not damaging the site, the detection technologies such as sound waves, infrared waves, electromagnetic waves and the like can realize penetration detection to different degrees. Among these, electromagnetic signals have long been used to detect, characterize, and identify targets in a variety of detection applications, including interferometry, navigation, and penetration detection; however, electromagnetic signals are greatly limited in penetration detection due to the typically low contrast between the target object and the background environment.
Based on the above problems, in order to avoid the exposure of the detection system, researchers need to further complicate the structural design, which leads to the problems of high complexity, difficulty in carrying, and the like of the conventional detection system. Therefore, the development and research of a portable novel detection mode capable of providing a characteristic unknown object at a long distance are valued by scholars at home and abroad.
Acoustic signals for detection applications have attracted renewed interest due to the limitations of detection methods in many environments. In a process similar to electromagnetic wave detection, information about scattering objects is contained in a scattered sound field generated by the interaction of incident sound waves with the objects. But for acoustic systems the high reflection coefficient of most materials prevents the coupling of acoustic energy into the object, while for electromagnetic systems electromagnetic losses and metallic objects prevent the coupling of electromagnetic energy into the object interior. Furthermore, in acoustically saturated environments where there is a large amount of background clutter, or when processing acoustic crossbars that do not produce direct reflected signals, acoustic detection of the desired reflected signal may not be possible. Therefore, it is difficult for both independent detection systems to obtain characteristic information about the object.
Disclosure of Invention
The invention provides an indoor target rapid detection method and system based on acoustic-electromagnetic intermodulation, which aim to solve the defects of the prior art.
In one aspect, the present invention provides a method for rapidly detecting an indoor target based on acoustic-electromagnetic intermodulation, which comprises the following steps:
s1: performing acoustic excitation on a plurality of indoor targets by using acoustic wave signals to induce mechanical vibration so as to generate acoustic excitation signals, and performing acoustic-electromagnetic mutual modulation on the electromagnetic wave signals and the acoustic excitation signals to generate acoustic-electromagnetic echoes;
s2: receiving acoustic electromagnetic echoes of each target in the plurality of indoor targets, and respectively extracting the characteristics of the acoustic electromagnetic echoes on the basis of two dimensions, namely Doppler effect and amplitude modulation, on each frequency band, so as to extract Doppler characteristics and amplitude characteristics of each acoustic electromagnetic echo in different frequency bands;
s3: performing feature fusion on the Doppler features in different frequency bands by using a splicing method to obtain a multi-band Doppler fusion feature of each target, and performing feature fusion on the amplitude features in different frequency bands by using a splicing method to obtain a multi-band amplitude fusion feature of each target;
s4: and constructing a plurality of machine learning models based on different classifiers according to the multi-band Doppler fusion characteristic and the multi-band amplitude fusion characteristic, classifying each acoustic electromagnetic echo by using the different machine learning models respectively to obtain classification results corresponding to the different machine learning models, and performing decision fusion on the classification results to judge the type of the target corresponding to each acoustic electromagnetic echo.
The method combines an acoustic detection system and an electromagnetic detection system, and represents the difference between an object and the surrounding environment thereof through Doppler frequency shift and amplitude modulation caused by the interaction of the transmitted electromagnetic wave and a wave sound source in the object, namely an acoustic electromagnetic detection method. For indoor targets, their contrast to the environment is enhanced by mechanical vibrations. The combined reflected and scattered electromagnetic signals are modulated with structure and composition specific characteristics of the vibrating object to provide information about the structure of the target and to distinguish it from clutter. And performing feature modeling by utilizing deep learning aiming at the extracted target information, and classifying various main target features. Compared with a single-mode detection system which possibly prevents sound waves from penetrating through the interior of an object or cannot receive reflected waves, the orthogonal detection system can better obtain object information, utilizes different characteristics of echo signals as much as possible, improves the accuracy of target detection, solves the problem of detection by a single means, and improves the target detection efficiency.
In a specific embodiment, the sound source level of the sound wave signal satisfies unidirectional penetration.
In a specific embodiment, the S1 specifically includes: the method comprises the steps of utilizing sound wave signals to conduct acoustic excitation on a plurality of indoor targets to cause mechanical vibration, enabling the indoor targets to generate acoustic excitation signals, enabling the acoustic excitation signals to pass through a power amplifier, adding the acoustic excitation signals into electromagnetic wave signals, modulating the electromagnetic wave signals, and obtaining acoustic electromagnetic echoes.
In a specific embodiment, the S2 specifically includes:
receiving the acoustic electromagnetic echo of each target in the plurality of indoor targets, preprocessing the acoustic electromagnetic echo, and then performing Fourier transform to obtain the frequency spectrum of each frequency band of each acoustic electromagnetic echo;
and respectively extracting the characteristics of the acoustic electromagnetic echoes on the basis of two dimensions of Doppler effect and amplitude modulation on each frequency band, thereby extracting the Doppler characteristics and the amplitude characteristics of each acoustic electromagnetic echo in different frequency bands.
In a specific embodiment, the extracting the features of the acoustic electromagnetic echo based on the doppler effect includes:
and analyzing the phase modulation of the electromagnetic wave signal caused by the Doppler effect in the acoustic electromagnetic echo to acquire the phase modulation amplitude generated by the phase modulation. Doppler effect, determining that the interaction of a propagating electromagnetic wave with a fluctuating sound source will result in a doppler shift of the electromagnetic wave, a vibratory motion that can alter the distance of an object relative to the electromagnetic wave source, the electromagnetic wave source modulating the phase of the reflected radio frequency signal, thereby modulating object features unique to the target in the phase, producing target information.
In a specific embodiment, the feature extraction of the acoustic electromagnetic echo based on amplitude modulation includes:
and analyzing the amplitude modulation of the electromagnetic wave signal caused by narrow relativity theory, path loss and radar scattering cross section in the acoustic electromagnetic echo respectively to obtain amplitude modulation caused by narrow relativity theory, amplitude modulation caused by path loss and amplitude modulation caused by radar scattering cross section respectively. Amplitude modulation, when analyzing electromagnetic scattering of a vibrating object, the overall contribution due to the modulation sideband power is much smaller than the phase modulation produced by the doppler effect and is typically below the lower noise limit measurable by the detection system. However, the dynamic range of the radio frequency detection system can be improved through the analog cancellation technology, and the detection capability of low-frequency and low-level sideband modulation in the reflected signals is remarkably improved, so that the narrow relativity theory, the road stiffness loss and the effect of RCS on amplitude modulation are reintroduced into the analysis of the scattered radio frequency signals of the vibrating object.
In a specific embodiment, the S4 specifically includes:
s401: respectively constructing a micro Doppler feature library and a multi-order amplitude feature library according to the multi-band Doppler fusion feature and the multi-band amplitude fusion feature, and performing preliminary matching judgment on the received acoustic electromagnetic echo of the target to be detected according to the micro Doppler feature library and the multi-order amplitude feature library to obtain a preliminary judgment result;
s402: respectively constructing a micro Doppler feature library and a multi-order amplitude feature library according to the multi-band Doppler fusion feature and the multi-band amplitude fusion feature to construct a plurality of sample data sets, and performing ensemble learning based on different classifiers by using each sample data set to construct different machine learning models;
s403: classifying and identifying the Doppler characteristic and the amplitude characteristic of the acoustic-electromagnetic echo of the target to be detected by using the different machine learning models respectively to obtain corresponding classification results, and voting the classification results to obtain decision results;
and performing decision fusion on the preliminary judgment result obtained after the S401 to the S403 are performed on the acoustic electromagnetic echoes received by different receiving points at the same time and the decision result, so as to judge the type of the target corresponding to each acoustic electromagnetic echo. By utilizing a splicing method, multi-band characteristic fusion is realized, a target signal which is easier to extract is obtained, and the detection accuracy is improved; meanwhile, for complex multi-signal sources captured from different receiving antennas, multi-target separation and separation are carried out through energy distribution in a threshold setting range, and the calculation complexity of a multi-target detection algorithm is reduced.
According to a second aspect of the present invention, a computer-readable storage medium is presented, having stored thereon a computer program which, when executed by a computer processor, implements the above-described method.
According to a third aspect of the present invention, an indoor target fast detection system based on acoustic electromagnetic intermodulation is provided, the system comprising:
an acoustic electromagnetic echo generation module: the indoor target acoustic excitation device is configured to perform acoustic excitation on a plurality of indoor targets by using acoustic wave signals to induce mechanical vibration so as to generate acoustic excitation signals, and perform acoustic-electromagnetic mutual modulation on the acoustic excitation signals by using electromagnetic wave signals to generate acoustic-electromagnetic echoes;
the acoustic electromagnetic echo characteristic extraction module: the method comprises the steps that the acoustic electromagnetic echoes of each target in a plurality of indoor targets are received, and feature extraction is carried out on the acoustic electromagnetic echoes on the basis of two dimensions of Doppler effect and amplitude modulation on each frequency band, so that Doppler features and amplitude features of each acoustic electromagnetic echo in different frequency bands are extracted;
a feature fusion module: the characteristic fusion method is configured to perform characteristic fusion on the Doppler characteristics located in different frequency bands by using a splicing method to obtain a multi-band Doppler fusion characteristic of each target, and perform characteristic fusion on the amplitude characteristics located in different frequency bands by using a splicing method to obtain a multi-band amplitude fusion characteristic of each target;
a classification decision module: the multi-band Doppler fusion system is configured and used for constructing a plurality of machine learning models based on different classifiers according to the multi-band Doppler fusion characteristics and the multi-band amplitude fusion characteristics, classifying each acoustic electromagnetic echo by using the different machine learning models respectively to obtain classification results corresponding to the different machine learning models, and then performing decision fusion on the classification results to judge the type of a target corresponding to each acoustic electromagnetic echo.
The invention combines an acoustic detection system and an electromagnetic detection system, and characterizes the difference between an object and the surrounding environment through Doppler frequency shift and amplitude modulation caused by the interaction of transmitted electromagnetic waves and a wave sound source in the object, namely an acoustic electromagnetic detection method. For indoor targets, their contrast to the environment is enhanced by mechanical vibrations. The combined reflected and scattered electromagnetic signals are modulated with structure and composition specific characteristics of the vibrating object to provide information about the structure of the target and to distinguish it from clutter. And performing feature modeling by utilizing deep learning aiming at the extracted target information, and classifying various main target features. Compared with a single-mode detection system which possibly prevents sound waves from penetrating through the interior of an object or cannot receive reflected waves, the orthogonal detection system can better obtain object information, utilizes different characteristics of echo signals as much as possible, improves the accuracy of target detection, solves the problem of detection by a single means, and improves the target detection efficiency.
Drawings
The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments and together with the description serve to explain the principles of the invention. Other embodiments and many of the intended advantages of embodiments will be readily appreciated as they become better understood by reference to the following detailed description. Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
fig. 1 is a flow chart of a method for fast detecting an indoor target based on acoustic-electromagnetic intermodulation according to an embodiment of the present invention;
fig. 2 is an overall framework diagram of an indoor target fast detection method based on acoustic-electromagnetic intermodulation according to a specific embodiment of the present invention;
FIG. 3 is a schematic diagram of attenuated channel modeling of acoustic signals according to a specific embodiment of the present invention;
FIG. 4 is a simulation diagram of the relationship between the target deformation amount and the power spectral density of the probe according to a specific embodiment of the present invention;
FIG. 5 is a schematic diagram of a simulation of the effects of different acoustic excitation effects for a specific embodiment of the present invention;
FIG. 6 is a diagram illustrating ensemble learning-based multi-scenario feature fusion according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of an indoor target identification scheme of a specific embodiment of the present invention;
fig. 8 is a block diagram of an indoor target fast detection system based on acoustic-electromagnetic intermodulation according to an embodiment of the present invention.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
Fig. 1 shows a flow chart of a method for rapidly detecting an indoor target based on acoustic-electromagnetic intermodulation according to an embodiment of the present invention. As shown in fig. 1, the method comprises the steps of:
s1: performing acoustic excitation on a plurality of indoor targets by using acoustic wave signals to induce mechanical vibration so as to generate acoustic excitation signals, and performing acoustic-electromagnetic mutual modulation on the electromagnetic wave signals and the acoustic excitation signals to generate acoustic-electromagnetic echoes;
s2: receiving acoustic electromagnetic echoes of each target in the plurality of indoor targets, and respectively extracting the characteristics of the acoustic electromagnetic echoes on the basis of two dimensions, namely Doppler effect and amplitude modulation, on each frequency band, so as to extract Doppler characteristics and amplitude characteristics of each acoustic electromagnetic echo in different frequency bands;
s3: performing feature fusion on the Doppler features in different frequency bands by using a splicing method to obtain a multi-band Doppler fusion feature of each target, and performing feature fusion on the amplitude features in different frequency bands by using a splicing method to obtain a multi-band amplitude fusion feature of each target;
s4: and constructing a plurality of machine learning models based on different classifiers according to the multi-band Doppler fusion characteristic and the multi-band amplitude fusion characteristic, classifying each acoustic electromagnetic echo by using the different machine learning models respectively to obtain classification results corresponding to the different machine learning models, and performing decision fusion on the classification results to judge the type of the target corresponding to each acoustic electromagnetic echo.
In a specific embodiment, the sound source level of the acoustic signal satisfies unidirectional penetration.
In a specific embodiment, the S1 specifically includes: the method comprises the steps of utilizing sound wave signals to conduct acoustic excitation on a plurality of indoor targets to cause mechanical vibration, enabling the indoor targets to generate acoustic excitation signals, enabling the acoustic excitation signals to pass through a power amplifier, adding the acoustic excitation signals into electromagnetic wave signals, modulating the electromagnetic wave signals, and obtaining acoustic electromagnetic echoes.
In a specific embodiment, the S2 specifically includes:
receiving the acoustic electromagnetic echo of each target in the plurality of indoor targets, preprocessing the acoustic electromagnetic echo, and then performing Fourier transform to obtain the frequency spectrum of each frequency band of each acoustic electromagnetic echo;
and respectively extracting the characteristics of the acoustic electromagnetic echoes on the basis of two dimensions of Doppler effect and amplitude modulation on each frequency band, thereby extracting the Doppler characteristics and the amplitude characteristics of each acoustic electromagnetic echo in different frequency bands.
In a specific embodiment, the extracting the features of the acoustic electromagnetic echo based on the doppler effect includes:
and analyzing the phase modulation of the electromagnetic wave signal caused by the Doppler effect in the acoustic electromagnetic echo to acquire the phase modulation amplitude generated by the phase modulation.
In a specific embodiment, the feature extraction of the acoustic electromagnetic echo based on amplitude modulation includes:
and analyzing the amplitude modulation of the electromagnetic wave signal caused by narrow relativity theory, path loss and radar scattering cross section in the acoustic electromagnetic echo respectively to obtain amplitude modulation caused by narrow relativity theory, amplitude modulation caused by path loss and amplitude modulation caused by radar scattering cross section respectively.
In a specific embodiment, the S4 specifically includes:
s401: respectively constructing a micro Doppler feature library and a multi-order amplitude feature library according to the multi-band Doppler fusion feature and the multi-band amplitude fusion feature, and performing preliminary matching judgment on the received acoustic electromagnetic echo of the target to be detected according to the micro Doppler feature library and the multi-order amplitude feature library to obtain a preliminary judgment result;
s402: respectively constructing a micro Doppler feature library and a multi-order amplitude feature library according to the multi-band Doppler fusion feature and the multi-band amplitude fusion feature to construct a plurality of sample data sets, and performing ensemble learning based on different classifiers by using each sample data set to construct different machine learning models;
s403: classifying and identifying the Doppler characteristic and the amplitude characteristic of the acoustic-electromagnetic echo of the target to be detected by using the different machine learning models respectively to obtain corresponding classification results, and voting the classification results to obtain decision results;
and performing decision fusion on the preliminary judgment result obtained after the steps from S401 to S403 are performed on the acoustic electromagnetic echoes received by different receiving points at the same moment and the decision result, so as to judge the type of the target corresponding to each acoustic electromagnetic echo.
The flow of this solution is fully described below using a specific embodiment:
1. indoor target rapid detection method based on acoustic-electromagnetic intermodulation
Fig. 2 is an overall framework diagram of an indoor target fast detection method based on acoustic-electromagnetic intermodulation according to a specific embodiment of the present invention.
FIG. 3 is a schematic diagram of attenuated channel modeling of acoustic signals according to a specific embodiment of the present invention; due to the need of exciting an indoor target, sound waves pass through a free space path from a sound source and penetrate through a target environment, and a certain sound source level is needed for exciting a detection target to generate signal characteristics. Therefore, acoustic energy attenuation models are established for acoustic wave attenuation of different propagation environments.
Fig. 4 is a simulation diagram of the relationship between the target deformation amount and the power spectral density of the probe according to a specific embodiment of the present invention.
According to the multi-modal indoor target detection method mainly integrating acoustic detection and electromagnetic detection, sound source level sound wave signals meeting unidirectional penetrability are designed, a scattering electromagnetic field is generated by exciting indoor target objects to interact with electromagnetic waves, an acoustic-electromagnetic integrated detection system is built, echo signals of indoor targets are collected, after preprocessing, Fourier transformation is carried out, single signal source frequency domain representation is obtained, Doppler effect and amplitude modulation are used as optimal features, feature extraction is carried out, and multi-scene data feature fusion and multi-recognition model decision fusion are carried out.
In order to solve the problem that in practice, due to the fact that an acoustic channel has serious attenuation influence, attenuation model simulation is respectively carried out on acoustic signals under different propagation media, the acoustic signals capable of meeting one-way penetrability can be designed through effective experimental conclusion, the phenomenon that target deformation is insufficient due to power reduction after the acoustic waves are attenuated through the channel is relieved, and the accuracy of target identification in the later period is improved. Electromagnetic wave modulation is induced by acoustic induced medium fluctuations, and phase and amplitude modulation occurring in comparative synthetic effects is studied on the basis of signals propagating in an acoustically excited environment and reflected from the surface of an object.
Aiming at the indoor target detection and identification tasks, the information fusion technology is further expanded into a data-feature-identification target classification processing process, and the indoor target detection method is improved. A multi-scene indoor target fusion classification model structure based on ensemble learning is divided into three fusion levels according to the degree of information fusion: data level fusion, feature level fusion and decision level fusion.
Under normal conditions, under the same-magnitude and homogeneous physical field of ships and warships, original data are collected from distributed acoustic and magnetic sensors and subjected to transverse multi-domain feature fusion, and then a principal component analysis method and a wavelet sub-band fusion method are adopted to reduce data volume.
On the premise of data level fusion, characteristic quantities such as a power spectrum of ship noise, a fundamental frequency of a ship, a field intensity change rate, a sound pressure level, a characteristic spectrum and the like are extracted from ship sound field signals, and then the characteristic vectors are comprehensively analyzed and processed to obtain fused characteristic vectors. And a D-S reasoning algorithm with high fusion efficiency is utilized to identify the information source of the characteristic level, so that the loss amount of main characteristic data is reduced, and the detection precision is improved.
Under the premise of feature level fusion, the information sources derived from the main feature values of the single-period and single-sensor are combined with the confidence coefficient of the time-frequency features, the Basic Probability Assignment (BPA) of the multiple information sources is calculated and used as the input of decision level fusion, the signal sources are judged, and the target signal detection result is output.
2. Comparison of different acoustoelectric modulation effects
As a means of improving the distinction between an object and its surroundings, sound waves are used to induce mechanical vibrations on the object, thereby modulating the scattered electromagnetic signal and enhancing the contrast of the excited object. The resulting reflected and scattered radio frequency signals are modulated with structure and composition-specific characteristics of the vibrating object, thereby providing information about the structure of the target and distinguishing it from clutter.
The analytic solution to the modulation effects introduced when high power sound waves propagate through a medium to structurally induce vibrations and compares the analytic results with the main modulation effects that appear as a direct result of object vibrations, including doppler phase modulation and amplitude modulation, path loss and Radar Cross Section (RCS) caused by narrow relativity. By combining all effects, the modulated received signal is as follows:
s(t)=α 0 A Acoust (t)A PL (t)A RCS (t)γ(t)cos[ω RF t-φ Doppler (t)] (1)
wherein alpha is 0 Is the transmission power; a. the PL (t) amplitude modulation due to path loss variation; a. the RCS (t) amplitude modulation due to RCS; γ (t) is amplitude modulation caused by a narrow sense of relativity; phi is a Doppler (t) is the phase modulation that produces the doppler effect of the reflected signal, and its expressions are:
the amplitude modulation due to the medium vibration is therefore complementary to the other modulation effects and when the vibration is caused by sound, i.e. ω V =ω A The modulated tones appear at the same frequency.
In order to further compare the results of the modulation processes, different modulation amplitudes due to acoustically induced medium fluctuations are given below,the amplitude is modulated for the phase of the doppler effect,for amplitude modulation caused by the narrow relativity theory,the amplitude is modulated for amplitude due to path loss variations,the amplitude is modulated for the amplitude caused by the RCS,the amplitude is modulated for the acoustic amplitude.
Referring to fig. 5, fig. 5 is a simulation diagram illustrating different acoustic excitation effect effects according to an embodiment of the present invention; it can be seen that in these intermodulation effects, the phase modulation of tone amplitude and doppler effect has the greatest influence on the acousto-electromagnetic interaction, which supports the basis of the optimization of subsequent characteristics from theoretical simulation.
3. Multi-scene feature fusion based on ensemble learning
Referring to fig. 6 and 7, fig. 6 is a schematic diagram of a multi-scenario feature fusion based on ensemble learning according to an embodiment of the present invention; fig. 7 is a schematic diagram of an indoor target identification scheme of a specific embodiment of the present invention.
As shown in fig. 6 and 7, for echo signals generated by the acousto-electromagnetic intermodulation effect, a micro doppler feature library and a multi-order amplitude feature library need to be further established through phase modulation analysis and amplitude modulation analysis, multi-feature fusion needs to be performed at a data level for different typical indoor scenes, matching initial judgment is performed according to frequency spectrum features and the micro doppler feature library, and finally, a material judgment result is obtained through fusion with matching results of the multi-order amplitude sample library. And constructing a machine learning model based on ensemble learning at a classification decision level for decision fusion.
The method comprises the following steps of fusing scene echo data of different measuring points measured at the same time, obtaining N data sets by adopting playback sampling from the scene echo data sets by using a bootstrap method, learning a model on each data set, and obtaining the final prediction result by using the output of N indoor sample classification models (as shown in figure 6): and predicting voting by adopting N indoor sample classification models. The specific process is as follows:
(1) resampling (repeated) n samples from the acoustic electromagnetic echo data samples;
(2) establishing classifiers (ID3, C4.5, CART, SVM, Logistic regression, etc.) on the n samples according to the typical sample attributes in all indoor target sample libraries;
(3) repeating the two steps for m times;
(4) and (4) putting the data on the m classifiers, and finally determining which class the sound electromagnetic echo data belong to according to the voting results of the m classifiers.
The invention has the beneficial effects that: the method combines the multi-mode system fusion of acoustic detection and electromagnetic detection, fully excites the hidden characteristic information in the target object, can generate the typical contrast between the target object and the background environment as much as possible, overcomes the defect that a single target signal is submerged by background clutter by utilizing the slightly-changed characteristic quantity of the target object, and improves the accuracy of target detection. Meanwhile, the invention carries out filtering decomposition on a large number of signals, reduces unnecessary calculation amount, improves detection efficiency and can reduce power consumption of target through-wall detection equipment.
Fig. 8 is a block diagram of an indoor target fast detection system based on acoustic-electromagnetic intermodulation according to an embodiment of the present invention. The system comprises an acoustic electromagnetic echo generating module 801, an acoustic electromagnetic echo feature extracting module 802, a feature fusion module 803 and a classification decision module 804.
In a specific embodiment, the acoustic electromagnetic echo generating module 801 is configured to perform acoustic excitation on a plurality of indoor targets by using an acoustic wave signal to induce mechanical vibration, so that the indoor targets generate acoustic excitation signals, and perform acoustic electromagnetic mutual modulation on the acoustic excitation signals by using an electromagnetic wave signal to generate acoustic electromagnetic echoes;
the acoustic electromagnetic echo feature extraction module 802 is configured to receive an acoustic electromagnetic echo of each of the plurality of indoor targets, and perform feature extraction on the acoustic electromagnetic echo on the basis of two dimensions, namely, doppler effect and amplitude modulation, in each frequency band, so as to extract doppler features and amplitude features of each acoustic electromagnetic echo in different frequency bands;
the feature fusion module 803 is configured to perform feature fusion on the doppler features located in different frequency bands by using a splicing method to obtain a multi-band doppler fusion feature of each target, and perform feature fusion on the amplitude features located in different frequency bands by using a splicing method to obtain a multi-band amplitude fusion feature of each target;
the classification decision module 804 is configured to construct a plurality of machine learning models based on different classifiers according to the multiband doppler fusion characteristic and the multiband amplitude fusion characteristic, classify each acoustic-electromagnetic echo by using the different machine learning models to obtain classification results corresponding to the different machine learning models, perform decision fusion on the classification results, and determine the type of a target corresponding to each acoustic-electromagnetic echo.
The system combines an acoustic detection system and an electromagnetic detection system, and represents the difference between an object and the surrounding environment thereof through Doppler frequency shift and amplitude modulation caused by the interaction of transmitted electromagnetic waves and a wave sound source in the object, namely an acoustic electromagnetic detection method. For indoor targets, their contrast to the environment is enhanced by mechanical vibrations. The combined reflected and scattered electromagnetic signals are modulated with structure and composition specific characteristics of the vibrating object to provide information about the structure of the target and to distinguish it from clutter. And performing feature modeling by utilizing deep learning aiming at the extracted target information, and classifying various main target features. Compared with a single-mode detection system which possibly prevents sound waves from penetrating through the interior of an object or cannot receive reflected waves, the orthogonal detection system can better obtain object information, utilizes different characteristics of echo signals as much as possible, improves the accuracy of target detection, solves the problem of detection by a single means, and improves the target detection efficiency.
Embodiments of the present invention also relate to a computer-readable storage medium having stored thereon a computer program which, when executed by a computer processor, implements the method above. The computer program comprises program code for performing the method illustrated in the flow chart. It should be noted that the computer readable medium of the present application can be a computer readable signal medium or a computer readable medium or any combination of the two.
The invention combines an acoustic detection system and an electromagnetic detection system, and characterizes the difference between an object and the surrounding environment thereof by Doppler frequency shift and amplitude modulation caused by the interaction of transmitted electromagnetic waves and a wave sound source in the object, namely an acoustic electromagnetic detection method. For indoor targets, their contrast to the environment is enhanced by mechanical vibrations. The combined reflected and scattered electromagnetic signals are modulated with structure and composition specific characteristics of the vibrating object to provide information about the structure of the target and to distinguish it from clutter. And performing feature modeling by utilizing deep learning aiming at the extracted target information, and classifying various main target features. Compared with a single-mode detection system which possibly prevents sound waves from penetrating into an object or cannot receive reflected waves, the orthogonal detection system can better obtain object information, utilizes different characteristics of echo signals as much as possible, improves the accuracy of target detection, solves the problem of detection by a single means, and improves the target detection efficiency.
The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements in which any combination of the features described above or their equivalents does not depart from the spirit of the invention disclosed above. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.
Claims (9)
1. An indoor target rapid detection method based on acoustic-electromagnetic intermodulation is characterized by comprising the following steps:
s1: performing acoustic excitation on a plurality of indoor targets by using acoustic wave signals to induce mechanical vibration so as to generate acoustic excitation signals, and performing acoustic-electromagnetic mutual modulation on the electromagnetic wave signals and the acoustic excitation signals to generate acoustic-electromagnetic echoes;
s2: receiving acoustic electromagnetic echoes of each target in the plurality of indoor targets, and respectively extracting the characteristics of the acoustic electromagnetic echoes on the basis of two dimensions, namely Doppler effect and amplitude modulation, on each frequency band, so as to extract Doppler characteristics and amplitude characteristics of each acoustic electromagnetic echo in different frequency bands;
s3: performing feature fusion on the Doppler features in different frequency bands by using a splicing method to obtain a multi-band Doppler fusion feature of each target, and performing feature fusion on the amplitude features in different frequency bands by using a splicing method to obtain a multi-band amplitude fusion feature of each target;
s4: and constructing a plurality of machine learning models based on different classifiers according to the multi-band Doppler fusion characteristic and the multi-band amplitude fusion characteristic, classifying each acoustic electromagnetic echo by using the different machine learning models respectively to obtain classification results corresponding to the different machine learning models, and performing decision fusion on the classification results to judge the type of the target corresponding to each acoustic electromagnetic echo.
2. The method of claim 1, wherein the acoustic signal has a source level that satisfies unidirectional penetration.
3. The method according to claim 1, wherein the S1 specifically includes: the method comprises the steps of utilizing sound wave signals to conduct acoustic excitation on a plurality of indoor targets to cause mechanical vibration, enabling the indoor targets to generate acoustic excitation signals, enabling the acoustic excitation signals to pass through a power amplifier, adding the acoustic excitation signals into electromagnetic wave signals, modulating the electromagnetic wave signals, and obtaining acoustic electromagnetic echoes.
4. The method according to claim 1, wherein the S2 specifically includes:
receiving the acoustic electromagnetic echo of each target in the plurality of indoor targets, preprocessing the acoustic electromagnetic echo, and then performing Fourier transform to obtain the frequency spectrum of each frequency band of each acoustic electromagnetic echo;
and respectively extracting the characteristics of the acoustic electromagnetic echoes on the basis of two dimensions of Doppler effect and amplitude modulation on each frequency band, thereby extracting the Doppler characteristics and the amplitude characteristics of each acoustic electromagnetic echo in different frequency bands.
5. The method of claim 1, wherein the characterizing the acoustic electromagnetic echoes based on the doppler effect comprises:
and analyzing the phase modulation of the electromagnetic wave signal caused by the Doppler effect in the acoustic electromagnetic echo to acquire the phase modulation amplitude generated by the phase modulation.
6. The method of claim 1, wherein the feature extracting the acoustic electromagnetic echo based on amplitude modulation comprises:
and analyzing the amplitude modulation of the electromagnetic wave signal caused by narrow-sense relativity, path loss and radar scattering cross section in the acoustic electromagnetic echo respectively to obtain amplitude modulation caused by the narrow-sense relativity, amplitude modulation caused by the path loss and amplitude modulation caused by the radar scattering cross section respectively.
7. The method according to claim 1, wherein the S4 specifically includes:
s401: respectively constructing a micro Doppler feature library and a multi-order amplitude feature library according to the multi-band Doppler fusion feature and the multi-band amplitude fusion feature, and performing preliminary matching judgment on the received acoustic electromagnetic echo of the target to be detected according to the micro Doppler feature library and the multi-order amplitude feature library to obtain a preliminary judgment result;
s402: respectively constructing a micro Doppler feature library and a multi-order amplitude feature library according to the multi-band Doppler fusion features and the multi-band amplitude fusion features to construct a plurality of sample data sets, and performing ensemble learning by using each sample data set based on different classifiers to construct different machine learning models;
s403: classifying and identifying the Doppler characteristic and the amplitude characteristic of the acoustic-electromagnetic echo of the target to be detected by using the different machine learning models respectively to obtain corresponding classification results, and voting the classification results to obtain decision results;
and performing decision fusion on the preliminary judgment result obtained after the S401 to the S403 are performed on the acoustic electromagnetic echoes received by different receiving points at the same time and the decision result, so as to judge the type of the target corresponding to each acoustic electromagnetic echo.
8. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a computer processor, carries out the method of any one of claims 1 to 7.
9. An indoor target rapid detection system based on acoustic-electromagnetic intermodulation, characterized by comprising:
an acoustic electromagnetic echo generation module: the indoor target acoustic excitation device is configured to perform acoustic excitation on a plurality of indoor targets by using acoustic wave signals to induce mechanical vibration so as to generate acoustic excitation signals, and perform acoustic-electromagnetic mutual modulation on the acoustic excitation signals by using electromagnetic wave signals to generate acoustic-electromagnetic echoes;
the acoustic electromagnetic echo characteristic extraction module: the method comprises the steps that the acoustic electromagnetic echoes of each target in a plurality of indoor targets are received, and feature extraction is carried out on the acoustic electromagnetic echoes on the basis of two dimensions of Doppler effect and amplitude modulation on each frequency band, so that Doppler features and amplitude features of each acoustic electromagnetic echo in different frequency bands are extracted;
a feature fusion module: the multi-band Doppler fusion characteristic of each target is obtained by performing characteristic fusion on the Doppler characteristics located in different frequency bands by using a splicing method, and the multi-band amplitude fusion characteristic of each target is obtained by performing characteristic fusion on the amplitude characteristics located in different frequency bands by using the splicing method;
a classification decision module: the multi-band Doppler fusion system is configured and used for constructing a plurality of machine learning models based on different classifiers according to the multi-band Doppler fusion characteristics and the multi-band amplitude fusion characteristics, classifying each acoustic electromagnetic echo by using the different machine learning models respectively to obtain classification results corresponding to the different machine learning models, and then performing decision fusion on the classification results to judge the type of a target corresponding to each acoustic electromagnetic echo.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210385718.1A CN115047448B (en) | 2022-04-13 | 2022-04-13 | Indoor target rapid detection method and system based on acoustic electromagnetic intermodulation |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210385718.1A CN115047448B (en) | 2022-04-13 | 2022-04-13 | Indoor target rapid detection method and system based on acoustic electromagnetic intermodulation |
Publications (2)
Publication Number | Publication Date |
---|---|
CN115047448A true CN115047448A (en) | 2022-09-13 |
CN115047448B CN115047448B (en) | 2024-09-10 |
Family
ID=83157984
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210385718.1A Active CN115047448B (en) | 2022-04-13 | 2022-04-13 | Indoor target rapid detection method and system based on acoustic electromagnetic intermodulation |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN115047448B (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115962845A (en) * | 2023-03-16 | 2023-04-14 | 保定天威新域科技发展有限公司 | Broad-spectrum mechanical wave intelligent sensor |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102713667A (en) * | 2009-11-23 | 2012-10-03 | 罗伯特·博世有限公司 | Method for detecting objects |
CN103858021A (en) * | 2011-09-23 | 2014-06-11 | 埃西斯创新有限公司 | Acousto-electromagnetic investigation of physical properties of an object |
US20200103499A1 (en) * | 2018-10-02 | 2020-04-02 | Fortem Technologies, Inc. | System and method for drone and object classification |
CN112285699A (en) * | 2020-12-25 | 2021-01-29 | 之江实验室 | Underwater medium-crossing communication and water surface target detection and tracking integrated system |
RU2757928C1 (en) * | 2021-03-03 | 2021-10-25 | Федеральное государственное унитарное предприятие "Ростовский-на-Дону" научно-исследовательский институт радиосвязи" (ФГУП "РНИИРС") | Multisensor method for detecting unmanned aerial vehicles |
-
2022
- 2022-04-13 CN CN202210385718.1A patent/CN115047448B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102713667A (en) * | 2009-11-23 | 2012-10-03 | 罗伯特·博世有限公司 | Method for detecting objects |
CN103858021A (en) * | 2011-09-23 | 2014-06-11 | 埃西斯创新有限公司 | Acousto-electromagnetic investigation of physical properties of an object |
US20140224021A1 (en) * | 2011-09-23 | 2014-08-14 | Isis Innovation Limited | Acousto-electromagnetic investigation of physical properties of an object |
US20200103499A1 (en) * | 2018-10-02 | 2020-04-02 | Fortem Technologies, Inc. | System and method for drone and object classification |
CN112285699A (en) * | 2020-12-25 | 2021-01-29 | 之江实验室 | Underwater medium-crossing communication and water surface target detection and tracking integrated system |
RU2757928C1 (en) * | 2021-03-03 | 2021-10-25 | Федеральное государственное унитарное предприятие "Ростовский-на-Дону" научно-исследовательский институт радиосвязи" (ФГУП "РНИИРС") | Multisensor method for detecting unmanned aerial vehicles |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115962845A (en) * | 2023-03-16 | 2023-04-14 | 保定天威新域科技发展有限公司 | Broad-spectrum mechanical wave intelligent sensor |
Also Published As
Publication number | Publication date |
---|---|
CN115047448B (en) | 2024-09-10 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN103718242B (en) | Adopt the system and method for the treatment of voice signal of spectrum motion transform | |
CN113049684A (en) | Ultrasonic lamb wave defect signal identification method based on variational modal decomposition | |
RU2681242C1 (en) | Intellectual system for detection and classification of marine targets | |
US20140311245A1 (en) | Pipe inspection system and related methods | |
CN103929706B (en) | Device and method for measuring the direct sound wave of sound source generation | |
CN103941232A (en) | Sound emission source locating method based on time reversal focusing technique | |
Van Komen et al. | Seabed type and source parameters predictions using ship spectrograms in convolutional neural networks | |
Conan et al. | Using the trapped energy ratio for source depth discrimination with a horizontal line array: Theory and experimental results | |
Alaie et al. | Passive sonar target detection using statistical classifier and adaptive threshold | |
CN115047448B (en) | Indoor target rapid detection method and system based on acoustic electromagnetic intermodulation | |
CN110850420B (en) | Fisher SVM sonar signal discrimination method based on marble loss | |
CN113884567B (en) | Rail weld damage detection method and device based on ultrasonic Lamb wave | |
CN101644768B (en) | Torpedo target recognition method based on cepstrum analysis | |
Ashok et al. | Absorption of echo signal for underwater acoustic signal target system using hybrid of ensemble empirical mode with machine learning techniques | |
Forman et al. | Validating deep learning seabed classification via acoustic similarity | |
Okopal et al. | Dispersion-invariant features for classification | |
Zhang et al. | Correlation network from multivariate time series: A new method for characterizing nonlinear dynamic behavior in marine acoustic signal | |
CN114739234B (en) | Land mine detection method and system based on electromagnetic sodar | |
CN110991507A (en) | Road underground cavity identification method, device and system based on classifier | |
Wu et al. | Acoustic tunnel lining cavity detection using cepstral coefficients with optimized filter bank | |
CN114778706A (en) | Indoor object echo characteristic processing method and system based on acoustic-electromagnetic intermodulation | |
KR101317887B1 (en) | Radar modulation identification method using gini's coefficient | |
Golato et al. | Multifrequency and multimodal sparse reconstruction in Lamb wave based structural health monitoring | |
Yang et al. | Novel method for detecting weak acoustic emission signals based on the similarity of time-frequency spectra | |
Reeder et al. | Walking on snow-covered Arctic sea ice to infer ice thickness |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |