JP2020198094A - Article detection method and device - Google Patents

Abstract

To provide an article detection method and a device.SOLUTION: The method includes the steps of: acquiring a piece of training data; performing training on a convolutional neural network using the training data; acquiring a piece of test data for items awaiting detection, the test data including a micro doppler image which is acquired by at least one radar based on a test echo signal; and performing a series of processing on the test data and inputting the processed image into the trained convolutional neural network to determine the detection result of the awaiting article.SELECTED DRAWING: Figure 16

Description

The present invention relates to the field of communication technology, and more particularly to article detection methods and devices.

In recent years, the issue of safety in public places has been emphasized. For example, how to detect dangerous goods such as regulated products, combustibles, and explosives has become an important issue. So far, hazardous material detectors are widely used in crowded areas such as airports, train stations, subway stations, and stadiums. Dangerous goods detection devices can be divided into two types: contact type and non-contact type. The contact type detection device needs to place a suspicious object (for example, a bottle containing a liquid) on the detection device before performing detection, and the non-contact type detection device moves the suspicious object within a predetermined range to the detection device. Then, the detection can be activated to recognize whether the suspicious object belongs to the dangerous substance.

For now, when it comes to non-contact detectors, one of the most common detection methods is the X-ray detection method, which is usually costly and can be used for long periods of time to improve worker health. It can also have an adverse effect. In addition, since the materials of different articles are different, there are differences in their reflection characteristics, and such differences can also be used for detecting articles, that is, a radar is provided to transmit a signal to an article waiting to be detected. Detection can be performed based on information such as intensity and radial velocity obtained by performing analysis using echo signals from articles waiting for detection. Further, in order to improve the accuracy of detection, a plurality of radars are installed and the analysis is performed based on the echo signals received by the plurality of radars, and the detection is performed based on the information such as the intensity and the radial velocity. Can also be done.

In recent years, deep learning has become a hotspot for research in various fields. Similarly, in the field of radar, a deep learning algorithm can be used to perform the corresponding information processing on a signal. Compared to the conventional method, the deep learning algorithm has advantages such as being able to automatically extract deep features and obtaining a high accuracy rate, but there is an overfit problem when training data is insufficient. To do.

To solve the above problems, the embodiments of the present invention provide article detection methods and devices.

According to the first aspect of the embodiment of the present invention, a device for detecting an article is provided, and the device is
A training data acquisition unit for obtaining training data, the training data including original training data and strengthening training data, and each training data is labeled as dangerous or safe;
In the training unit, the training data is used as an input of the convolutional neural network, the corresponding label is used as the output of the convolutional neural network, training is performed on the convolutional neural network by a predetermined algorithm, and the convolution after training is performed. What gets a neural network;
A test data acquisition unit for acquiring test data of an article awaiting detection, the test data including a microdoppler image acquired by at least one radar based on a test echo signal, the test echo signal being transmitted by the radar. The signal to be processed is a signal after being reflected by an article waiting to be detected; and a processing unit that processes the microdoppler image of at least one radar included in the test data, and the processed image. Is input to the convolutional neural network after training to determine the detection result of the awaiting detection article.
The training data acquisition unit
An original training data generation unit that generates the original training data based on a microdoppler image acquired by the at least one radar based on the training echo signal, and the training echo signal is known as a signal transmitted by the radar. What is the signal after being reflected by the article; and the reinforcement training data acquisition unit that performs position and / or quantity conversion and / or quantity for a given number of frames in each acquired microdoppler image. A predetermined number of frames are exchanged between multiple microdoppler images for each radar obtained by one or more acquisitions, and / or multiple microdoppler images for each acquired radar. Includes those that acquire the strengthening training data by performing merging according to the first order.

According to a second aspect of an embodiment of the invention, a method of detecting an article is provided, wherein the method is:
Training data is acquired, the training data includes original training data and reinforcement training data, and each training data is labeled as dangerous or safe;
The training data is used as the input of the convolutional neural network, the corresponding label is used as the output of the convolutional neural network, the convolutional neural network is trained by a predetermined algorithm, and the convolutional neural network after training is obtained;
The test data of the article awaiting detection is acquired, the test data includes a microdoppler image acquired by at least one radar based on the test echo signal, and the test echo signal is a signal transmitted by the radar reflected by the article awaiting detection. The signal after processing; and the microdoppler image of at least one radar included in the test data is processed, and the processed image is input to the convolutional neural network after training, and the article awaiting detection. Including confirming the detection result of
Obtaining training data is
The original training data is generated based on a microdoppler image acquired by the at least one radar based on the training echo signal, and the training echo signal is a signal after the signal transmitted by the radar is reflected by a known article. Yes; and position and / or quantity conversion for a given number of frames in each acquired microdoppler image, and / or multiple micros for each radar obtained by one or more acquisitions. This includes obtaining the enhanced training data by exchanging a predetermined number of frames between Doppler images and / or merging a plurality of micro Doppler images for each acquired radar according to the first order. ..

The beneficial effects of the embodiments of the present invention are as follows: one or more radars perform frame position or quantity conversion on the micro-Doppler image acquired based on the training echo signal. Alternatively, the problem of overfitting in deep learning can be solved by merging according to a plurality of types of orders and realizing the enhancement of training data.

It is a figure which shows the article detection apparatus in Example 1. FIG. It is a figure which shows the transmission signal of a microwave sensor. It is a figure which shows the swing of the article waiting for detection in Example 1. FIG. It is a figure which shows the micro Doppler spectrum in Example 1. FIG. It is a figure which shows the structure of the reinforcement unit in Example 1. FIG. It is a figure which shows the reinforcement training data in Example 1. It is a figure which shows the reinforcement training data in Example 1. It is a figure which shows the reinforcement training data in Example 1. It is a figure which shows the reinforcement training data in Example 1. It is a figure which shows the reinforcement training data in Example 1. It is a figure which shows the reinforcement training data in Example 1. It is a figure which shows the trimming of the image in Example 1. FIG. It is a figure which shows the adjustment of the image size in Example 1. FIG. It is a figure which shows the structure of the processing unit in Example 1. FIG. It is a figure which shows the hardware configuration of the article detection apparatus in Example 2. FIG. It is a flowchart of the article detection method in Example 3. It is a figure which shows the detection of the article in Example 3. FIG.

Hereinafter, suitable examples for carrying out the present invention will be described in detail with reference to the attached drawings.

In order for those skilled in the art to easily understand the principles and embodiments of the present invention, examples of the present invention will be described using a microwave radar as an example, but examples of the present invention will be described in the microwave radar. Not limited.

The first embodiment provides an article detection device. FIG. 1 is a block diagram of the article detection device. As shown in FIG. 1, the article detection device 100 includes the following.

Training Data Acquisition Unit 101: Acquires training data, which includes original training data and enhanced training data, and each training data is labeled as dangerous or safe;
Training unit 102: The training data is used as the input of the convolutional neural network, the corresponding label is used as the output of the convolutional neural network, the convolutional neural network is trained by a predetermined algorithm, and the convolutional neural network after training is performed. To get;
Test data acquisition unit 103: Acquires test data of an article awaiting detection, the test data includes a microdoppler image acquired by at least one radar based on a test echo signal, the test echo signal being a radar transmission signal. Is the signal after being reflected by the object awaiting detection;
Processing unit 104: Processes the micro-Doppler image of at least one radar included in the test data, inputs the processed image to the convolutional neural network after training, and confirms the detection result of the waiting article. To do.

Among them, the training data acquisition unit 101 includes the following.

Original training data generation unit (abbreviated as original unit) 1011: At least one radar generates the original training data based on the micro Doppler image acquired based on the training echo signal, and the training echo signal is the training echo signal. The signal after the radar transmission signal has been reflected by a known article;
Reinforcement training data acquisition unit (abbreviated as reinforcement unit) 1012: Converts the position and / or quantity of a predetermined number of frames in each acquired micro-Doppler image, and / or acquires one or more times. Swap a predetermined number of frames between multiple micro-Doppler images for each radar obtained by, and / or merge multiple micro-Doppler images for each acquired radar in a first order. By doing so, the strengthening training data is acquired.

With the above-mentioned device in this embodiment, one or more radars perform frame position or quantity conversion on the micro Doppler image acquired based on the training echo signal, or merge according to a plurality of types of sequences. By doing so and enhancing training data, the problem of overfitting in deep learning can be solved.

In this embodiment, the radar transmits an electromagnetic wave through a transmitting antenna and receives an echo signal after the electromagnetic wave is reflected by a different object. For example, the radar may be a microwave radar operating from 24.05GHz to 24.25GHz, which may be a microwave (eg, millimeter wave) signal on the waiting object, eg, Frequency-modulated Continuous Wave. , FMCW) can be transmitted, but the present embodiment is not limited to this, for example, the radar may be a microwave device operating in Ka band 27 GHz to 40 GHz, but here. Exhaustive enumeration is omitted.

For example, when the radar is a microwave sensor working in the FMCW (Frequency-modulated Continuous Wave) method, FIG. 2 is a diagram showing a transmission signal when working in the FMCW method, and the FMCW signal is a sawtooth. As shown in FIG. 2, B indicates the amount of change (frequency modulation bandwidth) of the frequency of the transmitted signal in one period, and the frequency changes linearly in one period and is the minimum. The frequency is f ₀ , the maximum frequency is f _T , and T indicates the length of the period. Hereinafter, for convenience of explanation, one cycle is referred to as one chirp, and the second predetermined number of chirps is referred to as one frame.

In this embodiment, the radar range resolution d _res may be determined by the frequency modulation bandwidth B and the speed of light c of the transmitted signal, i.e. d _res = c / 2B and the corresponding velocity resolution is v _res = λ. / 2T _f , where T _f represents the time length of one frame and is equal to mT. The above description is merely an example, and the present embodiment is not limited to this.

In this embodiment, the object waiting to be detected is carried by a moving body (for example, a human), and the moving body swings with respect to a predetermined axis (the human walking process also swings back and forth along the central axis of the human body). The moving body produces velocities that are opposite in direction along the radial direction on both sides of the axis, and the radial direction refers to the direction in which the moving body is directed toward the radar. FIG. 3 is a diagram showing the swing of the moving body. As shown in FIG. 3, when the moving body swings and swings with respect to a predetermined axis, the moving body moves to the C-axis and the D-axis whose directions are opposite to the radial direction of the C-axis on both sides of the axis. The swing may be a plurality of reciprocating motions, but the present embodiment is not limited to this.

In this embodiment, the moving body, for example, a human, may generate radial movement velocities toward the radar as a whole, in addition to generating motion velocities with opposite directions along the radial directions on both sides of the axis. it can. The radial movement speed is the same as the human walking speed. Hereinafter, for convenience of explanation, the radial motion velocity refers to the motion velocity with reference to a human.

The micro-Doppler effect is a physical phenomenon caused by an object and the resulting tremors, and a micro-Doppler image that reflects the target tremor features can be extracted from the radar echo signal. Specifically, the radar receives the echo signal due to the reflection of the moving body, and there is a frequency difference between the echo signal and the transmitting signal, and the frequency difference is directly proportional to the distance between the radar and the moving body. Then, processing is performed on the transmitted signal and the reflected signal to obtain a baseband signal, and when the moving body has motion velocities opposite to each other in the radial direction with respect to the radar, the baseband is obtained. The frequency of the signal changes, and the changing frequency includes radial motion velocity and distance information between the moving object and the radar, and by performing a two-dimensional Fourier transform (2D-FFT), the above-mentioned microdoppler You can get the image.

In this embodiment, the first predetermined number, that is, n baseband signal matrices can be obtained by performing frequency mixed sampling on the transmission signal and the echo signal. The transmitted signal is a periodic signal, the number of rows of one baseband signal matrix is equal to the second predetermined number m, and each row represents the baseband signal value of the sampling point of one period, and the baseband signal matrix The number of columns of is equal to the third predetermined number k, and the third predetermined number k is the number of sampling points in one period of the baseband signal. A two-dimensional Fourier transform is performed on each baseband signal matrix to obtain a second signal matrix, and the value of each element in the second signal matrix represents the signal strength. For each second signal matrix (each second signal matrix corresponds to one frame frame), one column out of k columns (for example, the column where the element having the maximum signal strength value is located is located. Selects (but is not limited to) and merges the n columns selected from the n second signal matrices to obtain the feature matrix. The value of each element in the feature matrix represents the signal strength. Then, a micro Doppler image can be acquired by converting the feature matrix.

One baseband signal matrix is as follows.
(Outside 1)

One second signal matrix is as follows.
(Outside 2)

The feature matrix is as follows.
(Outside 3)

FIG. 4 is a diagram showing a micro Doppler image obtained by the conversion. As shown in FIG. 4, the abscissa indicates the frame number corresponding to the frame numbers frame 1, frame 2, ..., Frame n of each column of the feature matrix, and the ordinate coordinates indicate the velocity value corresponding to one frame. Of these, the velocity value was obtained by performing a one-to-one mapping transformation based on ν _res , ..., _M × ν _res of each row of the feature matrix, and each horizontal and vertical. The gray scale of the coordinate points determined by the coordinates represents the signal strength value of the signal, that is, the signal strength value in the feature matrix. The velocity value in the vertical coordinates is only a relative reference value, and the method of one-to-one mapping can refer to the prior art, and detailed description thereof will be omitted here.

In this embodiment, since at least one radar (p, p ≧ 1) is installed, each radar obtains the corresponding micro Doppler image according to the above method at the time of training data acquisition. be able to. Therefore, p micro-Doppler images can be acquired. Hereinafter, how to process p micro-Doppler images to obtain training data will be described.

In this embodiment, the training data acquisition unit 101 generates a plurality of training data based on a micro Doppler image acquired by at least one radar based on the training echo signal. Each training data is labeled as dangerous or safe. Among them, the training echo signal is a signal after the radar transmission signal is reflected by a known article.

In this embodiment, the training data includes original training data and reinforcement training data. Hereinafter, how to acquire the original training data and the reinforcement training data will be described.

In this embodiment, for one known article, when the number p of the radar is 1, the original training data generation unit 1011 has one micro-Doppler image acquired by the one radar each time based on the training echo signal. Is one original training data. When the number of radars p is greater than 1, the original training data generation unit 1011 merges the p micro-Doppler images acquired by the p radars in the second order each time, and is obtained by the merge. One image is used as one original training data. Then, the original training data is labeled based on the classification (danger or safety) of the known article. Further, the original training data generation unit 1011 may obtain a plurality of original training data for the known article by performing the detection for one known article a plurality of times, and further, for a plurality of different known articles. By performing the above-mentioned processing, it is possible to acquire a plurality of original training data for a plurality of known articles.

In this embodiment, in order to avoid the problem of overfitting due to lack of training data, the enhanced training data acquisition unit 1012 is applied to a microdoppler image acquired by one or more radars based on the training echo signal. Therefore, a plurality of reinforcement training data can be acquired by converting the position or quantity of the frame or merging according to a plurality of types of orders. FIG. 5 is a configuration diagram of the reinforcement training data acquisition unit 1012. Hereinafter, an implementation method in which the reinforcement training data acquisition unit 1012 enhances data will be described with reference to FIG.

In one embodiment, the data can be enhanced by transforming the order of merging. When the number of radars p is greater than 1, the reinforcement training data acquisition unit 1012 includes a first reinforcement training data acquisition unit (abbreviated as the first reinforcement unit) 501, and the first reinforcement training data acquisition unit 501 , The microdoppler images corresponding to at least two radars acquired each time are merged according to the first order different from the second order, and one image obtained by each merge is combined with one strengthening training data. By doing so, it is possible to acquire a plurality of strengthening training data.

The above-mentioned images may be merged in the left-right direction or in the vertical direction. Hereinafter, merging in the left-right direction will be described as an example, but the present embodiment is not limited to this.

FIG. 6 is a diagram showing reinforcement training data acquired by the first reinforcement training data acquisition unit 501. As shown in FIG. 6, two micro-Doppler images X and Y are merged according to the order of X + Y for one original training data, and when the data is strengthened, they are merged according to the order of Y + X. It can be performed. Although p = 2 has been described above as an example, this embodiment is not limited to this, and for example, for one original training data, p = 3, that is, three microdoppler images X, Y, and Z are X. It is merged according to the order of + Y + Z, and when enhancing the data, the three microdoppler images X, Y, Z are combined with Z + Y + X, and / or X + Z + Y, etc. It is also possible to perform merging according to the order of the types of, and use one image obtained by each merging as one strengthening training data.

In one embodiment, the data can be enhanced by exchanging a predetermined number of frames in a plurality of micro-Doppler images acquired once or a plurality of times. When the number p of radars is 1 or more, the strengthening training data acquisition unit 1012 includes a second strengthening training data acquisition unit (abbreviated as a second strengthening unit) 502, and the second strengthening training data acquisition unit 502. With respect to the original training data and / or the strengthening training data acquired once, the one radar in the original training data and / or the strengthening training data by the second microdoppler image about one radar acquired in the other time. Or replace the first microdoppler image for another radar. Of these, when p is greater than 1, it is necessary to obtain reinforcement training data by merging multiple micro-Doppler images after replacement.

For example, when p = 1, one micro-Doppler image X is acquired as original training data by the acquisition related to the first test, and the micro-Doppler image obtained by the acquisition related to the second test is X1. In this case, X can be replaced by X1 and X1 can be acquired as reinforcement training data.

For example, when p = 2, two microdoppler images X and Y are acquired by the acquisition related to the first test, and after merging, the training data of X + Y or Y + X is acquired, and the second The two microdoppler images obtained from the acquisitions of the two tests are X1 and Y1, in which case X1 / Y1 + Y and / or by substituting X with X1 / Y1 and merging with Y. Y + X1 / Y1 can be acquired as reinforcement training data, or X1 / Y1 + X and / or X + X1 / Y1 can be strengthened by substituting Y with X1 / Y1 and merging with X. Although it can be acquired as data, this embodiment is not limited to this.

In this embodiment, when the number of radars is greater than 1, and the second micro-Doppler image does not contain a signal intensity value greater than or equal to the first threshold, the second reinforcement training data acquisition unit 502 is predetermined. The first micro-Doppler image is selected based on the conditions, and the predetermined conditions satisfy that the original training data and / or the strengthening training data after replacement include a signal strength value equal to or higher than the first threshold value. When the second micro-Doppler image contains a signal strength value equal to or higher than the first threshold value, the second reinforcement training data acquisition unit 502 may randomly select the first micro-Doppler image.

For example, whether to replace X or Y with X1 / Y1 may be determined according to the following rule, that is, when X1 / Y1 contains a signal strength value equal to or higher than the first threshold value, the first value is determined. The microdoppler image may be either X or Y, and when X1 / Y1 does not contain a signal strength value equal to or higher than the first threshold value, the original training data and / or the strengthening training data after replacement may be used. It is necessary to satisfy that the signal strength value equal to or higher than the first threshold value is included. For example, when Y does not include the signal strength value equal to or higher than the first threshold value and X contains the signal strength value equal to or higher than the first threshold value. , X1 / Y1 replaces Y, which can ensure that the merged image contains the feature of high signal strength values, which is advantageous for object detection. The above description is merely an example. For example, the predetermined condition may be another condition, or the first micro Doppler image may be arbitrarily selected. However, in this embodiment, the first micro Doppler image may be arbitrarily selected. Not limited to this.

FIG. 7 is a diagram showing reinforcement training data. As shown in FIG. 7, for X + Y, X1 may be used to replace X and merge with Y when enhancing the data. This makes it possible to guarantee that the merged image includes the feature that the signal strength value is large. Although p = 2 has been described above as an example, the present embodiment is not limited to this, and a comprehensive enumeration is omitted here.

In one embodiment, the data can be enhanced by converting positions for a predetermined number of frames in one or more micro-Doppler images. When the number p of radars is 1 or more, the reinforcement training data acquisition unit 1012 includes a third reinforcement training data acquisition unit (abbreviated as a third reinforcement unit) 503, and the third reinforcement training data acquisition unit 503. Swaps a predetermined number of frame positions in a microdoppler image for one or at least two radars according to a predetermined rule. If p is greater than 1, it is necessary to obtain reinforcement training data by merging multiple micro-Doppler images after exchange.

In the embodiment, the positions of a predetermined number of frames in the micro Doppler image corresponding to one radar may be exchanged according to a predetermined rule. For example, when p = 1, one micro-Doppler image X may be acquired by the acquisition related to one test, and the positions of a predetermined number of frames in X may be exchanged according to a predetermined rule. For example, in X. The i-th frame and the j-th frame (predetermined number is 2) may be exchanged, or all frames in X (predetermined number are all frames) are rearranged in reverse order or other order. By changing, the strengthening training data may be acquired. When p = 2, two micro-Doppler images X and Y may be acquired by one test acquisition, and the positions of a predetermined number of frames in X may be exchanged according to a predetermined rule, and / or The positions of a predetermined number of frames in Y may be exchanged according to a predetermined rule, for example, the i-frame and the j-th frame (the predetermined number is 2) in X and / or Y may be exchanged, or All frames in X and / or Y (a predetermined number are all frames) are rearranged in reverse or other order and merged using the micro-Doppler images X'and Y after frame repositioning. By performing the processing, X + Y and / or Y + X may be acquired as reinforcement training data.

8 and 9 are diagrams showing reinforcement training data. As shown in Figure 8, for X + Y, taking p = 2, when performing data enhancement, get X'after sorting all frames in X in reverse order and merge with Y. To do. As shown in FIG. 9, after exchanging the i-th frame and the j-th frame in X, they are merged with Y. It should be noted that one micro-Doppler image X or Y may be exchanged a plurality of times, but this embodiment is not limited to this.

Alternatively, in the embodiment, when the number p of radars is greater than 1, the positions of a predetermined number of frames in the micro-Doppler image corresponding to the p radars may be exchanged according to a predetermined rule. For example, when p = 2, two micro-Doppler images X and Y are acquired by acquisition related to one test, and a predetermined number of frame positions in X and a predetermined number of frame positions in Y are set as a predetermined rule. For example, the i-frame and the i + 1 frame in X and the j-frame and the j + 1-th frame in Y (the predetermined number is 2) are exchanged according to, and after the frame position exchange. X + Y and / or Y + X may be acquired as reinforcement training data by performing the merging process using the micro Doppler images X and Y of.

The third reinforcement training data acquisition unit 503 has been described above by taking as an example a case where p = 1 and 2, the predetermined number is 2, and the predetermined number is all frames. However, this embodiment describes this. Not limited to.

In the present embodiment, the third strengthening training data acquisition unit 503 further sets the radial velocity corresponding to the frame for exchanging the position where the signal strength value equal to or higher than the second threshold value is located to the target frame for exchanging the radial velocity. Change (correct) to match the corresponding radial velocity. As shown in FIG. 9, for the i-frame and the j-frame to be exchanged, the signal strength value of the i-frame is equal to or higher than the second threshold value, and after the exchange, the radial velocity of the i-frame is set to the j-frame. It must be modified to match the corresponding radial velocity before replacement. As an option, the speed may be corrected for the jth frame after replacement (the signal strength value is smaller than the second threshold value).

In one embodiment, the data can be enhanced by converting the quantity of a predetermined number of frames in one or more micro-Doppler images. When the number p of the radar is 1 or more, the strengthening training data acquisition unit 1012 includes a fourth strengthening training data acquisition unit (abbreviated as a fourth strengthening unit) 504, and the strengthening training data acquisition unit 1012 is Select a predetermined number of frames in the first microdoppler image for one or at least two radars, remove the selected predetermined number of frames from the location of the first microdoppler image, of which the predetermined number of frames. The signal strength value corresponding to is equal to or less than the third threshold value, or the selected predetermined number of frames are duplicated in the first microdoppler image in which the frame is located, and the signal strength value corresponding to the predetermined number of frames is included. Is greater than or equal to the fourth threshold. Of these, when p is greater than 1, it is necessary to obtain reinforcement training data by merging multiple micro-Doppler images after quantity conversion.

For example, when p = 1, one microdoppler image X may be acquired by the acquisition related to one test, and a predetermined number of frames may be selected from X, and a signal corresponding to the predetermined number of frames may be selected. The intensity value is equal to or less than the third threshold value, for example, the i-th frame and the j-th frame, and the i-frame and the j-th frame may be removed from X to acquire reinforcement training data. When p = 2, two microdoppler images X and Y may be acquired by the acquisition related to one test, and a predetermined number of frames may be selected from X and / or Y, and the predetermined number of frames may be selected. The signal strength value corresponding to is less than or equal to the third threshold, for example, the i-frame and the j-th frame, the i-frame and the j-th frame are removed from X, and the microdoppler image X and the frame-removed microdoppler image X By performing the merge process using Y, X + Y and / or Y + X may be acquired as reinforcement training data.

For example, when p = 1, one microdoppler image X may be acquired by the acquisition related to one test, and a predetermined number of frames may be selected from X, and a signal corresponding to the predetermined number of frames may be selected. The intensity value is equal to or greater than the fourth threshold, for example, the i-frame and the j-frame, and the i-frame and the j-frame are duplicated in X, for example, the i-frame is inserted after the original i-frame. Then, the strengthening training data may be acquired by inserting the jth frame after the original jth frame. When p = 2, two microdoppler images X and Y may be acquired by the acquisition related to one test, and a predetermined number of frames may be selected from X and / or Y, and the predetermined number of frames may be selected. The signal strength value corresponding to is equal to or greater than the fourth threshold value, for example, the i-frame and the j-th frame, and the i-frame and the j-th frame are duplicated in X, for example, after the original i-frame. X + Y and / or Y + X by inserting the i-frame, inserting the j-frame after the original j-frame, and performing the merge process using the micro-Doppler images X and Y after inserting the frame. May be acquired as strengthening training data. The fourth reinforcement training data acquisition unit 504 has been described above by taking as an example that p = 1 and 2 and the predetermined number is 2, but the present embodiment is not limited to this.

In this embodiment, since the number of frames (corresponding to time) of the first micro-Doppler image changes after the frame is removed or inserted, the radial velocity also changes. Therefore, the fourth reinforcement training data acquisition unit 504 is further used to correct the radial velocity corresponding to each frame in the first micro-Doppler image after frame removal or insertion, and the diameter at the time of frame removal. Increases directional velocity (which corresponds to increased velocity due to shorter time) and decreases radial velocity when the frame is inserted (this corresponds to decreased velocity due to longer time) To do). The ratio of increase or decrease is related to the number of frames removed or inserted. 10 and 11 are diagrams showing reinforcement training data and speed correction. As shown in FIG. 10, it is necessary to reduce the radial velocity after the overlapping insertion of frames, and as shown in FIG. 11, it is necessary to increase the radial velocity after removing the frame.

In this embodiment, the above-mentioned first reinforcement training data acquisition unit 501, second reinforcement training data acquisition unit 502, third reinforcement training data acquisition unit 503, and fourth reinforcement training data acquisition unit 504 are carried out independently. It may be carried out in combination of two, three, or four of them, but this embodiment is not limited to this.

In this embodiment, invalid features in the training data can be removed in order to improve the training speed. Therefore, the training data acquisition unit 101 further includes the following.

First trimming unit (optional, not shown): Trimming the original training data and reinforcement training data to remove continuous regions with weak reflection intensity in the training data; or first adjustment unit (optional) Yes, not shown): Adjust the size of the training data image to a predetermined size.

FIG. 12 is a diagram showing trimming for training data. As shown in FIG. 12, the signal reflection intensity corresponding to the region where the radial velocity is larger than v1 and smaller than v2 is weak, and this region is trimmed and removed from the merged microdoppler image to remove it in the radial direction. Regions with velocities greater than or equal to v2 and less than or equal to v1 may be left, for example, optionally trimming and removing regions with weaker signal reflection intensity and greater frames than N1 corresponding to regions with more frames than N1. However, this embodiment is not limited to this.

FIG. 13 is a diagram showing size adjustment for training data. As shown in FIG. 13, after trimming, the sizes of the images corresponding to each training data may be inconsistent, and the size of the images may be adjusted to a predetermined size M × M. , The number of pixels contained in each row / column in a micro-Doppler image (by zooming).

In this embodiment, after the training data acquisition unit 101 obtains the training data, the training unit 102 uses the training data in the training set as the input of the convolutional neural network and the corresponding label as the output of the convolutional neural network. , The convolutional neural network is trained by a predetermined algorithm, and the convolutional neural network after training is acquired. Since the prior art can be referred to for the specific training process, detailed description thereof will be omitted here.

In this embodiment, the convolutional neural networks are, in order, a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a third convolutional layer, a third pooling layer, a fourth convolutional layer, and a fourth convolutional layer. Includes layers, first fully connected layers, second fully connected layers, and third fully connected layers, of which the convolutional kernels of the first and fourth convolutional layers are 5x5, the second and third convolutional layers. The layer convolutional kernel is 3x3, and the first pooling layer, the second pooling layer, the third pooling layer, and the fourth pooling layer are 2x2 maximum pooling, have the same sliding steps, and have sides. With no edge filling, the output of the first fully connected layer is 1024 units and the output of the second fully connected layer is 10 units of the third fully connected layer. The output is two units, the activation function is a linear ramp function (ReLU), and the classification algorithm for the third fully connected layer is the normalization exponential function softmax. The convolutional neural network is merely an example, and the present embodiment is not limited to this. For example, the convolutional neural network has, in order, a first convolutional layer (64 3x1 size convolutional kernels), a first pooling layer, a second convolutional layer (128 3x1 size convolutional kernels), and a first. 2 pooling layer, 3rd convolutional layer (512 3x1 size convolutional kernels), 3rd pooling layer, 4th convolutional layer (256 3x1 size convolutional kernels), 4th pooling layer, 5th It may include a convolutional layer (256 convolutional kernels of 3x1 size), a fifth pooling layer, a first fully connected layer, a second fully connected layer, and a third fully connected layer, of which the first / first The output of the two fully connected layers is 4096 dimensions, the output of the third fully connected layer is two units, the activation function is a linear ramp function (ReLU), and the third fully connected layer. The classification algorithm of is the normalization exponential function softmax, and each convolutional layer contains two consecutive convolutional layers.

In this embodiment, the detection of the actual article may be started after obtaining the convolutional neural network after the training, and the p radars send a signal to a human (having an article waiting to be detected). Then, the test echo signal after being reflected by the object waiting to be detected (held by a human) is received, and the test data acquisition unit 103 receives the test echo signal of the at least one radar, and the microdoppler corresponding to each radar is used. Get an image. Since the specific method for acquiring a micro Doppler image based on the test echo signal is the same as the method for obtaining a micro Doppler image based on the training echo signal described above, detailed description thereof will be omitted here. ..

In this embodiment, the processing unit 104 processes the micro-Doppler image of the at least one radar included in the test data, and when the number of radars is 1, the processing unit 104 is the one. When one micro-Doppler image acquired by the radar based on the test echo signal is processed as test data, the detection result is confirmed, and the number of radars is larger than 1, FIG. 14 shows the configuration of the processing unit 104. FIG. 14 and, as shown in FIG. 14, the processing unit 104 includes:

First Merge Unit 1401: Merges the micro-Doppler images of at least two radars contained in the test data into one image. It should be noted that such merging of images may be merging in the left-right direction or merging in the vertical direction, but the method of merging must be the same as the method of merging training data.

Second trimming unit 1402 (optional): Trimming is performed on the one image, and a continuous region having a weak reflection intensity in the one image is removed. Since the first trimming unit can be referred to for the specific implementation method, detailed description thereof will be omitted here.

Second adjustment unit 1403 (optional): Adjusts the size of the test data image to a predetermined size. Since the first adjustment unit can be referred to for the specific implementation method, detailed description thereof will be omitted here.

The processing unit 104 inputs the processed image to the convolutional neural network after the training, and the output result is dangerous or safe, whereby the detection result of the waiting article can be obtained.

With the above-mentioned device in this embodiment, one or more radars perform frame position or quantity conversion on the micro Doppler image acquired based on the training echo signal, or merge according to a plurality of types of sequences. By doing so and enhancing training data, the problem of overfitting in deep learning can be solved.

The second embodiment provides an article detection device. FIG. 15 is a hardware configuration diagram of the article detection device according to the embodiment of the present invention. As shown in FIG. 15, device 1500 may include one interface (not shown), central processing unit (CPU) 1520, and storage 1510, which is connected to central processing unit 1520. .. Among them, the storage device 1510 can store various data, can also store a program for detecting an article, and executes the program under the control of the central processing unit 1520 to execute various predetermined values. , Predetermined conditions and the like can be stored. The article detection device 1500 is connected to at least two radars and is used to obtain a training echo signal, a test echo signal, and the like for subsequent processing.

In one embodiment, the functionality of the article detector may be integrated into the central processing unit 1520. Among them, the central processing apparatus 1520 may be configured as follows, that is, it acquires training data, the training data includes original training data and reinforcement training data, and each training data is dangerous or safe. The training data is used as the input of the convolutional neural network, the corresponding label is used as the output of the convolutional neural network, and the convolutional neural network is trained by a predetermined algorithm. Obtain a post-training convolutional neural network; obtain test data for an article awaiting detection, the test data containing a microdoppler image acquired by the at least one radar based on the test echo signal, the test echo signal , The signal transmitted by the radar after being reflected by the object awaiting detection; and after processing the microdoppler image of at least one radar contained in the test data and training the processed image. Input to the convolutional neural network of, and determine the detection result of the article waiting to be detected.

In one embodiment, the central processor 1520 may further be configured as follows, i.e., based on the microdoppler image acquired by at least one radar based on the training echo signal, the original training data. The training echo signal is the signal after the radar transmission signal has been reflected by a known article; and the position and / / for a predetermined number of frames in each acquired microdoppler image. Or a quantity conversion and / or a predetermined number of frames exchanged between multiple microdoppler images for each radar obtained by one or more acquisitions and / or acquired. The reinforcement training data is acquired by merging a plurality of microdoppler images for each radar according to the first order.

In one embodiment, the central processor 1520 may further be configured as follows, i.e., when the number of radars is greater than 1, at least two radars acquired each time out of multiple acquisitions. Multiple original training data were acquired by merging the corresponding microdoppler images according to the second order and using one image obtained by the merging as one original training data; and at least two acquired each time. A plurality of reinforcement training data are acquired by merging the micro-Doppler images corresponding to the radar according to the second order and using one image obtained by each merging as one reinforcement training data.

In one embodiment, the central processor 1520 may be further configured as follows: one radar acquired at one time and / or enhanced training data acquired at another time. The second micro-Doppler image for is replaced with the first micro-Doppler image for the one radar or the other radar in the original training data and / or the enhanced training data.

In one embodiment, the central processor 1520 may further be configured as follows, i.e., swapping the positions of a predetermined number of frames in a microDoppler image for one or at least two radars according to a predetermined rule. Then, the radial speed corresponding to the frame for which the position is exchanged, where the signal strength value equal to or higher than the second threshold value is located, is corrected so as to match the radial speed corresponding to the target frame to be exchanged with the frame.

In one embodiment, the central processor 1520 may further be configured as follows, i.e., select and select a predetermined number of frames in the first microDoppler image for one or at least two radars. The predetermined number of frames are removed from the first micro-Doppler image in which the predetermined number of frames are located, and the signal strength value corresponding to the predetermined number of frames is equal to or less than the third threshold value, or the selected predetermined number of frames are selected. The first micro-Doppler image in which the image is located is duplicated, and the signal strength value corresponding to the predetermined number of frames is equal to or higher than the fourth threshold value.

In one embodiment, the central processing apparatus 1520 may further be configured as follows, i.e., modifying the radial speed corresponding to each frame in the first microDoppler image after frame removal or insertion. Among them, the radial velocity is increased when the frame is removed, and the radial velocity is decreased when the frame is inserted.

In one embodiment, the central processing apparatus 1520 may be further configured as follows, i.e., a continuous region in which the original training data and the strengthening training data are trimmed and the reflection intensity in the training data is weak. Or adjust the size of the training data image to a predetermined size.

In one embodiment, the central processor 1520 may be further configured as follows, i.e., when the number of radars is greater than 1, the micro of at least two radars included in the test data. The Doppler images are merged as one image, the one image is trimmed to remove continuous regions with weak reflection intensity in the one image, or, optionally, the size of the test data image is determined. Adjust to size.

Since the first embodiment can be referred to for a specific implementation method of the central processing unit 1520, detailed description thereof will be omitted here.

In another embodiment, the above-mentioned article detection device may be configured as a chip (not shown) connected to the central processing unit 1520, and the function of the article detection device may be realized by controlling the central processing unit 1520.

The article detection device 1500 does not need to include all the parts shown in FIG. In addition, the article detection device 1500 may further include parts not shown in FIG. 15, for which prior art can be referred to.

The above-mentioned article detector of this embodiment transforms the position or quantity of frames on a micro-Doppler image acquired by one or more radars based on a training echo signal, or according to a plurality of types of sequences. By merging and strengthening training data, the problem of overfitting in deep learning can be solved.

Example 3 of the present invention provides a method for detecting an article. Since the principle of the method to solve the problem is the same as that of the device of the first embodiment, the specific implementation thereof can be referred to the practice of the device of the first embodiment, where the same content is duplicated. The explanation is omitted.

FIG. 16 is a flowchart of one embodiment of the article detection method in this embodiment. As shown in FIG. 16, the method includes the following steps.

Step 1601: Acquire training data, which includes original training data and intensive training data, and each training data is labeled as dangerous or safe;
Step 1602: The training data is used as the input of the convolutional neural network, the corresponding label is used as the output of the convolutional neural network, the convolutional neural network is trained by a predetermined algorithm, and the convolutional neural network after training is obtained. Acquired;
Step 1603: Acquire test data of an article awaiting detection, the test data including a microdoppler image acquired by the at least one radar based on a test echo signal, of which the test echo signal is a radar transmission signal. Is the signal after being reflected by the object awaiting detection;
Step 1604: Processing is performed on the micro-Doppler image of the at least one radar included in the test data, the processed image is input to the convolutional neural network after training, and the detection result of the waiting article is determined. To do.

In this embodiment, step 1601 generates the original training data based on a micro-Doppler image acquired by at least one radar based on the training echo signal, of which the training echo signal is a transmission signal of the radar. A signal after being reflected by a known article; and performing position and / or quantity conversion for a given number of frames in each acquired micro-Doppler image and / or one or more acquisitions. Swap a predetermined number of frames between multiple microdoppler images for each radar obtained by, and / or merge multiple microdoppler images for each acquired radar in a first order. By doing so, it includes acquiring the strengthening training data.

In this embodiment, the training data acquisition unit 101, the training unit 102, the test data acquisition unit 103, and the processing unit 104 in the first embodiment can be referred to for the specific implementation methods of steps 1601 to 1604. The contents are merged here, and duplicate description is omitted here.

In this embodiment, in one embodiment of step 1601, when the number of radars is greater than 1, the second of the multiple acquisitions is for the microdoppler image corresponding to at least two radars acquired each time. Multiple original training data are acquired by merging in order and using one image obtained by the merging as one original training data; and a microdoppler image corresponding to at least two radars acquired each time. On the other hand, a plurality of reinforcement training data are acquired by performing merging according to the second order and using one image obtained by each merging as one reinforcement training data.

In this embodiment, in one embodiment of step 1601, the original training data and / or the strengthening training data acquired once is based on the second micro-Doppler image of one radar acquired in the other times. And / or replace the first micro-Doppler image for that one radar or other radar in the training data.

In this embodiment, in one embodiment of step 1601, the positions of a predetermined number of frames in a microDoppler image for one or at least two radars are exchanged according to a predetermined rule. As an option, the radial speed corresponding to the frame for which the position is exchanged, where the signal strength value equal to or higher than the second threshold value is located, is modified so as to match the radial speed corresponding to the target frame to be exchanged with it.

In this embodiment, in one embodiment of step 1601, a predetermined number of frames in the first micro Doppler image for one or at least two radars are selected and the selected predetermined number of frames are located in the first micro. It is removed from the Doppler image, and the signal strength value corresponding to the predetermined number of frames is equal to or less than the third threshold value, or the selected predetermined number of frames are duplicated in the first micro Doppler image in which the predetermined number of frames are located. , The signal strength value corresponding to the predetermined number of frames is equal to or higher than the fourth threshold value, and optionally, the radial velocity corresponding to each frame in the first micro-Doppler image after frame removal or insertion is modified, among which frames. At the time of removal, the radial velocity is increased, and at the time of frame insertion, the radial velocity is decreased.

In this embodiment, in step 1604, when the number of radars is greater than 1, processing the microdoppler images of at least two radars included in the test data is included in the test data. The microdoppler images of at least two radars are merged into one image; and optionally trimming is performed on the one image to remove continuous regions with low reflection intensity in the one image. It further includes adjusting the size of the test data image to a predetermined size.

FIG. 17 is a diagram showing detection of an article according to this embodiment. As shown in FIG. 17, at least one radar (only two are shown) sends a signal to a walking human (with an article awaiting detection) and receives a reflected echo signal. The micro-Doppler images corresponding to the two radars are acquired respectively, the two micro-Doppler images are merged, and the image after trimming and sizing is input to the trained convolutional neural network. , Danger or safety can be obtained as a detection result.

According to the method described above according to this embodiment, one or more radars perform frame position or quantity conversion on the micro Doppler image acquired based on the training echo signal, or merge according to a plurality of types of orders. By doing this and realizing the enhancement of training data, the problem of overfitting in deep learning can be solved.

In the examples of the present invention, a computer-readable program is further provided, of which, when the program is executed in the article detection device, the program is sent to the computer in the article detection device according to the above-described third embodiment. Execute the article detection method.

In an embodiment of the present invention, a storage medium storing a computer-readable program is further provided, in which the computer-readable program causes a computer to execute the article detection method according to the above-described third embodiment in an article detection device.

The methods, devices, and the like described in the examples of the present invention can be realized by hardware, software modules executed by a processor, or a combination of both. For example, one or more functions in the functional block diagram shown in FIGS. 1, 5, 14, and 15 and / or a combination of one or more functions in the functional block diagram correspond to each software module in the computer program. It may be compatible with each hardware module. In addition, each of these software modules can correspond to each step shown in FIG. These hardware modules can be realized by solidifying these software modules using, for example, FPGA (field-programmable gate array).

Further, the apparatus, method, etc. according to the embodiment of the present invention may be realized by software, may be realized by hardware, or may be realized by a combination of hardware and software. The present invention also relates to such a computer-readable program, i.e., when the program is executed by a logic component, the logic component can realize the above-mentioned device or component, or the above-mentioned logic. The component can implement the method described above or a step thereof. Furthermore, the present invention also relates to a storage medium that stores the above-mentioned program, such as a hard disk, a magnetic disk, an optical disk, a DVD, or a fresh memory.

In addition, the following additional notes will be further disclosed with respect to the above-mentioned examples and the like.

(Appendix 1)
A device that detects articles
A training data acquisition unit that acquires training data, the training data including original training data and reinforcement training data, and each training data is labeled as dangerous or safe;
In the training unit, the training data is used as an input of the convolutional neural network, the corresponding label is used as the output of the convolutional neural network, training is performed on the convolutional neural network by a predetermined algorithm, and the convolution after training is performed. What gets a neural network;
A test data acquisition unit that acquires test data of an article awaiting detection, said test data including a microdoppler image acquired by at least one radar based on a test echo signal, the test echo signal being a radar transmission signal. Is the signal after being reflected by the article awaiting detection; and the processing unit that processes the microdoppler image of at least one radar included in the test data and trains the processed image. Includes those that are input to a later convolutional neural network to determine the detection result of the awaiting article.
The training data acquisition unit
An original training data generation unit that generates the original training data based on a microdoppler image acquired by the at least one radar based on the training echo signal, and the training echo signal is known as a transmission signal of the radar. What is the signal after being reflected by the article; and the reinforcement training data acquisition unit that performs position and / or quantity conversion and / or quantity for a given number of frames in each acquired microdoppler image. A predetermined number of frames are exchanged between multiple microdoppler images for each radar obtained by one or more acquisitions, and / or for multiple microdoppler images for each acquired radar. A device that acquires the strengthening training data by performing merging according to the first order.

(Appendix 2)
The device described in Appendix 1
When the number of radars is greater than one
The original training data generation unit merges the micro-Doppler images corresponding to at least two radars acquired each time in a plurality of times according to the second order, and one image obtained by the merge is subjected to one original training. By making the data, a plurality of original training data are acquired; and / or the reinforcement training data acquisition unit includes the first reinforcement training data acquisition unit.
The first reinforcement training data acquisition unit merges the micro Doppler images corresponding to at least two radars acquired each time according to the first order, and one image obtained by each merge is subjected to one reinforcement training. A device that acquires multiple reinforcement training data by using it as data.

(Appendix 3)
The device according to Appendix 1 or 2.
The reinforcement training data acquisition unit further includes a second reinforcement training data acquisition unit.
The second strengthening training data acquisition unit uses the original training data and / or the strengthening training data acquired once with the second microdoppler image of one radar acquired at another time. A device that replaces the first microdoppler image for the one radar or the other radar in the training data.

(Appendix 4)
The device according to Appendix 1 or 2.
The reinforcement training data acquisition unit further includes a third reinforcement training data acquisition unit.
The third reinforcement training data acquisition unit is a device that exchanges the positions of a predetermined number of frames in a micro Doppler image for one or at least two radars according to a predetermined rule.

(Appendix 5)
The device described in Appendix 4,
The third reinforcement training data acquisition unit further sets the radial speed corresponding to the frame for exchanging the position where the signal strength value equal to or higher than the second threshold value is located, and the radial speed corresponding to the target frame to be exchanged with the radial speed. A device that corrects to match.

(Appendix 6)
The device according to Appendix 1 or 2.
The reinforcement training data acquisition unit further includes a fourth reinforcement training data acquisition unit.
The fourth enhanced training data acquisition unit selects a predetermined number of frames in the first micro-Doppler image for one or at least two radars and removes the selected predetermined number of frames from the located first micro-Doppler image. Then, the signal strength value corresponding to the predetermined number of frames is equal to or less than the third threshold value, or the selected predetermined number of frames are duplicated in the first micro-Doppler image in which the predetermined number of frames are located to correspond to the predetermined number of frames. A device whose signal strength value is equal to or greater than the fourth threshold.

(Appendix 7)
The device described in Appendix 6
The fourth reinforcement training data acquisition unit further corrects the radial velocity corresponding to each frame in the first microdoppler image after frame removal or insertion, and increases the radial velocity at the time of frame removal. A device that reduces the radial velocity when the frame is inserted.

(Appendix 8)
The device described in Appendix 1
The training data acquisition unit
The first trimming unit includes a unit that trims the original training data and the strengthening training data to remove a continuous region having a weak reflection intensity in the training data.
Alternatively, the training data acquisition unit further includes a first adjustment unit that adjusts the size of the training data image to a predetermined size.

(Appendix 9)
The device described in Appendix 1
When the number of radars is greater than one
The processing unit
The first merge unit that merges at least two radar micro-Doppler images contained in the test data into one image; and the second trimming unit that trims the one image. An apparatus that removes a continuous region having a weak reflection intensity in the one image.

(Appendix 10)
The device described in Appendix 9
The processing unit further includes a second adjusting unit that adjusts the test data image size to a predetermined size.

(Appendix 11)
A method of detecting articles
Training data is acquired, the training data includes original training data and reinforcement training data, and each training data is labeled as dangerous or safe;
The training data is used as the input of the convolutional neural network, the corresponding label is used as the output of the convolutional neural network, the convolutional neural network is trained by a predetermined algorithm, and the convolutional neural network after training is obtained;
The test data of the awaiting detection article was acquired, the test data including a microdoppler image acquired by at least one radar based on the test echo signal, and the test echo signal was a radar transmission signal reflected by the awaiting detection article. It is a later signal; and the micro-Doppler image of the at least one radar contained in the test data is processed, and the processed image is input to the convolutional neural network after training to obtain the detection-waiting article. Including confirming the detection result
Obtaining training data is
The original training data is generated based on the microdoppler image acquired by the at least one radar based on the training echo signal, and the training echo signal is a signal after the transmission signal of the radar is reflected by a known article. Yes; and position and / or quantity conversion for a given number of frames in each acquired microdoppler image, and / or multiple micros for each radar obtained by one or more acquisitions. A predetermined number of frames are exchanged between Doppler images, and / or a plurality of micro Doppler images for each acquired radar are merged according to the first order to acquire the strengthening training data. Including, method.

(Appendix 12)
The method described in Appendix 11
When the number of radars is greater than one
Generating the original training data
By merging the micro Doppler images corresponding to at least two radars acquired each time according to the second order, and using one image obtained by the merging as one original training data, a plurality of times are performed. Acquiring the original training data; and / or merging multiple micro-Doppler images for each acquired radar in the first order to obtain the enhanced training data
Multiple reinforcement trainings are performed by merging the micro Doppler images corresponding to at least two radars acquired each time according to the first order, and using one image obtained by each merging as one reinforcement training data. How to get the data.

(Appendix 13)
The method described in Appendix 11 or 12.
Acquiring the enhanced training data by exchanging a predetermined number of frames between a plurality of micro-Doppler images for each radar obtained by one or a plurality of acquisitions is possible.
For the original training data and / or the strengthening training data acquired once, the one radar or the other in the original training data and / or the strengthening training data by the second micro-Doppler image of one radar acquired in the other time. A method that involves replacing a first micro-Doppler image about a radar.

(Appendix 14)
The method described in Appendix 11 or 12.
Performing position conversion for a predetermined number of frames in each acquired micro-Doppler image and acquiring the reinforcement training data is possible.
A method comprising exchanging a predetermined number of frame positions in a micro-Doppler image for one or at least two radars according to a predetermined rule.

(Appendix 15)
The method described in Appendix 14, and further
A method comprising modifying the radial velocity corresponding to a frame for which position exchange is performed, where a signal strength value greater than or equal to the second threshold is located, to match the radial velocity corresponding to the frame to be exchanged with it.

(Appendix 16)
The method described in Appendix 11 or 12.
Performing quantity conversion for a predetermined number of frames in each acquired micro-Doppler image and acquiring the reinforcement training data is possible.
Select a predetermined number of frames in the first micro-Doppler image for one or at least two radars, remove the selected predetermined number of frames from the located first micro-Doppler image, and correspond to the predetermined number of frames. When the signal strength value is equal to or less than the third threshold value, or when a predetermined number of selected frames are duplicated in the first micro-Doppler image where the selected frame is located and the signal strength value corresponding to the predetermined number of frames is equal to or higher than the fourth threshold value. Methods, including being.

(Appendix 17)
The method described in Appendix 16 and further
Correct the radial velocity corresponding to each frame in the first microdoppler image after frame removal or insertion, increase the radial velocity when removing the frame, and decrease the radial velocity when inserting the frame. Including, method.

(Appendix 18)
The method described in Appendix 11
Before getting the training data
The method further
Including trimming the original training data and the strengthening training data to remove continuous regions with weak reflection intensity in the training data.
Alternatively, a method further comprising adjusting the size of the training data image to a predetermined size.

(Appendix 19)
The method described in Appendix 11
When the number of radars is greater than 1, processing the micro-Doppler image of at least one radar contained in the test data can be done.
The micro-Doppler images of at least two radars contained in the test data are merged into one image; and the one image is trimmed to remove continuous regions with weak reflection intensity in the one image. Including, method.

(Appendix 20)
The method described in Appendix 19
Processing a microDoppler image of at least one radar included in the test data further comprises adjusting the test data image size to a predetermined size.

Although preferred embodiments of the present invention have been described above, the present invention is not limited to this embodiment, and any modification to the present invention belongs to the technical scope of the present invention unless the gist of the present invention is deviated.

Claims (10)

A device that detects articles
A training data acquisition unit that acquires training data, wherein the training data includes original training data and reinforcement training data, and each training data is labeled as dangerous or safe;
The training data is used as the input of the convolutional neural network, the label corresponding to the training data is used as the output of the convolutional neural network, the convolutional neural network is trained by a predetermined algorithm, and the convolutional neural network after training is used. Training unit to acquire;
A test data acquisition unit that acquires test data of an article awaiting detection, the test data including a microdoppler image acquired by at least one radar based on a test echo signal, the test echo signal being the at least one radar. The test data acquisition unit, which is the signal after the transmitted signal is reflected by the detection waiting article; and the microdoppler image of at least one radar included in the test data, and the processed image. Is input to the convolutional neural network after training, and includes a processing unit that determines the detection result of the awaiting detection article.
The training data acquisition unit
An original training data generation unit that generates the original training data based on a microdoppler image acquired by the at least one radar based on the training echo signal, and the training echo signal is transmitted by the at least one radar. An original training data generation unit that is the signal after the signal is reflected by a known article; and position and / or quantity conversion for a given number of frames in each acquired microdoppler image and / or A predetermined number of frames are exchanged between multiple microdoppler images for each radar obtained by one or more acquisitions, and / or for multiple microdoppler images for each acquired radar. An apparatus including an enhancement training data acquisition unit that merges according to the first order and acquires the enhancement training data. The device according to claim 1.
When the number of at least one radar is greater than one
The original training data generation unit merges the acquired micro-Doppler images corresponding to at least two radars in a second order each time, and one image obtained by the merge is combined into one. By using the original training data, a plurality of original training data are acquired; and / or the strengthening training data acquisition unit includes the first reinforcement training data acquisition unit, and the first reinforcement training data acquisition unit is always used. By merging the acquired micro-Doppler images corresponding to at least two radars according to the first order, and using one image obtained by the merging as one reinforcement training data, a plurality of reinforcement training data can be obtained. The device to get. The device according to claim 1 or 2.
The reinforcement training data acquisition unit further includes a second reinforcement training data acquisition unit, and the second reinforcement training data acquisition unit may be used for the original training data and / or the reinforcement training data acquired at one time at another time. A device that replaces a first microdoppler image for the one radar or other radar in the original training data and / or enhanced training data with a second microdoppler image for one radar acquired in. The device according to claim 1 or 2.
The reinforcement training data acquisition unit further includes a third reinforcement training data acquisition unit, and the third reinforcement training data acquisition unit positions a predetermined number of frames in a microdoppler image for one or at least two radars. A device that is replaced according to the prescribed rules. The device according to claim 4.
The third reinforcement training data acquisition unit further sets the radial velocity corresponding to the frame for which the position is exchanged, where the signal strength value equal to or higher than the second threshold value is located, in the radial direction corresponding to the target frame for exchanging the frame. A device that is modified to match the speed. The device according to claim 1 or 2.
The reinforcement training data acquisition unit further includes a fourth reinforcement training data acquisition unit, and the fourth reinforcement training data acquisition unit captures a predetermined number of frames in a first microdoppler image for one or at least two radars. A selected and selected predetermined number of frames are removed from the location of the first microdoppler image, and the signal strength value corresponding to the predetermined number of frames is equal to or less than the third threshold value, or the selected predetermined number of frames are selected. An apparatus in which a first microdoppler image in which the image is located is duplicated and the signal strength value corresponding to the predetermined number of frames is equal to or higher than the fourth threshold value. The device according to claim 6.
The fourth reinforcement training data acquisition unit further corrects the radial velocity corresponding to each frame in the first microdoppler image after frame removal or insertion, and increases the radial velocity at the time of frame removal. A device that reduces the radial velocity when the frame is inserted. The device according to claim 1.
The training data acquisition unit further includes a first trimming unit that trims the original training data and the strengthening training data to remove a continuous region having a weak reflection intensity in the training data; or the training data acquisition. The unit further comprises a first adjustment unit that adjusts the size of the training data image to a predetermined size. The device according to claim 1.
When the number of at least one radar is greater than one
The processing unit
The first merge unit that merges the micro-Doppler images of at least two radars included in the test data into one image; and the one image is trimmed and the reflection intensity in the one image is weak. A device comprising a second trimming unit that removes a continuous area. A method of detecting articles
Training data is acquired, the training data includes original training data and reinforcement training data, and each training data is labeled as dangerous or safe;
The training data is used as the input of the convolutional neural network, the label corresponding to the training data is used as the output of the convolutional neural network, the convolutional neural network is trained by a predetermined algorithm, and the convolutional neural network after training is used. Acquired;
The test data of the article awaiting detection is acquired, the test data includes a microdoppler image acquired by at least one radar based on the test echo signal, and the test echo signal is the signal transmitted by the at least one radar. It is the signal after being reflected by the waiting article; and the microdoppler image of the at least one radar included in the test data is processed, and the processed image is input to the convolution neural network after training. , Including determining the detection result of the awaiting article
Acquiring the training data
The original training data was generated based on the microdoppler image acquired by the at least one radar based on the training echo signal, and the training echo signal was a signal transmitted by the at least one radar reflected by a known article. Later signal; and each radar obtained by performing position and / or quantity conversion for a given number of frames in each acquired microdoppler image and / or one or more acquisitions. A predetermined number of frames are exchanged between the plurality of micro-Doppler images for each radar, and / or the plurality of micro-Doppler images for each acquired radar are merged according to the first order. Methods, including getting.

Images