WO2020095392A1 - Radio wave image identification device, radio wave image learning device, and radio wave image identification method - Google Patents

Abstract

This radio wave image identification device is provided with: a radio wave image acquisition unit (101) that acquires radio wave image information; a model acquisition unit (102) that acquires a model; a convolution processing unit (103) having a degree-of-separation calculation layer (131) that, on the basis of the model, sets, to the radio wave image information or feature map information generated on the basis of the radio wave image information, a region of interest being a region enclosed by a closed curve and a peripheral region being a region obtained by excluding the region of interest from a region including the region of interest and enclosed by a closed curve, and calculates a degree of separation between the region of interest and the peripheral region, and a region-of-interest convolution layer (132) that executes a convolution process on the region of interest in the radio wave image information or the feature map information on the basis of the model, assigns a weight based on the degree of separation to the processing result of the convolution process, and thus generates feature map information; an entire combination unit (106) that performs class sorting on the basis of the model and the feature map information; and a result output unit (107) that outputs the result of the class sorting performed by the entire combination unit (106).

Description

Radio wave image identification device, radio wave image learning device, and radio wave image identification method

The present invention relates to a radio wave image identification device, a radio wave image learning device, and a radio wave image identification method.

In the object identification technology for optical images using machine learning technology, it is common to use convolutional neural networks (hereinafter referred to as “CNN”), which is one of the deep learning technologies.

For example, Non-Patent Document 1 indicates that CNN called AlexNet can be used to perform highly accurate identification in object identification of an optical image.

In CNN, the feature amount is extracted from the input image, and the subjects captured in the image are classified based on the feature amount.
However, when the input image is a radio wave image, the radio wave image has less feature amount that can be extracted from the image as compared with the optical image, so that the identification accuracy is deteriorated in the conventional object identification using CNN. There was a problem.

The present invention is intended to solve the above-mentioned problems, and an object thereof is to provide a radio wave image identification device capable of performing highly accurate object identification even if an input image is a radio wave image.

A radio wave image identifying apparatus according to the present invention includes a radio wave image acquiring unit that acquires radio wave image information indicating a radio wave image, a model acquiring unit that acquires a model used for identification processing based on the radio wave image information, and the radio wave image information or the radio wave image information. Based on the model based on the feature map information generated based on the image information, an area of interest surrounded by a closed curve and an area of the area surrounded by a closed curve including the area of interest excluding the area of interest. A peripheral area is set, a separation degree calculation layer for calculating the degree of separation between the attention area and the peripheral area, and based on the model, convolution processing is performed on the attention area in the radio wave image information or the feature map information, One or more convolutions having a region of interest convolutional layer that generates feature map information by weighting the processing result of the convolutional processing based on the degree of separation A processing unit, based on the model and the characteristic map information, comprising a total binding unit that performs classification and a result output unit for total binding unit outputs a classification result of the classification, the.

According to the present invention, even if the input image is a radio wave image, highly accurate object identification can be performed.

FIG. 1 is a block diagram showing an example of the configuration of the radio wave image identifying apparatus according to the first embodiment. 2A and 2B are diagrams showing an example of a hardware configuration of a main part of the radio wave image identifying device according to the first embodiment. FIG. 3 is a diagram illustrating an example of the attention area and the peripheral area at a certain position on the radio wave image or the feature map. FIG. 4A is a diagram showing an example of a method of calculating the radius Ra indicating the size of the attention area corresponding to the position of each pixel on the radio wave image. FIG. 4B is a diagram showing an example of a method of calculating the radius Ra indicating the size of the attention area corresponding to the position of each element on the feature map. FIG. 5 is a diagram showing an example of processing a rectangular filter used in the conventional convolution processing into a circular filter corresponding to the attention area and the peripheral area according to the first embodiment. FIG. 6 is a flowchart illustrating an example of processing of the radio wave image identifying apparatus according to the first embodiment. 7A and 7B are diagrams showing an example of the radio wave image indicated by the radio wave image information acquired by the radio wave image identifying apparatus. FIG. 8 is a block diagram showing an example of a main part of the radio wave image learning apparatus according to the second embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

Embodiment 1.
A configuration of a main part of the radio wave image identifying apparatus 100 according to the first embodiment will be described with reference to FIG.
FIG. 1 is a block diagram showing an example of the configuration of the radio wave image identifying apparatus 100 according to the first embodiment.
As shown in FIG. 1, the radio wave image identification device 100 is applied to a radio wave image identification system 10.
The radio wave image identification system 10 includes a radio wave image identification device 100, a radio wave image output device 11, and a storage device 12.

The radio wave image identification device 100 identifies an object shown in the radio wave image indicated by the acquired radio wave image information, based on the radio wave image information acquired from the radio wave image output device 11 and the model acquired from the storage device 12. The identification processing is executed.
More specifically, the radio wave image identifying apparatus 100 executes an identification process for identifying an object included in a radio wave image indicated by the acquired radio wave image information, by a process using a neural network based on the acquired model. ..
Details of the radio wave image identifying apparatus 100 will be described later.

The radio wave image output device 11 demodulates a radio wave received by an antenna (not shown), and generates radio wave image information indicating a radio wave image based on the demodulated radio wave.
The radio wave image output device 11 outputs the generated radio wave image information to the radio wave image identification device 100.

The storage device 12 is a model that is information indicating parameters and programs necessary for the radio wave image identification device 100 to execute identification processing, and an ideal class for the radio wave image identification device 100 to perform classification in the identification processing. The values and are stored in the storage unit 13.
The storage device 12 outputs the ideal value of the model or class to the radio wave image identification device 100 in response to a request from the radio wave image identification device 100. In addition, the storage device 12 acquires a model from the radio wave image identification device 100 in response to a request from the radio wave image identification device 100, updates the model stored in the storage unit 13 with the acquired model, and stores the model in the storage unit 13. You may memorize it.
The ideal value of a class is represented by a two-dimensional vector when the number of classifications when classifying is 2, for example, the value of one element of the vector is 1 and the value of the other element is It is quantified to 0.

A main part of the radio wave image identifying apparatus 100 will be described.
The radio wave image identification device 100 according to the first embodiment includes a radio wave image acquisition unit 101, a model acquisition unit 102, a convolution processing unit 103, an activation unit 104, a pooling unit 105, a total combination unit 106, a result output unit 107, and an evaluation unit. And a model updating unit 109.
In the radio wave image identifying device 100, the activation unit 104, the pooling unit 105, the evaluation unit 108, and the model updating unit 109 are not essential components.

The radio wave image acquisition unit 101 acquires radio wave image information.
More specifically, for example, the radio wave image acquisition unit 101 acquires radio wave image information from the radio wave image output device 11.

The model acquisition unit 102 acquires a model used for identification processing based on radio wave image information.
More specifically, for example, the model acquisition unit 102 acquires a model from the storage device 12.

Each of the convolution processing unit 103, the activation unit 104, the pooling unit 105, the total combination unit 106, and the model updating unit 109 performs processing by a neural network based on the model acquired by the model acquiring unit 102. That is, each of the convolution processing unit 103, the activation unit 104, the pooling unit 105, the total connection unit 106, and the model update unit 109 corresponds to a layer in a layered connection neural network.
A plurality of each of the convolution processing unit 103, the activation unit 104, the pooling unit 105, the total combination unit 106, and the model updating unit 109 may exist. For example, when there are a plurality of convolution processing units 103, the plurality of convolution processing units 103 may exist in one layer or may be combined in layers. The same applies to each of the activation unit 104, the pooling unit 105, the total combination unit 106, and the model updating unit 109.

The convolution processing unit 103 includes a separation degree calculation layer 131 and a region-of-interest convolution layer 132.
The degree-of-separation calculation layer 131 uses the closed curve based on the model based on the radio wave image information acquired by the radio wave image acquisition unit 101 or the feature map information generated based on the radio wave image information acquired by the radio wave image acquisition unit 101. An attention area that is a surrounded area and a peripheral area that is an area that is the area surrounded by a closed curve that includes the attention area and that excludes the attention area are set, and the degree of separation between the attention area and the peripheral area is calculated.
The attention area convolutional layer 132 is a degree of separation in radio wave image information acquired by the radio wave image acquisition unit 101 based on the model, or in feature map information generated based on the radio wave image information acquired by the radio wave image acquisition unit 101. The convolution processing is executed on the attention area set by the calculation layer 131. Further, the attention area convolutional layer 132 generates feature map information by weighting the processing result of the convolution processing based on the degree of separation calculated by the degree-of-separation calculation layer 131.
The attention area convolutional layer 132 may perform a plurality of convolution processes on the attention area in the radio wave image information or the feature map information using a plurality of different parameters based on the model. The attention area convolutional layer 132 may generate a plurality of feature map information by weighting each processing result of the plurality of convolution processings based on the degree of separation.
In addition, the attention area convolutional layer 132 executes convolution processing on the attention area in the radio wave image information or the feature map information, and a plurality of parameters calculated based on a plurality of different parameters based on the model are used for the processing result of the convolution processing. A plurality of feature map information may be generated by giving weighting based on the degree of separation.
A method of weighting the processing result based on the degree of separation will be described later.

The closed curve in the attention area is, for example, a circle Ca having a radius Ra centered on a certain position on the radio wave image or the feature map. The circle Ca referred to here is not limited to a perfect circle, and may be, for example, a partially flattened substantially circle. Further, the shape of the closed curve in the attention area is not limited to the circular shape, and may be an elliptical shape, an oval shape, or a rounded rectangular shape. The elliptical shape, the oval shape, and the rounded rectangle include the substantially elliptical shape, the substantially oval shape, and the substantially rounded rectangle, respectively. A rounded rectangle is a figure having a shape in which some or all of the corners of the rectangle are rounded.
A method of calculating the size of the region of interest such as the radius Ra corresponding to the position of each pixel on the radio wave image or each element on the feature map will be described later.

The closed curve including the attention area in the peripheral area is, for example, a circle Cb having a radius Rb larger than the radius Ra of the circle Ca centered on the center of the circle Ca that is the closed curve in the attention area. The circle Cb referred to here is not limited to a perfect circle, and may be, for example, a partially flattened substantially circle. Further, the shape of the closed curve including the attention area in the peripheral area is not limited to the circular shape, and may be an elliptical shape, an oval shape, or a rounded rectangular shape. The elliptical shape, the oval shape, and the rounded rectangle include the substantially elliptical shape, the substantially oval shape, and the substantially rounded rectangle, respectively.
A method of calculating the size of the peripheral area such as the radius Ra and the radius Rb corresponding to the position of each pixel on the radio wave image or each element on the feature map will be described later.
The degree of separation between the region of interest and the peripheral region in the degree-of-separation calculation layer 131 and the method of calculating the degree of separation will be described later.

The activation unit 104, for example, performs a process of reducing an unnecessary feature amount by nonlinear conversion on the feature map information output by the convolution processing unit 103. For the non-linear conversion, an activation function such as a sigmoid function, a ramp function or a softmax function is used.
The pooling unit 105 performs, for example, a process of reducing the feature amount of the feature map information nonlinearly converted by the activation unit 104 so that the feature amount remains. The pooling unit 105 gives invariance to a minute position change of the target position on the feature map by the processing.

The total combination unit 106 performs class classification based on the model and the feature map information. More specifically, for example, the all-combining unit 106 classifies the feature map information output from the convolution processing unit 103 based on the feature map information processed by the activation unit 104 and the pooling unit 105. To do.
The total combination unit 106 extracts, for example, a feature amount from the feature map information and converts the extracted feature amount into a vector. When there are a plurality of all-joint units 106 and the plurality of all-joint units 106 form a layer, the vector output from the layer of the last all-joint unit 106 represents each class component in a plurality of identifiable classes. It becomes a vector containing. It is also possible to provide the total combining unit 106 that processes by the softmax function in the final layer of the total combining unit 106 that forms a plurality of layers, and convert the output vector into information indicating the probability of each class.
The total combination unit 106 may include the activation unit 104, the pooling unit 105, or the activation unit 104 and the pooling unit 105. That is, the activation unit 104, the pooling unit 105, and the total combination unit 106 may be collectively defined as the total combination unit 106.

The result output unit 107 outputs the classification result of the class combination performed by the total combination unit 106.
More specifically, for example, the result output unit 107 outputs the classification result obtained by performing the class classification by the all-combining unit 106 to the evaluation unit 108 described below.
Further, the result output unit 107 may output the classification result to a display control device (not shown). The display control device outputs the classification result by visually displaying the classification result on a display screen (not shown), for example.

The evaluation unit 108 evaluates the parameters included in the model based on the classification result output by the result output unit 107 and the ideal value in the class classification.
More specifically, for example, the evaluation unit 108 calculates the error between the probability of each class and the ideal value by an error function such as a cross entropy function based on the classification result and the ideal value of the class classification, and the error Find the parameter that minimizes.
The evaluation unit 108 acquires, for example, an ideal value in class classification from the storage device 12.

The model updating unit 109 updates the model by changing the parameters included in the model based on the evaluation result evaluated by the evaluation unit 108. The model updating unit 109 outputs the updated model to the storage device 12, for example. The storage device 12 updates the model stored in the storage unit 13 to the model output by the model updating unit 109 and stores the model in the storage unit 13. The model updating unit 109 may output the updated model to the model acquiring unit 102, for example.
FIG. 1 illustrates both the case where the model updating unit 109 outputs the updated model to the storage device 12 and the case where the model updating unit 109 outputs the updated model to the model acquisition unit 102.

In the radio wave image identifying apparatus 100, the evaluation unit 108 and the model updating unit 109 are not essential configurations. Since the radio wave image identifying apparatus 100 includes the evaluation unit 108 and the model updating unit 109, for example, the parameter for executing the identification process changes for each radio wave image information acquired by the radio wave image acquisition unit 101.
With this configuration, since the model is updated by learning for each radio wave image information acquired by the radio wave image acquisition unit 101, the radio wave image identification device 100 executes the object identification process using a more suitable model. It becomes possible to do.

With reference to FIG. 2A and FIG. 2B, a hardware configuration of a main part of the radio wave image identifying apparatus 100 according to the first embodiment will be described.
2A and 2B are diagrams showing an example of a hardware configuration of a main part of the radio wave image identifying apparatus 100 according to the first embodiment.

As shown in FIG. 2A, the radio wave image identifying apparatus 100 is composed of a computer, and the computer has a processor 201 and a memory 202. The memory 202 includes the radio wave image acquisition unit 101, model acquisition unit 102, convolution processing unit 103, activation unit 104, pooling unit 105, total combination unit 106, result output unit 107, evaluation unit 108, and model update in the memory 202. A program for causing the unit 109 to function is stored. When the processor 201 reads out and executes the program stored in the memory 202, the radio wave image acquisition unit 101, the model acquisition unit 102, the convolution processing unit 103, the activation unit 104, the pooling unit 105, the total combination unit 106, and the result. The output unit 107, the evaluation unit 108, and the model updating unit 109 are realized.

Further, as shown in FIG. 2B, the radio wave image identifying device 100 may be configured by the processing circuit 203. In this case, the functions of the radio wave image acquisition unit 101, model acquisition unit 102, convolution processing unit 103, activation unit 104, pooling unit 105, total combination unit 106, result output unit 107, evaluation unit 108, and model updating unit 109 are It may be realized by the processing circuit 203.

Further, the radio wave image identifying apparatus 100 may be composed of a processor 201, a memory 202 and a processing circuit 203 (not shown). In this case, the functions of the radio wave image acquisition unit 101, the model acquisition unit 102, the convolution processing unit 103, the activation unit 104, the pooling unit 105, the total combination unit 106, the result output unit 107, the evaluation unit 108, and the model updating unit 109 are performed. Some of the functions may be realized by the processor 201 and the memory 202, and the remaining functions may be realized by the processing circuit 203.

The processor 201 uses, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a microprocessor, a micro controller, or a DSP (Digital Signal Processor).

The memory 202 uses, for example, a semiconductor memory or a magnetic disk. More specifically, the memory 202 includes a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory-Memory), and an EEPROM (Electrically Integrated Memory). State Drive) or HDD (Hard Disk Drive) is used.

The processing circuit 203 includes, for example, an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field-Programmable Gate Array), and a So-C-Syc-SyC-SyC-Syc-SyC-Syc-System-Cyclic System. Is used.

The attention area and the peripheral area at a certain position on the radio wave image or the feature map will be described with reference to FIG.
FIG. 3 is a diagram illustrating an example of the attention area and the peripheral area at a certain position on the radio wave image or the feature map.

As shown in FIG. 3, for example, the attention area at a certain position on the radio wave image or the feature map is an area surrounded by a circle Ca having a radius Ra centered on the position. Further, the peripheral area at a certain position on the radio wave image or the feature map is an area excluding the attention area from the area surrounded by the circle Cb having the radius Rb centered on the position.

A method of calculating the size of the attention area and the size of the peripheral area corresponding to the position of each pixel on the radio wave image or each element on the feature map will be described with reference to FIG.
The convolution processing unit 103 includes, for example, a region size determination layer 133 (not shown) that executes the process before executing the process in the separation degree calculation layer 131.
The area size determination layer 133 executes convolution processing on the radio wave image information or the feature map information input to the convolution processing unit 103, for example, using a predetermined convolution filter (hereinafter referred to as “area determination filter”). By doing so, a value indicating the size of the attention area or the size of the peripheral area corresponding to the position of each pixel on the radio wave image or each element on the feature map is calculated.

FIG. 4A is a diagram showing an example of a method of calculating the radius Ra indicating the size of the attention area corresponding to the position of each pixel on the radio wave image.
Specifically, for example, radio wave image information in which the number of pixels in the vertical direction is H, the number of pixels in the horizontal direction is W, and the number of channels is C (hereinafter referred to as “H × W × C”). When input to the convolution processing unit 103, the area size determination layer 133 corresponds to each pixel of the radio wave image information by executing the convolution processing on the radio wave image information using the area determination filter. The area-of-interest size map information of “H × W × 1” in which the value of each element is the value of the radius Ra at the position of each pixel is output.
The channel in the radio wave image information corresponds to each color component such as red, blue and green in the radio wave image. That is, for example, when each pixel in the radio wave image information has three color components of a red component, a blue component, and a green component, the number of channels in the radio wave image is 3.

More specifically, for example, the radio wave image acquisition unit 101 acquires radio wave image information and generates a plurality of color component image information corresponding to each color component from the acquired radio wave image information. The area size determination layer 133 performs the convolution processing on the plurality of color component image information using the area determination filter, so that the value of each element corresponding to each pixel of the radio wave image information becomes The attention area size map information that is the value of the radius Ra at the position is output.

FIG. 4B is a diagram showing an example of a method of calculating the radius Ra indicating the size of the attention area corresponding to the position of each element on the feature map.
Specifically, for example, the feature map information in which the number of elements in the vertical direction is H, the number of elements in the horizontal direction is W, and the number of channels is C (hereinafter referred to as “H × W × C”). When input to the convolution processing unit 103, the area size determination layer 133 uses the area determination filter to perform the convolution processing on the feature map, so that each element corresponding to each element of the feature map The area-of-interest size map information of “H × W × 1” whose value is the value of the radius Ra at the position of each element of the feature map is output.

The channel in the feature map information is, for example, the number of feature map information input to the convolution processing unit 103. Specifically, for example, when the radio wave image identifying apparatus 100 includes a plurality of convolution processing units 103, and the plurality of convolution processing units 103 form a layer, the convolution processing units of the layer on which the process is executed first. One or more feature map information is output from 103, and the convolution processing unit 103 of the layer on which the process is performed later performs the convolution process on the one or more feature map information using the area determination filter. The channel is the number of one or more feature map information input to the convolution processing unit 103 of the layer to be processed later.

The radius Rb indicating the size of the outer circumference of the peripheral region corresponding to the position of each pixel on the radio wave image or each element on the feature map is calculated, for example, by the same method as the method of calculating the radius Ra described above. That is, the area size determination layer 133 determines the size of the outer circumference of the peripheral area corresponding to the position of each pixel on the radio wave image or each element on the feature map by the same method as outputting the attention area size map information. The peripheral area size map information shown is output.
In addition, for example, the size value of the outer circumference of the peripheral area corresponding to the position of each pixel on the radio wave image or each element on the feature map is preset to a value indicating the size of the attention area corresponding to the position. It may be calculated by multiplying by a value.

When the attention area or the peripheral area has a shape other than a circle, for example, an ellipse, the area size determination layer 133 includes an area determination filter for calculating the major axis of the ellipse and an area determination filter for calculating the minor axis. By using the area of interest size map information or area size map information for the major axis that indicates the size of the area of interest or the peripheral area corresponding to the position of each pixel on the radio wave image or each element on the feature map, and the minor axis Output the attention area size map information or the peripheral area size map information for.
That is, the area size determination layer 133 uses a plurality of area determination filters necessary for defining the sizes of the attention area and the peripheral area having a predetermined shape, thereby obtaining the necessary attention area size map information and the peripheral area size map. Output information.

The parameters of the area size determination layer 133 are learned and updated for each radio wave image information acquired by the radio wave image acquisition unit 101 by the processing in the evaluation unit 108 and the model update unit 109 described above. Therefore, the radio wave image identifying apparatus 100 can output the attention area size map information and the peripheral area size map information using a more suitable model.

The separation degree between the attention area and the peripheral area in the separation degree calculation layer 131 and a method of calculating the separation degree will be described.
The separation degree calculation layer 131 generates, for example, a plurality of filters used for calculating the degree of separation between the attention area and the peripheral area. The filters generated by the separability calculation layer 131 are, for example, an inter-region filter, a summation region filter, and a separability calculation filter.

The inter-region filter is a convolution filter for calculating the amount of change in the Radar Cross Section value (hereinafter referred to as “RCS value”) between the region of interest and the peripheral region.
Specifically, for example, the inter-region filter is the sum of the value indicating the variance of the RCS values in the attention region with respect to the summation region and the value indicating the variance of the RCS values in the peripheral regions (hereinafter referred to as “inter-region variance value”). Is a convolution filter for calculating
More specifically, for example, the inter-region filter is a convolution filter for calculating the inter-region variance value using Expression (1).

The RCS value is a value corresponding to the radio wave reflection intensity at a certain pixel on the radio wave image or a certain element on the feature map.

The summation area filter is a convolution filter for calculating the change amount of the RCS value in an area (hereinafter, referred to as “summation area”) in which the attention area and the peripheral area are combined.
Specifically, for example, the summation area filter is a convolution filter for calculating a value indicating the variance of RCS values in the summation area (hereinafter referred to as “summation area dispersion value”).
More specifically, for example, the summation area filter is a convolution filter for calculating the summation area variance value using Expression (2).

The summation area filter may be a convolution filter for calculating the summation area variance value using Expression (3) obtained by discretizing Expression (2).

The separation degree calculation filter is a filter for calculating a value indicating a ratio of the amount of change in RCS value between the attention area and the peripheral area to the amount of change in RCS value in the summation area.
Specifically, for example, the separation degree calculation filter is a filter for calculating the degree of separation between the attention area and the peripheral area by dividing the inter-area dispersion value by the total area dispersion value.
More specifically, for example, the separation degree calculation filter is a filter for calculating the degree of separation between the region of interest and the peripheral region using Expression (4).

The degree-of-separation calculation layer 131 calculates the degree of separation between the attention area and the peripheral area using an inter-area filter, a summation area filter, and a separation degree calculation filter.
The degree of separation between the attention area and the peripheral area approaches 0 when the change amount of the RCS value between the attention area and the peripheral area is small, and approaches 1 when the change amount is large.

A method of weighting the processing result based on the degree of separation will be described.
The attention area convolutional layer 132 generates a plurality of filters used for generating the feature map information. The filters generated by the attention area convolutional layer 132 are, for example, an attention area filter and a weighting filter.

The attention area filter is a convolution filter for calculating the characteristic RCS value at the position from the RCS value in the attention area corresponding to the position of an image on the radio wave image or the position of an element on the feature map. is there.
The attention area convolutional layer 132 calculates a characteristic RCS value at the position of each pixel on the radio wave image or the position of each element on the feature map using the attention area filter.

The weighting filter is calculated by the separation degree calculation layer 131, for example, to a characteristic RCS value at a position of an image on the radio wave image calculated using the attention area filter or a position of an element on the characteristic map. It is a filter for multiplying the degree of separation at the position.
The attention area convolutional layer 132 uses a weighting filter to multiply the position of each pixel on the radio wave image or the characteristic RCS value at the position of each element on the characteristic map by the degree of separation at each position. , Generate feature map information.
As described above, the degree of separation between the region of interest and the peripheral region approaches 1 when the amount of change in the RCS value between the region of interest and the peripheral region is large, and approaches 0 when the amount of change is small. Therefore, the characteristic map information generated by multiplying the characteristic RCS value at the position of each pixel on the radio wave image or the position of each element on the characteristic map by the degree of separation at each position is the degree of separation. Is high, the weight given to the attention area is large, and if the separation degree is low, the weight given to the attention area is small.

The inter-region filter, the summation region filter, and the attention region filter for performing the convolution processing on the RCS value in the attention region or the surrounding region correspond to the shape and size of the attention region or the surrounding region. The shape of the attention area and the peripheral area is a shape such as a circle, an ellipse, an oval, or a rounded rectangle.

The inter-region filter, the summation region filter, and the attention region filter are generated based on the attention region size map information and the peripheral region size map information output by the region size determination layer 133.
More specifically, for example, the inter-region filter, the summation region filter, and the region-of-interest filter are convolution filters having a predetermined shape such as a circular filter having a variable size included in the model acquired by the model acquisition unit 102. It is generated by applying a value indicating the size of the attention area or the surrounding area included in the attention area size map information and the surrounding area size map information.
Further, for example, the inter-region filter, the summation region filter, and the attention region filter, by processing each position in the rectangular filter used in the conventional convolution processing, so as to be arranged substantially evenly in the attention region or the peripheral region, It may be generated to have a predetermined shape such as a circle.

FIG. 5 is a diagram showing an example of processing a rectangular filter used in the conventional convolution processing into a circular filter corresponding to the attention area and the peripheral area according to the first embodiment.
As shown in FIG. 5, for example, the convolution processing unit 103 decomposes a rectangular filter, and arranges the decomposed rectangular filters so as to correspond to the shapes of the attention region and the peripheral region, so that the regions between the convolution filters are separated. A filter, a summation area filter, and an attention area filter may be generated.

The operation of the radio wave image identifying apparatus 100 according to the first embodiment will be described with reference to FIG.
FIG. 6 is a flowchart illustrating an example of processing of the radio wave image identifying apparatus 100 according to the first embodiment.
The radio wave image identifying apparatus 100, for example, repeatedly executes the process of the flowchart for each radio wave image information.

First, in step ST601, the radio wave image acquisition unit 101 acquires radio wave image information.
Next, in step ST602, the model acquisition unit 102 acquires a model.
Next, in step ST603, the convolution processing unit 103 determines the area of interest size map information indicating the size of the area of interest and the surrounding area at the position of each pixel on the radio wave image or the position of each element on the feature map and the surrounding area. Region size map information is generated.
Next, in step ST604, the convolution processing unit 103 calculates the degree of separation between the attention area and the peripheral area.
Next, in step ST605, the convolution processing unit 103 executes the convolution processing on the region of interest, and weights the processing result of the convolution processing based on the degree of separation to generate the feature map information.

When there are a plurality of convolution processing sections 103 and the plurality of convolution processing sections 103 form a layer, in step ST606, the convolution processing section 103 that executes the processing from step ST603 to step ST605 is the last layer. It is determined whether or not it is the convolution processing unit 103.
In step ST606, when the convolution processing unit 103 is not the convolution processing unit 103 of the last layer, the radio wave image identifying apparatus 100 returns to the processing of step ST603, and the convolution processing unit of the layer next to the layer that has executed the processing up to this point. The CPU 103 executes the processes from step ST603 to step ST606.
When the convolution processing unit 103 is the convolution processing unit 103 of the last layer in step ST606, the activation unit 104 performs processing of reducing unnecessary feature amounts in step ST607.

Next, in step ST608, the pooling unit 105 performs processing to reduce the feature amount so that it remains.
Next, in step ST609, the all-combining unit 106 performs class classification based on the model and the feature map information.
Next, in step ST610, the result output unit 107 outputs the classification result obtained by the total combining unit 106 performing the classification.
Next, in step ST611, the evaluation unit 108 evaluates the parameters included in the model based on the classification result output by the result output unit 107 and the ideal value in the class classification.
Next, in step ST612, the model updating unit 109 updates the model by changing the parameters included in the model based on the evaluation result evaluated by the evaluation unit 108.

After executing the process of step ST611, the radio wave image identifying apparatus 100 ends the process of the flowchart, returns to the process of step ST601, and repeatedly executes the process of the flowchart for each radio wave image information.
In the processing of the flowchart, the processing order of the processing of step ST601 and the processing of step ST602 may be reversed.

As described above, the radio wave image identifying apparatus 100 includes a radio wave image acquiring unit 101 that acquires radio wave image information indicating a radio wave image, a model acquiring unit 102 that acquires a model used for identification processing based on the radio wave image information, and a radio wave image. The feature map information generated based on the image information or the radio wave image information is used to exclude the attention area from the area enclosed by the closed curve including the attention area and the attention area, which is the area enclosed by the closed curve, based on the model. A surrounding area which is a closed area and calculates a degree of separation between the attention area and the surrounding area, and a convolution process for the attention area in the radio wave image information or the feature map information based on the model. The attention area convolutional layer 13 for generating the feature map information by executing the above processing and weighting the processing result of the convolution processing based on the degree of separation. And one or more convolution processing units 103 each having :, a total combining unit 106 that performs class classification based on the model and feature map information, and a result output unit 107 that outputs a classification result obtained by the total combining unit 106. Equipped with.
With this configuration, the radio wave image identifying apparatus 100 assigns weights based on the calculated degree of separation between the attention area and the peripheral area, so that even if the input image is a radio wave image, a highly accurate object can be obtained. Identification can be done.

Further, the radio wave image identifying apparatus 100 is configured such that the convolution processing unit 103 increases the weighting given to the attention area when the degree of separation is high, and reduces the weighting given to the attention area when the degree of separation is low.
With this configuration, the radio wave image identifying apparatus 100 can enhance the RCS value in the attention area when the RCS value in the attention area is characteristic.

In addition, the radio wave image identifying apparatus 100 is configured such that the size of the attention area is changed by the radio wave image information acquired by the radio wave image acquisition unit 101 by being updated by learning.
More specifically, the radio wave image identifying apparatus 100 updates the radius Ra of the circle Ca in the attention area and the radius Rb of the circle Cb in the peripheral area by learning so that the radio wave image acquiring unit 101 acquires the radio wave. It is configured to change for each image information.
With such a configuration, the radio wave image identifying apparatus 100 can calculate the feature amount that is more suitable, and can improve the identification accuracy.

In the radio wave image identifying apparatus 100, the convolution filter used when the convolution processing unit 103 performs the convolution process on the attention area in the radio wave image information or the feature map information is configured by processing a rectangular filter. I chose
With this configuration, the radio wave image identifying apparatus 100 does not need to prepare a convolution filter having a predetermined shape in advance, and can use a more general-purpose convolution filter.

Further, the radio wave image identifying apparatus 100 evaluates the parameters included in the model based on the classification result output by the result output unit 107 and the ideal value in the class classification, and the evaluation performed by the evaluation unit 108. And a model updating unit 109 that updates the model by updating the parameters included in the model based on the result.
With this configuration, the radio wave image identifying apparatus 100 can perform the identification process based on the model that is more suitable for each piece of radio wave image information to be acquired, and can improve the identification accuracy.

Further, the radio wave image identifying apparatus 100 is configured such that the shape of the attention area is a circle, an ellipse, an oval, or a rounded rectangle.
The feature of the object appearing in the radio wave image is a shape close to a circle. Therefore, the radio wave image identifying apparatus 100 uses a convolution filter having a shape such as a circle, an ellipse, an oval, or a rounded rectangle to extract a more suitable feature amount than when using a conventional rectangular filter. Therefore, even if the input image is a radio wave image, highly accurate object identification can be performed.

In the conventional convolution process, for each pixel of an image, the inner product of the k × k convolution filter and the region of vertical k (k is a natural number) pixel and horizontal k pixel centering on a certain pixel is calculated and the processing result is calculated. To do. Conventional convolution processing means calculating a degree of similarity indicating how similar the convolution area and the convolution filter are.
On the other hand, in the convolution processing in the convolution processing unit 103, in addition to the degree of similarity between the attention area centered around the attention point, such as a circle, an ellipse, an oval, or a rounded rectangle, the attention area filter , A degree of separation between the attention area and a peripheral area outside the attention area, that is, a value indicating how similar the attention area and the peripheral area are to each other is calculated, and the similarity and the degree of separation are combined. is there. With this configuration, the radio wave image identifying apparatus 100 not only simply measures the similarity between the RCS value in the attention area on the radio wave image and the RCS value in the entire radio wave image, but Since it is possible to measure the degree of peculiarity in the image information, it is possible to extract many features even in a radio wave image having a smaller feature amount than an optical image.

The radio wave image information acquired by the radio wave image identifying apparatus 100 according to the first embodiment will be described with reference to FIG. 7.
7A and 7B are diagrams showing an example of the radio wave image indicated by the radio wave image information acquired by the radio wave image identifying apparatus 100.

As illustrated in FIG. 7A, the radio wave image information acquired by the radio wave image identifying apparatus 100 is, for example, a range Doppler map represented by a two-dimensional image including a distance direction and a velocity direction.
FIG. 7A shows a radio wave image corresponding to an object that is two types of different identification targets.
In FIG. 7A, the light and shade in the radio wave image indicates the RCS intensity of the reflected wave of the radio wave received by the antenna and applied to the object as the identification target. In addition, the contrast in the radio wave image includes the influence of noise received by the antenna in addition to the reflected wave. Noise is represented by Signal-Noise Ratio (hereinafter referred to as “SN ratio”), and the smaller the SN ratio, the greater the influence of noise on the radio wave image.
The radio wave image information acquired by the radio wave image identifying apparatus 100 is not limited to the range Doppler map, for example.
As shown in FIG. 7B, the radio wave image information acquired by the radio wave image identifying apparatus 100 may be, for example, a micro-Doppler spectrumgram shown by a two-dimensional image in the frequency direction and the time direction.

The radio wave image on the left side in FIG. 7A shows the RCS intensity of the reflected wave from the object having a conical shape, and the radio wave image on the right side shows the RCS intensity of the reflected wave from the object having a cylindrical shape. Is shown. The RCS intensity of the identification target in each radio wave image in FIG. 7A is indicated by the RCS intensity in the area inside the closed curve surrounded by the broken line in the center of the image. On the other hand, what is indicated by the RCS intensity in the area other than the area is due to the influence of noise. As described above, in the radio wave image, it is difficult to distinguish the RCS intensity due to the influence of noise from the RCS intensity due to the reflected wave from the object which is the identification target, as compared with the optical image.

Further, as shown in FIG. 7A, the value of the RCS intensity in the radio wave image does not fluctuate abruptly with changes in the brightness value and the like, as compared with the optical image. Therefore, there is no clear contour in the RCS intensity due to the reflected wave from the object that is the identification target. Therefore, the conventional convolution processing that extracts the contour information or the shape information cannot extract a sufficient amount of features.
However, in the radio wave image identifying apparatus 100, the convolution processing unit 103 determines the RCS value in the attention area and the RCS value in the surrounding area even when there is no clear contour in the RCS intensity of the reflected wave from the object that is the identification target. By statistically comparing the difference between the RCS values, even if the RCS value changes smoothly, if there is a difference in the RCS value between the attention area and the peripheral area, the attention area has an important feature amount. The weighting can be increased as a matter of course.

Embodiment 2.
A radio wave image learning device 150 according to the second embodiment will be described with reference to FIG.
FIG. 8 is a block diagram showing an example of a main part of the radio wave image learning device 150 according to the second embodiment.
As shown in FIG. 8, the radio wave image learning device 150 is applied to, for example, the radio wave image learning system 15.

In the radio wave image learning apparatus 150 according to the second embodiment, the model updating unit 109 is changed to a model updating unit 109a and the model acquiring unit 102 is deleted, as compared with the radio wave image identifying apparatus 100 according to the first embodiment. It is a thing.
In the configuration of the radio wave image learning device 150 according to the second embodiment, the same components as those of the radio wave image identifying device 100 according to the first embodiment are denoted by the same reference numerals and duplicate description will be omitted. That is, the description of the configuration in FIG. 8 given the same reference numerals as those in FIG. 1 will be omitted.

The radio wave image learning system 15 includes, for example, a radio wave image learning device 150, a radio wave image output device 11, and a storage device 12.
Since the radio wave image output device 11 and the storage device 12 are the same as the radio wave image output device 11 and the storage device 12 described in the first embodiment, detailed description thereof will be omitted.

The radio wave image learning device 150 includes a radio wave image acquisition unit 101, a convolution processing unit 103, an activation unit 104, a pooling unit 105, a total combination unit 106, a result output unit 107, an evaluation unit 108, and a model update unit 109a.
The radio wave image acquisition unit 101, the convolution processing unit 103, the activation unit 104, the pooling unit 105, the total combination unit 106, the result output unit 107, and the evaluation unit 108 are the radio wave image acquisition unit 101, the convolution unit described in the first embodiment. Since it is equivalent to the processing unit 103, the activation unit 104, the pooling unit 105, the total combination unit 106, the result output unit 107, and the evaluation unit 108, detailed description thereof will be omitted.

The evaluation unit 108 holds in advance an ideal value of a class for the radio wave image learning apparatus 150 to perform class classification in the identification processing.
The model updating unit 109a holds a pre-learned model or a partially-learned model prepared in advance.
The radio wave image learning device 150 includes a model acquisition unit 102a (not shown in FIG. 8), and the model update unit 109a, for example, through the model acquisition unit 102a, before learning or partly prepared in the storage device 12. The learned model may be acquired.
The model updating unit 109a updates the model for each radio wave image information by updating the parameter included in the model based on the evaluation result evaluated by the evaluation unit 108, and holds the updated model.
In addition, the model updating unit 109a outputs the updated model to the storage device 12.
The convolution processing unit 103 generates feature map information based on the radio wave image information acquired by the radio wave image acquisition unit 101, based on the model held by the model updating unit 109a.

In the radio wave image learning apparatus 150 according to the second embodiment, the radio wave image acquisition unit 101, the model acquisition unit 102a, the convolution processing unit 103, the activation unit 104, the pooling unit 105, the total combination unit 106, the result output unit 107, and the evaluation. Each function of the unit 108 and the model updating unit 109a may be realized by the processor 201 and the memory 202 in the hardware configuration illustrated in FIGS. 2A and 2B in the first embodiment, or processing It may be realized by the circuit 203.

The operation of the main part of the radio wave image learning device 150 is the same as the operation of the radio wave image identification device 100 according to the first embodiment described with reference to FIG.

As described above, the radio wave image learning device 150 is based on the model based on the radio wave image acquisition unit 101 that acquires the radio wave image information indicating the radio wave image and the radio wave image information or the feature map information generated based on the radio wave image information. Then, a region of interest, which is a region surrounded by the closed curve, and a peripheral region, which is a region surrounded by the closed curve that includes the region of interest and excluding the region of interest, are set, and the degree of separation between the region of interest and the peripheral region is set. Based on the separation degree calculation layer 131 for calculating the, and the model, convolution processing is performed on the attention area in the radio wave image information or the feature map information, and weighting based on the separation degree is given to the processing result of the convolution processing. Based on the model and the feature map information, one or more convolution processing units 103 each having a region-of-interest convolutional layer 132 for generating the feature map information according to Based on the total combination unit 106 that performs the class classification, the result output unit 107 that outputs the classification result of the class combination performed by the total combination unit 106, the classification result that the result output unit 107 outputs, and the ideal value in the class classification. An evaluation unit 108 that evaluates parameters included in the model, and a model updating unit 109a that updates the model by updating the parameters included in the model based on the evaluation result evaluated by the evaluation unit 108. ..

With this configuration, the radio wave image learning apparatus 150 enables model learning for highly accurate object identification even if the input image is a radio wave image.

It should be noted that, within the scope of the invention, the present invention allows free combinations of the respective embodiments, modification of arbitrary constituent elements of each embodiment, or omission of arbitrary constituent elements in each embodiment. ..

The radio wave image identification device and the radio wave image learning device according to the present invention can be applied to a radio wave image identification system.

10 radio wave image identification system, 11 radio wave image output device, 12 storage device, 13 storage unit, 100 radio wave image identification device, 101 radio wave image acquisition unit, 102 model acquisition unit, 103 convolution processing unit, 104 activation unit, 105 pooling unit , 106 total combination unit, 107 result output unit, 108 evaluation unit, 109, 109a model update unit, 131 separation degree calculation layer, 132 attention area convolution layer, 201 processor, 202 memory, 203 processing circuit, 15 radio wave image learning system, 150 radio wave image learning device.

Claims (16)

1. A radio wave image acquisition unit that acquires radio wave image information indicating a radio wave image,
   A model acquisition unit that acquires a model used for identification processing based on the radio wave image information,
   An area of interest surrounded by a closed curve based on the model in the radio wave image information or the characteristic map information generated based on the radio wave image information, and an area surrounded by a closed curve including the area of interest. From the radio wave image information or the feature, based on the model, a separation degree calculation layer that sets a peripheral area that is an area excluding the attention area from, and calculates the degree of separation between the attention area and the peripheral area. And a region of interest convolution layer that generates the characteristic map information by performing a convolution process on the region of interest in the map information and weighting the processing result of the convolution process based on the degree of separation. And the convolution processing part of
   Based on the model and the feature map information, a total combining unit for classifying,
   A result output unit that outputs the classification result obtained by the all combining unit performing the class classification;
   A radio wave image identification device comprising:
2. The degree of separation is characterized in that an amount of change in RCS value between the attention area and the peripheral area is a proportion of an amount of change in RCS value in the combined area of the attention area and the peripheral area. The radio wave image identifying device according to claim 1.
3. The convolution processing unit increases the weighting given to the attention area when the separation degree is high, and decreases the weighting given to the attention area when the separation degree is low. Radio wave image identification device.
4. The radio wave image identification device according to claim 1, wherein the size of the attention area changes for each of the radio wave image information acquired by the radio wave image acquisition unit by being updated by learning.
5. The shape of the region of interest is a circle, and the radius of the circle is changed by learning, so that it changes for each of the radio wave image information acquired by the radio wave image acquisition unit. The radio wave image identification device described.
6. The convolution filter used when the convolution processing unit performs the convolution processing on the attention area in the radio wave image information or the characteristic map information is configured by processing a rectangular filter. The radio wave image identifying device according to Item 1.
7. The radio wave image identifying device according to claim 1, wherein the shape of the attention area is a circle, an ellipse, an oval, or a rounded rectangle.
8. Based on the classification result output by the result output unit and an ideal value in the class classification, an evaluation unit that evaluates a parameter included in the model,
   Based on the evaluation result evaluated by the evaluation unit, a model update unit for updating the model by updating the parameters included in the model,
   The radio wave image identification device according to any one of claims 1 to 7, further comprising:
9. A radio wave image acquisition unit that acquires radio wave image information indicating a radio wave image,
   Based on the model in the radio wave image information or the characteristic map information generated based on the radio wave image information, an attention area that is an area surrounded by a closed curve and an area surrounded by a closed curve that includes the attention area. A separation degree calculation layer that sets a peripheral area that is an area excluding the attention area and calculates the degree of separation between the attention area and the peripheral area, and the radio wave image information or the feature map based on the model. And a region of interest convolution layer that generates the feature map information by performing a convolution process on the region of interest in the information and weighting the processing result of the convolution process based on the degree of separation. A convolution processing unit,
   Based on the model and the feature map information, a total combining unit for classifying,
   A result output unit that outputs the classification result obtained by the all combining unit performing the class classification;
   Based on the classification result output by the result output unit and an ideal value in the class classification, an evaluation unit that evaluates a parameter included in the model,
   Based on the evaluation result evaluated by the evaluation unit, a model update unit for updating the model by updating the parameters included in the model,
   A radio wave image learning device characterized by being equipped with.
10. The degree of separation is characterized in that an amount of change in RCS value between the attention area and the peripheral area is a proportion of an amount of change in RCS value in the combined area of the attention area and the peripheral area. The radio wave image learning device according to claim 9.
11. The convolution processing unit increases the weighting given to the region of interest if the degree of separation is high, and reduces the weighting given to the region of interest if the degree of separation is low. Radio wave image learning device.
12. The radio wave image learning apparatus according to claim 9, wherein the size of the attention area changes for each radio wave image information acquired by the radio wave image acquisition unit by being updated by learning.
13. The shape of the region of interest is a circle, and the radius of the circle is changed by learning, thereby changing for each of the radio wave image information pieces acquired by the radio wave image acquisition unit. The described radio wave image learning device.
14. The convolution filter used when the convolution processing unit performs the convolution processing on the attention area in the radio wave image information or the characteristic map information is configured by processing a rectangular filter. The radio wave image learning device according to Item 9.
15. The radio wave image learning apparatus according to claim 9, wherein the shape of the region of interest is a circle, an ellipse, an oval, or a rounded rectangle.
16. The radio wave image acquisition unit acquires radio wave image information indicating the radio wave image,
    The model acquisition unit acquires a model used for identification processing based on the radio wave image information,
    The separation degree calculation layer included in the convolution processing unit, in the radio wave image information or the feature map information generated based on the radio wave image information, based on the model, the attention area that is an area surrounded by a closed curve, Setting a peripheral region which is a region excluding the attention region from a region surrounded by a closed curve including the attention region, and calculating the degree of separation between the attention region and the peripheral region,
    The attention area convolution layer included in the convolution processing unit performs a convolution processing on the attention area in the radio wave image information or the feature map information based on the model, and the processing result of the convolution processing includes the degree of separation. Generating the feature map information by giving a weighting based on
    The total combining unit performs class classification based on the model and the feature map information,
    The radio wave image identification method, wherein the result output unit outputs the classification result obtained by the total combination unit performing the class classification.

Images