JP2021110565A - Classification device and classification method - Google Patents

Abstract

To provide a labor-saving classification device and a classification method.SOLUTION: The classification device includes a priority determination unit 151 for determining a priority of acquisition of an ISAR image among a plurality of ships according to a learned model created by learning a relation between movements of the ships and the priority, an adjustment unit 152 for adjusting ISAR image settings according to a learned model created by learning a relation between the ISAR image settings and success or failure of the classification, a selection unit 153 for selecting the ISAR image according to a learned model created by learning a relation between the visibility of a ship's shape and success or failure of the classification, a discrimination unit 154 for discriminating the type of the ship according to a learned model created by learning a relation between feature values of the ship and a type of the ship, and a classification unit 155 for classifying the ship class of the ship according to a learned model created by learning a relation between the feature values of the ship and the ship class of the ship.SELECTED DRAWING: Figure 4

Description

The present embodiment relates to a classification device and a classification method.

Inverse Synthetic Aperture RADAR (ISAR) is known as one of the so-called imaging radar devices. ISAR images the target of a ship or the like from the reflected echo of the radar transmission signal. Many of these types of radar devices are mounted on aircraft and are used to categorize targets from a distance.

Japanese Patent No. 3412973

Currently, a person (operator) is involved in the process from the ISAR image to the classification of the target. Therefore, the operator is likely to be burdened. In addition, some processes from ISAR images to categorizing targets require a high degree of proficiency in equipment operation. In other words, it may not be possible to make an appropriate classification depending on the proficiency level of the operator.

An object to be solved by the present invention is to provide a labor-saving classification device and a classification method.

According to the embodiment, the classification device is a classification device for classifying ships based on an ISAR image, and includes a priority determination unit, an adjustment unit, a selection unit, a discrimination unit, and a classification unit. The priority determination unit determines the priority of acquiring an ISAR image among a plurality of ships searched by the radar device according to the first trained model created by learning the relationship between the movement of the ship and the priority. judge. The adjusting unit adjusts the settings of the acquired plurality of ISAR images according to the second trained model created by learning the relationship between the settings of the ISAR image and the success or failure of the classification. The selection unit selects an ISAR image from a plurality of adjusted ISAR images according to a third learned model created by learning the relationship between the visibility of the shape of the ship and the success or failure of the classification. The discriminating unit discriminates the type of the ship shown in the selected ISAR image according to the fourth learned model created by learning the relationship between the feature amount of the ship and the type of the ship. The classification section classifies the ship class shown in the selected ISAR image according to the fifth learned model created by learning the relationship between the ship's features and the ship's ship class.

FIG. 1 is a block diagram showing a configuration of a classification system of one embodiment. FIG. 2 is a block diagram showing a configuration of a radar device. FIG. 3 is a block diagram showing a server configuration. FIG. 4 is a functional block diagram of the processor. FIG. 5 is a flowchart showing the operation of the radar device. FIG. 6 is a flowchart showing the ISAR process.

Hereinafter, embodiments will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a classification system of one embodiment. The classification system 1 has a radar device 100 and a server 200. The radar device 100 is mounted on an aircraft, for example, and is used to search for an object O at sea. The object O is, for example, a ship. The server 200 is installed on land.

FIG. 2 is a block diagram showing the configuration of the radar device 100. The radar device 100 includes a transmitter 110, a circulator 120, an antenna 130, a receiver 140, a processor 150, a storage 160, an input device 170, and a display device 180.

The transmitter 110 generates a transmission signal at a predetermined transmission cycle. Then, the transmitter 110 transmits the generated transmission signal from the antenna 130 to the space via the circulator 120. A part of the transmission signal transmitted from the antenna 130 is reflected by the target and received by the antenna 130 as a reflected echo.

The circulator 120 is a circulation circuit for separating a transmission signal and a reception signal. The circulator 120 supplies the transmission signal generated by the transmitter 110 to the antenna 130, and supplies the reflected echo received by the antenna 130 to the receiver 140 as a reception signal.

The receiver 140 generates a radar video signal from the received signal based on the transmission cycle of the transmitter 110. The receiver 140 outputs the generated radar video signal to the processor 150.

The processor 150 includes a processor such as a CPU (Central Processing Unit) and a memory for a processor such as a ROM (Read Only Memory) and a RAM (Random Access Memory) to execute processing. The processor 150 controls the operation of the radar device 100. For example, the processor 150 switches the mode of the radar device 100 between the search mode and the ISAR mode. The search mode is a mode in which a large number of objects on the sea are searched by controlling the transmitter 110 so that the transmission signal is transmitted in the radial direction centered on the radar device 100. The ISAR mode is a mode for acquiring an ISAR image of an object instructed by the processor 150 by controlling the transmitter 110 so that a transmission signal is transmitted toward the object instructed by the processor 150. Further, in the search mode, the processor 150 generates a PPI (Plane Position Indicator) image from the received signal input from the receiver 140. Further, in the ISAR mode, the processor 150 generates an ISAR image from the received signal input from the receiver 140. In addition, the processor 150 determines the priority among a large number of ships searched by the radar device 100 in the search mode. Then, the processor 150 switches the mode of the radar device 100 to the ISAR mode in order to acquire the ISAR image of the ship having a high priority. In addition, the processor 150 selects a good ISAR image from a large number of ISAR images generated by the radar device 100 in order to classify the target ship class. The processor 150 also categorizes the target ship class from the selected ISAR images. The processor 150 may be configured to perform processing by a digital signal processor such as an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a DSP (Digital Signal Processor). Further, the processor 150 may be composed of a single CPU or the like, or may be composed of a plurality of CPUs or the like.

The storage 160 is a storage device for storing programs and parameters necessary for the operation of the radar device 100. Further, the storage 160 is a storage device for determining the priority of the ship, selecting the ISAR image, and storing the learned model used for classifying the ship from the ISAR image. The storage 160 may be a hard disk, a solid state drive, or the like. The trained model will be described in detail later.

The input device 170 is an interface for receiving an operation from the operator of the radar device 100. The input device 170 may be a keyboard, a mouse, a trackball, a joystick, a touch panel, or the like. The input device 170 supplies an operation signal corresponding to the operation from the operator to the processor 150. The processor 150 performs various processes in response to this operation signal.

The display device 180 includes a liquid crystal display, an organic EL display, and the like, and displays various images based on the signals processed by the processor 150.

FIG. 3 is a block diagram showing a configuration of the server 200 as an example of the classification device. The server 200 includes a processor 210, a storage 220, an input device 230, and a display device 240.

The processor 210 includes a processor such as a CPU (Central Processing Unit) and a memory for a processor such as a ROM (Read Only Memory) and a RAM (Random Access Memory) to execute processing. The processor 210 controls the operation of the server 200. The processor 210 may be configured to perform processing by a digital signal processor such as an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a DSP (Digital Signal Processor). Further, the processor 210 may be composed of a single CPU or the like, or may be composed of a plurality of CPUs or the like.

The storage 220 is a storage device for storing programs and parameters necessary for the operation of the server 200. The storage 220 may be a hard disk, a solid state drive, or the like. Further, the storage 220 is a storage device for determining the priority of the ship, selecting the ISAR image, and storing the learning model used for classifying the ship from the ISAR image. The learning model may be stored in a storage device other than the storage 220.

The input device 230 is an interface for receiving an operation from the operator of the server 200. The input device 230 may be a keyboard, a mouse, a touch panel, or the like. The input device 230 supplies an operation signal corresponding to the operation from the operator to the processor 210. The processor 210 performs various processes in response to this operation signal.

The display device 240 includes a liquid crystal display, an organic EL display, and the like, and displays various images based on the signals processed by the processor 210.

FIG. 4 is a functional block diagram of the processor 150. The processor 150 has a priority determination unit 151, an adjustment unit 152, a selection unit 153, a determination unit 154, and a classification unit 155.

The priority determination unit 151 determines the priority of acquiring the ISAR image between the ships. Then, the priority determination unit 151 gives an instruction to the transmitter 110 of the radar device 100 in order to acquire the ISAR image of the ship having a high priority. The priority determination unit 151 determines the priority by, for example, the trained model 161 by the SVM (Support Vector Machine).

The trained model 161 is, for example, a machine learning model stored in the storage 160, using data related to the movement of the ship as input data and outputting the priority classification result. The data relating to the movement of the ship may be vector data including the traveling direction of the ship, the speed of the ship, and the aspect angle (arrival angle of the received signal) of the radar device 100 with respect to the ship. The training of the trained model 161 is performed on the server 200. In learning, priority labels are attached to each of the vector data including the traveling direction, speed, and aspect angle. This priority may be given according to, for example, the operation of the input device 230 by the operator. The priority is divided into two classes, for example, "high" and "low". Then, from the combination of the vector data and the priority, the discriminating surface representing the boundary between the priority "high" and the priority "low" is learned. The trained model generated in the server 200 is stored as the trained model 161 in the radar device 100, for example, the storage 160. The priority determination unit 151 determines the priority of the ship from the input data regarding the movement of the ship based on the learned model 161. Then, the priority determination unit 151 instructs the transmitter 110 to shift to the ISAR mode in order to acquire the ISAR image of the ship having a high priority.

Here, in the example, it is assumed that the vector data relating to the movement of the ship is the traveling direction of the ship, the speed of the ship, and the aspect angle of the radar device 100. However, the vector data may include data other than these three. The priority is divided into two classes, "high" and "low". However, the priorities may be divided into three or more classes.

The adjusting unit 152 adjusts the settings of a plurality of ISAR images. The adjustment unit 152 makes a determination for adjustment by a trained model 162 by deep learning using a convolutional neural network, a random forest, or the like.

The trained model 162 is, for example, a machine learning model that is stored in the storage 160 and is configured to use the data of the ISAR image as input data and output the adjustment amount of the setting. The settings include, for example, the brightness and contrast of the image. The training of the trained model 162 is performed on the server 200. In learning, each of the ISAR images is labeled with a classification of success or failure. That is, the ISAR images that are correctly categorized, which will be described later, are labeled as successful, and the ISAR images that are not categorized correctly are labeled as failure. Whether or not the classification of ships is performed correctly may be specified by, for example, the operation of the input device 230 by the operator of the server 200. Then, from the combination of the setting of the ISAR image and the success or failure of the classification, the setting of the ISAR image of a good reference for the classification is learned. The trained model generated in the server 200 is stored as the trained model 162 in the radar device 100, for example, the storage 160. Based on such a learned model 162, the adjustment unit 152 determines the adjustment amount of the setting for bringing the input ISAR image setting closer to the reference ISAR image setting. Then, the adjustment unit 152 adjusts the brightness, contrast, and the like of the input ISAR image according to the determined adjustment amount.

Here, in the example, it is assumed that the ISAR image has two settings, brightness and contrast. However, the settings may include other than these two. Further, the setting may include data related to individual differences of the radar device 100. For example, when the reference ISAR image is associated with the data related to the individual difference of the radar device 100, the adjusting unit 152 may adjust the ISAR image including the deviation due to the individual difference.

Further, the data of the setting of the ISAR image for learning may be given by the operation of the input device 230 by the operator. The ISAR image in this case may be unconditionally labeled as successful.

The selection unit 153 selects an ISAR image suitable for each category from the plurality of ISAR images adjusted by the adjustment unit 152. The selection unit 153 selects the ISAR image by the trained model 163 by deep learning using a convolutional neural network or the like.

The trained model 163 is, for example, a machine stored in the storage 160, using the data of the ISAR image as input data, and outputting the data of the determination result as to whether or not the input ISAR image is suitable for classification. It is a learning model. The training of the trained model 163 is performed on the server 200. In learning, each of the data related to the visibility of the shape of the ship is labeled with a classification of success or failure. Data related to the visibility of the shape of the ship include, for example, ISAR image data showing the entire ship, ISAR image data showing only a part of the ship, and ISAR image data showing different orientations of the ship. , It is the data of various ISAR images of the same ship (ship class) having different shapes. Then, from the combination of the data related to the visibility of the shape of the ship and the success or failure of the classification, the feature amount of the ISAR image which is good for the classification is learned. Then, the trained model 163 outputs data to the effect that the input ISAR image features match the features of the ISAR images that are categorically good, and classify them when they do not match. Outputs data indicating that it is not suitable for. The detection of feature points and the extraction of feature quantities may be performed using SIFT (Scale Invariant Feature Transform) or the like. The trained model generated in the server 200 is stored as the trained model 163 in the radar device 100, for example, the storage 160. Based on such a trained model 163, the selection unit 153 selects an ISAR image in which the input ISAR image is good for classification.

The determination unit 154 determines the type of ship shown in the ISAR image selected by the selection unit 113. The discrimination unit 154 discriminates the type of the ship by the trained model 164 by deep learning using a convolutional neural network or the like.

The trained model 164 is, for example, a machine learning model that is stored in the storage 160 and is configured to use the ISAR image data as input data and output the ship type data. The ship type data is data indicating the types of various ships such as warships and commercial ships. The training of the trained model 164 is performed on the server 200. In learning, a label indicating the type of ship is attached to the ISAR image showing various ships. Then, the relationship between the feature quantity and the type of the ship is learned from the combination of the ISAR images of various ships and the type of the ship. Then, the trained model 164 determines an ISAR image having a feature amount similar to the feature amount of the input ISAR image, and obtains data representing the type of ship associated with the ISAR image having the similar feature amount. Output. The detection of feature points and the extraction of feature quantities may be performed using SIFT or the like. The trained model generated in the server 200 is stored as the trained model 164 in the radar device 100, for example, the storage 160. The discrimination unit 154 discriminates the type of the ship from the input ISAR image based on the trained model 164. When the discriminated ship is a merchant ship, the discriminating unit 154 gives an instruction to shift to the radar device 100 in order to return the radar device 100 to the search mode.

Here, in the example, it is assumed that there are two types of ships, warships and merchant ships. However, the type of ship may include other than these two.

The classification unit 155 classifies the ship class of the ship shown in the ISAR image determined by the discrimination unit 154 to be a non-merchant ship. The classification unit 155 classifies the ship class according to the trained model 165 by deep learning using a convolutional neural network or the like.

The trained model 165 is, for example, a machine learning model that is stored in the storage 160 and is configured to use the data of the ISAR image as input data and output the data of the ship class of the ship. The training of the trained model 165 is performed on the server 200. In learning, ISAR images showing ships of various ship classes are labeled to indicate the ship class. Then, the relationship between the feature quantity of the ship and the ship class is learned from the combination of the ISAR images of the ships of various ship classes and the ship class. Then, the trained model 165 determines an ISAR image having a feature amount similar to the feature amount of the input ISAR image, and outputs data representing the ship class associated with the ISAR image having the similar feature amount. do. The detection of feature points and the extraction of feature quantities may be performed using SIFT or the like. The trained model generated in the server 200 is stored as the trained model 165 in the radar device 100, for example, the storage 160. The classification unit 155 classifies the ship class of the ship from the input ISAR image based on the trained model 165.

The operation of the classification system 1 will be described below. FIG. 5 is a flowchart showing the operation of the radar device 100. In step S1, the processor 150 causes the transmitter 110 to transmit a transmission signal via the antenna 130 in the search mode. In the search mode, for example, the transmission signal is transmitted radially while rotating the antenna 130. Therefore, a wide range of searches centered on the radar device 100 can be performed.

In step S2, the processor 150 displays a PPI (Plane Position indicator) on the display device 180 based on the received signal received via the receiver 140. In the PPI display, the processor 150 displays on a plan view bright spots indicating each ship searched around the radar device 100.

In step S3, the processor 150 calculates the movement of each searched ship based on the received signal received via the receiver 140. As described above, the movement of the ship includes the direction of travel of the ship, the speed of the ship, and the aspect angle of the radar device 100 with respect to the ship. The traveling direction and speed of the ship can be calculated, for example, from the time change of the coordinates of the ship calculated based on the received signal. The aspect angle can be calculated, for example, by estimating the arrival angle of the received signal.

In step S4, the processor 150 makes a priority determination. That is, the processor 150 inputs the data regarding the movement of each ship received as the search result into the trained model 161 and acquires the priority of each of the searched ships. Then, the processor 150 determines, for example, the ship having the highest priority as the ship for which the ISAR image is acquired. The ships for which ISAR images are acquired are not limited to the ships with the highest priority.

In step S5, the processor 150 causes the transmitter 110 to transmit a transmission signal via the antenna 130 in ISAR mode. In the ISAR mode, the transmission signal is transmitted, for example, to the ship designated by the processor 150.

In step S6, processor 150 performs ISAR processing. After the ISAR process, the processor 150 ends the process of FIG.

FIG. 6 is a flowchart showing the ISAR process. In step S101, the processor 150 generates an ISAR image based on the received signal received via the receiver 140.

In step S102, processor 150 adjusts the ISAR image settings. That is, the processor 150 inputs the received ISAR image data to the trained model 162, and acquires the adjustment amount of the setting for bringing the input ISAR image setting closer to the reference ISAR image setting. Then, the processor 150 adjusts the brightness, contrast, and the like of the input ISAR image according to the adjustment amount.

In step S103, the processor 150 causes the display device 180 to display the ISAR image based on the adjusted ISAR image data.

In step S104, the processor 150 selects a categorically good ISAR image from the adjusted ISAR images. That is, the processor 150 inputs the adjusted ISAR image to the trained model 163 and acquires data as to whether or not it is suitable for classification. Upon receiving the data to the effect that it is suitable for the classification, the processor 150 selects the input ISAR image.

In step S105, the processor 150 determines whether or not the ISAR image could be selected. When it is determined in step S105 that the ISAR image could not be selected, the process proceeds to step S101. In this case, the processor 150 waits for the acquisition of the ISAR image at the next timing. When it is determined in step S105 that the ISAR image can be selected, the process proceeds to step S106.

In step S106, processor 150 determines the type of ship in the selected ISAR image. That is, the processor 150 inputs the selected ISAR image into the trained model 164 and acquires the ship type data.

In step S107, the processor 150 determines whether or not the determined ship type is a merchant ship. When it is determined in step S107 that the type of the determined ship is a merchant ship, the process proceeds to step S108. When it is determined in step S107 that the type of the determined ship is not a merchant ship, the process proceeds to step S109.

In step S108, the processor 150 ends the process of FIG. 6 and returns the process to step S1 of FIG. This process is a process when the determined ship type is not the target ship.

In step S109, processor 150 categorizes the ship class shown in the selected ISAR image. That is, the processor 150 inputs the selected ISAR image into the trained model 165 and acquires the ship class data of the ship. The ship class data may be only the ship class data of the ship with the highest similarity to the ship shown in the selected ISAR image, or the ship class data of a plurality of higher-ranking ships with high similarity. It may be.

In step S110, the processor 150 causes, for example, a display device 180 to display the classification result of the ship class. After that, the processor 150 ends the process of FIG. In this case, the process of FIG. 5 is also completed. Here, the method of displaying the classification result is not limited to a specific method. For example, the categorization result may be displayed on the ISAR image displayed on the display device 180, or may be displayed on another window. Furthermore, the categorization results may be listed in order of similarity.

In step S110, the final confirmation after the categorization result is displayed may be made by the operator of the radar device 100. In this case, the server 200 may be configured to listen for input of data as to whether or not the classification has been performed correctly. Further learning in the learning model in the server 200 is expected by inputting the success / failure data each time the classification is performed.

As described above, according to one embodiment, most of the series of processes from the acquisition of the ISAR image to the acquisition of the classification result is automated by using the machine learning model. As a result, the workload of the operator is significantly reduced. In addition, certain categorized results can be obtained regardless of the operator's experience.

Further, in the embodiment, the setting of the ISAR image is adjusted and then the classification is performed. As a result, the classification is performed after the individual differences of the radar device 100 and the like are absorbed. Therefore, it is expected that the accuracy of classification will be improved.

Here, in the embodiment, it is assumed that acquisition, adjustment, selection, ship classification, and ship class classification of ISAR images are performed in the radar device 100. Some of these processes may be performed on the server 200.

Further, in the above-described embodiment, the machine learning model is generated by learning by a support vector machine or deep learning. Learning may be performed in combination with zero-shot learning and the like.

The present invention is not limited to the above embodiment as it is, and at the implementation stage, the components can be modified and embodied within a range that does not deviate from the gist thereof. In addition, various inventions can be formed by an appropriate combination of the plurality of components disclosed in the above-described embodiment. For example, some components may be removed from all the components shown in the embodiments. In addition, components across different embodiments may be combined as appropriate.

1 Classification system, 100 radar device, 110 transmitter, 120 circulator, 130 antenna, 140 receiver, 150 processor, 151 priority judgment unit, 152 adjustment unit, 153 selection unit, 154 discrimination unit, 155 classification unit, 160 storage, 161 trained model, 162 trained model, 163 trained model, 164 trained model, 165 trained model, 70 input device, 180 display device, 200 server, 210 processor, 220 storage, 230 input device, 240 display device.

Claims (8)

A categorization device that categorizes ships based on ISAR images.
A priority determination unit that determines the priority of acquiring an ISAR image between a plurality of ships searched by a radar device according to a first trained model created by learning the relationship between ship movement and priority. When,
An adjustment unit that adjusts the settings of the plurality of acquired ISAR images according to the second trained model created by learning the relationship between the ISAR image settings and the success or failure of the classification.
A selection unit that selects an ISAR image from a plurality of the ISAR images adjusted according to a third trained model created by learning the relationship between the visibility of the shape of the ship and the success or failure of the classification.
A discriminator that discriminates the type of ship shown in the selected ISAR image according to the fourth learned model created by learning the relationship between the feature amount of the ship and the type of ship.
According to the fifth trained model created by learning the relationship between the ship's features and the ship's class, the classification section that classifies the ship's class shown in the selected ISAR image, and the classification section.
A categorization device equipped with. The classification device according to claim 1, wherein the first trained model is a machine learning model using a support vector machine. The classification device according to claim 1 or 2, wherein the movement includes the traveling direction and speed of the ship and the aspect angle of the radar device. The second trained model, the third trained model, the fourth trained model, and the fifth trained model are any one of claims 1 to 3 which are machine learning models by deep learning. The categorization device described in the section. The classification device according to any one of claims 1 to 4, which is mounted on an aircraft. A categorization device that categorizes ships based on ISAR images.
According to the first trained model created by learning the relationship between the movement of a ship and the priority, the priority of acquiring an ISAR image among a plurality of ships searched by a radar device is determined, and the high priority is obtained. A classification device including a priority determination unit that causes the radar device to acquire an ISAR image of a ship having the above. A categorization device that categorizes ships based on ISAR images.
A classification device including an adjustment unit that adjusts the settings of a plurality of acquired ISAR images according to a second trained model created by learning the relationship between the settings of the ISAR image and the success or failure of the classification. It is a categorization method that categorizes ships based on ISAR images.
According to the first trained model created by learning the relationship between the movement of a ship and the priority, the priority of acquiring an ISAR image among a plurality of ships searched by a radar device is determined.
Adjusting the settings of the plurality of acquired ISAR images according to the second trained model created by learning the relationship between the ISAR image settings and the success or failure of the classification, and
To select an ISAR image from a plurality of the ISAR images adjusted according to a third trained model created by learning the relationship between the shape of the ship and the success or failure of the above classification.
To determine the type of ship shown in the selected ISAR image according to the fourth trained model created by learning the relationship between the feature amount of the ship and the type of ship.
To classify the ship class shown in the selected ISAR image according to the fifth trained model created by learning the relationship between the ship's features and the ship's class.
Assortment method including.

Images