KR102327060B1 - Method, apparatus, computer-readable storage medium and computer program for extracting region of interest for identifying target - Google Patents

Abstract

A method for extracting a region of interest performed by an apparatus for extracting a region of interest according to an embodiment comprises the steps of: inputting a first synthesized aperture radar (SAR) image to a first deep learning model trained in advance to remove noise; inputting a second synthesized aperture radar image from which noise has been removed and the first synthesized aperture radar image to a second deep learning model trained in advance so that a region of interest is extracted in the first synthetic aperture radar image by the first deep learning model; merging a first feature map for the first synthesized aperture radar image and a second feature map for the second synthesized aperture radar image, which have been output from the second deep learning model; and extracting a region of interest for the first synthesized aperture radar image on the basis of the merged feature map. Images of interest with high accuracy are extracted.

Description

Method, apparatus, computer-readable recording medium and computer program for extracting a region of interest for target identification

The present invention relates to a method and apparatus for extracting a region of interest for target identification, a computer-readable recording medium, and a computer program.

Satellite Synthetic Aperture Radar (SAR) image is not affected by weather conditions such as rain, cloud, fog, or sunlight such as day or night, backlight, etc., and has very high military utility because it can acquire images over a wide area.

In particular, as a peninsula surrounded by the sea on three sides in terms of location, Korea is a peninsula surrounded by the sea on three sides and there is a high probability that there are many weather changes and there are many clouds. Compared to the satellite electro-optical image that is affected by the phenomenon, it is emerging as an important image.

Such a synthetic aperture radar (SAR) image is taken at a far distance (eg, 600 km), and the size of the object (or target) in the obtained satellite synthetic aperture radar image is very small. . In addition, since the satellite synthetic aperture radar image contains sparkle noise due to the phenomenon of the earth's surface, the object discrimination ability is limited compared to the general optical image.

In contrast, in order to discriminate an object (target) in the artificial satellite synthetic aperture radar image, it is necessary to identify the object (target) after performing data preprocessing to remove speckle noise. In the case of removing the time-consuming problem and the speckle noise, when the speckle noise includes information for object identification, there is a problem that the object identification performance is deteriorated.

Korean Patent Publication No. 10-2237994 (Registered on Feb. 2, 2021)

SUMMARY OF THE INVENTION An object of the present invention is to provide a method, an apparatus, a computer-readable recording medium, and a computer program for extracting a region of interest for target identification.

In addition, after extracting an image of interest with high accuracy through an ROI extraction method, apparatus, computer readable recording medium and computer program for target identification, identifying an object (target) included in the ROI is this It can be included in the problem to be solved of the invention.

However, the problems to be solved of the present invention are not limited to those mentioned above, and other problems to be solved that are not mentioned can be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description. will be.

In the method for extracting a region of interest according to an embodiment of the present invention, in the method for extracting a region of interest performed by an apparatus for extracting a region of interest, a first synthetic aperture radar (SAR) is applied to a first deep learning model previously trained to remove noise. inputting an image, and generating a second synthesized aperture radar image from which noise has been removed from the first synthesized aperture radar image by the first deep learning model and the first synthesized aperture radar image to extract a region of interest A first feature map for the first synthesized aperture radar image and a second feature map for the second synthesized aperture radar image output from the second deep learning model and extracting a region of interest for the first synthesized aperture radar image based on the merged feature map.

Also, the extracting of the region of interest may include extracting the region of interest by applying a non-maximum suppression (NMS) algorithm to the merged feature map.

In addition, the first deep learning model may be trained to output an image from which noise is removed from the training image when an image for training is input.

In addition, the first deep learning model may be further trained by receiving the second feature map and the region of interest for the first synthesized aperture radar image as a feedback value.

The method may further include identifying a target included in the first synthetic aperture radar image based on the region of interest.

An apparatus for extracting a region of interest according to an embodiment of the present invention includes: an input/output unit for receiving a first synthetic aperture radar (SAR) image; Memory; and a processor electrically connected to the memory, wherein the processor inputs the first synthesized aperture radar (SAR) image to a first deep learning model trained to remove noise; inputting a second synthesized aperture radar image and the first synthesized aperture radar image from which noise has been removed from the first synthesized aperture radar image by a model into a second deep learning model trained in advance so that a region of interest is extracted; , merging a first feature map for the first synthesized aperture radar image output from the second deep learning model and a second feature map for the second synthesized aperture radar image; A region of interest for the first synthesized aperture radar image may be extracted based on the ROI.

Also, the processor may extract the region of interest by applying a non-maximum suppression (NMS) algorithm to the merged feature map.

In addition, the first deep learning model may be trained to output an image from which noise is removed from the training image when an image for training is input.

In addition, the first deep learning model may be further trained by receiving the second feature map and the region of interest for the first synthesized aperture radar image as a feedback value. Also, the processor may identify a target included in the first synthetic aperture radar image based on the region of interest.

The apparatus for extracting a region of interest according to an embodiment of the present invention does not perform a data preprocessing process of removing noise from a synthesized aperture radar image, but only inputs the synthesized aperture radar image, and the ROI of the inputted synthesized aperture radar image is not performed. can be extracted.

In addition, the apparatus for extracting a region of interest according to an embodiment of the present invention uses a feature map for a region of interest calculated from each of the original synthesized aperture radar image and the noise-removed synthesized aperture radar image in which noise is removed from the original synthesized aperture radar image. Since the region of interest is extracted by doing so, it is possible to extract the region of interest with higher accuracy than extracting the region of interest using only the noise-removed synthetic aperture radar image.

However, the effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those of ordinary skill in the art to which the present disclosure belongs from the description below. will be able

1 is a block diagram of an apparatus for extracting a region of interest according to an embodiment of the present invention.
FIG. 2 is a diagram for explaining extracting a region of interest from a synthesized aperture radar image through an apparatus for extracting a region of interest according to an embodiment of the present invention.
3 is an exemplary flowchart of a procedure of a method for extracting a region of interest according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the art to which the present invention pertains It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims.

In describing the embodiments of the present invention, if it is determined that a detailed description of a well-known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, the terms to be described later are terms defined in consideration of functions in an embodiment of the present invention, which may vary according to intentions or customs of users and operators. Therefore, the definition should be made based on the content throughout this specification.

1 is a block diagram of an apparatus for extracting a region of interest according to an embodiment of the present invention.

The apparatus 100 for extracting a region of interest according to an embodiment of the present invention extracts a region of interest (ROI) from a synthetic aperture radar (SAR) image, and a synthetic aperture based on the extracted region of interest. It is possible to identify a target (or object) in a radar (SAR) image.

Referring to FIG. 1 , an apparatus for extracting a region of interest 100 according to an embodiment of the present invention may include an input/output unit 101 , a communication unit 102 , a memory 110 , and/or a processor 120 .

The input/output unit 101 transmits, for example, a command or data input from a user or other external device to other component(s) of the ROI extracting apparatus 100 according to an embodiment, or A command or data received from other component(s) of the ROI extracting apparatus 100 according to the ROI may be output to a user or an external device.

As an embodiment, the input/output unit 101 may receive a synthetic aperture radar (SAR) image, and the input/output unit 101 may include a keyboard, a mouse, a touch pad, and the like.

Here, the synthetic aperture radar (SAR) image may be a synthetic aperture radar (SAR) image captured by an artificial satellite, but is not limited thereto.

The communication unit 102 may support establishment of a wired or wireless communication channel between the ROI extracting apparatus 100 and an external device and performing communication through the established communication channel.

The memory 110 includes various data used by at least one component (the processor 120, the input/output unit 101, and/or the communication unit 102) of the ROI extraction apparatus 100, for example, software ( Example: program) and input data or output data for commands related thereto can be stored. The memory 110 may include a volatile memory or a non-volatile memory.

For example, the memory 110 may include an object identification model 115 .

Here, the object identification model 115 may include a pre-trained first deep learning model and a pre-trained second deep learning model.

The processor 120 (also referred to as a control unit, a control device, or a control circuit) includes at least one other component (eg, a hardware component (eg, an input/output unit 101 , a communication unit 102 ) of the connected region of interest extraction apparatus 100 . and/or the memory 110) or a software component), and may perform various data processing and operations.

In addition, the processor 120 may load and process commands or data received from at least one of the other components into the volatile memory, and store various data in the non-volatile memory.

To this end, the processor 120 executes a dedicated processor (eg, an embedded processor) or one or more software programs stored in a memory device for performing the corresponding operation, thereby performing the corresponding operation (generic-purpose processor). processor) (eg, a CPU or an application processor or a micro controller unit (MCU), etc.).

In more detail, the processor 120 inputs the first synthesized aperture radar (SAR) image received from the input/output unit 101 to the pre-trained first deep learning model to remove noise, and uses the first deep learning model to From the first synthesized aperture radar image, the second synthesized aperture radar image and the first synthesized aperture radar image from which noise has been removed are input to a second deep learning model trained to extract a region of interest, and output from the second deep learning model The first feature map for the synthesized first synthesized aperture radar image and the second feature map for the second synthesized aperture radar image are merged, and the region of interest for the first synthesized aperture radar image is extracted based on the merged feature map. can

As an embodiment, the processor 120 loads the object identification model 115 from the memory 110 and then uses the object identification model 115 to receive a synthetic aperture radar (SAR) image input from the input/output unit 101 . area of interest can be extracted.

Here, the object identification model 115 may include a pre-trained first deep learning model and a pre-trained second deep learning model.

In more detail, extraction of a region of interest from a synthetic aperture radar image through the region of interest extraction apparatus 100 according to an embodiment of the present invention will be described in detail with reference to FIG. 2 .

FIG. 2 is a diagram for explaining extracting a region of interest from a synthesized aperture radar image through an apparatus for extracting a region of interest according to an embodiment of the present invention.

Referring to FIG. 2 , the apparatus 100 for extracting a region of interest according to an embodiment of the present invention may extract a region of interest from a synthetic aperture radar image by using the object identification model 115 .

In more detail, the first deep learning model 210 may receive the first synthetic aperture radar (SAR) image 200 as an input.

Here, the first deep learning model 210 may be trained by unsupervised learning so that noise of input data is removed. For example, the first deep learning model 210 may be trained to output an image from which noise is removed from the training image when receiving the training image.

When the second synthetic aperture radar image 220 from which noise is removed from the first synthetic aperture radar image 200 is output by the first deep learning model 210, the second deep learning model 230 is the first The composite aperture radar image 200 and the second composite aperture radar image 220 may be input.

Here, the second deep learning model 230 may be trained in unsupervised learning or supervised learning so that a region of interest (ROI) including a target (or object) is extracted. have.

Then, by the second deep learning model 230 , the first feature map 205 for the first synthetic aperture radar image 200 and the second feature map 225 for the second synthetic aperture radar image 220 . When this is output, the ROI extraction apparatus 100 may merge the first feature map 205 and the second feature map 225 .

The ROI extracting apparatus 100 may extract the ROI from the first synthesized aperture radar image 200 based on the merged feature map 240 .

For example, the region of interest extraction apparatus 100 extracts the region of interest by applying a non-maximum suppression (NMS) algorithm to the merged feature map 240 as shown in Equation 1 below. can do.

는 관심 영역이고, NMS는 비-최대 억제(Non maximum Suppression) 알고리즘이고,

는 제 1 합성 개구 레이더 영상(200)이고,

는 제 2 합성 개구 레이더 영상(220)이고,

는 기 설정된 임계값이다.here,

is the region of interest, NMS is a non-maximum suppression algorithm,

is the first synthetic aperture radar image 200,

is the second synthetic aperture radar image 220,

is a preset threshold value.

Here, the non-maximum suppression (NMS) algorithm is an algorithm that filters out overlapping classes (or boxes).

On the other hand, when the first deep learning model 210 receives an image for training, it is trained to output an image in which noise has been removed from the training image, and then, on the second feature map 225 and the first synthetic aperture radar image 200 . It may be further learned by receiving an ROI as a feedback value.

In more detail, the first deep learning model 210 may be further learned by receiving information about the region of interest of the feature map 240 merged with the region of interest in the second feature map 225 as a feedback value. Here, the information on the region of interest of the merged feature map 240 includes information that classifies the region of interest of the merged feature map 240 for each class (or box) under a preset condition and the merged feature map 240 . It may include location information of the target included in the ROI.

At this time, when the region of interest extraction apparatus 100 according to an embodiment of the present invention receives a training image, it is trained to output an image in which noise is removed from the training image, and then, the second feature map 225 and the first synthesis aperture The second synthetic aperture radar from which noise is removed from the first synthetic aperture radar image 200 by using the first deep learning model 210 that has been further trained by receiving the region of interest for the radar image 200 as a feedback value. An image 220 may be acquired.

Meanwhile, the loss function used in the ROI extraction apparatus 100 according to the embodiment of the present invention may be calculated as in Equation 2 below.

은 본 발명의 실시예에 따른 관심 영역 추출 장치(100)에서 사용되는 손실함수이고,

은 제 1 딥러닝 모델(210)로부터 잡음이 제거된 영상을 학습시키기 위한 손실함수이고,

은 제 1 합성 개구 레이다 영상(200)에 대해 제 2 딥러닝 모델(230)로부터 관심영역을 추출하기 위한 손실 함수이고,

은 제 2 합성 개구 레이다 영상(220)에 대해 제 2 딥러닝 모델(230)로부터 관심영역을 추출하기 위한 손실 함수이고,

는 병합된 관심 영역(ROI: Region of Interest)에 대해 분류 및 표적 위치를 학습하기 위해 기 정의되는 손실 함수이고,

는 본 발명의 실시예에 따른 관심 영역 추출 장치(100)에서의 손실함수에 각각 대응되는 가중치 파라미터이다.here,

is a loss function used in the ROI extraction apparatus 100 according to an embodiment of the present invention,

is a loss function for learning an image from which noise has been removed from the first deep learning model 210,

is a loss function for extracting a region of interest from the second deep learning model 230 for the first synthesized aperture radar image 200,

is a loss function for extracting a region of interest from the second deep learning model 230 for the second synthesized aperture radar image 220,

is a pre-defined loss function to learn classification and target position for the merged region of interest (ROI),

is a weight parameter respectively corresponding to the loss function in the ROI extracting apparatus 100 according to an embodiment of the present invention.

The ROI extracting apparatus 100 may identify a target included in the first synthetic aperture radar image 200 based on the ROI.

For example, the ROI extracting apparatus 100 may classify the extracted ROI for each class (or box) under a preset condition, and then perform correction to identify a target included in the ROI.

As another example, the region of interest extraction apparatus 100 classifies the region of interest for each class (or box) in the merged feature map 240 using the pre-trained classification model, and then selects the target included in the region of interest. Identify or locate a target.

3 is an exemplary flowchart of a procedure of a method for extracting a region of interest according to an embodiment of the present invention. The method for extracting the region of interest of FIG. 3 may be performed by the apparatus 100 for extracting the region of interest shown in FIG. 1 . In addition, the method of extracting the region of interest shown in FIG. 3 is merely exemplary.

Referring to FIG. 3 , when the input/output unit 101 receives a first synthesized aperture radar (SAR) image 200 , the processor 120 transmits the first synthesized aperture radar image 200 to the first deep learning model 210 . ) can be input (step S10).

Here, the first deep learning model 210 may be trained by unsupervised learning to output an image from which noise is removed from the training image when receiving the training image.

Thereafter, the processor 120 performs a second synthetic aperture radar image 220 and a first synthetic aperture radar image 200 from which noise is removed from the first synthetic aperture radar image 200 by the first deep learning model 210 . may be input to the second deep learning model 230 (step S20).

Here, the second deep learning model 230 may be trained in unsupervised learning or supervised learning so that an image of interest including a target (or object) is extracted.

Then, the processor 120 outputs the first feature map 205 for the first synthetic aperture radar image 200 and the second for the second synthetic aperture radar image 220, output from the second deep learning model 230 . The two feature maps 225 may be merged (step S30).

Thereafter, the processor 120 may extract a region of interest for the first composite aperture radar 200 image based on the merged feature map (step S40 ).

On the other hand, when the first deep learning model 210 receives an image for training, it is trained to output an image in which noise has been removed from the training image, and then, on the second feature map 225 and the first synthetic aperture radar image 200 . It may be further learned by receiving an ROI as a feedback value.

For example, the processor 120 may extract a region of interest by applying a non-maximum suppression (NMS) algorithm to the merged feature map 240 .

Also, after extracting the ROI, the processor 120 may identify a target included in the extracted ROI according to a preset condition.

As described above, the apparatus for extracting a region of interest according to an embodiment of the present invention does not perform a data preprocessing process of removing noise from a synthesized aperture radar image, but only inputs a synthesized aperture radar image. A region of interest from the aperture radar image may be extracted.

In addition, the apparatus for extracting a region of interest according to an embodiment of the present invention uses a feature map for a region of interest calculated from each of the original synthesized aperture radar image and the noise-removed synthesized aperture radar image in which noise is removed from the original synthesized aperture radar image. Since the region of interest is extracted by doing so, it is possible to extract the region of interest with higher accuracy than extracting the region of interest using only the noise-removed synthetic aperture radar image.

Combinations of each block in the block diagram attached to the present invention and each step in the flowchart may be performed by computer program instructions. These computer program instructions may be embodied in an encoding processor of a general purpose computer, special purpose computer, or other programmable data processing equipment, such that the instructions executed by the encoding processor of the computer or other programmable data processing equipment may correspond to each block of the block diagram or Each step of the flowchart creates a means for performing the functions described. These computer program instructions may also be stored in a computer-usable or computer-readable memory that may direct a computer or other programmable data processing equipment to implement a function in a particular manner, and thus the computer-usable or computer-readable memory. The instructions stored in the block diagram may also produce an item of manufacture containing instruction means for performing a function described in each block of the block diagram or each step of the flowchart. The computer program instructions may also be mounted on a computer or other programmable data processing equipment, such that a series of operational steps are performed on the computer or other programmable data processing equipment to create a computer-executed process to create a computer or other programmable data processing equipment. It is also possible that instructions for performing the processing equipment provide steps for carrying out the functions described in each block of the block diagram and each step of the flowchart.

Further, each block or each step may represent a module, segment, or portion of code that includes one or more executable instructions for executing the specified logical function(s). It should also be noted that in some alternative embodiments it is also possible for the functions recited in blocks or steps to occur out of order. For example, it is possible that two blocks or steps shown one after another may in fact be performed substantially simultaneously, or that the blocks or steps may sometimes be performed in the reverse order according to the corresponding function.

The above description is merely illustrative of the technical idea of the present invention, and various modifications and variations will be possible without departing from the essential quality of the present invention by those skilled in the art to which the present invention pertains. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be interpreted as being included in the scope of the present invention.

100: region of interest extraction device
101: input/output unit
102: communication unit
110: memory
120: processor

Claims (12)

A method for extracting a region of interest performed by an apparatus for extracting a region of interest, the method comprising:
inputting a first synthesized aperture radar (SAR) image to a first deep learning model trained in advance to remove noise;
A second synthetic aperture radar image and the first composite aperture radar image in which noise is removed from the first composite aperture radar image by the first deep learning model, a first region of interest for the first composite aperture radar image inputting the extracted first feature map and the second region of interest for the second synthesized aperture radar image to a pre-trained second deep learning model to output the extracted second feature map;
generating a final feature map by merging the first feature map including the first region of interest and the second feature map including the second region of interest output from the second deep learning model;
and extracting a final region of interest for the first synthetic aperture radar image based on the final feature map.
A method of extracting a region of interest. According to claim 1,
Extracting the final region of interest comprises:
Extracting the region of interest by applying a Non-Maximum Suppression (NMS) algorithm to the final feature map
A method of extracting a region of interest. According to claim 1,
The first deep learning model is,
When an image for learning is input, it is learned to output an image from which noise has been removed from the image for learning.
A method of extracting a region of interest. 4. The method of claim 3,
The first deep learning model is,
The second feature map and the first region of interest are input as feedback values and are further learned.
A method of extracting a region of interest. According to claim 1,
Further comprising the step of identifying a target included in the first synthetic aperture radar image based on the final region of interest
A method of extracting a region of interest. an input/output unit receiving a first synthetic aperture radar (SAR) image;
Memory; and
a processor electrically connected to the memory;
The processor is
Input the first synthesized aperture radar (SAR) image to a first deep learning model trained so that noise is removed,
A second synthetic aperture radar image and the first composite aperture radar image in which noise is removed from the first composite aperture radar image by the first deep learning model, a first region of interest for the first composite aperture radar image The extracted first feature map and the second region of interest for the second synthesized aperture radar image are input to a pre-trained second deep learning model to output the extracted second feature map,
Merge the first feature map including the first region of interest and the second feature map including the second region of interest output from the second deep learning model to generate a final feature map,
extracting a final ROI for the first synthetic aperture radar image based on the final feature map
region of interest extraction device. 7. The method of claim 6,
The processor is
Extracting the region of interest by applying a Non-Maximum Suppression (NMS) algorithm to the final feature map
region of interest extraction device. 8. The method of claim 7,
The first deep learning model is,
When an image for learning is input, it is learned to output an image from which noise has been removed from the image for learning.
region of interest extraction device. 9. The method of claim 8,
The first deep learning model is,
The second feature map and the first region of interest are input as feedback values and are further learned.
region of interest extraction device. 7. The method of claim 6,
The processor is
identifying a target included in the first synthetic aperture radar image based on the final region of interest
region of interest extraction device. As a computer-readable recording medium storing a computer program,
The computer program, when executed by a processor,
inputting a first synthesized aperture radar (SAR) image to a first deep learning model trained in advance to remove noise;
A second synthetic aperture radar image and the first composite aperture radar image in which noise is removed from the first composite aperture radar image by the first deep learning model, a first region of interest for the first composite aperture radar image inputting the extracted first feature map and the second region of interest for the second synthesized aperture radar image to a pre-trained second deep learning model to output the extracted second feature map;
generating a final feature map by merging the first feature map including the first region of interest and the second feature map including the second region of interest output from the second deep learning model;
Including instructions for causing the processor to perform a method including extracting a final ROI for the first synthesized aperture radar image based on the final feature map
computer readable recording medium. As a computer program stored in a computer-readable recording medium,
The computer program, when executed by a processor,
inputting a first synthesized aperture radar (SAR) image to a first deep learning model trained in advance to remove noise;
A second synthetic aperture radar image and the first composite aperture radar image in which noise is removed from the first composite aperture radar image by the first deep learning model, a first region of interest for the first composite aperture radar image inputting the extracted first feature map and the second region of interest for the second synthesized aperture radar image to a pre-trained second deep learning model to output the extracted second feature map;
generating a final feature map by merging the first feature map including the first region of interest and the second feature map including the second region of interest output from the second deep learning model;
Including instructions for causing the processor to perform a method including extracting a final ROI for the first synthesized aperture radar image based on the final feature map
computer program.

Images