KR102523886B1 - A method and a device for detecting small target - Google Patents

Abstract

An embodiment of the present invention discloses a small target detection method and apparatus. A method of detecting a small target according to an aspect includes acquiring an original image including a small target; reducing the original image to a low-resolution image; identifying candidate regions containing small targets in the low-resolution image using a lightweight segmentation network; A region of the original image corresponding to the candidate region is set as a region of interest, and a pre-trained detection model is executed in the region of interest to determine a position of the small target in the original image. The above embodiment designs a two-step detection method, and first searches for a region of interest through a lightweight segmentation network and then executes a detection model in the region of interest, thereby greatly reducing the amount of computation.

Description

Small target detection method and apparatus {A METHOD AND A DEVICE FOR DETECTING SMALL TARGET}

Embodiments of the present invention relate to the field of computer technology, and specifically to a method and apparatus for detecting a small target.

Target detection is an important research direction in the field of autonomous driving. The main detection targets are divided into two types: stationary targets and moving targets. Stationary targets are traffic lights, traffic signs, lanes, obstacles, etc., and moving targets are vehicles, pedestrians, non-motorized vehicles, etc. Among them, traffic sign detection provides rich and essential navigation information to driving unmanned vehicles, which is a very important fundamental task.

In applications such as AR navigation, it is very important to detect traffic signs on the current road section in real time and prompt users accordingly. In vehicle-mounted video, traffic signs have a wide size distribution and a large number of small targets (20 pixels or less). It is a great test of the limited computing power of vehicle machines.

In order to ensure validity of traffic sign identification, most of the existing means train an input image using a YOLO model, and complete identification by predicting a classification to which a traffic sign belongs through an obtained prediction value. The training network of the YOLO model is a CNN model that includes a total of 7 convolutional training layers from C1 to C7 and 2 fully connected layers, so identification can be completed relatively quickly, but traffic signs are generally collected from original images. Since the size of the feature map continues to decrease every time it passes through one convolutional layer, when multiple convolutions are performed using the existing YOLO model method, relatively small image features are lost. Ease affects the success rate of traffic sign identification.

Embodiments of the present invention provide a method and apparatus for detecting a small target.

A method of detecting a small target according to an aspect includes acquiring an original image including a small target; reducing the original image to a low-resolution image; identifying candidate regions containing small targets in the low-resolution image using a lightweight segmentation network; and determining a position of the small target in the original image by determining a region of the original image corresponding to the candidate region as a region of interest and executing a pre-trained detection model in the region of interest.

In some embodiments, the detection model includes: determining a network structure of the initial detection model and initializing network parameters of the initial detection model; obtaining a training sample set, the training sample including a sample image and labeling information for indicating a position of a small target in the sample image; augmenting the training samples in at least one of replication, multi-scale transformation, and editing; take the sample images and labeling information in the training samples in the set of training samples after augmentation as inputs and expected outputs of the initial detection model, respectively, and train the initial detection model using a machine learning method; and a method of determining an initial detection model acquired through training as a pre-trained detection model.

In some embodiments, the training sample cuts a small target in the sample image; After zooming and/or rotating the small target, it is edited through a method of acquiring a new sample image by attaching it randomly to another location of the sample image.

In some embodiments, the method further includes, when constructing training samples of the segmentation network, setting pixel points within a rectangle box originally used for task detection as positive samples, and setting pixel points outside the rectangle boxes as negative samples; expanding outwardly a rectangular box of a small target whose length and width are smaller than a predetermined number of pixels; and setting all the pixels within the outwardly expanded rectangular box as positive samples.

In some embodiments, the detection model is a deep neural network.

In some embodiments, the detection model fuses each prediction layer feature and then introduces an attention module to learn one suitable weight for feature of different channels.

An apparatus for detecting a small target according to another aspect includes an acquiring unit acquiring an original image including a small target; a reduction unit that reduces the original image to a low-resolution image; a first detection unit for identifying a candidate region containing a small target in the low-resolution image using a lightweight segmentation network; and a second detection unit configured to set a region of the original image corresponding to the candidate region as a region of interest and execute a pre-trained detection model in the region of interest to determine the position of the small target in the original image.

In some embodiments, the apparatus provided in the embodiments of the present invention further includes a training unit, wherein the training unit determines a network structure of an initial detection model and initializes network parameters of the initial detection model; obtaining a training sample set, the training sample including a sample image and labeling information for indicating a position of a small target in the sample image; augmenting the training samples in at least one of replication, multi-scale transformation, and editing; The sample images and labeling information in the training samples in the set of training samples after augmentation are respectively used as inputs and expected outputs of the initial detection model, and the initial detection model is trained using a machine learning device; An initial detection model obtained by training is determined as a pre-trained detection model.

In some embodiments, the training unit cuts a small target in the sample image; After zooming and/or rotating the small target, a new sample image is obtained by randomly attaching the small target to a different position of the sample image.

In some embodiments, the first detection unit sets the pixel points within the rectangle box originally used for detecting the task as positive samples, and sets the pixel points outside the rectangle box as negative samples when preparing training samples of the segmentation network; expand outward the rectangle box of the small target, the length and width of which are less than a predetermined number of pixels; It is configured to set all pixels within the spherical box that extends outward as positive samples.

In some embodiments, the detection model is a deep neural network.

In some embodiments, the detection model fuses each prediction layer feature and then introduces an attention module to learn one suitable weight for feature of different channels.

An electronic device according to another aspect includes at least one processor; When the storage device includes a storage device in which at least one program is stored, and the at least one program is executed by at least one processor, the at least one processor implements the above method.

In a computer readable storage medium in which a computer program according to another aspect is stored, the above method is implemented when the program is executed by a processor.

A computer program stored in a computer readable storage medium according to another aspect, the above-described method is implemented when the computer program is executed by a processor.

The small target detection method and apparatus provided in the embodiments of the present invention are mainly solved from three aspects: a training method, a model structure, and a two-step detection. It is used to improve performance, and two-step detection is used to reduce the amount of computation in regions unrelated to the image, thereby improving computation speed.

The present invention can provide a real-time traffic sign detection algorithm for an AR navigation project, and can improve a user's navigation experience because it shows better performance in small target detection.

Other features, objects and advantages of the present invention will become more apparent upon reading and reference to the detailed description of the non-limiting embodiments shown in the accompanying drawings below.
1 is an exemplary system architecture to which one embodiment of the present invention may be applied.
2 is a flowchart of an embodiment of a small target detection method according to the present invention.
3 is a schematic diagram of one application scene of the small target detection method according to the present invention.
4 is a flowchart of another embodiment of a small target detection method according to the present invention.
5 is a network structure diagram of a detection model of a small target detection method according to the present invention.
6 is a structural schematic diagram of an embodiment of a small target detection device according to the present invention.
7 is a structural schematic diagram of a computer system of an electronic device suitable for implementing an embodiment of the present invention.

The terms used in the present embodiments have been selected from general terms that are currently widely used as much as possible while considering the functions in the present embodiments, but these may vary depending on the intention of a person skilled in the art or a precedent, the emergence of new technologies, etc. there is. In addition, in a specific case, there are also terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the relevant part. Therefore, the term used in the present embodiments should be defined based on the meaning of the term and the general content of the present embodiment, not a simple name of the term.

Since the present embodiments can have various changes and various forms, some embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the present embodiments to a specific disclosure, and should be understood to include all changes, equivalents, or substitutes included in the spirit and scope of the present embodiments. Terms used in this specification are only used for description of the embodiments, and are not intended to limit the embodiments.

Terms used in the present embodiments have the same meaning as commonly understood by a person of ordinary skill in the art to which the present embodiments belong, unless otherwise defined. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in the present embodiments, in an ideal or excessively formal meaning. should not be interpreted.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The detailed description of the present invention which follows refers to the accompanying drawings which illustrate, by way of illustration, specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable any person skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention are different from each other but are not necessarily mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented from one embodiment to another without departing from the spirit and scope of the present invention. It should also be understood that the location or arrangement of individual components within each embodiment may be changed without departing from the spirit and scope of the present invention. Therefore, the detailed description to be described later is not performed in a limiting sense, and the scope of the present invention should be taken as encompassing the scope claimed by the claims and all scopes equivalent thereto. Like reference numbers in the drawings indicate the same or similar elements throughout the various aspects.

Meanwhile, technical features individually described in one drawing in this specification may be implemented individually or simultaneously.

In this specification, “~ unit” may be a hardware component such as a processor or a circuit, and/or a software component executed by a hardware component such as a processor.

The present invention will be described in more detail with reference to the accompanying drawings and examples below. It will be understood that the specific embodiments described herein are only for interpreting the related invention, and the present invention is not limited thereto. In addition, it should be noted that only parts related to the invention are shown in the drawings for convenience of explanation.

It should be noted that the embodiments of the present invention and the features of the embodiments may be combined with each other unless contradictory. The present invention will be described in detail with reference to the accompanying drawings below and reference to examples.

1 is an exemplary system architecture to which one embodiment of the present invention may be applied.

As shown in FIG. 1 , system architecture 100 may include vehicle 101 and traffic sign 102 .

The vehicle 101 may be a general power engine vehicle or may be an unmanned vehicle. A controller 1011, a network 1012, and a sensor 1013 may be installed in the vehicle 101. Network 1012 is to provide a medium of communication link between controller 1011 and sensor 1013. Network 1012 may include a variety of connection types, such as wired, wireless communication links, or fiber optic cables.

The controller (or, also referred to as a vehicle brain) 1011 is responsible for smart control of the vehicle 101 . The controller 1011 may be a separately installed controller such as a programmable logic controller (PLC), a single needle microcomputer, an industrial control computer, etc., and is an electronic device having an input/output port and having operation and control functions. It may be another device configured, or a computer device in which a vehicle driving control type application is installed. The trained split network and detection model are installed in the controller.

The sensor 1013 may be various types of sensors, such as a camera, gravity sensor, wheel speed sensor, temperature sensor, humidity sensor, laser radar, millimeter wave radar, and the like. In some circumstances, a global navigation satellite system (GNSS) device and a strap-down inertial navigation system (SINS) may be installed in the vehicle 101 .

While the vehicle 101 is driving, the traffic sign 102 is photographed. Regardless of whether it is an image obtained by shooting from a relatively long distance or an image obtained by shooting from a short distance, all traffic signs in the image are small targets.

The vehicle 101 determines the position of the traffic sign by identifying the captured original image including the traffic sign by the controller. In addition, OCR identification can be performed to identify the contents of traffic signs. Next, the content of the traffic sign is output in the form of voice or text.

The small target detection method provided in the embodiments of the present invention is generally performed by the controller 1011, and correspondingly, the small target detection device is generally installed in the controller 1011.

The number of controllers 1011, networks 1012, and sensors 1013 in FIG. 1 is merely exemplary, and the vehicle 101 may include any number of controllers, networks, and sensors according to actual needs.

2 is a flowchart of an embodiment of a small target detection method according to the present invention.

The small target detection method includes the following steps.

In step 201, an original image including a small target is acquired.

In this embodiment, a subject performing the small target detection method (for example, the controller 1011 shown in FIG. 1 ) may collect front images through a vehicle camera, and the collected original image includes the small target. . The small target refers to an image of a target object in which the number of pixels of length and width is smaller than predetermined values (eg, 20).

In step 202, the original image is reduced to a lower resolution image.

In this embodiment, a low-resolution image can be obtained by dividing by 4 (or other multiples) in the length and width directions of the original photo. During the reduction process, the ratio of length and width does not change.

In step 203, a lightweight segmentation network is used to identify candidate regions containing small targets in the low-resolution image.

In this embodiment, since only the approximate location where the target can be present must be located during the first step detection, and an accurate outer border is not required, a lightweight segmentation network is used, and the final output heat map A point larger than a certain threshold value in is regarded as a suspicious point where the target exists. A split network similar to U-Net can be used, and as a backbone network, shufflenet is used for light weight.

When creating a training sample for the segmentation network, the pixel points inside the rectangle box originally used for task detection are set as positive samples, and the pixel points outside the rectangle box are set as negative samples. Since there is a zoom in the length and width directions, in order to guarantee a recall rate in a small target, when making a training sample, the length and width are smaller than predetermined values, for example 20 After extending the rectangular box of the target, which is a pixel, by a factor of 1, all pixels within the expanded rectangular box are set as positive samples.

In step 204, a region of the original image corresponding to the candidate region is set as a region of interest, and a pre-trained detection model is executed in the region of interest to determine the position of the small target in the original image.

In this embodiment, after filtering noise points in the results output by the segmentation network, a minimum circumscribed sphere is formed enclosing all remaining suspect target points, and an area corresponding to the sphere is obtained in a high-resolution image that has not been zoomed. as an area of interest. After that, a detection model is run in the region of interest. In this way, only a part of the high-resolution picture needs to be processed, thereby reducing the amount of calculation.

As described above, in order to better detect a small target, the high resolution of the photo must be maintained, and if the photo is large, the amount of calculation increases, so real-time processing cannot be performed in a vehicle machine environment. On the other hand, traffic signs occupy a very small percentage of the picture and most of them are background areas, so the background area calculation amount occupies a large part of the total calculation amount, and processing the background areas at high resolution is time consuming and meaningless. Therefore, the present invention uses a two-step detection method, first, through one lightweight segmentation network, the approximate position of the suspect target is determined in the low-resolution photo, and then the minimum circumscribed sphere containing all the suspect targets is obtained, Finally, by executing the detection model on the high-resolution image block corresponding to the minimum circumscribed rectangle, the amount of calculation is reduced under the condition that the detection rate of the small target is guaranteed.

After the two-step process, the average computational amount of the detection model is reduced to about 25% of the original computational amount, and the average computational amount of the sum of the two models is reduced to about 45% of the original computational amount.

3 is a schematic diagram of one application scene of the small target detection method according to the present invention.

In the application scene of FIG. 3 , the vehicle collects front images in real time while driving. Divide the length and width of the acquired original image by 4 to reduce it to a low-resolution image. A low-resolution image is fed into a lightweight segmentation network to identify candidate regions containing traffic signs. Next, a region of the original image corresponding to the candidate region is found in the original image and set as a region of interest. The image of the region of interest is cut out, and a pre-trained detection model inputs it to determine the specific location of the traffic sign in the original image, which is indicated by a dotted line box.

The method provided in the above embodiment of the present invention undergoes secondary detection, thereby reducing the amount of calculation and improving identification speed and accuracy.

4 is a flowchart of another embodiment of a small target detection method according to the present invention.

The process of the small target detection method includes the following steps.

In step 401, a network structure of an initial detection model is determined, and network parameters of the initial detection model are initialized.

In this embodiment, an electronic device (eg, the controller 1011 shown in FIG. 1 ) performing the small target detection method may train a detection model. Alternatively, the detection model may be trained by a third server and then installed in the controller of the vehicle. The detection model is a neural network model, and may be any existing neural network for target detection.

In some optional implementations of this embodiment, the detection model is a deep neural network, such as a YOLO-based network. YOLO (You Only Look Once) is an object identification and location determination algorithm based on a deep neural network, and the greatest feature is that the execution speed is very fast, and it can be used in a real-time system. Currently, YOLO has already developed to the v3 version (YOLO3), but the new version is also evolved by continuing to develop based on the previous version. In the original structural design of YOLO3, low-resolution feature maps are fused with high-resolution feature maps through upsampling. However, such fusion occurs only in high-resolution feature maps, and features of different scales cannot be sufficiently fused.

In order to better fuse the features of different layers, the present invention first selects 8-fold, 16-fold, and 32-fold downsampled features in the backbone network as basic features, and then, to predict targets of different sizes, the predicted feature map The size of is set to the size of downsampling the picture to 8x, 16x, and 32x, respectively. The features of each predictive feature map are all obtained from the three basic feature layers, unified into the same size through downsampling or upsampling, and then fused. Taking a picture as an example of a prediction layer downsampled by a factor of 16, its features are obtained from each of the three basic feature layers, and 1x downsampled for the basic feature layer downsampled by a factor of 8 to unify them into a unified size. Sampling is performed, 1x upsampling is performed on the basic feature layer downsampled by a factor of 32, and then the two feature layers are merged with the basic feature layer downsampled by a factor of 16.

If only features of different scales are simply fused, the proportions of features in these three prediction layers are all the same, so it cannot be used intensively according to different prediction targets. Therefore, after the features of each prediction layer are fused, the attention module is introduced again to learn one suitable weight for the features of different channels. Features after fusion can be used intensively. The network structure is as shown in FIG. 5 . Since the learning method of the parameters of the attention module is a prior art, it is not further described here.

The present invention can use YOLO3 as a detection network, design and allocation of anchors are very important in this detection method based on anchor points, and the number of anchors that can be matched to small targets is very small. can directly lead to a lack of learning on small targets by To this end, a dynamic anchor matching mechanism is used to adaptively select an IOU (confidence score) threshold when matching anchors and ground truth according to the size of the ground truth (default true value), and when the target is relatively small, the IOU threshold By adjusting the value low, more small targets can participate in training, and the model can improve its performance in small target detection. When creating a training sample, the size of the target is already known, so an appropriate IOU threshold is selected according to the size of the target.

At step 402, a set of training samples is obtained.

In this embodiment, the training sample includes a sample image and labeling information for displaying a position of a small target among the sample images.

In step 403, the training samples are augmented in at least one of replication, multi-scale transformation, and editing.

In this embodiment, this is a strategy mainly used for the case where the quantity of small targets in the training data is insufficient. On the one hand, a plurality of photos containing small targets in the data set are duplicated, and the number of small targets in the data is directly increased; , and attach it randomly to another location on the image again. In this way, not only can the quantity of small targets be increased, but also more variations can be introduced, enriching the allocation of training data.

Optionally, by zooming the training picture to different scales and then training, it is possible to enrich the target scale change in the existing data set, and make the model adapt to the detection task of the different scale target.

In step 404, sample images and labeling information in training samples in the training sample set after augmentation are used as inputs and expected outputs of the initial detection model, respectively, and the initial detection model is trained using a machine learning method.

In this embodiment, the performing entity inputs a sample image of a training sample in a training sample set to an initial detection model, obtains position information of a small target in the sample image, and sets labeling information in the training sample as an expected output of the initial detection model. and train an initial detection model using a machine learning method. Specifically, first, the difference between the position information obtained by calculating by a preset loss function and the labeling information in the training sample is used, for example, the position information obtained by using the L2 function as a loss function and the position information obtained by calculating the training sample. It uses the difference between the labeling information in Next, based on the calculated difference, network parameters of the initial detection model are adjusted, and training may be terminated when a preset training end condition is satisfied. For example, the preset training end condition here is that the training time exceeds the preset time length, the number of training times exceeds the preset number of times, and the calculated difference is less than the preset difference threshold. It may include one of, but is not limited thereto.

Here, various implementations may be used, and network parameters of the initial detection model may be adjusted according to the difference between the generated location information and the labeling information in the training sample. For example, network parameters of an initial detection model may be adjusted using a Back Propagation (BP) algorithm or a Stochastic Gradient Descent (SGD) algorithm.

In step 405, an initial detection model obtained by training is determined as a pre-trained detection model.

In this embodiment, the subject performing the training step may determine the initial detection model obtained by training in step 404 as a pre-trained detection model.

6 is a structural schematic diagram of an embodiment of a small target detection device according to the present invention.

Referring to FIG. 6 , an embodiment of the device 600 corresponds to the embodiment of the method shown in FIG. 2 , and the device 600 may be specifically applied to various electronic devices.

As shown in FIG. 6 , the small target detection apparatus 600 of this embodiment includes an acquisition unit 601, a reduction unit 602, a first detection unit 603 and a second detection unit 604. . Here, the acquiring unit 601 acquires an original image containing a small target; the reduction unit 602 reduces the original image to a low-resolution image; The first detection unit 603 uses a lightweight segmentation network to identify candidate regions containing small targets in the low-resolution image; The second detection unit 604 sets the region of the original image corresponding to the candidate region as the region of interest, and executes a pre-trained detection model in the region of interest to determine the position of the small target in the original image.

In this embodiment, the specific processing of the acquisition unit 601, the reduction unit 602, the first detection unit 603, and the second detection unit 604 of the small target detection device 600 corresponds to the corresponding implementation in FIG. Step 201, step 202, step 203, step 204 in the example can be referred to.

In some optional implementation forms of this embodiment, the device 600 determines the network structure of the initial detection model, initializes network parameters of the initial detection model; A training sample set is obtained, the training sample including a sample image and labeling information for indicating a position of a small target in the sample image; augmenting the training sample through at least one of replication, multi-scale transformation, and editing; The sample images and labeling information in the training samples in the set of training samples after augmentation are respectively used as inputs and expected outputs of the initial detection model, and the initial detection model is trained using a machine learning device; It further includes a training unit (not shown) for determining an initial detection model obtained by training as a pre-trained detection model.

In some optional implementations of this embodiment, the training unit (not shown) also cuts a small target in the sample image; After zooming and/or rotating the small target, a new sample image is obtained by randomly attaching the small target to a different position of the sample image.

In some optional implementation forms of this embodiment, the first detection unit 603, when creating training samples of the segmentation network, sets the pixel points within the rectangle box originally used for task detection as positive samples, and the pixels outside the rectangle box. set points to audio samples; expand outward the rectangle box of the small target, the length and width of which are less than a predetermined number of pixels; Set all pixels within the outwardly expanded rectangle box as positive samples.

In some optional implementations of this embodiment, the detection model is a deep neural network.

In some optional implementations of this embodiment, the detection model introduces an attention module after merging features of each prediction layer to learn a suitable weight for features of different channels.

7 is a structural schematic diagram of a computer system of an electronic device suitable for implementing an embodiment of the present invention.

The electronic device 700 shown in FIG. 7 (eg, the controller 1011 of FIG. 1 ) is only an example, and is not intended to be any limitation on the function or scope of use of an embodiment of the present invention.

As shown in FIG. 7 , the electronic device 700 varies according to a program stored in a read-only memory (ROM) 702 or a program loaded into a random access memory (RAM) 703 from a storage device 708 and and a processing unit (eg, central processing unit, graphics processing unit, etc.) 701 capable of performing appropriate operations and processing. The RAM 703 also stores various programs and data necessary for operating the electronic device 700 . The processing unit 701 , ROM 702 and RAM 703 are connected to each other via a bus 704 . An input/output (I/O) interface 705 is also connected to bus 704.

Input devices 706 generally include, for example, touch screens, touch pads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, and the like; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, and the like; storage device 708 including, for example, magnetic tape, hard drives, etc.; and communication device 709 can be coupled to I/O interface 705 . The communication device 709 may allow the electronic device 700 to exchange data by communicating with another device wirelessly or wired. Although FIG. 7 shows an electronic device 700 with various devices, it should be understood that it is not necessary to implement or have all of the illustrated devices. The electronic device 700 may alternatively implement or include more or fewer devices. Each block shown in FIG. 7 may represent one device, or may represent multiple devices according to demand.

In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart may be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product including a computer program embodied in a computer readable medium, and the computer program includes program code for performing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network via communication device 709 and/or installed from ROM 702 . When the computer program is executed by the processing unit 701, the functions defined in the method of the present invention are performed. It should be noted that computer readable media described herein may be computer readable signal media or computer readable media or any combination of both. A computer readable medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or element, or any combination thereof. More specific examples of computer readable media are electrically connected by one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination thereof. In the present invention, a computer readable medium can be any type of medium that can contain or store an instruction execution system, device or element, or a program that can be used in combination therewith. In the present invention, a computer readable signal medium may include a data signal propagated as part of a baseband or carrier bearing computer readable program code. These propagated data signals can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer readable signal medium can also be any computer readable medium other than a computer readable medium that can transmit, propagate, or transport a program for use in or in conjunction with an instruction execution system, device, or element. The program code embodied in a computer readable medium may be transmitted by any suitable medium, including but not limited to electrical wire, fiber optic cable, radio frequency (RF), or the like, or any suitable combination of the foregoing.

The computer readable medium may be included in the electronic device or may exist separately without being assembled in the electronic device. acquiring an original image including a small target by storing at least one program in the computer readable medium and, when the at least one program is executed by the electronic device; reducing the original image to a low-resolution image; identifying candidate regions containing small targets in the low-resolution image using a lightweight segmentation network; and setting a region of the original image corresponding to the candidate region as a region of interest and executing a pre-trained detection model in the region of interest to determine the position of the small target in the original image.

Computer program code for performing the operations of the present invention may be written in one or more programming languages, or a combination thereof. The programming languages include object oriented programming languages including Java, Smalltalk, C++ and conventional procedural programming languages including the "C" language or similar programming languages. The program code may run entirely on the user's computer, partly on the user's computer, as a standalone software package, partly on the user's computer and partly on a remote computer, or entirely on a remote computer or server. can be executed In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a LAN or WAN, or to an external computer (for example, through the Internet using an Internet Service Provider).

The flow diagrams and block diagrams in the drawings illustrate the implementable architectures, functions and operations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block of a flowchart or block diagram may represent a module, program segment, or portion of code that includes one or more executable instructions for implementing a specified logical function. It should be noted that in some alternative implementations, the functions indicated in the blocks may also be implemented in a different order than shown in the figures. For example, two blocks presented sequentially may actually be executed in parallel, and sometimes in reverse order depending on the function involved. In addition, each block in the block diagram and/or flowchart and the combination of blocks in the block diagram and/or flowchart may be implemented in a dedicated hardware-based system that performs a designated function or operation, or may be implemented by combining dedicated hardware and computer instructions. It should be noted that there are

Units described in the embodiments of the present invention may be implemented by software or hardware. The described unit may also be installed in a processor, eg a processor comprising an acquisition unit, a reduction unit, a first detection unit and a second detection unit. Here, the names of these units are not limited to the units themselves in some cases, and for example, the acquisition unit may be described as “a unit that receives a user webpage browsing request”.

The above description is merely a description of the preferred embodiment of the present invention and the principles of applied technology. Those skilled in the art to which the present invention pertains, the scope of the present invention mentioned in the present invention is not limited to the technical solutions according to the specific combination of the above technical features, and at the same time, without departing from the spirit of the present invention, the technical features or Other technical solutions formed by arbitrarily combining the equivalent features, for example, including technical solutions formed by exchanging the above features with technical features having similar functions disclosed in the present invention (but not limited thereto). should understand that

Claims (15)

As a small target detection method,
obtaining an original image including a small target;
reducing the original image to a low-resolution image;
identifying a candidate region containing the small target in the low-resolution image using a lightweight segmentation network; and
determining a location of the small target in the original image by determining a region of the original image corresponding to the candidate region as a region of interest and executing a pre-trained detection model in the region of interest;
The training sample of the split network is
Fabrication by outwardly extending a rectangle box of a small target whose length and width are smaller than a predetermined number of pixels, setting pixels within the expanded rectangle box as positive samples, and setting pixels outside the expanded rectangle box as negative samples. how to be According to claim 1,
The detection model,
determining a network structure of an initial detection model and initializing network parameters of the initial detection model;
obtaining a training sample set, the training sample including a sample image and labeling information for indicating a position of a small target in the sample image;
augmenting the training sample through at least one of replication, multi-scale transformation, and editing;
taking sample images and labeling information of training samples in the set of training samples after augmentation as inputs and expected outputs of the initial detection model, respectively, and training the initial detection model using a machine learning method; and
A method of training in such a way that the initial detection model obtained by training is determined as the pre-trained detection model. According to claim 2,
The training sample is
cutting a small target in the sample image; and
A method of editing by performing at least one of zoom and rotation operations on the small target, and then obtaining a new sample image by attaching it randomly to another location of the sample image. delete According to any one of claims 1 to 3,
The detection model,
A method that is a deep neural network. According to claim 5,
The detection model,
A method of learning one suitable weight for features of different channels by merging the features of each prediction layer and then introducing the attention module. As a small target detection device,
an acquiring unit acquiring an original image including a small target;
a reduction unit that reduces the original image to a low-resolution image;
a first detection unit for identifying a candidate region containing the small target in the low-resolution image using a lightweight segmentation network; and
a second detection unit configured to set a region of the original image corresponding to the candidate region as a region of interest, and execute a pre-trained detection model in the region of interest to determine the position of the small target in the original image; ,
The training sample of the split network is
Fabrication by outwardly extending a rectangle box of a small target whose length and width are smaller than a predetermined number of pixels, setting pixels within the expanded rectangle box as positive samples, and setting pixels outside the expanded rectangle box as negative samples. device to be. According to claim 7,
The apparatus further comprises a training unit, wherein the training unit comprises:
determining a network structure of an initial detection model and initializing network parameters of the initial detection model;
obtaining a training sample set, the training sample including a sample image and labeling information for indicating a position of a small target in the sample image;
augmenting the training sample through at least one of replication, multi-scale transformation, and editing;
take sample images and labeling information of training samples in the set of training samples after augmentation as inputs and expected outputs of the initial detection model, respectively, and train the initial detection model using a machine learning device;
An apparatus for determining the initial detection model obtained by training as the pre-trained detection model. According to claim 8,
The training unit,
cutting a small target in the sample image;
Apparatus for obtaining a new sample image by randomly attaching the small target to another location of the sample image after performing at least one of zoom and rotation operations on the small target. delete According to any one of claims 7 to 9,
The detection model,
A device that is a deep neural network. According to claim 11,
The detection model,
A device that learns one suitable weight for features of different channels by merging the features of each prediction layer and then introducing the attention module. As an electronic device,
at least one processor;
A storage device in which at least one program is stored;
An electronic device in which the at least one processor implements the method according to claim 1 when the at least one program is executed by the at least one processor. A computer-readable storage medium in which a computer program is stored,
A computer-readable storage medium on which the method according to claim 1 is implemented when the program is executed by a processor. A computer program stored on a computer readable storage medium,
A computer program in which the method according to claim 1 is implemented when the computer program is executed by a processor.

Images