JP2021179971A - Method and apparatus for detecting small target, electronic device, computer readable storage medium, and computer program - Google Patents

Abstract

To disclose a method and an apparatus for detecting small targets, an electronic device, a computer readable storage medium, and a computer program, in embodiments of the present disclosure.SOLUTION: A specific embodiment of a method includes: acquiring an original image containing a small target; reducing an original image to a low-resolution image; identifying a candidate area containing the small target from the low-resolution image using a lightweight division network; and determining the position of the small target in the original image by setting an area of the original image corresponding to the candidate area as an interest area and executing a detection model trained in advance on the interest area. In the embodiment, a two-stage detection method is designed, first the interest area is searched for via the lightweight division network, and next the detection model is executed on the interest area. Thereby, an amount of calculation can be substantially saved.SELECTED DRAWING: Figure 2

Description

The embodiments of the present disclosure relate to the field of computer technology, specifically to methods and devices for detecting sub-targets, electronic devices, computer-readable storage media and computer programs.

Target detection is an important research direction in the field of autonomous driving. The main detection targets are classified into two types: stationary targets and motor targets. Stationary targets include traffic lights, traffic signs, roadways, obstacles, and exercise targets include cars, pedestrians, and non-automobiles. Here, the detection of traffic signs is a basic task that is important because it provides abundant and necessary navigation information while an unmanned driving vehicle is driving.

In applications such as AR navigation, it is important to detect traffic signs in the current section in real time and give appropriate hints to the user. In in-vehicle video, the size distribution range of traffic signs is wide and there are a large number of small targets (20 pixels or less), and detection of small targets requires not only the detection algorithm itself but also maintaining high resolution of the image. This is also a big test for the finite computational performance of in-vehicle machines.

To ensure the effectiveness of traffic sign recognition, many existing methods identify by training the input image using a YOLO model and predicting the classification to which the traffic sign belongs from the resulting predictions. Since the training network of the YOLO model is a CNN model that includes a total of 7 convolutional training layers of C1-C7 and 2 fully connected layers, recognition can be completed at a relatively high speed, but traffic signs are usually collected. It occupies only a small part of the original image and is reduced in size each time the feature map passes through the convolutional layer, so the method of adopting the existing YOLO model is to go through the multi-layered convolutional and then the smaller image. It is easy to lose its characteristics and affects the success rate of traffic sign recognition.

The embodiments of the present disclosure propose methods and devices, electronic devices, computer-readable storage media and computer programs for detecting sub-targets.

In a first aspect, the embodiments of the present disclosure are small from a low resolution image using an original image containing a small target, reducing the original image to a low resolution image, and using a lightweight split network. The position of the small target in the original image is determined by identifying the candidate area including the target, setting the area of the original image corresponding to the candidate area as the area of interest, and executing a pre-trained detection model on the area of interest. And how to detect small goals, including.

In some embodiments, the detection model is trained in the following way, i.e., determining the network structure of the initial detection model, initializing the network parameters of the initial detection model, and obtaining a training sample set. Here, the training sample contains a sample image and annotation information for characterizing the position of the small target in the sample image, and the training sample is enhanced and enhanced by at least one method of copying, multi-scale change, and editing. The sample image and annotation information in the training sample in the training sample set are used as the input and desired output of the initial detection model, the initial detection model is trained by the machine learning method, and the initial detection model obtained by the training is pre-trained. Confirmed as a detection model.

In some examples, the training sample is edited as follows, i.e., a small target is extracted from the sample image, the small target is scaled and / or rotated, and then randomly pasted elsewhere in the sample image. This will give you a new sample image.

In some embodiments, the method sets the pixel points within the rectangular frame for detecting the task to the positive sample and negatives the pixel points outside the rectangular frame when creating the training sample for the split network. To set the sample of, to expand the rectangular frame of the small target whose length is smaller than the predetermined number of pixels to the outside, and to set the pixels in the rectangular frame expanded to the outside to the positive sample. , Further including.

In some embodiments, the detection model is a deep neural network.

In some embodiments, attention modules are introduced after feature fusion of each predictor layer to learn appropriate weighting for features of different channels.

In the second aspect, the embodiments of the present disclosure include an acquisition unit arranged to acquire an original image including a small target, a reduction unit arranged to reduce the original image to a low resolution image, and a lightweight class. The first detection unit arranged to identify the candidate area including the small target from the low-resolution image and the area of the original image corresponding to the candidate area as the area of interest were pre-trained using the divided network of. It relates to a second detection unit arranged to determine the position of the small target in the original image by running the detection model on the region of interest, and a device for detecting the small target including.

In some embodiments, the apparatus according to the embodiments of the present disclosure further comprises a training unit arranged as follows, i.e., determining the network structure of the initial detection model and providing the network parameters of the initial detection model. Initialize and get the training sample set, where the training sample contains the sample image and annotation information to characterize the position of the small target in the sample image, and at least copy, multiscale change, edit the training sample. The sample images and annotation information in the training sample in the training sample enhanced by one method are used as the input and desired output of the initial detection model, respectively, and the initial detection model is trained by the machine learning method and obtained by training. The initial detection model obtained is determined as a pre-trained detection model.

In some embodiments, the training unit further extracts a small target from the sample image, scales and / or rotates the small target, and then randomly pastes it elsewhere in the sample image to create a new sample image. Is arranged to get.

In some embodiments, the first detection unit further sets the pixel points within the rectangular frame for detecting the task to the positive sample when creating the training sample for the split network, and outside the rectangular frame. Pixel points are set as a negative sample, a rectangular frame with a small target whose length is smaller than a predetermined number of pixels is expanded outward, and all the pixels in the rectangular frame expanded outward are set as positive samples. Arranged like this.

In some embodiments, the detection model is a deep neural network.

In some embodiments, attention modules are introduced after feature fusion of each predictor layer to learn appropriate weighting for features of different channels.

In a third aspect, the embodiments of the present disclosure include one or more processors and a storage device in which one or more programs are stored, and one or more programs are executed by one or more processors. When it comes to electronic devices for detecting sub-targets that enable one or more processors to implement the method according to any one of the first embodiments.

In a fourth aspect, an embodiment of the present disclosure is a computer-readable medium in which a computer program is stored, which realizes the method according to any one of the first aspects when the computer program is executed by a processor. Regarding storage media.

In a fifth aspect, the embodiments of the present disclosure relate to a computer program that, when executed by a processor, realizes the method according to any one of the first aspects.

The methods and devices for detecting sub-targets according to the embodiments of the present disclosure are mainly solved from three aspects of training method, model structure, and two-step detection, where the training method and model structure are mainly described. Used to improve the ability of the model to detect small targets, two-step detection is used to reduce the amount of computation in areas unrelated to the image, resulting in increased computational speed.

The present invention can provide real-time traffic sign detection algorithms for AR navigation projects, show better performance in detecting small targets, and improve the user's navigation experience.

To better clarify the other features, objectives and advantages of the present disclosure, reference is made to the detailed description of the non-limiting examples made with reference to the following drawings.
It is an exemplary system architecture diagram to which one embodiment of the present disclosure is applicable. It is a flowchart of one Embodiment of the method for detecting a small target by this disclosure. It is a schematic diagram which shows one application scene of the method for detecting a small target by this disclosure. It is a flowchart of another embodiment of the method for detecting a small target by this disclosure. It is a network configuration diagram of the detection model of the method for detecting a small target by this disclosure. It is a schematic block diagram of an Example of the apparatus for detecting a small target by this disclosure. It is a schematic block diagram of the computer system of the electronic device applied to realize the embodiment of this disclosure.

Hereinafter, the present disclosure will be described in more detail with reference to the drawings and examples. It can be understood that the specific examples described herein are for illustration purposes only and are not intended to limit the invention. For ease of explanation, the drawings show only the parts related to the related invention.

If there is no contradiction, the embodiments of the present disclosure and the features of the embodiments can be combined with each other. Hereinafter, the present disclosure will be described in detail according to examples with reference to the drawings.

FIG. 1 shows an exemplary system architecture 100 of an embodiment of a method for detecting a small target or an apparatus for detecting a small target to which the present invention may be applied.

As shown in FIG. 1, the system architecture 100 can include a vehicle 101 and a traffic sign 102.

The vehicle 101 may be an ordinary automobile or an unmanned driving vehicle. The controller 1011 and the network 1012 and the sensor 1013 may be attached to the vehicle 101. The network 1012 is used to provide a medium for communication links between the controller 1011 and the sensor 1013. The network 1012 can include various connection types such as, for example, wired, wireless communication links, or fiber optic cables.

The controller (also called an in-vehicle brain) 1011 is in charge of intelligent control of the vehicle 101. The controller 1011 may be an individually arranged controller such as a programmable logic controller (PLC), a one-chip computer, an industrial controller, or the like, or may have an input / output port and perform an operation. It may be a device composed of other electronic devices having a control function, or it may be a computer device in which an application of vehicle driving control is installed. The controller is equipped with a trained split network and detection model.

The sensor 1013 may be, for example, various sensors such as a camera, a gravity sensor, a wheel speed sensor, a temperature sensor, a humidity sensor, a laser radar, and a millimeter wave radar. In some cases, the vehicle 101 may be equipped with GNSS (Global Navigation Satellite System) equipment, SINS (Strap-down Inertial Navigation System, strap-down type inertial navigation system), or the like.

The vehicle 101 photographs the traffic sign 102 while traveling. Traffic signs in images, whether taken at long distances or at short distances, are both small targets.

The vehicle 101 determines the position of the traffic sign by causing the controller to identify the original image including the captured traffic sign. By performing OCR identification, the content of the traffic sign can also be identified. Then, the content of the traffic sign is output in the form of voice or characters.

The method for detecting a small target according to the embodiment of the present invention is generally executed by the controller 1011. Correspondingly, the device for detecting the small target is generally arranged in the controller 1011. Will be done.

It should be understood that the number of controllers, networks, and sensors in FIG. 1 is merely exemplary. It can be equipped with any number of controllers, networks, and sensors as needed.

Subsequently, with reference to FIG. 2, a flow 200 of an embodiment of the method for detecting a small target according to the present disclosure is shown. The method for detecting the sub-target includes the following steps.

In step 201, an original image including a small goal is acquired.

In this embodiment, the execution subject of the method for detecting a small target (for example, the controller shown in FIG. 1) can collect a front image via an in-vehicle camera, and the collected original image is small. The goal is included. The small target refers to an image of a target object having a number of vertical and horizontal pixels less than a predetermined value (for example, 20).

In step 202, the original image is reduced to a low resolution image.

In this embodiment, a low resolution image can be obtained by dividing the vertical and horizontal directions of the original image by 4 (or other multiples). Do not change the aspect ratio during reduction.

In step 203, a lightweight divided network is used to identify candidate regions containing small goals from low resolution images.

In this embodiment, during the detection of the first stage, it is only necessary to identify the approximate position where the target may exist, and an accurate outer frame is not required. Therefore, a lightweight divided network is used. The point larger than a certain threshold value in the final output heat map is regarded as the point where the existence of the target is suspicious. A split network such as U-Net can be adopted, and the backbone network adopts a shufflenet for weight reduction.

When creating a training sample for a divided network, the pixel points inside the rectangular frame for detecting the task are set as positive samples, and the pixel points outside the rectangular frame are set as negative samples. Since there is vertical and horizontal scaling, in order to guarantee the recall rate for small goals, when creating a training sample, the rectangular frame of the target whose vertical and horizontal dimensions are smaller than a predetermined value, for example, 20 pixels, is expanded outward by 1 time. After that, all the pixels in the rectangular frame spread outward are set as positive samples.

In step 204, the region of the original image corresponding to the candidate region is set as the region of interest, and the position of the small target in the original image is determined by executing the pre-trained detection model on the region of interest.

In this example, after filtering the noise points in the result output from the split network, a minimum circumscribed rectangle surrounding all the remaining pseudo target points is formed, which corresponds to the unscaled high resolution image of the rectangle. The area is the area of interest. After that, when the detection model is executed on the region of interest, it is necessary to process only a part of the high-resolution image, and as a result, the amount of calculation can be reduced.

As mentioned earlier, higher resolutions need to be maintained in order to better detect small targets, larger images double the amount of computation, and real-time processing is difficult to achieve in an in-vehicle machine environment. be. On the other hand, traffic signs occupy a small proportion on the image, most of which is the background area, the amount of calculation in the background area occupies a considerable proportion of the total amount of calculation, and it takes time to process the background area with high resolution. It takes and is meaningless. Therefore, the present invention employs a two-step detection method, first identifying the approximate location of a suspicious target on a low resolution image via a lightweight split network, and then a minimum including all suspicious targets. If the detection rate for a small target is guaranteed by finding the circumscribing rectangle of, and finally running the detection model on the high-resolution image block corresponding to the smallest circumscribing rectangle, the amount of calculation is reduced.

After the above two steps, the average complexity of the detection model is reduced to about 25% of the original complexity, and the combined complexity of the two models is about 45% of the original complexity. Become.

With reference to FIG. 4, FIG. 4 is a schematic diagram showing an application scene of the method for detecting a small target according to the present embodiment. In the application scene of FIG. 4, the vehicle collects a front image in real time while driving. After dividing the vertical and horizontal directions of the acquired original image by 4, the image is reduced to a low resolution image. Input a low-resolution image into a lightweight split network to identify candidate areas containing traffic signs. After that, the area of the original image corresponding to the candidate area is found as the area of interest from the original image. An image of the region of interest is extracted and a pre-trained detection model is entered to determine the specific position of the traffic sign in the original image, as shown by the dotted frame.

The method according to the above embodiment of the present disclosure reduces the amount of calculation and improves the discrimination speed and the accuracy rate by the secondary detection.

Further, with reference to FIG. 4, a flow 400 of another embodiment of the method for detecting a small target is shown. Flow 400 of the method for detecting the sub-target includes the following steps.

In step 401, the network structure of the initial detection model is determined, and the network parameters of the initial detection model are initialized.

In this embodiment, an electronic device (eg, the controller shown in FIG. 1) in which the method for detecting a small target is performed can train a detection model. You can also train the detection model on a third-party server and then install it on the vehicle's controller. The detection model is a neural network model and may be any existing neural network for target detection.

In some selectable embodiments of this embodiment, the detection model is a deep neural network, such as a YOLO network. YOLO (You Only Look None) is an object identification and positioning algorithm based on a deep neural network, and its greatest feature is that it operates at a high speed and can be used in a real-time system. Currently, YOLO has evolved into the V3 version (YOLO3), but the new version has also evolved steadily based on the original version. In YOLO3's original structural design, low-resolution feature maps and high-resolution feature maps are fused by upsampling. However, such fusion occurred only in the high resolution feature map and could not sufficiently fuse features of different scales.

In order to better integrate the features of different hierarchies, the invention first selects features 8x, 16x, and 32x the subsampling in the backbone network as basic features, followed by different sizes. In order to predict the target, the size of the predicted feature map is set to 8 times, 16 times, and 32 times the size of the subsampling of the image, respectively, and the features of each predicted feature map are all three basic features. It is from a layer and is unified to the same size by subsampling or upsampling before fusion. Taking the prediction layer 16 times the subsampling of the image as an example, the features are from each of the three basic feature layers, and the basics are 8 times the subsampling in order to unify them to the same size. 1x subsampling for a typical feature layer, 1x upsampling for a basic feature layer 32x the subsampling, then 16x the two feature layers and subsampling It fuses with the basic feature layer of.

By simply fusing features of different scales, the proportions of features are the same in the three prediction layers and cannot be weighted according to their different prediction goals. Therefore, after the feature fusion of each prediction layer, an attention module is further introduced to learn appropriate weighting for the features of different channels, so that each prediction layer depends on the characteristics of the prediction target that it needs. Therefore, the characteristics after fusion can be used with a heavy weight. The network structure is shown in FIG. Since the method of learning the parameters of the attention module is prior art, the description thereof is omitted here.

In the present disclosure, YOLO3 can be adopted as a detection network, and the design and allocation of anchors are very important in such anchor-based detection methods, and the number of anchors that can meet the sub-goals is small. , The learning of the small goal by the model becomes insufficient, and the small goal cannot be detected well. For this purpose, a dynamic anchor matching mechanism is adopted, and the IOU (reliability score) threshold value when the anchor and the ground truth are matched according to the magnitude of the ground truth (basic truth value) is adaptively selected. If the goal is small, lower the IOU threshold to allow more sub-goals to participate in the training, improving the performance of the model in detecting sub-goals. When creating a training sample, you already know the size of the target and select the appropriate IOU threshold according to the size of the target.

In step 402, a training sample set is obtained.

In this example, the training sample contains a sample image and annotation information for characterizing the position of the small target in the sample image.

In step 403, the training sample is enhanced by at least one method of copying, multiscale variation, and editing.

In this example, this is primarily a trick when the number of sub-goals in the training data is inadequate. By copying multiple images containing small goals in the data set, you can directly increase the number of small goals in the data, while extracting the small goals in the image, scaling and rotating them, and then randomly locating them elsewhere in the image. By pasting in, you can not only increase the number of small goals, but also introduce more changes and enrich the distribution of training data.

As an option, training images can be scaled to different scales and then trained to enrich the target scale changes in the original dataset and adapt the model to the task of detecting targets of different scales.

In step 404, the sample images and annotation information in the training sample in the enhanced training sample set are used as inputs and desired outputs of the initial detection model, and the initial detection model is trained by a machine learning method.

In this embodiment, the execution subject can obtain the position information of the small target in the sample image by inputting the sample image in the training sample in the training sample set into the initial detection model, and the annotation information in the training sample. Is the desired output of the initial detection model, and the initial detection model is trained by a machine learning method. Specifically, first, the difference between the obtained position information and the annotation information in the training sample can be calculated using the preset loss function. For example, the L2 norm can be used as the loss function to obtain the difference. It is possible to calculate the difference between the obtained position information and the annotation information in the training sample. After this, the network parameters of the initial detection model can be adjusted based on the differences obtained by the calculation, and if the preset training end conditions are met, the training ends. For example, the training end condition preset here includes at least one that the training time exceeds the preset time, the number of trainings exceeds the preset number, and the difference obtained by the calculation is smaller than the preset difference threshold. However, it is not limited to this.

Here, in various embodiments, the network parameters of the initial detection model can be adjusted based on the difference between the generated location information and the annotation information in the training sample. For example, a BP (Backpropagation) algorithm or an SGD (Stochastic Gradient Descent) algorithm can be used to adjust the network parameters of the initial detection model.

In step 405, the initial detection model obtained by training is determined as a pre-trained detection model.

In this embodiment, the execution subject of the training step can determine the initial detection model obtained by training in step 404 as a pre-trained detection model.

Further, with reference to FIG. 6, as an embodiment of the method shown in each of the above figures, the present invention provides an embodiment of an apparatus for detecting a small target, and an embodiment of the apparatus is shown in FIG. Corresponding to the method embodiment shown, the apparatus can be specifically applied to various electronic devices.

As shown in FIG. 6, the apparatus 600 for detecting the small target according to the present embodiment includes an acquisition unit 601, a reduction unit 602, a first detection unit 603, and a second detection unit 604. .. Here, the acquisition unit 601 is arranged to acquire the original image including the small target, the reduction unit 602 is arranged to reduce the original image to a low resolution image, and the first detection unit 603 is lightweight. A class division network is used to identify candidate regions containing small targets from low-resolution images, and the second detection unit 604 pre-trains the region of the original image corresponding to the candidate region as the region of interest. By executing the detected detection model on the region of interest, it is arranged so as to determine the position of the small target in the original image.

In the present embodiment, the corresponding implementation of FIG. 2 relates to the specific processing of the acquisition unit 601, the reduction unit 602, the first detection unit 603, and the second detection unit 604 of the device 600 for detecting the small target. You can refer to step 201, step 202, step 203, and step 204 in the example.

In some selectable embodiments of this embodiment, the apparatus 600 further comprises a training unit (not shown) arranged as follows, i.e., determining and initializing the network structure of the initial detection model. Initialize the network parameters of the detection model and obtain the training sample set, where the training sample contains the sample image and annotation information to characterize the position of the small target in the sample image, copy the training sample, multi. Enhanced by at least one method of scaling and editing, the sample images and annotation information in the training sample in the enhanced training sample set are the input and desired output of the initial detection model, respectively, and the initial detection model is machine-learned. Train and establish the initial detection model obtained by training as a pre-trained detection model.

In some selectable embodiments of this example, the training unit further extracts a small target from the sample image, scales and / or rotates the small target, and then randomly pastes it elsewhere in the sample image. By attaching, it is arranged so as to obtain a new sample image.

In some selectable embodiments of this embodiment, the first detection unit further positively sets the pixel points in the rectangular frame for detecting the task when creating a training sample of the split network. Set it as a sample, set the pixel points outside the rectangular frame to the negative sample, expand the small target rectangular frame whose length is smaller than the predetermined number of pixels to the outside, and all the pixels inside the rectangular frame expanded to the outside. Arranged to set to a positive sample.

In some selectable embodiments of this embodiment, the detection model is a deep neural network.

In some selectable embodiments of this embodiment, attention modules are introduced after feature fusion of each predictor layer to learn appropriate weighting for features of different channels.

Hereinafter, with reference to FIG. 7, a schematic configuration diagram of an electronic device (for example, the controller shown in FIG. 1) 700 applied to realize the embodiment of the present disclosure is shown. The controller shown in FIG. 7 is merely an example and does not limit the functionality and scope of use of the embodiments of the present disclosure.

As shown in FIG. 7, the electronic device 700 performs various appropriate operations and processes according to a program stored in the read-only memory (ROM) 702 or a program loaded into the random access memory (RAM) 703 from the storage unit 708. A processing unit capable of performing the above (for example, a central processing unit, a graphics processor, etc.) 701 can be included. The RAM 703 stores various programs and data necessary for operating the electronic device 700. The processing apparatus 701, ROM 702, and RAM 703 are connected to each other by a bus 704. The input / output (I / O) interface 705 is also connected to the bus 704.

Typically, the I / O interface 705 includes an input device 706 including, for example, a touch screen, touch panel, keyboard, mouse, camera, microphone, accelerometer, gyro, etc., and, for example, a liquid crystal display (LCD), speaker, vibrator, etc. An output device 707, a storage device 708 including, for example, a magnetic tape and a hard disk, and a communication device 709 are connected. The communication device 709 can allow the electronic device 700 to perform wireless or wired communication with other devices for exchanging data. FIG. 7 shows an electronic device 700 with various devices, but it should be understood that it is not required to implement or equip all the devices shown. Alternatively, more or less equipment can be implemented or equipped. Each block shown in FIG. 7 may represent one device, or may represent a plurality of devices as needed.

In particular, according to the embodiments of the present disclosure, the process described with reference to the flowchart above can be implemented as a program of computer software. For example, the embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program including program code for performing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 709, installed from the storage device 708, or installed from the ROM 702. When this computer program is executed by the processor 701, it performs the above functions limited by the methods of the embodiments of the present disclosure. The computer-readable medium described in the examples of the present disclosure may be a computer-readable signal medium, a computer-readable storage medium, or any combination of both of the above. The computer-readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above, but is not limited thereto. More specific examples of computer-readable storage media include electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory. (EPROM or flash memory), optical fiber, compact disk read-only memory (CD-ROM), optical storage, magnetic storage, or any combination of the above, but not limited to these. In the embodiments of the present disclosure, the computer-readable storage medium can be any tangible medium containing or storing a program, which program is used by, or combined with, a command execution system, device or device. Can be used. In the embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagating within the baseband or as part of a carrier wave carrying a computer-readable program code. Such propagated data signals can adopt a variety of formats, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination described above. The computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium, which is used by, or in combination with, a command execution system, device or device. Can send, propagate or transmit a program for. The program code contained on a computer-readable medium can be transmitted on any suitable medium, including wires, fiber optic cables, RF (radio frequency), etc., or any suitable combination described above. Not exclusively.

The computer-readable medium may be included in the electronic device, or may exist individually and may not be incorporated in the electronic device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device acquires an original image including a small target and the original image. Is reduced to a low resolution image, the area of the original image corresponding to the candidate area is set as the area of interest, and the position of the small target in the original image is determined by executing the pre-trained detection model on the area of interest.

Computer program code for performing the operations of the embodiments of the present disclosure can be written in one or more programming languages or combinations thereof, and the programming languages include object-oriented programming languages such as Java, Smalltalk, and C ++. Also includes traditional procedural programming languages such as the "C" language or similar programming languages. The program code is entirely on the user's computer, partly on the user's computer, as a single software package, partly on the user's computer and partly on the remote computer, or entirely on the remote computer or It can be run on the server. When it comes to remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or wide area network (WAN), or to an external computer. Can be connected (eg, connect via the Internet using an internet service provider).

The flowcharts and block diagrams in the figure show the feasible architectures, functions, and operations of the systems, methods, and computer program products according to the various embodiments of the present disclosure. In this regard, each block of the flowchart or block diagram can represent one module, programming segment, or part of code, and that module, programming segment, or part of code implements a given logic function. Includes one or more actionable directives to do so. Also note that in some alternative embodiments, the functions shown in the blocks may occur in a different order than shown in the drawings. For example, two blocks shown in succession may actually be executed in parallel, and they may sometimes be executed in reverse order depending on the function concerned. Each block of block diagrams and / or flowcharts, and a combination of blocks of block diagrams and / or flowcharts, may be implemented by a system of dedicated hardware that performs specific functions or operations, or dedicated hardware and a computer. It should also be noted that this may be achieved by a combination of directives.

The units according to the embodiments of the present disclosure may be realized by software or hardware. The described unit may be installed in a processor and may be described as, for example, "a processor including an acquisition unit, a reduction unit, a first detection unit, and a second detection unit". .. Here, the names of these units are not limited to the units themselves in some cases, and for example, the acquisition unit may be described as "a unit that receives a user's web page browsing request".

The above description is merely a description of the preferred embodiments of the present disclosure and the principles of the applied technique. The scope of the invention according to the embodiment of the present disclosure is not limited to the technical proposal obtained by combining the above technical features, and at the same time, the above technical features or the above technical features shall not deviate from the concept of the above invention. Those skilled in the art should understand that other technical proposals obtained by any combination of equivalent technical features should be included. For example, there may be a technical proposal in which the above features are interchangeably formed with technical features having functions similar to those disclosed in the present disclosure (but not limited to this).

Claims (15)

To get the original image including the small goal,
Reducing the original image to a low resolution image and
Using a lightweight split network to identify candidate regions containing the small target from the low resolution image,
By using the region of the original image corresponding to the candidate region as the region of interest and executing a pre-trained detection model on the region of interest, the position of the small target in the original image can be determined.
A method for detecting small goals, including. The detection model is trained in the following way, i.e.
The network structure of the initial detection model is fixed, and the network parameters of the initial detection model are initialized.
Obtain a training sample set, where the training sample contains the sample image and annotation information to characterize the position of the small goal in the sample image.
The training sample was enhanced by at least one method of copying, multiscale variation, and editing.
The sample images and annotation information in the training sample in the enhanced training sample set are used as inputs and desired outputs of the initial detection model, respectively, and the initial detection model is trained by a machine learning method.
The initial detection model obtained by training is determined as the pre-trained detection model.
The method according to claim 1. Edit the training sample as follows, ie
Extract small goals from sample images and
Obtain a new sample image by scaling and / or rotating the small target and then randomly pasting it elsewhere in the sample image.
The method according to claim 2. When creating the training sample of the divided network, the pixel points inside the rectangular frame for detecting the task are set to the positive sample, and the pixel points outside the rectangular frame are set to the negative sample.
Expanding the rectangular frame of the small target whose length is smaller than the predetermined number of pixels to the outside,
Setting all the pixels in the rectangular frame spread out to be positive samples,
The method according to claim 1, further comprising. The method according to any one of claims 1 to 3, wherein the detection model is a deep neural network. The method of claim 5, wherein an attention module is introduced after feature fusion of each predictor layer to learn appropriate weighting for features of different channels. With the acquisition unit arranged to acquire the original image including the small target,
A reduction unit arranged to reduce the original image to a low resolution image, and
A first detection unit arranged to identify a candidate region containing the small target from the low resolution image using a lightweight divided network.
A region of the original image corresponding to the candidate region is set as the region of interest, and a pre-trained detection model is executed on the region of interest to determine the position of the small target in the original image. 2 detection units and
A device for detecting small targets, including. It further includes training units arranged as follows, i.e.
The network structure of the initial detection model is fixed, and the network parameters of the initial detection model are initialized.
Obtain a training sample set, where the training sample contains the sample image and annotation information to characterize the position of the small goal in the sample image.
The training sample was enhanced by at least one method of copying, multiscale variation, and editing.
The sample images and annotation information in the training sample in the enhanced training sample set are used as inputs and desired outputs of the initial detection model, respectively, and the initial detection model is trained by a machine learning method.
The initial detection model obtained by training is determined as the pre-trained detection model.
The device according to claim 7. The training unit further
Extract small goals from sample images and
Obtain a new sample image by scaling and / or rotating the small target and then randomly pasting it elsewhere in the sample image.
Arranged like
The device according to claim 8. The first detection unit further
When creating the training sample of the divided network, the pixel points inside the rectangular frame for detecting the task are set as positive samples, and the pixel points outside the rectangular frame are set as negative samples.
Expand the rectangular frame of the small target whose length is smaller than the predetermined number of pixels to the outside,
Set all the pixels in the rectangular frame spread outward as a positive sample,
Arranged like
The device according to claim 7. The apparatus according to any one of claims 7 to 10, wherein the detection model is a deep neural network. 11. The apparatus of claim 11, wherein an attention module is introduced after feature fusion of each predictor layer to learn appropriate weighting for features of different channels. With one or more processors
A storage device in which one or more programs are stored, and
Including
When the one or more programs are executed by the one or more processors, the method according to any one of claims 1 to 6 is realized in the one or more processors.
An electronic device for detecting small targets. A computer-readable medium in which a computer program is stored.
The method according to any one of claims 1 to 6 is realized when the computer program is executed by a processor.
Computer-readable medium. It ’s a computer program,
A computer program that, when executed by a processor, implements the method of any one of claims 1-6.

Images