KR102659027B1 - Method for detecting moving objects, device and program using the same - Google Patents

Abstract

The moving object detection method according to an embodiment of the present invention selects sample sliding windows that correspond to some of a plurality of sliding windows for scanning the collected images according to a preset movement width. A step of scoring the features extracted from the collected images using selected sample sliding windows through a weak classifier. According to the scoring result through the weak classifier, a relatively high scoring result among the sample sliding windows It includes selecting at least one candidate sliding window, scoring features extracted from the collected images using a strong classifier using the selected at least one candidate sliding window, and detecting a moving object according to the scoring result. .

Description

Moving object detection method, moving object detection device and program using the same {METHOD FOR DETECTING MOVING OBJECTS, DEVICE AND PROGRAM USING THE SAME}

The present invention relates to a moving object detection method, a moving object detection device and program using the same, and more specifically, to scoring features extracted using selected sample sliding windows through a weak classifier and having a relatively high scoring result. It relates to a moving object detection method that selects a candidate sliding window, scores the features extracted using the candidate sliding window through a strong classifier, and detects a moving object according to the scoring result, and a moving object detection device and program using the same. .

Moving object detection technology is a technology that finds moving objects included in images from collected images.

Recently, technological development related to autonomous driving of vehicles is progressing rapidly, and moving object detection technology specifically detects moving objects from images collected using cameras installed in vehicles, and utilizes moving object detection information to drive the vehicle. It is widely used to assist or control vehicle driving.

The technical problem to be achieved by the present invention is to score the features extracted using selected sample sliding windows through a weak classifier, select a candidate sliding window with a relatively high scoring result, and extract the features extracted using the candidate sliding window. A moving object detection method capable of detecting a moving object according to the scoring result by scoring through a strong classifier, and a moving object detection device and program using the method are provided.

The moving object detection method according to an embodiment of the present invention selects sample sliding windows that correspond to some of a plurality of sliding windows for scanning the collected images according to a preset movement width. A step of scoring the features extracted from the collected images using selected sample sliding windows through a weak classifier. According to the scoring result through the weak classifier, a relatively high scoring result among the sample sliding windows It includes selecting at least one candidate sliding window, scoring features extracted from the collected images using a strong classifier using the selected at least one candidate sliding window, and detecting a moving object according to the scoring result. can do.

Depending on the embodiment, the moving object detection method may further include converting the collected image into a gray scale image before selecting the sample sliding windows.

According to an embodiment, the moving object detection method further includes, after converting the collected image into a gray scale image, setting a region of interest in the converted image, and selecting the sample sliding windows. may select the sample sliding windows from among sliding windows located within the region of interest.

Depending on the embodiment, the step of setting the region of interest may include setting a central region excluding the edge region of the collected image as the region of interest according to a preset ratio, and the preset ratio is set to the region of interest by the camera collecting the image. It can be set differently depending on the height.

Depending on the embodiment, the step of selecting the sample sliding windows includes selecting a plurality of sample sliding windows at different heights among the plurality of sliding windows, and the step of selecting the sample sliding windows includes selecting a plurality of sample sliding windows at the same height. The sample sliding windows may be selected so that they do not overlap each other, and sample sliding windows selected at different heights overlap each other.

Depending on the embodiment, the sample sliding windows include sample sliding windows corresponding to different moving object types, and the sample sliding windows corresponding to different moving object types have different horizontal and vertical ratios, and have different It can have a rectangular shape.

Depending on the embodiment, the moving object detection method may set processing priorities of sample sliding windows corresponding to the different moving object types differently.

Depending on the embodiment, the moving object detection method may further include training the strong classifier using features extracted from the collected images containing color data.

A moving object detection device according to an embodiment of the present invention uses a sample sliding method to select sample sliding windows corresponding to some of a plurality of sliding windows for scanning collected images according to a preset movement width. A window selector, a weak classifier for scoring features extracted from the collected image using selected sample sliding windows, and at least one having a relatively high scoring result among the sample sliding windows according to the scoring result through the weak classifier. It may include a candidate sliding window selector for selecting a candidate sliding window, a strong classifier for scoring features extracted from the collected images using at least one selected candidate sliding window, and a moving object detector for detecting a moving object according to the scoring result. You can.

A program stored in a medium for performing a method of identifying traffic signals in combination with a processor according to an embodiment of the present invention includes a plurality of sliding windows for scanning collected images according to a preset movement width. Selecting sample sliding windows that correspond to some of the (sliding windows), scoring features extracted from the collected images using the selected sample sliding windows through a weak classifier, according to the scoring result through the weak classifier. , selecting at least one candidate sliding window with a relatively high scoring result among the sample sliding windows, and using the selected at least one candidate sliding window to score the features extracted from the collected images through a strong classifier; , may include program code that performs the step of detecting a moving object according to the scoring result.

Methods and devices according to embodiments of the present invention score features extracted using selected sample sliding windows through a weak classifier, select a candidate sliding window with a relatively high scoring result, and use the candidate sliding window to By scoring the extracted features through a strong classifier and detecting moving objects according to the scoring results, the processing amount required to detect moving objects is reduced, enabling rapid detection of moving objects.

In order to more fully understand the drawings cited in the detailed description of the present invention, a brief description of each drawing is provided.
1 is a block diagram of a moving object detection device according to an embodiment of the present invention.
Figure 2 is a flowchart of a moving object detection method according to an embodiment of the present invention.
Figures 3 and 4 are diagrams showing the process of detecting a moving object according to the moving object detection method of Figure 2.

Since the technical idea of the present invention can be subject to various changes and can have various embodiments, specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the technical idea of the present invention to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the scope of the technical idea of the present invention.

In explaining the technical idea of the present invention, if it is determined that a detailed description of related known technology may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. In addition, numbers (eg, first, second, etc.) used in the description of this specification are merely identifiers to distinguish one component from another component.

In addition, in this specification, when a component is referred to as "connected" or "connected" to another component, the component may be directly connected or directly connected to the other component, but specifically Unless there is a contrary description, it should be understood that it may be connected or connected through another component in the middle.

In addition, terms such as “unit”, “unit”, “unit”, and “module” used in this specification refer to a unit that processes at least one function or operation, which refers to a processor, micro Processor (Micro Processer), Micro Controller, CPU (Central Processing Unit), GPU (Graphics Processing Unit), APU (Accelerate Processor Unit), DSP (Drive Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA It may be implemented as hardware or software, such as a Field Programmable Gate Array, or a combination of hardware and software, and may also be implemented in a form combined with a memory that stores data necessary for processing at least one function or operation. .

In addition, it is intended to be clear that the division of components in this specification is merely a division according to the main function each component is responsible for. That is, two or more components, which will be described below, may be combined into one component, or one component may be divided into two or more components for more detailed functions. In addition to the main functions it is responsible for, each of the components described below may additionally perform some or all of the functions handled by other components, and some of the main functions handled by each component may be performed by other components. Of course, it can also be carried out exclusively by .

1 is a block diagram of a moving object detection device according to an embodiment of the present invention.

Referring to FIG. 1 , the moving object detection device 100 may include an image capture unit 110, a memory 120, a communication interface 130, and a processor 140.

Depending on the embodiment, the moving object detection device 100 may be implemented with various devices (eg, a vehicle black box, etc.) that can acquire and process images around the vehicle.

According to another embodiment, the image capture unit 110 may be implemented separately from the moving object detection device 100, in which case the moving object detection device 100 may receive image data from the image capture unit 110. You can.

Depending on the embodiment, the moving object detection apparatus 100 or at least some of the components constituting the moving object detection apparatus 100 may be implemented as an embedded system.

The image capture unit 110 may capture and acquire images around the vehicle (eg, in front of the vehicle).

In this specification, “image” may mean a concept that includes both static images and dynamic images (i.e., video).

Depending on the embodiment, the image capture unit 110 includes an optical system including an image sensor (e.g., CCD (Charge Coupled Device) image sensor, CMOS (Complementary Metal-Oxide Semiconductor) image sensor, etc.) for capturing an image and a lens. It can be included.

The memory 120 may store data or programs necessary for the overall operation of the moving object detection device 100.

Memory 120 may store images acquired by image capture unit 110 or images that are being processed or processed by processor 140 .

The communication interface 130 may perform communication between the moving object detection device 100 and another external device and interface the communication.

For example, the communication interface 130 may transmit information about the moving object detected by the moving object detection device 100 to the vehicle's driving assistance control system or autonomous driving control system.

In this specification, “moving object” may include various objects that can move. For example, when detecting moving objects around a vehicle, the moving objects may include various types of moving objects that require detection due to the possibility of collision while the vehicle is driving, such as pedestrians, bicycles, motorcycles, or personal mobility.

Processor 140 may include a sample sliding window selector 142, a weak classifier 144, a candidate sliding window selector 146, and a strong classifier 148.

This indicates that each of the window selector 142, weak classifier 144, candidate sliding window selector 146, and strong classifier 148 can be separated functionally and logically, and each of the components must be a separate physical device. This does not mean that it is separated or written as separate code.

Depending on the embodiment, the processor 140 may be implemented in various forms, such as a Central Processing Unit (CPU), Neural Processing Unit (NPU), or Graphic Processing Unit (GPU).

Depending on the embodiment, the processor 140 may be implemented as a combination of a plurality of processors (eg, CPU and NPU) with different processing performances.

Depending on the embodiment, the processor 140 may further include an image pre-processing module. The image pre-processing module improves the image quality of acquired images by removing noise, gamma correction, color filter array interpolation, color correction, and color enhancement. Image processing to reduce image processing, such as image processing or color conversion of a color image to a gray scale image, can be performed.

The sample sliding window selector 142 can select sample sliding windows corresponding to some of a plurality of sliding windows for scanning the collected images according to a preset movement width (stride). there is.

Depending on the embodiment, the sample sliding window selector 142 may set a region of interest in an image converted to a gray scale image. In this case, the sample sliding window selector 142 can select sample sliding windows only from among sliding windows located within a set region of interest.

The detailed process of the sample sliding window selector 142 setting a region of interest and selecting sliding windows within the region of interest will be described later with reference to FIGS. 2 to 4.

A weak classifier (144) can broadly refer to a classifier that can perform classification at a relatively low throughput compared to a strong classifier (148). Depending on the embodiment, the weak classifier 144 may be implemented using an Artificial Neural Network (ANN) model.

The weak classifier 144 may classify and score features extracted from the collected images using sample sliding windows corresponding to a portion selected from a plurality of sliding windows by the sample sliding window selector 142.

The candidate sliding window selector 146 may select at least one candidate sliding window with a relatively high scoring result among the sample sliding windows according to the scoring result by the weak classifier 144.

Depending on the embodiment, the candidate sliding window selector 146 selects the top M sample sliding windows (where M is a natural number smaller than N) with high scoring results among N sample sliding windows (where N is a natural number of 2 or more). Candidates can be selected as sliding windows.

The strong classifier 148 may score features extracted from the collected images using at least one candidate sliding window selected by the candidate sliding window selector 146.

According to the scoring result, the strong classifier 148 may determine that a moving object has been detected when the scoring result exceeds the standard value. Depending on the embodiment, the strong classifier 148 may be implemented using a CNN (Convolutional Neural Network) model.

Depending on the embodiment, the strong classifier 148 may be trained using features extracted from an image including color data, that is, an image before conversion to gray scale.

The detailed process of detecting a moving object will be described later with reference to FIGS. 2 to 4.

Figure 2 is a flowchart of a moving object detection method according to an embodiment of the present invention. Figures 3 and 4 are diagrams showing the process of detecting a moving object according to the moving object detection method of Figure 2.

Referring to FIGS. 1 to 4 , the image capture unit 110 may collect images for an area corresponding to the coverage of the moving object detection device 100 (S210).

Depending on the embodiment, the image collected in step S210 may be a color image, and the moving object detection method according to the embodiment of the present invention may further include a pre-processing process of converting the collected image into a gray scale image.

The sample sliding window selector 142 may select sample sliding windows that correspond to some of the plurality of sliding windows for scanning the image collected in step S210 (or the image collected and converted to gray scale) (S220).

Referring to Figure 3, the sliding window is configured to scan the entire collected image while moving according to a preset movement width (stride(STR)). That is, if rapid scanning is not required, a process of extracting features and performing classification and scoring can be performed for all sliding windows (eg, SW-1 to SW-N) that move according to a preset movement width.

Referring to Figure 4 together, in the present invention, sample sliding windows (e.g., SSW-11 to SW-N) corresponding to some of a plurality of sliding windows (e.g., SW-1 to SW-N) that move according to a preset movement width. SSW-16) may be selected, and feature extraction and scoring of the image may be performed only for the selected sample sliding windows (eg, SSW-11 to SSW-16).

Depending on the embodiment, the sample sliding window selector 142 sets a region of interest (ROI) and selects the region of interest (ROI) set from among a plurality of sliding windows configured to scan the entire area of the collected image (IMG). Sample sliding windows (eg, SSW-11 to SSW-16, SSW-21 to SSW-26, SSW-31 to SSW-36) can be selected only from among the sliding windows located within.

Depending on the embodiment, the region of interest (ROI) may be set as the center area excluding the edge area of the image (IMG) collected according to a preset ratio.

Depending on the embodiment, a preset ratio for selecting a region of interest (ROI) from the entire image (IMG) may be set differently depending on the height of the camera that collects the image.

For example, if the height of the camera is relatively high, the region of interest (ROI) may be set to a relatively low position within the image (IMG), and conversely, if the height of the camera is relatively low, the region of interest (ROI) may be set to a relatively low position in the image (IMG). It can be set to a relatively high position within the (IMG).

For example, if the height of the camera is relatively high, the region of interest (ROI) may be set to a value with a relatively low ratio of the vertical length to the horizontal length (i.e., set to a short ratio of the vertical length to the horizontal length), and conversely, When the height of the camera is relatively low, the region of interest (ROI) may be set to a value where the ratio of the vertical length to the horizontal length is relatively high (i.e., the ratio of the vertical length to the horizontal length is long).

Depending on the embodiment, the region of interest (ROI) may be set among areas excluding areas where the probability of a moving object being present is significantly low according to the pattern of the image (IMG). For example, an area in the image (IMG) where a building is determined to exist may be excluded from the region of interest (ROI).

Depending on the embodiment, the sample sliding window selector 142 may select a plurality of sample sliding windows at different heights. For example, select a plurality of sample sliding windows (SSW-11 to SSW-16) at the first height, select a plurality of sample sliding windows (SSW-21 to SSW-26) at the second height, and select a plurality of sample sliding windows (SSW-21 to SSW-26) at the third height. Multiple sample sliding windows (SSW-31 to SSW-36) can be selected in height.

Depending on the embodiment, each of the sample sliding windows selected at different heights may be a sliding window for detecting moving objects located at different distances.

Depending on the embodiment, the sample sliding window selector 142 does not overlap between sample sliding windows selected at the same height (e.g., SSW-11 to SSW-16), but selects sample sliding windows selected at different heights (e.g., Between SSW-12 and SSW21), sample sliding windows can be selected to overlap each other. In this case, the sample sliding windows may be selected so that the overlap ratio between the sample sliding windows (eg, SSW-12 and SSW21) selected at different heights is a preset ratio. Depending on the embodiment, the sample sliding window selector 142 adjusts the preset ratio to increase in places where moving objects are detected frequently, and to increase the preset ratio in places where moving objects are detected frequently, according to the history of moving objects being detected. In some places, the overlap ratio can be adaptively adjusted to lower the preset ratio.

Depending on the embodiment, sample sliding windows selected at different heights may have the same horizontal and vertical ratio, but may have rectangular shapes of different sizes. At this time, the size of the sample sliding window selected at a relatively higher height may be configured to be relatively smaller.

Depending on the embodiment, the sample sliding window selector 142 may select sample sliding windows such that the central axes of sample sliding windows selected at different heights do not coincide with each other.

Depending on the embodiment, the rate at which the sample sliding window selector 142 selects sample sliding windows among all sliding windows is at least one of the performance of the processor 140 and the quality of service (QoS) required for moving object detection. It can be set based on .

According to another embodiment, the rate at which the sample sliding window selector 142 selects sample sliding windows among all sliding windows may be set based on a history of detection of a moving object. The sample sliding window selector 142 can adaptively adjust the selection ratio so that the ratio is set to increase in areas where moving objects are detected frequently, and the ratio is lowered in areas where moving objects are detected frequently. .

Returning to FIG. 2, the moving object detection apparatus 100 may score features extracted from the image collected using the sample sliding windows selected in step S220 through a weak classifier (S230).

Although not shown in FIG. 4, the sample sliding windows may include sample sliding windows corresponding to different moving object types (eg, pedestrian type, bicycle type, motorcycle type, or personal mobility type, etc.). In this case, sample sliding windows corresponding to different moving object types may have different ratios of horizontal length to vertical length and may have rectangular shapes of different sizes.

Depending on the embodiment, when the sample sliding windows include sample sliding windows corresponding to different moving object types, steps S240 to S260 are performed first for sample sliding windows of the high priority moving object type. You can.

Depending on the embodiment, the priority may be set according to the movement speed of the moving object for each moving object type. For example, a bicycle type with a fast moving speed may be given higher priority than a pedestrian type with a slow moving speed.

The moving object detection apparatus 100 may select at least one candidate sliding window with a relatively high scoring result, that is, a high scoring result, from among the sample sliding windows according to the scoring result of step S230 (S240).

Depending on the embodiment, the rate at which the moving object detection apparatus 100 selects a candidate sliding window from sample sliding windows may be set based on the rate at which sample sliding windows are selected from all sliding windows.

Depending on the embodiment, the rate of selecting a candidate sliding window from among sample sliding windows may be proportional to the rate of selecting sample sliding windows from all sliding windows. For example, when the ratio of selecting sample sliding windows among all sliding windows is relatively high, the ratio of selecting candidate sliding windows from sample sliding windows may be set to be high. For example, when the ratio of selecting sample sliding windows from all sliding windows is relatively low, the ratio of selecting candidate sliding windows from sample sliding windows may be set to be low.

According to another embodiment, the rate of selecting a candidate sliding window from among sample sliding windows may be inversely proportional to the rate of selecting sample sliding windows from all sliding windows. For example, when the ratio of selecting sample sliding windows from all sliding windows is relatively high, the ratio of selecting candidate sliding windows from sample sliding windows may be set to be low. For example, when the ratio of selecting sample sliding windows from all sliding windows is relatively low, the ratio of selecting candidate sliding windows from sample sliding windows may be set to be high.

The moving object detection apparatus 100 may extract features from the collected images using at least one candidate sliding window selected in step S240 and score the extracted features through a strong classifier (S250).

The moving object detection apparatus 100 may detect a moving object according to the scoring result of step S250 (S260).

Depending on the embodiment, the scoring value that serves as a standard when the moving object detection apparatus 100 detects a moving object may be set to vary depending on the ratio of selecting a candidate sliding window among sample sliding windows. The scoring value that serves as a standard when the moving object detection apparatus 100 detects a moving object may be proportional to the ratio of selecting a candidate sliding window from sample sliding windows. For example, when the rate of selecting a candidate sliding window among sample sliding windows was relatively high, the scoring value that serves as a standard when the moving object detection device 100 detects a moving object is set to be high (i.e., The standards for viewing can be set to be strengthened. For example, when the rate of selecting a candidate sliding window among sample sliding windows was relatively low, the scoring value that serves as a standard when the moving object detection device 100 detects a moving object is set to be low (i.e., The viewing standards can be set to be relaxed.

The moving object detection method according to an embodiment of the present invention may be implemented as a program code and stored in a memory, and the memory may be combined with a processor to perform the moving object detection method according to an embodiment of the present invention.

Above, the present invention has been described in detail with preferred embodiments, but the present invention is not limited to the above embodiments, and various modifications and changes can be made by those skilled in the art within the technical spirit and scope of the present invention. This is possible.

100: moving object detection device
110: image capture unit
120: memory
130: communication interface
140: processor

Claims (10)

Selecting some sample sliding windows located within a region of interest from among a plurality of sliding windows for scanning collected images;
Scoring features extracted from the collected images using the selected sample sliding windows through a weak classifier;
According to the scoring result through the weak classifier, selecting at least one candidate sliding window that has a relatively high scoring result and is located within the region of interest among the selected sample sliding windows; and
Scoring features extracted from the collected images using a strong classifier using the selected at least one candidate sliding window, and detecting a moving object according to the scoring result,
The method of detecting a moving object, wherein the selected at least one candidate sliding window is a sliding window of some of the plurality of sliding windows.
According to paragraph 1,
The moving object detection method is,
Before selecting the sample sliding windows,
A moving object detection method further comprising converting the collected image into a gray scale image.
According to paragraph 2,
The moving object detection method is,
After converting the collected image into a gray scale image, further comprising setting the region of interest in the converted image,
The step of selecting the sample sliding windows is,
A moving object detection method for selecting the sample sliding windows from among sliding windows located within the region of interest.
According to paragraph 3,
The step of setting the region of interest is,
Set the center area excluding the edge area of the collected image as the area of interest according to a preset ratio,
The preset ratio is,
A moving object detection method that is set differently depending on the height of the camera collecting the image.
According to paragraph 1,
The step of selecting the sample sliding windows is,
Selecting a plurality of sample sliding windows at different heights from the plurality of sliding windows,
The step of selecting the sample sliding windows is,
A moving object detection method, wherein the sample sliding windows are selected so that sample sliding windows selected at the same height do not overlap each other, and sample sliding windows selected at different heights overlap each other.
According to paragraph 1,
The sample sliding windows are,
Contains sample sliding windows corresponding to different moving object types,
A moving object detection method in which sample sliding windows corresponding to different moving object types have different horizontal and vertical ratios and have different sized rectangular shapes.
According to clause 6,
The moving object detection method is,
A moving object detection method that sets different processing priorities of sample sliding windows corresponding to the different moving object types.
According to paragraph 2,
The moving object detection method is,
A moving object detection method further comprising training the strong classifier using features extracted from the collected images containing color data.
a sample sliding window selector that selects some sample sliding windows located within a region of interest among a plurality of sliding windows for scanning collected images;
a weak classifier that scores features extracted from the collected images using the selected sample sliding windows;
a candidate sliding window selector that selects at least one candidate sliding window that has a relatively high scoring result and is located within the region of interest among the selected sample sliding windows, according to the scoring result through the weak classifier;
a strong classifier that scores features extracted from the collected images using the selected at least one candidate sliding window; and
It includes a moving object detector that detects a moving object according to the scoring result,
The moving object detection apparatus, wherein the selected at least one candidate sliding window is a sliding window of some of the plurality of sliding windows.
A program stored on a medium coupled with a processor to perform a method of detecting a moving object,
Selecting some sample sliding windows located within a region of interest from among a plurality of sliding windows for scanning collected images;
Scoring features extracted from the collected images using selected sample sliding windows through a weak classifier;
According to the scoring result through the weak classifier, selecting at least one candidate sliding window that has a relatively high scoring result and is located within the region of interest among the selected sample sliding windows; and
A program code for scoring features extracted from the collected images using the selected at least one candidate sliding window through a strong classifier and detecting a moving object according to the scoring result,
The program stored on a medium, wherein the selected at least one candidate sliding window is a sliding window of some of the plurality of sliding windows.