JP2021128759A - Method and device for detecting objects - Google Patents

Abstract

To provide a method and device for detecting objects.SOLUTION: An object detection method comprises: performing matching between a radar detection result and a video detection result; generating grid cells in a radar coordinate system based on the radar detection result; generating a border frame in an image coordinate system based on the grid cells; and performing object detection based on the border frame.SELECTED DRAWING: Figure 2

Description

The present invention relates to the art of object detection, and more particularly to object detection methods and devices.

The purpose of object detection is to understand what objects are and where they are in an image (or frame in a video). A fast and accurate target object detection algorithm can cause a computer to drive a car and an auxiliary device to transmit real-time scene information to the user and various other applications. be able to.

In recent years, according to a large number of related technologies, good performance object detection has already been developed. However, it still faces challenges in two aspects: the length of processing time and the accuracy of the object's position on the image. Systems such as DPM (Deformable Part Models) use a window sliding method, in which the classifier operates at a position that is evenly distributed over the image. However, in terms of performance, due to the design of the sliding steps, such methods reduce efficiency and accuracy. Modern methods such as R-CNN (Region-Convolutional Neural Networks) use region definition methods, first generating potential borders in an image and then classifying on these defined frames. Run the vessel. After classification, post-processing is used to simplify the borders to eliminate duplicate detection and re-evaluate (score) the borders based on other objects in the scene. However, the execution of such complex pipelines is slow and difficult to optimize.

The inventor discovered the following. That is, the latest methods such as YOULO (you look only once) can completely solve the slow processing problem by redefining object detection as one regression problem. It uses a single convolutional network to simultaneously predict multiple boundary frames and the class probabilities of these frames.

As shown in FIG. 1, YOLO first divides the input image into S × S grids. For each grid cell, it can predict the B border frame and the confidence score of these frames. Each boundary frame consists of five predicted quantities, namely x, y, w, h and reliability. The confidence score includes the confidence of the C-class probability of the B boundary frame. Therefore, in total, these predicted quantities are coded as S × S × (B × 5 + C) tensors. Network complexity is determined by S ^2. In consideration of processing time, B is also restricted, because it is because a multiple of S ^2. Therefore, one of the problems of YOLO is that the accuracy of the boundary frame is relatively low. However, in reality, the vast majority of grids contain no content, and predicting these empty grid cells is a waste of processing time.

To solve at least one of the above problems or other similar problems, the embodiments of the present invention provide object detection methods and devices.

According to the first aspect of the embodiment of the present invention, an object detection method is provided, in which the method is described.
Matching radar detection results with video detection results;
Generate grid cells based on the radar detection result in the radar coordinate system;
It includes generating a border based on the grid cells in the image coordinate system; and performing object detection based on the border.

According to the second aspect of the embodiment of the present invention, an object detection device is provided, in which the device is
A matching unit that matches radar detection results with video detection results;
The first generation unit that generates grid cells based on the radar detection result in the radar coordinate system;
It includes a second generation unit that generates a boundary frame based on the grid cell in the image coordinate system; and a detection unit that performs object detection based on the boundary frame.

According to a third aspect of an embodiment of the present invention, an image processing device is provided, of which the image processing device includes a processor and a storage device, the storage device stores a computer program, and the processor is said to be said. It is configured to run a computer program to implement the object detection method described above.

One of the beneficial effects of the embodiments of the present invention is as follows, i.e., the present invention indicates the possible position of the border frame on the video frame based on the radar detection result, so that the complete video Blind searches, subdivisions and predictions on the frame can be avoided. As a result, the processing time of the object detection method (for example, DPM, R-CNN, etc.) can be significantly reduced, or the size and position accuracy of the boundary frame can be improved under the same processing time conditions. ..

It is a figure which shows the process which performs the object detection by adopting the YOLO method. It is a figure which shows the object detection method in the Example of this invention. It is another figure which shows the object detection method in the Example of this invention. It is a figure which shows the object detection apparatus in the Example of this invention. It is a figure which shows the image processing apparatus in the Example of this invention.

Hereinafter, preferred examples for carrying out the present invention will be described in detail with reference to the attached drawings.

<Example of the first aspect>
An embodiment of the present invention provides an object detection method. FIG. 2 is a diagram showing an object detection method according to an embodiment of the present invention. As shown in FIG. 2, the method includes the following operations (steps).

Operation 201: Matching the radar detection result and the video detection result;
Operation 202: Generate a grid cell based on the radar detection result in the radar coordinate system;
Operation 203: Generate a border frame based on the grid cells in the image coordinate system;
Operation 204: Object detection is performed based on the boundary frame.

The present invention avoids blind searches, subdivisions and predictions on a complete video frame by indicating possible positions on the border frame video frame based on radar detection results, thereby object detection. The processing time of the method (eg, DPM, R-CNN, etc.) can be significantly reduced, or the size and position accuracy of the border frame can be improved under the same processing time conditions.

In the embodiment of the present invention, the radar and the camera simultaneously acquire the sensing data asynchronously, the radar captures the original data and outputs the detection result (referred to as the radar detection result), and the camera captures and detects the video frame. Output the result (called video detection result).

In an embodiment of the invention, the radar detection result includes a point cloud, the velocity of each point in the point cloud, and other information. Based on the points in the point cloud, clustering or tracking results can be obtained using an algorithm, for example, dBscan (Density-Based Spatial Crusting of Applications with Noise) or EKF (Exted Kalman Filter). The clustering or tracking result represents the location of the object, which is the coordinates of the object with respect to the radar in a (planar) Cartesian coordinate system and is represented as (x, y).

In operation 201, the radar detection result and the video detection result are matched, that is, the radar frame and the video frame are matched to find the radar frame and the video frame at the same time, or the difference in time stamps is different. It finds radar and video frames that are less than a certain threshold, which calibrates these coordinates (x, y) in the radar coordinate system to their position in the video frame, which is (u, v). It is expressed as. By matching the radar frame with the video frame, the point cloud in the radar frame can be mapped to the video frame, that is, the position of each point on the video frame in the radar detection result can be determined. In the present invention, the related technology can be referred to without limiting the specific matching method.

In operation 202, if the point cloud can be clustered as a cluster, i.e., if some points of the point cloud can be clustered as a cluster in a radar frame, then the cluster can be clustered in width (W _c ⁱ ) and height (W c i). _{Can be described by Hc} ⁱ ), of which "i" indicates the i-th cluster, in which case the points in the cluster are located within one rectangular area, which is the grid cell. You can see it.

For example, in operation 202, in some embodiments, generating grid cells based on radar detection results first clusters the radar detection results to obtain at least one cluster, and then. Generating grid cells based on the coordinates or signal strength of points in the cluster. Hereinafter, an example will be described.

In some examples, for each cluster, the coordinates of the center point of the cluster are the center of the grid cell, and the abscissa of the rightmost point in the cluster (that is, the maximum value of the x coordinate in the cluster). abscissa of the point of the leftmost (i.e., the minimum value of the x coordinate in a cluster) the difference between the width W _c ⁱ of the grid cell and the ordinate of the point of the uppermost end in the cluster (i.e., the The difference between the maximum value of the y-coordinate in the cluster) and the abscissa of the lowest point (that is, the minimum value of the y-coordinate in the cluster) is defined as the height H _c ⁱ of the grid cell, whereby each You can get the grid cells for the cluster.

In some embodiments, for each cluster, the coordinates of the point with the highest signal strength in the cluster are the center of the grid cell, and the preset width and height are the grid cell width W _c ⁱ and height H _c ^i. Or, the product of the preset width and height and the weight is set to the width W _c ⁱ and the height H _c ⁱ of the network unit, whereby a grid cell corresponding to each cluster can be obtained. Here, the preset width and height may be set based on experience, and the above-mentioned signal strength may be determined based on the distance between the maximum point and the radar. Not limited to. It also presets fixed widths and heights and determines grid cell weights based on other policies or factors (eg, the distance between the point with the highest signal strength and the radar mentioned above). , The product of the weight and the above-mentioned preset fixed width and height may be the width W _c ⁱ and the height H _c ⁱ of the grid cell.

In the above examples, different clusters have different grid sizes. The present invention is not limited to this, and since it is easy to addless to video frames, one unified grid size may be determined in some embodiments. The unified grid size is determined by factors such as processing efficiency and detection accuracy. For different objects, the unified grid size calculation method is also different.

For example, in order to improve object detection efficiency, the width (W _g ) of the unified grid size is equal to max (W _c ⁱ ), and the height (H _g ) of the unified grid size is max (H _c ⁱ ). It may be equal.

Further, for example, in order to obtain a detection result with higher accuracy, the width (W _g ) of the unified grid size is equal to min (W _c ⁱ ), and the height (H _g ) of the unified grid size is min (H g). It may be equal to H _c ^i).

Further, the present invention is not limited to the clustering method, and may be the above-mentioned dBscan algorithm or another clustering algorithm, but detailed description thereof will be omitted here.

In operation 202, when it is difficult to cluster the point cloud as a cluster, that is, in the radar frame, when the cluster cannot be clustered from the points in the point cloud, each point in the radar detection result (that is, the radar frame). The coordinates of each point in the above point cloud) may be set as the center of the grid cell, and then the size of the grid cell (that is, width W _c ⁱ and height H _c ⁱ ) may be determined by any method.

For example, based on the distance between each point and the radar, the size of the grid cell centered on the coordinates of the point is determined. In some embodiments, the size of the grid cell corresponding to the point near the radar may be relatively large, and the size of the grid cell corresponding to the point far from the radar may be relatively small. The specific s size may be determined based on experience, but the present invention is not limited thereto.

Further, for example, based on the signal strength of each point, the size of the grid cell whose center is the coordinate of the point is determined. Generally speaking, a strong signal means a large dielectric constant and a large reflection area of the measurement medium because the strength of the signal is affected by a plurality of factors. Therefore, in some embodiments, the size of the corresponding grid cell may be relatively large for points with high signal strength, and conversely, the size of the corresponding grid cell may be increased for points with low signal strength. It may be relatively small. The specific size may be determined based on experience, and the present invention is not limited thereto.

In the embodiment of the present invention, the distance between each point in the point cloud of the radar frame and the radar may be obtained by calculating from the radar detection result, but the detailed description thereof will be omitted here. Further, in the embodiment of the present invention, the calculation method, the detection method, or the determination method of the signal strength at each point is not limited, and the signal strength may be obtained by any conventional method, but detailed description thereof will be omitted here.

Further, similar to the method when the point cloud can be clustered as a cluster described above, even when the cluster cannot be clustered from the point cloud, one unified grid size may be determined, and the method is the point described above. Since it is similar to the method when the cloud can be clustered as a cluster, the detailed description thereof is omitted here.

In the embodiment of the present invention, the grid cell is acquired based on the radar detection result (point in the point cloud), the area not including the point is ignored, and thus the calibration position (grid cell) from the radar frame is video. It points to a possible position of the border on the frame, which can make predictions based on the grid cells.

In operation 203, in some embodiments, each grid cell is covered by sliding left or right or up and down along the u-axis and v-axis of the center point of the grid cell on a video frame. At least one frame can be generated as a boundary frame corresponding to the grid cell.

In the present invention, the unit of sliding is not limited. For example, by sliding left and right or up and down along the u-axis and v-axis of the center point of the grid cell with the size (width and height) of the grid cell as the unit. A boundary frame covering the above-mentioned grid cell may be obtained, and for example, by using a preset unit size as a unit and sliding left or right or up and down along the u-axis and v-axis of the center point of the grid cell. , The boundary frame covering the above-mentioned grid cells may be acquired.

In some embodiments, the position of the boundary frame may be represented by the coordinates of the upper left corner of the boundary frame, and the size of the boundary frame may be represented by the width and height of the boundary frame. The present invention is not limited to this, and the position and size of the boundary frame may be indicated by other quantities, and specifically, it is determined by means for obtaining the boundary frame or other factors.

In some examples, the reliability of the boundary frame may be determined, for example, the reliability of the boundary frame may be determined based on the number of points in the grid cell, and in some examples, the boundary frame may be determined. The more points in the grid cell that generates the frame, the greater the possibility that the object exists, and the higher the reliability of the boundary frame corresponding to the grid cell. Further, for example, the reliability of the boundary frame is determined based on the signal strength of points in the grid cell, and in some embodiments, the signal strength in the grid cell that generates the boundary frame is the maximum. The stronger the signal strength of a point, the more likely it is that an object will exist, and the greater the reliability of the border frame corresponding to the grid cell.

In the embodiments of the present invention, the speed of points in the point cloud on the radar frame may further provide prior information (preliminary information) for the classification of objects in the video frame, i.e., in the object detection method of the invention. Further, the possible classification of objects (referred to as object type) may be determined based on the speed of points in the radar detection result, whereby each point in the boundary frame and radar detection result obtained in the above operation 203. Object detection can be detected based on the possible classifications corresponding to. For example, if the speed of a point is close to zero, it means that the point represents a static object in an intelligent road intersection scene, such as a fence, fire plug, road sign, etc., otherwise the point is a moving object, eg, a road sign. , Means to represent a car, motorcycle, bicycle or pedestrian. The object detection efficiency can be improved by the prior test information.

In operation 204, an object detection result can be obtained by adopting an algorithm known to those skilled in the art based on the above-mentioned boundary frame or based on the above-mentioned boundary frame and the above-mentioned prior test information. The present invention is not limited to the algorithm here, and may be any artificial intelligence (AI) processing method, for example, the YOLO method.

According to the method of the embodiment of the present invention, blind search, subdivision and prediction on the complete video frame can be avoided by indicating the possible position of the boundary frame on the video frame based on the radar detection result. This makes it possible to significantly reduce the processing time of the object detection method (for example, DPM, R-CNN, etc.) or improve the accuracy of the size and position of the boundary frame under the same processing time condition.

Although the method of the embodiment of the present invention has been described above with reference to FIG. 2, the present invention is not limited to this, and the execution order between each operation is appropriately adjusted when carrying out concretely. For example, the execution order of the operations 201 and 202 may be exchanged, and some operations may be increased or decreased. That is, a person skilled in the art may appropriately change the above-mentioned contents based on the above-mentioned description of FIG.

FIG. 3 is another diagram showing the object detection method in the embodiment of the present invention, and shows the entire process from the acquisition of the radar original data and the camera video frame to the output of the object detection result. As shown in FIG. 3, the method includes the following operations (steps).

Operation 301: Obtain radar original data by radar;
Operation 302: Signals the radar original data to obtain the position of the object with respect to the radar and the velocity of the object;
Operation 303: Generate a grid cell based on the point cloud on the radar frame;
Operation 304: Acquire a video frame by the camera;
Operation 305: Calibrate radar and video frames and map the above grid cells to video frames;
Operation 306: Synchronize the time with the radar frame and the video frame;
Operation 307: Indicate possible positions of the border frame based on the grid cells;
Operation 308: Limit classification types using point velocities in point clouds on radar frames;
Operation 309: Performs AI processing and obtains an object detection result.

Although the method of the embodiment of the present invention has been described above with reference to FIG. 3, the present invention is not limited to this, and the execution order between each operation is appropriately adjusted when carrying out concretely. For example, the operation 301 and the operation 304 may be performed asynchronously at the same time, or the execution order of the operation 303 and the operation 305-306 may be exchanged. For example, the operation 305 and the operation 306 may be executed first. Then, the operation 303 may be executed. Further, some operations may be increased or decreased, and for example, operation 308 may be omitted. That is, a person skilled in the art may appropriately change the above-mentioned contents based on the above-mentioned description of FIG.

Further, in the example of FIG. 3, the operation 303 corresponds to the operation 202 of FIG. 2, and the operation 307 corresponds to the operation 203 of FIG. Since the operation 202 and the operation 203 have been described in detail in the previous embodiment, the detailed description thereof will be omitted here.

Although only each step or process according to the present invention has been described above, the present invention is not limited thereto. The method may further include other steps or processes, with reference to prior art for the specific content of these steps or processes.

According to the method of the embodiment of the present invention, as described above, the processing time of the object detection method can be significantly reduced, or the size and position accuracy of the boundary frame can be improved under the same processing time condition. ..

<Example of the second aspect>
An embodiment of the present invention provides an object detection device. Since the principle by which the device solves the problem is similar to that of the first aspect embodiment, the first aspect embodiment can be referred to for its specific implementation, and detailed description thereof will be omitted here. ..

FIG. 4 is a diagram showing an object detection device according to an embodiment of the present invention. As shown in FIG. 4, the object detection device 400 according to the embodiment of the present invention includes a matching unit 401, a first generation unit 402, and a second generation. Includes unit 403 and detection unit 404. The matching unit 401 matches the radar detection result with the video detection result; the first generation unit 402 generates a grid cell based on the radar detection result in the radar coordinate system; and the second generation unit 403 is in the image coordinate system. A boundary frame is generated based on the grid cell; the detection unit 404 is used to perform object detection based on the boundary frame.

In some embodiments, as shown in FIG. 4, the object detection device 400 in the embodiments of the present invention further includes a determination unit 405, which determines the object type based on the velocity of the point in the radar detection result. The detection unit 404 performs object detection based on the above-mentioned boundary frame and the object type of each point in the radar detection result.

In some embodiments, the first generation unit 402 clusters on the radar detection results to obtain at least one cluster; for each cluster, the coordinates of the center point of the cluster are the center of the grid cell. The difference between the abscissa of the rightmost point in the cluster and the abscissa of the leftmost point is the width of the grid cell, and the abscissa of the top point and the abscissa of the bottom point in the cluster. By using the difference between the two as the height of the grid cells, the grid cells for each cluster can be obtained.

In some embodiments, the first generation unit 402 clusters on the radar detection results to obtain at least one cluster; for each cluster, grids the coordinates of the points with the highest signal strength in the cluster. For each cluster, center the cell and let the preset width and height be the width and height of the grid cell, or the product of the preset width and height and weight be the width and height of the network unit. You can get a grid cell.

In some embodiments, the first generation unit 402 has the coordinates of each point in the radar detection result as the center of the grid cell, and based on the distance between the point and the radar, the coordinates of the point are the center of the grid cell. The size of the grid cell to be used may be determined.

In some embodiments, the first generation unit 402 has the coordinates of each point in the radar detection result as the center of the grid cell, and the coordinates of the point as the center of the grid cell based on the signal strength of the point. The size of the grid cell can be determined.

In some embodiments, for each grid cell, the second generation unit 403 covers at least one grid cell by sliding left or right or up and down along the u-axis and v-axis of the center point of the grid cell. It can be generated as a corresponding boundary frame that makes the frame the grid cell.

In some embodiments, the second generation unit 403 further determines the reliability of the boundary frame based on the number of points in the grid cell, or determines the signal strength of the points in the grid cell. Based on this, the reliability of the boundary frame may be determined.

In some embodiments, the second generation unit 403 slides left and right or up and down along the u-axis and v-axis of the center point of the grid cell in units of the width and height of the grid cell. The boundary frame of the above is generated, or the second generation unit 403 slides left and right or up and down along the u-axis and v-axis of the center point of the grid cell in the preset unit size as a unit. Borders can be generated.

Although only each component or module according to the present invention has been described above, the present invention is not limited thereto. The object detection device 400 may further include other parts or modules, and the related art can be referred to for the specific contents of these parts or modules.

As can be seen from the above-described embodiment, the object detection device in the embodiment of the present invention significantly reduces the processing time of the object detection method, or improves the size and position accuracy of the boundary frame under the same processing time condition. Can be made to.

<Example of the third aspect>
An embodiment of the present invention provides an image processing apparatus, and the image processing apparatus may be, for example, a computer, a server, a workstation, a laptop computer, a smartphone, or the like, but the embodiment of the present invention is not limited thereto.

FIG. 5 is a diagram showing an image processing device according to an embodiment of the present invention. As shown in FIG. 5, the image processing device 500 according to an embodiment of the present invention has at least one interface (not shown in FIG. 5) and a processor. (For example, a central processing unit (CPU)) 501 and a storage device 502 are included, and the storage device 502 is connected to the processing device 501. Among them, the storage device 502 can store various data, stores the program 503 for object detection, executes the program 503 under the control of the processor 501, and performs various preset values and various preset values. It is also possible to store predetermined conditions and the like.

In one embodiment, the functions of the object detection device 400 described in the second aspect embodiment may be integrated in the processor 501 to realize the object detection method described in the first aspect embodiment. For example, the processor 501 may be configured as follows, i.e.
Matching radar detection results with video detection results;
Generate grid cells based on the radar detection result in the radar coordinate system;
A boundary frame is generated based on the grid cells in the image coordinate system; and object detection is performed based on the boundary frame.

In another embodiment, the object detection device 400 described in the second aspect embodiment may be configured separately from the processor 501, for example, the object detection device 400 is connected to the processor 501. The function of the object detection device 400 may be realized by controlling the processor 501.

The image processing device 500 may further include a display 505 and an I / O device 504, or does not need to include all the parts shown in FIG. 5, for example, further inputting a camera head (not shown). It may be included to obtain an image frame. Further, the image processing apparatus 500 may further include parts not shown in FIG. 5, and related techniques can be referred to for this.

In the embodiments of the present invention, the processor 501 may be referred to as a controller or operation controller and may include a microprocessor or other processor sum / or logic device. The processor 501 can receive the input and control the operation of each component of the image processing device 500.

In the embodiments of the present invention, the storage device 502 may be, for example, one or more of a buffer, a fresh memory, an HDD, a mobile medium, a volatile storage device, a non-volatile storage device, or other suitable device. , Various information and processing programs can be stored. The processor 501 may execute the program of storage in the storage 502 to realize storage, processing, and the like of information. Since the functions of other parts are similar to those of the conventional ones, detailed description thereof will be omitted here. Each component of the image processing apparatus 500 may be realized by dedicated hardware, firmware, software, or a combination thereof, but all of them belong to the scope of the present invention.

By detecting an object with the image processing apparatus according to the embodiment of the present invention, the processing time of the object detection method can be significantly reduced, or the size and position accuracy of the boundary frame can be improved under the same processing time condition. can.

An embodiment of the present invention further provides a computer-readable program, wherein when the program is executed in the image processing apparatus, the program provides the image processing apparatus with the method described in the first aspect of the embodiment. Let it run.

An embodiment of the present invention further provides a storage medium in which a computer-readable program is stored, wherein the computer-readable program executes the method described in the first aspect of the embodiment in an image processing apparatus.

The apparatus or method described with reference to the examples of the present invention can be realized by hardware, a software module executed by a processor, or a combination of both. For example, one or more functions in a functional block diagram and / or a combination of one or more functions in a functional block diagram may correspond to each software module in a computer program and correspond to each hardware module. Is also good. In addition, each of these software modules can correspond to each step shown in the figure showing the method. These hardware modules can be realized by solidifying these software modules using, for example, FPGA (field-programmable gate array).

Further, the apparatus, method, etc. according to the embodiment of the present invention may be realized by software, may be realized by hardware, or may be realized by a combination of hardware and software. The present invention also relates to such a computer-readable program, i.e., when the program is executed by a logic component, the logic component can realize the above-mentioned device or component, or the above-mentioned logic. The component can implement the method described above or a step thereof. Furthermore, the present invention also relates to a storage medium that stores the above-mentioned program, such as a hard disk, a magnetic disk, an optical disk, a DVD, or a fresh memory.

In addition, the following additional notes will be further disclosed with respect to the above examples.

(Appendix 1)
It is an object detection method
Matching radar detection results with video detection results;
Generate grid cells based on the radar detection result in the radar coordinate system;
A method comprising generating a border based on the grid cells in an image coordinate system; and performing object detection based on the border.

(Appendix 2)
The method described in Appendix 1
The method further includes determining the object type based on the velocity of the points in the radar detection result, and performing object detection based on the boundary frame is an object of the boundary frame and each point in the radar detection result. A method that involves performing object detection based on type.

(Appendix 3)
The method described in Appendix 1
Generating a grid cell based on the radar detection result in the radar coordinate system
Clustering is performed on the radar detection result to obtain at least one cluster; and for each cluster, the coordinate of the center point of the cluster is the center of the grid cell, and the abscissa and the most rightmost point in the cluster. The difference between the abscissa and the ordinate of the leftmost point is the width of the grid cell, and the difference between the ordinate of the topmost point and the ordinate of the bottommost point in the cluster is the height of the grid cell. A method that involves determining the corresponding grid cells.

(Appendix 4)
The method described in Appendix 1
Generating a grid cell based on the radar detection result in the radar coordinate system
Clustering is performed on the radar detection result to obtain at least one cluster; and for each cluster, the coordinates of the point with the highest signal strength in the cluster are set as the center of the grid cell, and the preset width and height are set. A method comprising obtaining a grid cell corresponding to each cluster by making the width and height of grid cells or the product of a preset width and height and weight being the width and height of a network unit.

(Appendix 5)
The method described in Appendix 1
Generating a grid cell based on the radar detection result in the radar coordinate system
Including determining the size of the grid cell with the coordinates of each point in the radar detection result as the center of the grid cell and the coordinates of the point as the center of the grid cell based on the distance between the point and the radar. ,Method.

(Appendix 6)
The method described in Appendix 1
Generating a grid cell based on the radar detection result in the radar coordinate system
A method comprising determining the size of a grid cell with the coordinates of each point in the radar detection result as the center of the grid cell and the coordinates of the point as the center of the grid cell based on the signal strength of the point.

(Appendix 7)
The method described in Appendix 1
Generating a border frame based on the grid cells in the image coordinate system
For each grid cell, by sliding left and right or up and down along the u-axis and v-axis of the center point of the grid cell, at least one frame covering the grid cell is generated as a boundary frame corresponding to the grid cell. The method, including that.

(Appendix 8)
The method described in Appendix 7
Generating a border frame based on the grid cells in the image coordinate system further
A method comprising determining the reliability of the boundary frame based on the number of points in the grid cell; or determining the reliability of the boundary frame based on the signal strength of the points in the grid cell.

(Appendix 9)
The method described in Appendix 7
Sliding left and right or sliding up and down along the u-axis and v-axis of the center point of the grid cell
Sliding left or right or up and down along the u-axis and v-axis of the center point of the grid cell in units of the width and height of the grid cell; or the center point of the grid cell in units of a preset unit size. A method comprising sliding left or right or up and down along the u-axis and v-axis of.

Although the preferred embodiment of the present invention has been described above, the present invention is not limited to this embodiment, and any modification to the present invention belongs to the technical scope of the present invention unless the gist of the present invention is deviated.

Claims (10)

An object detector
A matching unit that matches radar detection results with video detection results;
The first generation unit that generates grid cells based on the radar detection result in the radar coordinate system;
An object detection device including a second generation unit that generates a boundary frame based on the grid cells in an image coordinate system; and a detection unit that detects an object based on the boundary frame. The object detection device according to claim 1.
It further includes a confirmation unit that determines the object type based on the velocity of the point in the radar detection result.
The detection unit is an object detection device that detects an object based on the object type of each point in the boundary frame and the radar detection result. The object detection device according to claim 1.
The first generation unit clusters on the radar detection result and acquires at least one cluster; and for each cluster, the coordinates of the center point of the cluster are set as the center of the grid cell, and the most in the cluster. The difference between the abscissa of the rightmost point and the abscissa of the leftmost point is the width of the grid cell, and the difference between the abscissa of the uppermost point and the abscissa of the lowest point in the cluster is the height of the grid cell. By doing so, an object detection device that acquires the grid cells corresponding to each cluster. The object detection device according to claim 1.
The first generation unit clusters the radar detection result and acquires at least one cluster; and for each cluster, the coordinates of the point where the signal strength in the cluster is maximum is set as the center of the grid cell. , The preset width and height are the width and height of the grid cell, or the product of the preset width and height and the weight is the width and height of the network unit, so that the grid cell corresponding to each cluster is used. To get the object detector. The object detection device according to claim 1.
The first generation unit has the coordinates of each point in the radar detection result as the center of the grid cell, and the coordinates of the point as the center of the grid cell based on the distance between the point and the radar. An object detector that determines the size. The object detection device according to claim 1.
The first generation unit determines the size of a grid cell with the coordinates of each point in the radar detection result as the center of the grid cell and the point coordinates as the center of the grid cell based on the signal strength of the points. Object detector. The object detection device according to claim 1.
For each grid cell, the second generation unit slides left and right or up and down along the u-axis and v-axis of the center point of the grid cell to provide at least one frame covering the grid cell into the grid cell. An object detector that is generated as the corresponding border frame. The object detection device according to claim 7.
The second generation unit further determines the reliability of the boundary frame based on the number of points in the grid cell, or determines the reliability of the boundary frame based on the signal strength of the points in the grid cell. , Object detector. The object detection device according to claim 7.
The second generation unit slides left and right or up and down along the u-axis and v-axis of the center point of the grid cell in units of the width and height of the grid cell, or the second generation unit may An object detection device that slides left and right or up and down along the u-axis and v-axis of the center point of the grid cell with a preset unit size as a unit. An image processing device including a processor and a storage device.
A computer program is stored in the storage device.
The processor executes the computer program and
Matching radar detection results with video detection results;
Generate grid cells based on the radar detection result in the radar coordinate system;
An image processing device configured to generate a boundary frame based on the grid cells in an image coordinate system; and to realize object detection based on the boundary frame.

Images