KR102585525B1 - Object detection device and method thereof - Google Patents

Abstract

The present disclosure relates to an object detection apparatus and method, which includes a receiver for receiving image data and radar data about the surroundings of the vehicle from a camera and a radar, respectively, image object information about at least one object from the image data and radar data, and Extracts detected object information, maps the detected object information to image data based on preset image coordinate system information, calculates mapping object information, and matches objects determined to be the same object based on the mapping object information and image object information. It includes a matching unit that calculates object information, and a tracking unit that tracks each object based on the matching object information and updates the matching object information based on image data and radar data.

Description

Object detection device and method {OBJECT DETECTION DEVICE AND METHOD THEREOF}

The present disclosure relates to an object detection device and method, and more specifically, to an object detection device and method that can match and track object sensing information from different sensors.

ADAS (Advanced Driver Assistance Systems) technology is a general term for technologies provided to help drive a vehicle or improve driving stability, and is considered a part of autonomous driving technology or a necessary technology in the previous stage of fully autonomous driving.

In general, ADAS technology operates based on the vehicle's recognition of its surroundings to help drive the vehicle, and detecting objects around the vehicle is very important in the operation of various ADAS systems.

Considering this, it is necessary to develop technology that can improve object detection performance by accurately matching and fusing the sensing information of each sensor while complementing the strengths and weaknesses of the various sensors attached to the vehicle.

The present disclosure seeks to provide an object detection device and method that can detect an object by fusing sensing information from different sensors.

Additionally, the present disclosure seeks to provide an object detection device and method that can match information about the same object in sensing information from cameras and radars.

Additionally, the present disclosure seeks to provide an object detection device and method that can improve object detection accuracy by using the characteristics of each sensor.

In one aspect, the present embodiments include a receiver that receives image data and radar data about the surroundings of the host vehicle from a camera and a radar, respectively, and image object information and detection object information about at least one object from the image data and radar data, respectively. Extract and map the detected object information to image data based on preset image coordinate system information to calculate mapping object information, and calculate matching object information for objects determined to be the same object based on the mapping object information and image object information. An object detection device including a matching unit that tracks each object based on matching object information and a tracking unit that updates matching object information based on image data and radar data.

In another aspect, the present embodiments include a data receiving step of receiving image data and radar data about the surroundings of the own vehicle from a camera and a radar, respectively, and image object information and detection object about at least one object from the image data and radar data, respectively. Extract information, map detected object information to image data based on preset image coordinate system information to calculate mapping object information, and match object information for objects determined to be the same object based on mapping object information and image object information. An object detection method including an object matching step of calculating and an object tracking step of updating matching object information based on image data and radar data while tracking each object based on matching object information can be provided.

According to the present disclosure, it is possible to provide an object detection device and method that can detect an object by fusing sensing information from different sensors.

Additionally, according to the present disclosure, it is possible to provide an object detection device and method that can match information about the same object in sensing information from cameras and radars.

Additionally, according to the present disclosure, it is possible to provide an object detection device and method that can improve object detection accuracy by using the characteristics of each sensor.

1 is a block diagram of an object detection device according to the present disclosure.
FIG. 2 is a diagram illustrating a configuration for matching object information based on sensing fusion using an object detection device according to an embodiment.
FIG. 3 is a diagram illustrating a configuration for matching information about the same object among a plurality of object information using an object detection device according to an embodiment.
Figure 4 is a flowchart of an object detection method according to an embodiment.
Figure 5 is a flowchart of an object matching step in an object detection method according to an embodiment.
Figure 6 is a flowchart illustrating an object detection method according to another embodiment.
Figure 7 is a flowchart illustrating a matching step in an object detection method according to another embodiment.
Figure 8 is a flowchart illustrating a tracking step in an object detection method according to another embodiment.

Hereinafter, some embodiments of the present disclosure will be described in detail with reference to illustrative drawings. In adding reference numerals to components in each drawing, the same components may have the same reference numerals as much as possible even if they are shown in different drawings. Additionally, in describing the present embodiments, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present technical idea, the detailed description may be omitted. When “comprises,” “has,” “consists of,” etc. mentioned in the specification are used, other parts may be added unless “only” is used. When a component is expressed in the singular, it can also include the plural, unless specifically stated otherwise.

Additionally, in describing the components of the present disclosure, terms such as first, second, A, B, (a), and (b) may be used. These terms are only used to distinguish the component from other components, and the nature, sequence, order, or number of the components are not limited by the term.

In the description of the positional relationship of components, when two or more components are described as being “connected,” “coupled,” or “connected,” the two or more components are directly “connected,” “coupled,” or “connected.” ", but it should be understood that two or more components and other components may be further "interposed" and "connected," "combined," or "connected." Here, other components may be included in one or more of two or more components that are “connected,” “coupled,” or “connected” to each other.

In the description of temporal flow relationships related to components, operation methods, production methods, etc., for example, temporal precedence relationships such as “after”, “after”, “after”, “before”, etc. Or, when a sequential relationship is described, non-continuous cases may be included unless “immediately” or “directly” is used.

On the other hand, when a numerical value or corresponding information (e.g., level, etc.) for a component is mentioned, even if there is no separate explicit description, the numerical value or corresponding information is related to various factors (e.g., process factors, internal or external shocks, It can be interpreted as including the error range that may occur due to noise, etc.).

1 is a block diagram of an object detection device according to the present disclosure.

Referring to FIG. 1, the object detection device 100 according to the present disclosure may include a receiving unit 110, a matching unit 120, and a tracking unit 130. And the receiving unit 110, matching unit 120, and tracking unit 130 may be connected to each other.

As an example, the object detection device 100 includes a receiver that receives image data and radar data about the surroundings of the vehicle from a camera and a radar, respectively, and image object information and detection about at least one object from the image data and radar data, respectively. Extract object information, calculate mapping object information by mapping detected object information to image data based on preset image coordinate system information, and match objects for objects determined to be the same object based on mapping object information and image object information. It may include a matching unit that calculates information, and a tracking unit that tracks each object based on the matching object information and updates the matching object information based on image data and radar data.

The receiving unit 110 may be connected to a sensor device or other devices in the vehicle, and may receive certain signals or data from the sensor device or other devices in the vehicle. The receiving unit 110 may receive signals or data at any time or at a certain period. Data received by the receiver 110 may include image data and radar data.

In this case, image data may be received from a camera sensor installed in the vehicle, and radar data may be received from a radar sensor installed in the vehicle. In some cases, image data and radar data may be received from sensors or communication devices installed on other vehicles or other structures.

In some cases, the receiver 110 may further receive vehicle trajectory information for moving the host vehicle to a preset target location. In this case, the vehicle trajectory information may include information estimated using a vehicle trajectory estimation device installed in the own vehicle. Additionally, the receiver 110 may further receive heading angle information regarding the heading angle of the host vehicle from a yaw rate sensor.

The matching unit 120 may be connected to at least one of the receiving unit 110 and the tracking unit 130. In this case, new information may be calculated based on the information received from the receiving unit 110, or the received information may be recalculated using other information or a preset value. And the calculated information can be provided to the tracking unit 130.

As an example, the matching unit 120 may extract image object information and detection object information about at least one object from image data and radar data, respectively. Additionally, the matching unit 120 may calculate mapping object information by mapping the detected object information to image data based on preset image coordinate system information.

As an example, the matching unit 120 may calculate matching object information for objects that are determined to be the same object based on mapping object information and image object information.

As an example, the matching unit 120 may set a matching boundary area for each image object information. Additionally, when mapping object information is included in the matching boundary area, matching object information for objects determined to be the same object can be calculated by matching image object information and mapping object information.

As an example, when the matching boundary area includes two or more mapping object information, the matching unit 120 matches one of the mapping object information with the image object information based on a preset matching judgment standard to create a matching object. Information can be calculated.

As an example, the matching determination criterion may be set so that mapping object information determined to have the closest distance to the image object information is matched with the image object information based on the distance matching information.

As another example, the matching determination criterion may be set so that the mapping object information determined to have the smallest difference in moving direction from the image object information is matched with the image object information based on the direction matching information.

As another example, the matching determination criterion may be set so that mapping object information determined to have the smallest size difference from the image object information is matched with the image object information based on the size matching information.

As another example, the matching judgment standard calculates a matching evaluation score for each mapping object information based on distance matching information, direction matching information, size matching information, and preset weight information, and the mapping object information with the highest matching evaluation score is It can be set to match video object information.

As an example, the matching unit 120 may calculate distance matching information regarding the distance between image object information and mapping object information for each piece of mapping object information. In some cases, distance matching information may be set to be calculated only when two or more mapping object information are included in the matching boundary area.

Additionally, the matching unit 120 may calculate matching object information by matching mapping object information determined to have the closest distance to the image object information with the image object information based on the distance matching information.

As an example, the matching unit 120 may calculate direction matching information regarding the difference in movement direction between image object information and mapping object information for each piece of mapping object information. In some cases, direction matching information may be set to be calculated only when two or more mapping object information are included in the matching boundary area.

Additionally, the matching unit 120 may calculate matching object information by matching mapping object information determined to have the smallest movement direction difference with the image object information with the image object information based on the direction matching information.

As an example, the matching unit 120 may calculate size matching information regarding the size difference between image object information and mapping object information for each piece of mapping object information. In some cases, size matching information may be set to be calculated only when two or more pieces of mapping object information are included in the matching boundary area.

Additionally, the matching unit 120 may calculate matching object information by matching mapping object information determined to have the smallest size difference with the image object information with the image object information based on the size matching information.

As an example, the matching unit 120 may calculate a matching evaluation score for each mapping object information based on distance matching information, direction matching information, size matching information, and preset weight information. In some cases, the matching evaluation score may be set to be calculated only when two or more mapping object information are included in the matching boundary area.

Additionally, the matching unit 120 may calculate matching object information by matching mapping object information with the highest matching evaluation score with video object information.

As an example, the matching unit 120 may calculate object lateral distance information to the host vehicle based on image data, and calculate object longitudinal distance information to the host vehicle based on radar data.

And, the matching unit 120 may calculate matching object information based on the object horizontal distance information and the object vertical distance information. For example, the matching unit 120 calculates information about the horizontal component of the matching object information based on the object horizontal distance information, and information about the vertical component of the matching object information is calculated based on the object vertical distance information. It can be calculated based on

As an example, the matching unit 120 may extract information on at least one point on the trajectory based on vehicle trajectory information. Next, the matching unit 120 may calculate the lateral distance information of a point on a trajectory with the host vehicle based on the image data, and calculate the longitudinal distance information of a point on the trajectory with the host vehicle based on radar data. there is.

The tracking unit 130 may track each object based on matching object information. Additionally, the tracking unit 130 may update matching object information based on newly received image data and radar data.

As an example, the tracking unit 130 updates information about the lateral component of the vehicle trajectory information based on the lateral distance information of the point on the trajectory, and updates the longitudinal component of the vehicle trajectory information based on the longitudinal distance information of the point on the trajectory. Information about ingredients can be updated.

In some cases, the tracking unit 130 may control vehicle trajectory information to be updated using a preset PID controller. In this case, the PID controller is a first PID controller that calculates and updates the lateral component information of the vehicle trajectory based on the image data, the longitudinal distance information of the point on the trajectory, and the heading angle information, and the radar data and the lateral distance of the point on the trajectory. It may include a second PID controller that calculates and updates longitudinal component information of the vehicle trajectory based on the information and heading angle information.

Additionally, the tracking unit 130 can control vehicle trajectory information to be updated by fusing information calculated from the first PID controller and the second PID controller.

Through this process, the object detection apparatus 100 according to the present disclosure can improve the accuracy of the lateral component of vehicle trajectory information based on image data received from the camera. Additionally, the accuracy of the longitudinal component of vehicle trajectory information can be improved based on radar data received from the radar.

FIG. 2 is a diagram illustrating a configuration for matching object information based on sensing fusion using an object detection device according to an embodiment.

Referring to FIG. 2, the object detection device 100 according to an embodiment detects an object 220 using data received from a camera 212 and a radar 214 mounted on the own vehicle 210. You can.

As an example, the object detection device 100 may extract image object information 222 from image data received from the camera 212. And, the matching boundary area 226 can be set based on the image object information 222. In this case, the matching boundary area 226 may be set as an area within the range of a preset boundary distance from the image object information 222. Additionally, image coordinate system information may be set based on the image object information 222.

As an example, the object detection device 100 extracts detection object information from radar data received from the radar 214, maps the detection object information to image data based on image coordinate system information, and provides mapping object information 224. It can be calculated.

As an example, the object detection device 100 may determine whether the image object information 222 and the mapping object information 224 are information about the same object. And, if it is determined to be the same object, matching object information for the object can be calculated based on the image object information 222 and the mapping object information 224.

As an example, the object detection device 100 includes image object information 222 extracted from image data received from the camera 212 and mapping object information 224 extracted and mapped from radar data received from the radar 214. It can be determined whether is information about the same object.

For example, when the condition that the mapping object information 224 is included in the matching boundary area 226 and the condition that there is one mapping object included in the matching boundary area 226 are satisfied, the image object information 222 and The mapping object information 224 may be determined to be information about the same object, and in this case, matching object information may be calculated based on the image object information 222 and the mapping object information 224.

In some cases, when there are two or more mapping objects included in the matching boundary area, the object detection device 100 may determine video object information and mapping object information about the same object based on preset matching judgment criteria. , Configurations related to this will be described in more detail in FIG. 3 below.

In general, camera data has relatively higher accuracy in the lateral component, and radar data has relatively higher accuracy in the longitudinal component, making it possible to detect objects through sensor fusion between camera sensors and radar sensors. 100) object detection accuracy can be improved.

For example, when the object detection device 100 determines that the objects indicated by the image object information 222 and the mapping object information 224 are the same object and calculates matching object information, the sensor between the camera sensor and the radar sensor Matching object information can be calculated through fusion.

In this case, the object detection device 100 may calculate the object's lateral distance information to the host vehicle based on image data, and calculate the object's longitudinal distance information to the host vehicle based on radar data.

Next, the object detection device 100 calculates information about the horizontal component of the matching object information based on the object horizontal distance information, and calculates information about the vertical component of the matching object information based on the object vertical distance information. Information can be calculated. And, the matching object information can be finally calculated by fusing the information about the horizontal component and the information about the vertical component of the matching object information.

Through this process, the object detection device 100 can improve the accuracy of the lateral component of matching object information using image data received from the camera 212 and radar data received from the radar 214. Thus, the accuracy of the longitudinal component of matching object information can be improved.

FIG. 3 is a diagram illustrating a configuration for matching information about the same object among a plurality of object information using an object detection device according to an embodiment.

Referring to FIG. 3, the object detection device 100 has two or more mapping object information in the first matching boundary area 328 set based on the first image object information 320 for the first object 310. When included, mapping object information regarding the same object as the first image object information 320 may be determined based on a preset matching decision criterion.

As an example, the object detection device 100 may calculate distance matching information regarding the distance between image object information and mapping object information for each piece of mapping object information. In this case, the matching determination standard may be set to determine mapping object information that is determined to have the closest distance to the image object information based on the distance matching information.

As an example, the object detection device 100 may include first distance matching information regarding the distance between the first image object information 320 and the first mapping object information 330, the first image object information 320, and the second mapping Second distance matching information regarding the distance between objects 360 may be calculated.

In this case, the first distance matching information may be calculated as the distance between the first image object center 322 and the first mapping object center 332, and the second distance matching information may be calculated as the distance between the first image object center 322 and the first mapping object center 332. 2 It can be calculated as the distance between the centers of the mapping objects 362.

And, the object detection device 100 compares the sizes of the first distance matching information and the second distance matching information, and when it is determined that the first distance matching information is smaller, the distance to the first image object information 320 is The closest first mapping object information 330 may be determined as mapping object information about the same object as the first image object information 320.

As an example, the object detection device 100 includes first direction matching information regarding the difference in movement direction between the first image object information 320 and the first mapping object information 330, the first image object information 320, and the first image object information 320. Second direction matching information regarding the difference in movement directions between the two mapping objects 360 may be calculated.

In this case, the first direction matching information is the direction difference between the first image object difference width 324 and the first mapping object difference width 334 or the direction difference between the first image object full length 326 and the first mapping object full length 336. It can be calculated as, and the second direction matching information is the direction difference between the first image object difference width 324 and the second mapping object difference width 364 or the first image object full length 326 and the second mapping object full length 366. It can be calculated as the difference in direction between the two sides.

And, the object detection device 100 compares the sizes of the first direction matching information and the second direction matching information, and when it is determined that the first direction matching information is smaller, the moving direction with the first image object information 320 The first mapping object information 330 with the smallest difference may be determined as mapping object information about the same object as the first image object information 320.

As an example, the object detection device 100 may include first size matching information regarding the size difference between the first image object information 320 and the first mapping object information 330, the first image object information 320, and the second Second size matching information regarding the size difference between the mapping object information 360 may be calculated.

In this case, the first size matching information is the size difference between the first image object difference width 324 and the first mapping object difference width 334 or the size difference between the first image object full length 326 and the first mapping object full length 336. It can be calculated as, and the second size matching information is the size difference between the first image object difference width 324 and the second mapping object difference width 364 or the first image object full length 326 and the second mapping object full length 366. It can be calculated as the size difference between the liver.

And, the object detection device 100 compares the sizes of the first size matching information and the second size matching information, and when it is determined that the first size matching information is smaller, the size difference with the first image object information 320 The first mapping object information 330 with the smallest value may be determined as mapping object information about the same object as the first image object information 320.

As an example, the object detection device 100 includes a first matching evaluation score related to the first image object information 320 and the first mapping object information 330, and the first image object information 320 and the first mapping object ( 360), each of the second matching evaluation scores can be calculated.

As an example, the matching evaluation score may be set based on distance matching information, direction matching information, size matching information, and preset weight information. In this case, the weight information may be set so that the distance weight, direction weight, and size weight have different values.

For example, in a situation where the own vehicle is driving on a straight road, the angle weight can be set to be relatively small and the distance weight can be set to be relatively large.

As another example, in a situation where the host vehicle is driving on a curved road or performing lane change control, the angle weight can be set to be relatively larger.

In the embodiment shown in FIG. 3, two mapping objects exist in the first matching boundary area 328, but as seen above, the first distance matching information is smaller than the second distance matching information, and the first direction matching information is It is smaller than the second direction matching information, and since the first size matching information is judged to be smaller than the second size matching information, the first matching evaluation score can be calculated to be higher than the second matching evaluation score no matter how the weight information is set. .

And according to this calculation result, the mapping object with the highest matching evaluation score can be determined as mapping object information about the same object as the first image object information 320.

Below, an object detection method using the object detection device 100, which can perform all of the above-described content in the present disclosure, will be described. Content that overlaps with the content described above may be omitted in some cases, but also from the perspective of the method below. All can be applied.

Figure 4 is a flowchart of an object detection method according to an embodiment.

Referring to FIG. 4, the object detection method according to one embodiment may include a data receiving step (S410), an object matching step (S420), and an object tracking step (S430).

As an example, the object detection method includes a data receiving step of receiving image data and radar data about the surroundings of the own vehicle from a camera and a radar, respectively, and image object information and detection object about at least one object from the image data and radar data, respectively. Extract information, calculate mapping object information by mapping detected object information to image data based on preset image coordinate system information, and match object information for objects determined to be the same object based on mapping object information and image object information. It may include an object matching step of calculating and an object tracking step of updating matching object information based on image data and radar data while tracking each object based on matching object information.

The data reception step (S410) may include receiving image data and radar data from a camera sensor and a radar sensor. In some cases, in the data reception step (S410), the vehicle trajectory information for moving the host vehicle to a preset target position received from the vehicle trajectory estimation device and the heading angle of the host vehicle received from the yaw rate sensor. Additional heading angle information may be received.

In the object matching step (S420), image object information and detection object information are extracted from image data and radar data, mapping detection object information is mapped to image data to calculate mapping object information, and mapping object information is calculated based on the mapping object information and image object information. This may include calculating matching object information for objects that are determined to be the same object.

The object tracking step (S430) may include tracking each object based on matching object information. And, it may include updating matching object information based on newly received image data and radar data.

Figure 5 is a flowchart of an object matching step in an object detection method according to an embodiment.

Referring to FIG. 5, the object matching step (S420) according to one embodiment may include a distance matching step (S510), a direction matching step (S520), a size matching step (S530), and a weight evaluation matching step (S540). You can.

In the distance matching step (S510), distance matching information regarding the distance between the image object information and the mapping object information can be calculated for each piece of mapping object information. Then, based on the distance matching information, mapping object information determined to have the closest distance to the image object information may be matched with the image object information to calculate matching object information.

In the direction matching step (S520), direction matching information regarding the difference in movement direction between the image object information and the mapping object information can be calculated for each piece of mapping object information. Then, based on the direction matching information, mapping object information determined to have the smallest movement direction difference with the image object information is matched with the image object information to calculate matching object information.

In the size matching step (S530), size matching information regarding the size difference between the image object information and the mapping object information can be calculated for each mapping object information. Then, based on the size matching information, mapping object information determined to have the smallest size difference from the image object information may be matched with the image object information to calculate matching object information.

In the weight evaluation matching step (S540), a matching evaluation score can be calculated for each mapping object information based on distance matching information, direction matching information, size matching information, and preset weight information. Additionally, matching object information can be calculated by matching mapping object information with the highest matching evaluation score with video object information.

Figure 6 is a flowchart illustrating an object detection method according to another embodiment.

Referring to FIG. 6, an object detection method according to another embodiment includes a radar data reception step (S610), an image data reception step (S612), a data input step (S620), a matching step (S630), and a tracking step (S640). ), a buffer processing and completion determination step (S650), and an estimated data reflection step (S652).

The radar data receiving step (S610) and the image data receiving step (S612) may include receiving radar data and image data from a radar sensor and a camera sensor mounted on the own vehicle, respectively.

The data input step (S620) may include inputting the received radar data and image data for use in the matching and tracking process.

The matching step (S630) may include calculating matching object information for objects that are determined to be information about the same object, based on the input image data and radar data. The matching step (S630) will be described in more detail in FIG. 7 below.

The tracking step (S640) may include updating matching object information based on newly received image data and radar data while tracking each object based on the matching object information calculated in the matching step (S630). The tracking step (S640) will be described in more detail in FIG. 8 below.

In the buffer processing and completion determination step (S650), the result data of the matching step (S630) and the tracking step (S640) are performed based on the radar data and image data input in the data input step (S620) and input into the track buffer. , It may include determining whether the object detection process is complete. Here, if it is determined that the object detection process is not completed, the process may proceed to the estimated data reflection step (S652).

In the estimated data reflection step (S652), the result data of the matching step (S630) and the tracking step (S640) may be reflected in the object estimation data, and the reflected data may be input again into the data input step (S620). there is.

Figure 7 is a flowchart illustrating a matching step in an object detection method according to another embodiment.

Referring to FIG. 7, the matching step (S630) according to another embodiment includes an estimated buffer determining step (S710), a matching data generating step (S712), an initialization data generating step (S714), and a matching data checking step (S720). , a matching determination step (S730), an initialization data generation step (S732), a matching data input step (S740), a matching buffer input step (S750), a buffer cleaning step (S752), and a matching information update step (S760). You can.

As an example, it can be assumed that the matching step (S630) is performed after radar data and image data have already been input. And in this case, the radar data may be radar data converted based on image data.

The estimated buffer determination step (S710) may include determining whether the estimated buffer is NULL. If it is determined that it is not NULL (NO), it can proceed to the matching data generation step (S712), and if it is determined to be NULL (YES), it can proceed to the matching data generation step (S712).

The matching data generation step (S712) may include generating matching data for use in determining whether or not a match is made based on radar data and image data when the estimation buffer is not NULL. Then, the process may proceed to the matching data checking step (S720).

In the initialization data generation step (S714), if the estimation buffer is NULL, it may include initializing the estimation data by reflecting the radar data and image data.

The matching data checking step (S720) may include confirming checks to determine whether the generated matching data matches the data stored in the estimation buffer.

In the matching determination step (S730), a matching determination may be performed to determine whether the generated matching data and the data stored in the estimation buffer are data related to the same object. Here, if it is determined that there is no match, the process may proceed to the initialization data generation step (S732).

The initialization data generation step (S732) may include initializing the matching data when it is determined that the generated matching data and the data stored in the estimation buffer do not match.

The matching data input step (S740) may include inputting matching data when it is determined that the generated matching data and the data stored in the estimation buffer match.

The matching buffer input step (S750) may include inputting the input matching data into the matching buffer. And, in this case, the buffer cleaning step (S752) may proceed.

The buffer cleaning step (S752) may include inputting matching data into the matching buffer and deleting and organizing some of the existing data stored in the matching buffer. For example, in the case of a stack buffer, the most recently stored data among existing data can be deleted, and in the case of a queue buffer, the most recently stored data among existing data can be deleted.

The matching information updating step (S760) may include updating matching-related information that has changed according to matching data input.

Figure 8 is a flowchart illustrating a tracking step in an object detection method according to another embodiment.

Referring to FIG. 8, the tracking step (S640) according to another embodiment includes a tracking mode determination step (S810), a creation mode step (S812), a tracking mode step (S814), an extinction mode step (S816), and a Kalman filter. It may include a processing step (S820) and a tracking completion determination step (S830).

As an example, the tracking step (S640) is performed after matching is made based on radar data and camera data in the matching step (S630), and fusion matching data fused by the sensor using the result is input to the fusion buffer. It can be assumed that

The tracking mode determination step (S810) may include determining the tracking mode based on fusion matching data. In this case, tracking modes may include creation mode, tracking mode, and destruction mode.

In the creation mode step (S812), if the object determined based on the fusion matching data is determined to be an object that did not exist in the existing fusion buffer, it may include creating the object to be added to the tracking target. In this case, this may include adding the ID of the object to the fusion buffer.

In the tracking mode step (S814), if the object determined based on the fusion matching data is determined to be an object that existed in the existing fusion buffer, it may include maintaining it as a tracking target.

In the extinction mode step (S816), if the object determined based on the fusion matching data is determined to no longer be a tracking target, it may include deleting it from the tracking target. In this case, this may include deleting the ID of the corresponding object from the fusion buffer.

The Kalman filter processing step (S820) may include performing Kalman filter processing based on the input fusion matching data, thereby removing noise from the data and improving detection accuracy.

The tracking completion determination step (S830) may include determining whether tracking has been completed for all objects. If tracking is determined to be complete for all objects, the tracking step (S640) is terminated, and if tracking is not determined to be complete for even one object, the tracking step (S640) can be returned to the beginning and repeated.

And, when the tracking step (S640) is completed, the result can be generated as Output_message according to a preset method. In this case, Output_message contains information about the fusion_id item to distinguish each object, the class_id item to distinguish the type of object information, the SFOut.msg item regarding the contents of the control message, and the fusion_flag item to distinguish whether sensor fusion is performed. may be included.

As described above, the present disclosure can provide an object detection device and method that can detect an object by fusing sensing information from different sensors.

Additionally, the present disclosure can provide an object detection device and method that can match information about the same object in sensing information from cameras and radars.

Additionally, the present disclosure can provide an object detection device and method that can improve object detection accuracy by using the characteristics of each sensor.

The above description is merely an illustrative explanation of the technical idea of the present disclosure, and those skilled in the art will be able to make various modifications and variations without departing from the essential characteristics of the present disclosure. In addition, the present embodiments are not intended to limit the technical idea of the present disclosure, but rather to explain it, so the scope of the present technical idea is not limited by these embodiments. The scope of protection of this disclosure should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of rights of this disclosure.

Claims (14)

a receiving unit that receives image data and radar data about the surroundings of the vehicle from a camera and a radar, respectively;
Extract image object information and detection object information about at least one object from the image data and the radar data, respectively, and map the detection object information to the image data based on preset image coordinate system information to calculate mapping object information. and a matching unit that calculates matching object information for objects determined to be the same object based on the mapping object information and the image object information; and
A tracking unit that tracks each object based on the matching object information and updates the matching object information based on the image data and the radar data,
The matching unit,
An object detection device that sets a matching boundary area for each image object information, and calculates the matching object information by matching the image object information and the mapping object information when the matching boundary area includes the mapping object information. delete According to claim 1,
The matching unit,
When the matching boundary area includes two or more pieces of mapping object information, calculate distance matching information regarding the distance between the image object information and the mapping object information for each piece of mapping object information,
An object detection device that calculates the matching object information by matching the mapping object information, which is determined to have the closest distance to the image object information, with the image object information based on the distance matching information. According to claim 1,
The matching unit,
When the matching boundary area includes two or more pieces of mapping object information, calculate direction matching information about a difference in movement direction between the image object information and the mapping object information for each piece of mapping object information,
An object detection device that calculates the matching object information by matching the mapping object information, which is determined to have the smallest movement direction difference with the image object information, with the image object information based on the direction matching information. According to claim 1,
The matching unit,
When the matching boundary area includes two or more pieces of mapping object information, calculate size matching information about the size difference between the image object information and the mapping object information for each piece of mapping object information,
An object detection device that calculates the matching object information by matching the mapping object information, which is determined to have the smallest size difference with the image object information, with the image object information based on the size matching information. According to claim 1,
The matching unit,
When the matching boundary area includes two or more pieces of mapping object information, distance matching information about the distance between the image object information and the mapping object information, direction matching information about the difference in movement direction, and size matching information about the size difference Calculate each,
A matching evaluation score is calculated for each mapping object information based on the distance matching information, the direction matching information, the size matching information, and the preset weight information, and the mapping object information with the highest matching evaluation score is assigned to the video object. An object detection device that matches information and calculates the matching object information. According to claim 1,
The matching unit,
Calculating lateral distance information of the object with the host vehicle based on the image data, and calculating longitudinal distance information of the object with the host vehicle based on the radar data,
An object detection device that calculates the matching object information based on the object lateral distance information and the object longitudinal distance information. A data receiving step of receiving image data and radar data about the surroundings of the vehicle from a camera and a radar, respectively;
Extract image object information and detection object information about at least one object from the image data and the radar data, respectively, and map the detection object information to the image data based on preset image coordinate system information to calculate mapping object information. and an object matching step of calculating matching object information for an object determined to be the same object based on the mapping object information and the image object information; and
An object tracking step of tracking each object based on the matching object information and updating the matching object information based on the image data and the radar data,
The object matching step is,
An object detection method for setting a matching boundary area for each image object information, and calculating the matching object information by matching the image object information and the mapping object information when the matching boundary area includes the mapping object information. delete According to claim 8,
The object matching step is,
When the matching boundary area includes two or more pieces of mapping object information, calculate distance matching information regarding the distance between the image object information and the mapping object information for each piece of mapping object information,
An object detection method comprising a distance matching step of calculating the matching object information by matching the mapping object information, which is determined to have the closest distance to the image object information, with the image object information based on the distance matching information. According to claim 8,
The object matching step is,
When the matching boundary area includes two or more pieces of mapping object information, calculate direction matching information about a difference in movement direction between the image object information and the mapping object information for each piece of mapping object information,
An object detection method comprising a direction matching step of calculating the matching object information by matching the mapping object information determined to have the smallest movement direction difference with the image object information based on the direction matching information with the image object information. . According to claim 8,
The object matching step is,
When the matching boundary area includes two or more pieces of mapping object information, calculate size matching information about the size difference between the image object information and the mapping object information for each piece of mapping object information,
An object detection method comprising a size matching step of calculating the matching object information by matching the mapping object information determined to have the smallest size difference with the image object information based on the size matching information with the image object information. According to claim 8,
The object matching step is,
When the matching boundary area includes two or more pieces of mapping object information, distance matching information about the distance between the image object information and the mapping object information, direction matching information about the difference in movement direction, and size matching information about the size difference Calculate each,
A matching evaluation score is calculated for each mapping object information based on the distance matching information, the direction matching information, the size matching information, and the preset weight information, and the mapping object information with the highest matching evaluation score is assigned to the video object. An object detection method including a weight evaluation matching step of calculating the matching object information by matching information. According to claim 8,
The object matching step is,
Calculating lateral distance information of the object with the host vehicle based on the image data, and calculating longitudinal distance information of the object with the host vehicle based on the radar data,
The object tracking step is,
An object detection method for calculating the matching object information based on the object lateral distance information and the object longitudinal distance information.

Images