JP7382799B2 - Object recognition method and object recognition device - Google Patents

Description

The present invention relates to an object recognition method and an object recognition device.

Patent Document 1 proposes a device that acquires position information of surrounding objects at a plurality of observation positions (specifically, a vehicle) and supports driving operation of the vehicle based on this position information.

JP 2017-111565 Publication

In the device proposed in Patent Document 1, for example, in a case where each observation position is located relatively close to each other, it is assumed that the same surrounding object is observed from each of these observation positions. However, depending on the relative positional relationship between the observation positions, the observation point for the same object may differ depending on the observation position. Therefore, even if the surrounding object is actually the same, the position information of the surrounding object observed at each observation position may not match each other, and there is a possibility that it may be mistakenly recognized as a different surrounding object. Ta.

In view of these circumstances, an object of the present invention is to provide an object recognition method and an object recognition device that can further improve the accuracy of object recognition when observing surrounding objects from a plurality of observation positions. .

According to an aspect of the present invention, an object recognition method is provided for recognizing surrounding objects that are objects existing around each observation position from a plurality of observation positions. In this object recognition method, the first object relative position, which is the relative position of the first surrounding object with respect to the first observation position, and the first relative motion vector, which is the motion vector of the first surrounding object with respect to the first observation position, are determined. get. Further, a second object relative position, which is the relative position of the second surrounding object with respect to the second observation position, and a second relative motion vector, which is a motion vector of the second surrounding object with reference to the second observation position, are obtained. and determining whether the first surrounding object and the second surrounding object are the same based on the first object relative position, the first relative motion vector, the second object relative position, and the second relative motion vector. Execute object determination processing.

According to the present invention, it is possible to further improve the accuracy of object recognition when observing surrounding objects from a plurality of observation positions.

FIG. 1 is a diagram illustrating a vehicle configuration common to each embodiment for carrying out the object recognition method of the present invention. FIG. 2 is a block diagram illustrating the functions of a control device that executes an object recognition method. FIG. 3 is a flowchart illustrating the object recognition method. FIG. 4 is a diagram illustrating an overview of coordinate transformation from relative coordinates to absolute coordinates. FIG. 5 is a flowchart illustrating the same object determination process according to the first embodiment. FIG. 6 is a diagram illustrating an example in which the object recognition method is applied to a specific scene (object recognition on a road). FIG. 7 is a flowchart illustrating identical object determination processing according to the second embodiment. FIG. 8 is a flowchart illustrating motion vector correction processing. FIG. 9A is a diagram for explaining details of motion vector correction processing. FIG. 9B is a diagram for explaining details of the motion vector correction process. FIG. 9C is a diagram for explaining details of the motion vector correction process. FIG. 10 is a flowchart illustrating an object recognition method according to the third embodiment.

Embodiments of the present invention will be described below with reference to the drawings and the like. In each embodiment described below, the first observation position and the second observation position are different vehicles (first vehicle 10-1 and second vehicle 10-2), and the first vehicle 10-1 is the first observation position and the second observation position is different from each other. Assume that a first surrounding object O1 is observed, and a second vehicle 10-2 observes a second surrounding object O2. Furthermore, from the viewpoint of simplification of description, when describing matters common to the first surrounding object O1 and the second surrounding object O2, the term "surrounding object O" encompassing them will be used as appropriate.

(First embodiment)
First, an overview of the configurations of the first vehicle 10-1 and the second vehicle 10-2 will be explained with reference to FIGS. 1 and 2. In the following description of the configuration, from the viewpoint of simplifying the description, the first vehicle 10-1 and the second vehicle 10-2 will be collectively referred to as "vehicle 10". That is, the configuration of the vehicle 10 described below is based on the assumption that the first vehicle 10-1 and the second vehicle 10-2 each have the same configuration unless otherwise specified.

FIG. 1 is a diagram illustrating the configuration of a vehicle 10. As shown in FIG.

As illustrated, the vehicle 10 includes a peripheral sensor 1, an external communication device 2, an output device 3, and a control device 20.

The surrounding sensor 1 is a detection device that detects the surrounding situation of the vehicle 10. In particular, the surrounding sensor 1 includes a camera, a radar, a lidar (LIDER: Laser Imaging Detection and Ranging), etc. as an imaging device that images the surroundings of the vehicle 10. The surrounding sensor 1 outputs detected surrounding information (a captured image of a camera or a reflected wave of a radar or lidar) to a control device 20 at a predetermined detection cycle (in the case of a camera, a predetermined frame rate). In this embodiment, when the peripheral sensor 1 is configured with a camera, a configuration (two or more monocular cameras, or a stereo camera, etc.).

The external communication device 2 is a device that receives position information of the vehicle 10 measured by GPS (Global Positioning System) from a predetermined external server.

The output device 3 performs various outputs based on commands from the control device 20. Specifically, the output device 3 is configured as predetermined software for creating and updating map information (information indicating real-time road conditions) configured inside or outside the vehicle 10.

The control device 20 is configured to be able to communicate various signals with the peripheral sensor 1 and the external communication device 2, and has a hardware configuration and a software configuration for executing each process related to the object recognition method of this embodiment. Be prepared.

Specifically, the control device 20 includes a central processing unit (CPU), a read-only memory (ROM), a random access memory (RΑM), and an input/output interface (I/O interface), and performs each process related to the object recognition method. It consists of a computer that stores programs to be executed in read-only memory. Note that the computer constituting the control device 20 may be composed of one computer hardware, or may be divided into a plurality of computer hardware to perform the above-mentioned processes in a distributed manner.

FIG. 2 is a block diagram illustrating the configuration of the control device 20. As shown in FIG.

As illustrated, the control device 20 includes a self-position calculation section 11, an object relative position calculation section 13, a relative motion vector calculation section 14, and an object recognition processing section 15.

Based on the position information of the vehicle 10 acquired via the external communication device 2, the self-position calculation unit 11 calculates the position of the surrounding sensor 1 and its sensor direction using a global coordinate system or a fixed coordinate system (absolute) based on the self-position measurement method. Calculate the value expressed in the coordinate system).

More specifically, the self-position calculation unit 11 calculates the surrounding sensor in the absolute coordinate system based on the position information of the vehicle 10 that can be specified by GPS, and the predetermined position information and direction of the surrounding sensor 1 on the vehicle 10. 1's position coordinates and sensor direction are calculated.

Note that the position coordinates and sensor direction of the peripheral sensor 1 are obtained by adding or subtracting known values to or from the position and orientation of the vehicle 10. Therefore, in the following, these will be treated as the same for the purpose of simplifying the explanation.

Particularly, hereinafter, the position coordinates of the peripheral sensor 1 in the absolute coordinate system will be referred to as "vehicle absolute coordinates (X, Y)" as the same as the position coordinates of the vehicle 10. Further, the sensor direction of the peripheral sensor 1 in the absolute coordinate system is equated with the direction of the vehicle 10, and this is quantified and referred to as "vehicle absolute direction angle A." In particular, the vehicle absolute direction angle A of this embodiment is defined as the angle formed by the direction of the vehicle 10 and a predetermined reference direction (for example, the north direction) fixed in the absolute coordinate system.

Furthermore, below, as necessary, the vehicle absolute coordinates (X, Y) and the vehicle absolute direction angle A of the first vehicle 10-1 will be referred to as "first vehicle absolute coordinates (X 1 , Y 1 )" and "first vehicle absolute coordinates (X ₁ , Y ₁ )," respectively. 1 vehicle absolute direction angle A ₁ ". Further, the vehicle absolute coordinates (X, Y) and the vehicle absolute direction angle A of the second vehicle 10-2 are respectively "second vehicle absolute coordinates (X ₂ , Y ₂ )" and "second vehicle absolute direction angle A ₂ ". It is called.

The object relative position calculation unit 13 calculates relative position information of the surrounding object O as seen from the vehicle 10 based on the surrounding information detected by the surrounding sensor 1. More specifically, the object relative position calculation unit 13 calculates the position of the surrounding object O (hereinafter referred to as ``object relative coordinates (u, w)'' in a coordinate system having the position of the vehicle 10 as its origin (hereinafter referred to as ``vehicle coordinate system''). ) is calculated. Note that, hereinafter, the vehicle coordinate system of the first vehicle 10-1 will be particularly referred to as the "first vehicle coordinate system" and the vehicle coordinate system of the second vehicle 10-2 will be referred to as the "second vehicle coordinate system."

For example, when a captured image of a camera is acquired as the peripheral information of the peripheral sensor 1, the object relative position calculation unit 13 performs a predetermined object recognition process on the captured image to detect surrounding objects O on the captured image. The three-dimensional position (position in the horizontal direction of the image, position in the front-rear direction of the image plane, and position in the vertical direction of the image) is calculated.

More specifically, the object relative position calculation unit 13 calculates the three-dimensional position coordinates of the surrounding object O on the image using, for example, a coordinate system set with the image center position (position of the vehicle 10) as the origin. In particular, the object relative position calculation unit 13 specifies the positional coordinates of the surrounding object O in the front-rear direction of the image plane by calculating the parallax in the pixel region corresponding to the part of the surrounding object O included in the captured image. Then, the object relative position calculation unit 13 calculates object relative coordinates (u, w) representing the three-dimensional position coordinates of the surrounding object O on the calculated image in the vehicle coordinate system in real space.

On the other hand, when radar or lidar detection results are acquired as surrounding information, the surroundings are determined based on the shape and size of the reflection points of surrounding objects O that can be identified from the detection results (reflected waves). The object relative coordinates (u, w) of the object O can be determined.

Hereinafter, the object relative coordinates (u, w) calculated by the object relative position calculation unit 13 of the first vehicle 10-1 will be particularly referred to as "first object relative coordinates (u ₁ , w ₁ )." On the other hand, the object relative coordinates (u, w) calculated by the object relative position calculation unit 13 of the second vehicle 10-2 are referred to as "second object relative coordinates (u ₂ , w ₂ )."

Next, the relative motion vector calculation section 14 will be explained. The relative motion vector calculation unit 14 calculates a relative motion vector (vu, vw) of the surrounding object O and a relative motion vector direction angle vb that quantifies its directional component, based on the peripheral information detected by the peripheral sensor 1. .

Here, the term "motion vector" in this embodiment means a vector quantity representing a temporal change in the position coordinates of a predetermined observation point (that is, a velocity vector of the movement of the observation point). Therefore, the relative motion vector (vu, vw) of the surrounding object O means the motion vector of the surrounding object O expressed in the vehicle coordinate system. Further, the relative motion vector direction angle vb is defined as the angle formed by the direction of the vehicle 10 (that is, the sensor direction of the peripheral sensor 1) and the relative motion vector (vu, vw) in the vehicle coordinate system.

In particular, the relative motion vector calculation unit 14 calculates the relative motion vector (vu, vw) and the relative motion vector direction angle vb using the position information of the vehicle 10 obtained from the peripheral information of the peripheral sensor 1 in a plurality of detection cycles. do.

This will be explained more specifically. For example, when a captured image of a camera is acquired as peripheral information of the peripheral sensor 1, the relative motion vector calculation unit 14 calculates a set of points constituting at least a part of the surrounding object O in the captured image detected at a certain detection time T1. A feature point having a feature that is relatively easy to track is extracted from the (pixel set), and the three-dimensional position coordinates of this feature point on the image are specified.

Next, at a detection time T2 corresponding to the next detection cycle, the relative motion vector calculation unit 14 performs matching processing as appropriate to extract feature points common to those extracted at the detection time T1, and extracts the feature points in common with those extracted at the detection time T1, and the three-dimensional position thereof. Identify coordinates. Then, the relative motion vector calculation unit 14 calculates a three-dimensional motion vector by dividing the difference between the three-dimensional position coordinates obtained at the detection time T1 and the three-dimensional position coordinates obtained at the detection time T2 by the detection period. Further, the relative motion vector calculation unit 14 obtains relative motion vectors (vu, vw) by converting this three-dimensional motion vector into a vehicle coordinate system in real space. Note that the relative motion vector calculation unit 14 calculates the angle formed between the relative motion vectors (vu, vw) and the coordinate axes (coordinate axes on the vehicle coordinate system) as the relative motion vector direction angle vb.

On the other hand, when radar or lidar detection results are acquired as peripheral information, the relative motion vector calculation unit 14 extracts a point group of reflection points of the surrounding object O that can be identified from the detection results (reflected waves). . Then, the relative motion vector calculation unit 14 obtains a change in the point group position over the course of the detection period (a change in the point group position between detection time T1 and detection time T2), and displays this in the vehicle coordinate system. Relative motion vectors (vu, vw) can be calculated.

In the following, it is assumed that the position of the feature point extracted in the motion vector calculation is the same as the position of the observed surrounding object O, and both are expressed in object relative coordinates (u, w). is called "observation point Cr". In particular, the observation point Cr of the first vehicle 10-1 will be referred to as a "first observation point Cr ₁ ", and the observation point Cr of the second vehicle 10-2 will be referred to as a "second observation point Cr ₂ ".

Further, the relative motion vectors (vu, vw) and the relative motion vector direction angle vb calculated by the relative motion vector calculation unit 14 of the first vehicle 10-1 are respectively referred to as "first relative motion vectors (vu ₁ , vw ₁ )". and “first relative motion vector direction angle vb ₁ ”. On the other hand, the relative motion vectors (vu, vw) and the relative motion vector direction angle vb calculated by the relative motion vector calculation unit 14 of the second vehicle 10-2 are respectively referred to as "second relative motion vectors (vu ₂ , vw ₂ )". and “second relative motion vector direction angle vb ₂ ”.

Next, the object recognition processing section 15 will be explained. The object recognition processing section 15 uses vehicle absolute coordinates (X, Y) and vehicle absolute direction angle A calculated by the self-position calculation section 11 and object relative coordinates (u, w) calculated by the object relative position calculation section 13. , the relative motion vector (vu, vw) and the relative motion vector direction angle vb computed by the relative motion vector computation unit 14, the first surrounding object O1 observed by the first vehicle 10-1 and the A process is performed to determine the identity between the second surrounding object O2 observed by the second vehicle 10-2.

In this embodiment, it is assumed that the object recognition processing section 15 is provided only in the second vehicle 10-2. Therefore, the object recognition processing unit 15 of the second vehicle 10-2 uses the first observation information through known vehicle-to-vehicle communication or the like to obtain the above-mentioned first vehicle absolute coordinates (X ₁ , Y ₁ ), the first vehicle absolute direction angle A ₁ , the first object relative coordinates (u ₁ , w ₁ ), the first relative motion vector (vu ₁ , vw ₁ ), and the first relative motion vector direction angle vb _{1 .} get.

More specifically, the object recognition processing section 15 includes a coordinate unification section 24, an identical object determination processing section 25, and an object integration processing section 26.

The coordinate unification unit 24 calculates the relative position and relative motion vector of the first surrounding object O1 in the first vehicle 10-1 and the relative position of the second surrounding object O2 calculated in the second vehicle 10-2. and the relative motion vector into a common coordinate system, which is an absolute coordinate system.

The same object determination processing unit 25 identifies the first surrounding object O1 observed by the first vehicle 10-1 and the first surrounding object O1 observed by the first vehicle 10-1 based on the respective position information and motion vector information changed into a common coordinate system in the coordinate unification unit 24. The identity of the second surrounding object O2 observed by the second vehicle 10-2 is determined.

The object integration processing section 26 performs a process of integrating the first surrounding object O1 and the second surrounding object O2, which are determined to be the same by the same object determination processing section 25, as the same surrounding object O. Further, the object integration processing unit 26 integrates the first surrounding object O1 and the second surrounding object O2 as the same surrounding object O into a predetermined map held in the control device 20-2 of the second vehicle 10-2 or externally. The information on the map will be updated to reflect this fact.

Note that the processing of the coordinate unification section 24, the same object determination processing section 25, and the object integration processing section 26 will be explained in more detail later. Next, the flow of the object recognition method executed in the vehicle 10 that constitutes the object recognition device having each of the above configurations will be described.

FIG. 3 is a flowchart illustrating the object recognition method of this embodiment. Note that each process described below is repeatedly executed at a predetermined calculation cycle by the control device 20 of the vehicle 10 (particularly the control device 20-2 of the second vehicle 10-2).

In step S100, the control device 20-2 executes position and motion vector acquisition processing. Specifically, the control device 20-2 calculates the first vehicle absolute coordinates (X ₁ , Y ₁ ), the first vehicle absolute direction angle A ₁ , and the first object relative coordinates calculated by the first vehicle 10-1. (u ₁ , w ₁ ), the first relative motion vector (vu ₁ , vw ₁ ), and the first relative motion vector direction angle vb ₁ are obtained from the control device 20-1 of the first vehicle 10-1.

Further, the control device 20-2 automatically determines the second vehicle absolute coordinates (X ₂ , Y ₂ ), the second vehicle absolute direction angle A ₂ , the second object relative coordinates (u ₂ , w ₂ ), and the second relative motion vector. (vu ₂ , vw ₂ ) and a second relative motion vector direction angle vb ₂ .

In step S200, the control device 20-2 executes coordinate unification processing. Specifically, the control device 20-2 controls the first object relative coordinates (u ₁ , w ₁ ), the first relative motion vector (vu ₁ , vw ₁ ), the first relative motion vector direction angle vb ₁ , and the second vehicle The absolute direction angle A ₂ , the second object relative coordinates (u ₂ , w ₂ ), the second relative motion vector (vu ₂ , vw ₂ ), and the second relative motion vector direction angle vb ₂ are displayed on the absolute coordinate system. Convert.

For the sake of simplicity, the following explanation regarding conversion to an absolute coordinate system includes the relative coordinates, motion vectors, and motion vector directions of the first vehicle 10-1 and the second vehicle 10-2. Let absolute coordinates (X, Y), vehicle absolute direction angle A, object relative coordinates (u, w), relative motion vector (vu, vw), and relative motion vector direction angle vb.

In addition, the values of object relative coordinates (u, w), relative motion vector (vu, vw), and relative motion vector direction angle vb after conversion to the absolute coordinate system are respectively expressed as "object absolute coordinates (U, W)". ”, “absolute motion vector (VU, VW)”, and “absolute motion vector direction angle VB”.

FIG. 4 is a diagram illustrating an overview of coordinate transformation from relative coordinates to absolute coordinates. In addition, in FIG. 4, the direction of the vehicle is shown by a dotted line arrow, and the direction of a motion vector is shown by a chain line. Furthermore, it is also referred to as a direction from the position of the vehicle 10 toward the observation point Cr of the surrounding object O (hereinafter, "observation direction DO"). In particular, the direction from the position of the first vehicle 10-1 toward the first observation point Cr ₁ of the first surrounding object O1 is referred to as a "first observation direction DO ₁ ." The direction from the position of the second vehicle 10-2 toward the second observation point Cr ₂ of the second surrounding object O2 is referred to as a "second observation direction DO ₂ ."

As shown in the figure, the absolute coordinate system (XY coordinate plane in FIG. 4) allows for parallel movement of the vehicle absolute coordinates (X, Y) with respect to the vehicle coordinate system (xy coordinate plane in FIG. 4), and The relationship is obtained by performing inverse rotational transformation corresponding to the absolute direction angle A.

Therefore, the control device 20-2 determines the object absolute coordinates (U, W) using the following equation (1).

Note that the matrix of the second term on the right side of the first equation of equation (1) corresponds to inverse rotational transformation, that is, rotational transformation when the rotation angle is set to "-A".

Next, motion vector conversion will be explained. Since the motion vector is a change in the position of the surrounding object O per time, the term of the vehicle absolute coordinates (X, Y) corresponding to the parallel movement component is not included when converting from the vehicle coordinate system to the absolute coordinate system. Therefore, the absolute motion vector (VU, VW) can be expressed only by a term obtained by subjecting the relative motion vector (vu, vw) to an inverse rotational transformation corresponding to the vehicle absolute direction angle A.

Specifically, the control device 20-2 calculates the absolute motion vector (VU, VW) using the following equation (2).

Furthermore, as is clear from FIG. 4, the absolute motion vector direction angle VB is equal to the sum of the vehicle absolute direction angle A and the relative motion vector direction angle Vb. Therefore, the control device 20-2 determines the absolute motion vector direction angle VB using the following equation (3).

In the following, the object absolute coordinates (U, W), absolute motion vector (VU, VW), and absolute motion vector direction angle VB of the first vehicle 10-1 will be referred to as "first object absolute coordinates" as necessary. (V ₁ , W ₁ ),” “first absolute motion vector (VU ₁ , VW ₁ ),” and “first absolute motion vector direction angle VB ₁ .”

Further, the object absolute coordinates (U, W), absolute motion vector (VU, VW), and absolute motion vector direction angle VB of the second vehicle 10-2 are respectively expressed as "second object absolute coordinates (U ₂ , W ₂ )". ", "second absolute motion vector (VU ₂ , VW ₂ )", and "second absolute motion vector direction angle VB ₂ ".

Returning to FIG. 3, in step S300, the control device 20-2 executes identical object determination processing.

FIG. 5 is a flowchart illustrating the same object candidate extraction process of this embodiment.

As shown in the figure, in the same object candidate extraction process, first in step S310, the control device 20-2 performs a process to extract the same object candidates.

Specifically, when a plurality of first surrounding objects O1 are observed in the first vehicle 10-1 and a plurality of second surrounding objects O2 are observed in the second vehicle 10-2, the control device 20-2 , among these, those that may actually be the same object are extracted.

More specifically, the control device 20-2 calculates the distance between the plurality of first surrounding objects O1 and the plurality of second surrounding objects O2 from the position coordinates in the absolute coordinate system for any combination of both objects. do. Then, the control device 20-2 extracts the first surrounding object O1 and the second surrounding object O2 for which the inter-object distance is equal to or less than a predetermined distance threshold THs as identical object candidates.

Note that the distance threshold THs has the effect of suppressing the omission of extraction of the first surrounding object O1 and the second surrounding object O2 that are the same, and the effect of extracting a large number of identical object candidates and performing subsequent motion vector calculation. This is set to a suitable value in consideration of the balance between the effect of avoiding an increase in calculation load due to

Next, in step S340, the control device 20-2 determines whether the respective absolute motion vectors (VU, VW) of the first surrounding object O1 and the second surrounding object O2 extracted as the same object candidates satisfy a prescribed correlation. Determine whether or not.

Here, the prescribed correlation in this embodiment is defined as a relationship in which the difference in absolute motion vectors between the first surrounding object O1 and the second surrounding object O2 is small enough to determine that they are the same object. .

In other words, the prescribed correlation is the difference in magnitude (difference in absolute value) between the first absolute motion vector (VU ₁ , VW ₁ ) and the second absolute motion vector (VU ₂ , VW ₂ ), and As a relationship such that the motion vector difference ΔV ₁₂ determined from the viewpoint of the difference in direction (the difference between the first absolute motion vector direction angle VB ₁ and the second absolute motion vector direction angle VB ₂ ) is equal to or less than a predetermined motion vector difference threshold THv. determined.

The motion vector difference threshold THv is determined by determining that the first surrounding object O1 and the second surrounding object O2 are the same, taking into consideration the characteristics of the measurement system of the surrounding sensor 1, measurement errors, calculation errors, etc. It is determined from the viewpoint of whether the motion vector difference ΔV ₁₂ is as close to 0 as possible.

In particular, when an image captured by a camera is acquired as peripheral information detected by the peripheral sensor 1, errors in parallax calculation, the size of the angle to the pixel that can be calculated from the number of pixels of the camera and the angle of view of the lens, and the above-mentioned movement It is preferable to set this in consideration of measurement errors during tracking of feature points for vector calculation.

Therefore, if the motion vector difference ΔV ₁₂ exceeds the motion vector difference threshold THv, the control device 20-2 determines that the prescribed correlation is not satisfied, and proceeds to the process of step S360. That is, in this case, the control device 20-2 determines that the first surrounding object O1 and the second surrounding object O2 are not the same object, and ends this process.

On the other hand, the control device 20-2 determines that the prescribed correlation is satisfied when the motion vector difference ΔV ₁₂ is less than or equal to the motion vector difference threshold THv, and executes the process of step S350. That is, in this case, the control device 20-2 determines that the first surrounding object O1 and the second surrounding object O2 are the same object, and proceeds to the process of step S400.

Returning to FIG. 3, in step S400, the control device 20-2 executes integration processing. Specifically, the control device 20-2 updates the map information via the output device 3 by treating the first surrounding object O1 and the second surrounding object O2 as the same surrounding object O. In this case, the information on the surrounding objects O that is actually reflected in the map information is acquired by the more accurate one of the surrounding sensor 1 of the first vehicle 10-1 and the surrounding sensor 1 of the second vehicle 10-2. It is preferable to use the information provided.

An example in which the object recognition method described above is applied to a specific scene will be described.

FIG. 6 is a diagram illustrating an example in which the object recognition method is applied to a specific scene.

In particular, in FIG. 6, the first vehicle 10-1 and the second vehicle 10-2 traveling in opposite lanes are surrounded by surrounding objects O (vehicles other than the vehicle 10, obstacles on the road, pedestrians, etc.) on the road. ) is assumed to be observed. Note that FIG. 6 shows an example in which four surrounding objects O exist around the first vehicle 10-1 and the second vehicle 10-2, and these are marked with "a", "c", and "d", respectively. Attach a sign. Further, in FIG. 6, the directions of the absolute motion vectors (VU, VW) of object a, object c, and object d are schematically shown by dashed-dotted line arrows.

Furthermore, the circle α indicated by the broken line in FIG. 6 is used to extract identical object candidates in the above-mentioned step S310 when focusing on object c (especially when focusing on the second observation point Cr ₂ of object c). It represents the range of the determined distance threshold THs. That is, this circle α is a circle whose radius is centered on the second observation point Cr ₂ and corresponds to the distance threshold THs.

Here, the first vehicle 10-1 is traveling in the opposite direction to the second vehicle 10-2 in the opposite lane. Therefore, as is clear from FIG. 6, the first observation point Cr ₁ detected when the first vehicle 10-1 observes the object c is detected when the second vehicle 10-2 observes the object c. It is different from the second observation point Cr ₂ .

Therefore, even if the object c is the same, the position information of the object c observed by the first vehicle 10-1 and the position information of the object c observed by the second vehicle 10-2 are different from each other. . Therefore, from the viewpoint of the control device 20-2 of the second vehicle 10-2, the observation information of the object c acquired from the first vehicle 10-1 and the observation information of the object c observed by itself do not match. It is assumed that the objects will be recognized as different objects. On the other hand, it is also conceivable to set a relatively large threshold value for the deviation between the first observation point Cr ₁ and the second observation point Cr ₂ for determining that they are the same object. However, in this case, there is a possibility that the first vehicle 10-1 and the second vehicle 10-2 may be mistakenly determined to be the same object, even though they are actually observing different objects. There is.

In response to this situation, in this embodiment, in addition to the position information of the object, absolute motion vectors (VU, VW) that do not depend on the difference in observation position are referred to. By preventing recognition, the accuracy of object recognition can be further improved. This will be explained in more detail by applying it to the scene shown in FIG.

First, in the same object candidates (step S310), the object d existing outside the circle α is removed from the same object candidates for the object c.

Next, by comparing the motion vector difference ΔV ₁₂ between the object c and the object a with the motion vector difference threshold THv (step S340), it is determined that the object c is not the same as the object a.

More specifically, the first absolute motion vector (VU ₁ , VW ₁ ) determined by tracking the first observation point Cr ₁ of the object a by the first vehicle 10-1 is The direction is opposite to the second absolute motion vector (VU ₂ , VW ₂ ) determined by tracking the second observation point Cr ₂ of the object c (more specifically, the directions are opposite to each other in the vertical direction in the figure). Therefore, the motion vector difference ΔV ₁₂ (absolute value difference in FIG. 6) between object c and object a exceeds the motion vector difference threshold THv, and it is determined that object a and object c are not the same. (No in step S340 and step S360).

On the other hand, the first absolute motion vector (VU ₁ , VW ₁ ) determined by tracking the first observation point Cr ₁ of the object c by the first vehicle 10-1 is the same as that of the same object c by the second vehicle 10-2. This substantially matches the second absolute motion vector (VU ₂ , VW ₂ ) determined by tracking the second observation point Cr ₂ .

Therefore, the motion vector difference ΔV ₁₂ between the first observation point Cr ₁ and the second observation point Cr ₂ of the object c is less than the motion vector difference threshold THv. That is, it is determined that the first observation point Cr ₁ observed by the first vehicle 10-1 and the second observation point Cr ₂ observed by the second vehicle 10-2 belong to the same object c. (Yes in step S340 and step S350).

In particular, in the example shown in FIG. 6, since the first vehicle 10-1 and the second vehicle 10-2, which are the observation positions, are moving relatively, the motion vector of the object c as seen from each position is It is assumed that the first relative motion vector (vu ₁ , vw ₁ ) and the second relative motion vector (vu ₂ , vw ₂ ) are shifted from each other. However, if these are the first absolute motion vectors (VU ₁ , VW ₁ ) and second absolute motion vectors (VU ₂ , VW ₂ ) converted into a common coordinate system, the influence of the relative movement of each observation position is eliminated. Since it is not included, it is possible to suitably determine object identity.

According to this embodiment having the configuration described above, the following effects are achieved.

This embodiment provides an object recognition method for recognizing surrounding objects O, which are objects existing around each observation position, from a plurality of observation positions.

In this object recognition method, first object relative coordinates (u ₁ , w ₁ ) as the first object relative position, which is the relative position of the first surrounding object O1 with respect to the first observation position (first vehicle 10-1), and A first relative motion vector (vu 1 , vw 1 ), which is a motion vector of the first surrounding object O1 with respect to the first vehicle 10-1, is obtained, and a first relative motion vector (vu ₁ , vw ₁ ) is obtained, and The second object relative coordinates (u 2 , w 2 ) are the relative positions of the second surrounding objects, and the second object relative coordinates (u ₂ , w ₂ ) are the relative positions of the second surrounding objects, and the second object relative coordinates (u 2 , w 2 ) are the relative positions of the second surrounding objects O2, and Two relative motion vectors (vu ₂ , vw ₂ ) are obtained (step S100 in FIG. 3). and,

Then, the first object relative coordinates (u ₁ , w ₁ ), the first relative motion vector (vu ₁ , vw ₁ ), the second object relative coordinates (u ₂ , w ₂ ), and the second relative motion vector (vu ₂ , vw ₂ ), the same object determination process (steps S300 to S360 in FIG. 5) is executed to determine whether the first surrounding object O1 and the second surrounding object O2 are the same.

As a result, the object identity of the first surrounding object O1 and the second surrounding object O2 can be determined based on the first object relative coordinates (u ₁ , w ₁ ) and the second object relative coordinates (u ₂ , w ₂ ), as well as the first relative motion vector (vu ₁ , vw ₁ ) and the second relative motion vector (vu ₂ , vw ₂ ) seen from each observation position. This will be done based on the following. Here, if the first surrounding object O1 and the second surrounding object O2 are the same surrounding object O, the observation points of the surrounding object O from both observation positions (first observation point Cr ₁ and second observation point Cr ₂ ) Even if the values are different, the motion vector representing the trajectory of the movement (translational movement) of the surrounding object O is basically unchanged. That is, unlike position information, the motion vector of the surrounding object O is not affected by the difference in observation position itself. Therefore, when determining object identity, by using the motion vector of the surrounding object O, even though the same surrounding object O is observed from multiple observation positions, it may be mistakenly recognized as a different object. It is possible to suppress the occurrence of such situations, and the accuracy of object recognition is further improved.

In particular, in the object recognition method of this embodiment, the first object relative coordinates (u ₁ , w ₁ ), the first relative motion vector (vu ₁ , vw ₁ ), the second object relative coordinates (u ₂ , w ₂ ), and the second relative motion vectors (vu ₂ , vw ₂ ) are respectively converted into a common coordinate system (absolute coordinate system) to obtain the first object absolute coordinates (V ₁ , W ₁ ) as the first object position and the first movement. A first absolute motion vector (VU ₁ , VW ₁ ) as a vector, a second object absolute coordinate (U ₂ , W ₂ ) as a second object position, and a second absolute motion vector (VU _{2 )} as a second motion vector. , VW ₂ ) (step S200 in FIG. 3). In the same object determination process, the first object absolute coordinates (V ₁ , W ₁ ), the first absolute motion vector (VU ₁ , VW ₁ ), the second object absolute coordinates (U ₂ , W ₂ ), and the second Based on the absolute motion vectors (VU ₂ , VW ₂ ), it is determined whether the first surrounding object O1 and the second surrounding object O2 are the same (steps S310 to S360 in FIG. 5).

Thereby, information on the positions and motion vectors of the first surrounding object O1 and the second surrounding object O2, which are respectively observed at different observation positions, can be handled in a unified coordinate system. Therefore, it is possible to suppress the complexity of the process related to determining the object identity between the first surrounding object O1 and the second surrounding object O2.

Furthermore, the interrelationship (difference between them) of the first absolute motion vector (VU ₁ , VW ₁ ) and the second absolute motion vector (VU ₂ , VW ₂ ) is the first relative motion vector (vu ₁ , vw ₁ ) and the second relative motion vector (vu ₂ , vw ₂ ), it remains unchanged with respect to the relative movement of each observation position. Therefore, even if the first vehicle 10-1 and the second vehicle 10-2 constituting the observation position are moving, object identity determination can be performed with high accuracy.

Therefore, in particular, as in the present embodiment, observation is performed in which the first vehicle 10-1 and the second vehicle 10-2, which are equipped with the surrounding sensor 1 that detects the surrounding situation, are set at the first observation position and the second observation position, respectively. This contributes to improving the determination accuracy in systems (particularly in observation systems where the observation position moves).

Further, in the object recognition method of the present embodiment, in the same object determination process, the mutual distance between the first object absolute coordinates (V ₁ , W ₁ ) and the second object absolute coordinates (U ₂ , W ₂ ) is a predetermined threshold ( A first surrounding object O1 and a second surrounding object O2 whose distance is less than or equal to a distance threshold THs are extracted (step S310 in FIG. 5). Then, it is determined whether the extracted first surrounding object O1 and second surrounding object O2 are the same as each other using the first absolute motion vector (VU ₁ , VW ₁ ) and the second absolute motion vector (VU ₂ , VW ₂ ) (steps S340 to S360).

That is, first, the first surrounding object O1 and the second surrounding object O2, which are observed to be less than or equal to the distance threshold THs, which may realistically be the same, are extracted as identical object candidates, and then the objects are extracted based on their motion vectors. A determination of identity will be performed. Therefore, in the first place, for the first surrounding object O1 and the second surrounding object O2, which are unlikely to be the same object because their observed distances are far apart from each other, object identity analysis using motion vectors is difficult. Processing related to determination can be omitted. Therefore, it is possible to reduce the calculation load on the control device 20-2 while maintaining the accuracy of the identity of the first surrounding object O1 and the second surrounding object O2.

Furthermore, in the object recognition method of this embodiment, the first absolute motion vector (VU ₁ , VW ₁ ) of the first surrounding object O1 and the second absolute motion vector (VU ₂ , VW _{2 )} of the second surrounding object O2 are extracted. ) satisfies a predefined correlation (step S340 in FIG. 5). Then, when it is determined that the first absolute motion vector (VU ₁ , VW ₁ ) and the second absolute motion vector (VU ₂ , VW ₂ ) satisfy the above correlation, the first surrounding object O1 and the second surrounding object It is assumed that O2 is the same (Yes in step S340 and step S350).

That is, from the viewpoint of determining that the first surrounding object O1 and the second surrounding object O2 are the same according to various conditions, the respective first absolute motion vectors (VU ₁ , VW ₁ ) and second absolute motion vectors (VU ₂ , VW ₂ ) is set as a correlation, and object identity is determined based on whether the correlation is satisfied. In particular, by determining the correlation according to factors such as the properties of the measurement system of the peripheral sensor 1, measurement errors, motion vector calculation errors, and the above-mentioned observation difference angle Δθ, object identity can be determined based on motion vectors. can be made to match the actual observation system with higher precision.

Furthermore, in the object recognition method of the present embodiment, the above correlation is such that the motion vector difference, which is the difference between the first absolute motion vector (VU ₁ , VW ₁ ) and the second absolute motion vector (VU ₂ , VW ₂ ), is a predetermined difference. The relationship is such that the motion vector difference threshold value THv is less than or equal to the motion vector difference threshold value THv.

As mentioned above, the motion vector of the surrounding object O that basically moves in translation is the position of the first observation point Cr ₁ and the second observation point Cr ₂ between the first vehicle 10-1 and the second vehicle 10-2. It remains unchanged with respect to differences in . Therefore, by setting a correlation such that the motion vector difference is less than or equal to the motion vector difference threshold THv, it is possible to suitably determine the identity of the first surrounding object O1 and the second surrounding object O2. In particular, the motion vector difference threshold THv is basically set to a value close to 0, but the object identity can be determined with higher precision by setting it to a value that appropriately takes into account errors in the sensor system.

Further, according to the present embodiment, an object recognition device (control device 20-2 of the second vehicle 10-2) for executing the object recognition method described above is provided.

The control device 20-2 includes an object relative position acquisition section (object relative position calculation section 13), a relative motion vector acquisition section (relative motion vector calculation section 14), and an identical object determination processing section (object recognition processing section). 15) and (see FIG. 2).

The object relative position calculation unit 13 calculates first object relative coordinates (u ₁ , w ₁ ) as the first object relative position, which is the relative position of the first surrounding object O1 with respect to the first observation position (first vehicle 10-1). , and second object relative coordinates (u ₂ , w ₂ ) that are the second object relative position that is the relative position of the second surrounding object with respect to the second observation position (second vehicle 10-2).

The relative motion vector calculation unit 14 calculates a first relative motion vector (vu 1 , vw 1 ) which is a motion vector of the first surrounding object O1 with the first vehicle 10-1 as a reference, and a first relative motion vector (vu ₁ , vw ₁ ) with the second vehicle 10-2 as a reference. A second relative motion vector (vu ₂ , vw ₂ ) that is a motion vector of the second surrounding object O2 is obtained.

The object recognition processing unit 15 then generates the first object relative coordinates (u ₁ , w ₁ ), the first relative motion vector (vu ₁ , vw ₁ ), the second object relative coordinates (u ₂ , w ₂ ), and the Based on the two relative motion vectors (vu ₂ , vw ₂ ), it is determined whether the first surrounding object O1 and the second surrounding object O2 are the same.

Thereby, a suitable control configuration (program configuration) for executing the object recognition method of this embodiment is realized.

(Second embodiment)
The second embodiment will be described below. Note that the same elements as in the first embodiment are denoted by the same reference numerals, and the description thereof will be omitted. In particular, in this embodiment, a mode will be described in which the identity of the first surrounding object O1 and the second surrounding object O2 is determined when the surrounding object O is performing a turning motion.

FIG. 7 is a flowchart illustrating the same object candidate extraction process of this embodiment. As shown in the figure, the control device 20-2 of the present embodiment controls the first surrounding object O1 and the second surrounding object O2 extracted in the same object candidate extraction process described in the first embodiment from step S320 onwards. Execute the process.

Specifically, in step S320, the control device 20-2 determines whether the surrounding object O is turning. Specifically, the control device 20-2 determines that the surrounding object O is turning when the time change in the direction of the absolute motion vector (VU, VW) is larger than a predetermined threshold. On the other hand, the control device 20-2 determines that the surrounding object O is not turning when the time change in the direction of the absolute motion vector (VU, VW) is less than or equal to this threshold value.

In particular, the control device 20-2 determines the magnitude between each of the time change of the first absolute motion vector direction angle VB ₁ and the time change of the second absolute motion vector direction angle VB ₂ and a predetermined turning determination threshold. compare. Then, the control device 20-2 causes the surrounding object O (more specifically, both the first surrounding object O1 and the second surrounding object O2) to turn when at least one of these exceeds the turning determination threshold. If not, it is determined that the vehicle is not turning.

In this way, the configuration is such that it is determined that the surrounding object O is turning when at least one of the temporal changes in the first absolute motion vector direction angle VB ₁ and the second absolute motion vector direction angle VB ₂ exceeds the threshold value. Therefore, the rotation of the surrounding object O can be detected more reliably. More specifically, even in a scene where the temporal change in the absolute motion vector direction angle VB of one of the first observation point Cr 1 and the second observation point Cr ₂ is small, such as when one of the first observation point Cr ₁ and the second observation point Cr 2 is located near the turning center, the error can be avoided. Judgment is suitably suppressed.

In addition, in determining the turning of the surrounding object O, instead of using the time change of each absolute motion vector direction angle VB, or in addition to this, the magnitude of the first absolute motion vector (VU ₁ , VW ₁ ) and the second absolute motion vector are used. The relative relationship between the sizes of (VU ₂ , VW ₂ ) may also be considered.

Then, when the control device 20-2 determines that the surrounding object O is not turning, it executes the processes from step S340 onwards, as in the first embodiment. On the other hand, if the control device 20-2 determines that the surrounding object O is turning, the control device 20-2 shifts to the process of step S330.

In step S330, the control device 20-2 executes motion vector correction processing. Here, the motion vector correction process of this embodiment is a process of correcting the motion vector difference ΔV ₁₂ that is compared with the motion vector difference threshold THv in step S340.

As already explained, when the surrounding object O is not rotating (translational motion), the difference in position between the first observation point Cr ₁ and the second observation point Cr ₂ is determined by the respective first absolute motion vectors (VU ₁ , VW ₁ ) and the second absolute motion vector (VU ₂ , VW ₂ ).

On the other hand, when the surrounding object O turns (rotational movement), the first observation point Cr ₁ and the second observation point Cr ₂ have different moving speeds depending on the difference in the distance from the center of rotation. Depending on the positions of the first vehicle 10-1 and the second vehicle 10-2 (more specifically, the relative positional relationship between the first vehicle 10-1 and the second vehicle 10-2 with respect to the surrounding object O), the first absolute motion vector (VU ₁ , VW ₁ ) and the second absolute motion vector (VU ₂ , VW ₂ ) are different from each other.

Therefore, when turning, even if the first surrounding object O1 and the second surrounding object O2 are actually the same, it depends on the relative positional relationship between the first observation point Cr ₁ and the second observation point Cr ₂ and the turning center. In this case, it is assumed that the motion vector difference ΔV ₁₂ may or may not exceed the motion vector difference threshold THv in the determination in step S340.

In such a situation, the control device 20-2 of the present embodiment appropriately corrects the motion vector difference ΔV ₁₂ itself to be compared with the motion vector difference threshold THv when the turning of the surrounding object O is detected. Execute motion vector correction processing from the viewpoint. The motion vector correction process will be explained in more detail below.

FIG. 8 is a flowchart illustrating motion vector correction processing.

First, in step S331, the control device 20-2 calculates the observation difference angle Δθ as a parameter representing the deviation between the first observation direction DO ₁ and the second observation direction DO ₂ .

More specifically, the control device 20-2 controls the vector corresponding to the first observation direction DO ₁ in the absolute coordinate system (from the first vehicle absolute coordinates (X ₁ , Y ₁ ) to the first object absolute coordinates (V ₁ , W ₁ )), and a vector corresponding to the second observation direction DO ₂ (vector in the direction from the second vehicle absolute coordinates (X ₂ , Y ₂ ) to the second object absolute coordinates (U ₂ , W ₂ ) (vector), and calculate its absolute value as the observed difference angle Δθ.

Next, in step S332, the control device 20-2 determines whether the calculated observed difference angle Δθ is less than or equal to a predetermined first difference angle threshold THa1. When the control device 20-2 determines that the observed difference angle Δθ is less than or equal to the first difference angle threshold THa1, the control device 20-2 proceeds to the process of step S340. That is, in this case, the motion vector correction process ends without substantially correcting the motion vector difference ΔV ₁₂ .

On the other hand, if the control device 20-2 determines that the observed difference angle Δθ exceeds the first difference angle threshold THa1, the process proceeds to step S333.

In step S333, the control device 20-2 determines whether the observed difference angle Δθ is less than or equal to a predetermined second difference angle threshold THa2. That is, it is determined whether the observed difference angle Δθ is in a range that exceeds the first difference angle threshold THa1 and is equal to or less than the second difference angle threshold THa2.

When the control device 20-2 determines that the observed difference angle Δθ is less than or equal to the second difference angle threshold THa2, it performs a first correction to be described later on the motion vector difference ΔV ₁₂ (step S334). Then, the control device 20-2 uses the motion vector difference ΔV ₁₂ after the first correction to execute the determination in step S340 of FIG. 7 and the subsequent processing.

On the other hand, if the control device 20-2 determines that the observed difference angle Δθ exceeds the second difference angle threshold THa2, it performs a second correction to be described later on the motion vector difference ΔV ₁₂ (step S335). Then, the control device 20-2 uses the motion vector difference ΔV ₁₂ after the second correction to execute the determination in step S340 of FIG. 7 and the subsequent processing.

Below, the mode of correction of the motion vector difference ΔV ₁₂ determined based on the observed difference angle Δθ in the motion vector correction process will be described in more detail.

9A to 9C are diagrams for explaining details of motion vector correction processing. In particular, FIG. 9A schematically shows the mode of the actual observation system when the observation difference angle Δθ is less than or equal to the first difference angle threshold THa1 (when the determination in step S332 is Yes). FIG. 9B also shows the actual situation when the observed difference angle Δθ exceeds the first difference angle threshold THa1 and is less than or equal to the second difference angle threshold THa2 (when the determination in step S332 is No and the determination in step S333 is Yes). This diagram schematically shows the aspect of the observation system. Further, FIG. 9C schematically shows the state of the actual observation system when the observation difference angle Δθ exceeds the second difference angle threshold THa2 (when the determination in step S333 is No).

Furthermore, in FIGS. 9A to 9C, a part of the set of points constituting the surrounding object O is shown as a black dot, and the motion vector of the black dot is schematically shown as a dot-double chain line.

First, if the observation difference angle Δθ shown in _FIG . The two observation points Cr ₂ are both located on the same observation plane H1 that is substantially orthogonal to the turning direction. Therefore, even if the surrounding object O is turning, the first absolute motion vector direction angle VB ₁ related to the first observation point Cr ₁ and the second absolute motion vector direction angle VB ₂ related to the second observation point Cr ₂ are mutually different. This is approximately the same as .

Therefore, in this case, as in the case of non-turning described in the first embodiment, the control device 20-2 does not correct the motion vector difference ΔV ₁₂ but based on the magnitude relationship with the motion vector difference threshold THv. to determine the identity of the object. For this purpose, the first difference angle threshold THa1 is such that the first observation point Cr ₁ by the first vehicle 10-1 and the second observation point Cr ₂ by the second vehicle 10-2 are on the same observation plane H1. The value is set to a value that enables suitable estimation of whether or not to do so.

Next, when the observation difference angle Δθ shown in FIG. 9B exceeds the first difference angle threshold THa1 and is less than or equal to the second difference angle threshold THa2, the second observation point Cr ₂ by the second vehicle 10-2 is located at the above observation plane. Even if it is located at H1, the first observation point _Cr1 by the first vehicle 10-1 is located at the observation plane H2 (that is, the plane along the side surface of the surrounding object O) orthogonal to the observation plane H1. Become. Therefore, when the surrounding object O turns, the second absolute motion vector (VU ₂ , VW ₂ ) related to the second observation point Cr ₂ located on the observation plane H1 is the same as that of the first observation point Cr 2 located on the observation plane H2. This results in a deviation from the first absolute motion vector (VU ₁ , VW ₁ ) related to Cr ₁ in terms of both magnitude and direction.

Therefore, in this case, the control device 20-2 controls each absolute motion vector caused by the difference between the observation plane H1 where the second observation point Cr ₂ is located and the observation plane H2 where the first observation point Cr ₁ is located. Taking into account the difference in (VU, VW), the motion vector difference ΔV ₁₂ is corrected (first correction).

More specifically, the control device 20-2 controls the movement locus of the second observation point Cr ₂ on the observation plane H1 perpendicular to the direction of rotation and the observation plane different from the observation plane H1 when the surrounding object O turns. The motion vector difference ΔV ₁₂ is corrected based on a dynamic model that suitably represents the relationship between the motion trajectories of the first observation point Cr ₁ on H2.

In particular, when the surrounding object O is a vehicle or a similar traveling object, by using a two-wheeled model as the dynamic model, the respective motion trajectories of the second observation point Cr ₂ and the first observation point Cr ₁ described above can be It becomes possible to appropriately correct the motion vector difference ΔV ₁₂ according to the difference.

More specifically, in the example of FIG. 9B, the second absolute motion vector (VU ₂ , VW ₂ ) is the motion vector of the rear wheel in the two-wheel model, and the first absolute motion vector (VU ₁ , VW ₁ ) is the motion vector of the rear wheel in the two-wheel model. By setting the motion vector of the two-wheel side surface in , it is possible to appropriately evaluate the difference in the motion trajectories of the second observation point Cr ₂ and the first observation point Cr ₁ during a turn that matches the two-wheel model. As a result, it is possible to appropriately correct the motion vector difference ΔV ₁₂ according to this motion trajectory.

Furthermore, if the observation difference angle Δθ shown in FIG. 9C exceeds the second difference angle threshold THa2, even if the second observation point Cr ₂ by the second vehicle 10-2 is located on the observation plane H1, the first The first observation point Cr ₁ by the vehicle 10-1 is located on an observation plane H3 that is parallel to the observation plane H1 and on the opposite side (opposite side in the longitudinal direction of the surrounding object O).

Therefore, when the surrounding object O turns, the second absolute motion vector (VU ₂ , VW ₂ ) related to the second observation point Cr ₂ located on the observation plane H1 is the same as that of the first observation point Cr 2 located on the observation plane H3. This results in a deviation from the first absolute motion vector (VU ₁ , VW ₁ ) related to Cr ₁ in terms of both magnitude and direction.

Therefore, in this case, the control device 20-2 calculates the difference in motion vectors caused by the difference between the observation plane H1 where the second observation point Cr ₂ is located and the observation plane H3 where the first observation point Cr ₁ is located. Taking this into account, the motion vector difference ΔV ₁₂ is corrected (second correction).

More specifically, the control device 20-2 controls the movement locus of the second observation point Cr ₂ on the observation plane H1 perpendicular to the direction of rotation and the observation plane different from the observation plane H1 when the surrounding object O turns. The motion vector difference ΔV ₁₂ is corrected based on a dynamic model that suitably represents the relationship between the motion trajectories of the first observation point Cr ₁ on H3.

In particular, when the surrounding object O is a vehicle or a similar traveling object, by using a two-wheeled model as the dynamic model, the respective motion trajectories of the second observation point Cr ₂ and the first observation point Cr ₁ described above can be It becomes possible to appropriately correct the motion vector difference ΔV ₁₂ according to the difference.

More specifically, in the example of FIG. 9C, the second absolute motion vector (VU ₂ , VW ₂ ) is the motion vector of the rear wheel in the two-wheel model, and the first absolute motion vector (VU ₁ , VW ₁ ) is the motion vector of the rear wheel in the two-wheel model. By setting the motion vector of the front wheel in , it is possible to appropriately evaluate the difference between the motion trajectories of the second observation point Cr ₂ and the first observation point Cr ₁ during a turn that matches the two-wheel model. As a result, it is possible to appropriately correct the motion vector difference ΔV ₁₂ according to this motion trajectory.

According to this embodiment having the configuration described above, the following effects are achieved.

In the object recognition method of this embodiment, the surrounding object O turns based on the temporal change in at least one direction of the first absolute motion vector (VU ₁ , VW ₁ ) and the second absolute motion vector (VU ₂ , VW ₂ ). If it is determined that the surrounding object O is turning (step S320 in FIG. 7), the above correlation (motion vector difference ΔV ₁₂ ) is corrected for when the surrounding object O is turning. A corrected correlation (motion vector difference ΔV ₁₂ after first correction or second correction) is set (step S330). Then, when it is determined that the first absolute motion vector (VU ₁ , VW ₁ ) and the second absolute motion vector (VU ₂ , VW ₂ ) satisfy the corrected correlation, the first surrounding object O1 and the second surrounding object Assume that O2 is the same.

As a result, the first absolute motion vector (VU ₁ , VW ₁ ) and the second absolute motion vector (VU ₂ , VW ₂ ) determined from the viewpoint of determining that the first surrounding object O1 and the second surrounding object O2 are the same. The correlation between them is calculated by taking into account the deviation of the mutual motion vectors caused by the difference in the respective motion trajectories of the second observation point Cr ₂ and the first observation point Cr ₁ when the surrounding object O is rotating. Can be set appropriately. As a result, the accuracy of determining object identity can be further improved.

In particular, in the object recognition method of this embodiment, the first observation direction Do ₁ , which is the observation direction of the first surrounding object O1 by the first observation position (first vehicle 10-1), is acquired, and the second observation position (first vehicle 10-1) A second observation direction Do ₂ , which is the observation direction of the second vehicle 10-2), is obtained. When the observation difference angle Δθ, which is the difference between the first observation direction Do ₁ and the second observation direction Do ₂ , exceeds a predetermined first threshold (first difference angle threshold THa1), the A corrected correlation is set by applying a dynamic model that describes the difference between the motion trajectories of the first observation point Cr ₁ and the second observation point Cr ₂ from one observation position (No in step S332 in FIG. 8, FIG. 9B , and see FIG. 9C).

As a result, when the observation difference angle Δθ has a specific relationship, the difference in the positions of the first observation point Cr ₁ and the second observation point Cr ₂ can be more appropriately estimated, and the difference in the motion trajectory between them can be more accurately estimated. A corrected correlation that is suitably reflected can be set. As a result, the accuracy of determining object identity can be further improved.

In addition, in the object recognition method of this embodiment, the first difference angle threshold THa1 is set to the first observation point Cr ₁ when it is assumed that the first surrounding object O1 and the second surrounding object O2 are the same object. and the second observation point Cr ₂ are determined from the viewpoint of determining whether or not they exist on the same observation plane (for example, observation plane H1 in FIG. 9A) on the surrounding object O.

That is, the first difference angle threshold THa1 is such that the difference between the first absolute motion vector (VU ₁ , VW ₁ ) and the second absolute motion vector (VU ₂ , VW ₂ ) changes significantly when the surrounding object O turns. This will be set as a boundary point (a point where the consistency of the respective movement trajectories of the first observation point Cr ₁ and the second observation point Cr ₂ is significantly lost). Therefore, after the observed difference angle Δθ crosses the first difference angle threshold THa1 set in this way, the correction of the correlation by applying the above dynamic model will be performed, so the accuracy of determining object identity will be can be further enhanced.

Furthermore, a two-wheel model is used as a dynamic model. If the observed difference angle Δθ exceeds the first difference angle threshold THa1 and is less than or equal to the predetermined second threshold (second difference angle threshold THa2) (No in step S332 and Yes in step S333), the first movement The second absolute motion vector (VU ₂ , VW ₂ ), which is one of the vector and the second motion vector, is regarded as the motion vector of the wheels (especially the rear wheels) in the two-wheel model, and the other first absolute motion vector (VU ₁ , VW ₁ ) is regarded as the motion vector of the side of the vehicle body in the two-wheeled model. Thereby, the mutual deviation between the first absolute motion vector (VU ₁ , VW ₁ ) and the second absolute motion vector (VU ₂ , VW ₂ ) caused by the rotation of the surrounding object O is evaluated. Further, as the corrected correlation, a first corrected correlation (movement vector difference ΔV ₁₂ after the first correction) in which this deviation is reflected in the correlation is set.

Further, if the observed difference angle Δθ exceeds the predetermined second difference angle threshold THa2 (No in step S333), the first absolute motion vector (VU ₁ , VW ₁ ) is regarded as the motion vector of the front wheel in the two-wheel model, and the other second absolute motion vector (VU ₂ , VW ₂ ) is regarded as the motion vector of the rear wheel in the two-wheel model. Thereby, the mutual deviation between the first absolute motion vector (VU ₁ , VW ₁ ) and the second absolute motion vector (VU ₂ , VW ₂ ) caused by the rotation of the surrounding object O is evaluated. Further, as the corrected correlation, a second corrected correlation (movement vector difference ΔV ₁₂ after second correction) in which this deviation is reflected in the correlation is set.

As a result, when the surrounding object O is a vehicle or a similar traveling object, by using an existing two-wheel model, the first absolute motion vector when turning the surrounding object O can be calculated without constructing a new dynamic model. It is possible to set a corrected correlation in which the deviation between (VU ₁ , VW ₁ ) and the second absolute motion vector (VU ₂ , VW ₂ ) is suitably reflected.

(Third embodiment)
The third embodiment will be described below. Note that elements similar to those in the first embodiment or the second embodiment are denoted by the same reference numerals, and explanations thereof will be omitted.

FIG. 10 is a flowchart illustrating the object recognition method of this embodiment. As shown in the figure, in this embodiment, the specific aspects of the coordinate unification process related to step S200' are different from the coordinate unification process (step S200 in FIG. 3) described in the first embodiment.

Specifically, in step S200', the control device 20-2 determines the object relative coordinates (u, w), the relative motion vector (vu, vw), and the relative motion vector direction angle vb relative to the second vehicle 10-2. The display is converted into a display on the second vehicle coordinate system as a reference. That is, as a common coordinate system used for determining object identity, the second vehicle coordinate system is used instead of the absolute coordinate system described in the first embodiment. Then, the control device 20-2 executes the processing from step S300 onward according to the first embodiment or the second embodiment, based on each quantity converted to the display on the second vehicle coordinate system.

In this way, even when the position information and motion vector information of the surrounding object O are displayed on the second vehicle coordinate system instead of the absolute coordinate system, the information can be displayed without changing the mathematical processing. Similarly, determination of object identity can be performed.

Specifically, by replacing the above equations (1) to (3) with the following equations (4) to (6), it is possible to execute each process after step S300.

Note that in each formula, the characters with "'" are the parts that have been changed to be displayed on the second vehicle coordinate system. In particular, "A'" in the formula represents the deviation angle of the direction of the vehicle 10 with respect to the reference direction (i.e., the direction of the second vehicle 10-2) in the second vehicle coordinate system, as in the case of the absolute coordinate system ( Hereinafter, this will be referred to as "vehicle coordinate system relative direction angle A'"). As is clear from this definition, the vehicle coordinate system relative direction angle A _{' 2} with respect to the second vehicle 10-2, which is the reference, is zero. On the other hand, the vehicle coordinate system relative direction angle A _{' 1} with respect to the first vehicle 10-1 corresponds to a deviation with respect to the direction of the second vehicle 10-2.

Moreover, (X', Y') in formula (4) are the position coordinates of the vehicle 10 displayed in the second vehicle coordinate system. Therefore, (X ₂ ′, Y ₂ ′) representing the position of the second vehicle 10-2 in the second vehicle coordinate system coincides with the origin (0, 0). On the other hand, (X ₁ ', Y ₁ ') representing the position of the first vehicle 10-1 in the second vehicle coordinate system corresponds to the relative position of the second vehicle 10-2 with respect to the first vehicle 10-1.

Therefore, for example, when applying the second object relative coordinates (u ₂ , w ₂ ) to the object relative coordinates (u, w) in equation (4), X', Y', and A' are all 0. Therefore, the first term on the right side becomes a 0 vector, and the matrix of the second term becomes a unit matrix. Therefore, in this case, the second object relative coordinates (u ₂ , w ₂ ) remain unchanged with respect to the transformation of equation (4). Similarly, when applying the second relative motion vector (vu ₂ , vw ₂ ) and the second relative motion vector direction angle vb ₂ to the conversion expressed by equations (5) and (6), these The value of will remain unchanged.

According to this embodiment having the configuration described above, the following effects are achieved.

In the object recognition method of this embodiment, when the control device 20-2 that executes the same object determination process is installed in the second vehicle 10-2 serving as the second observation position, the first object relative coordinates (u ₁ , w ₁ ) and the first relative motion vectors (vu ₁ , vw ₁ ) are converted into a coordinate system (second vehicle coordinate system) with the second vehicle 10-2 as a reference as a common coordinate system.

As a result, in the case where the control device 20-2 that executes the same object determination process is placed at the observation position (second vehicle 10-2) where the surrounding object O is observed, each quantity is calculated from the position of the second vehicle 10-2. It is possible to perform the same object determination process by representing the coordinate system based on the reference coordinate system. Therefore, when the second vehicle 10-2, which is a moving body, is equipped with the control device 20-2 for executing the object recognition method of the present embodiment, the position of the second vehicle 10-2 itself is used as a reference. Since each calculation can be executed in the coordinate system, convenience is improved.

Although the embodiments of the present invention have been described above, each of the above embodiments merely shows a part of the application examples of the present invention, and the technical scope of the present invention is limited to the specific configuration of the above embodiments. isn't it.

For example, the result of determining that the first surrounding object O1 and the second surrounding object O2 are the same surrounding object O is used in the driving control (drive control and steering control) of the second vehicle 10-2 in automatic driving control or driving support control. It may also be used as appropriate.

Furthermore, in each of the above embodiments, an example has been described in which the control device 20-2 of the second vehicle 10-2 is configured with functions for executing each process related to the object recognition method. However, the present invention is not limited to this, and the function may be configured in the control device 20-1 of the first vehicle 10-1. Further, the function may be realized by distributed processing by the control device 20-1 of the first vehicle 10-1 and the control device 20-2 of the second vehicle 10-2. Furthermore, the function may be configured in an external server (such as a cloud) other than the first vehicle 10-1 and the second vehicle 10-2.

Furthermore, in each of the above embodiments, a case has been described in which the surrounding object O is observed at two locations (first vehicle 10-1 and second vehicle 10-2). However, when observing the surrounding object O from three or more observation positions, the object recognition method described in the above embodiment may be applied. In this case, for example, one or more combinations of two observation positions are selected from among three or more observation positions, the processing described in the above embodiment is executed for each selected combination, and the resulting plurality of Processing results can be integrated as appropriate.

Furthermore, in each of the embodiments described above, an example has been described in which both the first observation position and the second observation position are configured in the vehicle 10, which is a moving body. However, the present invention is not limited to this, and if at least one of the first observation position and the second observation position is configured as a fixed observation facility (for example, a fixed observation facility such as a fixed point camera installed on a road), the above-mentioned Processing related to the object recognition method described in the embodiment can also be applied. Further, the first observation position and the second observation position are configured by arbitrary observation means other than the vehicle 10, and the surrounding object O is also an arbitrary observation target other than those moving on the road such as a vehicle, and each of the above embodiments The object recognition method described above may be applied while being modified as appropriate.

Furthermore, when applying the object recognition method described in each of the above embodiments to an observation system consisting of arbitrary observation means and observation targets, it is assumed that the recognition accuracy may be reduced by regarding the surrounding object O as a rigid body. Ru. In other words, if the same surrounding object O is accompanied by deformation, the relative positional relationship between the first observation point Cr ₁ and the second observation point Cr ₂ in the surrounding object O changes over time, so that the first It is assumed that a deviation between the absolute motion vector (VU ₁ , VW ₁ ) and the second absolute motion vector (VU ₂ , VW ₂ ) occurs. Therefore, in this case, the deformation characteristics of the surrounding object O are acquired, and the motion vector difference ΔV ₁₂ or the motion vector difference threshold THv used for the determination in step S340 (FIG. 5 or FIG. 7) is appropriately determined in consideration of the deformation characteristics. It may be corrected.

Further, in the above embodiment, as a common coordinate system used for determining object identity, an absolute coordinate system based on GPS information or the like, or a control device 20-2 equipped with a control device 20-2 that mainly executes processing related to an object recognition method is used. An example in which a coordinate system based on the position of two vehicles 10-2 is adopted has been described. However, as a common coordinate system used for determining object identity, any suitable coordinate system other than these may be adopted as appropriate. Furthermore, without necessarily using a common coordinate system, first object relative coordinates (u ₁ , w ₁ ), first relative motion vector (vu ₁ , vw ₁ ), second object relative coordinates (u ₂ , w ₂ ), The same object determination process may be performed using the second relative motion vectors (vu ₂ , vw ₂ ) as they are. In this case, the motion vector difference ΔV ₁₂ (between the first relative motion vector (vu ₁ , vw ₁ ) and the second relative motion vector (vu ₂ , vw ₂ ) used for the determination in step S340 (FIG. 5 or 7) The difference) or the motion vector difference threshold THv is appropriately corrected in consideration of the deviation of these relative motion vectors caused by the movement of the first vehicle 10-1 and the second vehicle 10-2 themselves, which are the observation positions.

Furthermore, an object recognition program for causing the control device 20, which is a computer, to execute the object recognition method described in each of the above embodiments, and a storage medium storing the object recognition program are also described in the specification etc. at the time of filing of this application. Included within the scope of the stated matters.

1 Peripheral sensor 2 Internal sensor 3 External communication device 10 Vehicle 10-1 First vehicle 10-2 Second vehicle 11 Self-position calculation section 13 Object relative position calculation section 14 Relative motion vector calculation section 15 Object recognition processing section 20 Control device 24 Coordinate unification section 25 Same object determination processing section 26 Object integration processing section

Claims (9)

An object recognition method that recognizes surrounding objects that are objects that exist around each from multiple observation positions, the method comprising:
Obtaining a first object relative position that is a relative position of a first surrounding object with respect to a first observation position, and a first relative motion vector that is a motion vector of the first surrounding object with respect to the first observation position,
Obtaining a second object relative position that is a relative position of a second surrounding object with respect to a second observation position, and a second relative motion vector that is a motion vector of the second surrounding object with respect to the second observation position,
Based on the first object relative position, the first relative motion vector, the second object relative position, and the second relative motion vector, whether the first surrounding object and the second surrounding object are the same. Execute the same object determination process to determine the
The first object relative position, the first relative motion vector, the second object relative position, and the second relative motion vector are each converted into a common coordinate system to obtain a first object position, a first motion vector, and a second object relative position. Determine the object position and the second motion vector,
In the same object determination process,
extracting the first surrounding object and the second surrounding object for which a mutual distance between the first object position and the second object position is equal to or less than a predetermined threshold;
determining whether the first motion vector of the extracted first surrounding object and the second motion vector of the second surrounding object satisfy a predefined correlation;
determining whether the surrounding object is turning based on a temporal change in at least one direction of the first motion vector and the second motion vector;
When it is determined that the surrounding object is rotating,
setting a corrected correlation corrected for when the surrounding object turns as the correlation;
If it is determined that the first motion vector and the second motion vector satisfy the corrected correlation, the first surrounding object and the second surrounding object are considered to be the same;
Object recognition method. The object recognition method according to claim 1 ,
The correlation is a relationship in which a motion vector difference, which is a difference between the first motion vector and the second motion vector, is less than or equal to a predetermined motion vector difference threshold;
Object recognition method. The object recognition method according to claim 1 or 2 ,
obtaining a first observation direction that is an observation direction of the first surrounding object by the first observation position;
obtaining a second observation direction that is an observation direction of the second surrounding object by the second observation position;
If the difference between the first observation direction and the second observation direction exceeds a predetermined first threshold, a first observation point and a second observation point from the first observation position according to the rotation of the surrounding object. setting the corrected correlation by applying a dynamic model that describes the difference between the respective movement trajectories of the second observation point from
Object recognition method. The object recognition method according to claim 3 ,
The first threshold value is
When it is assumed that the first surrounding object and the second surrounding object are the same object, the first observation point and the second observation point exist on the same observation plane on the surrounding object. determined from the perspective of determining whether or not
Object recognition method. The object recognition method according to claim 4 ,
Using a two-wheel model as the dynamic model,
If the difference between the first observation direction and the second observation direction exceeds the first threshold and is less than or equal to a predetermined second threshold,
By regarding one of the first motion vector and the second motion vector as a motion vector of a wheel in the two-wheeled model, and regarding the other as a motion vector of a side surface of the vehicle body in the two-wheeled model, Evaluating the mutual deviation of the first motion vector and the second motion vector,
setting a first corrected correlation in which the deviation is reflected in the correlation as the corrected correlation;
Object recognition method. The object recognition method according to claim 5 ,
Using a two-wheel model as the dynamic model,
If the difference between the first observation direction and the second observation direction exceeds a predetermined second threshold,
By regarding one of the first motion vector and the second motion vector as the motion vector of the front wheel in the two-wheel model and the other as the motion vector of the rear wheel in the two-wheel model, Evaluating the mutual deviation of the first motion vector and the second motion vector,
setting a second corrected correlation in which the deviation is reflected in the correlation as the corrected correlation;
Object recognition method. The object recognition method according to any one of claims 1 to 6 ,
When a control device that executes the same object determination process is installed at the second observation position,
converting the first object relative position and the first relative motion vector into a coordinate system based on the second observation position as the common coordinate system;
Object recognition method. The object recognition method according to any one of claims 1 to 7 ,
carried out by setting a first vehicle and a second vehicle equipped with sensors for detecting surrounding conditions at the first observation position and the second observation position, respectively;
Object recognition method. An object recognition device that recognizes surrounding objects that are objects that exist around each from a plurality of observation positions,
Comprising an object relative position acquisition unit, a relative motion vector acquisition unit, and an identical object determination processing unit,
The object relative position acquisition unit includes:
Obtaining a first object relative position that is the relative position of the first surrounding object with respect to the first observation position, and a second object relative position that is the relative position of the second surrounding object with respect to the second observation position,
The relative motion vector acquisition unit includes:
a first relative motion vector that is a motion vector of the first surrounding object relative to the first observation position; and a second relative motion vector that is a motion vector of the second surrounding object relative to the second observation position. get
The first object relative position, the first relative motion vector, the second object relative position, and the second relative motion vector are each converted into a common coordinate system to obtain a first object position, a first motion vector, and a second object relative position. Determine the object position and the second motion vector,
The same object determination processing unit includes:
extracting the first surrounding object and the second surrounding object for which a mutual distance between the first object position and the second object position is equal to or less than a predetermined threshold;
determining whether the first motion vector of the extracted first surrounding object and the second motion vector of the second surrounding object satisfy a predefined correlation;
determining whether the surrounding object is turning based on a temporal change in at least one direction of the first motion vector and the second motion vector;
When it is determined that the surrounding object is rotating,
setting a corrected correlation corrected for when the surrounding object turns as the correlation;
If it is determined that the first motion vector and the second motion vector satisfy the corrected correlation, the first surrounding object and the second surrounding object are considered to be the same;
Object recognition device.

Images