JP6908993B2 - Information processing device, information processing method, and driving support system - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and a driving support system.

Conventionally, a recognition device for recognizing an attitude angle of a vehicle in front traveling in front of the vehicle with respect to the vehicle traveling direction is known. The front vehicle recognition device disclosed in Patent Document 1 includes a first detection means for detecting the inter-vehicle distance between the own vehicle and the vehicle in front. Further, the front vehicle recognition device has a size in each of the left and right images of the symmetrical portion based on a partial image extracted from an image input by the image input means for a predetermined symmetrical portion of the preceding vehicle. A second detecting means for detecting the distance between the symmetrical portions is provided. Further, the front vehicle recognition device is relative to the own vehicle of the vehicle in front based on the inter-vehicle distance detected by these detecting means, the size of the symmetrical portion in the left and right images, and the distance between the symmetrical portions. It is provided with a calculation means for calculating the target posture angle.

Japanese Unexamined Patent Publication No. 3-535

In the conventional technology, it is not considered to grasp the behavior of another vehicle that is detected to exist around the own vehicle. If the situation of another vehicle that affects the own vehicle, such as the behavior of the other vehicle, can be appropriately determined, the burden on the driver in automatic parking or the like can be reduced, for example.

An object of the present invention is to provide a technique capable of appropriately determining the situation of another vehicle existing around the own vehicle in view of the above problems.

The information processing device according to the present invention is an information processing device that processes information about another vehicle existing in the vicinity of the own vehicle, and the first information of the other vehicle is obtained from a camera image acquired by the camera of the own vehicle. The first information generation unit that generates the second information, the second information generation unit that generates the second information of the other vehicle based on the detection result of the radar device of the own vehicle, and the first information and the second information. It is a configuration (first configuration) including a third information generation unit that uses the third information generation unit to generate the third information based on the attitude of the other vehicle.

Further, the information processing device having the first configuration preferably further includes a posture change detection unit that detects a posture change of the other vehicle based on the third information (second configuration).

Further, the information processing device having the second configuration stores information for tilt estimation, which classifies the relationship between the width of the front or rear surface of the vehicle and the width of the side surface and the inclination value of the vehicle for each type of vehicle. A storage unit, a tilt derivation unit for deriving the inclination of the other vehicle based on the width and side width of the front or rear surface of the other vehicle detected from the camera image, and the information for estimating the inclination are further provided. The configuration (third configuration) is preferable.

Further, the information processing device having the third configuration further includes an action determination unit that determines the behavior of the other vehicle based on the information acquired from the attitude change detection unit and the inclination derivation unit (fourth configuration). The configuration of) is preferable.

Further, in the information processing device having any of the first to fourth configurations, the first information is information about a predetermined corner portion of the other vehicle, and the second information is obtained from the radar device. The information regarding the position of the other vehicle, and the third information is the distance difference information between the predetermined corner portion and the position of the other vehicle obtained from the radar device (fifth configuration). You can.

Further, in the information processing device having the first or second configuration, the first information is information about the inclination of the other vehicle, and the second information is the information of the transmitted wave from the radar device in the other vehicle. It is the distribution information of the reflection points, and the third information may be a configuration (sixth configuration) which is the shortest distance position information of the other vehicle based on the own vehicle.

In addition, the information processing device having the sixth configuration stores information for tilt estimation that classifies the relationship between the width of the front or rear surface of the vehicle and the side width and the tilt value of the vehicle for each type of vehicle. The first information generation unit further includes a storage unit, and the first information generation unit tilts the other vehicle based on the width and side width of the front or rear surface of the other vehicle detected from the camera image and the information for estimating the inclination. It is preferable that the configuration is such that information about the above is generated (seventh configuration).

Further, the information processing device having the sixth or seventh configuration may further include an action determination unit that determines the behavior of the other vehicle based on the third information (eighth configuration).

The driving support system according to the present invention is automatically operated based on the camera, the radar device, the information processing device having any of the first to eighth configurations, and the information output from the information processing device. It is a configuration (nineth configuration) including a driving support device that makes a determination regarding support control.

The information processing method according to the present invention is an information processing method for processing information on another vehicle existing in the vicinity of the own vehicle by an information processing device, and the other vehicle is obtained from a camera image acquired by the camera of the own vehicle. The first information generation step of generating the first information of the above, the second information generation step of generating the second information of the other vehicle based on the detection result of the radar device of the own vehicle, the first information and the first information. It is a configuration (10th configuration) including a third information generation step of generating a third information based on the attitude of the other vehicle using the two information.

According to the present invention, it is possible to provide a technique capable of appropriately determining the situation of another vehicle existing around the own vehicle.

Block diagram showing the configuration of the parking support system of the first embodiment Schematic diagram for explaining the first information in the first embodiment A flowchart showing a procedure for generating the first information in the first embodiment. Schematic diagram for explaining the second information in the first embodiment A flowchart showing a procedure for generating the second information in the first embodiment. Schematic diagram for explaining the third information in the first embodiment Flow chart showing the posture change detection flow by the posture change detection unit Schematic diagram for explaining the method of deriving the inclination of another vehicle by the inclination deriving unit Flowchart showing the inclination derivation flow of another vehicle by the inclination derivation unit Flow chart showing the behavior judgment flow of other vehicles by the other vehicle behavior judgment department Schematic diagram illustrating a situation in which parking assistance is provided by a parking assistance device Flow chart showing parking support flow by parking support device Block diagram showing the configuration of the parking support system of the second embodiment Schematic diagram for explaining the method of generating the third information in the second embodiment. Block diagram showing the configuration of the driving support system of the third embodiment

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. In the following description, the direction in which the vehicle travels straight from the driver's seat to the steering wheel is referred to as the "forward direction". Further, the direction in which the vehicle travels straight from the steering wheel to the driver's seat is defined as the "rear direction". Further, the direction from the right side to the left side of the driver who is facing the front direction, which is the direction perpendicular to the straight line and the vertical direction of the vehicle, is defined as the "left direction". Further, the direction from the left side to the right side of the driver who is facing the front direction, which is the direction perpendicular to the straight line and the vertical direction of the vehicle, is defined as the "right direction".

<< 1. First Embodiment >>
<1-1. Parking support system configuration>
The first embodiment illustrates a case where the driving support system of the present invention is a parking support system. The parking support system is an example of a driving support system. FIG. 1 is a block diagram showing a configuration of the parking support system SYS1 of the first embodiment. The parking support system SYS1 is a system for automatically driving a vehicle from a current position to a target parking position to park the vehicle. As shown in FIG. 1, the parking support system SYS1 includes an information processing device 1, a camera 2, a radar device 3, and a parking support device 4.

The information processing device 1 performs processing related to information on other vehicles existing in the vicinity of the own vehicle. Here, the own vehicle is a vehicle on which the information processing device 1 is mounted, and in the present embodiment, is a vehicle that receives parking support. The information processing device 1 includes a microcomputer 11 and a storage unit 12. The microcomputer 11 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory) (not shown). The microcomputer 11 processes and transmits / receives information based on the program stored in the storage unit 12. The microcomputer 11 is connected to the camera 2, the radar device 3, and the parking support device 4 by wire or wirelessly. The details of the information processing device 1 will be described later.

The camera 2 is provided in the own vehicle and photographs the surroundings of the own vehicle. In this embodiment, the number of cameras 2 is four. The four cameras 2 are composed of a front camera, a back camera, a left side camera, and a right side camera. The front camera is arranged at the front end of the vehicle, for example, and photographs the front of the vehicle. The back camera is arranged at the rear end of the vehicle, for example, and photographs the rear of the vehicle. The left side camera is arranged on the left side door mirror of the vehicle, for example, and photographs the left side of the vehicle. The right side camera is arranged, for example, on the right door mirror of the vehicle and photographs the right side of the vehicle. The camera 2 transmits the shooting information to the information processing device 1.

The number and arrangement of the cameras 2 are examples. For example, when the purpose is to support only the back parking of the vehicle, the camera 2 may be composed of only the back camera, or may be composed of the back camera and the left and right side cameras.

The radar device 3 is provided in the own vehicle and detects a target around the own vehicle. In this embodiment, the radar device 3 is provided in front of and behind the vehicle. The radar device 3 is mounted inside a bumper, for example. The radar axis of the radar device 3 coincides with the straight-ahead traveling direction of the vehicle. The radar device 3 of the present embodiment transmits and receives a frequency-modulated electromagnetic wave as a radar signal. The radar device 3 transmits a radar signal for each azimuth angle in front of or behind the own vehicle, and receives a reflected signal from a target. The radar device 3 generates a beat signal from the transmission / reception signal, processes the beat signal by a signal processing device such as a microcomputer, and detects target information such as the relative distance, relative velocity, and azimuth of the target. The target is, for example, a vehicle in front or behind.

The number and arrangement of the radar devices 3 are examples. For example, the radar device 3 may be provided only in the rear when the purpose is to support only the back parking of the vehicle.

The parking support device 4 is a device that controls for automatically parking the own vehicle at the target parking position. The parking support device 4 is an example of a driving support device. In the present embodiment, the parking support device 4 includes a route calculation unit 41 and a guidance unit 42. The route calculation unit 41 calculates, for example, a parking route from the current position to the target parking position. The parking space for parking the own vehicle may be automatically detected or may be selected by the driver. The guidance unit 42 includes an engine control unit, a brake control unit, a steering control unit, and a shift control unit (not shown). The guidance unit 42 controls the vehicle to guide the vehicle along the parking route calculated by the route calculation unit 41. The vehicle is automatically parked at the target parking position by the vehicle control of the guidance unit 42.

In the present embodiment, the parking support device 4 determines the automatic parking support control based on the information output from the information processing device 1 that processes the information of the other vehicle. Specifically, the parking support device 4 determines whether or not to perform automatic parking support control based on the information output from the information processing device 1. Details of this point will be described later.

<1-2. Information processing device details>
As shown in FIG. 1, the microcomputer 11 of the information processing apparatus 1 includes a first information generation unit 111, a second information generation unit 112, a third information generation unit 113, a posture change detection unit 114, and a tilt derivation unit. It is provided with 115 and another vehicle behavior determination unit 116. The functions of the respective units 111 to 116 included in the microcomputer 11 are realized by the CPU performing arithmetic processing according to a program.

The first information generation unit 111 generates the first information of the other vehicle from the camera image acquired by the camera 2 of the own vehicle. The first information generation unit 111 does not generate the first information when it is determined that there is no other vehicle in the camera image. In the present embodiment, the first information is information about a predetermined corner portion of another vehicle. The predetermined corner portion may be a corner portion in the left-right direction (hereinafter, may be referred to as a lateral direction) of the front surface or the rear surface of the other vehicle. In the present embodiment, in detail, the predetermined corner portion is a corner portion in the front lateral direction of the following vehicle (another vehicle) reflected in the camera image of the back camera 2. This is because the parking support system SYS1 is a system that assumes that the vehicle is parked in the parking space while being retracted. In the present embodiment, the corner portion corresponds to an end portion (for example, a portion at the left end or a right end of the bumper) of the vehicle body front portion of another vehicle.

FIG. 2 is a schematic diagram for explaining the first information in the first embodiment. FIG. 2 is a schematic diagram of a camera image acquired from the back camera 2. FIG. 3 is a flowchart showing a procedure for generating the first information in the first embodiment. As shown in FIG. 3, the first information generation unit 111 acquires a camera image from the back camera 2 (step S101). For example, a camera image as shown in FIG. 2 is acquired.

The first information generation unit 111 performs edge extraction processing of the acquired camera image (step S102). Here, the edge means a boundary line between light and dark of an image, for example, the outline of the subject corresponds to the edge. The first information processing unit 111 obtains the change in brightness of the image by obtaining the difference in density gradation between each pixel of the camera image and the adjacent pixel, and extracts the change point of the difference in brightness above a certain level as an edge.

The first information generation unit 111 performs the corner extraction process based on the result of the edge process (step S103). In the present embodiment, the first information generation unit 111 extracts the corners in the front lateral direction of the other vehicle. The first information generation unit 111 recognizes the front surface of the other vehicle by using the edge shape pattern data stored in the storage unit 12, and extracts the recognized corner portion in the front lateral direction of the other vehicle.

The first information generation unit 111 calculates the first lateral distance X1 which is the distance from the center position in the front lateral direction of the other vehicle to the corner position (step S104). In FIG. 2, the alternate long and short dash line C1 indicates the center position in the front direction of the other vehicle, and the dotted line L1 indicates the position of the corner portion of the other vehicle. In FIG. 2, the first lateral distance X1 is the distance from C1 to L1. The first lateral distance X1 is an example of the first information of the present invention. The center position C1 in the front lateral direction of the other vehicle can be calculated because the two corners in the front lateral direction of the other vehicle are extracted in step S103.

Returning to FIG. 1, the second information generation unit 112 generates the second information of the other vehicle based on the detection result of the radar device 3 of the own vehicle. In the present embodiment, the second information is compared with the first information. In the present embodiment, the second information is information regarding the position of another vehicle obtained from the radar device 3. In the present embodiment, in detail, the second information is information regarding the position of another vehicle obtained from the radar device 3 provided behind the own vehicle. This is because the parking support system SYS1 is a system that assumes that the vehicle is parked in the parking space while being retracted. For example, when the parking support system SYS1 also supports parking in a parking space while advancing the vehicle, the second information includes information on the position of another vehicle obtained from the radar device 3 provided in front of the own vehicle. It may be.

FIG. 4 is a schematic diagram for explaining the second information in the first embodiment. FIG. 4 is a schematic diagram of a camera image acquired from the back camera 2. FIG. 5 is a flowchart showing a procedure for generating the second information in the first embodiment. As shown in FIG. 5, the second information generation unit 112 acquires the result detected by the radar device 3 behind the own vehicle (step S201). Specifically, the second information generation unit 112 acquires the relative distance, the relative velocity, and the azimuth angle of the target. There may be a plurality of targets, and in that case, target information such as a relative distance is acquired for each target. In this example, as shown in FIG. 4, there are two vehicles behind, and target information for these two vehicles can be obtained.

The second information generation unit 112 converts the coordinate position of the acquired target (step S202). Specifically, the second information generation unit 112 converts the position of the target acquired by the radar device 3 into the coordinate position on the camera image (see FIG. 4). The camera 2 and the radar device 3 are fixed to the own vehicle. Therefore, the coordinate position of the target acquired by the radar device 3 in the real space can be converted into the coordinate position of the camera image in a one-to-one correspondence. The second information generation unit 112 calculates the lateral position of the target with respect to the own vehicle from the acquired target information. Then, the second information generation unit 112 converts this lateral position into the coordinate position on the camera image. As a result, the horizontal position line shown by the dotted line L2 in FIG. 4 is obtained.

In the example shown in FIG. 4, behind the own vehicle, there is a vehicle that faces sideways with respect to the own vehicle in addition to the vehicle that faces the own vehicle in front. Target information is also acquired for the vehicle facing sideways, and based on this, a lateral position line different from the dotted line L2 shown in FIG. 4 is obtained. However, the second information is information to be compared with the first information. The first information is information from a vehicle facing the front of the own vehicle, and information that is determined to be position information of a vehicle different from this vehicle is excluded from the generation target of the second information.

The second information generation unit 112 is the distance from the center position C1 in the front lateral direction of the other vehicle to the lateral position L2 of the other vehicle calculated based on the target information of the radar device 3 in the coordinates on the camera image. The second lateral distance X2 is calculated (step S203). The second lateral distance X2 is the distance from C1 to L2 in FIG. The second lateral distance X2 is an example of the second information of the present invention.

Returning to FIG. 1, the third information generation unit 113 generates the third information based on the posture of the other vehicle by using the first information and the second information. In the present embodiment, the third information serves as an index for determining the situation of the other vehicle. The condition of another vehicle refers to the condition or operation of another vehicle that affects the driving of the own vehicle. In the present embodiment, more specifically, the third information serves as an index for determining the posture change of the other vehicle. In the present embodiment, the situation of other vehicles can be determined based on the third information obtained from the camera 2 and the radar device 3 that monitor the periphery of the own vehicle. Therefore, the situation of another vehicle can be accurately determined as compared with the case where the determination is based on one sensor.

In the present embodiment, the third information is the distance difference information between the predetermined corner portion of the other vehicle obtained from the camera image and the position of the other vehicle obtained from the radar device 3. Specifically, the third information is a difference value between the first lateral distance X1 and the second lateral distance X2. When the other vehicle changes its inclination with respect to the own vehicle, the angle at which the transmitted wave transmitted from the radar device 3 of the own vehicle is reflected by the other vehicle changes. For this reason, the positional relationship between the corner portion in the front lateral direction of the other vehicle and the position of the other vehicle detected by the radar device 3 changes on the camera image. This will be described by way of illustration in FIG.

FIG. 6 is a schematic diagram for explaining the third information in the first embodiment. FIG. 6 is a schematic view of a camera image acquired from the back camera 2. FIG. 6 shows a state in which one vehicle existing behind the own vehicle has changed its posture from the state shown in FIGS. 2 and 4. Further, FIG. 6 shows a first lateral distance X1 generated by the first information generation unit 111 and a second lateral distance X2 generated by the second information generation unit 112.

As shown in FIG. 6, when another vehicle (see FIGS. 2 and 4) whose front surface is visible from the own vehicle tilts and changes to a posture showing the side surface, it is detected by the radar device 3 on the camera image. The position of the car (see the black dot in the figure) approaches the corner in the front lateral direction of the other car. Therefore, the posture change can be detected by detecting the change of the third information which is the difference value between the first lateral distance X1 and the second lateral distance X2. That is, the third information serves as an index for determining the attitude change of another vehicle. According to this configuration, the attitude change of another vehicle can be detected by the third information obtained based on the camera image information and the target information by the radar device 3. Therefore, for example, when the attitude change is determined only by the camera image. Compared with, the posture change can be detected appropriately. Here, the relationship between the information detected by the radar device 3 and the camera image will be described in more detail based on FIGS. 4 and 6. When a plurality of reflection points are detected from one object (following vehicle in this example), the radar device sets the reflection point having the highest signal level among the plurality of reflection points as a representative point, and positions the representative points. Is the position of the object. When the following vehicle faces substantially directly in front of the own vehicle (state shown in FIG. 4), the reflection point S becomes a higher level than other reflection points belonging to the following vehicle and becomes a representative point. Then, when the following vehicle is in a state of being relatively greatly tilted with respect to the own vehicle (state shown in FIG. 6), the reflection point T becomes higher than the other reflection points of the following vehicle and becomes a representative point. Specifically, when the following vehicle is tilted relatively large, at least two reflection points, the reflection point S and the reflection point T, are detected, but the level of the reflection point T that is the closest to the own vehicle. Is larger than the reflection point S. Therefore, the representative point is the reflection point T. In this way, by operating the steering wheel of the following vehicle, the vehicle body of the following vehicle tilts with respect to the own vehicle, and the position of the following vehicle (position of the representative point) changes. Then, based on the change, it is determined which position of the vehicle image of the following vehicle in the image corresponds to the position detected by the radar device.

Returning to FIG. 1, the posture change detection unit 114 detects the posture change of another vehicle based on the third information. When the information processing device 1 is provided with the posture change detection unit 114 as in this configuration, it becomes possible to predict the behavior of another vehicle on the own vehicle side, and the safety can be improved. FIG. 7 is a flowchart showing a posture change detection flow by the posture change detection unit 114.

The posture change detection unit 114 acquires the third information which is the difference value between the first horizontal distance X1 and the second horizontal distance X2 (step S301). The posture change detection unit 114 performs a process of storing the acquired difference value information (step S302). The difference value may be temporarily stored, for example, stored in RAM. The posture change detection unit 114 confirms whether or not a predetermined period has elapsed (step S303). The posture change detection unit 114 waits for the next process until a predetermined period elapses. A predetermined period may be set as appropriate.

When the predetermined period elapses, the posture change detection unit 114 again acquires the difference value between the first horizontal distance X1 and the second horizontal distance X2 (step S304). The posture change detection unit 114 compares the re-acquired difference value with the previously stored difference value, and confirms whether or not there is a change in the difference value (step S305). When it is determined that there is a change in the difference value (Yes in step S305), the attitude change detection unit 114 determines that there is a change in attitude (step S306). When it is determined that there is no change in the difference value (No in step S305), the attitude change detection unit 114 determines that there is no change in attitude (step S307).

Returning to FIG. 1, the tilt derivation unit 115 of the other vehicle, based on the width and side width of the front or rear surface of the other vehicle detected from the camera image and the inclination estimation information stored in the storage unit 12. Derived the slope. The inclination estimation information is information that classifies the relationship between the ratio of the width of the front or rear surface of the vehicle to the side width and the inclination value of the vehicle for each type of vehicle. By deriving the inclination of the other vehicle, it becomes possible to grasp the situation of the other vehicle in detail. Therefore, it becomes possible to accurately predict the behavior of other vehicles on the own vehicle side.

In the present embodiment, the inclination deriving unit 115 derives the inclination of the other vehicle by using the front width and the side width of the other vehicle detected from the camera image. For this purpose, the inclination estimation information is information in which the relationship between the ratio of the front width and the side width of the vehicle and the inclination value of the vehicle is classified for each type of vehicle. These are derived from the fact that the parking support system SYS1 is a system that assumes that the vehicle is parked in the parking space while being retracted. More specifically, the inclination of the other vehicle is a lateral inclination of the back camera 2 of the own vehicle with respect to the optical axis. The front width and the side width are both horizontal widths.

In the present embodiment, the inclination estimation information is stored in the storage unit 12 as table information for each type of vehicle. The inclination estimation information can be obtained by calculation in advance based on the design value and the typical value of the vehicle for each type. The type of vehicle may be, for example, a classification of a small vehicle, an ordinary vehicle, a large vehicle, or the like. As another example, the type of vehicle may be, for example, information for each model of the vehicle. For example, it is assumed that the size of an ordinary car is 1.8 m in width and 5.0 m in length. In this case, on the table of an ordinary car, for example, when the ratio of the front width to the side width is 1.34 (= (1.8 × cos15 °) /5.0 × sin15 °), the inclination of the car is 15 °. Information such that the inclination of the car is 30 ° when the ratio of the front width to the side width is 0.62 (= (1.8 × cos30 °) /5.0 × sin30 °) is described.

FIG. 8 is a schematic diagram for explaining a method of deriving the inclination of another vehicle by the inclination deriving unit 115. FIG. 8 is a schematic view of a camera image acquired from the back camera 2. FIG. 9 is a flowchart showing the inclination derivation flow of another vehicle by the inclination derivation unit 115. As shown in FIG. 9, the tilt deriving unit 115 calculates the front width W1 (see FIG. 8) of the other vehicle reflected in the camera image of the back camera 2 (step S401). In the present embodiment, when the first information generation unit 111 generates the first information, the front surface of the other vehicle is subjected to edge extraction and corner extraction. Using this, the front width W1 can be calculated.

The tilt deriving unit 115 calculates the side width W2 (see FIG. 8) of the other vehicle reflected in the camera image of the back camera 2 (step S402). The side width W2 can be calculated by using the edge-processed image processed when the first information is generated. As shown in FIG. 8, the side surface width W2 is obtained by determining the boundary line position in the left-right direction of the vehicle side surface from the edge-processed image and obtaining the distance between the boundary line positions. The order in which steps 401 and S402 are performed may be reversed from that of this example.

The inclination deriving unit 115 calculates the ratio of the front width W1 and the side width W2 (step S403). If either the front width W1 or the side width W2 is zero, the ratio need not be calculated. When the side width W2 is zero, the inclination value is determined to be 0 °. When the front width W1 is zero, the inclination value is determined to be 180 °.

The tilt derivation unit 115 determines the type of another vehicle reflected in the camera image of the back camera 2 (step S404). For example, the tilt derivation unit 115 determines the type of vehicle from the information written on the license plate displayed on the camera image. Examples of the information written on the license plate include 5 numbers, 3 numbers and the like. Further, as another example, the inclination deriving unit 115 may determine the type of the vehicle from the ratio of the width of the license plate of the other vehicle to the front width of the other vehicle shown in the camera image. The size of the license plate is fixed by the standard. Therefore, the front width of the other vehicle in the real space can be calculated from the ratio of the width of the license plate of the other vehicle to the front width of the other vehicle and the standard value of the width of the license plate shown in the camera image. From the calculated front width of another vehicle in the real space, the vehicle type can be determined based on the predetermined front width data for each vehicle type. The vehicle type determination is not limited to the timing of this example, and may be performed before any of steps S401 to S403.

The inclination deriving unit 115 refers to the inclination estimation information (table information) based on the ratio of the front width W1 and the side width W2 calculated in step S403 and the vehicle type determined in step S404, and others. Derivation of the inclination of the car (step S405).

Returning to FIG. 1, the other vehicle behavior determination unit 116 determines the behavior of the other vehicle based on the information acquired from the attitude change detection unit 114 and the inclination derivation unit 115. In this configuration, not only the inclination change of the other vehicle but also the inclination information of the other vehicle is taken into consideration to determine the behavior of the other vehicle, so that the behavior of the other vehicle can be determined more appropriately.

FIG. 10 is a flowchart showing a behavior judgment flow of another vehicle by the other vehicle behavior judgment unit 116. The other vehicle behavior determination unit 116 acquires information from the attitude change detection unit 114 and confirms whether or not the other vehicle has a posture change (step S501).

When there is a change in the attitude of the other vehicle (step S501), the other vehicle behavior determination unit 116 confirms the relative speed of the other vehicle acquired from the radar device 3. Specifically, the other vehicle behavior determination unit 116 confirms whether or not the relative speed of the other vehicle is other than 0 km / h (step S502). When the relative speed of the other vehicle is other than 0 km / h (Yes in step S502), the other vehicle behavior determination unit 116 determines that the other vehicle is overtaking (step S503). On the other hand, when the relative speed of the other vehicle is 0 km / h (No in step S502), the other vehicle behavior determination unit 116 determines that the other vehicle is stopping (step S504).

In the above example, when the relative speed of the other vehicle is other than 0 km / h, it is immediately determined to be an overtaking action. This is just an example. For example, when the relative speed of the other vehicle is other than 0 km / h, the configuration may be such that the time change of the relative speed of the other vehicle is confirmed. If it is determined that the relative speed of the other vehicle is decelerating and approaching 0 km / h, it may be determined as a stop action, and in other cases, it may be determined as an overtaking action.

When there is no change in the posture of the other vehicle (No in step S501), the other vehicle behavior determination unit 116 acquires the inclination of the other vehicle from the inclination derivation unit 115 (step S505). The other vehicle behavior determination unit 116 confirms whether or not the inclination of the other vehicle is other than 0 ° (step S506).

When the inclination of the other vehicle is other than 0 ° (Yes in step S506), the other vehicle behavior determination unit 116 confirms whether or not the relative speed of the other vehicle is other than 0 km / h (step S507). When the relative speed of the other vehicle is other than 0 km / h (Yes in step S507), the other vehicle behavior determination unit 116 determines that the other vehicle is overtaking (step S508). On the other hand, when the relative speed of the other vehicle is 0 km / h (Yes in step S507), the other vehicle behavior determination unit 116 determines that the other vehicle is stopping (step S509). When the relative speed of the other vehicle is other than 0 km / h as in the modification of step S503 described above, the time change of the relative speed of the other vehicle is confirmed without immediately determining the overtaking behavior. It may be configured to judge the behavior of another vehicle.

When the inclination of the other vehicle is 0 ° (No in step S506), the other vehicle behavior determination unit 116 confirms whether or not the relative speed of the other vehicle is other than 0 km / h (step S510). When the relative speed of the other vehicle is other than 0 km / h (Yes in step S510), the other vehicle behavior determination unit 116 determines that the other vehicle is approaching (step S511). On the other hand, when the relative speed of the other vehicle is 0 km / h (Yes in step S510), the other vehicle behavior determination unit 116 determines that the other vehicle is stopping (step S512). As in the modification of step S503 described above, when the relative speed of the other vehicle is other than 0 km / h, the time change of the relative speed of the other vehicle is confirmed without immediately determining the approaching behavior. It may be configured to judge the behavior of another vehicle.

For example, when the steering wheel is returned to the original position and the vehicle goes straight after changing the inclination of another vehicle, the attitude change detection unit 114 may not detect the attitude change. Even in such a case, the overtaking behavior can be reliably detected by using the inclination information of the other vehicle. Further, even when the inclination of the other vehicle is 0 °, the other vehicle may go straight and approach, and in the configuration of the present embodiment, it is possible to determine the behavior of the other vehicle.

<1-3. Relationship between information processing device and parking support device>
FIG. 11 is a schematic diagram illustrating a situation in which parking support is performed by the parking support device 4. The parking lot P has a parking space P1 surrounded by a plurality of white lines. The own vehicle PC1 is scheduled to be parked in one of the plurality of parking spaces P1 by using the parking support device 4. Another vehicle PC2 exists behind the vehicle PC1 in a direction parallel to the straight-ahead direction.

FIG. 12 is a flowchart showing a parking support flow by the parking support device 4. The parking support control by the parking support device 4 is started, for example, by the driver pressing a parking support button (not shown). The start of the parking support control is not limited to this, and may be started in conjunction with the position of the shift lever, for example. Specifically, it may be initiated when the shift lever is retracted (R).

The parking support device 4 first orders the information processing device 1 to acquire information about another vehicle PC2 for which an action determination should be made (step S601). The information processing device 1 appropriately functions each of the functional units 111 to 116 according to the command to acquire information on the behavior of the other vehicle PC2. In this example, since there is another vehicle PC2 that faces the front of the own vehicle PC1 behind the own vehicle PC1, information on the behavior of the other vehicle PC2 is acquired. If the other vehicle PC2 does not exist behind the own vehicle PC1, the information regarding the behavior of the other vehicle PC2 is not acquired.

The parking support device 4 confirms whether or not the information processing device 1 has acquired information on the behavior of the other vehicle PC2 (step S602). When the information regarding the behavior of the other vehicle PC2 is acquired (Yes in step S602), it is confirmed whether or not the behavior of the other vehicle PC2 is a stop behavior (step S603).

When the information processing device 1 does not acquire information about the behavior of the other vehicle PC2 (No in step S602), or when it is determined that the behavior of the other vehicle PC2 is a stop behavior (Yes in step S603), parking is performed. The support device 4 generates support information (step S604). In the example shown in FIG. 11, since the other vehicle PC2 exists, the support information is generated when the other vehicle PC2 is stopped in that case. The support information includes, for example, parking route information calculated by the route calculation unit 41. The parking support device 4 controls the vehicle based on the generated support information (step S605). Vehicle control includes, for example, control of an engine control unit, a brake control unit, a steering control unit, and a shift control unit of the own vehicle PC1.

On the other hand, when it is determined that the action of the other vehicle PC2 is not a stop action (No in step S603), the parking support device 4 stops the parking support control (step S606). In the example shown in FIG. 11, when the other vehicle PC2 performs the approaching action indicated by the dotted arrow and the overtaking action indicated by the alternate long and short dash arrow, it is determined that the vehicle is not stopped. When the parking support device 4 cancels the parking support control, it is preferable to notify the driver of this. As the means of notification, for example, a display device (not shown) or a voice device provided in the own vehicle PC1 is used.

According to the parking support system SYS1 of the present embodiment, the parking support device 4 acquires the behavior information of the other vehicle PC2 from the information processing device 1 when the driver indicates the intention to perform automatic parking. Determine whether it is okay to continue parking support control. Therefore, according to the parking support system SYS1 of the present embodiment, it is possible to reduce the burden on the driver himself / herself to check the surrounding environment of his / her vehicle when starting automatic parking.

<< 2. Second Embodiment >>
<2-1. Parking support system configuration>
Similar to the first embodiment, the second embodiment also illustrates the case where the driving support system of the present invention is a parking support system. FIG. 13 is a block diagram showing a configuration of the parking support system SYS2 of the second embodiment. The configuration of the parking support system SYS2 of the second embodiment is substantially the same as the configuration of the parking support system SYS1 of the first embodiment. The parts that overlap with the first embodiment will be described with the same reference numerals, and the description will be omitted if there is no particular need. The function exhibited by the microcomputer 51 included in the information processing device 5 of the second embodiment is different from the function exhibited by the microcomputer 11 included in the information processing device 1 of the first embodiment. This difference will be described below.

<2-2. Information processing device details>
Similar to the first embodiment, the microcomputer 51 of the information processing device 5 of the second embodiment also has the first information generation unit 511, the second information generation unit 512, the third information generation unit 513, and the other vehicle behavior determination unit. 514 is provided. The functions of the respective units 511 to 514 included in the microcomputer 51 are realized by the CPU performing arithmetic processing according to the program stored in the storage unit 12.

Similar to the first embodiment, the first information generation unit 511 generates the first information of the other vehicle from the camera image acquired by the camera 2 of the own vehicle. However, in the second embodiment, the first information is information regarding the inclination of another vehicle. This point is different from the first embodiment. The first information generation unit 511 obtains information on the inclination of the other vehicle based on the width and side width of the front or rear surface of the other vehicle detected from the camera image and the inclination estimation information stored in the storage unit 12. Generate. The inclination estimation information is information that classifies the relationship between the ratio of the width of the front or rear surface of the vehicle to the side width and the inclination value of the vehicle for each type of vehicle.

In the present embodiment, in detail, the first information generation unit 511 derives the inclination of the other vehicle by using the front width and the side width of the other vehicle detected from the camera image. For this purpose, the inclination estimation information is information in which the relationship between the ratio between the front width and the side width of the vehicle and the inclination value of the vehicle is classified for each type of vehicle. These are derived from the fact that the parking support system SYS2 is a system that assumes that the vehicle is parked in the parking space while being retracted. In the present embodiment, the first information generation unit 511 does not generate the first information when it is determined that there is no other vehicle in the camera image acquired from the back camera 2.

The method by which the first information generating unit 511 derives the inclination of the other vehicle is the same as the method of deriving the inclination of the inclination deriving unit 115 of the first embodiment. Further, the configuration of the tilt estimation information in the second embodiment is the same as the configuration of the tilt estimation information in the first embodiment. For this reason, these detailed descriptions will be omitted. The first information generation unit 511 can appropriately estimate the inclination of the other vehicle only from the camera image in order to generate information on the inclination of the other vehicle from the camera image using the inclination estimation information prepared in advance. .. The first information generation unit 511 creates a bird's-eye view BW (see FIG. 14 described later) from the inclination of the derived vehicle. In the present embodiment, the shape of the other vehicle viewed from above is approximated to be rectangular. The first information generation unit 511 tilts the rectangle corresponding to the derived tilt to generate the bird's-eye view BW. When the inclination value is zero, the short side of the rectangle is parallel to the left-right direction, and the long side of the rectangle is parallel to the front-back direction.

Similar to the first embodiment, the second information generation unit 512 generates the second information of the other vehicle to be compared with the first information based on the detection result of the radar device 3 of the own vehicle. However, in the second embodiment, the second information is the distribution information of the reflection points of the transmitted wave from the radar device 3 in another vehicle. This point is different from the first embodiment. In the present embodiment, the second information generation unit 512 does not generate the second information when the first information is not generated by the first information generation unit 511.

The radar device 3 detects information on the positions of a plurality of reflection points when the target has a reflection surface having a certain degree of spread. The radar device 3 usually determines a representative reflection point position based on a predetermined reference among the above-mentioned plurality of reflection point positions, and outputs target information of the representative reflection point position to the outside. The number of representative reflection point positions is not necessarily limited to one. The second information generation unit 512 radars not only the target information for the representative reflection point position but also the target information when there are a plurality of other reflection point positions detected by the radar device 3. Obtained from device 3. Then, the second information generation unit 512 is a distribution map DM of the reflection point positions assuming that the target is viewed from above based on the acquired target information of the plurality of reflection point positions (see FIG. 14 described later). To generate.

The third information generation unit 513 generates the third information based on the posture of the other vehicle by using the first information and the second information. Also in this embodiment, the third information serves as an index for determining the situation of the other vehicle. In detail, the third information serves as an index for determining a movement including a change in the posture of another vehicle. In the present embodiment, the third information is the shortest distance position information of another vehicle based on the own vehicle. In other words, the third information is information determined based on the posture of another vehicle. For example, when the other vehicle is facing almost directly in front of the own vehicle, the front grille of the other vehicle is at the shortest distance position, and when the other vehicle is relatively tilted with respect to the own vehicle, the other vehicle is in the shortest distance position. The corner of the other vehicle is the shortest distance position. In the present embodiment, the shortest distance position information of another vehicle is detected based on the camera image information and the target information obtained from the radar device 3. Therefore, the operation of the other vehicle can be appropriately determined as compared with the case where the operation of the other vehicle is determined using only the distance information of the other vehicle obtained from the radar device 3, for example.

FIG. 14 is a schematic diagram for explaining the method for generating the third information in the second embodiment. As shown in FIG. 14, the third information generation unit 513 compares the bird's-eye view BW that reflects the inclination of the other vehicle obtained as the first information with the distribution map DM of the reflection point positions obtained as the second information. .. The third information generation unit 513 estimates the most probable position as the position of another vehicle from the distribution of the reflection point positions. For the estimation, for example, the least squares method or the like may be used. Further, in the estimation, for example, the position estimation may be performed assuming that the above-mentioned representative reflection point position (shown by a square in FIG. 14) always overlaps with another vehicle. Further, the position estimation may be performed by excluding the points that are clearly unsuitable as the reflection point positions of other vehicles from the camera image.

When the third information generation unit 513 estimates the most probable position of another vehicle in the distribution map DM, the third information generation unit 513 generates the shortest distance position information based on this. In the example shown in FIG. 14, the positions of the vertices of two sides orthogonal to each other (indicated by white circles in FIG. 14), which are considered to exist in the vicinity of the plurality of reflection points, are determined to be the shortest distance positions. If there is only one reflection point position, that point may be determined as the shortest distance position.

Returning to FIG. 13, the other vehicle behavior determination unit 514 determines the behavior of the other vehicle based on the third information. As in this configuration, if the information processing device 1 is provided with the other vehicle behavior determination unit 514 to determine the behavior of the other vehicle, it is possible to support automatic driving while appropriately determining the behavior of the other vehicle. become. Therefore, improvement in safety can be expected.

The other vehicle behavior determination unit 514 monitors the change in the shortest distance position of the other vehicle obtained as the third information. The other vehicle behavior determination unit 514 can detect a change in posture by changing the position at the shortest distance. That is, the other vehicle behavior determination unit 514 has a function as a posture change detection unit. As in the first embodiment, the posture change detection unit may be provided separately from the other vehicle behavior determination unit 514. In addition, the other vehicle behavior determination unit 514 can acquire the inclination of the other vehicle from the first information generation unit 511. Further, the other vehicle behavior determination unit 514 can acquire the relative speed of the other vehicle from the radar device 3. Therefore, the other vehicle behavior determination unit 514 can determine the behavior such as overtaking behavior and stopping behavior of the other vehicle by, for example, the same method as in the first embodiment (see FIG. 10).

<< 3. Third Embodiment >>
FIG. 15 is a block diagram showing a configuration of the driving support system SYS3 of the third embodiment. The driving support system SYS3 is applicable to a system that supports automatic parking, but is also applicable to other systems that support automatic driving. This embodiment exemplifies the case where the driving support system SYS3 is an automatic emergency braking (AEB) system.

The schematic configuration of the driving support system SYS3 of the third embodiment is substantially the same as the configuration of the parking support system SYS2 of the second embodiment. The parts that overlap with the second embodiment will be described with the same reference numerals, and the description will be omitted if there is no particular need. The functions exhibited by the microcomputer 61 included in the information processing device 6 of the third embodiment are substantially the same as the functions exhibited by the microcomputer 51 included in the information processing device 5 of the second embodiment, but the other vehicle behavior determination unit It differs in that it does not have 514. Further, the driving support system SYS3 is different in that it includes a driving support device 7 instead of the parking support device 4. Hereinafter, these differences will be described.

Similar to the second embodiment, the third information generation unit 513 of the information processing device 6 generates the shortest distance position information of another vehicle based on the own vehicle as the third information. However, the AEB system is usually a system that prevents the own vehicle from colliding with the vehicle in front. For this purpose, the first information generation unit 511 acquires the image information of the other vehicle by using the front camera 2, and derives the inclination of the other vehicle by using the rear surface width instead of the front width of the other vehicle. Further, the second information generation unit 512 derives the distribution information of the reflection points by using the radar device 3 in front of the own vehicle. The shortest distance position information generated by the third information generation unit 513 is used as an index for determining the state of another vehicle as seen from the own vehicle. In the present embodiment, the main body for determining the state of the other vehicle is the driving support device 7. However, the information processing device 6 may determine the state of another vehicle and output the determination information to the driving support device 7.

For example, when the own vehicle starts moving, the driving support device 7 periodically acquires the third information (shortest distance position information) from the information processing device 6. Then, the driving support device 7 periodically compares the shortest distance between the own vehicle and the other vehicle with a predetermined threshold value based on the acquired third information. When the above-mentioned shortest distance is smaller than a predetermined threshold value, the driving support device 7 determines that another vehicle is abnormally approaching the own vehicle. When the driving support device 7 detects an abnormal approach of another vehicle, the driving support device 7 controls a brake system (not shown) to prevent a collision between the own vehicle and the other vehicle.

In the present embodiment, the driving support device 7 controls the brake system when another vehicle approaches abnormally, but this is an example. For example, the driving support device 7 may be configured to control the steering system instead of the brake system or in addition to the brake system when another vehicle approaches abnormally.

According to the system configuration of the present embodiment, the positional relationship between the other vehicle and the own vehicle is determined based on the camera image and the detection result of the radar device. Therefore, the accuracy of determining the positional relationship can be improved as compared with the case where the positional relationship of another vehicle with the own vehicle is determined using only a single sensor. That is, the collision prevention performance can be improved as compared with the case where only a single sensor is used.

<< 4. Points to keep in mind >>
The various technical features disclosed herein can be modified in addition to the above embodiments without departing from the gist of the technical creation. Further, the plurality of embodiments and modifications shown above may be combined and implemented to the extent possible.

For example, in the above, the information processing devices 1, 5 and 6 are configured to be provided as separate devices from the parking support device 4 and the driving support device 7. However, this is an example, and the information processing devices 1, 5 and 6 may be a part of the parking support device and the driving support device.

Further, in the above, the configuration is such that the attitude change of another vehicle is detected by using the camera image and the detection result by the radar device. In some cases, the attitude change is detected based on the change in the inclination of the other vehicle derived by the inclination deriving unit 115 of the first embodiment or the first information generating unit 511 of the second and third embodiments. The configuration may be adopted.

Further, in the above-described embodiment, it has been described that various functions are realized by software by the arithmetic processing of the CPU according to the program, but at least a part of these functions is electrical hardware. It may be realized by a circuit. On the contrary, at least a part of the functions realized by the hardware circuit may be realized by software.

1, 5, 6 Information processing device 2 Camera 3 Radar device 4 Parking support device (driving support device)
7 Driving support device 12 Storage unit 111, 511 First information generation unit 112, 512 Second information generation unit 113, 513 Third information generation unit 114 Posture change detection unit 115 Tilt derivation unit 116, 514 Other vehicle behavior judgment unit

Claims (10)

An information processing device that processes information about other vehicles that exist around the vehicle.
A first information generation unit that generates first information of the other vehicle from a camera image acquired by the camera of the own vehicle, and
A second information generation unit that generates second information of the other vehicle based on the detection result of the radar device of the own vehicle, and
A third information generation unit that generates third information based on the posture of the other vehicle by using the first information and the second information.
A storage unit that stores tilt estimation information that classifies the relationship between the ratio of the width of the front or rear surface of the vehicle to the width of the side surface and the tilt value of the vehicle according to the type of vehicle.
A tilt derivation unit that derives the tilt of the other vehicle based on the width and side width of the front or rear surface of the other vehicle detected from the camera image and the tilt estimation information.
Information processing device. The information processing device according to claim 1, further comprising a posture change detection unit that detects a posture change of the other vehicle based on the third information. The information processing device according to claim 2 , further comprising an action determination unit that determines the behavior of the other vehicle based on the information acquired from the attitude change detection unit and the inclination derivation unit. The first information is information about a predetermined corner portion of the other vehicle.
The second information is information regarding the position of the other vehicle obtained from the radar device.
The information processing device according to any one of claims 1 to 3 , wherein the third information is distance difference information between the predetermined corner portion and the position of the other vehicle obtained from the radar device. An information processing device that processes information about other vehicles that exist around the vehicle.
A first information generation unit that generates first information of the other vehicle from a camera image acquired by the camera of the own vehicle, and
A second information generation unit that generates second information of the other vehicle based on the detection result of the radar device of the own vehicle, and
A third information generation unit that generates third information based on the posture of the other vehicle by using the first information and the second information.
A storage unit that stores tilt estimation information that classifies the relationship between the ratio of the width of the front or rear surface of the vehicle to the width of the side surface and the tilt value of the vehicle according to the type of vehicle.
With
The first information is information regarding the inclination of the other vehicle.
The first information generation unit generates information on the inclination of the other vehicle based on the width and side surface width of the front or rear surface of the other vehicle detected from the camera image and the information for estimating the inclination. Information processing device. The second information is distribution information of reflection points of the transmitted wave from the radar device in the other vehicle.
The information processing device according to claim 5, wherein the third information is the shortest distance position information of the other vehicle based on the own vehicle. The information processing device according to claim 5 or 6 , further comprising an action determination unit that determines the behavior of the other vehicle based on the third information. With the camera
With the radar device
The information processing device according to any one of claims 1 to 7.
A driving support device that makes a decision regarding automatic driving support control based on the information output from the information processing device, and
A driving support system equipped with. It is an information processing method that uses an information processing device to process information about other vehicles that exist around the vehicle.
The first information generation step of generating the first information of the other vehicle from the camera image acquired by the camera of the own vehicle, and
A second information generation step of generating second information of the other vehicle based on the detection result of the radar device of the own vehicle, and
A third information generation step of generating the third information based on the posture of the other vehicle by using the first information and the second information.
Tilt estimation information that classifies the relationship between the width of the front or rear surface of the vehicle and the side width and the inclination value of the vehicle for each type of vehicle, and the front or rear surface of the other vehicle detected from the camera image. The inclination derivation step of deriving the inclination of the other vehicle based on the width and the side width of
Information processing method. It is an information processing method that uses an information processing device to process information about other vehicles that exist around the vehicle.
The first information generation step of generating the first information of the other vehicle from the camera image acquired by the camera of the own vehicle, and
A second information generation step of generating second information of the other vehicle based on the detection result of the radar device of the own vehicle, and
A third information generation step of generating the third information based on the posture of the other vehicle by using the first information and the second information.
With
The first information is information regarding the inclination of the other vehicle.
The first information generation step is detected from the inclination estimation information in which the relationship between the width of the front or rear surface of the vehicle and the side width and the inclination value of the vehicle is classified for each type of vehicle, and the camera image. An information processing method for generating information on the inclination of the other vehicle based on the width of the front surface or the rear surface of the other vehicle and the width of the side surface.

Images