JP2018103730A - Information processing device, information processing method, and operation support system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique capable of appropriately determining a state of a different vehicle existing around an own vehicle.SOLUTION: An information processing device performs processing relating to information on a different vehicle existing around an own vehicle. The information processing device includes: a first information generation part for generating first information of the different vehicle from a camera image acquired by a camera of the own vehicle; a second information generation part for generating second information of the different vehicle on the basis of a detection result of a radar device of the own vehicle; and a third information generation part for generating third information based on a posture of the different vehicle by using the first information and the second information.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information processing apparatus, an information processing method, and a driving support system.

  2. Description of the Related Art Conventionally, a recognition device for recognizing a posture angle with respect to a traveling direction of a preceding vehicle traveling in front of the own vehicle is known. The forward vehicle recognition device disclosed in Patent Literature 1 includes first detection means for detecting an inter-vehicle distance between the host vehicle and the forward vehicle. Further, the forward vehicle recognizing device has a size in the left and right images of the symmetric portion based on a partial image extracted for a predetermined symmetric portion of the forward vehicle from the image input from the image input means. Second detection means for detecting the distance between the symmetrical portions is provided. Further, the forward vehicle recognition device is configured to detect the relative position of the forward vehicle with respect to the own vehicle based on the inter-vehicle distance detected by these detection means, the size in the left and right images of the symmetrical portion, and the distance between the symmetrical portions. Computation means for computing a desired posture angle is provided.

Japanese Unexamined Patent Publication No. 3-535

  In the conventional technology, it is not considered to grasp the behavior of other vehicles detected to be present around the host vehicle. If the situation of the other vehicle that affects the host vehicle such as the behavior of the other vehicle can be appropriately determined, for example, the burden on the driver in automatic parking or the like can be reduced.

  An object of this invention is to provide the technique which can judge appropriately the condition of the other vehicle which exists around the own vehicle in view of said subject.

  The information processing apparatus according to the present invention is an information processing apparatus that performs processing related to information on other vehicles existing around the own vehicle, and includes first information on the other vehicle from a camera image acquired by the camera of the own vehicle. A first information generation unit that generates a second information generation unit that generates second information of the other vehicle based on a detection result of the radar device of the host vehicle, and the first information and the second information. And a third information generation unit that generates third information based on the attitude of the other vehicle.

  The information processing apparatus having the first configuration preferably has a configuration (second configuration) further including an attitude change detection unit that detects an attitude change of the other vehicle based on the third information.

  In addition, the information processing apparatus having the second configuration stores inclination estimation information in which the relationship between the ratio between the width of the front or rear surface of the vehicle and the width of the vehicle and the inclination value of the vehicle is classified for each type of vehicle. A storage unit; and a tilt deriving unit that derives the tilt of the other vehicle based on the front and rear widths and side widths of the other vehicle detected from the camera image and the tilt estimation information. A configuration (third configuration) is preferable.

  The information processing apparatus having the third configuration further includes a behavior determination unit that determines a behavior of the other vehicle based on information acquired from the posture change detection unit and the inclination derivation unit (fourth). It is preferable that

  In the information processing apparatus having any one of the first to fourth configurations, the first information is information related to a predetermined corner portion of the other vehicle, and the second information is obtained from the radar apparatus. It is information on the position of the other vehicle, and the third information is a configuration (fifth configuration) that is distance difference information between the predetermined corner and the position of the other vehicle obtained from the radar device. It's okay.

  In the information processing apparatus having the first or second configuration, the first information is information related to an inclination of the other vehicle, and the second information is information on a transmitted wave from the radar device in the other vehicle. The distribution information of reflection points, and the third information may be a configuration (sixth configuration) that is the shortest distance position information of the other vehicle based on the host vehicle.

  The information processing apparatus having the sixth configuration stores inclination estimation information in which the relationship between the ratio of the front or rear width of the vehicle and the width of the vehicle and the vehicle inclination value is classified for each type of vehicle. A storage unit, wherein the first information generation unit is configured to determine the inclination of the other vehicle based on the front or rear width and side width of the other vehicle detected from the camera image and the inclination estimation information. It is preferable that it is the structure (7th structure) which produces | generates the information regarding.

  The information processing apparatus having the sixth or seventh configuration may further include a configuration (eighth configuration) further including a behavior determination unit that determines the behavior of the other vehicle based on the third information.

  The driving support system according to the present invention is configured to perform automatic driving based on information output from the camera, the radar device, the information processing device having any one of the first to eighth configurations, and the information processing device. And a driving support device that makes a determination regarding support control (a ninth configuration).

  An information processing method according to the present invention is an information processing method for performing processing related to information on other vehicles existing around the own vehicle by an information processing device, and the other vehicle from a camera image acquired by the camera of the own vehicle. A first information generating step for generating the first information, a second information generating step for generating second information of the other vehicle based on a detection result of the radar device of the own vehicle, the first information, and the first information And a third information generation step of generating third information based on the posture of the other vehicle using two information (tenth configuration).

  ADVANTAGE OF THE INVENTION According to this invention, the technique which can judge appropriately the condition of the other vehicle which exists around the own vehicle can be provided.

The block diagram which shows the structure of the parking assistance system of 1st Embodiment. Schematic diagram for explaining the first information in the first embodiment The flowchart which shows the procedure which produces | generates 1st information in 1st Embodiment. The schematic diagram for demonstrating the 2nd information in 1st Embodiment The flowchart which shows the procedure which produces | generates 2nd information in 1st Embodiment. Schematic diagram for explaining the third information in the first embodiment Flowchart showing detection flow of posture change by posture change detector Schematic diagram for explaining the method of deriving the inclination of another vehicle by the inclination deriving unit Flow chart showing the flow for deriving the inclination of another vehicle by the inclination deriving unit The flowchart which shows the action judgment flow of the other car by the other car action judgment part Schematic diagram illustrating a situation where parking assistance is performed by a parking assistance device The flowchart which shows the parking assistance flow by a parking assistance apparatus The block diagram which shows the structure of the parking assistance system of 2nd Embodiment. Schematic for demonstrating the production | generation method of the 3rd information in 2nd Embodiment. The block diagram which shows the structure of the driving assistance system of 3rd Embodiment.

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. In the following description, the forward traveling direction of the vehicle and the direction from the driver's seat toward the steering is referred to as “front direction”. In addition, the direction in which the vehicle travels straight and is directed from the steering to the driver's seat is referred to as “rearward direction”. Further, the direction from the right side to the left side of the driver who faces the forward direction and is the direction perpendicular to the straight traveling direction of the vehicle and the vertical line is referred to as “left direction”. Further, the direction from the left side to the right side of the driver who faces the forward direction and is the direction perpendicular to the straight traveling direction of the vehicle and the vertical line is referred to as “right direction”.

<< 1. First embodiment >>
<1-1. Configuration of parking support system>
1st Embodiment illustrates the case where the driving assistance system of this invention is a parking assistance system. The parking assistance system is an example of a driving assistance system. FIG. 1 is a block diagram illustrating a configuration of the parking support system SYS1 according to the first embodiment. The parking assist system SYS1 is a system that automatically parks the vehicle from the current position to the target parking position. As illustrated in FIG. 1, the parking support system SYS1 includes an information processing device 1, a camera 2, a radar device 3, and a parking support device 4.

  The information processing apparatus 1 performs processing related to information on other vehicles existing around the host vehicle. Here, the own vehicle is a vehicle on which the information processing apparatus 1 is mounted, and in the present embodiment, is a vehicle that receives parking assistance. The information processing apparatus 1 includes a microcomputer 11 and a storage unit 12. The microcomputer 11 includes a CPU (Central Processing Unit) (not shown), a RAM (Random Access Memory), and a ROM (Read Only Memory). The microcomputer 11 performs information processing and transmission / reception based on a program stored in the storage unit 12. The microcomputer 11 is connected to the camera 2, the radar device 3, and the parking assistance device 4 by wire or wireless. Details of the information processing apparatus 1 will be described later.

  The camera 2 is provided in the own vehicle and photographs the periphery of the own vehicle. In the present embodiment, the number of cameras 2 is four. The four cameras 2 include a front camera, a back camera, a left side camera, and a right side camera. A front camera is arrange | positioned at the front-end part of a vehicle, for example, and image | photographs the front of a vehicle. A back camera is arrange | positioned at the rear-end part of a vehicle, for example, and image | photographs the back of a vehicle. The left side camera is disposed, for example, on the left door mirror of the vehicle and photographs the left side of the vehicle. The right side camera is disposed, for example, on the right door mirror of the vehicle and photographs the right side of the vehicle. The camera 2 transmits shooting information to the information processing apparatus 1.

  The number and arrangement of the cameras 2 are examples. For example, when the purpose is to support only the back parking of the vehicle, the camera 2 may be composed of only the back camera, or may be composed of the back camera and the left and right side cameras.

  The radar device 3 is provided in the own vehicle and detects a target around the own vehicle. In the present embodiment, the radar device 3 is provided in front of and behind the vehicle. The radar device 3 is mounted, for example, inside a bumper. The radar axis of the radar device 3 coincides with the straight traveling direction of the vehicle. The radar apparatus 3 according to the present embodiment transmits and receives frequency-modulated electromagnetic waves as radar signals. The radar apparatus 3 transmits a radar signal for each azimuth angle in front of or behind the host vehicle, and receives a reflection signal from a target. The radar apparatus 3 generates a beat signal from a transmission / reception signal, processes it by a signal processing apparatus such as a microcomputer, and detects target information such as a relative distance, a relative speed, and an azimuth angle of the target. The target is, for example, a front or rear vehicle.

  Note that the number and arrangement of the radar devices 3 are examples. For example, when the purpose is to support only the back parking of the vehicle, the radar device 3 may be provided only in the rear.

  The parking assistance device 4 is a device that performs control for automatically parking the own vehicle at the target parking position. The parking assistance device 4 is an example of a driving assistance device. In the present embodiment, the parking assistance device 4 includes a route calculation unit 41 and a guidance unit 42. For example, the route calculation unit 41 calculates a parking route from the current position to the target parking position. In addition, about the parking space which parks the own vehicle, the structure which detects automatically may be sufficient and the structure which a driver | operator selects may be sufficient. The guide unit 42 includes an engine control unit, a brake control unit, a steering control unit, and a shift control unit (not shown). The guiding unit 42 performs vehicle control for guiding the vehicle along the parking route calculated by the route calculating unit 41. The vehicle is automatically parked at the target parking position by the vehicle control of the guide unit 42.

  In the present embodiment, the parking support device 4 makes a determination regarding automatic parking support control based on information output from the information processing device 1 that performs processing related to information on other vehicles. Specifically, the parking assistance device 4 determines whether to perform automatic parking assistance control based on the information output from the information processing device 1. Details of this point will be described later.

<1-2. Details of Information Processing Device>
As shown in FIG. 1, the microcomputer 11 of the information processing apparatus 1 includes a first information generation unit 111, a second information generation unit 112, a third information generation unit 113, an attitude change detection unit 114, and an inclination derivation unit. 115 and another vehicle action determination unit 116. The functions of these units 111 to 116 included in the microcomputer 11 are realized by the CPU performing arithmetic processing according to a program.

  The 1st information generation part 111 generates the 1st information on other vehicles from the camera picture acquired with camera 2 of the own vehicle. Note that the first information generation unit 111 does not generate the first information when it is determined that there is no other vehicle in the camera image. In the present embodiment, the first information is information related to a predetermined corner portion of another vehicle. The predetermined corner portion may be a corner portion in the left-right direction (hereinafter sometimes referred to as a lateral direction) of the front surface or the rear surface of another vehicle. In the present embodiment, specifically, the predetermined corner portion is a corner portion in the front lateral direction of the succeeding vehicle (other vehicle) reflected in the camera image of the back camera 2. This is because the parking support system SYS1 is a system that assumes that the vehicle is parked in the parking space while retreating. In the present embodiment, the corner portion is a portion corresponding to an end portion (for example, a left end or a right end portion of a bumper) of a vehicle body front portion of another vehicle.

  FIG. 2 is a schematic diagram for explaining the first information in the first embodiment. FIG. 2 is a schematic diagram of a camera image acquired from the back camera 2. FIG. 3 is a flowchart showing a procedure for generating the first information in the first embodiment. As shown in FIG. 3, the first information generation unit 111 acquires a camera image from the back camera 2 (step S101). For example, a camera image as shown in FIG. 2 is acquired.

  The first information generation unit 111 performs edge extraction processing of the acquired camera image (step S102). Here, the edge refers to a light / dark border line of an image, and for example, a contour line of a subject corresponds to the edge. The first information processing unit 111 obtains a change in light / darkness of the image by obtaining a difference in density gradation from each pixel of each pixel of the camera image, and extracts a change point of the light / dark difference of a certain level or more as an edge.

  The first information generation unit 111 performs corner extraction processing based on the result of the edge processing (step S103). In the present embodiment, the first information generation unit 111 extracts a corner portion in the front lateral direction of another vehicle. The first information generation unit 111 recognizes the front surface of the other vehicle using the edge shape pattern data stored in the storage unit 12, and extracts a corner portion in the front lateral direction of the recognized other vehicle.

  The first information generation unit 111 calculates a first lateral distance X1 that is a distance from the center position in the front lateral direction of the other vehicle to the corner position (step S104). In FIG. 2, an alternate long and short dash line C1 indicates the center position of the other vehicle in the front direction, and a dotted line L1 indicates the position of the corner of the other vehicle. In FIG. 2, the first lateral distance X1 is a distance from C1 to L1. The first lateral distance X1 is an example of the first information of the present invention. The center position C1 in the front lateral direction of the other vehicle can be calculated because two corners in the front lateral direction of the other vehicle are extracted in step S103.

  Returning to FIG. 1, the second information generation unit 112 generates the second information of the other vehicle based on the detection result of the radar device 3 of the own vehicle. In the present embodiment, the second information is compared with the first information. In the present embodiment, the second information is information relating to the position of another vehicle obtained from the radar device 3. In the present embodiment, specifically, the second information is information related to the position of another vehicle obtained from the radar device 3 provided behind the host vehicle. This is because the parking support system SYS1 is a system that assumes that the vehicle is parked in the parking space while retreating. For example, when the parking assistance system SYS1 also provides assistance for parking the parking space while moving the vehicle forward, the second information includes information on the position of the other vehicle obtained from the radar device 3 provided in front of the host vehicle. May be.

  FIG. 4 is a schematic diagram for explaining the second information in the first embodiment. FIG. 4 is a schematic diagram of a camera image acquired from the back camera 2. FIG. 5 is a flowchart showing a procedure for generating the second information in the first embodiment. As illustrated in FIG. 5, the second information generation unit 112 acquires a result detected by the radar device 3 behind the host vehicle (step S <b> 201). Specifically, the second information generation unit 112 acquires the relative distance, the relative speed, and the azimuth angle of the target. There may be a plurality of targets, and in such a case, target information such as a relative distance is acquired for each target. In this example, as shown in FIG. 4, there are two vehicles behind, and target information for these two vehicles is obtained.

  The second information generation unit 112 converts the coordinate position of the acquired target (step S202). Specifically, the second information generation unit 112 converts the position of the target acquired by the radar device 3 into a coordinate position on the camera image (see FIG. 4). The camera 2 and the radar device 3 are fixed to the own vehicle. For this reason, the coordinate position in the real space of the target acquired by the radar apparatus 3 can be converted to the coordinate position of the camera image in a one-to-one correspondence. The second information generation unit 112 calculates the lateral position of the target with respect to the own vehicle from the acquired target information. Then, the second information generation unit 112 converts the horizontal position into a coordinate position on the camera image. As a result, a horizontal position line indicated by a dotted line L2 in FIG. 4 is obtained.

  In the example shown in FIG. 4, there is a vehicle that faces sideways with respect to the own vehicle in addition to the vehicle that faces the front of the own vehicle behind the own vehicle. Target information is also acquired for the vehicle facing sideways, and based on this, a lateral position line different from the dotted line L2 shown in FIG. 4 is obtained. However, the second information is information to be compared with the first information. The first information is information from a vehicle facing the front of the vehicle, and information determined as position information of a vehicle different from the vehicle is excluded from the generation target of the second information.

  The second information generation unit 112 has a distance from the center position C1 in the front lateral direction of the other vehicle to the lateral position L2 of the other vehicle calculated based on the target information of the radar device 3 in the coordinates on the camera image. The second lateral distance X2 is calculated (step S203). The second lateral distance X2 is a distance from C1 to L2 in FIG. The second lateral distance X2 is an example of the second information of the present invention.

  Returning to FIG. 1, the third information generation unit 113 generates the third information based on the posture of the other vehicle using the first information and the second information. In the present embodiment, the third information is an index for determining the situation of other vehicles. The situation of the other vehicle indicates a state or operation of the other vehicle that affects the driving of the own vehicle. More specifically, in the present embodiment, the third information is an index for determining a change in the posture of another vehicle. In the present embodiment, the other vehicle situation can be determined based on the third information obtained from the camera 2 and the radar apparatus 3 that monitor the periphery of the host vehicle. For this reason, compared with the case where it judges based on one sensor, the condition of other vehicles can be judged correctly.

  In the present embodiment, the third information is distance difference information between a predetermined corner of the other vehicle obtained from the camera image and the position of the other vehicle obtained from the radar device 3. Specifically, the third information is a difference value between the first lateral distance X1 and the second lateral distance X2. When the other vehicle changes its inclination with respect to the own vehicle, the angle at which the transmission wave transmitted from the radar device 3 of the own vehicle is reflected by the other vehicle changes. For this reason, on the camera image, the positional relationship between the corners in the front lateral direction of the other vehicle and the position of the other vehicle detected by the radar device 3 changes. This will be described with reference to FIG.

  FIG. 6 is a schematic diagram for explaining the third information in the first embodiment. FIG. 6 is a schematic diagram of a camera image acquired from the back camera 2. FIG. 6 shows a state in which the posture of one vehicle existing behind the host vehicle has changed from the state shown in FIGS. 2 and 4. In addition, FIG. 6 illustrates a first lateral distance X1 generated by the first information generation unit 111 and a second lateral distance X2 generated by the second information generation unit 112.

  As shown in FIG. 6, when the other vehicle (see FIGS. 2 and 4) in which only the front surface is visible from the own vehicle is tilted and changed to a posture in which the side surface can be seen, The position of the car (see the black dot in the figure) approaches the corner of the front side of the other car. For this reason, it is possible to detect a change in posture by detecting a change in the third information, which is a difference value between the first lateral distance X1 and the second lateral distance X2. That is, the third information is an index for determining the posture change of the other vehicle. According to this configuration, since it is possible to detect the posture change of the other vehicle based on the third information obtained based on the camera image information and the target information obtained by the radar device 3, for example, when the posture change is determined only from the camera image Compared to the above, it is possible to appropriately detect the posture change. Here, based on FIG.4 and FIG.6, the relationship between the information detected with the radar apparatus 3 and a camera image is demonstrated in detail. When a plurality of reflection points are detected from one object (following vehicle in this example), the radar apparatus uses the reflection point having the highest signal level among the plurality of reflection points as a representative point, and the position of the representative point. Is the position of the object. When the succeeding vehicle is facing substantially in front of the host vehicle (the state shown in FIG. 4), the reflection point S is higher than the other reflection points belonging to the succeeding vehicle and becomes a representative point. Then, when the following vehicle is relatively inclined with respect to the own vehicle (the state shown in FIG. 6), the reflection point T becomes a representative point because the level is higher than the other reflection points of the subsequent vehicle. Specifically, when the succeeding vehicle is relatively inclined, at least two reflection points of the reflection point S and the reflection point T are detected, but the level of the reflection point T that is closest to the own vehicle is detected. Becomes larger than the reflection point S. Therefore, the representative point is the reflection point T. As described above, the position of the succeeding vehicle (the position of the representative point) is changed by tilting the body of the succeeding vehicle relative to the host vehicle by operating the steering wheel of the succeeding vehicle. Based on the change, it is determined which position in the vehicle image of the following vehicle in the image corresponds to the position detected by the radar device.

  Returning to FIG. 1, the posture change detection unit 114 detects the posture change of the other vehicle based on the third information. When the information processing apparatus 1 includes the posture change detection unit 114 as in this configuration, it is possible to predict the behavior of another vehicle on the own vehicle side, and safety can be improved. FIG. 7 is a flowchart showing a posture change detection flow by the posture change detection unit 114.

  The posture change detection unit 114 acquires third information that is a difference value between the first horizontal distance X1 and the second horizontal distance X2 (step S301). The posture change detection unit 114 performs a process of storing the acquired difference value information (step S302). The difference value may be temporarily stored, for example, stored in the RAM. The posture change detection unit 114 checks whether or not a predetermined period has elapsed (step S303). The posture change detection unit 114 waits for the next process until a predetermined period elapses. The predetermined period may be set as appropriate.

  When the predetermined period has elapsed, the posture change detection unit 114 obtains a difference value between the first horizontal distance X1 and the second horizontal distance X2 again (step S304). The posture change detection unit 114 compares the difference value acquired again with the difference value previously stored and checks whether or not there is a change in the difference value (step S305). When it is determined that there is a change in the difference value (Yes in step S305), the posture change detection unit 114 determines that there is a posture change (step S306). When it is determined that there is no change in the difference value (No in step S305), the posture change detection unit 114 determines that there is no posture change (step S307).

  Returning to FIG. 1, the inclination deriving unit 115 detects the other vehicle's front or rear width and side width detected from the camera image and the inclination estimation information stored in the storage unit 12. Deriving the slope. The information for estimating the inclination is information obtained by classifying the relationship between the ratio of the width of the front or rear surface of the vehicle and the width of the vehicle and the inclination value of the vehicle for each type of vehicle. By deriving the inclination of the other vehicle, the situation of the other vehicle can be grasped in detail. For this reason, it becomes possible to accurately predict the behavior of other vehicles on the own vehicle side.

  In the present embodiment, the inclination deriving unit 115 derives the inclination of the other vehicle using the front width and side width of the other vehicle detected from the camera image. For this reason, the inclination estimation information is information in which the relationship between the ratio of the front width and the side width of the car and the inclination value of the car is classified for each type of car. These are derived from the fact that the parking support system SYS1 is a system that assumes that the vehicle is parked in the parking space while retreating. Specifically, the inclination of the other vehicle is an inclination in the lateral direction with respect to the optical axis of the back camera 2 of the own vehicle. The front width and the side width are both horizontal widths.

  In the present embodiment, the inclination estimation information is stored in the storage unit 12 as table information for each type of vehicle. The information for estimating the inclination can be obtained by calculation in advance based on the design value and representative value of the vehicle in each type. The type of vehicle may be, for example, a classification such as a small vehicle, a normal vehicle, or a large vehicle. As another example, the type of car may be information for each car model, for example. For example, assume that the size of a normal vehicle is assumed to be 1.8 m wide and 5.0 m long. In this case, for example, when the ratio of the front width to the side width is 1.34 (= (1.8 × cos15 °) /5.0×sin15°), the inclination of the car is 15 °. When the ratio of the front width to the side width is 0.62 (= (1.8 × cos 30 °) /5.0×sin 30 °), information such as the inclination of the vehicle is 30 ° is described.

  FIG. 8 is a schematic diagram for explaining a method of deriving the inclination of another vehicle by the inclination deriving unit 115. FIG. 8 is a schematic diagram of a camera image acquired from the back camera 2. FIG. 9 is a flowchart showing a flow for deriving the inclination of another vehicle by the inclination deriving unit 115. As illustrated in FIG. 9, the inclination deriving unit 115 calculates the front width W1 (see FIG. 8) of the other vehicle reflected in the camera image of the back camera 2 (step S401). In the present embodiment, when the first information is generated by the first information generation unit 111, edge extraction and corner extraction are performed on the front surface of the other vehicle. By utilizing this, the front width W1 can be calculated.

  The inclination deriving unit 115 calculates the side width W2 (see FIG. 8) of the other vehicle shown in the camera image of the back camera 2 (step S402). The side surface width W2 can be calculated using an edge processed image that has been processed when the first information is generated. As shown in FIG. 8, by using the edge processing image, the boundary line position is determined in the left-right direction of the vehicle side surface, and the distance between the boundary line positions is obtained to obtain the side surface width W2. Note that the order in which step 401 and step S402 are performed may be the reverse of this example.

  The inclination deriving unit 115 calculates the ratio between the front surface width W1 and the side surface width W2 (step S403). In addition, a ratio does not need to be calculated when either one of the front surface width W1 and the side surface width W2 is zero. When the side surface width W2 is zero, the inclination value is determined to be 0 °. When the front width W1 is zero, the inclination value is determined to be 180 °.

  The inclination deriving unit 115 determines the type of the other vehicle shown in the camera image of the back camera 2 (step S404). For example, the inclination deriving unit 115 determines the type of the car from information described on a license plate reflected in the camera image. As information described on the license plate, for example, 5 number, 3 number and the like can be mentioned. As another example, the inclination deriving unit 115 may determine the type of the vehicle from the ratio between the width of the license plate of the other vehicle and the width of the front surface of the other vehicle, which are reflected in the camera image. The size of the license plate is fixed according to the standard. For this reason, the front width of the other vehicle in the real space can be calculated from the ratio between the width of the license plate of the other vehicle and the front width of the other vehicle, and the standard value of the width of the license plate. From the calculated front width of the other vehicle in the real space, the vehicle type can be determined based on the front width data for each predetermined vehicle type. Note that the vehicle type determination is not limited to the timing of this example, and may be performed before any of steps S401 to S403.

  The tilt deriving unit 115 refers to the tilt estimation information (table information) based on the ratio between the front width W1 and the side width W2 calculated in step S403 and the type of the vehicle determined in step S404. The vehicle inclination is derived (step S405).

  Returning to FIG. 1, the other vehicle behavior determination unit 116 determines the behavior of the other vehicle based on information acquired from the posture change detection unit 114 and the inclination derivation unit 115. In this configuration, the behavior of the other vehicle can be determined more appropriately in order to determine the behavior of the other vehicle in consideration of not only the inclination change of the other vehicle but also the tilt information of the other vehicle.

  FIG. 10 is a flowchart showing a behavior determination flow of another vehicle by the other vehicle behavior determination unit 116. The other vehicle behavior determination unit 116 acquires information from the posture change detection unit 114 and confirms whether or not the other vehicle has a posture change (step S501).

  When there is a change in the posture of the other vehicle (step S501), the other vehicle action determination unit 116 confirms the relative speed of the other vehicle acquired from the radar device 3. Specifically, the other vehicle behavior determination unit 116 checks whether or not the relative speed of the other vehicle is other than 0 km / h (step S502). When the relative speed of the other vehicle is other than 0 km / h (Yes in step S502), the other vehicle action determining unit 116 determines that the other car is performing the overtaking action (step S503). On the other hand, when the relative speed of the other vehicle is 0 km / h (No in step S502), the other vehicle behavior determining unit 116 determines that the other vehicle is performing the stopping behavior (step S504).

  In the above example, when the relative speed of the other vehicle is other than 0 km / h, it is immediately determined to be an overtaking action. This is only an example. For example, when the relative speed of the other vehicle is other than 0 km / h, the time variation of the relative speed of the other vehicle may be confirmed. When it is determined that the relative speed of the other vehicle decelerates and approaches 0 km / h, it may be determined to be a stop action, and otherwise, it may be determined to be an overtaking action.

  When there is no change in the posture of the other vehicle (No in step S501), the other vehicle action determination unit 116 acquires the inclination of the other vehicle from the inclination deriving unit 115 (step S505). The other vehicle behavior determination unit 116 checks whether the inclination of the other vehicle is other than 0 ° (step S506).

  When the inclination of the other vehicle is other than 0 ° (Yes in step S506), the other vehicle action determining unit 116 checks whether or not the relative speed of the other vehicle is other than 0 km / h (step S507). When the relative speed of the other vehicle is other than 0 km / h (Yes in step S507), the other vehicle behavior determining unit 116 determines that the other vehicle is performing the overtaking behavior (step S508). On the other hand, when the relative speed of the other vehicle is 0 km / h (Yes in step S507), the other vehicle behavior determining unit 116 determines that the other vehicle is performing the stopping behavior (step S509). If the relative speed of the other vehicle is other than 0 km / h as in the modified example of step S503 described above, the time change of the relative speed of the other vehicle is confirmed without immediately determining that the vehicle is overtaking. It is also possible to determine the behavior of other vehicles.

  When the inclination of the other vehicle is 0 ° (No in step S506), the other vehicle action determination unit 116 checks whether the relative speed of the other vehicle is other than 0 km / h (step S510). When the relative speed of the other vehicle is other than 0 km / h (Yes in step S510), the other vehicle action determining unit 116 determines that the other car is performing an approaching action (step S511). On the other hand, when the relative speed of the other vehicle is 0 km / h (Yes in step S510), the other vehicle behavior determining unit 116 determines that the other vehicle is performing the stopping behavior (step S512). As with the above-described modification of step S503, when the relative speed of the other vehicle is other than 0 km / h, the time change of the relative speed of the other vehicle is confirmed without immediately determining the approaching action. It is good also as a structure which judges the action of another vehicle.

  For example, when the other vehicle changes its inclination and then returns to its original position and moves straight, the posture change detection unit 114 may not detect the posture change. Even in such a case, the overtaking action can be reliably detected by using the inclination information of the other vehicle. Even when the inclination of the other vehicle is 0 °, the other vehicle may go straight ahead and approach, and in the configuration of the present embodiment, it is possible to determine such behavior of the other vehicle.

<1-3. Relationship between Information Processing Device and Parking Support Device>
FIG. 11 is a schematic view illustrating a situation where parking assistance is performed by the parking assistance device 4. The parking lot P has a parking space P1 surrounded by a plurality of white lines. The host vehicle PC1 is scheduled to park in one of the plurality of parking spaces P1 using the parking assistance device 4. There is another vehicle PC2 behind the direction parallel to the straight direction of the host vehicle PC1.

  FIG. 12 is a flowchart showing a parking support flow by the parking support device 4. The parking assistance control by the parking assistance device 4 starts, for example, when the driver presses a parking assistance button (not shown). In addition, the start of parking assistance control is not limited to this, For example, you may start in response to the position of a shift lever. Specifically, it may be started when the shift lever is moved backward (R).

  The parking assistance device 4 first instructs the information processing device 1 to acquire information related to the other vehicle PC2 that should make a behavior determination (step S601). The information processing apparatus 1 causes each of the functional units 111 to 116 to function appropriately according to the instruction, and acquires information related to the behavior of the other vehicle PC2. In this example, since there is another vehicle PC2 facing the front of the host vehicle PC1 behind the host vehicle PC1, information on the behavior of the other vehicle PC2 is acquired. When the other vehicle PC2 does not exist behind the host vehicle PC1, information regarding the behavior of the other vehicle PC2 is not acquired.

  The parking assistance apparatus 4 confirms whether the information regarding the behavior of the other vehicle PC2 has been acquired by the information processing apparatus 1 (step S602). When the information regarding the behavior of the other vehicle PC2 is acquired (Yes in Step S602), it is confirmed whether or not the behavior of the other vehicle PC2 is a stopping behavior (Step S603).

  If the information processing device 1 does not acquire information related to the behavior of the other vehicle PC2 (No in step S602), or if the behavior of the other vehicle PC2 is determined to be a stop behavior (Yes in step S603), parking The support device 4 generates support information (step S604). In the example shown in FIG. 11, since the other vehicle PC2 exists, the support information is generated when the other vehicle PC2 is stopped in that case. The support information includes, for example, parking route information calculated by the route calculation unit 41. The parking assistance device 4 performs vehicle control based on the generated assistance information (step S605). The vehicle control includes, for example, control of an engine control unit, a brake control unit, a steering control unit, and a shift control unit of the host vehicle PC1.

  On the other hand, when it is determined that the action of the other vehicle PC2 is not a stop action (No in step S603), the parking assistance device 4 stops the parking assistance control (step S606). In the example illustrated in FIG. 11, when the other vehicle PC2 performs an approaching action indicated by a dotted line arrow and performs an overtaking action indicated by a one-dot chain line arrow, it is determined that the other vehicle PC2 is not a stopping action. When the parking assistance device 4 stops the parking assistance control, it is preferable to notify the driver of this. As the notification means, for example, a display device (not shown) or a sound device provided in the host vehicle PC1 is used.

  According to the parking support system SYS1 of the present embodiment, the parking support device 4 acquires the behavior information of the other vehicle PC2 from the information processing device 1 when there is an intention to perform automatic parking from the driver, It is determined whether or not the parking support control can be continued. For this reason, according to the parking assistance system SYS1 of the present embodiment, it is possible to reduce the burden of the driver himself / herself confirming the surrounding environment of the own vehicle when starting automatic parking.

<< 2. Second Embodiment >>
<2-1. Configuration of parking support system>
Similarly to the first embodiment, the second embodiment also illustrates a case where the driving support system of the present invention is a parking support system. FIG. 13 is a block diagram illustrating a configuration of the parking support system SYS2 of the second embodiment. The configuration of the parking support system SYS2 of the second embodiment is substantially the same as the configuration of the parking support system SYS1 of the first embodiment. The same parts as those in the first embodiment will be described with the same reference numerals, and the description thereof will be omitted unless particularly necessary. The function exhibited by the microcomputer 51 included in the information processing apparatus 5 of the second embodiment is different from the function exhibited by the microcomputer 11 included in the information processing apparatus 1 of the first embodiment. Hereinafter, this difference will be described.

<2-2. Details of Information Processing Device>
Similarly to the first embodiment, the microcomputer 51 of the information processing apparatus 5 according to the second embodiment also includes a first information generation unit 511, a second information generation unit 512, a third information generation unit 513, and an other vehicle action determination unit. 514. The functions of these units 511 to 514 included in the microcomputer 51 are realized by the CPU performing arithmetic processing according to a program stored in the storage unit 12.

  Similar to the first embodiment, the first information generation unit 511 generates first information of the other vehicle from the camera image acquired by the camera 2 of the own vehicle. However, in 2nd Embodiment, 1st information is the information regarding the inclination of other vehicles. This is different from the first embodiment. The first information generation unit 511 obtains information related to the inclination of the other vehicle based on the front and rear widths and side widths of the other vehicle detected from the camera image and the inclination estimation information stored in the storage unit 12. Generate. The information for estimating the inclination is information obtained by classifying the relationship between the ratio of the width of the front or rear surface of the vehicle and the width of the vehicle and the inclination value of the vehicle for each type of vehicle.

  In the present embodiment, in detail, the first information generation unit 511 derives the inclination of the other vehicle using the front width and side width of the other vehicle detected from the camera image. Therefore, the inclination estimation information is information obtained by classifying the relationship between the ratio of the front width and the side width of the car and the inclination value of the car for each type of car. These are derived from the fact that the parking support system SYS2 is a system that assumes that the vehicle is parked in the parking space while retreating. In the present embodiment, the first information generation unit 511 does not generate the first information when it is determined that there is no other vehicle in the camera image acquired from the back camera 2.

  The method by which the first information generation unit 511 derives the inclination of the other vehicle is the same as the inclination derivation method of the inclination derivation unit 115 of the first embodiment. Further, the configuration of the information for estimating the inclination in the second embodiment is the same as the configuration of the information for estimating the inclination in the first embodiment. For this reason, these detailed description is abbreviate | omitted. The first information generation unit 511 can appropriately estimate the inclination of the other vehicle from only the camera image in order to generate information related to the inclination of the other vehicle from the camera image using the inclination estimation information prepared in advance. . The first information generation unit 511 creates an overhead view BW (see FIG. 14 described later) from the derived inclination of the other vehicle. In the present embodiment, the shape of the other vehicle viewed from above is approximated as a rectangular shape. The first information generation unit 511 generates an overhead view BW by inclining a rectangle corresponding to the derived inclination. When the inclination value is zero, the short side of the rectangle is parallel to the left-right direction, and the long side of the rectangle is parallel to the front-rear direction.

  Similarly to the first embodiment, the second information generation unit 512 generates second information of another vehicle to be compared with the first information based on the detection result of the radar device 3 of the own vehicle. However, in the second embodiment, the second information is the distribution information of the reflection points in the other vehicle of the transmission wave from the radar device 3. This is different from the first embodiment. In the present embodiment, the second information generation unit 512 does not generate the second information when the first information generation unit 511 does not generate the first information.

  The radar device 3 detects information of a plurality of reflection point positions when the target has a reflection surface having a certain extent. The radar apparatus 3 usually determines a representative reflection point position based on a predetermined reference among the plurality of reflection point positions described above, and outputs the target information of the representative reflection point position to the outside. The representative reflection point position is not necessarily limited to one. When there are a plurality of other reflection point positions detected by the radar apparatus 3 as well as the target information for the representative reflection point position, the second information generation unit 512 also converts the target information to the radar. Obtained from device 3. Then, the second information generation unit 512, based on the acquired target information of the plurality of reflection point positions, the reflection point position distribution map DM (see FIG. 14 described later) assuming that the target is viewed from above. Is generated.

  The third information generation unit 513 generates third information based on the attitude of the other vehicle using the first information and the second information. Also in the present embodiment, the third information is an index for determining the situation of other vehicles. Specifically, the third information serves as an index for determining a motion including a posture change of another vehicle. In this Embodiment, 3rd information is the shortest distance position information of the other vehicle on the basis of the own vehicle. In other words, the third information is information determined based on the attitude of the other vehicle. For example, when the other vehicle is in a posture that faces almost directly in front of the own vehicle, the front grill portion of the other vehicle is in the shortest distance position, The corner of the other vehicle is the shortest distance position. In the present embodiment, the shortest distance position information of the other vehicle is detected based on the camera image information and the target information obtained from the radar device 3. For this reason, for example, the operation of the other vehicle can be appropriately determined as compared with the case where the operation of the other vehicle is determined using only the distance information of the other vehicle obtained from the radar device 3.

  FIG. 14 is a schematic diagram for describing a third information generation method according to the second embodiment. As illustrated in FIG. 14, the third information generation unit 513 compares the overhead view BW reflecting the inclination of the other vehicle obtained as the first information with the reflection point position distribution map DM obtained as the second information. . The third information generation unit 513 estimates the most likely position as the position of the other vehicle from the distribution of reflection point positions. For the estimation, for example, a least square method or the like may be used. In the estimation, for example, the position estimation may be performed on the assumption that the above-described representative reflection point positions (indicated by squares in FIG. 14) necessarily overlap with other vehicles. Further, the position may be estimated by excluding points that are clearly estimated as inappropriate from the camera image as the reflection point position of the other vehicle.

  When the third information generating unit 513 estimates the most likely position of the other vehicle in the distribution map DM, the third information generating unit 513 generates the shortest distance position information based on the estimated position. In the example shown in FIG. 14, the vertex positions of two sides that are considered to exist in the vicinity of a plurality of reflection points (indicated by white circles in FIG. 14) are determined as the shortest distance positions. If there is only one reflection point position, that point may be determined as the shortest distance position.

  Returning to FIG. 13, the other vehicle behavior determination unit 514 determines the behavior of the other vehicle based on the third information. As in this configuration, when the information processing apparatus 1 includes the other vehicle behavior determination unit 514 and performs the behavior determination of the other vehicle, it is possible to support automatic driving while appropriately determining the behavior of the other vehicle. become. For this reason, improvement in safety can be expected.

  The other vehicle behavior determination unit 514 monitors the change in the shortest distance position of the other vehicle obtained as the third information. The other vehicle behavior determination unit 514 can detect a change in posture based on a change in the shortest distance position. That is, the other vehicle behavior determination unit 514 has a function as a posture change detection unit. Similar to the first embodiment, a posture change detection unit may be provided separately from the other vehicle behavior determination unit 514. Further, the other vehicle action determination unit 514 can acquire the inclination of the other vehicle from the first information generation unit 511. Further, the other vehicle action determination unit 514 can acquire the relative speed of the other vehicle from the radar device 3. For this reason, the other vehicle behavior determination unit 514 can perform behavior determination such as an overtaking behavior or a stop behavior of the other vehicle, for example, by a method similar to that of the first embodiment (see FIG. 10).

<< 3. Third Embodiment >>
FIG. 15 is a block diagram illustrating a configuration of the driving support system SYS3 of the third embodiment. The driving support system SYS3 can be applied to a system that supports automatic parking, but can also be applied to other systems that support automatic driving. The present embodiment illustrates a case where the driving support system SYS3 is an automatic emergency braking (AEB) system.

  The schematic configuration of the driving support system SYS3 of the third embodiment is substantially the same as the configuration of the parking support system SYS2 of the second embodiment. The same parts as those in the second embodiment will be described with the same reference numerals, and the description thereof will be omitted unless particularly necessary. The function exhibited by the microcomputer 61 included in the information processing apparatus 6 of the third embodiment is substantially the same as the function exhibited by the microcomputer 51 included in the information processing apparatus 5 of the second embodiment, but the other vehicle behavior determination unit The difference is that 514 is not provided. Further, the driving support system SYS3 is different in that it includes a driving support device 7 instead of the parking support device 4. Hereinafter, these differences will be described.

  Similar to the second embodiment, the third information generation unit 513 of the information processing device 6 generates the shortest distance position information of the other vehicle based on the own vehicle as the third information. However, the AEB system is a system that normally prevents the host vehicle from colliding with a preceding vehicle. For this purpose, the first information generation unit 511 acquires the image information of the other vehicle using the front camera 2 and derives the inclination of the other vehicle using the rear width instead of the front width of the other vehicle. The second information generation unit 512 derives reflection point distribution information using the radar device 3 in front of the host vehicle. The shortest distance position information generated by the third information generation unit 513 is used as an index for determining the state of another vehicle viewed from the own vehicle. In the present embodiment, the driving assistance device 7 is the subject that determines the state of the other vehicle. However, the state of the other vehicle may be determined on the information processing device 6 side, and the determination information may be output to the driving support device 7.

  For example, when the host vehicle starts moving, the driving support device 7 periodically acquires third information (shortest distance position information) from the information processing device 6. Then, the driving support device 7 periodically compares the shortest distance between the host vehicle and the other vehicle with a predetermined threshold based on the acquired third information. The driving support device 7 determines that the other vehicle is abnormally approaching the host vehicle when the shortest distance is smaller than a predetermined threshold. When the driving assistance device 7 detects an abnormal approach of another vehicle, the driving assistance device 7 controls a brake system (not shown) to prevent a collision between the own vehicle and the other vehicle.

  In the present embodiment, the driving support device 7 is configured to control the brake system when another vehicle is abnormally approaching, but this is merely an example. For example, the driving support device 7 may be configured to control the steering system instead of the brake system or in addition to the brake system when another vehicle is abnormally approaching.

  According to the system configuration of the present embodiment, the positional relationship between the other vehicle and the own vehicle is determined based on the camera image and the detection result of the radar device. For this reason, compared with the case where the positional relationship with the own vehicle of another vehicle is determined using only a single sensor, the determination accuracy of the positional relationship can be improved. That is, the collision prevention performance can be improved as compared with the case where only a single sensor is used.

<< 4. Points to remember >>
Various technical features disclosed in the present specification can be variously modified within the scope of the technical creation in addition to the above-described embodiment. Further, the plurality of embodiments and modification examples described above may be combined and implemented within a possible range.

  For example, in the above, the information processing devices 1, 5, and 6 are configured to be provided as separate devices from the parking support device 4 and the driving support device 7. However, this is merely an example, and the information processing apparatuses 1, 5, and 6 may be part of a parking assistance apparatus or a driving assistance apparatus.

  Moreover, in the above, it was set as the structure which detects the attitude | position change of another vehicle using a camera image and the detection result by a radar apparatus. In some cases, the attitude change is detected based on the change in the inclination of the other vehicle derived by the inclination deriving unit 115 of the first embodiment or the first information generating unit 511 of the second and third embodiments. A configuration may be employed.

  Further, in the above-described embodiment, it has been described that various functions are realized in software by the arithmetic processing of the CPU according to the program. However, at least a part of these functions is electric hardware. It may be realized by a circuit. Conversely, at least some of the functions realized by the hardware circuit may be realized by software.

1, 5, 6 Information processing device 2 Camera 3 Radar device 4 Parking support device (driving support device)
7 Driving Support Device 12 Storage Unit 111, 511 First Information Generation Unit 112, 512 Second Information Generation Unit 113, 513 Third Information Generation Unit 114 Attitude Change Detection Unit 115 Inclination Derivation Unit 116, 514 Other Vehicle Behavior Determination Unit

Claims (10)

An information processing apparatus that performs processing related to information on other vehicles existing around the vehicle,
A first information generating unit that generates first information of the other vehicle from a camera image acquired by the camera of the own vehicle;
A second information generation unit that generates second information of the other vehicle based on a detection result of the radar device of the host vehicle;
A third information generating unit that generates third information based on an attitude of the other vehicle using the first information and the second information;
An information processing apparatus comprising:   The information processing apparatus according to claim 1, further comprising an attitude change detection unit that detects an attitude change of the other vehicle based on the third information. A storage unit that stores information for estimating a tilt that classifies the relationship between the ratio of the width of the front or rear surface of the vehicle and the width of the side and the vehicle inclination value for each type of vehicle;
An inclination deriving unit for deriving the inclination of the other vehicle based on the front and rear widths and side widths of the other vehicle detected from the camera image, and the inclination estimation information;
The information processing apparatus according to claim 2, further comprising:   The information processing apparatus according to claim 3, further comprising an action determination unit that determines an action of the other vehicle based on information acquired from the posture change detection unit and the inclination derivation unit. The first information is information related to a predetermined corner of the other vehicle,
The second information is information on the position of the other vehicle obtained from the radar device,
5. The information processing apparatus according to claim 1, wherein the third information is distance difference information between the predetermined corner and the position of the other vehicle obtained from the radar apparatus. The first information is information related to the inclination of the other vehicle,
The second information is distribution information of a reflection point in the other vehicle of a transmission wave from the radar device,
The information processing apparatus according to claim 1, wherein the third information is shortest distance position information of the other vehicle with respect to the own vehicle. A storage unit for storing information for estimating a slope in which the ratio between the width of the front or rear surface of the car and the width of the side and the slope value of the car is classified for each type of car;
The first information generation unit generates information related to the inclination of the other vehicle based on the front and rear widths and side widths of the other vehicle detected from the camera image, and the inclination estimation information. The information processing apparatus according to claim 6.   The information processing apparatus according to claim 6, further comprising an action determination unit that determines an action of the other vehicle based on the third information. The camera;
The radar device;
The information processing apparatus according to any one of claims 1 to 8,
A driving support device that makes a determination on support control of automatic driving based on information output from the information processing device;
A driving support system comprising: An information processing method for performing processing related to information on other vehicles existing around the own vehicle by an information processing device,
A first information generation step of generating first information of the other vehicle from a camera image acquired by the camera of the host vehicle;
A second information generation step of generating second information of the other vehicle based on a detection result of the radar device of the own vehicle;
A third information generating step of generating third information based on a posture of the other vehicle using the first information and the second information;
An information processing method comprising:

Images