WO2016129704A1 - Onboard camera calibration apparatus for identifying mounting orientation - Google Patents

Abstract

The camera calibration apparatus is provided with: onboard cameras (5, 6, 7, 8) mounted at predetermined positions in a vehicle; camera orientation detectors (51, 61, 71, 81), provided in the onboard cameras, each of which successively detects a camera orientation index value indicating the orientation of the corresponding onboard camera with respect to the horizontal plane; detection result storage units (M11, M12, M13, M14), for cameras, each of which stores the camera orientation index values obtained by the corresponding camera orientation detector through a predetermined required number of times of detection; and a mounting orientation identification unit (F5) that identifies the mounting orientation of each of the onboard cameras with respect to the vehicle on the basis of the most frequent value among the camera orientation index values accumulated in the corresponding detection result storage unit for the camera.

Description

In-vehicle camera calibration device that identifies mounting orientation

This disclosure relates to an in-vehicle camera. More specifically, the present invention relates to a camera calibration device that calibrates data representing the mounting posture of a vehicle-mounted camera on a vehicle.

Conventionally, there is a camera calibration device that calibrates data representing the mounting posture of a vehicle-mounted camera on a vehicle. For example, in the camera calibration device disclosed in Japanese Patent No. 5606770, the three-axis acceleration sensor having a known positional relationship with the in-vehicle camera detects the acceleration detected when the vehicle is stationary and the vehicle goes straight ahead. Based on the detected acceleration, the mounting posture of the in-vehicle camera is calculated. Then, the calculated mounting posture is adopted as data representing the current mounting posture of the in-vehicle camera. The mounting posture of the in-vehicle camera here corresponds to the shooting direction of the in-vehicle camera with respect to the vehicle.

Japanese Patent No. 5606770

In the method disclosed in Japanese Patent No. 5606770, the influence of the road gradient on the output value of the acceleration sensor is not considered. When the vehicle is on a road with a slope such as a slope, the direction in which the gravitational acceleration acts on the acceleration sensor is affected by the slope of the road, so the ratio of the output value for each axis direction of the acceleration sensor is The ratio is different from the case of being on a horizontal road.

That is, in the method disclosed in Japanese Patent No. 5606770, the inclination of the vehicle body caused by the road gradient is regarded as the inclination of the in-vehicle camera, and there is a possibility that the current mounting posture of the in-vehicle camera is erroneous.

The present disclosure has been made based on this situation, and the purpose of the present disclosure is a camera that can prevent the mounting posture of the in-vehicle camera from being specified as an incorrect posture due to the inclination of the vehicle. It is to provide a calibration device.

The camera calibration apparatus according to the first disclosure of the present application includes an in-vehicle camera (5, 6, 7, 8), a camera attitude detector (51, 61, 71, 81), and a camera detection result storage unit (M11, M12). , M13, M14) and an attachment posture specifying part (F5). The in-vehicle camera is installed at a predetermined position of the vehicle, and a predetermined area around the vehicle is an imaging range. The camera attitude detector is provided in the in-vehicle camera, and sequentially detects a camera attitude index value that represents the attitude of the in-vehicle camera with respect to a horizontal plane that is a plane perpendicular to the direction in which gravity acts. The camera detection result storage unit stores the camera posture index value detected by the camera posture detector for a predetermined required number of detection times. The mounting posture specifying unit specifies the mounting posture of the in-vehicle camera with respect to the vehicle based on the mode value of the camera posture index value stored in the camera detection result storage unit.

In the above configuration, the mounting posture specifying unit specifies the mounting posture of the in-vehicle camera using the mode value of the camera posture index value stored in the camera detection result storage unit. The camera posture index value detected by the camera posture detector is a value affected by the road gradient. The slope of the road corresponds to the slope of the vehicle. In general, since a vehicle is assumed to travel on roads with various slopes, the detection result output by the camera posture detector is also a value affected by various slopes. However, the road with the highest travel frequency is likely to be a horizontal road (a road where the gradient can be ignored).

In other words, if the required number of detections is set to a sufficiently large number, the mode value of the camera posture index value stored in the camera detection result storage unit is the state in which a vehicle is present on a horizontal road. It corresponds to the detection result. That is, the mode value of the camera posture index value stored in the camera detection result storage unit corresponds to a detection result in a state where the vehicle is not tilted.

Therefore, according to the above configuration, it is possible to prevent the mounting posture of the in-vehicle camera from being specified as an incorrect posture due to the inclination of the vehicle.

The camera calibration device according to the second disclosure of the present application includes an in-vehicle camera (5, 6, 7, 8), a camera attitude detector (51, 61, 71, 81), and a camera detection result storage unit (M11, M12, M13, M14), a vehicle posture specifying unit (F4), and an attachment posture specifying unit (F5). The in-vehicle camera is installed at a predetermined position of the vehicle, and a predetermined area around the vehicle is an imaging range. The camera attitude detector is provided in the in-vehicle camera, and sequentially detects a camera attitude index value that represents the attitude of the in-vehicle camera with respect to a horizontal plane that is a plane perpendicular to the direction in which gravity acts. The camera detection result storage unit stores the camera posture index value detected by the camera posture detector for a predetermined required number of detection times. The vehicle attitude specifying unit specifies the attitude of the vehicle with respect to the horizontal plane. The mounting posture specifying unit determines the mounting posture of the in-vehicle camera with respect to the vehicle based on the mode value of the camera posture index value accumulated in the camera detection result storage unit and the vehicle posture specified by the vehicle posture specifying unit. Identify.

In the above configuration, the mode value of the camera posture index value stored in the camera detection result storage unit represents the posture of the in-vehicle camera with respect to the horizontal plane. Of course, the attitude of the in-vehicle camera with respect to the horizontal plane detected by the camera attitude detector includes the influence of the vehicle inclination caused by the road gradient. The presence / absence or the magnitude of the influence of the inclination of the vehicle is specified by the vehicle posture specifying unit. The mounting posture specifying unit specifies the mounting posture of the in-vehicle camera with respect to the vehicle from the vehicle posture specified by the vehicle posture specifying unit and the posture of the in-vehicle camera with respect to the horizontal plane detected by the camera posture detector.

Therefore, according to the above configuration, it is possible to prevent the mounting posture of the in-vehicle camera from being specified as an incorrect posture due to the road gradient.

In addition, the code | symbol in the parenthesis described in the claim shows the correspondence with the specific means as described in embodiment mentioned later as one aspect, Comprising: The technical scope of this indication is limited is not.

In the attached drawing:
1 is a block diagram illustrating an example of a schematic configuration of a driving support system 100 according to a first embodiment. It is a conceptual diagram for demonstrating the installation position and detection direction of various inclination sensors. 3 is a block diagram illustrating an example of a schematic configuration of a control unit 1. FIG. It is a flowchart for demonstrating the attitude | position update related process which the control part 1 implements. It is a block diagram which shows an example of schematic structure of the driving assistance system 100A concerning 2nd Embodiment. It is a conceptual diagram for demonstrating the installation position and detection direction of various inclination sensors. It is a block diagram which shows an example of a schematic structure of 1 A of control parts. It is a flowchart for demonstrating the attitude | position update related process which 1A of control parts implement. It is a conceptual diagram for demonstrating the action | operation of the vehicle attitude | position specific part F4. It is a block diagram which shows an example of a schematic structure of the driving assistance system 100B in 3rd Embodiment.

<First Embodiment>
Hereinafter, a first embodiment of the present disclosure will be described with reference to the drawings. FIG. 1 is a diagram illustrating an example of a schematic configuration of a driving support system 100 to which the camera calibration apparatus according to the present embodiment is applied. The driving support system 100 assists the driver in recognizing the situation around the vehicle by converting a captured image of a camera that captures a predetermined area outside the passenger compartment into, for example, a bird's-eye view image and displaying it on a display. is there.

Hereinafter, a vehicle equipped with the driving support system 100 is referred to as a host vehicle. Further, hereinafter, a right-handed three-dimensional coordinate system having a predetermined position in the own vehicle as an origin and an X axis, a Y axis, and a Z axis orthogonal to each other will be introduced, and the positional relationship of each part included in the own vehicle will be described. To do.

The X-axis is parallel to the vehicle front-rear direction and has a positive direction from the rear to the front of the vehicle. The Y-axis is parallel to the vehicle width direction of the host vehicle and Let the direction from the left to the left be the positive direction. The Z-axis is parallel to the height direction of the vehicle, and the direction from the vehicle floor to the roof is defined as a positive direction. As an example, the origin is a point where the distance from the front end of the vehicle to the rear end is equal on the center of the host vehicle, that is, on the center line of the vehicle equidistant from both sides of the host vehicle. Of course, the origin may be another position, for example, a position that is the center of the rear wheel shaft in the vehicle width direction.

The driving support system 100 according to this embodiment includes a control unit 1, a vehicle speed sensor 2, a vehicle body side tilt sensor 3, a display 4, a front camera 5, and a rear camera 6, as shown in FIG.

The control unit 1 controls the operation of the driving support system 100, and each of the vehicle speed sensor 2, the vehicle body side tilt sensor 3, the display 4, the front camera 5, and the rear camera 6 is a well-known in-vehicle network. Are configured to be able to communicate with each other. Details of the control unit 1 will be described later.

The vehicle speed sensor 2 is a sensor that detects the traveling speed of the host vehicle. The vehicle body side inclination sensor 3 is a sensor (that is, an inclination sensor) that detects the inclination of the vehicle body of the host vehicle with respect to a horizontal plane. Here, the horizontal plane refers to a plane perpendicular to the direction in which gravity acts. The vehicle body side inclination sensor 3 of the present embodiment detects the inclination of the vehicle body with respect to a horizontal plane by resolving the rotation angle with respect to each of two mutually orthogonal axes (X0 axis and Y0 axis). Let it be a sensor. The biaxial tilt sensor may be realized using a triaxial acceleration sensor, or may be realized by combining a pendulum and a magnetic sensor.

For example, as shown in FIG. 2, the vehicle body side inclination sensor 3 has a predetermined position (for example, a floor) on the vehicle body so that the X0 axis and the X axis are in the same direction and the Y0 axis is in the same direction as the Y axis. Part) or the like. The rotation angle around the X0 axis (referred to as roll angle θr0) detected by the vehicle body side inclination sensor 3 represents the inclination angle in the vehicle width direction of the vehicle body, and the rotation angle around the Y0 axis (referred to as pitch angle θp0) is Represents the tilt angle in the vehicle longitudinal direction.

In the present embodiment, as an example, it is assumed that the vehicle body side tilt sensor 3 is installed in a portion parallel to the horizontal plane in the host vehicle. Accordingly, the roll angle θr0 and the pitch angle detected by the vehicle body side inclination sensor 3 are detected when the host vehicle is on a horizontal road, except in special circumstances such as when the air pressure of each tire is unbalanced. Both θp0 are 0 degrees.

The vehicle body side tilt sensor 3 sequentially outputs a signal representing the roll angle θr0 and a signal representing the pitch angle θp0 to the control unit 1 sequentially. Here, as an example, for the roll angle θr0, the rotation angle in the direction of the right screw of the X0 axis from the neutral state is represented by a positive value, and the rotation angle in the reverse direction is represented by a negative value. In addition, the pitch angle θp0 represents a rotation angle in the direction of the right screw of the Y0 axis from a neutral state as a positive value, and a rotation angle in the reverse direction as a negative value.

Since the vehicle is a rigid body, the road surface and the bottom of the vehicle body are substantially parallel. That is, the pitch angle θp0 detected by the vehicle body side tilt sensor 3 represents the gradient in the vehicle front-rear direction of the road where the host vehicle exists, and the roll angle θr0 detected by the vehicle body side tilt sensor 3 exists. This represents the gradient of the road in the vehicle width direction. Therefore, when the vehicle front-rear direction and the road extension direction coincide with each other, the pitch angle θp0 detected by the vehicle body side inclination sensor 3 represents the longitudinal gradient of the road, and the vehicle body side inclination sensor 3 detects it. The roll angle θr0 represents the single slope of the road.

The vehicle body side tilt sensor 3 corresponds to the vehicle attitude detector described in the claims, and the pitch angle θp0 and the roll angle θr0 correspond to an example of the vehicle attitude index value described in the claims. Further, the extending direction of the X0 axis and the extending direction of the Y0 axis correspond to the vehicle side detection direction described in the claims.

The display 4 displays text and images based on signals input from the control unit 1. The display 4 is capable of full color display, for example, and can be configured using a liquid crystal display, an organic EL display, or the like. Here, as an example, the display 4 is a display arranged near the center in the vehicle width direction of the instrument panel. As another aspect, the display 4 may be a display provided in the meter unit or a known head-up display.

The front camera 5 is a camera provided to photograph a predetermined range in front of the host vehicle. As the front camera 5, for example, a well-known CMOS camera, CCD camera, or the like whose shooting range is set to a wide angle (for example, an angle of view of 175 °) by a wide angle lens can be used. For example, the front camera 5 may be installed near the center of the front bumper in the vehicle width direction.

Of course, the installation position of the front camera 5 is not limited to the vicinity of the center portion in the vehicle width direction of the front bumper, but is a position that does not obstruct the driver's field of view in front of the host vehicle, for example, near the rearview mirror in the vehicle interior or the upper end of the windshield. It only has to be attached.

The front camera 5 has a vertical direction (vertical direction) and a horizontal direction (horizontal direction) defined in advance. The front camera 5 is mounted so that the orthogonal projection vector onto the horizontal plane of the optical axis, which is the central axis in the photographing direction, is the same direction as the X axis and the horizontal direction is parallel to the Y axis at the predetermined installation location described above. . Video signals taken by the front camera 5 are sequentially output to the control unit 1.

Further, the front camera 5 includes an inclination sensor (hereinafter referred to as a first inclination sensor) 51 that detects the inclination of the front camera 5 with respect to the horizontal plane. The first inclination sensor 51 detects the inclination angle of the front camera 5 with respect to the horizontal plane by dividing it into rotation angles for each of two axes (X1 axis and Y1 axis), similarly to the vehicle body side inclination sensor 3 described above. This is a biaxial tilt sensor.

The first tilt sensor 51 is placed in the housing of the front camera 5 so that the X1 axis of the first tilt sensor 51 coincides with the optical axis of the front camera 5 and the Y1 axis coincides with the lateral direction of the front camera 5. Is provided. That is, when the front camera 5 is attached to the vehicle body in the above-described posture, the orthogonal projection vector of the X1 axis to the XY plane is the same direction as the positive direction of the X axis and the orthogonal projection of the Y1 axis to the XY plane. The vector is in the same direction as the positive direction of the Y axis.

Thereby, the first inclination sensor 51 has an inclination angle (pitch angle θp1) with respect to the horizontal plane of the front camera 5 in the vehicle front-rear direction and an inclination angle with respect to the horizontal plane of the front camera 5 in the vehicle width direction (denoted roll angle θr1). Is detected.

The detection results (pitch angle θp1 and roll angle θr1) of the first tilt sensor 51 are sequentially output to the control unit 1. As with the pitch angle θp0, the roll angle θr1 represents the rotation angle in the direction of the right-handed screw with the X1 axis as the rotation axis from the neutral state as a positive value, and the reverse rotation angle as a negative value. I will do it. In addition, the pitch angle θp1 represents a rotation angle in the direction of the right screw of the Y1 axis from a neutral state as a positive value, and a rotation angle in the reverse direction as a negative value.

The first tilt sensor 51 corresponds to the camera posture detector described in the claims, and the pitch angle θp1 and the roll angle θr1 correspond to an example of the camera posture index value described in the claims. Further, the extending direction of the X1 axis and the extending direction of the Y1 axis correspond to the camera side detection direction described in the claims.

In the present embodiment, the first inclination sensor 51 is configured to be installed in the casing of the front camera 5, but as another aspect, it may be configured to be integrally attached to the outside of the casing.

The rear camera 6 is a camera provided to photograph a predetermined range behind the host vehicle. For example, similar to the front camera 5, the rear camera 6 may be a known CMOS camera, CCD camera, or the like whose shooting range is set to a wide angle by a wide angle lens. The rear camera 6 may be installed, for example, near the center of the rear bumper in the vehicle width direction.

Of course, the installation position of the rear camera 6 is not limited to the vicinity of the center of the rear bumper in the vehicle width direction, and may be attached to a position that does not obstruct the driver's field of view for confirmation behind the rear window, for example, near the upper end of the rear window.

Also in the rear camera 6, the vertical direction (vertical direction) and the horizontal direction (horizontal direction) are defined in advance. In the rear camera 6, the orthogonal projection vector onto the horizontal plane of the optical axis that is the central axis of the imaging direction is the same direction as the negative direction of the X axis, and the left-right direction is parallel to the Y axis at the predetermined installation location described above. It is attached as follows. Video signals captured by the rear camera 6 are sequentially output to the control unit 1.

Further, the rear camera 6 includes a tilt sensor (hereinafter referred to as a second tilt sensor) 61 that detects the tilt of the rear camera 6 with respect to the horizontal plane. The second tilt sensor 61 decomposes the tilt angle of the rear camera 6 with respect to the horizontal plane into rotation angles for two orthogonal axes (X2 axis and Y2 axis) in the same manner as the vehicle body side tilt sensor 3 described above. This is a two-axis tilt sensor to detect.

The second tilt sensor 61 is built in the rear camera 6 so that the X2 axis of the second tilt sensor 61 coincides with the optical axis direction of the rear camera 6 and the Y2 axis coincides with the lateral direction of the rear camera 6. . That is, when the rear camera 6 is attached to the vehicle body in the above-described posture, the orthogonal projection vector of the X2 axis to the XY plane is the same direction as the negative direction of the X axis, and the orthogonal projection of the Y2 axis to the XY plane. The vector is in the same direction as the negative direction of the Y axis.

Thus, the second inclination sensor 61 has an inclination angle (pitch angle θp2) with respect to the horizontal plane of the rear camera 6 in the vehicle front-rear direction and an inclination angle (set to roll angle θr2) with respect to the horizontal plane of the rear camera 6 in the vehicle width direction. Is detected.

The detection results (pitch angle θp2 and roll angle θr2) of the second tilt sensor 61 are sequentially output to the control unit 1. As for the roll angle θr2, the rotation angle in the direction of the right screw with the X2 axis as the rotation axis from the neutral state is expressed as a positive value, and the rotation angle in the reverse direction is expressed as a negative value. In addition, the pitch angle θp2 represents a rotation angle in the direction of the right screw of the Y2 axis from a neutral state as a positive value, and a rotation angle in the reverse direction as a negative value.

The second tilt sensor 61 corresponds to the camera posture detector described in the claims, and the pitch angle θp2 and the roll angle θr2 correspond to an example of the camera posture index value described in the claims. Further, the extending direction of the X2 axis and the extending direction of the Y2 axis correspond to the camera side detection direction described in the claims.

In the present embodiment, the second tilt sensor 61 is installed in the housing of the rear camera 6, but as another embodiment, the second tilt sensor 61 may be integrally attached to the outside of the housing.

Hereinafter, when the front camera 5 and the rear camera 6 are not distinguished from each other, they are simply referred to as an in-vehicle camera. Further, when the vehicle body side tilt sensor 3, the first tilt sensor 51, and the second tilt sensor 61 are not distinguished, they are referred to as tilt sensors. In particular, the first tilt sensor 51 and the second tilt sensor 61 that are tilt sensors attached to the in-vehicle camera are referred to as camera-side tilt sensors.

The control unit 1 is configured as a normal computer, and includes a well-known CPU 11, a memory 12, a storage 13, an input / output interface (hereinafter referred to as I / O) 14, and a bus line that connects these configurations. .

The CPU 11 is a well-known central processing unit, and executes various arithmetic processes by using the memory 12 as an arithmetic area. The memory 12 may be realized by a temporary storage medium such as a RAM, for example, and functions as a main storage device for the CPU 11. The storage 13 may be realized by a non-volatile storage medium such as a ROM or a flash memory, and functions as an auxiliary storage device for the CPU 11. Although only one CPU 11 is shown here, a plurality of CPUs 11 may be provided.

The I / O 14 controls transmission / reception of data performed between the control unit 1 and a device connected to the control unit 1 such as the front camera 5. For example, the I / O 14 converts video signals input from the front camera 5 and the rear camera 6 into image data in a format that can be processed by an image processing unit F7 described later, and stores the image data in the memory 12.

The storage 13 stores a program for executing various processes and camera information for each in-vehicle camera. The camera information includes installation position data indicating the installation position of each in-vehicle camera in the own vehicle, installation attitude data indicating the installation attitude of the in-vehicle camera with respect to the vehicle, lens distortion coefficient, focal length, optical axis center, pixel size, pixel Includes internal data indicating ratios and so on.

The installation position of each in-vehicle camera may be represented by, for example, the coordinates of the aforementioned three-dimensional coordinate system. Further, the mounting posture may be represented by a pitch angle, a roll angle, and a yaw angle determined with reference to the X axis, the Y axis, and the Z axis. That is, the storage 13 stores the pitch angle P1, roll angle R1, and yaw angle Y1 of the front camera 5, and the pitch angle P2, roll angle R2, and yaw angle Y2 of the rear camera 6.

The pitch angle P1 of the front camera 5 is simply the angle formed by the XY plane and the optical axis of the front camera 5, the roll angle R1 is the angle formed by the XY plane and the lateral direction of the front camera 5, and the yaw angle. Y1 is an angle (0 degree) between the orthogonal projection of the optical axis onto the XY plane and the X axis. The pitch angle P2 of the rear camera 6 is simply an angle formed by the XY plane and the optical axis of the rear camera 6, and the roll angle R2 is an angle formed by the XY plane and the lateral direction of the rear camera 6, The angle Y2 is an angle (180 degrees) between the orthogonal projection of the optical axis onto the XY plane and the X axis.

The yaw angle defined above represents the general shooting direction of the in-vehicle camera. In this embodiment, it is assumed that the deviation of the yaw angle of each in-vehicle camera from the initial mounting posture can be ignored.

The installation position data and the installation posture data are values measured in advance in a predetermined test environment or design values in the initial state. Of the various elements (roll angle and pitch angle) indicating the mounting posture, the values stored in the storage 13 in the initial state represent the current mounting posture for elements that have not been updated by the posture update related processing described later. The value to represent. Of the various elements indicating the mounting posture, for the elements updated by the posture update related processing described later, the updated value is stored in the storage 13 (or memory 12) as a value representing the current mounting posture. Is done.

In addition, the storage 13 stores camera parameters as parameters used for image processing (for example, well-known viewpoint conversion processing) on the captured image of the in-vehicle camera as camera information for each in-vehicle camera. The camera parameter is a parameter that is determined according to the installation position and mounting orientation of the in-vehicle camera.

Furthermore, the storage 13 stores data indicating the correspondence relationship between the detection directions of the tilt sensors included in each in-vehicle camera. The detection direction of each inclination sensor should just be described as the direction of the detection axis with which each inclination sensor is provided.

As shown in FIG. 3, the control unit 1 includes a stop determination unit F1, a detection result management unit F2, an update necessity determination unit F3, an attachment as functional blocks realized by executing a program stored in the storage 13. A posture specifying unit F5, a parameter adjusting unit F6, an image processing unit F7, and a vehicle posture specifying unit F4 are provided. Note that some or all of the functions of the control unit 1 may be configured by hardware using one or a plurality of ICs.

The stop determination unit F1 determines whether the host vehicle is stopped based on the vehicle speed input from the vehicle speed sensor 2. For example, the stop determination unit F1 determines that the host vehicle is stopped when the current vehicle speed is 0 km / h, and the host vehicle when the current vehicle speed is greater than 0 km / h. Is determined not to stop. Of course, as another aspect, when the vehicle speed is less than a predetermined threshold (for example, 5 km / h), it is determined that the vehicle is stopped, and when the vehicle speed is equal to or higher than the threshold, it is determined that the vehicle is not stopped. May be.

Moreover, although the aspect which determines whether the own vehicle has stopped based on the vehicle speed was illustrated here as an example, what is necessary is just to determine whether the own vehicle has stopped by the well-known various methods. . For example, you may determine whether the own vehicle has stopped based on the shift position which a shift position sensor detects.

The detection result management unit F2 sequentially acquires the detection results of the vehicle body side tilt sensor 3, the first tilt sensor 51, and the second tilt sensor 61 (for example, every 50 milliseconds), and detects the acquired detection results. Each result output source is distinguished and stored in the memory 12. The detection results for each inclination sensor may be stored in the memory 12 in the order of acquisition, for example. In addition, the detection result for each inclination sensor is further stored by distinguishing between the pitch angle and the roll angle.

The first detection result storage unit M11 illustrated in FIG. 3 is an area that stores detection results that are sequentially acquired from the first inclination sensor 51 among the storage areas of the memory 12. The second detection result storage unit M <b> 12 is an area that stores detection results that are sequentially acquired from the second inclination sensor 61 among the storage areas of the memory 12. The vehicle body side detection result storage unit M <b> 2 is an area that stores detection results sequentially acquired from the vehicle body side inclination sensor 3 among the storage areas of the memory 12. The first detection result storage unit M11 and the second detection result storage unit M12 correspond to the camera detection result storage unit described in the claims, and the vehicle body side detection result storage unit M2 includes the vehicle detection result storage unit described in the claims. It corresponds to.

Further, each time the detection result management unit F2 acquires the pitch angle θp1 from the first inclination sensor 51, the pitch angle displacement amount obtained by subtracting the pitch angle P1 employed as the current mounting posture from the acquired pitch angle θp1. Δθp1 is calculated and stored in the first detection result storage unit M11. Similarly, each time the roll angle θr1 is acquired, the roll angle displacement amount Δθr1 obtained by subtracting the roll angle R1 adopted as the current mounting posture from the acquired roll angle θr1 is calculated for the roll angle θr1. The result is stored in the detection result storage unit M11.

The pitch angle P1 and roll angle R1 adopted as the current mounting posture of the front camera 5 are values measured in a predetermined test environment, or the pitch angle and roll specified by the posture update-related processing performed previously. Point to the corner.

The detection result management unit F2 performs the same processing as when the pitch angles θp1 and θr1 are acquired from the first tilt sensor 51 even when the pitch angle θp2 and the roll angle θr2 are acquired from the second tilt sensor 61. That is, every time the pitch angle θp2 is acquired from the second tilt sensor 61, the pitch angle displacement amount Δθp2 obtained by subtracting the pitch angle P2 employed as the current mounting posture from the acquired pitch angle θp2 is calculated, and the second Stored in the detection result storage unit M12. Further, every time the roll angle θr2 is acquired, a roll angle displacement amount Δθr2 that is a difference from the roll angle R2 adopted as the current mounting posture is calculated from the acquired roll angle θr2, and the second detection result storage unit Store in M12.

The pitch angle displacement amount Δθp1, the roll angle displacement amount Δθr1, the pitch angle displacement amount Δθp2, and the roll angle displacement amount Δθr2 that are sequentially calculated by the detection result management unit F2 are collectively referred to as a displacement amount Δθ. The displacement amount calculation unit F21 provided in the detection result management unit F2 is a functional block that calculates the displacement amount Δθ. The displacement amount calculation unit F21 corresponds to the angular displacement amount calculation unit described in the claims.

Furthermore, the detection result management unit F2 determines whether or not the detection results for each tilt sensor have been collected in a predetermined amount (a required number of detection times) sufficient to specify the posture of the in-vehicle camera. judge. Note that a period during which a sufficient amount of detection results are collected to identify the posture of the in-vehicle camera is also referred to as a data collection period. The required number of detections may be designed as appropriate.

When the detection result management unit F2 determines that the detection results for a predetermined number of detection times can be collected, the detection result management unit F2 specifies the mode value of the detection results for each inclination sensor. The mode value here is synonymous with the mode value used in statistics, and is the value that appears most frequently as a result of multiple detections for one state quantity.

More specifically, the detection result management unit F2 uses the plurality of pitch angles θp1 stored in the first detection result storage unit M11 as a population, and determines the mode value of the pitch angle θp1 detected by the first tilt sensor 51. Identify. Further, the mode value of the roll angle θr1 detected by the first tilt sensor 51 is specified using the plurality of roll angles θr1 stored in the first detection result storage unit M11 as a population. Similarly, the detection result management unit F2 detects the mode value of the pitch angle θp2 detected by the second tilt sensor 61, the mode value of the roll angle θr2 within the data collection period, and the pitch angle detected by the vehicle body side tilt sensor 3. The mode value of θp0 and the mode value of the roll angle θr0 are specified. The mode value of the pitch angle θp1, the mode value of the roll angle θr1, the mode value of the pitch angle θp2, and the mode value of the roll angle θr2 correspond to the camera-side mode value described in the claims, and the pitch angle θp0 The mode value and the mode value of the roll angle θr0 correspond to the vehicle-side mode value recited in the claims.

The update necessity determination unit F3 determines whether or not the mounting posture data of the front camera 5 needs to be updated and whether or not the mounting posture data of the rear camera 6 needs to be updated. Details of the update necessity determination unit F3 will be described later.

The vehicle attitude specifying unit F4 specifies the attitude of the host vehicle with respect to the horizontal plane based on the detection result of the vehicle body side tilt sensor 3. The attitude of the host vehicle with respect to the horizontal plane here refers to the presence or absence of a tilt, and the size of the tilt when tilted. Furthermore, in this embodiment, the inclination in the vehicle front-rear direction and the inclination in the vehicle width direction are distinguished and handled.

More specifically, the vehicle posture specifying unit F4 is based on the mode value of the pitch angle θp0 and the mode value of the roll angle θr0 detected by the vehicle body side tilt sensor 3 specified by the detection result management unit F2. Identify the position of the vehicle relative to the horizontal plane. That is, the mode value of the pitch angle θp0 is adopted as the vehicle body inclination angle in the vehicle longitudinal direction. Therefore, when the mode value of the pitch angle θp0 is 0 degrees (or a value that can be regarded as 0 degrees), it is considered that the vehicle body is not inclined in the vehicle front-rear direction. Further, the mode value of the roll angle θr0 is adopted as the vehicle body inclination angle in the vehicle width direction. Therefore, when the mode value of the roll angle θr0 is 0 degree, it is determined that the vehicle body is not inclined in the vehicle width direction.

In the present embodiment, when the X1 axis provided in the vehicle body side tilt sensor 3 is in the same direction as the X axis, the Y1 axis is in the same direction as the Y axis, and the host vehicle is in a horizontal posture, The vehicle body side tilt sensor 3 is installed on the vehicle body so that the pitch angle θp0 and the roll angle θr0 are 0 degrees. Therefore, the mode value of the pitch angle θp0 is used as the vehicle body inclination angle in the vehicle front-rear direction, and the mode value of the roll angle θr0 is used as the vehicle body inclination angle in the vehicle width direction, but this is not restrictive. The posture of the host vehicle with respect to the horizontal plane may be specified by appropriately correcting according to the mounting posture of the vehicle body side tilt sensor 3 with respect to the vehicle body.

For example, when the X1 axis and the X axis of the vehicle body side tilt sensor 3 and the Y1 axis and the Y axis are deviated, the detection result may be corrected and used in view of the degree of deviation. In addition, the basic output value is measured in advance even when the vehicle is installed to output a predetermined value (basic output value) that is not 0 as a detection result when the vehicle is in a horizontal posture. The posture of the host vehicle with respect to the horizontal plane may be specified using the basic output value as a reference.

The mounting posture specifying unit F5 specifies the pitch angle P1 and the roll angle R1 as the current mounting posture of the front camera 5 with respect to the vehicle body. Further, the pitch angle P2 and the roll angle R2 as the mounting posture of the rear camera 6 with respect to the vehicle body are specified. Details of the mounting posture specifying portion F5 will be described later.

The parameter adjustment unit F6 corrects the camera parameter corresponding to the in-vehicle camera based on the identification result of the mounting posture identification unit F5. That is, when the mounting posture specifying unit F5 specifies the current mounting posture of the front camera 5 with respect to the current vehicle body, the camera parameter corresponding to the front camera 5 is corrected based on the specified current mounting posture. When the mounting posture specifying unit F5 specifies the current mounting posture of the rear camera 6 with respect to the current vehicle body, the camera parameter corresponding to the rear camera 6 is corrected based on the specified current mounting posture.

The image processing unit F7 performs various known image processing on the image data input from the in-vehicle camera, and generates an image to be displayed on the display 4. For example, the image processing unit F7 performs a process of converting an image captured by the front camera 5 into an overhead image using camera parameters corresponding to the front camera 5. The image processing unit F7 outputs to the display 4 and displays the image input from each in-vehicle camera and the image data generated by performing various image processing on the image input from the in-vehicle camera.

(Camera posture update related processing in the first embodiment)
Next, a series of processing (referred to as camera posture update related processing) performed by the control unit 1 in order to update the posture data of each in-vehicle camera will be described using the flowchart shown in FIG. The camera posture update related process shown in FIG. 4 may be performed independently for each of the front camera 5 and the rear camera 6.

Here, as an example, a case will be described in which the front camera 5 is a processing target and the pitch angle P1 of the front camera 5 is specified and updated. Of course, the same operation may be performed when the roll angle R1 of the front camera 5 is updated. Moreover, what is necessary is just to implement in the same process sequence, when the vehicle-mounted camera made into a process target is made into the rear camera 6. FIG.

The flowchart shown in FIG. 4 may be started, for example, when the ignition power of the host vehicle is turned on or when a certain time has elapsed since this processing was performed last time.

First, in step S101, the stop determination unit F1 determines whether or not the host vehicle is stopped. If the stop determination unit F1 determines that the host vehicle is stopped, step S101 is YES and the process proceeds to step S103. On the other hand, when the stop determination part F1 determines with the own vehicle not stopping, step S101 becomes NO and moves to step S102.

In step S102, the detection result (for example, pitch angle θp1) stored in the memory 12 and the data calculated from the detection result (for example, pitch angle displacement amount Δθp1) are discarded, and the process returns to step S101.

In step S103, the detection result management unit F2 acquires the pitch angle θp1 detected by the first tilt sensor 51 and stores it in the first detection result storage unit M11, and acquires the pitch angle θp0 detected by the vehicle body side tilt sensor 3. And stored in the vehicle body side detection result storage unit M2. When the process of step S103 is completed, the process proceeds to step S104.

In step S104, the displacement amount calculation unit F21 calculates a pitch angle displacement amount Δθp1 that is a difference between the pitch angle θp1 acquired this time and the pitch angle P1 that is currently employed, and stores it in the first detection result storage unit M11. The process moves to step S105. The pitch angle θp1 acquired this time means the pitch angle θp1 newly added to the first detection result storage unit M11 in step S103.

In step S105, the detection result management unit F2 determines whether or not a predetermined number of detection results are accumulated in the first detection result storage unit M11 and the vehicle body side detection result storage unit M2. The predetermined number here is a sufficient number of times for specifying the current mounting orientation of the front camera 5 (here, the pitch angle P1), and may be, for example, a detection result for 50 times. Since the number of detections per unit time is constant, the time required to collect the detection results for a predetermined number of times is a constant time. That is, the determination content in step S105 may be to determine whether or not a certain time has elapsed since the collection of detection results was started. The process from step S101 to YES until step S105 is determined to be YES corresponds to the data collection period described above.

If the detection results for a predetermined number of times are accumulated in the first detection result storage unit M11 and the vehicle body side detection result storage unit M2 at the time of moving to step S105, step S105 is YES. Then, the process proceeds to step S106. On the other hand, if the predetermined number of detection results have not yet been accumulated in each of the first detection result storage unit M11 and the vehicle body side detection result storage unit M2, step S105 is NO and the process returns to step S101.

Note that the data collected between step S101 and step S105 is data for specifying the current mounting posture. By the way, when data acquired at a plurality of points is mixed with data for specifying the current mounting posture, the pitch angle P1 is updated using data affected by different gradients. As described in the third embodiment, the accuracy may be deteriorated unless the data collection time is very long.

In the present embodiment, when the host vehicle starts to travel from the state where the host vehicle is stopped (NO in step S101), in step S102, the detection results collected while the vehicle is stopped are discarded. That is, the case where the pitch angle P1 is specified and the mounting posture data is updated in the configuration of the present embodiment is a case where a predetermined number of detection results are accumulated while the vehicle stops at one point. Therefore, according to the configuration of the present embodiment, the detection results used for specifying the current mounting posture are all values affected by the common road gradient, in other words, the own vehicle with respect to the horizontal plane is constant. Since it is the detection result at the time, the mounting posture can be specified with higher accuracy.

In step S106, the update necessity determination unit F3 determines whether or not it is necessary to update the attachment posture data (the pitch angle P1 here) to be processed. When the update necessity determination unit F3 determines that the pitch angle P1 needs to be updated, the mode value of the pitch angle displacement amount Δθp1 accumulated during the data collection period is a predetermined threshold (for example, 3 degrees). What is necessary is just to be the above. By using the most frequent value determined by the plurality of pitch angle displacement amounts Δθp1 as a population, it is possible to suppress the determination that the pitch angle P1 needs to be updated due to a temporary disturbance.

In step S106, if the update necessity determination unit F3 determines that the pitch angle P1 needs to be updated, step S106 is YES and the process proceeds to step S107. On the other hand, when the update necessity determination unit F3 determines that the pitch angle P1 does not need to be updated, step S106 is NO and the process proceeds to step S102.

In the present embodiment, even if the update necessity determination unit F3 determines that the pitch angle P1 does not need to be updated, the process returns to step S101 via step S102 and continues this process. Not limited to. As another aspect, when the update necessity determination unit F3 determines that it is not necessary to update the pitch angle P1, this flow may be terminated. In that case, this process may be started again at a predetermined timing, for example, when the ignition power is turned on next time or when a predetermined time has elapsed since the end of this flow.

In step S <b> 107, the mounting posture specifying unit F <b> 5 sets the amount of influence of the posture of the host vehicle with respect to the detection result of the first inclination sensor 51 (the vehicle posture influence amount) as the mode value of the pitch angle θp <b> 1. The value subtracted from is calculated, and the process proceeds to step S108. The vehicle attitude influence amount corresponds to the amount that the road gradient affects the detection result of the first inclination sensor 51.

As described above, the posture of the host vehicle with respect to the horizontal plane is represented by the mode value of the pitch angle θp0 and the mode value of the roll angle θr0 detected by the vehicle body side tilt sensor 3. The vehicle attitude influence amount is determined based on the correspondence between the mode value of the pitch angle θp0 and the mode value of the roll angle θr0, and the detection direction of the vehicle body side tilt sensor 3 and the detection direction of the first tilt sensor 51. Here, since the direction in which the first inclination sensor 51 detects the pitch angle θp1 and the direction in which the vehicle body side inclination sensor 3 detects the pitch angle θp0 are the same direction, the mounting posture specifying unit F5 includes the first inclination sensor 51. The mode value of the pitch angle θp0 is used as it is as the vehicle posture influence amount with respect to the pitch angle θp1 detected by. That is, the vehicle attitude influence amount is the mode value of the pitch angle θp0 collected while repeating steps S101 to S105.

If the pitch angle P2 of the second tilt sensor 61 is to be processed, the vehicle attitude influence amount for the pitch angle θp2 detected by the second tilt sensor 61 is set to −1 as the mode value of the pitch angle θp0. A value obtained by multiplication may be used. This is because the direction in which the second tilt sensor 61 detects the pitch angle θp2 and the direction in which the vehicle body side tilt sensor 3 detects the pitch angle θp0 are opposite directions.

The subtraction value calculated in step S102 represents a value obtained by subtracting the influence amount (that is, the vehicle posture influence amount) of the inclination of the host vehicle from the pitch angle θp1 detected by the first inclination sensor 51. That is, the calculated subtraction value represents the actual pitch angle P1 of the front camera 5 with respect to the vehicle body at the current time. For convenience, the calculated subtraction value is set as the pitch angle P1a.

In step S108, the mounting posture specifying unit F5 determines whether or not the pitch angle P1a calculated in step S107 matches the pitch angle P1 adopted as the current mounting posture. The coincidence here is not limited to perfect coincidence, but may be regarded as coincidence when the difference is within a predetermined allowable range (for example, within ± 0.5 degrees). The case where it is determined in step S108 that the pitch angle P1a coincides with the pitch angle P1 even though it is determined in step S106 that the mode value of the pitch angle displacement amount Δθp1 is equal to or greater than a predetermined threshold value. This is a case where step S106 is determined as YES due to the influence of the inclination of the vehicle.

If the pitch angle P1a calculated in step S108 matches the pitch angle P1 currently employed, step S108 is YES and this flow is finished. On the other hand, if the pitch angle P1a calculated in step S108 does not coincide with the current pitch angle P1, step S108 is NO and the process proceeds to step S109.

In step S109, the pitch angle P1a is adopted as the current pitch angle P1, and is registered in the storage 13 as the current pitch angle P1 of the front camera 5. Further, the parameter adjustment unit F6 updates the camera parameter stored in the storage 13 to a value corresponding to the newly adopted pitch angle P1.

In the above, as described at the beginning of the description of this processing, the case where the pitch angle P1 of the front camera 5 is updated has been exemplified. The process of updating the roll angle R1 of the front camera 5 may be performed in parallel with the pitch angle P1 of the front camera 5 described above.

Further, the processing for processing the rear camera 6 may be performed in parallel with the processing for the front camera 5.

(Summary of the first embodiment)
According to the above configuration, in step S107, the detection result of the vehicle body side inclination sensor 3, the detection direction of the camera side inclination sensor attached to the vehicle-mounted camera to be processed, and the detection direction of the vehicle body side inclination sensor 3 From the corresponding relationship, the vehicle attitude influence amount with respect to the detection result of the camera side tilt sensor is specified. And the attachment attitude | position of the vehicle-mounted camera with respect to a vehicle body is specified by subtracting the vehicle attitude | position influence amount from the detection result of a camera side inclination sensor.

Therefore, according to the above configuration, it is possible to prevent the mounting posture of the in-vehicle camera from being specified as an incorrect posture due to the inclination of the vehicle.

In the above configuration, in step S106, the update necessity determination unit F3 determines that it is not necessary to update the element to be processed (for example, the pitch angle P1) among the various elements representing the mounting posture. Sometimes the value of the element is not updated. In other words, only when it is determined that the element to be processed needs to be updated, the current value of the element can be identified and updated.

Further, according to the above configuration, the degree of inclination of the vehicle body can be specified from the detection result of the vehicle body side inclination sensor 3 provided on the vehicle body. For this reason, in the second embodiment to be described later, three or more in-vehicle cameras with a built-in tilt sensor are required, whereas according to the configuration of the present embodiment, the tilt sensor included in the driving support system 100 is built-in. There may be one in-vehicle camera.

Also, the pitch angle and roll angle detected by the tilt sensor are usually affected by vehicle acceleration / deceleration and turning. However, in this embodiment, the mounting posture of the in-vehicle camera is specified based on the detection result detected by the tilt sensor while the vehicle is stopped. Therefore, the influence of the temporary noise resulting from such a vehicle behavior can be suppressed, and the mounting posture of the in-vehicle camera can be specified with higher accuracy.

In the present embodiment, the driving support system 100 is illustrated as an in-vehicle camera including the front camera 5 and the rear camera 6, but is not limited thereto. The driving support system 100 only needs to include at least one in-vehicle camera, and the number and photographing range are not limited.

<Second Embodiment>
Next, a second embodiment of the present disclosure will be described with reference to FIGS. In the following, members having the same functions as those shown in the drawings used in the description of the first embodiment are denoted by the same reference numerals, and description thereof is omitted. Further, when only a part of the configuration is mentioned, the first embodiment described above can be applied to the other parts.

FIG. 5 is a diagram illustrating an example of a schematic configuration of the driving support system 100A according to the present embodiment. The driving support system 100A includes a control unit 1A, a vehicle speed sensor 2, a display 4, a front camera 5, a rear camera 6, a right side camera 7, and a left side camera 8.

The control unit 1A corresponds to the control unit 1 described above, and each of the vehicle speed sensor 2, the display 4, the front camera 5, the rear camera 6, the right side camera 7, and the left side camera 8 is a well-known vehicle interior. They are connected to each other via a network.

The right side camera 7 is a camera provided so as to photograph a predetermined range on the right side of the host vehicle. As the right side camera 7, for example, a well-known CMOS camera or CCD camera whose shooting range is set to a wide angle (for example, an angle of view of 175 °) by a wide angle lens can be used. The right side camera 7 may be installed at a position that is appropriately designed, for example, near the right side mirror or on the right side of the vehicle body. In the present embodiment, for simplification of description, the right side camera 7 is assumed to be installed in the vicinity of the center in the vehicle front-rear direction at the upper end of the right side surface of the vehicle body.

Further, the right side camera 7 has a vertical direction (vertical direction) and a horizontal direction (horizontal direction) defined in advance. In the right side camera 7, the orthogonal projection vector onto the XY plane of the optical axis, which is the central axis of the photographing direction, is the same direction as the negative direction of the Y axis, and the left-right direction is parallel to the X axis at the predetermined installation location described above. It is attached to become. Video signals captured by the right side camera 7 are sequentially output to the control unit 1.

Also, the right side camera 7 includes an inclination sensor (hereinafter, third inclination sensor) 71 that detects the inclination of the right side camera 7 (that is, the inclination with respect to the horizontal plane). The third tilt sensor 71 is a biaxial tilt sensor that detects the tilt of the right side camera 7 with respect to the horizontal plane by decomposing it into rotation angles for each of two axes (X3 axis and Y3 axis) as shown in FIG. It is.

The third inclination sensor 71 includes an inclination angle of the right side camera 7 in the vehicle longitudinal direction (inclination angle with respect to the horizontal plane (referred to as roll angle θr3)) and an inclination angle of the right side camera 7 in the vehicle width direction (inclination angle with respect to the horizontal plane). (Pitch angle θp3) is incorporated in the right side camera 7 so as to detect.

More specifically, in the third inclination sensor 71, the orthogonal projection vector of the X3 axis to the XY plane is the same direction as the negative direction of the Y axis, and the orthogonal projection vector of the Y3 axis of the XY plane is the X axis. It is built in the right side camera 7 so as to be in the same direction as the positive direction. The X3 axis of the third tilt sensor 71 coincides with the optical axis of the right side camera 7, and the Y3 axis is fixed in the casing of the right side camera 7 so as to coincide with the lateral direction of the right side camera 7. Yes.

The detection results (pitch angle θp3 and roll angle θr3) of the third tilt sensor 71 are sequentially output to the control unit 1A. In the pitch angle θp3, the rotation angle in the direction of the right-handed screw with the Y3 axis as the rotation axis from the neutral state is represented by a positive value, and the rotation angle in the reverse direction is represented by a negative value. In addition, the pitch angle θp3 represents a rotation angle in the direction of the right screw of the Y3 axis from a neutral state as a positive value, and a rotation angle in the reverse direction as a negative value.

The third tilt sensor 71 corresponds to the camera posture detector described in the claims, and the pitch angle θp3 and the roll angle θr3 correspond to an example of the camera posture index value described in the claims. Further, the extending direction of the X3 axis and the extending direction of the Y3 axis correspond to the camera side detection direction described in the claims.

In the present embodiment, the third tilt sensor 71 is configured to be installed in the casing of the right side camera 7. However, as another aspect, the third tilt sensor 71 is integrally attached to the right side camera 7 outside the casing. It is good.

The left side camera 8 is a camera provided to photograph a predetermined range on the left side of the host vehicle. As the left side camera 8, for example, a well-known CMOS camera or CCD camera whose shooting range is set to a wide angle (for example, an angle of view of 185 °) by a wide angle lens can be used. The left side camera 8 may be installed at a position that is appropriately designed, for example, near the left side mirror or on the left side surface of the vehicle body. In the present embodiment, for the sake of simplification, it is assumed that the left side camera 8 is installed in the vicinity of the center in the vehicle front-rear direction at the upper end of the left side surface of the vehicle body.

Further, the left side camera 8 is preliminarily defined in the vertical direction (vertical direction) and the horizontal direction (horizontal direction). In the above-described predetermined installation location, the left side camera 8 has an orthogonal projection vector onto the XY plane of the optical axis, which is the central axis in the shooting direction, in the same direction as the positive direction of the Y axis, and the horizontal direction is parallel to the X axis. It is attached to become. Video signals taken by the left side camera 8 are sequentially output to the control unit 1.

Also, the left side camera 8 includes an inclination sensor (hereinafter referred to as a fourth inclination sensor) 81 that detects the inclination of the left side camera 8 (inclination with respect to the horizontal plane). The fourth tilt sensor 81 is a two-axis tilt sensor that detects the tilt of the left side camera 8 with respect to the horizontal plane by decomposing the tilt into rotation angles for two axes (X4 axis and Y4 axis).

The fourth inclination sensor 81 includes an inclination angle (referred to as roll angle θr4) of the left side camera 8 in the vehicle longitudinal direction with respect to the horizontal plane and an inclination angle (pitch angle θp4) of the left side camera 8 in the vehicle width direction with respect to the horizontal plane. It is built in the left side camera 8 so as to detect.

More specifically, in the fourth tilt sensor 81, the orthogonal projection vector of the X4 axis to the XY plane is the same as the positive direction of the Y axis, and the orthogonal projection vector of the Y4 axis of the XY plane is the X axis. Built in the left side camera 8 so as to be in the same direction as the negative direction. Note that the X4 axis of the fourth tilt sensor 81 is fixed in the housing of the left side camera 8 so as to coincide with the optical axis of the left side camera 8, and the Y4 axis coincides with the lateral direction of the left side camera 8. Yes.

The detection results (pitch angle θp4 and roll angle θr4) of the fourth tilt sensor 81 are sequentially output to the control unit 1A. In the pitch angle θp4, the rotation angle in the direction of the right screw with the Y4 axis as the rotation axis from the neutral state is expressed as a positive value, and the rotation angle in the reverse direction is expressed as a negative value. In addition, the pitch angle θp4 represents a rotation angle in the direction of the right screw of the Y4 axis from a neutral state as a positive value, and a rotation angle in the reverse direction as a negative value.

The fourth tilt sensor 81 corresponds to the camera posture detector described in the claims, and the pitch angle θp4 and the roll angle θr4 correspond to an example of the camera posture index value described in the claims. Further, the extending direction of the X4 axis and the extending direction of the Y4 axis correspond to the camera side detection direction described in the claims.

In the present embodiment, the fourth tilt sensor 81 is configured to be installed in the housing of the left side camera 8, but as another aspect, the configuration is integrally attached to the left side camera 8 outside the housing. It is good.

Hereinafter, when the front camera 5, the rear camera 6, the right side camera 7, and the left side camera 8 are not distinguished from each other, they are described as an in-vehicle camera. Further, when the first inclination sensor 51, the second inclination sensor 61, the third inclination sensor 71, and the fourth inclination sensor 81 are not distinguished, they are described as inclination sensors.

The storage 13 stores the camera information of the right side camera 7 and the left side camera 8 in addition to the camera information of the front camera 5 and the rear camera 6. In other words, the storage 13 includes the installation position data, mounting orientation data (pitch angle P3, roll angle R3, yaw angle Y3) of the right side camera 7, camera parameters, installation position data, mounting orientation data of the left side camera 8. (Pitch angle P4, roll angle R4, yaw angle Y4) and camera parameters are stored.

The pitch angle P3 of the right side camera 7 is simply the angle formed by the XY plane and the optical axis of the right side camera 7, and the roll angle R3 is the angle formed by the XY plane and the lateral direction of the right side camera 7. The yaw angle Y3 is an angle (180 degrees) between the orthogonal projection of the optical axis onto the XY plane and the Y axis. The pitch angle P4 of the left side camera 8 is simply an angle formed by the XY plane and the optical axis of the left side camera 8, and the roll angle R4 is an angle formed by the XY plane and the lateral direction of the left side camera 8. The yaw angle Y4 is an angle (0 degree) formed by the orthogonal projection of the optical axis onto the XY plane and the Y axis. As in the first embodiment, also in this embodiment, the deviation of the yaw angle of each in-vehicle camera from the initial mounting posture can be ignored. The mounting posture data includes the pitch angle and roll angle specified as the current mounting posture in addition to the mounting posture at the initial setting.

In addition, the storage 13 stores data indicating the correspondence relationship between the detection directions of the tilt sensors included in each in-vehicle camera. The detection direction of each inclination sensor may be represented by the direction of the detection axis included in each inclination sensor.

For example, the storage 13 has a correspondence relationship between detection directions in which the X1 axis of the first tilt sensor 51 and the Y3 axis of the third tilt sensor 71 are in the same direction, the X2 axis of the second tilt sensor 61, and the fourth tilt sensor 81. The fact that the direction is opposite to the Y4 axis is stored as a correspondence relationship. Further, the Y1 axis of the first inclination sensor 51 and the X4 axis of the fourth inclination sensor 81 are in the same direction, and the X3 axis of the third inclination sensor 71 and the Y2 axis of the second inclination sensor 61 are opposite to each other. Is stored as a correspondence relationship.

Further, the storage 13 stores a combination of angles having a corresponding relationship among various elements representing the mounting posture. There are two types of combinations of angles that have a corresponding relationship.

One is a combination of elements that are similarly affected by the inclination of the vehicle body with respect to the horizontal plane among the elements representing various mounting postures. More specifically, the roll angle θr1, the roll angle θr2, the pitch angle θp3, and the pitch angle θp4 are all affected by the inclination of the vehicle body in the vehicle width direction as shown in FIG. That is, the roll angle θr1, the roll angle θr2, the pitch angle θp3, and the pitch angle θp4 are combinations of angles that have a corresponding relationship.

Further, the pitch angle θp1, the pitch angle θp2, the roll angle θr3, and the roll angle θr4 are all angles affected by the inclination of the vehicle body in the longitudinal direction of the vehicle. That is, the pitch angle θp1, the pitch angle θp2, the roll angle θr3, and the roll angle θr4 are combinations of elements that have a corresponding relationship. Hereinafter, other elements having a correspondence relationship with a certain element are referred to as corresponding angles.

FIG. 7 is a block diagram illustrating a configuration of the control unit 1A in the present embodiment, and the control unit 1A includes various functional blocks described in the first embodiment. However, the operation of the vehicle posture specifying unit F4 in the present embodiment is different from the operation of the vehicle posture specifying unit F4 in the first embodiment described above. The operation of the vehicle attitude specifying unit F4 in this embodiment will be described later separately.

The detection result management unit F2 of the present embodiment sequentially acquires the detection results of the first tilt sensor 51, the second tilt sensor 61, the third tilt sensor 71, and the fourth tilt sensor 81, and acquires the acquired detection results. The detection results are stored in the memory 12 separately for each output source. The detection results for each inclination sensor may be stored in the memory 12 in the order of acquisition, for example.

The displacement amount calculation unit F21 obtains the pitch angle θp3 and the roll angle θr3 from the third tilt sensor 71, and the pitch angle displacement that is the difference between the pitch angle P3 and the roll angle R3 that are employed as the current mounting posture. The amount Δθp3 and the roll angle displacement amount Δθr3 are calculated and stored in the third detection result storage unit M13. Further, every time the pitch angle θp4 and the roll angle θr4 are acquired from the fourth inclination sensor 81, the pitch angle displacement amount Δθp4 and the roll angle which are the differences from the pitch angle P4 and the roll angle R4 adopted as the current mounting posture. A displacement amount Δθr4 is calculated and stored in the fourth detection result storage unit M14.

That is, as the displacement amount Δθ, the pitch angle displacement amount Δθp3, the roll angle displacement amount Δθr3, the pitch angle displacement amount Δθp4, and the roll angle displacement amount Δθr4 are also applicable.

The third detection result storage unit M <b> 13 provided in the memory 12 is an area that stores detection results that are sequentially acquired from the third inclination sensor 71 among the storage areas included in the memory 12. The fourth detection result storage unit M <b> 14 is an area that stores detection results sequentially acquired from the fourth inclination sensor 81 among the storage areas of the memory 12.

(Camera posture update related processing in the second embodiment)
Next, a series of processing (camera posture update related processing) performed by the control unit 1A in order to update the posture data of each in-vehicle camera will be described with reference to the flowchart shown in FIG. The flowchart shown in FIG. 8 may be started, for example, when the ignition power supply of the host vehicle is turned on or when a certain time has elapsed since this processing was performed last time.

First, in step S201, the stop determination unit F1 determines whether or not the host vehicle is stopped. If the stop determination unit F1 determines that the host vehicle is stopped, step S201 is YES and the process proceeds to step S203. On the other hand, when the stop determination part F1 determines with the own vehicle not stopping, step S201 becomes NO and moves to step S202.

In step S202, the detection result stored in the memory 12 and the data calculated from the detection result (for example, the displacement amount Δθ) are discarded, and the process returns to step S201.

In step S203, the detection result management unit F2 acquires the pitch angles θp1 to θp4 and roll angles θr1 to θr4 detected by each inclination sensor, and stores them in the storage area of the memory 12 corresponding to each output source. The process proceeds to S204.

In step S204, the displacement amount calculation unit F21 calculates various displacement amounts Δθ based on the various detection results acquired in step S203, stores them in the corresponding storage areas, and proceeds to step S205. That is, in step S204, the pitch angle displacement amount Δθp1 and the roll angle displacement amount Δθr1 are calculated and stored in the first detection result storage unit M11, and the pitch angle displacement amount Δθp2 and the roll angle displacement amount Δθr2 are calculated and the second detection result. Store in the storage unit M12. Further, the pitch angle displacement amount Δθp3 and the roll angle displacement amount Δθr3 are calculated and stored in the third detection result storage unit M13, and the pitch angle displacement amount Δθp4 and the roll angle displacement amount Δθr4 are calculated and the fourth detection result storage unit M14. To store.

In step S205, the detection result management unit F2 determines whether or not a predetermined number of detection results are accumulated in the memory 12. If the detection results for a predetermined number of times are accumulated in the memory 12 at the time of moving to step S205, step S205 becomes YES and the process moves to step S206. On the other hand, if the predetermined number of detection results have not yet been accumulated in the memory 12, step S205 is NO and the process returns to step S201.

In step S206, the update necessity determination unit F3 specifies the mode values of the pitch angle displacement amounts Δθp1, Δθp2, Δθp3, Δθp4, roll angle displacement amounts Δθr1, Δθr2, Δθr3, and Δθr4 in the data collection period, and step S207. Move on. For convenience, when referring to the mode value of each displacement amount, Δθp1 (m), Δθp2 (m), Δθp3 (m), Δθp4 (m), roll angle displacement amounts Δθr1 (m), Δθr2 (m) are sequentially arranged. , Δθr3 (m), and Δθr4 (m).

In addition, when Δθp1 to θp4 and Δθp1 to Δθp4 are not distinguished and indicate a displacement amount of a certain element, they are described as Δθ, and further, Δθp1 (m) to Δθp4 (m) are not distinguished, and a certain element In this case, Δθ (m) is used.

In step S207, the update necessity determination unit F3 determines whether it is necessary to update any one or a plurality of mounting posture data of the plurality of in-vehicle cameras. Here, first, all of the absolute values of the pitch angle displacement amounts Δθp1 (m) to Δθp4 (m) and roll angle displacement amounts Δθr1 (m) to Δθr4 (m) acquired in step S206 are set to predetermined threshold values (for example, 3 It is determined whether or not it is within (degree).

When the absolute values of the pitch angle displacement amounts Δθp1 (m) to Δθp4 (m) and the roll angle displacement amounts Δθr1 (m) to Δθr4 (m) are all equal to or less than a predetermined threshold, It is determined that there is no need to update the mounting posture data, and the process proceeds to step S202.

On the other hand, when any one of the pitch angle displacement amounts Δθp1 (m) to Δθp4 (m) and the roll angle displacement amounts Δθr1 (m) to Δθr4 (m) has an absolute value equal to or greater than a predetermined threshold value. Is determined that it is necessary to update any one or a plurality of mounting posture data of a plurality of in-vehicle cameras, and the process proceeds to step S208.

In step S208, the vehicle posture specifying unit F4 is set to an angle that is affected by the inclination of the vehicle body in the vehicle longitudinal direction among the various displacement amounts Δθp1 (m) to Δθp4 (m) and Δθr1 (m) to Δθr4 (m). By comparing the derived displacement amounts Δθ (m) with each other, it is determined whether the vehicle body is tilted in the front-rear direction. That is, it is determined whether or not the vehicle body is tilted in the longitudinal direction of the vehicle by comparing the displacement amounts Δθ (m) of the pitch angle θp1, the pitch angle θp2, the roll angle θr3, and the roll angle θr4.

More specifically, it is as follows. First, when the angle adopted as the current mounting posture matches the actual current angle in the elements that are affected by the tilt in the longitudinal direction of the vehicle, the displacement amounts corresponding to these various elements Δθ is a value reflecting only the inclination in the front-rear direction of the vehicle.

Therefore, in the element that is affected by the inclination of the vehicle in the front-rear direction, the angle adopted as the current mounting posture matches the actual current angle, and the vehicle has a front-rear inclination. If not, the absolute value of each displacement amount Δθ (m) is less than a predetermined threshold value.

On the other hand, among the elements that are affected by the vehicle's front-rear direction tilt, the vehicle has a front-rear direction tilt with respect to an element that does not match the angle currently used as the mounting orientation and the actual current angle. Even if it does not occur, the absolute value of the displacement Δθ (m) is a value that reflects the deviation between the angle used as the current mounting posture and the actual current angle. That is, the absolute value of the displacement Δθ (m) of an element whose current mounting posture does not coincide with the actual current angle among the elements affected by the vehicle front-rear inclination. Exceeds a predetermined threshold.

Also, it is relatively unlikely that the majority of the four elements that have a corresponding relationship with each other are out of mounting orientation. For this reason, as shown in FIG. 9, the number of elements in which the absolute value of the displacement amount Δθ (m) is less than a predetermined threshold among the four having a corresponding relationship becomes a majority (three or more). If it is determined that the vehicle body is not tilted in the longitudinal direction of the vehicle.

On the other hand, when the number of elements whose absolute value of the displacement amount Δθ (m) is equal to or greater than a predetermined threshold among the four corresponding to each other is greater than a majority, It is determined that an inclination has occurred.

That is, the vehicle attitude specifying unit F4 determines whether or not the vehicle body is tilted in the vehicle longitudinal direction by majority vote. If the vehicle posture specifying unit F4 determines that the vehicle body is not tilted in the vehicle front-rear direction, step S208 is YES and the process proceeds to step S209. On the other hand, when the vehicle posture specifying unit F4 determines that the vehicle body is tilted in the vehicle front-rear direction, step S208 is NO and the process proceeds to step S211.

In step S209, among the elements affected by the inclination of the vehicle body in the longitudinal direction of the vehicle, the element whose absolute value of the displacement amount Δθ (m) is equal to or greater than a predetermined threshold value is transferred to step S210 as the update target angle. For example, in the example of FIG. 9, the pitch angle θp1 of the front camera 5 corresponding to the pitch angle displacement amount Δθp1 (m) is the update target angle. At this time, the front camera 5 corresponds to an in-vehicle camera (that is, a camera to be updated) that should update the mounting posture data. Hereinafter, as an example, the operation of this process will be described with the pitch angle θp1 of the front camera 5 as the update target angle. Note that if there is no element whose absolute value of the displacement Δθ (m) is equal to or greater than a predetermined threshold among the elements affected by the vehicle body inclination in the vehicle longitudinal direction in step S209, the process proceeds to step S211. It ’s fine.

In step S210, the mode value of the pitch angle θp1 adopted as the update target angle is adopted as the current pitch angle P1, and is registered in the storage 13 as the current pitch angle P1 of the front camera 5. In addition, the camera parameter stored in the storage 13 is updated to a value corresponding to the newly adopted pitch angle P1. When the process in step S210 is completed, the process proceeds to step S211.

In step S211, the vehicle posture specifying unit F4 is similar to step S208 from the displacement amount Δθ (m) of the roll angle θr1, the roll angle θr2, the pitch angle θp3, and the pitch angle θp4 that is affected by the inclination of the vehicle body in the vehicle width direction. In addition, it is determined whether or not the vehicle body is tilted in the vehicle width direction by majority vote.

In this step S211, if the vehicle attitude specifying unit F4 determines that the vehicle body is not tilted in the vehicle width direction, step S211 is YES and the process proceeds to step S212. On the other hand, when the vehicle posture specifying unit F4 determines that the vehicle body is not inclined in the vehicle width direction, step S211 is NO and this flow is ended.

In step S212, among the elements affected by the inclination of the vehicle body in the vehicle width direction, the element whose absolute value of the displacement amount Δθ (m) is equal to or greater than a predetermined threshold value is transferred to step S213 as the update target angle. In step S212, if there is no element that is affected by the inclination of the vehicle body in the vehicle width direction and the absolute value of the displacement amount Δθ (m) is equal to or greater than the predetermined threshold value, this flow ends. do it.

(Summary of the second embodiment)
In the above configuration, the vehicle posture specifying unit F4 specifies whether or not the vehicle body of the host vehicle is in a horizontal posture based on the detection results of a plurality of inclination sensors attached to the in-vehicle camera. More specifically, it is determined whether or not the vehicle body is tilted in the vehicle longitudinal direction and whether or not the vehicle body is tilted in the vehicle width direction.

Then, when it is determined that the vehicle body tilt in the vehicle front-rear direction has not occurred, the value of the element affected by the vehicle body tilt in the vehicle front-rear direction is updated. Further, when it is determined that the vehicle body is not tilted in the vehicle width direction, the value of the element that is affected by the vehicle body tilt in the vehicle width direction is updated.

Therefore, according to the above configuration, it is possible to prevent the mounting posture of the in-vehicle camera from being specified as a wrong posture due to the inclination of the vehicle, as in the first embodiment.

In the above configuration, when the update necessity determination unit F3 determines in step S207 that there is no need to update the attachment posture data, the value of the element is not updated. In other words, only when it is determined that the attachment posture data needs to be updated, the processing after step S208 can be performed to update the value of a predetermined element among various elements representing the attachment posture. In the above-described embodiment, elements that do not need to be updated can be updated only for elements that need to be updated without unnecessary updating.

Furthermore, in the first embodiment described above, the vehicle body side inclination sensor 3 needs to be installed in the vehicle body, but in the second embodiment, the vehicle body side inclination sensor 3 does not need to be installed.

In addition, although the aspect provided with four vehicle-mounted cameras was illustrated here as an example, it is not restricted to this. It is only necessary that the update target angle can be specified by majority vote among the plurality of corresponding angles. That is, it is sufficient that at least three in-vehicle cameras are provided.

Here, the current value of the element affected by the vehicle body tilt in the longitudinal direction of the vehicle is specified and updated (S210), and the current value of the element affected by the vehicle body tilt in the rear direction of the vehicle width is specified and updated. Although the aspect which performs update (S213) independently was illustrated, it is not restricted to this.

As another aspect, when the vehicle posture specifying unit F4 determines that the vehicle body does not tilt in the vehicle front-rear direction and determines that the vehicle body tilt does not occur in the vehicle width direction, Or it is good also as an aspect which specifies the mounting attitude | position of a vehicle-mounted camera provided with the element whose absolute value of displacement amount (DELTA) (theta) is more than a predetermined threshold value, and updates mounting attitude data.

When the vehicle posture specifying unit F4 determines that the vehicle body is not inclined in the vehicle front-rear direction and it is determined that the vehicle body is not inclined in the vehicle width direction, the host vehicle is in a horizontal posture. Means that

<Third Embodiment>
Next, a third embodiment of the present disclosure will be described with reference to FIG. In the following, members having the same functions as those shown in the drawings used in the description of the first and second embodiments described above are denoted by the same reference numerals and description thereof is omitted. When only a part of the configuration is mentioned, the first and second embodiments described above can be applied to the other parts.

FIG. 10 is a diagram illustrating an example of a schematic configuration of the driving support system 100B according to the present embodiment. The driving support system 100B includes a control unit 1B, a vehicle speed sensor 2, a display 4, and a front camera 5. The control unit 1B is connected to the vehicle speed sensor 2, the display 4, and the front camera 5 so as to be able to communicate with each other via a known in-vehicle network.

The control unit 1B includes various functional blocks (see FIG. 4) described in the first embodiment. However, it is not necessary to include the second detection result storage unit M12 and the vehicle body side detection result storage unit M2. The detection result management unit F2 accumulates the pitch angle θp1 and roll angle θr1 acquired from the first tilt sensor 51 in the first detection result storage unit M11. Further, every time the pitch angle θp1 and the roll angle θr1 are acquired from the first tilt sensor 51, the displacement amount calculation unit F21 calculates the pitch angle displacement amounts Δθp1 and Δθr1 and accumulates them in the first detection result storage unit M11.

As a more preferable aspect, the detection result management unit F2 according to the present embodiment stores, in the first detection result storage unit M11, only the detection results acquired while the vehicle determination unit F1 determines that the host vehicle is stopped. It shall be accumulated. However, unlike the first and second embodiments described above, even if the host vehicle starts traveling, the data accumulated in the first detection result storage unit M11 is retained without being discarded.

Furthermore, the detection result management unit F2 determines whether or not a sufficient amount (the required number of detection times) of detection results has been accumulated in the first detection result storage unit M11. The required number of detections here is larger than the number of detections assumed in the first and second embodiments, and the mode value is a detection result in a state where a vehicle is present on a horizontal road. Is the number of detections.

As described above, the pitch angle θp1 and the roll angle θr1 detected by the first inclination sensor 51 are values affected by the road gradient (that is, the inclination of the vehicle body). Therefore, the pitch angle θp1 and the roll angle θr1 detected by the first inclination sensor 51 do not always represent the mounting posture of the front camera 5 with respect to the vehicle body, but are affected by the vehicle body inclination caused by the road gradient. It is the value. Further, since the host vehicle is assumed to travel on roads with various slopes, the detection result output by the first inclination sensor 51 is also a value affected by various slopes.

However, when the required number of detections is set to a sufficiently large number of detections, the road with the highest traveling frequency is a horizontal (including substantially horizontal) road. That is, if the required number of detections is sufficiently increased, the mode values of the pitch angle θp1 and the roll angle θr1 accumulated in the first detection result storage unit M11 are in a state where the host vehicle is present on a horizontal road. It can be expected to be a detection result at.

Therefore, by adopting the mode value of the pitch angle θp1 and the roll angle θr1 accumulated over a sufficiently long time as the pitch angle P1 and roll angle R1 of the front camera 5, the in-vehicle camera caused by the inclination of the own vehicle. It is possible to suppress erroneous determination of the mounting posture.

Further, as a more preferable aspect of the third embodiment, it is not sufficient to determine that a sufficient amount of data is accumulated only by the number of detections, but when the detection results are collected at a predetermined number of different points, it is sufficient. It may be determined that the amount detection results are collected. For example, when data is collected at 50 locations, it may be determined that a sufficient amount of data has been collected. Further, an upper limit may be set for the number of detection results collected at one place.

In addition, although the aspect which accumulate | stores the detection result when the own vehicle has stopped as an example here was illustrated, it is not restricted to this. Detection results while the host vehicle is traveling may also be accumulated.

Also, if the number of detection results required for specifying the mounting posture of the front camera 5 is increased, it may be difficult to collect a sufficient amount of data in a single trip. Therefore, the first detection result storage unit M11 may be provided in the storage 13 instead of the memory 12. According to such an aspect, data being collected can be retained even after the ignition power is turned off. Here, the trip refers to movement from turning on the ignition power supply to turning off the ignition power supply.

In addition, here, as an example, the driving support system 100B exemplifies a mode in which only the front camera 5 is provided as a vehicle-mounted camera, but the present invention is not limited thereto. The driving support system 100B may include the rear camera 6, the right side camera 7, and the left side camera 8 described above. For each in-vehicle camera, the mounting posture may be specified in the same manner as the processing for the front camera 5 described above.

As described above, the first, second, and third embodiments have been described as embodiments of the present disclosure. However, the present disclosure is not limited to the above-described embodiments, and the following modifications are also included in the technical scope of the present disclosure. In addition, the present invention can be implemented with various modifications other than those described below without departing from the scope of the invention.

<Modification 1>
In the first and second embodiments, an example of performing the process of specifying the mounting posture of the in-vehicle camera when the detection results for a desired number of times can be collected while stopping at one point is illustrated. However, it is not limited to this. You may perform the process which pinpoints the mounting attitude | position of a vehicle-mounted camera based on the detection result collected when stopping at several points. Moreover, although the aspect which collects data when the own vehicle has stopped was illustrated, you may implement during driving | running | working as another aspect.

<Modification 2>
In the first embodiment, when the vehicle posture specifying unit F4 determines that the vehicle body is not tilted in the longitudinal direction of the vehicle, the mounting posture specifying unit F5 stores the pitch accumulated in the first detection result storage unit M11. The mode value of the angle θp1 may be adopted as the current pitch angle of the front camera 5 to update the mounting posture data. The same applies to the pitch angle P2 of the rear camera 6.

Further, when the vehicle posture specifying unit F4 determines that the vehicle body is not tilted in the vehicle width direction, the mounting posture specifying unit F5 sets the maximum roll angle θr1 stored in the first detection result storage unit M11. The mode value may be adopted as the current roll angle of the front camera 5 to update the mounting posture data. The same applies to the roll angle of the rear camera 6.

<Modification 3>
In the above, although the aspect which employ | adopts a 2 axis inclination sensor as a camera attitude | position detector as described in a claim was illustrated, it is not restricted to this. It may be a uniaxial tilt sensor for detecting the pitch angle or roll angle of the in-vehicle camera. The same applies to the vehicle body side inclination sensor 3 as a vehicle attitude detector.

Further, although a configuration in which a sensor that outputs an angle such as a pitch angle or a roll angle is employed as the posture index value is exemplified, the present invention is not limited thereto. The camera attitude detector (and vehicle attitude detector) may be a triaxial acceleration sensor or a biaxial acceleration sensor. In this case, the posture index value is an acceleration for each axial direction.

<Modification 4>
Each in-vehicle camera is provided with a geomagnetic sensor (for example, a triaxial geomagnetic sensor) in addition to the tilt sensor, and the control unit 1 can specify the azimuth angle of the host vehicle by a gyro sensor or a geomagnetic sensor provided on the vehicle body. It is good also as a structure. According to such an aspect, the yaw angle of each in-vehicle camera may be specified from the azimuth angle detected by the geomagnetic sensor provided in the in-vehicle camera and the azimuth angle of the host vehicle.

100 / 100A / 100B driving support system, 1.1A / 1B control unit, 3 vehicle body side tilt sensor (vehicle attitude detector), 5 front camera, 6 rear camera, 7 right side camera, 8 left side camera (5-8) In-vehicle camera), 12 memory, 13 storage, 51 1st tilt sensor, 61 2nd tilt sensor, 71 3rd tilt sensor, 81 4th tilt sensor (51 · 61 · 71 · 81 camera attitude detector), F1 stop determination Unit, F2 detection result management unit, F21 displacement amount calculating unit (angular displacement amount calculating unit), F3 updating necessity determining unit, F4 vehicle posture specifying unit, F5 mounting posture specifying unit, F6 parameter adjusting unit, F7 image processing unit, M11 first detection result storage unit, M12 second detection result storage unit, M13 third detection result storage unit, M14 fourth detection result憶部, (M11 ~ 14 camera detection result storage unit) M2 vehicle body side detection result storing unit (detection result storage unit for a vehicle)

Claims (9)

1. An in-vehicle camera (5, 6, 7, 8) installed at a predetermined position of the vehicle and having a predetermined area around the vehicle as a shooting range;
   A camera attitude detector (51, 61, 71, 81) that sequentially detects a camera attitude index value that represents the attitude of the in-vehicle camera with respect to a horizontal plane that is a plane perpendicular to the direction in which gravity acts, provided in the in-vehicle camera;
   A camera detection result storage unit (M11, M12, M13, M14) for storing a result of detecting the camera posture index value by the camera posture detector a predetermined required number of times;
   A mounting posture specifying unit (F5) for specifying a mounting posture of the in-vehicle camera with respect to the vehicle based on a mode value of the camera posture index value accumulated in the camera detection result storage unit. A camera calibration device.
2. An in-vehicle camera (5, 6, 7, 8) installed at a predetermined position of the vehicle and having a predetermined area around the vehicle as a shooting range;
   A camera attitude detector (51, 61, 71, 81) that sequentially detects a camera attitude index value that represents the attitude of the in-vehicle camera with respect to a horizontal plane that is a plane perpendicular to the direction in which gravity acts, provided in the in-vehicle camera;
   A camera detection result storage unit (M11, M12, M13, M14) for storing the camera posture index value detected by the camera posture detector for a predetermined required number of detection times;
   A vehicle attitude specifying unit (F4) for specifying the attitude of the vehicle with respect to the horizontal plane;
   Based on the mode value of the camera posture index value stored in the camera detection result storage unit and the posture of the vehicle specified by the vehicle posture specifying unit, the mounting posture of the in-vehicle camera with respect to the vehicle is determined. A camera calibration device comprising: a mounting orientation specifying unit (F5) for specifying.
3. In claim 2,
   A vehicle attitude detector (3), which is installed in the vehicle body of the vehicle and sequentially detects a vehicle attitude index value representing the attitude of the vehicle body with respect to a horizontal plane that is a plane perpendicular to the direction in which gravity acts;
   A vehicle detection result storage unit (M2) for storing the vehicle posture index value detected by the vehicle posture detector for the required number of detection times,
   The vehicle attitude specifying unit is a vehicle attitude that is an attitude of the vehicle with respect to the horizontal plane based on a vehicle-side mode value that is a mode value of the vehicle attitude index value stored in the vehicle detection result storage unit. Identify
   The mounting posture specifying unit includes a camera-side mode value that is a mode value of the camera posture index value accumulated in the camera detection result storage unit, and the vehicle posture specified by the vehicle posture specifying unit. A camera calibration device characterized in that the mounting posture of the in-vehicle camera is specified based on the camera orientation.
4. In claim 3,
   The camera posture detector detects an inclination angle of the in-vehicle camera with respect to a horizontal plane that is a plane perpendicular to a direction in which gravity acts in a camera side detection direction that is a predetermined detection direction, as the camera posture index value.
   The vehicle attitude detector detects an inclination angle of the vehicle with respect to the horizontal plane in a vehicle-side detection direction which is a predetermined detection direction as the vehicle attitude index value;
   The mounting posture of the in-vehicle camera is represented by an inclination angle of the in-vehicle camera with respect to the horizontal plane in the camera-side detection direction when the vehicle is in a horizontal posture,
   A storage (13) that is a nonvolatile storage medium that stores the vehicle-side detection direction and the camera-side detection direction;
   The mounting posture specifying part is
   The vehicle attitude affects the detection result of the camera attitude detector from the vehicle attitude specified by the vehicle attitude specifying unit and the correspondence between the vehicle-side detection direction and the camera-side detection direction. The amount of vehicle attitude influence that is
   A camera calibration apparatus characterized by specifying the mounting posture of the in-vehicle camera by subtracting the vehicle posture influence amount from the camera-side mode value.
5. In claim 4,
   The storage stores mounting posture data representing the mounting posture of the in-vehicle camera,
   Each time the tilt angle as the camera posture index value detected by the camera posture detector is added to the camera detection result storage unit, the mounting posture stored in the storage is added from the added tilt angle. An angular displacement amount calculation unit (F2) that calculates an angular displacement amount that is a value obtained by subtracting the angle indicated in the data;
   An update necessity determination unit (F3) for determining whether or not the mounting posture data of the in-vehicle camera needs to be updated,
   The camera detection result storage unit stores the angular displacement amount calculated by the angular displacement amount calculation unit,
   The update necessity determination unit obtains the mounting posture data of the in-vehicle camera when the mode value of the angular displacement amount stored in the camera detection result storage unit is equal to or greater than a predetermined threshold. Determine that it needs to be updated,
   When it is determined by the update necessity determination unit that it is necessary to update the mounting posture data of the in-vehicle camera, the mounting posture specifying unit specifies the vehicle posture influence amount and the camera side maximum amount. A camera calibration apparatus characterized by subtracting the vehicle posture influence amount from a frequent value to identify the mounting posture of the in-vehicle camera and update the mounting posture data stored in the storage.
6. In claim 2,
   Comprising at least three in-vehicle cameras,
   Each of the plurality of in-vehicle cameras includes the camera posture detector corresponding to the in-vehicle camera,
   Each of the plurality of camera posture detectors detects, as the camera posture index value, an inclination angle of the in-vehicle camera corresponding to the camera posture detector with respect to the horizontal plane in the camera side detection direction which is a predetermined detection direction. ,
   The camera detection result storage unit stores the detection results of the plurality of camera posture detectors separately for each camera posture detector,
   The mounting posture of the in-vehicle camera is represented by an inclination angle of the in-vehicle camera with respect to the horizontal plane in the camera-side detection direction when the vehicle is in a horizontal posture,
   A storage (13) that is a non-volatile storage medium that stores the camera-side detection direction for each of the plurality of camera attitude detectors and the installation attitude data representing the installation attitude for each of the plurality of in-vehicle cameras;
   Each time an inclination angle detected by the camera attitude detector is added to the camera detection result storage unit, an angle indicated in the mounting attitude data stored in the storage is subtracted from the added inclination angle. An angular displacement amount calculation unit (F2) that calculates an angular displacement amount that is a calculated value and stores the calculated angular displacement amount in the camera detection result storage unit,
   The vehicle posture specifying unit is configured such that the vehicle is horizontal when a majority of the mode values of the angular displacement amount for each in-vehicle camera are less than a predetermined threshold value. It is determined that it is in posture,
   The mounting posture specifying part is
   When the vehicle posture specifying unit determines that the vehicle is in a horizontal posture, among the most frequent values of the angular displacement amount for each of the in-vehicle cameras, the most frequent value is the threshold value or more. The in-vehicle camera corresponding to the value is an update target camera that is the in-vehicle camera whose attachment posture data should be updated.
   Adopting the mode value of the camera posture index value output from the camera posture detector corresponding to the update target camera, stored in the camera detection result storage unit, as the mounting posture of the in-vehicle camera. A camera calibration device.
7. In claim 1,
   The required number of detections is sufficient so that the mode value of the camera posture index value accumulated in the camera detection result storage unit is a value detected when the vehicle is traveling on a horizontal road. The camera calibration device is characterized in that the number of times is high.
8. In any one of Claims 1-7,
   The mounting posture of the in-vehicle camera is represented as a pitch angle, a roll angle, and a yaw angle that are determined based on the vehicle front-rear direction, the vehicle width direction, and the height direction
   The camera orientation detector detects an angle corresponding to at least one of the pitch angle, the roll angle, and the yaw angle as the camera orientation index value.
9. In claim 8,
   The camera posture detector is realized by a biaxial tilt sensor.

Images