JP7081481B2 - Vehicle video processing equipment, vehicle video processing systems, vehicle video processing methods and programs - Google Patents

Description

The present invention relates to a vehicle image processing device, a vehicle image processing system, a vehicle image processing method and a program.

There is known a technique for confirming the surroundings of a vehicle by using video data obtained by photographing the surroundings of the vehicle (see, for example, Patent Document 1). The technique described in Patent Document 1 causes a camera image to be displayed on a display at an intersection with poor visibility registered in a car navigation system.

Japanese Unexamined Patent Publication No. 08-293098

For example, when the vehicle travels on a road having a steep slope, if the inclination angle of the vehicle becomes large, the display area of the video data taken by the photographing unit installed in the vehicle may deviate from the appropriate display area.

The present invention has been made in view of the above, and an object of the present invention is to make it possible to display an image in an appropriate display area regardless of the inclination angle of the vehicle.

In order to solve the above-mentioned problems and achieve the object, the vehicle video processing apparatus according to the present invention includes a video data acquisition unit that acquires video data taken by a shooting unit that shoots the periphery of the vehicle, and the video data. The distortion correction unit that corrects the distortion of the video data acquired by the acquisition unit using the distortion parameter and outputs the distortion correction video data, the tilt angle detection unit that detects the tilt angle of the vehicle, and the video data acquisition unit. Based on the road surface area detection unit that detects the road surface area captured in the video data acquired by the vehicle, the inclination angle of the vehicle detected by the inclination angle detection unit, and the road surface area detected by the road surface area detection unit. The strain correction unit includes a determination unit for determining a change in the inclination angle of the vehicle and the area of the road surface region with respect to the threshold value. When it is determined that the area of the road surface region has changed by the threshold value or more, the strain parameter is changed according to the change in the area of the road surface region to correct the strain.

The vehicle image processing system according to the present invention includes at least one of the above-mentioned vehicle image processing device, the photographing unit, and a display unit that displays image data generated by the vehicle image processing device. It is characterized by that.

In the vehicle video processing method according to the present invention, a video data acquisition step of acquiring video data taken by a shooting unit that shoots the periphery of the vehicle and a distortion parameter of the video data acquired by the video data acquisition step are used as distortion parameters. A distortion correction step that corrects using and outputs distortion-corrected video data, a tilt angle detection step that detects the tilt angle of the vehicle, and a road surface that is copied to the video data acquired by the video data acquisition step. The inclination angle of the vehicle and the road surface based on the road surface area detection step for detecting the region, the inclination angle of the vehicle detected by the inclination angle detection step, and the road surface area detected by the road surface area detection step. The strain correction step includes a determination step for determining a change in the area of the region with respect to the threshold value, and the determination step changes the inclination angle of the vehicle by the threshold value or more and the area of the road surface region by the threshold value or more. If it is determined, the strain parameter is changed according to the change in the area of the road surface region to correct the strain.

The program according to the present invention corrects the distortion of the video data acquired by the video data acquisition step of acquiring the video data captured by the photographing unit that captures the periphery of the vehicle and the distortion of the video data by using the distortion parameter. Then, the distortion correction step for outputting the distortion correction video data, the tilt angle detection step for detecting the tilt angle of the vehicle, and the road surface area reflected in the video data acquired by the video data acquisition step are detected. Based on the road surface area detection step, the inclination angle of the vehicle detected by the inclination angle detection step, and the road surface area detected by the road surface area detection step, the inclination angle of the vehicle and the area of the road surface area are determined. The strain correction step includes a determination step for determining a change with respect to a threshold value, and it is determined that the inclination angle of the vehicle is equal to or greater than the threshold value and the area of the road surface region has changed by the determination step or more. In this case, the computer operating as the vehicle image processing device is made to perform the correction of the distortion by changing the distortion parameter according to the change in the area of the road surface region.

According to the present invention, it is possible to display an image in an appropriate display area regardless of the inclination angle of the vehicle.

FIG. 1 is a block diagram showing a configuration example of an image processing device for a vehicle according to an embodiment. FIG. 2 is a diagram illustrating a state in which a vehicle travels on a flat road. FIG. 3 is a diagram illustrating a state in which a vehicle travels on a flat road. FIG. 4 is a diagram showing an example of video data taken by a camera when a vehicle travels on a flat road. FIG. 5 is a diagram showing an example of a display area in which distortion correction processing is performed in the video data shown in FIG. FIG. 6 is a diagram illustrating a state in which a vehicle travels on a road having a steep uphill slope. FIG. 7 is a diagram illustrating a state in which a vehicle travels on a road having a steep uphill slope. FIG. 8 is a diagram showing an example of video data taken by a camera when a vehicle travels on a road having a steep uphill slope. FIG. 9 is a diagram showing an example of a display area for performing distortion correction processing in the video data shown in FIG. FIG. 10 is a diagram illustrating a state in which a vehicle travels on a road having a steep downward slope. FIG. 11 is a diagram illustrating a state in which a vehicle travels on a road having a steep downward slope. FIG. 12 is a diagram showing an example of video data taken by a camera when a vehicle travels on a road having a steep downward slope. FIG. 13 is a diagram showing an example of a display area for performing distortion correction processing in the video data shown in FIG. 12. FIG. 14 is a flowchart showing a processing flow in the vehicle video processing apparatus according to the embodiment. FIG. 15 is a diagram showing an example of a display area for performing distortion correction processing when an intersecting road is curved. FIG. 16 is a diagram illustrating a state in which a road intersecting a road on which a vehicle is traveling is curved. FIG. 17 is a diagram showing an example of a display area in which distortion correction processing is performed when a vehicle travels on a snowy road.

Hereinafter, embodiments of a vehicle image processing apparatus, a vehicle image processing system, a vehicle image processing method, and a program according to the present invention will be described in detail with reference to the accompanying drawings. The present invention is not limited to the following embodiments.

FIG. 1 is a block diagram showing a configuration example of an image processing device for a vehicle according to an embodiment. The vehicle image processing system 1 displays, for example, an image for confirming the surroundings of the vehicle V when traveling at a low speed or when parking. The vehicle image processing system 1 displays an image in an appropriate display area regardless of the inclination angle of the vehicle V. The vehicle image processing system 1 includes a camera unit (shooting unit) 10, a display unit 21, a headlight switch 22, a vehicle speed sensor 23, a 3-axis gyro sensor 24, and a vehicle image processing device 30.

The camera unit 10 has a camera that photographs the surroundings of the vehicle V. A plurality of cameras may be arranged. In the present embodiment, the camera unit 10 has a front camera 11, a right side camera 12, a left side camera 13, and a rear camera 14.

The front camera 11 is arranged in front of the vehicle V and photographs the surroundings centered on the front of the vehicle V. The front camera 11 captures a super wide angle, for example, a shooting range of about 190 ° in the horizontal direction. The front camera 11 outputs the captured image to the image processing unit 34 of the vehicle image processing device 30.

The right-side camera 12 is arranged on the right side of the vehicle V, and photographs the surroundings centered on the right side of the vehicle V. The right-side camera 12 captures an ultra-wide-angle lens, for example, a shooting range of about 190 ° in the horizontal direction. The right-side camera 12 outputs the captured image to the image processing unit 34 of the vehicle image processing device 30.

The left-side camera 13 is arranged on the left side of the vehicle V, and photographs the surroundings centered on the left side of the vehicle V. The left-side camera 13 captures an ultra-wide-angle lens, for example, a shooting range of about 190 ° in the horizontal direction. The left-side camera 13 outputs the captured image to the image processing unit 34 of the vehicle image processing device 30.

The rear camera 14 is arranged behind the vehicle V and photographs the surroundings centered on the rear of the vehicle V. The rear camera 14 captures a super wide angle, for example, a shooting range of about 190 ° in the horizontal direction. The rear camera 14 outputs the captured image to the image processing unit 34 of the vehicle image processing device 30.

The camera unit 10 including the front camera 11, the right side camera 12, the left side camera 13, and the rear camera 14 captures all directions of the vehicle V.

The display unit 21 is, for example, a display including a liquid crystal display (LCD: Liquid Crystal Display) or an organic EL (Organic Electro-Luminence) display. The display unit 21 displays, for example, an image around the vehicle V based on the image signal output from the vehicle image processing device 30 of the vehicle image processing system 1. The display unit 21 may be dedicated to the vehicle image processing system 1 or may be used jointly with other systems including, for example, a navigation system. The display unit 21 is arranged at a position that is easily visible to the user. In the present embodiment, the display unit 21 is arranged on the center console at the center of the dashboard in the vehicle width direction.

The headlight switch 22 outputs a signal capable of detecting whether the headlight is on or off to the headlight lighting detection unit 31 of the vehicle image processing device 30.

The vehicle speed sensor 23 detects the vehicle speed of the vehicle V. More specifically, the vehicle speed sensor 23 is arranged on the drive shaft or the tire of the vehicle V. The vehicle speed sensor 23 detects a pulse signal according to the rotation of the drive shaft or the tire. The vehicle speed sensor 23 outputs the detected pulse signal to the vehicle speed detection unit 32 of the vehicle image processing device 30.

The 3-axis gyro sensor 24 detects the slope of the road on which the vehicle V travels. The 3-axis gyro sensor 24 is a sensor that detects the inclination of the vehicle V. The 3-axis gyro sensor 24 detects, for example, the angular velocity of the vehicle V. More specifically, the 3-axis gyro sensor 24 has a roll rate gyro, a pitch rate gyro, and a yaw rate gyro. The roll rate gyro detects the roll angular velocity, which is the angular velocity of rotation about the vehicle V in the front-rear direction. The pitch rate gyro detects the pitch angular velocity, which is the angular velocity of rotation about the vehicle V in the left-right direction. The yaw rate gyro detects the yaw angular velocity, which is the angular velocity of rotation of the vehicle V about the vertical direction. The 3-axis gyro sensor 24 uses the roll angular velocity detected by the roll rate gyro, the pitch angular velocity detected by the pitch rate gyro, and the yaw angular velocity detected by the yaw rate gyro as angular velocity signals as the tilt angle detection unit of the vehicle image processing device 30. Output to 33.

The vehicle image processing device 30 generates an image for confirming the surroundings of the vehicle V and displays it on the display unit 21. The vehicle image processing device 30 loads the program stored in the storage unit 39 into the memory and executes the instruction included in the program. The vehicle image processing device 30 includes a headlight lighting detection unit 31, a vehicle speed detection unit 32, an inclination angle detection unit 33, an image processing unit 34, a determination unit 35, a photometric area control unit 36, and a display control unit. It has 37 and a storage unit 39.

The headlight lighting detection unit 31 detects whether the headlight is on or off from the signal output from the headlight switch 22. When the headlights are on, the road surface illuminated by the headlights is captured in the front image data captured by the front camera 11. The road surface illuminated by the headlights reflected in the front image data has a difference in hue, brightness, saturation, and brightness from the road surface not illuminated by the headlights. The headlight lighting detection unit 31 outputs the detection result to the determination unit 35.

The vehicle speed detection unit 32 detects the vehicle speed of the vehicle V from the pulse signal acquired from the vehicle speed sensor 23. The vehicle speed detection unit 32 outputs vehicle speed information indicating the calculated vehicle speed to the determination unit 35.

The inclination angle detection unit 33 detects the inclination angle of the road on which the vehicle V travels. In the present embodiment, the tilt angle detection unit 33 acquires the angular velocity signal output from the 3-axis gyro sensor 24. The tilt angle detection unit 33 calculates the roll angle, pitch angle, and yaw angle, which indicate the tilt of the vehicle V, from the roll angular velocity, pitch angular velocity, and yaw angular velocity included in the angular velocity signal. The tilt angle detection unit 33 outputs the calculated roll angle, pitch angle, and yaw angle to the determination unit 35 as tilt angle information. The tilt angle detection unit 33 may calculate only the pitch angle and output only the pitch angle as tilt angle information to the determination unit 35.

The image processing unit 34 generates display image data of the image for display to be displayed on the display unit 21 based on the image data captured by the camera unit 10. The video processing unit 34 includes a video data acquisition unit 341, a road surface area detection unit 342, a distortion correction unit 343, and an image generation unit 344.

The video data acquisition unit 341 acquires video data taken by the camera unit 10. The video data acquisition unit 341 has captured the front video data captured by the front camera 11, the right video data captured by the right camera 12, the left video data captured by the left camera 13, and the rear camera 14. Acquires the rear video data.

The road surface area detection unit 342 detects the road surface area captured in the video data acquired by the video data acquisition unit 341. The road surface area is an area of the road surface captured in the video data. The road surface area is defined by the shape of the road, the width of the road, and the depth of the road. In the present embodiment, the road surface area detection unit 342 will be described as detecting the road surface area captured in the forward video data. The road surface area detection unit 342 stores the road surface area detected when the vehicle V is traveling on a flat road as a reference road surface area. It is preferable that the reference road surface area is stored for each road width of the road on which the vehicle V travels. The method for detecting the road surface may be any known method and is not limited. When the detection result of the headlight lighting detection unit 31 indicates that the headlight is lit, the road surface area detection unit 342 indicates that the road surface reflected in the video data is illuminated by the headlight, and therefore the hue of the video data. , Brightness, saturation, and brightness are corrected to detect the road surface area of the road surface.

The distortion correction unit 343 corrects the distortion of the display area, which is a part of the ultra-wide-angle video data acquired by the video data acquisition unit 341, using the distortion parameter, and outputs the rectangular distortion correction video data. .. The distortion-corrected video data includes forward distortion-corrected video data in which the front video data is corrected, right-side distortion-corrected video data in which the right-side video data is corrected, left-side distortion-corrected video data in which the left-side video data is corrected, and rear. Includes backward distortion corrected video data with video data corrected. When it is not necessary to distinguish between the front distortion correction video data, the right side distortion correction video data, the left side distortion correction video data, and the rear distortion correction video data, the data will be described as distortion correction video data.

When the determination unit 35 determines that the inclination angle of the vehicle V is equal to or greater than the threshold value, the vehicle speed of the vehicle V is equal to or less than the threshold value, and the area of the road surface region is equal to or greater than the threshold value, the strain correction unit 343 determines that the road surface region has changed. The distortion parameter is changed according to the change in the area to correct the distortion, and the distortion-corrected video data is output. More specifically, the distortion correction unit 343 corrects the distortion by changing the distortion parameter so as to shift the display area for correcting the distortion upward when the area of the road surface area is increased as compared with the reference road surface area. And output the distortion correction video data. When the area of the road surface area is smaller than the reference road surface area, the distortion correction unit 343 changes the distortion parameter so as to shift the display area for correcting the distortion downward, and corrects the distortion. Output the corrected video data.

A process in the strain correction unit 343 when the vehicle V is traveling on a flat road will be described with reference to FIGS. 2 to 5. FIG. 2 is a diagram illustrating a state in which a vehicle travels on a flat road. FIG. 3 is a diagram illustrating a state in which a vehicle travels on a flat road. FIG. 4 is a diagram showing an example of video data taken by a camera when a vehicle travels on a flat road. FIG. 5 is a diagram showing an example of a display area in which distortion correction processing is performed in the video data shown in FIG. As shown in FIGS. 2 and 3, when the vehicle V is traveling on a flat road, the front image data captured by the front camera 11 is, for example, as shown in FIG. Such forward video data is used as reference video data, and a display area indicating an area to be visually recognized as an image by the user and a distortion parameter are set. Further, the road surface area detected from the forward video data is the reference road surface area. For example, the display area is set to A1, which is a barrel-shaped area shown in the figure. Distortion parameters are set so that distortion is corrected for such a display area A1 and rectangular forward distortion correction video data is generated.

Here, the camera unit 10 including the front camera 11 is capturing an image having an ultra-wide viewing angle. For applications that confirm a wide range, such as when generating an image that confirms the surroundings of the vehicle V, it is desirable to correct the distortion so that the entire viewing angle can be secured in the horizontal direction. In such a case, if the distortion is corrected while ensuring a natural sense of distance from the front, the display area in the vertical direction as opposed to the horizontal direction becomes as if a part of the shooting range of the camera is cut out. On the contrary, if the vertical display area is secured as it is and the distortion is corrected, the image becomes uncomfortable so that the sense of distance in front is felt farther than it actually is. Therefore, it is necessary to adjust the display area in the vertical direction to be displayed depending on the inclination angle of the vehicle V. Specifically, the distortion parameter is changed so as to shift the display area for correcting the distortion from the center of the shooting range of the camera to the upper side or the lower side according to the increase / decrease ratio of the road surface area.

6 to 9, the processing in the strain correction unit 343 when the vehicle V is traveling on the road having a steep uphill slope will be described. FIG. 6 is a diagram illustrating a state in which a vehicle travels on a road having a steep uphill slope. FIG. 7 is a diagram illustrating a state in which a vehicle travels on a road having a steep uphill slope. FIG. 8 is a diagram showing an example of video data taken by a camera when a vehicle travels on a road having a steep uphill slope. FIG. 9 is a diagram showing an example of a display area for performing distortion correction processing in the video data shown in FIG. As shown in FIGS. 6 and 7, when the vehicle V is traveling on a road having a steep uphill slope, the front image data captured by the front camera 11 is, for example, as shown in FIG. Here, it is assumed that there is an intersection that intersects the intersection road R1 in the middle of the steep uphill slope. Another vehicle V1 is traveling on the intersection road R1. When the vehicle V rises forward, the shooting range of the front camera 11 faces upward. As a result, the road surface area in the front image data display area is reduced. Therefore, the strain parameter is changed so that the position of the road surface region in the display region becomes appropriate, and the strain parameter is changed so that A2, which is a barrel-shaped region shown in the figure, becomes the display region. In this way, the distortion is corrected for the display area A2 and the forward distortion correction video data is output. The display area A1 shown by the broken line in the figure is a display area when the distortion parameter is not changed. The same applies to the following figure.

A process in the strain correction unit 343 when the vehicle V is traveling on a road having a steep downward slope will be described with reference to FIGS. 10 to 13. FIG. 10 is a diagram illustrating a state in which a vehicle travels on a road having a steep downward slope. FIG. 11 is a diagram illustrating a state in which a vehicle travels on a road having a steep downward slope. FIG. 12 is a diagram showing an example of video data taken by a camera when a vehicle travels on a road having a steep downward slope. FIG. 13 is a diagram showing an example of a display area for performing distortion correction processing in the video data shown in FIG. 12. As shown in FIGS. 10 and 11, when the vehicle V is traveling on a road having a steep downward slope, the front image data captured by the front camera 11 is, for example, as shown in FIG. Here, it is assumed that there is an intersection that intersects the intersection road R1 in the middle of the steep descent. Another vehicle V1 is traveling on the intersection road R1. When the vehicle V is lowered forward, the shooting range of the front camera 11 is downward. As a result, the road surface area in the front image data display area increases. Therefore, the strain parameter is changed so that the position of the road surface region in the display region becomes appropriate, and the strain parameter is changed so that the barrel-shaped region A3 shown in the figure becomes the display region. In this way, the distortion is corrected for the display area A3 and the forward distortion correction video data is output.

The image generation unit 344 generates display image data for confirming the periphery of the vehicle V based on the distortion correction image data output by the distortion correction unit 343. In the present embodiment, the image generation unit 344 generates display image data that can confirm the front of the vehicle V based on the forward distortion correction image data.

The image generation unit 344 may generate display image data of a bird's-eye view image including a virtual own vehicle image looking down on the vehicle V from above based on the distortion correction image data output by the distortion correction unit 343. In this case, the image generation unit 344 synthesizes the viewpoint conversion process and the own vehicle icon indicating the vehicle V with the forward distortion correction video data, the right side distortion correction video data, the left side distortion correction video data, and the rear distortion correction video data. Performs synthesis processing to generate display image data of a bird's-eye view image that displays the surroundings of the vehicle V. The method for generating the bird's-eye view image may be any known method and is not limited.

The determination unit 35 changes the inclination angle of the vehicle V and the threshold value of the area of the road surface area based on the inclination angle of the vehicle V detected by the inclination angle detection unit 33 and the road surface area detected by the road surface area detection unit 342. judge. More specifically, the determination unit 35 is based on the inclination angle of the vehicle V detected by the inclination angle detection unit 33, the vehicle speed detected by the vehicle speed detection unit 32, and the road surface area detected by the road surface area detection unit 342. It is determined whether or not the inclination angle of V is equal to or more than the threshold value, the vehicle speed of vehicle V is equal to or less than the threshold value, and the area of the road surface region has changed by not more than the threshold value. The threshold value of the inclination angle is, for example, about 5 °. The threshold value of the vehicle speed is, for example, about 10 km / h. The threshold value of the area of the road surface region is, for example, about 20%.

Whether or not the area of the road surface area has changed by the threshold value or more is determined by whether the area of the road surface area detected by the road surface area detection unit 342 is about 20% or more of the area of the standard road surface area of a road having the same road width. , Judge whether it has increased or decreased. When it is determined that the area of the road surface area detected by the road surface area detection unit 342 has increased or decreased by about 20% or more with respect to the area of the reference road surface area, it is determined that the area of the road surface area has changed by the threshold value or more. When it is determined that the area of the road surface area detected by the road surface area detection unit 342 has not increased or decreased by about 20% or more with respect to the area of the reference road surface area, the area of the road surface area has not changed by the threshold value or more. judge.

Alternatively, whether or not the area of the road surface area has changed by more than the threshold value depends on whether or not the area of the road surface area detected by the road surface area detection unit 342 has increased or decreased by about 20% or more in a predetermined period or a predetermined mileage. You may judge. When it is determined that the area of the road surface area detected by the road surface area detection unit 342 has increased or decreased by about 20% or more in a predetermined period or a predetermined mileage, it is determined that the area of the road surface area has changed by a threshold value or more. When it is determined that the area of the road surface area detected by the road surface area detection unit 342 has not increased or decreased by about 20% or more in a predetermined period or a predetermined mileage, it is determined that the area of the road surface area has not changed by the threshold value or more. ..

When the distortion correction unit 343 changes the distortion parameter, the photometric area control unit 36 controls the position of the photometric area when the camera unit 10 takes a picture in accordance with the change in the distortion parameter. For example, the position of the photometric region when the camera unit 10 shoots in a normal state is the central portion of the display region. When the distortion correction unit 343 shifts the display area for correcting distortion upward, the metering area control unit 36 shifts the metering area upward so that the central portion of the display area shifted upward becomes the metering area. The control signal is output to the camera unit 10. When the distortion correction unit 343 shifts the display area for correcting distortion downward, the metering area control unit 36 lowers the metering area so that the central portion of the display area shifted downward becomes the metering area. A control signal to be shifted to is output to the camera unit 10.

The display control unit 37 controls the display unit 21 to display the display image data generated by the image generation unit 344 of the image processing unit 34.

The storage unit 39 stores data required for various processes in the vehicle image processing device 30 and various processing results. The storage unit 39 stores, for example, a semiconductor memory element such as a RAM (Random Access Memory), a ROM (Read Only Memory), or a flash memory (Flash Memory), or an external storage device via a hard disk, an optical disk, or a communication network. It is a device.

Next, the flow of processing in the vehicle image processing apparatus 30 will be described with reference to FIG. FIG. 14 is a flowchart showing a processing flow in the vehicle video processing apparatus 30 according to the embodiment. Here, a case will be described in which an image for confirming the safety in front of the vehicle V is displayed based on the front image data captured by the front camera 11.

It is assumed that the vehicle image processing system 1 is started when the engine of the vehicle V is started. While the vehicle image processing system 1 is activated, the camera unit 10 photographs the surroundings of the vehicle V. While the vehicle video processing system 1 is being activated, the video data acquisition unit 341 acquires video data taken by the camera unit 10.

The vehicle image processing device 30 detects the lighting of the headlights (step S101). The vehicle image processing device 30 detects whether the headlights are on or off from the signal output from the headlight switch 22 by the headlight lighting detection unit 31. The vehicle image processing device 30 proceeds to step S102.

The vehicle image processing device 30 detects the road surface region (step S102). The vehicle image processing device 30 uses a known road surface detection method to detect the road surface area reflected in the front image data by the road surface area detection unit 342. The vehicle image processing device 30 proceeds to step S103.

The vehicle image processing device 30 determines whether the vehicle speed is equal to or lower than the threshold value by the determination unit 35 (step S103). When the determination unit 35 determines that the vehicle speed detected by the vehicle speed detection unit 32 is equal to or less than the threshold value (Yes in step S103), the vehicle image processing device 30 proceeds to step S104. When the determination unit 35 determines that the vehicle speed detected by the vehicle speed detection unit 32 is not equal to or less than the threshold value (No in step S103), the vehicle image processing device 30 re-executes the process of step S101.

When it is determined that the vehicle speed detected by the vehicle speed detection unit 32 is equal to or less than the threshold value (Yes in step S103), the vehicle image processing apparatus 30 determines whether the pitch angle is equal to or more than the threshold value by the determination unit 35 (step). S104). When the determination unit 35 determines that the pitch angle detected by the inclination angle detection unit 33 is equal to or greater than the threshold value (Yes in step S104), the vehicle image processing device 30 proceeds to step S105. When the determination unit 35 determines that the pitch angle detected by the inclination angle detection unit 33 is not equal to or greater than the threshold value (No in step S104), the vehicle image processing device 30 proceeds to step S111.

When it is determined that the pitch angle detected by the inclination angle detection unit 33 is equal to or greater than the threshold value (Yes in step S104), the vehicle image processing device 30 determines the road surface area detected by the road surface area detection unit 342 by the determination unit 35. Based on this, it is determined whether the area of the road surface region has increased or decreased (step S105). When the determination unit 35 determines that the area of the road surface area has increased or decreased (Yes in step S105), the vehicle image processing device 30 proceeds to step S106. When the determination unit 35 determines that the area of the road surface area has not increased or decreased (No in step S105), the vehicle image processing device 30 re-executes the process of step S101.

When it is determined that the area of the road surface area has increased or decreased (Yes in step S105), the vehicle image processing apparatus 30 performs correction processing of the display area (step S106). The vehicle image processing device 30 performs correction processing of a display area for correcting distortion by changing the distortion parameter according to a change in the area of the road surface area by the distortion correction unit 343. The vehicle image processing device 30 changes the distortion parameter so as to shift the display area for correcting the distortion upward when the area of the road surface area is increased by the distortion correction unit 343, and the display area for correcting the distortion. Perform correction processing. When the area of the road surface area is reduced by the distortion correction unit 343, the vehicle image processing device 30 changes the distortion parameter so as to shift the display area for correcting the distortion downward to the display area for correcting the distortion. Performs correction processing. The vehicle image processing device 30 proceeds to step S107.

The vehicle image processing device 30 generates display image data (step S107). The vehicle image processing device 30 generates display image data from the forward distortion correction image data by the image generation unit 344. The vehicle image processing device 30 proceeds to step S108.

The vehicle image processing device 30 causes the display unit 21 to display the generated display image data by the display control unit 37 (step S108). The vehicle image processing device 30 proceeds to step S109.

The vehicle image processing device 30 determines whether the pitch angle is less than the threshold value by the determination unit 35 (step S109). When the determination unit 35 determines that the pitch angle detected by the inclination angle detection unit 33 is less than the threshold value (Yes in step S109), the vehicle image processing device 30 proceeds to step S110. When the determination unit 35 determines that the pitch angle detected by the inclination angle detection unit 33 is not less than the threshold value (No in step S109), the vehicle image processing device 30 re-executes the process of step S109.

When it is determined that the pitch angle detected by the inclination angle detecting unit 33 is less than the threshold value (Yes in step S109), the vehicle image processing apparatus 30 returns the display area to the initial setting (step S110). In the vehicle image processing device 30, the distortion correction unit 343 returns the distortion parameter changed in step S106 to the initial value. The vehicle image processing device 30 returns the distortion parameter to the distortion parameter when the vehicle V is traveling on a flat road by the distortion correction unit 343. The vehicle image processing device 30 proceeds to step S113.

When it is determined that the pitch angle detected by the tilt angle detection unit 33 is not equal to or greater than the threshold value (No in step S104), the vehicle image processing device 30 generates display image data (step S111). The vehicle video processing device 30 generates display video data from the video data by the video generation unit 344. The vehicle image processing device 30 proceeds to step S112.

The vehicle image processing device 30 causes the display unit 21 to display the display image data by the display control unit 37 (step S112). The vehicle image processing device 30 proceeds to step S113.

The vehicle image processing device 30 determines whether the vehicle speed is higher than the threshold value by the determination unit 35 (step S113). When the determination unit 35 determines that the vehicle speed detected by the vehicle speed detection unit 32 is larger than the threshold value (Yes in step S113), the vehicle image processing device 30 proceeds to step S114. When the determination unit 35 determines that the vehicle speed detected by the vehicle speed detection unit 32 is not greater than the threshold value (No in step S113), the vehicle image processing device 30 re-executes the process of step S113.

When it is determined that the vehicle speed detected by the vehicle speed detection unit 32 is higher than the threshold value (Yes in step S113), the vehicle image processing device 30 stops displaying the displayed image data (step S114). The vehicle image processing device 30 proceeds to step S115.

The vehicle image processing device 30 determines whether to end the processing (step S115). More specifically, the vehicle image processing device 30 determines whether or not to end the processing based on the presence or absence of the end trigger. The end trigger is, for example, a combination of engine off, gears in parking, side brake operation, and the like. The end trigger may be an end trigger by a user operation. If there is an end trigger, the vehicle image processing device 30 determines that the processing is completed (Yes in step S115), and ends the processing. If there is no end trigger, the vehicle video processing apparatus 30 determines that the processing is not completed (No in step S115), and executes the processing in step S101 again.

In this way, when the determination unit 35 determines that the inclination angle of the vehicle V is equal to or greater than the threshold value, the vehicle speed of the vehicle V is equal to or less than the threshold value, and the area of the road surface region is equal to or greater than the threshold value, the distortion correction unit 343 determines. Corrects the strain by changing the strain parameter according to the change in the area of the road surface region. When the area of the road surface area increases, the distortion correction unit 343 corrects the distortion by changing the distortion parameter so as to shift the display area for correcting the distortion upward. When the area of the road surface area is reduced, the distortion correction unit 343 corrects the distortion by changing the distortion parameter so as to shift the display area for correcting the distortion downward.

As described above, in the present embodiment, when it is determined that the inclination angle of the vehicle V is equal to or greater than the threshold value, the vehicle speed of the vehicle V is equal to or less than the threshold value, and the area of the road surface region is equal to or greater than the threshold value and has changed, the area of the road surface region is determined. The distortion can be corrected by changing the distortion parameter according to the change in. In the present embodiment, when the area of the road surface area increases, the distortion can be corrected by changing the distortion parameter so as to shift the display area for correcting the distortion upward. When the area of the road surface area is reduced, the distortion can be corrected by changing the distortion parameter so as to shift the display area for correcting the distortion downward. In this way, according to the present embodiment, it is possible to display an image in an appropriate display area regardless of the inclination angle of the vehicle V.

In the present embodiment, when the distortion parameter is changed, the position of the photometric region when the camera unit 10 takes a picture can be controlled according to the change of the distortion parameter. According to the present embodiment, an appropriate photometric region can be set regardless of the inclination angle of the vehicle V, and photometry can be appropriately performed for shooting.

As described above, according to the present embodiment, it is possible to confirm the surroundings of the vehicle V with an appropriate image regardless of the inclination angle of the vehicle V.

Although the vehicle image processing system 1 according to the present invention has been described so far, it may be implemented in various different forms other than the above-described embodiment.

Each component of the illustrated vehicle image processing system 1 is a functional concept and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of each device is not limited to the one shown in the figure, and all or part of the device is functionally or physically dispersed or integrated in an arbitrary unit according to the processing load and usage status of each device. You may.

The configuration of the vehicle video processing system 1 is realized, for example, by a program loaded in a memory as software. In the above embodiment, it has been described as a functional block realized by cooperation of these hardware or software. That is, these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

The above-mentioned components include those that can be easily assumed by those skilled in the art and those that are substantially the same. Further, the above configurations can be combined as appropriate. Further, various omissions, substitutions or changes of the configuration can be made without departing from the gist of the present invention.

A case where the road on which the vehicle V is traveling and the crossing road are curved will be described with reference to FIGS. 15 and 16. FIG. 15 is a diagram showing an example of a display area for performing distortion correction processing when an intersecting road is curved. FIG. 16 is a diagram illustrating a state in which a road intersecting a road on which a vehicle is traveling is curved. When the road on which the vehicle V is traveling and the intersecting road are curved, the distortion correction unit 343 may change the distortion parameter according to the shape of the curve to correct the distortion. The vehicle image processing device 30 has a navigation information acquisition unit that acquires the current position information of the current position of the vehicle V and the map information around the vehicle V from a navigation system (not shown). Further, the vehicle image processing device 30 has a road shape detecting unit that detects the shape of the road based on the information acquired by the navigation information acquisition unit. The strain correction unit 343 corrects the strain by changing the strain parameter according to the change in the area of the road surface region and the shape of the road detected by the road shape detection unit. The distortion correction unit 343 shifts the display area for correcting distortion to the upper side or the lower side so that the display surface area on the curved side can be displayed far away according to the curve shape of the crossing road. As described above, the distortion parameter is changed so that A4, which is a region having a shape shown in the figure, becomes a display region. As a result, when the crossing road is curved, it is possible to display an image around the vehicle V in the display area that matches the curved shape, which is difficult for the user to directly see.

A case where the vehicle V is traveling on a snowy road or the like will be described with reference to FIG. FIG. 17 is a diagram showing an example of a display area in which distortion correction processing is performed when a vehicle travels on a snowy road. For example, if the road surface area detection unit 342 cannot detect the road surface due to snow or flooding, the strain parameter is changed so that the display area is wider, and A5, which is the area of the shape shown in the figure, is the display area. Change the distortion parameter so that As a result, when it is difficult to detect the road surface area due to snow or flooding, it is possible to display an image around the vehicle V in a display area in which the road surface area is wider than usual.

A case of displaying an image for confirming the surroundings of the vehicle V when parking will be described. In this case, the vehicle image processing device 30 executes the processing of the flowchart shown in FIG. 14 when it is determined that there is a reverse trigger. The reverse trigger means, for example, that the shift position is set to "reverse" or that the traveling direction of the vehicle V is behind the vehicle V in the front-rear direction. Then, in step S102, the road surface area captured in the rear video data is detected, and in step S105, it is determined whether or not the area of the road surface area of the rear video data has increased or decreased. Then, in step S106, the distortion parameter is changed to correct the distortion of the rear video data. In this way, when parked, the image behind the vehicle V in an appropriate display area can be displayed regardless of the inclination angle of the vehicle V.

The vehicle image processing system 1 can also be applied to a so-called drive recorder that captures and records surrounding images while the vehicle V is traveling. In this case, regardless of the vehicle speed of the vehicle V, the determination unit 35 inclines the vehicle V based on the inclination angle of the vehicle V detected by the inclination angle detection unit 33 and the road surface area detected by the road surface area detection unit 342. It is determined whether or not the angle is equal to or greater than the threshold value and the area of the road surface area is equal to or greater than the threshold value and has changed. Further, when the determination unit 35 determines that the inclination angle of the vehicle V is equal to or greater than the threshold value and the area of the road surface region has changed by the threshold value or more, the distortion correction unit 343 distorts according to the change in the area of the road surface region. Change the parameters to correct the distortion. The vehicle video processing apparatus 30 executes processing other than steps S103, S113, and S114 in the flowchart shown in FIG. Further, when applied to a drive recorder, apart from the storage unit 39, an external recording device including, for example, a card-type recording medium for storing video data is provided outside the vehicle video processing device 30. You may.

The vehicle image processing device 30 may have an optical axis control unit that controls the position of the optical axis of the headlight according to the change of the distortion parameter when the distortion correction unit 343 changes the distortion parameter. This makes it possible to display an image around the appropriate vehicle V.

Instead of the above-mentioned 3-axis gyro sensor 24, a 6-axis sensor having a function of a 3-axis acceleration sensor capable of detecting acceleration in the X-direction, Y-direction, and Z-direction and a 3-axis gyro sensor may be used. ..

1 Vehicle image processing system 10 Camera unit 21 Display unit 22 Headlight switch 23 Vehicle speed sensor 24 3-axis gyro sensor 30 Vehicle image processing device 31 Headlight lighting detection unit 32 Vehicle speed detection unit 33 Tilt angle detection unit 34 Video processing unit 341 Video data acquisition unit 342 Road surface area detection unit 343 Distortion correction unit 344 Image generation unit 35 Judgment unit 36 Measurement area control unit 37 Display control unit 39 Storage unit

Claims (8)

The video data acquisition unit that acquires the video data taken by the shooting unit that shoots the surroundings of the vehicle, and the video data acquisition unit.
A distortion correction unit that corrects the distortion of the video data acquired by the video data acquisition unit using a distortion parameter and outputs the distortion correction video data.
An inclination angle detection unit that detects the inclination angle of the vehicle,
A road surface area detection unit that detects a road surface area captured in the video data acquired by the video data acquisition unit, and a road surface area detection unit.
A determination unit that determines a change in the inclination angle of the vehicle and the threshold value of the area of the road surface region based on the inclination angle of the vehicle detected by the inclination angle detection unit and the road surface region detected by the road surface region detection unit. When,
Equipped with
When the determination unit determines that the inclination angle of the vehicle is equal to or greater than the threshold value and the area of the road surface region has changed by the threshold value or more, the distortion correction unit determines that the area of the road surface region has changed. Correct the distortion by changing the distortion parameter,
An image processing device for vehicles characterized by this. The strain correction unit shifts the area for correcting distortion upward when the area of the road surface area increases, and shifts the area for correcting distortion downward when the area of the road surface area decreases. , Correct the distortion by changing the distortion parameter,
The vehicle image processing apparatus according to claim 1. Vehicle speed detection unit that detects the vehicle speed of the vehicle,
Equipped with
The determination unit has an inclination angle of the vehicle based on the inclination angle of the vehicle detected by the inclination angle detection unit, the vehicle speed detected by the vehicle speed detection unit, and the road surface area detected by the road surface area detection unit. And the change with respect to the vehicle speed of the vehicle and the threshold value of the area of the road surface area are determined.
When the strain correction unit determines that the inclination angle of the vehicle is equal to or greater than the threshold value, the vehicle speed of the vehicle is equal to or less than the threshold value, and the area of the road surface region has changed by the threshold value or more, the area of the road surface region changes. The distortion is corrected by changing the distortion parameter according to the above.
The vehicle image processing apparatus according to claim 1 or 2. When the distortion correction unit changes the distortion parameter, the photometric area control unit controls the position of the photometric region when the photographing unit changes the distortion parameter.
The vehicle image processing apparatus according to any one of claims 1 to 3. An image generation unit that generates image data for displaying the surroundings of the vehicle based on the distortion correction image data corrected by the distortion correction unit.
The vehicle image processing apparatus according to any one of claims 1 to 4. The vehicle video processing apparatus according to claim 5,
At least one of the photographing unit and the display unit that displays the image data generated by the vehicle image processing device.
A video processing system for vehicles characterized by being equipped with. A video data acquisition step to acquire video data taken by the shooting unit that shoots the surroundings of the vehicle, and
A distortion correction step that corrects the distortion of the video data acquired by the video data acquisition step using a distortion parameter and outputs the distortion correction video data.
The tilt angle detection step that detects the tilt angle of the vehicle,
A road surface area detection step for detecting a road surface area captured in the video data acquired by the video data acquisition step, and a road surface area detection step.
Based on the inclination angle of the vehicle detected by the inclination angle detection step and the road surface area detected by the road surface area detection step, the change with respect to the threshold value of the inclination angle of the vehicle and the area of the road surface area is determined. Judgment step and
Including
In the strain correction step, when it is determined by the determination step that the inclination angle of the vehicle is equal to or greater than the threshold value and the area of the road surface region has changed by the threshold value or more, the change in the area of the road surface region is determined. The distortion is corrected by changing the distortion parameter.
Video processing method for vehicles. A video data acquisition step to acquire video data taken by the shooting unit that shoots the surroundings of the vehicle, and
A distortion correction step that corrects the distortion of the video data acquired by the video data acquisition step using a distortion parameter and outputs the distortion correction video data.
The tilt angle detection step that detects the tilt angle of the vehicle,
A road surface area detection step for detecting a road surface area captured in the video data acquired by the video data acquisition step, and a road surface area detection step.
Based on the inclination angle of the vehicle detected by the inclination angle detection step and the road surface area detected by the road surface area detection step, the change with respect to the threshold value of the inclination angle of the vehicle and the area of the road surface area is determined. Judgment step and
Including
In the strain correction step, when it is determined by the determination step that the inclination angle of the vehicle is equal to or greater than the threshold value and the area of the road surface region has changed by the threshold value or more, the change in the area of the road surface region is determined. The distortion is corrected by changing the distortion parameter.
A program that causes a computer that operates as a video processing device for vehicles to execute this.

Images