JP7363965B2 - Vehicle video processing device, vehicle video processing system, vehicle video processing method and program - Google Patents

Description

The present invention relates to a vehicle video processing device, a vehicle video processing system, a vehicle video processing method, and a program.

2. Description of the Related Art There is a known technique for checking the surroundings of a vehicle using video data of the surroundings of the vehicle (for example, see Patent Document 1). The technology described in Patent Document 1 displays camera images on a display at intersections with poor visibility that are registered in a car navigation system.

Japanese Patent Application Publication No. 08-293098

For example, if a road that intersects with the road on which the vehicle is traveling is curved, it is difficult for the user to see it directly, and the display area of the video data captured by the imaging unit installed in the vehicle cannot be displayed far away.

The present invention has been made in view of the above, and an object of the present invention is to appropriately display an image around a vehicle in a display area that matches the shape of a curve.

In order to solve the above-mentioned problems and achieve the purpose, a vehicle video processing device according to the present invention has the following features:
a video data acquisition unit that acquires video data captured by a photography unit that photographs the surroundings of a vehicle; and a video data acquisition unit that corrects distortion of the video data acquired by the video data acquisition unit using a distortion parameter to generate distortion-corrected video data. a distortion correction unit to output, a road shape detection unit that detects the shape of a road based on current position information of the current position of the vehicle, and map information around the vehicle; and a road shape detection unit that detects the shape of the road, a road surface area detection unit that detects a road surface area that is captured in the video data; The method is characterized in that the distortion is corrected by changing the distortion parameter.

A vehicle video processing system according to the present invention includes at least one of the vehicle video processing device described above, the photographing section, and a display section that displays video data generated by the vehicle video processing device. It is characterized by

The video processing method for a vehicle according to the present invention includes a video data acquisition step of acquiring video data photographed by a photographing unit that photographs the surroundings of a vehicle, and a distortion parameter that defines the distortion of the video data acquired in the video data acquisition step. a distortion correction step of outputting distortion corrected video data; and a road shape detecting the shape of the road based on current position information of the current position of the vehicle and map information around the vehicle. a detection step, and a road surface area detection step of detecting a road surface area captured in the video data acquired by the video data acquisition step, and the distortion correction step includes: a change in the area of the road surface area; The distortion parameter is changed to correct the distortion according to the shape of the road detected in the road shape detection step.

The program according to the present invention includes a video data acquisition step of acquiring video data photographed by a photographing unit that photographs the surroundings of a vehicle, and a correction of distortion of the video data acquired by the video data acquisition step using a distortion parameter. a distortion correction step of outputting distortion corrected video data; a road shape detection step of detecting the shape of the road based on current position information of the vehicle's current location and map information around the vehicle; a road surface area detection step of detecting a road surface area captured in the video data acquired by the video data acquisition step, and the distortion correction step includes detecting a change in the area of the road surface area and the road shape detection step. A computer operating as a vehicle video processing device is caused to correct the distortion by changing the distortion parameter according to the shape of the road detected in the step.

Advantageous Effects of Invention According to the present invention, it is possible to appropriately display an image of the surroundings of a vehicle in a display area that matches the shape of a curve.

FIG. 1 is a block diagram showing a configuration example of a vehicle video processing device according to an embodiment. FIG. 2 is a diagram illustrating a state in which a vehicle travels on a flat road. FIG. 3 is a diagram illustrating a state in which a vehicle travels on a flat road. FIG. 4 is a diagram showing an example of video data captured by a camera when a vehicle travels on a flat road. FIG. 5 is a diagram showing an example of a display area on which distortion correction processing is performed in the video data shown in FIG. 4. FIG. 6 is a diagram illustrating a state in which a vehicle travels on a road with a steep upward slope. FIG. 7 is a diagram illustrating a state in which a vehicle travels on a road with a steep upward slope. FIG. 8 is a diagram showing an example of video data captured by a camera when a vehicle travels on a road with a steep upward slope. FIG. 9 is a diagram showing an example of a display area on which distortion correction processing is performed in the video data shown in FIG. 8. FIG. 10 is a diagram illustrating a state in which a vehicle travels on a road with a steep downward slope. FIG. 11 is a diagram illustrating a state in which a vehicle travels on a road with a steep downward slope. FIG. 12 is a diagram showing an example of video data captured by a camera when a vehicle travels on a road with a steep downward slope. FIG. 13 is a diagram showing an example of a display area on which distortion correction processing is performed in the video data shown in FIG. 12. FIG. 14 is a flowchart showing the flow of processing in the vehicle video processing device according to the embodiment. FIG. 15 is a diagram illustrating an example of a display area in which distortion correction processing is performed when an intersecting road is curved. FIG. 16 is a diagram illustrating a state in which a road intersecting a road on which a vehicle is traveling is curved. FIG. 17 is a diagram illustrating an example of a display area in which distortion correction processing is performed when a vehicle travels on a snowy road.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of a vehicle video processing device, a vehicle video processing system, a vehicle video processing method, and a program according to the present invention will be described in detail below with reference to the accompanying drawings. Note that the present invention is not limited to the following embodiments.

FIG. 1 is a block diagram showing a configuration example of a vehicle video processing device according to an embodiment. The vehicle video processing system 1 displays a video for checking the surroundings of the vehicle V, for example, when driving at low speed or when parking. The vehicle video processing system 1 displays video in an appropriate display area regardless of the inclination angle of the vehicle V. The vehicle video processing system 1 includes a camera unit (photographing section) 10,
Display unit 21, headlight switch 22, vehicle speed sensor 23, and 3-axis gyro sensor 2
4 and a vehicle video processing device 30.

The camera unit 10 includes a camera that photographs the surroundings of the vehicle V. A plurality of cameras may be arranged. In this embodiment, the camera unit 10 includes a front camera 11, a right camera 12, a left camera 13, and a rear camera 14.

The front camera 11 is arranged in front of the vehicle V, and photographs the surrounding area centered on the front of the vehicle V. The front camera 11 takes an ultra-wide-angle photograph, for example, a photographing range of about 190 degrees in the horizontal direction. The front camera 11 outputs the captured video to the video processing section 34 of the vehicle video processing device 30.

The right side camera 12 is arranged on the right side of the vehicle V, and photographs the surrounding area centered on the right side of the vehicle V. The right camera 12 takes an ultra-wide-angle photograph, for example, a photographing range of approximately 190° in the horizontal direction. The right side camera 12 sends the captured image to the image processing unit 3 of the vehicle image processing device 30.
Output to 4.

The left side camera 13 is arranged on the left side of the vehicle V, and photographs the surrounding area centered on the left side of the vehicle V. The left side camera 13 takes an ultra-wide-angle photograph, for example, a photographing range of about 190° in the horizontal direction. The left side camera 13 sends the captured image to the image processing unit 3 of the vehicle image processing device 30.
Output to 4.

The rear camera 14 is arranged at the rear of the vehicle V, and photographs the surrounding area centered on the rear of the vehicle V. The rear camera 14 takes an ultra-wide-angle photograph, for example, a photographing range of approximately 190° in the horizontal direction. The rear camera 14 outputs the captured video to the video processing section 34 of the vehicle video processing device 30.

The camera unit 10 including such a front camera 11, right side camera 12, left side camera 13, and rear camera 14 photographs the vehicle V in all directions.

The display unit 21 is, for example, a liquid crystal display (LCD).
Display) or Organic EL (Organic Electro-Luminesc)
ence) display. The display unit 21 displays, for example, a video around the vehicle V based on a video signal output from the vehicle video processing device 30 of the vehicle video processing system 1. The display unit 21 may be dedicated to the vehicle video processing system 1, or may be used jointly with other systems including, for example, a navigation system. The display unit 21 is arranged at a position that is easily visible to the user. In this embodiment,
The display unit 21 is arranged on a center console at the center of the dashboard in the vehicle width direction.

The headlight switch 22 outputs a signal capable of detecting whether the headlights are on or off to the headlight on detection section 31 of the vehicle video processing device 30 .

Vehicle speed sensor 23 detects the vehicle speed of vehicle V. More specifically, the vehicle speed sensor 23 is arranged on the drive shaft or tire of the vehicle V. The vehicle speed sensor 23 detects a pulse signal corresponding to the rotation of the drive shaft or tires. Vehicle speed sensor 23 outputs the detected pulse signal to vehicle speed detection section 32 of vehicle video processing device 30 .

The three-axis gyro sensor 24 detects the slope of the road on which the vehicle V travels. The three-axis gyro sensor 24 is a sensor that detects the inclination of the vehicle V. The three-axis gyro sensor 24 detects, for example, the angular velocity of the vehicle V. More specifically, the three-axis gyro sensor 24 includes a roll rate gyro, a pitch rate gyro, and a yaw rate gyro. The roll rate gyro detects a roll angular velocity that is the angular velocity of rotation of the vehicle V around the longitudinal direction. The pitch rate gyro detects a pitch angular velocity that is the angular velocity of rotation of the vehicle V about the left-right direction. The yaw rate gyro detects the yaw angular velocity, which is the angular velocity of rotation of the vehicle V around the vertical direction. The 3-axis gyro sensor 24 uses the roll angular velocity detected by the roll rate gyro, the pitch angular velocity detected by the pitch rate gyro, and the yaw angular velocity detected by the yaw rate gyro as angular velocity signals, and sends them to the tilt angle detection section of the vehicle video processing device 30. Output to 33.

The vehicle video processing device 30 generates a video for checking the surroundings of the vehicle V, and causes the display unit 21 to display the video. The vehicle video processing device 30 loads the program stored in the storage unit 39 into memory and executes the instructions included in the program. The vehicle video processing device 30 includes a headlight lighting detection section 31, a vehicle speed detection section 32, a tilt angle detection section 33, a video processing section 34, a determination section 35, a photometry area control section 36, and a display control section. 37 and a storage section 39.

The headlight lighting detection section 31 detects, from the signal output from the headlight switch 22,
Detects whether headlights are on or off. When the headlights are on, the front image data captured by the front camera 11 shows the road surface illuminated by the headlights. The road surface illuminated by the headlights shown in the forward image data differs from the road surface not illuminated by the headlights in hue, brightness, saturation, and brightness. The headlight lighting detection section 31 outputs the detection result to the determination section 35.

The vehicle speed detection unit 32 detects the vehicle speed of the vehicle V from the pulse signal acquired from the vehicle speed sensor 23. The vehicle speed detection unit 32 outputs vehicle speed information indicating the calculated vehicle speed to the determination unit 35.

The inclination angle detection unit 33 detects the inclination angle of the road on which the vehicle V travels. In this embodiment, the inclination angle detection unit 33 acquires the angular velocity signal output from the three-axis gyro sensor 24. The tilt angle detection unit 33 calculates a roll angle, a pitch angle, and a yaw angle that indicate the tilt of the vehicle V from the roll angular velocity, pitch angular velocity, and yaw angular velocity included in the angular velocity signal. The tilt angle detection section 33 outputs the calculated roll angle, pitch angle, and yaw angle to the determination section 35 as tilt angle information. Note that the tilt angle detection section 33 may calculate only the pitch angle and output only the pitch angle to the determination section 35 as tilt angle information.

The video processing unit 34 displays the display unit 21 based on the video data captured by the camera unit 10.
Generates display video data of the video to be displayed on the screen. The video processing section 34 includes a video data acquisition section 341, a road surface area detection section 342, a distortion correction section 343, and a video generation section 344.

The video data acquisition unit 341 acquires video data captured by the camera unit 10. The video data acquisition unit 341 receives front video data captured by the front camera 11 and the right side camera 1.
2, left side image data taken by the left side camera 13, and rearward image data taken by the rear camera 14 are acquired.

The road surface area detection unit 342 detects the road surface area captured in the video data acquired by the video data acquisition unit 341. The road surface area is the area of the road surface shown in the video data. The road surface area is defined by the shape of the road, the width of the road, and the depth of the road. In this embodiment, the road surface area detection unit 342 will be described as one that detects a road surface area shown in the forward image data. The road surface area detection unit 342 stores a road surface area detected when the vehicle V is traveling on a flat road as a reference road surface area. It is preferable that the reference road surface area is stored for each road width of the road on which the vehicle V travels. The method of detecting the road surface is not limited and may be any known method. If the detection result of the headlight lighting detection unit 31 indicates that the headlights are on, the road surface area detection unit 342 detects the hue of the video data because the road surface depicted in the video data is illuminated by the headlights. The road surface area of the road surface is detected by correcting the brightness, saturation, and brightness.

The distortion correction unit 343 uses distortion parameters to correct distortion in a display area that is a part of the ultra-wide-angle video data acquired by the video data acquisition unit 341, and outputs rectangular distortion-corrected video data. . The distortion-corrected video data includes forward distortion-corrected video data that corrects front video data, right-side distortion-corrected video data that corrects right-side video data, left-side distortion-corrected video data that corrects left-side video data, and rearward distortion-corrected video data that corrects left-side video data. This includes backward distortion corrected video data obtained by correcting video data. If there is no need to distinguish between forward distortion corrected video data, right side distortion corrected video data, left side distortion corrected video data, and backward distortion corrected video data, they will be described as distortion corrected video data.

When the determination unit 35 determines that the inclination angle of the vehicle V is greater than or equal to the threshold value, the vehicle speed of the vehicle V is less than or equal to the threshold value, and the area of the road surface area has changed by more than the threshold value, the distortion correction unit 343 adjusts the area of the road surface area. According to the change in area, the distortion parameters are changed to correct the distortion, and distortion-corrected video data is output. More specifically, when the area of the road surface area increases compared to the reference road surface area, the distortion correction unit 343 corrects the distortion by changing the distortion parameter so as to shift the display area for correcting the distortion upward. and outputs distortion-corrected video data. When the area of the road surface area decreases compared to the reference road surface area, the distortion correction unit 343 corrects the distortion by changing the distortion parameter so as to shift the display area for correcting the distortion downward. Output corrected video data.

Processing performed by the distortion correction unit 343 when the vehicle V is traveling on a flat road will be described with reference to FIGS. 2 to 5. FIG. 2 is a diagram illustrating a state in which a vehicle travels on a flat road. FIG. 3 is a diagram illustrating a state in which a vehicle travels on a flat road. FIG. 4 is a diagram showing an example of video data captured by a camera when a vehicle travels on a flat road. Figure 5 shows
5 is a diagram showing an example of a display area on which distortion correction processing is performed among the video data shown in FIG. 4. FIG. Figure 2,
As shown in FIG. 3, when the vehicle V is traveling on a flat road, the forward image data captured by the front camera 11 is as shown in FIG. 4, for example. Such forward image data is used as reference image data, and a display area indicating an area that the user can view as an image and a distortion parameter are set. Furthermore, the road surface area detected from this forward image data is the reference road surface area. For example, the display area is set to A1, which is a barrel-shaped area shown in the figure. Distortion parameters are set so that distortion is corrected for such display area A1 and rectangular forward distortion corrected video data is generated.

Here, the camera unit 10 including the front camera 11 captures an image with an ultra-wide viewing angle. For purposes of checking a wide range, such as when generating an image for checking the surroundings of the vehicle V, it is desirable to correct distortion so that a full viewing angle can be ensured in the horizontal direction. In such a case,
If the distortion is corrected while maintaining a natural sense of distance to the front, the display area in the vertical direction, as opposed to the horizontal direction, will look like a portion of the camera's shooting range. On the other hand, if the vertical display area is maintained as is and the distortion is corrected, the resulting image will look unnatural, giving the impression that the sense of distance in front is further away than it actually is. Therefore, depending on the inclination angle of the vehicle V, it is necessary to adjust the display area in the vertical direction. Specifically, the distortion parameter is changed so as to shift the display area for correcting distortion upward or downward from the center of the photographing range of the camera according to the increase/decrease ratio of the road surface area.

Processing performed by the distortion correction unit 343 when the vehicle V is traveling on a road with a steep upward slope will be described with reference to FIGS. 6 to 9. FIG. 6 is a diagram illustrating a state in which a vehicle travels on a road with a steep upward slope. FIG. 7 is a diagram illustrating a state in which a vehicle travels on a road with a steep upward slope. FIG. 8 shows that when a vehicle travels on a road with a steep upward slope,
FIG. 3 is a diagram showing an example of video data captured by a camera. FIG. 9 is a diagram showing an example of a display area on which distortion correction processing is performed in the video data shown in FIG. 8. As shown in FIGS. 6 and 7, the vehicle V
When the vehicle is traveling on a road with a steep upward slope, the forward image data captured by the front camera 11 is as shown in FIG. 8, for example. Here, it is assumed that there is an intersection that intersects with the cross road R1 on the way up a steep slope. Another vehicle V1 is traveling on the intersecting road R1.
When the vehicle V is tilted forward, the photographing range of the front camera 11 is tilted upward. This results in
The road surface area in the display area of the forward video data is reduced. Therefore, the distortion parameters are changed so that the position of the road surface area in the display area is appropriate, and the distortion parameters are changed so that the barrel-shaped area A2 shown in the figure becomes the display area. In this way, distortion is corrected for the display area A2 and forward distortion corrected video data is output. Note that a display area A1 indicated by a broken line in the figure is a display area when the distortion parameter is not changed. The same applies to the following figures.

Processing performed by the distortion correction unit 343 when the vehicle V is traveling on a road with a steep downward slope will be described with reference to FIGS. 10 to 13. FIG. 10 is a diagram illustrating a state in which a vehicle travels on a road with a steep downward slope. FIG. 11 is a diagram illustrating a state in which a vehicle travels on a road with a steep downward slope. FIG. 12 is a diagram showing an example of video data captured by a camera when a vehicle travels on a road with a steep downward slope. FIG. 13 is a diagram showing an example of a display area on which distortion correction processing is performed in the video data shown in FIG. 12. As shown in FIGS. 10 and 11, when the vehicle V is traveling on a road with a steep downward slope, the forward image data captured by the front camera 11 is, for example, as shown in FIG. 12. Here, it is assumed that there is an intersection that intersects with the cross road R1 in the middle of a steep downward slope. Another vehicle V1 is traveling on the intersecting road R1. When the vehicle V is tilted forward, the photographing range of the front camera 11 is directed downward. This increases the road surface area in the display area of the forward image data. Therefore, the distortion parameters are changed so that the position of the road surface area in the display area is appropriate, and the distortion parameters are changed so that the barrel-shaped area A3 shown in the figure becomes the display area. in this way,
Distortion is corrected for the display area A3 and forward distortion corrected video data is output.

The video generation unit 344 generates the vehicle V based on the distortion corrected video data output by the distortion correction unit 343.
Generates display video data that confirms the surrounding area. In the present embodiment, the video generation unit 344 generates display video data that allows confirmation of the front of the vehicle V based on the forward distortion corrected video data.

The video generation unit 344 generates the vehicle V based on the distortion corrected video data output by the distortion correction unit 343.
Display video data of an overhead video including a virtual self-vehicle image looking down from above may be generated. In this case, the video generation unit 344 performs viewpoint conversion processing and synthesizes the host vehicle icon indicating the vehicle V with the forward distortion corrected video data, the right side distortion corrected video data, the left side distortion corrected video data, and the backward distortion corrected video data. A synthesis process is performed to generate display video data of an overhead video that displays the surroundings of the vehicle V. The method of generating the bird's-eye view image may be any known method and is not limited.

The determination unit 35 uses the inclination angle of the vehicle V detected by the inclination angle detection unit 33 and the road surface area detection unit 342.
Based on the detected road surface area, a change in the inclination angle of the vehicle V and the area of the road surface area with respect to the threshold value is determined. More specifically, the determination unit 35 determines whether the vehicle It is determined whether the inclination angle of vehicle V is greater than or equal to a threshold value, the vehicle speed of vehicle V is less than or equal to a threshold value, and the area of the road surface region has changed by more than the threshold value. The threshold value of the inclination angle is, for example, about 5°. The vehicle speed threshold is, for example, about 10 km/h. The threshold value of the area of the road surface area is, for example,
It is about 20%.

Whether the area of the road surface area has changed by more than the threshold value is determined if the area of the road surface area detected by the road surface area detection unit 342 is approximately 20% or more compared to the area of the reference road surface area of a road having the same road width. , the determination is made based on whether it has increased or decreased. When determining that the area of the road surface area detected by the road surface area detection unit 342 has increased or decreased by about 20% or more compared to the area of the reference road surface area, it is determined that the area of the road surface area has changed by a threshold value or more. When determining that the area of the road surface area detected by the road surface area detection unit 342 has not increased or decreased by about 20% or more compared to the area of the reference road surface area, it is determined that the area of the road surface area has not changed by more than a threshold value. judge.

Alternatively, whether the area of the road surface area has changed by more than a threshold value is determined by whether the area of the road surface area detected by the road surface area detection unit 342 has increased or decreased by about 20% or more over a predetermined period or a predetermined travel distance. You may judge. If it is determined that the area of the road surface area detected by the road surface area detection unit 342 has increased or decreased by about 20% or more over a predetermined period or a predetermined travel distance, it is determined that the area of the road surface area has changed by a threshold value or more. If it is determined that the area of the road surface area detected by the road surface area detection unit 342 has not increased or decreased by about 20% or more over a predetermined period or a predetermined travel distance, it is determined that the area of the road surface area has not changed by a threshold value or more. .

When the distortion correction unit 343 changes the distortion parameter, the photometry area control unit 36 controls the position of the photometry area when the camera unit 10 takes an image in accordance with the change in the distortion parameter. For example, the position of the photometry area when the camera unit 10 takes pictures in normal times is the center of the display area. When the distortion correction unit 343 shifts the display area in which distortion is corrected upward, the photometry area control unit 36 shifts the photometry area upward so that the center of the display area shifted upward becomes the photometry area. A control signal is output to the camera unit 10. When the distortion correction unit 343 shifts the display area in which distortion is corrected downward, the photometry area control unit 36 shifts the photometry area downward so that the center of the display area shifted downward becomes the photometry area. A control signal for shifting the camera unit 10 is output to the camera unit 10.

The display control unit 37 controls displaying the display video data generated by the video generation unit 344 of the video processing unit 34 on the display unit 21.

The storage unit 39 stores data required for various processes in the vehicle video processing device 30 and various process results. The storage unit 39 is, for example, a RAM (Random Access Me
memory), ROM (Read Only Memory), flash memory (Fla.
Semiconductor memory elements such as (sh Memory), hard disks, optical disks,
This is a storage device such as an external storage device via a communication network.

Next, the flow of processing in the vehicle video processing device 30 will be described using FIG. 14. FIG. 14 is a flowchart showing the flow of processing in the vehicle video processing device 30 according to the embodiment. Here, based on the front image data photographed by the front camera 11, the vehicle V
A case will be explained in which an image is displayed to confirm the safety in front of the vehicle.

It is assumed that the vehicle video processing system 1 is started when the engine of the vehicle V is started. While the vehicle video processing system 1 is activated, the camera unit 10 photographs the surroundings of the vehicle V. During startup of the vehicle video processing system 1, the video data acquisition section 341 acquires the camera unit 10.
Obtain video data shot with.

The vehicle video processing device 30 detects the lighting of the headlights (step S101). The vehicle video processing device 30 detects whether the headlights are on or off based on the signal output from the headlight switch 22 by the headlight on detection section 31 . The vehicle video processing device 30 proceeds to step S102.

The vehicle video processing device 30 detects a road surface area (step S102). The vehicle image processing device 30 uses a road surface area detection unit 342 to detect a road surface area shown in the forward image data using a known road surface detection method. The vehicle video processing device 30 performs step S
Proceed to step 103.

The vehicle video processing device 30 uses the determination unit 35 to determine whether the vehicle speed is equal to or less than a threshold (step S103). When the determination unit 35 determines that the vehicle speed detected by the vehicle speed detection unit 32 is equal to or lower than the threshold (Yes in step S103), the vehicle video processing device 30 performs step S
Proceed to step 104. When the determining unit 35 determines that the vehicle speed detected by the vehicle speed detecting unit 32 is not equal to or lower than the threshold value (No in step S103), the vehicle video processing device 30 executes the process of step S101 again.

If the vehicle speed detected by the vehicle speed detection unit 32 determines that the vehicle speed is equal to or less than the threshold value (Yes in step S103), the vehicle video processing device 30 causes the determination unit 35 to determine whether the pitch angle is equal to or greater than the threshold value (step S104). In the vehicle video processing device 30, when the determination unit 35 determines that the pitch angle detected by the inclination angle detection unit 33 is greater than or equal to the threshold value (Y in step S104),
es), the process advances to step S105. In the vehicle video processing device 30, when the determination unit 35 determines that the pitch angle detected by the inclination angle detection unit 33 is not equal to or greater than the threshold value (N in step S104),
o), proceed to step S111.

When determining that the pitch angle detected by the inclination angle detection unit 33 is equal to or greater than the threshold (step S1
04), the vehicle video processing device 30 determines that the road surface area detection unit 34
Based on the road surface area detected by step S2, it is determined whether the area of the road surface area has increased or decreased (step S
105). When the determination unit 35 determines that the area of the road surface area has increased or decreased (Yes in step S105), the vehicle video processing device 30 proceeds to step S106. In the vehicle video processing device 30, when the determination unit 35 determines that the area of the road surface area has not increased or decreased (step S1
05), the process of step S101 is executed again.

When determining that the area of the road surface area has increased or decreased (Yes in step S105), the vehicle video processing device 30 performs a correction process on the display area (step S106). The vehicular video processing device 30 uses the distortion correction unit 343 to perform display area correction processing that corrects distortion by changing distortion parameters in accordance with changes in the area of the road surface area. The vehicle video processing device 30 includes a distortion correction section 3
43, when the area of the road surface area increases, a display area correction process for correcting distortion is performed by changing the distortion parameter so as to shift the display area for correcting distortion upward. The vehicular video processing device 30 uses the distortion correction unit 343 to adjust the display area for correcting the distortion by changing the distortion parameter so that when the area of the road surface area decreases, the display area for correcting the distortion is shifted downward. Performs correction processing. The vehicle video processing device 30 proceeds to step S107.

The vehicle video processing device 30 generates display video data (step S107). The vehicle video processing device 30 uses the video generation unit 344 to generate display video data from the forward distortion corrected video data. The vehicle video processing device 30 proceeds to step S108.

The vehicle video processing device 30 causes the display control unit 37 to display the generated display video data on the display unit 21 (step S108). The vehicle video processing device 30 performs step S10.
Proceed to step 9.

The vehicle video processing device 30 uses the determination unit 35 to determine whether the pitch angle is less than a threshold (step S109). In the vehicle video processing device 30, the determination unit 35
If it is determined that the pitch angle detected by No. 3 is less than the threshold value (Yes in step S109),
The process advances to step S110. In the vehicle video processing device 30, the determination unit 35 detects the tilt angle detection unit 33.
If it is determined that the detected pitch angle is not less than the threshold (No in step S109), the process of step S109 is executed again.

When determining that the pitch angle detected by the inclination angle detection unit 33 is less than the threshold value (step S1
09: Yes), the vehicle video processing device 30 returns the display area to the initial setting (step S1
10). In the vehicle video processing device 30, the distortion correction unit 343 returns the distortion parameters changed in step S106 to the initial values. The vehicular video processing device 30 uses the distortion correction unit 343 to return the distortion parameters to the distortion parameters when the vehicle V is traveling on a flat road. The vehicle video processing device 30 proceeds to step S113.

When it is determined that the pitch angle detected by the inclination angle detection unit 33 is not equal to or greater than the threshold (step S
104), the vehicle video processing device 30 generates display video data (step S1
11). The vehicular video processing device 30 generates display video data from the video data using the video generation unit 344. The vehicle video processing device 30 proceeds to step S112.

The vehicle video processing device 30 causes the display control unit 37 to display the display video data on the display unit 21 (step S112). The vehicle video processing device 30 proceeds to step S113.

The vehicle video processing device 30 uses the determination unit 35 to determine whether the vehicle speed is greater than a threshold value (step S113). When the determination unit 35 determines that the vehicle speed detected by the vehicle speed detection unit 32 is greater than the threshold (Yes in step S113), the vehicle video processing device 30 performs step S
Proceed to 114. When the determination unit 35 determines that the vehicle speed detected by the vehicle speed detection unit 32 is not greater than the threshold (No in step S113), the vehicle video processing device 30 performs step S113.
Execute the process again.

When determining that the vehicle speed detected by the vehicle speed detection unit 32 is greater than the threshold value (Yes in step S113), the vehicle video processing device 30 stops displaying the display video data (step S1
14). The vehicle video processing device 30 proceeds to step S115.

The vehicle video processing device 30 determines whether to end the process (step S115). More specifically, the vehicle video processing device 30 determines whether or not to end the process based on the presence or absence of the end trigger. The end trigger is, for example, a combination of turning off the engine, putting the gear in parking, or operating the handbrake. The termination trigger may be activated by a user's operation. If there is an end trigger, the vehicle video processing device 30 determines to end the process (Yes in step S115), and ends the process. If there is no end trigger, the vehicle video processing device 30 determines not to end the process (step S11).
5), the process of step S101 is executed again.

In this way, when the determination unit 35 determines that the inclination angle of the vehicle V is greater than or equal to the threshold value, the vehicle speed of the vehicle V is less than or equal to the threshold value, and the area of the road surface area has changed by more than the threshold value, the distortion correction unit 343 corrects distortion by changing distortion parameters according to changes in the area of the road surface area. Distortion correction section 3
43 corrects the distortion by changing the distortion parameter so that when the area of the road surface area increases, the display area where the distortion is corrected is shifted upward. When the area of the road surface area decreases, the distortion correction unit 343 corrects the distortion by changing the distortion parameter so as to shift the display area in which the distortion is corrected downward.

As described above, in this embodiment, when it is determined that the inclination angle of the vehicle V is greater than or equal to the threshold value, the vehicle speed of the vehicle V is less than or equal to the threshold value, and the area of the road surface region has changed by more than the threshold value, the area of the road surface region is The distortion can be corrected by changing the distortion parameter according to the change in the distortion parameter. In this embodiment, when the area of the road surface area increases, the distortion can be corrected by changing the distortion parameter so that the display area for correcting the distortion is shifted upward. When the area of the road surface area decreases, the distortion can be corrected by changing the distortion parameter so that the display area for correcting the distortion is shifted downward. In this way, according to the present embodiment, an image can be displayed in an appropriate display area regardless of the inclination angle of the vehicle V.

In this embodiment, when changing the distortion parameter, the position of the photometry area when the camera unit 10 takes an image can be controlled in accordance with the change in the distortion parameter. According to this embodiment, regardless of the inclination angle of the vehicle V, it is possible to set an appropriate photometry area and perform appropriate photometry to take an image.

In this way, according to the present embodiment, the surroundings of the vehicle V can be confirmed with an appropriate image regardless of the inclination angle of the vehicle V.

Now, although the vehicle video processing system 1 according to the present invention has been described so far, it may be implemented in various different forms other than the embodiments described above.

Each component of the illustrated vehicle video processing system 1 is functionally conceptual, and does not necessarily have to be physically configured as illustrated. In other words, the specific form of each device is not limited to what is shown in the diagram, and all or part of it may be functionally or physically distributed or integrated into arbitrary units depending on the processing load and usage status of each device. It's okay.

The configuration of the vehicle video processing system 1 is realized by, for example, a program loaded into a memory as software. The above embodiments have been described as functional blocks realized by cooperation of these hardware or software. That is, these functional blocks can be realized in various forms using only hardware, only software, or a combination thereof.

The above-mentioned components include those that can be easily imagined by those skilled in the art and that are substantially the same.
Furthermore, the above configurations can be combined as appropriate. Furthermore, various omissions, substitutions, or changes in the configuration are possible without departing from the gist of the present invention.

A case where the road on which the vehicle V is traveling and the intersecting road are curved will be described with reference to FIGS. 15 and 16. FIG. 15 is a diagram illustrating an example of a display area in which distortion correction processing is performed when an intersecting road is curved. FIG. 16 is a diagram illustrating a state in which a road intersecting a road on which a vehicle is traveling is curved. When the road on which the vehicle V is traveling and the intersecting road are curved, the distortion correction unit 343 may correct the distortion by changing the distortion parameters according to the shape of the curve. The vehicle video processing device 30 is a vehicle
The vehicle V has a navigation information acquisition unit that acquires current position information of the current position of the vehicle V and map information around the vehicle V. Further, the vehicle video processing device 30 includes a road shape detection section that detects the shape of the road based on the information acquired by the navigation information acquisition section. The distortion correction unit 343 corrects the distortion by changing the distortion parameter according to the change in the area of the road surface area and the shape of the road detected by the road shape detection unit. The distortion correction unit 343 shifts the right or left side of the display area for which distortion is to be corrected upward or downward so that the display area on the curved side can be displayed far away in accordance with the curved shape of the intersecting road. The distortion parameters are changed so that the area A4 having the shape shown in the figure becomes the display area. Thereby, when the intersecting road is curved, it is possible to display an image of the surroundings of the vehicle V in a display area that matches the shape of the curve, which is difficult for the user to see directly.

A case where the vehicle V is traveling on a snowy road or the like will be described using FIG. 17. Figure 17 shows
FIG. 6 is a diagram illustrating an example of a display area in which distortion correction processing is performed when a vehicle travels on a snowy road. for example,
If the road surface area detection unit 342 is unable to detect the road surface due to snow accumulation or flooding, the distortion parameter is changed to display a wider range, and the display area is A5, which is the area with the shape shown in the figure.
Change the distortion parameters so that is the display area. Thereby, when it is difficult to detect the road surface area due to snow accumulation or flooding, it is possible to display an image around the vehicle V in a display area in which the road surface area is wider than usual.

A case will be described in which an image for checking the surroundings of the vehicle V is displayed when parking the vehicle. In this case, when it is determined that there is a reverse trigger, the vehicle video processing device 30 executes the process of the flowchart shown in FIG. 14 . The reverse trigger means, for example, that the shift position has been set to "reverse" or that the traveling direction of the vehicle V has become rearward in the longitudinal direction of the vehicle V. Then, in step S102, a road surface area shown in the rear video data is detected,
In step S105, it is determined whether the area of the road surface area of the rear video data has increased or decreased.
Then, in step S106, the distortion parameters are changed to correct the distortion of the rear video data. In this way, when parking, an image behind the vehicle V can be displayed in an appropriate display area regardless of the inclination angle of the vehicle V.

The vehicle video processing system 1 can also be applied to a so-called drive recorder that photographs and records surrounding images when the vehicle V is running. In this case, regardless of the vehicle speed of the vehicle V, the determination unit 3
5 is based on the inclination angle of the vehicle V detected by the inclination angle detection unit 33 and the road surface area detected by the road surface area detection unit 342, and the inclination angle of the vehicle V is equal to or greater than the threshold value, and the area of the road surface area is the threshold value. that's all,
Determine whether there has been a change. Further, when the determination unit 35 determines that the inclination angle of the vehicle V is equal to or greater than the threshold value and the area of the road surface region has changed by equal to or more than the threshold value, the distortion correction unit 343 adjusts the distortion according to the change in the area of the road surface region. Correct the distortion by changing the parameters. The vehicle video processing device 30 is
Processing other than step S103, step S113, and step S114 in the flowchart shown in FIG. 14 is executed. Furthermore, when applied to a drive recorder, the storage unit 39
Separately, an external recording device including, for example, a card-type recording medium may be provided outside the vehicle video processing device 30 for storing video data.

The vehicle video processing device 30 may include an optical axis control unit that controls the position of the optical axis of the headlight in accordance with the change in the distortion parameter when the distortion correction unit 343 changes the distortion parameter. Thereby, an appropriate image of the surroundings of the vehicle V can be displayed.

Instead of the 3-axis gyro sensor 24 described above, a 6-axis sensor having the functions of a 3-axis acceleration sensor and a 3-axis gyro sensor capable of detecting acceleration in the X direction, Y direction, and Z direction may be used. .

1 Vehicle video processing system 10 Camera unit 21 Display section 22 Headlight switch 23 Vehicle speed sensor 24 3-axis gyro sensor 30 Vehicle video processing device 31 Headlight lighting detection section 32 Vehicle speed detection section 33 Tilt angle detection section 34 Video processing section 341 Video data acquisition unit 342 Road surface area detection unit 343 Distortion correction unit 344 Image generation unit 35 Determination unit 36 Photometry area control unit 37 Display control unit 39 Storage unit

Claims (5)

a video data acquisition unit that acquires video data captured by a photography unit that photographs the surroundings of the vehicle;
a distortion correction unit that corrects distortion of the video data acquired by the video data acquisition unit using a distortion parameter and outputs distortion-corrected video data;
a road shape detection unit that detects the shape of a road based on current location information of the current location of the vehicle and map information around the vehicle;
a road surface area detection unit that detects a road surface area captured in the video data acquired by the video data acquisition unit;
Equipped with
The distortion correction unit adjusts an area for correcting distortion of the display surface area on the curved side to the upper side or the lower side according to the change in the area of the road surface area and the curved shape of the intersecting road detected by the road shape detection unit. correcting the distortion by changing the distortion parameter so as to shift it downward ;
A vehicle video processing device characterized by: a video generation unit that generates video data displaying the surroundings of the vehicle based on the distortion corrected video data corrected by the distortion correction unit;
The vehicle video processing device according to claim 1 , comprising: A vehicle video processing device according to claim 2 ;
At least one of the photographing unit and a display unit that displays video data generated by the vehicle video processing device,
A vehicle video processing system comprising: a video data acquisition step of acquiring video data shot by a photography unit that photographs the surroundings of the vehicle;
a distortion correction step of correcting the distortion of the video data acquired in the video data acquisition step using a distortion parameter, and outputting distortion-corrected video data;
a road shape detection step of detecting the shape of a road based on current location information of the current location of the vehicle and map information around the vehicle;
a road surface area detection step of detecting a road surface area captured in the video data acquired in the video data acquisition step;
including;
In the distortion correction step, an area for correcting distortion of the display surface area on the curved side is adjusted to the upper or lower side according to the change in the area of the road surface area and the curved shape of the intersecting road detected in the road shape detection step . correcting the distortion by changing the distortion parameter so as to shift it downward ;
Video processing method for vehicles. a video data acquisition step of acquiring video data shot by a photography unit that photographs the surroundings of the vehicle;
a distortion correction step of correcting the distortion of the video data acquired in the video data acquisition step using a distortion parameter, and outputting distortion-corrected video data;
a road shape detection step of detecting the shape of a road based on current location information of the current location of the vehicle and map information around the vehicle;
a road surface area detection step of detecting a road surface area captured in the video data acquired in the video data acquisition step;
including;
In the distortion correction step, an area for correcting distortion of the display surface area on the curved side is adjusted to the upper or lower side according to the change in the area of the road surface area and the curved shape of the intersecting road detected in the road shape detection step . correcting the distortion by changing the distortion parameter so as to shift it downward ;
A program that causes a computer that operates as a vehicle video processing device to execute this.

Images