JP2022126659A - Vehicle image processing device, vehicle image processing system, vehicle image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To appropriately display an image around a vehicle in a display area matched with a curve shape.

SOLUTION: A vehicle image processing device includes: a video data acquisition unit 341 that acquires video data captured by a camera unit 10; a distortion correction unit 343 that corrects distortion of the video data using a distortion parameter and outputs distortion-corrected video data; a road shape detection unit that detects a shape of a road on the basis of current position information of a current position of a vehicle and map information around the vehicle; and a road surface area detection unit 342 that detects a road surface area captured in the video data. The distortion correction unit 343 corrects the distortion by changing the distortion parameter according to change in acreage of the road surface area and the shape of the road detected by the road shape detection unit.

SELECTED DRAWING: Figure 15

COPYRIGHT: (C)2022,JPO&INPIT

Description

The present invention relates to a vehicle image processing device, a vehicle image processing system, a vehicle image processing method, and a program.

2. Description of the Related Art There is known a technique for checking the surroundings of a vehicle using image data obtained by photographing the surroundings of the vehicle (see, for example, Patent Document 1). The technology described in Patent Literature 1 causes a camera image to be displayed on a display at an intersection with poor visibility registered in a car navigation system.

JP-A-08-293098

For example, when the road on which the vehicle is traveling and the road that intersects is curved, it is difficult for the user to directly visually recognize it, and the display area of the video data captured by the imaging unit installed in the vehicle cannot be displayed far.

SUMMARY OF THE INVENTION It is an object of the present invention to appropriately display an image of the surroundings of a vehicle in a display area adapted to the shape of a curve.

In order to solve the above-described problems and achieve the object, the vehicle image processing device according to the present invention includes:
A video data acquisition unit that acquires video data captured by a shooting unit that captures the surroundings of a vehicle; and a distortion of the video data acquired by the video data acquisition unit is corrected using a distortion parameter to generate distortion-corrected video data. Based on the output distortion correction unit, the current position information of the current position of the vehicle, and the map information around the vehicle, the road shape detection unit for detecting the shape of the road, and the image data acquisition unit acquired a road surface area detection unit that detects the road surface area captured in the video data, and the distortion correction unit detects the road surface area according to the change in the area of the road surface area and the shape of the road detected by the road shape detection unit. and correcting the distortion by changing the distortion parameter.

A vehicle image processing system according to the present invention includes at least one of the above vehicle image processing device, the photographing unit, and a display unit that displays image data generated by the vehicle image processing device. It is characterized by

A video image processing method for a vehicle according to the present invention includes a video data obtaining step of obtaining video data captured by a capturing unit that captures the surroundings of a vehicle; and a distortion correction step of outputting distortion-corrected video data; and a road shape of detecting a road shape based on current position information of the current position of the vehicle and map information around the vehicle. a detection step; and a road surface area detection step of detecting a road surface area shown in the image data acquired by the image data acquisition step, wherein the distortion correction step includes a change in the area of the road surface area, According to the shape of the road detected by the road shape detection step, the distortion parameter is changed to correct the distortion.

A program according to the present invention includes a video data obtaining step of obtaining video data captured by a capturing unit that captures an image around a vehicle, and correcting distortion of the video data obtained by the video data obtaining step using a distortion parameter. a distortion correction step of outputting distortion-corrected image data; a road shape detection step of detecting a shape of a road based on current position information of the current position of the vehicle and map information around the vehicle; a road surface area detection step of detecting a road surface area captured in the image data acquired by the image data acquisition step, and According to the shape of the road detected in the step, the distortion parameter is changed to correct the distortion.

ADVANTAGE OF THE INVENTION According to this invention, it is effective in the ability to appropriately display the image|video of the circumference|surroundings of the vehicle of the display area according to the curve shape.

FIG. 1 is a block diagram showing a configuration example of a vehicle image processing device according to an embodiment. FIG. 2 is a diagram illustrating a state in which a vehicle travels on a flat road. FIG. 3 is a diagram illustrating a state in which the vehicle travels on a flat road. FIG. 4 is a diagram showing an example of image data captured by a camera when a vehicle travels on a flat road. FIG. 5 is a diagram showing an example of a display area on which distortion correction processing is performed in the image data shown in FIG. FIG. 6 is a diagram illustrating a state in which a vehicle travels on a road having a steep uphill gradient. FIG. 7 is a diagram illustrating a state in which a vehicle travels on a road having a steep uphill gradient. FIG. 8 is a diagram showing an example of image data captured by a camera when a vehicle travels on a road having a steep uphill gradient. FIG. 9 is a diagram showing an example of a display area on which distortion correction processing is performed in the image data shown in FIG. FIG. 10 is a diagram illustrating a state in which a vehicle travels on a road having a steep downhill gradient. FIG. 11 is a diagram illustrating a state in which a vehicle travels on a road having a steep downhill gradient. FIG. 12 is a diagram showing an example of image data captured by a camera when a vehicle travels on a road with a steep downhill gradient. FIG. 13 is a diagram showing an example of a display area on which distortion correction processing is performed in the video data shown in FIG. 12. In FIG. FIG. 14 is a flowchart showing the flow of processing in the vehicle image processing device according to the embodiment. FIG. 15 is a diagram showing an example of a display area in which distortion correction processing is performed when an intersecting road is curved. FIG. 16 is a diagram illustrating a state in which a road that intersects a road on which a vehicle is traveling is curved. FIG. 17 is a diagram showing an example of a display area in which distortion correction processing is performed when the vehicle travels on a snowy road.

Exemplary embodiments of a vehicle image processing device, a vehicle image processing system, a vehicle image processing method, and a program according to the present invention will be described below in detail with reference to the accompanying drawings. In addition, the present invention is not limited by the following embodiments.

FIG. 1 is a block diagram showing a configuration example of a vehicle image processing device according to an embodiment. The vehicle image processing system 1 displays an image for confirming the surroundings of the vehicle V, for example, when the vehicle is traveling at a low speed or when the vehicle is parked. The vehicle image processing system 1 displays an image in an appropriate display area regardless of the tilt angle of the vehicle V. FIG. The vehicle image processing system 1 includes a camera unit (image capturing unit) 10,
A display unit 21, a headlight switch 22, a vehicle speed sensor 23, and a three-axis gyro sensor 2
4 and a vehicle image processing device 30 .

The camera unit 10 has a camera that photographs the surroundings of the vehicle V. As shown in FIG. A plurality of cameras may be arranged. In this embodiment, the camera unit 10 has a front camera 11 , a right side camera 12 , a left side camera 13 and a rear camera 14 .

The front camera 11 is arranged in front of the vehicle V and captures an image of the surroundings centered on the front of the vehicle V. As shown in FIG. The front camera 11 takes a super-wide angle, for example, a shooting range of about 190° in the horizontal direction. The front camera 11 outputs the captured image to the image processing unit 34 of the vehicle image processing device 30 .

The right side camera 12 is arranged on the right side of the vehicle V and photographs the surroundings centered on the right side of the vehicle V. - 特許庁The right side camera 12 takes a super-wide angle, for example, a shooting range of about 190° in the horizontal direction. The right side camera 12 transmits the captured image to the image processing unit 3 of the vehicle image processing device 30.
Output to 4.

The left side camera 13 is arranged on the left side of the vehicle V, and photographs the surroundings with the left side of the vehicle V as the center. The left side camera 13 takes a super-wide angle, for example, a shooting range of about 190° in the horizontal direction. The left side camera 13 transmits the captured image to the image processing unit 3 of the vehicle image processing device 30.
Output to 4.

The rear camera 14 is arranged at the rear of the vehicle V and photographs the surroundings centered on the rear of the vehicle V. - 特許庁The rear camera 14 takes a super-wide angle, for example, a shooting range of about 190° in the horizontal direction. The rear camera 14 outputs the captured image to the image processing unit 34 of the vehicle image processing device 30 .

The camera unit 10 including the front camera 11, the right side camera 12, the left side camera 13, and the rear camera 14 is used to photograph the vehicle V in all directions.

The display unit 21 is, for example, a liquid crystal display (LCD).
Display) or organic EL (Organic Electro-Luminesc
ence) display that contains the display. The display unit 21 displays an image around the vehicle V, for example, based on the image signal output from the vehicle image processing device 30 of the vehicle image processing system 1 . The display unit 21 may be dedicated to the vehicle image processing system 1, or may be used in collaboration with other systems including, for example, a navigation system. The display unit 21 is arranged at a position where it can be easily viewed by the user. In this embodiment,
The display unit 21 is arranged on the center console at the center of the dashboard in the vehicle width direction.

The headlight switch 22 outputs a signal capable of detecting whether the headlight is on or off to the headlight lighting detection unit 31 of the vehicle video processing device 30 .

The vehicle speed sensor 23 detects the vehicle speed of the vehicle V. FIG. More specifically, the vehicle speed sensor 23 is arranged on the drive shaft of the vehicle V or on the tires. A vehicle speed sensor 23 detects a pulse signal corresponding to the rotation of the drive shaft or tires. The vehicle speed sensor 23 outputs the detected pulse signal to the vehicle speed detector 32 of the vehicle video processing device 30 .

The 3-axis gyro sensor 24 detects the gradient of the road on which the vehicle V travels. The 3-axis gyro sensor 24 is a sensor that detects the inclination of the vehicle V. As shown in FIG. The 3-axis gyro sensor 24 detects the angular velocity of the vehicle V, for example. More specifically, the 3-axis gyro sensor 24 has a roll rate gyro, a pitch rate gyro, and a yaw rate gyro. The roll rate gyro detects the roll angular velocity, which is the angular velocity of rotation of the vehicle V about the front-rear direction. The pitch rate gyro detects the pitch angular velocity, which is the angular velocity of rotation of the vehicle V about the left-right direction. The yaw rate gyro detects the yaw angular velocity, which is the angular velocity of rotation of the vehicle V about the vertical direction. The 3-axis gyro sensor 24 uses the roll angular velocity detected by the roll rate gyro, the pitch angular velocity detected by the pitch rate gyro, and the yaw angular velocity detected by the yaw rate gyro as angular velocity signals, and outputs the tilt angle detection unit of the vehicle image processing device 30. 33.

The vehicle image processing device 30 generates an image for confirming the surroundings of the vehicle V and causes the display unit 21 to display the image. The vehicle image processing device 30 loads the program stored in the storage unit 39 into the memory and executes the instructions included in the program. The vehicle image processing device 30 includes a headlight lighting detection unit 31, a vehicle speed detection unit 32, an inclination angle detection unit 33, an image processing unit 34, a determination unit 35, a photometry area control unit 36, and a display control unit. 37 and a storage unit 39 .

Based on the signal output from the headlight switch 22, the headlight lighting detection unit 31
Detects whether the headlights are on or off. When the headlights are on, the front image data captured by the front camera 11 shows the road surface illuminated by the headlights. The road surface illuminated by the headlights shown in the front image data differs from the road surface not illuminated by the headlights in hue, brightness, saturation, and luminance. The headlight lighting detection unit 31 outputs the detection result to the determination unit 35 .

The vehicle speed detector 32 detects the vehicle speed of the vehicle V from the pulse signal obtained from the vehicle speed sensor 23 . The vehicle speed detection unit 32 outputs vehicle speed information indicating the calculated vehicle speed to the determination unit 35 .

The tilt angle detection unit 33 detects the tilt angle of the road on which the vehicle V travels. In this embodiment, the tilt angle detector 33 acquires an angular velocity signal output from the triaxial gyro sensor 24 . The tilt angle detection unit 33 calculates roll angle, pitch angle, and yaw angle indicating the tilt of the vehicle V from the roll angular velocity, pitch angular velocity, and yaw angular velocity included in the angular velocity signal. The tilt angle detection unit 33 outputs the calculated roll angle, pitch angle, and yaw angle to the determination unit 35 as tilt angle information. Note that the tilt angle detection unit 33 may calculate only the pitch angle and output only the pitch angle to the determination unit 35 as tilt angle information.

The image processing unit 34 displays the image data of the display unit 21 based on the image data captured by the camera unit 10.
display video data of a display video to be displayed on the . The image processing unit 34 has an image data acquisition unit 341 , a road surface area detection unit 342 , a distortion correction unit 343 and an image generation unit 344 .

The image data acquisition unit 341 acquires image data captured by the camera unit 10 . Image data acquisition unit 341 obtains front image data captured by front camera 11 and right side camera 1
2, the left image data captured by the left camera 13, and the rear image data captured by the rear camera 14 are obtained.

The road surface area detection unit 342 detects the road surface area shown in the image data acquired by the image data acquisition unit 341 . The road surface area is the area of the road surface represented in the video data. The road surface area is defined by the shape of the road, the width of the road, and the depth of the road. In this embodiment, the road surface area detection unit 342 will be described as detecting the road surface area shown in the front image data. The road surface area detection unit 342 stores a road surface area detected when the vehicle V is traveling on a flat road as a reference road surface area. The reference road area is preferably stored for each width of the road on which the vehicle V travels. A method for detecting the road surface may be any known method, and is not limited. When the detection result of the headlight lighting detection unit 31 indicates that the headlights are on, the road surface area detection unit 342 determines the hue of the video data because the road surface captured in the video data is illuminated by the headlights. , lightness, saturation, and brightness are corrected to detect the road surface area.

The distortion correction unit 343 corrects the distortion of the display area, which is a partial area of the ultra-wide-angle image data acquired by the image data acquisition unit 341, using a distortion parameter, and outputs rectangular distortion-corrected image data. . The distortion-corrected image data includes front distortion-corrected image data obtained by correcting front image data, right-side distortion-corrected image data obtained by correcting right-side image data, left-side distortion-corrected image data obtained by correcting left-side image data, and rear image data. and backward distortion corrected image data obtained by correcting the image data. When it is not necessary to distinguish between the front distortion corrected image data, the right side distortion corrected image data, the left side distortion corrected image data and the rear distortion corrected image data, they will be described as distortion corrected image data.

When the determination unit 35 determines that the inclination angle of the vehicle V is equal to or greater than the threshold, the vehicle speed of the vehicle V is equal to or less than the threshold, and the area of the road surface area has changed by the threshold or more, the distortion correction unit 343 corrects the road surface area. The distortion parameter is changed according to the change of the area to correct the distortion, and the distortion corrected image data is output. More specifically, when the area of the road surface area increases compared to the reference road surface area, the distortion correction unit 343 corrects the distortion by changing the distortion parameter so that the display area for distortion correction is shifted upward. to output distortion-corrected video data. When the area of the road surface area decreases compared to the reference road surface area, the distortion correction unit 343 corrects the distortion by changing the distortion parameter so as to shift the display area for correcting the distortion downward. Output corrected image data.

Processing in the distortion corrector 343 when the vehicle V is traveling on a flat road will be described with reference to FIGS. 2 to 5. FIG. FIG. 2 is a diagram illustrating a state in which a vehicle travels on a flat road. FIG. 3 is a diagram illustrating a state in which the vehicle travels on a flat road. FIG. 4 is a diagram showing an example of image data captured by a camera when a vehicle travels on a flat road. Figure 5 shows
FIG. 5 is a diagram showing an example of a display area on which distortion correction processing is performed in the image data shown in FIG. 4; Figure 2,
As shown in FIG. 3, when the vehicle V is traveling on a flat road, the front image data captured by the front camera 11 is as shown in FIG. 4, for example. Such front image data is used as reference image data, and a display area indicating an area to be visually recognized as an image by the user and a distortion parameter are set. Further, the road surface area detected from the front image data is the reference road surface area. For example, the display area is set to A1, which is a barrel-shaped area shown in the figure. The distortion parameter is set so that the distortion is corrected in such a display area A1 to generate rectangular forward distortion correction image data.

Here, the camera unit 10 including the front camera 11 captures an image with an ultra-wide viewing angle. For applications where a wide range is to be checked, such as when generating an image for checking the surroundings of the vehicle V, it is desirable to correct the distortion so that the entire viewing angle can be secured in the horizontal direction. In such cases,
When the distortion is corrected while maintaining a natural sense of distance to the front, the display area in the vertical direction, as opposed to the horizontal direction, becomes a part of the shooting range of the camera. Conversely, if the vertical display area is secured as it is and the distortion is corrected, the image in front will appear to be farther than it actually is. Therefore, depending on the tilt angle of the vehicle V, it is necessary to adjust the displayed vertical display area. Specifically, the distortion parameter is changed so as to shift the display area for distortion correction upward or downward from the center of the imaging range of the camera according to the increase/decrease ratio of the road surface area.

Processing in the distortion corrector 343 when the vehicle V is traveling on a road with a steep uphill gradient will be described with reference to FIGS. 6 to 9. FIG. FIG. 6 is a diagram illustrating a state in which a vehicle travels on a road having a steep uphill gradient. FIG. 7 is a diagram illustrating a state in which a vehicle travels on a road having a steep uphill gradient. FIG. 8 shows that when the vehicle travels on a road with a steep uphill gradient,
It is a figure which shows an example of the video data which the camera image|photographed. FIG. 9 is a diagram showing an example of a display area on which distortion correction processing is performed in the image data shown in FIG. As shown in FIGS. 6 and 7, the vehicle V
is traveling on a steep uphill road, the front image data captured by the front camera 11 is as shown in FIG. 8, for example. Here, it is assumed that there is an intersection that intersects with the intersection road R1 on the way up the steep slope. Another vehicle V1 is traveling on the cross road R1.
When the vehicle V is raised forward, the photographing range of the front camera 11 is directed upward. This will
The road surface area in the forward image data display area is reduced. Therefore, the distortion parameter is changed so that the position of the road surface area in the display area becomes appropriate, and the distortion parameter is changed so that the barrel-shaped area A2 shown in the figure becomes the display area. In this manner, the distortion is corrected for the display area A2, and the forward distortion correction image data is output. A display area A1 indicated by a dashed line in the figure is a display area when the distortion parameter is not changed. The same applies to the following figures.

The processing in the distortion corrector 343 when the vehicle V is traveling on a road with a steep downhill gradient will be described with reference to FIGS. 10 to 13. FIG. FIG. 10 is a diagram illustrating a state in which a vehicle travels on a road having a steep downhill gradient. FIG. 11 is a diagram illustrating a state in which a vehicle travels on a road having a steep downhill gradient. FIG. 12 is a diagram showing an example of image data captured by a camera when a vehicle travels on a road with a steep downhill gradient. FIG. 13 is a diagram showing an example of a display area on which distortion correction processing is performed in the video data shown in FIG. 12. In FIG. As shown in FIGS. 10 and 11, when the vehicle V is traveling on a steep downhill road, the front image data captured by the front camera 11 is as shown in FIG. 12, for example. Here, it is assumed that there is an intersection that intersects with the cross road R1 in the middle of the downhill steep slope. Another vehicle V1 is traveling on the cross road R1. When the vehicle V is lowered forward, the photographing range of the front camera 11 is downward. As a result, the road surface area in the front image data display area is increased. Therefore, the distortion parameter is changed so that the position of the road surface area in the display area becomes appropriate, and the distortion parameter is changed so that the barrel-shaped area A3 shown in the figure becomes the display area. in this way,
Distortion is corrected with respect to the display area A3, and forward distortion correction image data is output.

The image generator 344 generates the vehicle V based on the distortion-corrected image data output from the distortion corrector 343 .
display image data for checking the surroundings of the In this embodiment, the image generation unit 344 generates display image data that allows confirmation of the front of the vehicle V based on the forward distortion correction image data.

The image generator 344 generates the vehicle V based on the distortion-corrected image data output from the distortion corrector 343 .
display video data of a bird's-eye view video including a virtual own vehicle image looking down from above may be generated. In this case, the image generation unit 344 combines the viewpoint conversion processing and the own vehicle icon indicating the vehicle V with the front distortion corrected image data, the right side distortion corrected image data, the left side distortion corrected image data, and the rear distortion corrected image data. Synthesis processing is performed to generate display image data of a bird's-eye view image that displays the surroundings of the vehicle V. FIG. The method for generating the bird's-eye view image may be any known method, and is not limited.

The determination unit 35 determines the tilt angle of the vehicle V detected by the tilt angle detection unit 33 and the road surface area detection unit 342.
Based on the detected road surface area, a change in the inclination angle of the vehicle V and the area of the road surface area with respect to the threshold value is determined. More specifically, the determination unit 35 detects the vehicle V based on the tilt angle of the vehicle V detected by the tilt angle detection unit 33, the vehicle speed detected by the vehicle speed detection unit 32, and the road surface area detected by the road surface area detection unit 342. It is determined whether or not the inclination angle of V is greater than or equal to a threshold value, the vehicle speed of vehicle V is less than or equal to the threshold value, and the area of the road surface region has changed by more than or equal to the threshold value. The threshold for the tilt angle is, for example, about 5 degrees. The vehicle speed threshold is, for example, about 10 km/h. The threshold for the area of the road surface area is, for example,
It is about 20%.

Whether or not the area of the road surface area has changed by a threshold value or more is determined when the area of the road surface area detected by the road surface area detection unit 342 is about 20% or more of the area of the reference road surface area of the road having the same road width. , has increased or decreased. When it is determined that the area of the road surface area detected by the road surface area detection unit 342 has increased or decreased by about 20% or more compared to the area of the reference road surface area, it is determined that the area of the road surface area has changed by a threshold value or more. When it is determined that the area of the road surface area detected by the road surface area detection unit 342 has not increased or decreased by about 20% or more compared to the area of the reference road surface area, it is determined that the area of the road surface area has not changed by the threshold value or more. judge.

Alternatively, whether or not the area of the road surface area has changed by a threshold value or more depends on whether or not the area of the road surface area detected by the road surface area detection unit 342 has increased or decreased by about 20% or more during a predetermined period of time or a predetermined travel distance. You can judge. When it is determined that the area of the road surface area detected by the road surface area detection unit 342 has increased or decreased by about 20% or more during the predetermined period or the predetermined travel distance, it is determined that the area of the road surface area has changed by a threshold value or more. When it is determined that the area of the road surface area detected by the road surface area detection unit 342 has not increased or decreased by about 20% or more in the predetermined period or the predetermined travel distance, it is determined that the area of the road surface area has not changed by the threshold value or more. .

When the distortion correction unit 343 changes the distortion parameter, the photometry area control unit 36 controls the position of the photometry area when the camera unit 10 takes an image in accordance with the change of the distortion parameter. For example, the position of the photometry area when the camera unit 10 takes an image in a normal state is the central portion of the display area. When the distortion correcting unit 343 shifts the display area for correcting distortion upward, the photometric area control unit 36 shifts the photometric area upward so that the central portion of the display area shifted upward becomes the photometric area. A control signal is output to the camera unit 10 . When the distortion correction unit 343 shifts the display area for distortion correction downward, the photometry area control unit 36 shifts the photometry area downward so that the central portion of the display area shifted downward becomes the photometry area. to the camera unit 10.

The display control unit 37 controls the display unit 21 to display the display image data generated by the image generation unit 344 of the image processing unit 34 .

The storage unit 39 stores data required for various processes in the vehicle image processing device 30 and various processing results. The storage unit 39 is, for example, a RAM (Random Access Me
memory), ROM (Read Only Memory), flash memory (Fl
(sh Memory) or other semiconductor memory elements, or hard disks, optical disks,
It is a storage device such as an external storage device via a communication network.

Next, the flow of processing in the vehicle image processing device 30 will be described with reference to FIG. 14 . FIG. 14 is a flow chart showing the flow of processing in the vehicle image processing device 30 according to the embodiment. Here, based on the front image data captured by the front camera 11, the vehicle V
A case of displaying an image for confirming the safety in front of the vehicle will be described.

It is assumed that the vehicle image processing system 1 is started when the engine of the vehicle V is started. The camera unit 10 photographs the surroundings of the vehicle V while the vehicle image processing system 1 is activated. While the vehicle image processing system 1 is running, the image data acquisition unit 341
Acquire video data captured by

The video processing device 30 for vehicles detects lighting of a headlight (step S101). The vehicle image processing device 30 detects whether the headlights are on or off from the signal output from the headlight switch 22 by the headlight lighting detection unit 31 . The vehicle image processing device 30 proceeds to step S102.

The vehicle image processing device 30 detects a road surface area (step S102). The vehicle image processing device 30 uses a road surface area detection unit 342 to detect the road surface area shown in the front image data using a known road surface detection method. The vehicle image processing device 30 performs step S
Go to 103.

The vehicle image processing device 30 determines whether the vehicle speed is equal to or less than the threshold value by the determination unit 35 (step S103). When the determination unit 35 determines that the vehicle speed detected by the vehicle speed detection unit 32 is equal to or less than the threshold value (Yes in step S103), the vehicle video processing device 30 performs step S
Go to 104. When the determination unit 35 determines that the vehicle speed detected by the vehicle speed detection unit 32 is not equal to or lower than the threshold value (No in step S103), the vehicle video processing device 30 executes the process of step S101 again.

When it is determined that the vehicle speed detected by the vehicle speed detection unit 32 is equal to or lower than the threshold (Yes in step S103), the vehicle video processing device 30 causes the determination unit 35 to determine whether the pitch angle is equal to or greater than the threshold (step S104). When the determination unit 35 determines that the pitch angle detected by the tilt angle detection unit 33 is equal to or greater than the threshold value (Y in step S104), the vehicle video processing device 30
es), and proceed to step S105. When the determination unit 35 determines that the pitch angle detected by the tilt angle detection unit 33 is not equal to or greater than the threshold value (N in step S104), the vehicle video processing device 30
o), go to step S111.

When it is determined that the pitch angle detected by the tilt angle detection unit 33 is equal to or greater than the threshold (step S1
04), the vehicle image processing device 30 causes the determination unit 35 to detect the road surface area detection unit 34
Based on the road surface area detected by 2, it is determined whether the area of the road surface area has increased or decreased (step S
105). When the determination unit 35 determines that the area of the road surface area has increased or decreased (Yes in step S105), the vehicle image processing device 30 proceeds to step S106. When the determination unit 35 determines that the area of the road surface area has not increased or decreased (step S1
05: No), the process of step S101 is executed again.

When determining that the area of the road surface area has increased or decreased (Yes in step S105), the vehicle image processing device 30 performs correction processing of the display area (step S106). The vehicle image processing device 30 performs display area correction processing for correcting distortion by changing a distortion parameter according to a change in the area of the road surface area by the distortion correction unit 343 . The vehicle image processing device 30 includes the distortion correction unit 3
In 43, when the area of the road surface area increases, the distortion parameter is changed so as to shift the display area for distortion correction upward, and the display area correction process for correcting the distortion is performed. When the area of the road surface area is reduced by the distortion correction unit 343, the vehicle image processing device 30 changes the distortion parameter so as to shift the display area for correcting the distortion downward. correction processing is performed. The vehicle image processing device 30 proceeds to step S107.

The vehicle image processing device 30 generates display image data (step S107). The vehicle image processing device 30 uses the image generation unit 344 to generate display image data from the forward distortion correction image data. The vehicle image processing device 30 proceeds to step S108.

The vehicle image processing device 30 causes the display control unit 37 to display the generated display image data on the display unit 21 (step S108). The vehicle video processing device 30 performs step S10
Go to 9.

The vehicle image processing device 30 determines whether the pitch angle is less than the threshold by the determination unit 35 (step S109). In the vehicle video processing device 30, the determination unit 35 determines whether the tilt angle detection unit 3
If it is determined that the pitch angle detected by 3 is less than the threshold (Yes in step S109),
Proceed to step S110. In the vehicle video processing device 30 , the determination unit 35 determines whether the tilt angle detection unit 33
If it is determined that the detected pitch angle is not less than the threshold value (No in step S109), the process of step S109 is executed again.

When it is determined that the pitch angle detected by the tilt angle detection unit 33 is less than the threshold (step S1
09: Yes), the vehicle image processing device 30 returns the display area to the initial setting (step S1
10). In the vehicle image processing device 30, the distortion correction unit 343 restores the distortion parameter changed in step S106 to the initial value. The vehicle image processing device 30 restores the distortion parameter to the distortion parameter when the vehicle V is traveling on a flat road by the distortion correction unit 343 . The vehicle image processing device 30 proceeds to step S113.

When it is determined that the pitch angle detected by the tilt angle detection unit 33 is not equal to or greater than the threshold (step S
No at 104), the vehicle video processing device 30 generates display video data (step S1
11). The vehicle video processing device 30 generates display video data from the video data by the video generation unit 344 . The vehicle image processing device 30 proceeds to step S112.

The vehicle image processing device 30 causes the display control unit 37 to display the display image data on the display unit 21 (step S112). The vehicle image processing device 30 proceeds to step S113.

The vehicle video processing device 30 determines whether or not the vehicle speed is greater than the threshold by the determination unit 35 (step S113). When the determination unit 35 determines that the vehicle speed detected by the vehicle speed detection unit 32 is greater than the threshold value (Yes in step S113), the vehicle video processing device 30 performs step S
Proceed to 114. When the determination unit 35 determines that the vehicle speed detected by the vehicle speed detection unit 32 is not greater than the threshold value (No in step S113), the vehicle video processing device 30 performs step S113.
process again.

When determining that the vehicle speed detected by the vehicle speed detection unit 32 is greater than the threshold value (Yes in step S113), the vehicle video processing device 30 stops displaying the display video data (step S1
14). The vehicle image processing device 30 proceeds to step S115.

The vehicle image processing device 30 determines whether to end the process (step S115). More specifically, the vehicle image processing device 30 determines whether or not to end the process based on the presence or absence of an end trigger. The termination trigger is, for example, a combination of engine off, gear shift to parking, parking brake operation, and the like. A termination trigger may be a termination trigger by a user's operation. If there is an end trigger, the vehicle image processing device 30 determines to end the process (Yes in step S115), and ends the process. If there is no end trigger, the vehicle image processing device 30 determines not to end the processing (step S11
5: No), the process of step S101 is executed again.

In this way, when the determination unit 35 determines that the inclination angle of the vehicle V is equal to or greater than the threshold, the vehicle speed of the vehicle V is equal to or less than the threshold, and the area of the road surface area has changed by the threshold or more, the distortion correction unit 343 corrects the distortion by changing the distortion parameter according to the change in the area of the road surface area. Distortion corrector 3
43 corrects the distortion by changing the distortion parameter so that the display area for correcting the distortion is shifted upward when the area of the road surface area increases. When the area of the road surface area decreases, the distortion correction unit 343 corrects the distortion by changing the distortion parameter so as to shift the display area for correcting the distortion downward.

As described above, in this embodiment, when it is determined that the inclination angle of the vehicle V is greater than or equal to the threshold, the vehicle speed of the vehicle V is less than or equal to the threshold, and the area of the road surface area has changed by more than the threshold, the area of the road surface area , the distortion parameter can be changed to compensate for the distortion. In this embodiment, when the area of the road surface area increases, the distortion can be corrected by changing the distortion parameter such that the display area for distortion correction is shifted upward. When the area of the road surface area decreases, the distortion can be corrected by changing the distortion parameter such that the display area for distortion correction is shifted downward. In this manner, according to the present embodiment, regardless of the tilt angle of the vehicle V, it is possible to display an image in an appropriate display area.

When the distortion parameter is changed, this embodiment can control the position of the photometry area when the camera unit 10 takes an image in accordance with the change of the distortion parameter. According to the present embodiment, regardless of the tilt angle of the vehicle V, it is possible to set an appropriate photometry area, perform photometry appropriately, and perform photographing.

As described above, according to the present embodiment, regardless of the inclination angle of the vehicle V, the surroundings of the vehicle V can be confirmed with an appropriate image.

Now, the video processing system 1 for vehicle according to the present invention has been described so far, but it may be implemented in various different forms other than the above-described embodiment.

Each component of the vehicle image processing system 1 illustrated is functionally conceptual, and does not necessarily have to be physically configured as illustrated. In other words, the specific form of each device is not limited to the illustrated one, and all or part of it may be functionally or physically distributed or integrated in arbitrary units according to the processing load and usage conditions of each device. may

The configuration of the video processing system 1 for vehicle is realized by, for example, a program loaded in a memory as software. In the above embodiments, functional blocks realized by cooperation of these hardware or software have been described. That is, these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

The components described above include those that can be easily assumed by those skilled in the art and those that are substantially the same.
Furthermore, the above configurations can be combined as appropriate. Also, various omissions, substitutions, or modifications of the configuration are possible without departing from the gist of the present invention.

A case where the road on which the vehicle V is traveling and the cross road are curved will be described with reference to FIGS. 15 and 16 . FIG. 15 is a diagram showing an example of a display area in which distortion correction processing is performed when an intersecting road is curved. FIG. 16 is a diagram illustrating a state in which a road that intersects a road on which a vehicle is traveling is curved. If the road on which the vehicle V is traveling and the cross road are curved, the distortion correction unit 343 may correct the distortion by changing the distortion parameter according to the shape of the curve. The vehicle image processing device 30 receives a signal from a navigation system (not shown) to the vehicle V.
and a navigation information acquisition unit that acquires the current position information of the current position of the vehicle V and the map information around the vehicle V. FIG. Further, the vehicle image processing device 30 has a road shape detection section that detects the shape of the road based on the information acquired by the navigation information acquisition section. The distortion correction unit 343 corrects the distortion by changing the distortion parameter according to the change in the area of the road surface area and the shape of the road detected by the road shape detection unit. The distortion correction unit 343 shifts the right or left side of the display area to correct the distortion upward or downward so that the display area on the curved side can be displayed far away according to the curve shape of the cross road. , the distortion parameter is changed so that A4, which is the area having the shape shown in the figure, becomes the display area. As a result, when the cross road is curved, it is possible to display the image around the vehicle V in a display area adapted to the shape of the curve, which is difficult for the user to directly visually recognize.

A case where the vehicle V is traveling on a snowy road or the like will be described with reference to FIG. 17 . FIG. 17 shows
FIG. 10 is a diagram showing an example of a display area in which distortion correction processing is performed when the vehicle travels on a snowy road; for example,
When the road surface area detection unit 342 cannot detect the road surface due to snow or flooding, the distortion parameter is changed so that a wider range is used as the display area.
Change the distortion parameter so that is the display area. As a result, when it is difficult to detect the road surface area due to snow or flooding, it is possible to display an image of the surroundings of the vehicle V in a display area in which the road surface area is wider than usual.

A case where an image for checking the surroundings of the vehicle V is displayed when the vehicle is parked will be described. In this case, the vehicle image processing device 30 executes the processing of the flowchart shown in FIG. 14 when it is determined that there is a reverse trigger. The reverse trigger means, for example, that the shift position is set to "reverse" or that the traveling direction of the vehicle V becomes backward in the longitudinal direction of the vehicle V. As shown in FIG. Then, in step S102, the road surface area shown in the rear image data is detected,
In step S105, it is determined whether the area of the road surface area of the rear image data has increased or decreased.
Then, in step S106, the distortion parameter is changed to correct the distortion of the rear image data. In this way, when the vehicle is parked, regardless of the inclination angle of the vehicle V, the image behind the vehicle V can be displayed in an appropriate display area.

The vehicle image processing system 1 can also be applied to a so-called drive recorder that captures and records surrounding images while the vehicle V is running. In this case, regardless of the vehicle speed of the vehicle V, the determination unit 3
5, based on the inclination angle of the vehicle V detected by the inclination angle detection unit 33 and the road surface area detected by the road surface area detection unit 342, the inclination angle of the vehicle V is equal to or greater than the threshold, and the area of the road surface area is the threshold. that's all,
Determine whether or not it has changed. Further, when the determination unit 35 determines that the inclination angle of the vehicle V is equal to or greater than the threshold and the area of the road surface area has changed by the threshold or more, the distortion correction unit 343 corrects the distortion according to the change in the area of the road surface area. Modify parameters to compensate for distortion. The vehicle video processing device 30 is
Processes other than steps S103, S113, and S114 of the flowchart shown in FIG. 14 are executed. Furthermore, when applying to a drive recorder, the storage unit 39
Separately, an external recording device including, for example, a card-type recording medium for storing video data may be provided outside the video processing device 30 for vehicle.

The vehicle image processing device 30 may include an optical axis control section that controls the position of the optical axis of the headlights in accordance with the change in the distortion parameter when the distortion correction section 343 changes the distortion parameter. Accordingly, an appropriate image of the surroundings of the vehicle V can be displayed.

Instead of the 3-axis gyro sensor 24, a 6-axis sensor having the functions of a 3-axis acceleration sensor and a 3-axis gyro sensor capable of detecting acceleration in the X, Y, and Z directions may be used. .

1 vehicle image processing system 10 camera unit 21 display unit 22 headlight switch 23 vehicle speed sensor 24 3-axis gyro sensor 30 vehicle image processing device 31 headlight lighting detection unit 32 vehicle speed detection unit 33 tilt angle detection unit 34 image processing unit 341 Image data acquisition unit 342 road surface area detection unit 343 distortion correction unit 344 image generation unit 35 determination unit 36 photometry area control unit 37 display control unit 39 storage unit

Claims (6)

a video data acquisition unit that acquires video data captured by an imaging unit that captures an image of the surroundings of the vehicle;
a distortion correction unit that corrects distortion of the image data acquired by the image data acquisition unit using a distortion parameter and outputs distortion-corrected image data;
a road shape detection unit that detects the shape of a road based on current position information of the current position of the vehicle and map information around the vehicle;
a road surface area detection unit that detects a road surface area shown in the video data acquired by the video data acquisition unit;
with
The distortion correction unit corrects the distortion by changing the distortion parameter according to a change in the area of the road surface area and the shape of the road detected by the road shape detection unit.
A video processing device for a vehicle characterized by: The distortion correction unit changes the distortion parameter so as to shift the distortion correction area of the display area on the curving side upward or downward according to the curve shape of the cross road. to correct,
The vehicle image processing device according to claim 1 . an image generation unit that generates image data for displaying the surroundings of the vehicle based on the distortion-corrected image data corrected by the distortion correction unit;
The vehicle video processing device according to claim 1 or 2, comprising: A video processing device for a vehicle according to claim 3;
at least one of the imaging unit and a display unit that displays video data generated by the vehicle video processing device;
A video processing system for a vehicle, comprising: a video data acquisition step of acquiring video data captured by the imaging unit that captures the surroundings of the vehicle;
a distortion correction step of correcting the distortion of the video data obtained by the video data obtaining step using a distortion parameter and outputting distortion-corrected video data;
a road shape detection step of detecting a shape of a road based on current position information of the current position of the vehicle and map information around the vehicle;
a road surface area detection step of detecting a road surface area shown in the image data acquired by the image data acquisition step;
including
In the distortion correction step, the distortion is corrected by changing the distortion parameter according to a change in the area of the road surface area and the shape of the road detected by the road shape detection unit.
Video processing method for vehicle. a video data acquisition step of acquiring video data captured by the imaging unit that captures the surroundings of the vehicle;
a distortion correction step of correcting the distortion of the video data obtained by the video data obtaining step using a distortion parameter and outputting distortion-corrected video data;
a road shape detection step of detecting a shape of a road based on current position information of the current position of the vehicle and map information around the vehicle;
a road surface area detection step of detecting a road surface area shown in the image data acquired by the image data acquisition step;
including
In the distortion correction step, the distortion is corrected by changing the distortion parameter according to a change in the area of the road surface area and the shape of the road detected by the road shape detection unit.
A program for causing a computer that operates as a video processing device for a vehicle to execute.

Images