WO2016035271A1

WO2016035271A1 - Image processing device for vehicle

Info

Publication number: WO2016035271A1
Application number: PCT/JP2015/004133
Authority: WO
Inventors: 祥吾大宮; 博彦柳川; 敦志村
Original assignee: 株式会社デンソー
Priority date: 2014-09-02
Filing date: 2015-08-19
Publication date: 2016-03-10

Abstract

An image processing device for a vehicle, which generates a vehicle surroundings image of an overhead view of the surroundings of the vehicle as seen from above, the image processing device being provided with: a surroundings image acquisition unit (S30) that acquires a photographed front image and a photographed rear image from a front photographic device (11) and a rear photographic device (12); a calculation unit (S80) that calculates the moving distance of the vehicle using vehicle state information, including the information associated with the tire rotation angle of the vehicle, and parameter information associated with the tire outer diameter stored in a storage unit (34); a composite image creation unit (S100, S110, S120) that creates a vehicle surroundings image by using, as a history image, one past image of a region in the traveling direction of the vehicle among the photographed front image and photographed rear image, and compositing the history image with the other image on the basis of the moving distance; and a tire parameter updating unit (S250 to S360, S440, S480, S490) that updates the parameter information stored in the storage unit so as to reduce the deviation between the history image and the other image composited on the basis of the moving distance.

Description

Image processing apparatus for vehicle

Cross-reference of related applications

This application is based on Japanese Application No. 2014-178098 filed on September 2, 2014 and Japanese Application No. 2015-010341 filed on January 22, 2015, the contents of which are described herein. Is used.

The present disclosure relates to an image processing apparatus for a vehicle that performs processing on an image captured by an imaging apparatus installed in a vehicle.

Conventionally, a part of the vehicle periphery is continuously photographed, and the photographed image taken at the present time and the image photographed before the present time are converted into an image looking down from above the vehicle, and joined together. A technique for creating a single top-view image overlooking the entire periphery is known (see, for example, Patent Document 1).

JP 2002-373327 A

In the technique described in Patent Document 1, when the outer diameter of a tire of a vehicle changes due to tire replacement or the like, in one top view image, a captured image captured at the present time, a history image captured before the current time, There is a risk of misalignment near the boundary.

The present disclosure has been made in view of the above points, and even if a vehicle peripheral image is created by synthesizing a history image with a captured image, by smoothly connecting the captured image and the history image. An object of the present invention is to provide a vehicular image processing device that reduces the sense of discomfort.

An image processing device for a vehicle according to an aspect of the present disclosure generates a vehicle peripheral image in which a periphery of the vehicle is viewed from above from an image of the periphery of the vehicle captured by a plurality of imaging devices installed in the vehicle, A peripheral image acquisition unit, a calculation unit, a composite image creation unit, and a tire parameter update unit are provided.

The peripheral image acquisition unit acquires a front captured image and a rear captured image from each of a front image capturing device that continuously images a front region of the vehicle and a rear image capturing device that continuously images a rear region of the vehicle.

The calculation unit calculates the travel distance of the vehicle using vehicle state information including information related to the rotation angle of the tire of the vehicle and parameter information related to the outer diameter of the tire stored in the storage unit.

The composite image creating unit uses one past image that shows the traveling direction of the vehicle among the front photographed image and the rear photographed image as a history image, and moves to the other image of the front photographed image and the rear photographed image. A vehicle peripheral image is created by synthesizing history images based on the above.

The tire parameter update unit updates the parameter information stored in the storage unit so that the deviation between the history image synthesized based on the moving distance and the other image is reduced.

The vehicular image processing device of the present disclosure configured as described above is synthesized based on the other image not used for the history image among the front captured image and the rear captured image, and the moving distance calculated by the calculation unit. The parameter information can be updated so as to reduce the deviation from the recorded history image. Therefore, even when the outer diameter of the tire changes for some reason, the vehicular image processing apparatus determines that the travel distance calculated by using the tire rotation angle specified by the vehicle state information and the parameter information is the actual distance of the vehicle. It is possible to suppress the occurrence of a situation in which the distance is different from the movement distance.

For this reason, the vehicle image processing device of the present disclosure can smoothly connect the captured image and the history image even if the vehicle peripheral image is generated by synthesizing the history image with the captured image. Thereby, the vehicular image processing device according to the present disclosure can realize generation of a vehicle periphery image with reduced discomfort.

The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description with reference to the accompanying drawings. The drawing
FIG. 1 is a block diagram showing a configuration of a vehicle display system, FIG. 2 is a flowchart showing the peripheral display process. FIG. 3 is a diagram showing a configuration of a display top view image. FIG. 4 is a flowchart showing the tire parameter update process of the first embodiment. FIG. 5 is a view showing a top-view image for display when the tire outer diameter is increased, FIG. 6 is a view showing a top-view image for display when the tire outer diameter is reduced, FIG. 7 is a flowchart showing the tire parameter update process of the second embodiment, FIG. 8 is a diagram showing a display top view image in which measurement objects are displayed in the rear display area and the intermediate display area. FIG. 9 is a flowchart showing the tire parameter update process of the third embodiment, FIG. 10 is a diagram showing a top view image for explaining a method of calculating the speed of a moving object. FIG. 11A is a diagram showing a luminance change pattern in the front view top-view image; FIG. 11B is a diagram showing a luminance change pattern in the rear display top view image.

(First embodiment)
Hereinafter, a first embodiment of the present disclosure will be described with reference to the drawings.

The vehicle display system 1 of this embodiment is mounted on a vehicle and includes an in-vehicle camera group 2, a sensor group 3, an image processing device 4, and a display device 5, as shown in FIG. The image processing device 4 is a vehicle image processing device according to the present disclosure.

The in-vehicle camera group 2 includes a front camera 11 and a rear camera 12. The front camera 11 is attached to the front side of a vehicle (hereinafter referred to as the host vehicle) on which the vehicle display system 1 is mounted, and repeatedly photographs the ground surface in front of the host vehicle. The rear camera 12 is attached to the rear side of the host vehicle, and repeatedly photographs the ground surface behind the host vehicle.

The sensor group 3 includes a vehicle speed sensor 21, a yaw rate sensor 22, a shift lever position sensor 23, and a steering angle sensor 24. The vehicle speed sensor 21 outputs a vehicle speed signal that changes from a low level to a high level at every predetermined angle according to the rotation of the axle. The yaw rate sensor 22 detects the yaw rate of the host vehicle and outputs a yaw rate signal indicating the detection result. The shift lever position sensor 23 detects the position of the shift lever of the vehicle and outputs a shift position signal indicating the detection result. The steering angle sensor 24 detects the steering angle of the steering wheel of the vehicle, and outputs a steering angle signal indicating the detection result.

The image processing apparatus 4 includes an image input signal processing unit 31, an input signal processing unit 32, an image conversion unit 33, a storage unit 34, a control unit 35, and an image output signal processing unit 36.

The image input signal processing unit 31 is an interface for inputting captured images from the front camera 11 and the rear camera 12.

The input signal processing unit 32 is an interface for inputting signals from the various sensors 21 to 24.

The image conversion unit 33 obtains a front photographed image and a rear photographed image from the front camera 11 and the rear camera 12 via the image input signal processing unit 31, and performs coordinate conversion of the photographed image, so that the image is viewed from the viewpoint above the vehicle. A converted image (hereinafter referred to as a top view image) is created. Hereinafter, a top view image created by performing coordinate conversion on a captured image of the front camera 11 is referred to as a front top image. In addition, a top view image created by coordinate-transforming a captured image of the rear camera 12 is referred to as a rear top image. The image conversion unit 33 outputs the created front top image and rear top image to the control unit 35.

The storage unit 34 is a storage device for storing various data. The front top image and the rear top image created by the image conversion unit 33, and the outer diameter of the tire of the host vehicle (hereinafter referred to as the host vehicle tire outer diameter). Is stored. Note that the tire parameter information indicates the outer diameter of one tire, not the outer diameter of each of the four tires, assuming that all four tires of the host vehicle have the same outer diameter.

The control unit 35 is mainly configured by a microcomputer including a CPU 41, a ROM 42, a RAM 43, and the like. The control unit 35 controls the image processing apparatus 4 by the CPU 41 executing a process based on a program stored in the ROM 42.

The image output signal processing unit 36 is an interface for outputting image data processed by the control unit 35 to the display device 5.

The display device 5 is a color display device having a display screen such as a liquid crystal display, and displays various images on the display screen in response to input of image data from the image processing device 4.

In the vehicular display system 1 configured as described above, the image processing device 4 executes a peripheral display process and a tire parameter update process which will be described later.

First, the procedure of the peripheral display process executed by the control unit 35 of the image processing apparatus 4 will be described. This peripheral display process is a process that is repeatedly executed every preset execution cycle during the operation of the image processing apparatus 4.

When this peripheral display process is executed, the control unit 35, as shown in FIG. 2, first from the vehicle speed sensor 21, the yaw rate sensor 22, and the shift lever position sensor 23 via the input signal processing unit 32 in S10, respectively. A vehicle speed signal, a yaw rate signal, and a shift position signal are acquired. And the vehicle speed information which shows the vehicle speed specified by the vehicle speed signal acquired by S10, the yaw rate information which shows the yaw rate specified by the yaw rate signal acquired by S10, and the shift position information specified by the shift position signal acquired by S10 Is stored in a vehicle information storage area provided in the storage unit 34. The vehicle speed indicated by the vehicle speed information is calculated based on the time interval input by the vehicle speed signal. Thereby, a plurality of vehicle speed information, yaw rate information, and shift position information are stored in time series in the vehicle information storage area of the storage unit 34.

Next, in S30, the front top image and the rear top image are acquired from the image conversion unit 33. Further, in S40, the front top image and the rear top image acquired in S30 are stored in the front / rear top image storage area provided in the storage unit 34. Accordingly, a plurality of front top images and rear top images are stored in time series in the front / rear top image storage area of the storage unit 34.

Thereafter, in S50, based on the acquired vehicle speed signal, the traveling speed of the host vehicle is equal to or less than a preset low-speed traveling determination speed (in this embodiment, for example, 30 km / h) indicating that the vehicle is traveling at a low speed. Judge whether there is.

If the traveling speed of the host vehicle exceeds the low-speed traveling determination speed (S50: NO), the front top image and the rear top image acquired in S30 are output to the display device 5 in S60, The display process is temporarily terminated. Thereby, the front top image and the rear top image are displayed on the display screen of the display device 5.

On the other hand, when the traveling speed of the host vehicle is equal to or lower than the low-speed traveling determination speed (S50: YES), the shift position of the host vehicle is set to P (parking) or N (based on the acquired shift position signal in S70. It is determined whether it is neutral). If the shift position is P or N (S70: YES), the data indicating the front top image and the rear top image acquired in S30 is output to the display device 5 in S60, and the peripheral display processing is performed. Exit once.

If the shift position is neither P nor N in S70 (S70: NO), a plurality of vehicle speed information, yaw rate information, and shift position information stored in time series in the storage unit 34 in S80. Based on the vehicle speed, yaw rate, and shift position specified by the above, the moving distance and moving direction of the host vehicle are sequentially calculated. In S80, the left and right moving directions are calculated based on the yaw rate specified by the plurality of pieces of yaw rate information stored in time series in the storage unit 34. Further, based on the shift positions specified by the plurality of shift position information stored in time series in the storage unit 34, the front and rear moving directions are calculated. Further, based on the vehicle speed specified by the plurality of vehicle speed information stored in the storage unit 34 in time series and the own vehicle tire outer diameter indicated by the tire parameter information stored in the storage unit 34, Calculate the travel distance.

In S90, it is determined whether or not the shift position of the host vehicle is D (drive). If the shift position is D (S90: YES), an intermediate top image is created using the front top image stored in the storage unit 34 in S100, and the created intermediate top image is After storing in the intermediate top image storage area provided in the storage unit 34, the process proceeds to S120. The intermediate top image is a top view image in an area (hereinafter referred to as an intermediate area) located between the area indicated by the front top image and the area indicated by the rear top image. Note that the intermediate top image corresponds to a history image in the present disclosure.

In S100, based on the movement distance and movement direction calculated in S80, the front top image capturing the intermediate region is extracted from the storage unit. Further, in S100, an intermediate top image is created by performing coordinate transformation in which the extracted front top image is translated or rotated in accordance with the difference between the extracted front top image and the above intermediate region. Note that the front top image extracted in S100 corresponds to one image in the present disclosure, and a history image in the present disclosure is created by the extracted front top image.

On the other hand, if the shift position of the host vehicle is not D (S90: NO), it is determined that the shift position of the host vehicle is R (reverse), and the top top image stored in the storage unit 34 is stored in S110. The intermediate top image is created by using the same method as in S100, and the created intermediate top image is stored in the intermediate top image storage area provided in the storage unit 34, and then the process proceeds to S120. Note that the rear top image extracted in S110 corresponds to one image in the present disclosure, and a history image in the present disclosure is created by the extracted rear top image.

Then, when the process proceeds to S120, a top view image (hereinafter referred to as a display top view image) to be displayed on the display device 5 is created.

As shown in FIG. 3, the display top view image includes a front display area R1, a rear display area R2, and an intermediate display area R3. The front display area R1 is an area for displaying a top view image in front of the host vehicle. The rear display area R2 is an area for displaying a top view image behind the host vehicle. The intermediate display area R3 is an area for displaying a top view image between the front and rear of the host vehicle.

In S120, the front top image and the rear top image acquired in S30 are allocated to the front display region R1 and the rear display region R2, respectively, and the latest stored in the intermediate top image storage region of the storage unit 34 is stored in the intermediate display region R3. Assign an intermediate top image. Further, in S120, the host vehicle image CG is arranged in the intermediate display region R3. Thereby, the creation of the display top view image is completed.

Then, when the process of S120 is completed, as shown in FIG. 2, in S130, data indicating the display top view image created in S120 is output to the display device 5, and the peripheral display process is temporarily terminated. As a result, the display top view image is displayed on the display screen of the display device 5.

Next, the procedure of the tire parameter update process executed by the control unit 35 of the image processing apparatus 4 will be described. This tire parameter update process is a process that is started immediately after the image processing apparatus 4 is started.

When this tire parameter update process is executed, the control unit 35 first determines in S210 whether or not a preset process start condition is satisfied, as shown in FIG. The process start condition of the present embodiment is that the host vehicle has traveled a predetermined process start determination distance from the establishment of the previous process start condition.

Here, when the process start condition is not satisfied (S210: NO), the process of S210 is repeated to wait until the process start condition is satisfied.

On the other hand, if the process start condition is satisfied (S210: YES), whether or not the traveling speed of the host vehicle is equal to or lower than a preset process start determination speed (40 km / h in the present embodiment, for example) in S220. Judge whether or not. Here, if the traveling speed of the host vehicle exceeds the processing start determination speed (S220: NO), the process of S220 is repeated until the traveling speed becomes equal to or lower than the processing start determination speed.

On the other hand, if the traveling speed of the host vehicle is equal to or lower than the processing start determination speed (S220: YES), whether or not the host vehicle is traveling straight based on the steering angle signal from the steering angle sensor 24 in S230. Judging. Specifically, when the steering angle indicated by the steering angle signal is within a preset straight traveling determination angle range (for example, −3 ° to + 3 °), it is determined that the host vehicle is traveling straight.

Here, when the host vehicle is not traveling straight (S230: NO), the process of S230 is repeated to wait until the host vehicle is traveling straight. On the other hand, if the host vehicle is traveling straight (S230: YES), whether or not the shift position of the host vehicle is R (reverse) based on the shift position signal from the shift lever position sensor 23 in S240. Determine whether.

If the shift position is not R (S240: NO), it is determined that the shift position is D, and image recognition is performed using the latest rear top image stored in the storage unit 34 in S250. By performing processing (for example, pattern matching), the measurement object shown in the rear top image is detected. The measurement object of the present embodiment is, for example, a pylon, a sign, a road surface printing, a manhole, and a pole.

In S260, it is determined whether or not the measurement object is detected in S250. Here, when the measurement object is not detected (S260: NO), the process proceeds to S250 and the above process is repeated. On the other hand, when a measurement object is detected (S260: YES), the same measurement object as the measurement object detected in S250 is captured from the front top image stored in the storage unit 34 in S270. Extract the image that is.

In S280, a time difference (hereinafter referred to as a time difference between before and after) between the shooting timing of the rear top image where the measurement object is detected in S250 and the shooting timing of the front top image extracted in S270 is calculated. Further, in S290, based on the position where the measurement object appears in the rear top image where the measurement object is detected in S250 and the position where the measurement object appears in the front top image extracted in S270, The distance that the host vehicle has traveled within the time difference (hereinafter referred to as the front-rear travel distance) is calculated, and the process proceeds to S350.

In S240, when the shift position is R (S240: YES), the image recognition process is performed in the same manner as S250 using the latest front top image stored in the storage unit 34 in S300. By doing so, the measurement object in the front top image is detected.

In S310, it is determined whether or not a measurement object is detected in S300. If no measurement object has been detected (S310: NO), the process proceeds to S300 and the above-described processing is repeated. On the other hand, when the measurement object is detected (S310: YES), the same measurement object as the measurement object detected in S300 is captured from the rear top image stored in the storage unit 34 in S320. Extract the image that is.

In S330, a time difference (front-rear time difference) between the shooting timing of the front top image at which the measurement object is detected in S300 and the shooting timing of the rear top image extracted in S320 is calculated. Further, in S340, based on the position where the measurement object is reflected in the front top image where the measurement object is detected in S300 and the position where the measurement object is reflected in the rear top image extracted in S320, The distance traveled by the host vehicle within the time difference (front / rear travel distance) is calculated, and the process proceeds to S350.

Then, when the process proceeds to S350, first, an angle (hereinafter referred to as a front and rear tire rotation angle) in which the tire of the host vehicle is rotated by the front and rear time difference is calculated from the front and rear time difference calculated in S280 or S330 and the vehicle speed signal. Note that the front and rear tire rotation angles can be calculated based on the number of times the vehicle speed signal is input during the time period difference. The front and rear tire rotation angles can be calculated beyond 360 °. Further, in S350, the vehicle tire outer diameter is calculated based on the front / rear travel distance calculated in S290 or S340 and the front / rear tire rotation angle calculated in S350. Specifically, based on the front and rear running distance and the front and rear tire rotation angle, the distance traveled while the tire rotates 360 ° is calculated as the tire circumference, and the tire circumference is divided by π. The vehicle tire outer diameter.

In S360, the value of the tire parameter information stored in the storage unit 34 is updated to the value calculated in S350, and the tire parameter update process is terminated.

The image processing apparatus 4 configured as described above includes a front top image obtained by performing coordinate conversion for performing a bird's-eye view from above the vehicle on an image captured by the front camera 11 that continuously captures a front area of the vehicle, A rear top image obtained by performing coordinate transformation for bird's-eye view from above the vehicle on the captured image by the rear camera 12 that continuously captures the rear region is acquired (S30).

Further, the image processing device 4 calculates the moving distance of the vehicle using the vehicle speed information, the yaw rate information, the shift position information, and the tire parameter information related to the outer diameter of the tire stored in the storage unit 34 (S80).

Further, when the vehicle is moving forward, the image processing device 4 uses the past front top image stored in the storage unit 34 as a history image, and records the history based on the movement distance in the rear top image acquired at the present time. A top view image for display is created by synthesizing the images (S100, S120). Further, when the vehicle is moving backward, the image processing device 4 uses the past rear top image stored in the storage unit 34 as a history image, and records the history based on the moving distance in the front top image acquired at the present time. A top-view image for display is created by combining the images (S110, S120).

Then, the image processing device 4 stores the tire parameter information stored in the storage unit 34 so that the deviation between the history image synthesized based on the moving distance and the front top image or the rear top image acquired at the present time is reduced. Is updated (S250 to S360).

In this way, the image processing device 4 reduces the deviation between the other image not used for the history image among the front top image and the rear top image, and the history image synthesized based on the calculated movement distance. Thus, the tire parameter information can be updated. Therefore, even when the outer diameter of the tire changes for some reason, the image processing apparatus 4 determines that the travel distance calculated using the tire rotation angle specified by the vehicle speed information and the tire parameter information is the actual vehicle distance. Occurrence of a situation that is different from the moving distance can be suppressed.

For this reason, the image processing apparatus 4 can smoothly connect the photographed image and the history image even when the top image for display is generated by synthesizing the history image with the photographed image. As a result, the image processing apparatus 4 can realize generation of a display top view image with reduced discomfort.

For example, when the actual vehicle tire outer diameter is larger than the vehicle tire outer diameter indicated by the tire parameter information, for example, as shown in FIG. 5, both the rear display region R2 and the intermediate display region R3 are the same. The pylons PL2 and PL3 are displayed, and a deviation occurs in the traveling direction DM of the vehicle near the boundary between the rear display area R2 and the intermediate display area R3. This is because the actual travel distance of the host vehicle is longer than the travel distance calculated by the image processing device 4, and thereby the distance DC3 recognized by the image processing device 4 with respect to the distance between the host vehicle and the pylon. This is because the actual distance DC2 becomes longer.

Conversely, when the actual vehicle tire outer diameter is smaller than the vehicle tire outer diameter indicated by the tire parameter information, the actual travel distance of the host vehicle is shorter than the travel distance calculated by the image processing device 4. Become. For this reason, for example, as shown in FIG. 6, the distance DC2A recognized by the image processing device 4 with respect to the distance between the host vehicle and the pylon (actually, the pylon PL2A in FIG. 6 is not displayed in the rear display region R2). The distance DC2 (the pylon PL2 in FIG. 6 is displayed in the rear display area R2) is shorter. As a result, deviation occurs along the traveling direction DM of the vehicle near the boundary between the rear display region R2 and the intermediate display region R3.

Further, the image processing apparatus 4 detects a preset measurement object from each of the plurality of front top images and the plurality of rear top images (S250 to S270, S290 to S310).

Furthermore, the image processing device 4 measures the time difference between the times when the front top image and the rear top image in which the common measurement object is captured are taken, and further determines the travel distance traveled by the vehicle within the time difference. Measurement is performed based on a common measurement object appearing in the top image (S280, S290, S330, S340).

Then, the image processing device 4 calculates the vehicle tire outer diameter for update based on the measured time difference and travel distance and the tire rotation angle rotated within the time difference (S350).

For this reason, the image processing apparatus 4 can measure the vehicle tire outer diameter using the in-vehicle camera group 2 necessary for creating the display top view image. That is, the image processing device 4 does not need to add a new measuring device for measuring the vehicle tire outer diameter, and can simplify the configuration of the image processing device 4. Moreover, since the image processing device 4 compares the photographed image of the front camera 11 and the photographed image of the rear camera 12, the vehicle tire outer diameter can be measured based on the result of rotating the tire a plurality of times. It is possible to accurately detect a change in the circumference along with a change in the tire outer diameter.

Further, the image processing device 4 determines whether or not the vehicle is in a straight traveling state (S230), and performs a process of updating the tire parameter information when it is determined that the vehicle is traveling straight ahead. As a result, the image processing device 4 calculates the value indicated by the tire parameter information only when traveling straight ahead, and can update the tire parameter information while substantially ignoring the lateral displacement of the vehicle. The accuracy of information can be improved.

Further, the image processing device 4 determines whether or not the traveling speed of the vehicle is equal to or lower than a preset processing start determination speed (S220), and when it is determined that the traveling speed of the vehicle is equal to or lower than the processing start determination speed, Processing to update tire parameter information is performed. As a result, the image processing apparatus 4 calculates the value indicated by the tire parameter information only at a low speed, can reduce the influence of slip generated on the tire, and can easily ensure the accuracy of the tire parameter information. .

In the embodiment described above, the front camera 11 and the rear camera 12 are the imaging devices in the present disclosure, the processing in S30 is the peripheral image acquisition unit in the present disclosure, the processing in S80 is the calculation unit in the present disclosure, and the processing in S100, S110, and S120. Is a composite image creation unit in the present disclosure, and the processes in S250 to S360 are a tire parameter update unit in the present disclosure.

The processing of S250 to S270 and S290 to S310 is the measurement object detection unit in the present disclosure, the processing of S280, S290, S330, and S340 is the measurement unit in the present disclosure, the processing of S230 is the straight traveling determination unit in the present disclosure, and the processing of S220. The process is a vehicle speed determination unit in the present disclosure.

(Second Embodiment)
Hereinafter, a second embodiment of the present disclosure will be described with reference to the drawings. In the second embodiment, parts different from the first embodiment will be described.

The vehicle display system 1 of the second embodiment is the same as that of the first embodiment except that the tire parameter update process is changed.

The tire parameter update process of the second embodiment is the same as that of the first embodiment except that the processes of S270 to S290 and S320 to S360 are omitted and the processes of S410 to S490 are added.

That is, as shown in FIG. 7, when a measurement object is detected (S260: YES), the moving distance and moving direction of the host vehicle are calculated in S410 in the same manner as in S80. In S420, a position comparison image indicating the rear display region R2 and the intermediate display region R3 is created using the front top image stored in the storage unit 34, and the created position comparison image is stored in the storage unit 34. , The process proceeds to S430.

In S420, based on the movement distance and movement direction calculated in S410, the front top image capturing the rear display area R2 and the intermediate display area R3 is extracted from the storage unit 34. Further, in S420, a position comparison image is created by performing coordinate conversion in which the extracted front top image is translated or rotated in accordance with the difference between the extracted front top image and the intermediate region.

Next, in S430, by performing an image recognition process using the latest position comparison image stored in the position comparison image storage area of the storage unit 34, the measurement object shown in the position comparison image is obtained. To detect.

Thereafter, in S440, the vehicle tire outer diameter is calculated, and the process proceeds to S490. Specifically, first, by using the distance between the measurement object of the rear top image detected in S250 and the measurement object of the position comparison image detected in S430, the change amount of the vehicle tire outer diameter is calculated. calculate. For example, as shown in FIG. 8, when the measurement object PL2 displayed in the rear display area R2 and the measurement object PL3 displayed in the intermediate display area R3 are the same measurement object, measurement is performed. The amount of change in the outer diameter of the vehicle tire is calculated using the distance DC between the object PL2 for measurement and the object PL3 for measurement. Then, the calculated change amount is added to or subtracted from the value of the tire parameter information stored in the storage unit 34, and this added value or subtracted value is set as a new own vehicle tire outer diameter.

As shown in FIG. 7, when a measurement object is detected in S310 (S310: YES), the moving distance and moving direction of the host vehicle are calculated in S450 in the same manner as in S80. In S460, using the rear top image stored in the storage unit 34, a position comparison image indicating the front display region R1 and the intermediate display region R3 is created, and the created position comparison image is stored in the storage unit 34. After the image is stored in the position comparison image storage area provided in step S470, the process proceeds to S470.

In S470, based on the movement distance and movement direction calculated in S450, the rear top image capturing the front display area R1 and the intermediate display area R3 is extracted from the storage unit 34. Further, in S470, a position comparison image is created by performing coordinate conversion in which the extracted rear top image is translated or rotated in accordance with the difference between the extracted rear top image and the intermediate region.

Next, in S470, in the same manner as in S430, image recognition processing is performed using the latest position comparison image stored in the position comparison image storage area of the storage unit 34, so that the image is captured in the position comparison image. The measuring object is detected.

Thereafter, in S480, using the distance between the measurement object of the front top image detected in S300 and the measurement object of the position comparison image detected in S470, the vehicle tire outer diameter is determined in the same manner as in S440. Calculate and shift to S490.

And if it transfers to S490, the value of the tire parameter information memorize | stored in the memory | storage part 34 will be updated to the value calculated by S440 or S480, and a tire parameter update process will be complete | finished.

The image processing apparatus 4 configured as described above selects a measurement object set in advance from each of the front top image or the rear top image acquired at the present time and the position comparison image stored in the storage unit 34. It detects (S250, S420, S300, S470). Then, the image processing device 4 updates the tire parameter information so that the deviation between the measurement object shown in the front top image or the rear top image acquired at the present time and the measurement object shown in the position comparison image is reduced. (S440, S480, S490).

For this reason, the image processing apparatus 4 can measure the vehicle tire outer diameter using the in-vehicle camera group 2 necessary for creating the display top view image. That is, the image processing device 4 does not need to add a new measuring device for measuring the vehicle tire outer diameter, and can simplify the configuration of the image processing device 4.

In the embodiment described above, the processing of S250, S420, S300, and S470 is a measurement object detection unit in the present disclosure, and the processing of S440, S480, and S490 is a tire parameter update unit in the present disclosure.

(Third embodiment)
Hereinafter, a third embodiment of the present disclosure will be described with reference to the drawings. In the third embodiment, parts different from the first embodiment will be described.

The vehicular display system 1 of the third embodiment is the same as that of the first embodiment except that the tire parameter update process is changed.

The tire parameter update process of the third embodiment is a process that is repeatedly executed at preset execution cycles during the operation of the image processing apparatus 4.

When this tire parameter update process is executed, the control unit 35 first determines in S610 whether or not a preset process start condition is satisfied, as shown in FIG. The processing start condition of the present embodiment is that a tire parameter update button provided in the passenger compartment is pushed down.

Here, when the process start condition is not satisfied (S610: NO), the tire parameter update process is temporarily ended. On the other hand, if the processing start condition is satisfied (S610: YES), it is determined in S620 whether the shift position of the host vehicle is P or N in the same manner as S70. If the shift position is P or N (S620: YES), the process of S620 is repeated.

If the shift position is neither P nor N at S620 (S620: NO), the travel distance of the host vehicle is calculated at S630 in the same manner as S80.

In S640, it is determined whether or not the shift position of the host vehicle is D in the same manner as S90. Here, when the shift position is D (S640: YES), an intermediate top image is created using the front top image stored in the storage unit 34 in S650 in the same manner as S100, and created. After storing the intermediate top image in the intermediate top image storage area provided in the storage unit 34, the process proceeds to S670.

On the other hand, if the shift position of the host vehicle is not D (S640: NO), it is determined that the shift position of the host vehicle is R, and is stored in the storage unit 34 in S660 similarly to S110. After the intermediate top image is created using the rear top image and the created intermediate top image is stored in the intermediate top image storage area, the process proceeds to S670.

In S670, based on the acquired vehicle speed signal, it is determined whether or not the host vehicle is traveling at a constant speed from the present time to a preset speed determination period. If the host vehicle is not traveling at a constant speed (S670: NO), the process proceeds to S620.

On the other hand, when the host vehicle is traveling at a constant speed (S670: YES), in S680, a plurality of front top images and a plurality of rear tops acquired in S30 between the current time and before the speed determination period. Optical flow is detected from the image.

Then, in S690, using the optical flow detected in S680, the velocity vector of the moving object (see moving objects MO1, MO2, MO3, MO4, and MO5 in FIG. 10) shown in the front top image and the rear top image (see FIG. 10). Vector V1, V2, V3, V4, V5).

Next, in S700, an optical flow is detected from a plurality of intermediate top images created in S650 or S660 between the current time and before the speed determination period.

In S710, the velocity vector (see vectors V6 and V7 in FIG. 10) of the moving object (see moving objects MO6 and MO7 in FIG. 10) in the intermediate top image is calculated using the optical flow detected in S700. To do.

Thereafter, in S720, based on the number and direction of the velocity vectors obtained in the optical flow detected in S680 and S700, it is determined whether or not the velocity vector calculated in S690 and S710 is valid. If the velocity vector is not valid (S720: NO), the process proceeds to S620.

On the other hand, if the speed vector is valid (S720: YES), the vehicle tire outer diameter X is calculated by the following equation (1) using the speed vector calculated in S690 and S710 in S730. . In the following formula (1), L is the vehicle tire outer diameter indicated by the tire parameter information stored in the storage unit 34. v is the magnitude of the velocity vector calculated in S690. va is the magnitude of the velocity vector calculated in S710.

X = L × v / va (Formula 1)
In S740, the value of the tire parameter information stored in the storage unit 34 is updated to the value of the vehicle tire outer diameter X calculated in S730, and the tire parameter update process is terminated.

The image processing apparatus 4 configured as described above detects the moving speed of the moving object (hereinafter referred to as the first moving speed) in the front top image and the back top image (S680, S690). Further, the image processing device 4 detects the moving speed of the moving object (hereinafter referred to as the second moving speed) in the intermediate top image (S700, S710). Then, the image processing device 4 calculates the vehicle tire outer diameter for update based on the first movement speed and the second movement speed (S730).

For this reason, the image processing device 4 moves the moving speed (first moving speed) of the object that actually moves in the front top image and the rear top image, and the moving speed (the moving speed of the object that actually moves in the intermediate top image). The tire parameter information can be updated so that the second movement speed matches.

Thereby, the image processing apparatus 4 can smoothly connect the front top image and the rear top image to be actually combined with the intermediate top image, and can generate a display top view image with reduced discomfort.

In the embodiment described above, the processes of S680 and S690 are the first movement speed detection unit in the present disclosure, the processes of S700 and S710 are the second movement speed detection unit in the present disclosure, and the processes of S730 and S740 are the tire parameters in the present disclosure. It is an update unit.

As mentioned above, although one embodiment of this indication was explained, this indication is not limited to the above-mentioned embodiment, and can take various forms, as long as it belongs to the technical scope of this indication.

(Modification 1)
For example, in the above-described embodiment, the display top view image is displayed on the display screen of the display device 5 when the vehicle speed is equal to or lower than the low-speed traveling determination speed. However, the display top view image is displayed regardless of the vehicle speed. You may make it do.

(Modification 2)
In the above embodiment, the tire parameter information indicates the tire outer diameter. However, the tire parameter information is not limited to this as long as the tire outer diameter is information that can specify the tire outer diameter. It may be a value indicating the circumference or a correction value for correcting the outer diameter or circumference as a reference.

(Modification 3)
Moreover, in the said embodiment, what calculated the outer diameter of the tire of the own vehicle based on the time difference between the imaging timing of the front top image which detected the measurement object, and the imaging timing of the back top image was shown. However, the outer diameter of the tire of the host vehicle may be calculated using a luminance change pattern instead of the measurement object of the front top image and the rear top image. For example, as shown in FIG. 11A, it is assumed that a luminance change pattern PB1 in which the luminance changes along the traveling direction DM of the host vehicle is detected in the front top image. The luminance change pattern PB1 is generated by, for example, a broken line white line WL (roadway center line) that divides a road lane. Thereafter, as shown in FIG. 11B, when a luminance change pattern PB2 having the same waveform as the luminance change pattern PB1 is detected in the rear top image, the time difference between the shooting time of the front top image and the shooting time of the rear top image. Based on the vehicle speed of the host vehicle, the outer diameter of the tire of the host vehicle can be calculated.

Specifically, first, the time difference (front-rear time difference) between the shooting timing of the front top image that detects the luminance change pattern PB1 and the shooting timing of the rear top image that detects the luminance change pattern PB2, and the vehicle speed signal, An angle (front / rear tire rotation angle) by which the tire of the host vehicle is rotated by the time difference between the front and rear is calculated.

Further, based on the position where the luminance change pattern PB1 is reflected in the front top image and the position where the luminance change pattern PB2 is reflected in the rear top image, the distance traveled by the host vehicle within the time difference (front and rear traveling). Distance). Then, the vehicle tire outer diameter is calculated based on the front-rear travel distance and the front-rear tire rotation angle. Specifically, based on the front and rear running distance and the front and rear tire rotation angle, the distance traveled while the tire rotates 360 ° is calculated as the tire circumference, and the tire circumference is divided by π. The vehicle tire outer diameter.

(Modification 4)
Moreover, in the said embodiment, what used as a process start condition that the own vehicle drive | worked the process start determination distance set beforehand was shown. However, the processing start condition may be that tire replacement is detected. Further, the processing start condition may be that the tire air pressure fluctuates. For example, when refueling is detected, it may be determined that the tire air pressure has increased. Further, the process start condition may be that a preset process start determination time elapses from the previous establishment of the process start condition. Further, the process start condition may be that the driver performs a process start operation that is set in advance to start the tire parameter update process (for example, operation of an operation button provided for calculating the tire outer diameter). .

Also, the functions of one component in the above embodiment may be distributed as a plurality of components, or the functions of a plurality of components may be integrated into one component. Moreover, you may abbreviate | omit a part of structure of the said embodiment. Further, at least a part of the configuration of the above embodiment may be added to or replaced with the configuration of the other embodiment. In addition, all the aspects included in the technical idea specified only by the wording described in the claims are embodiments of the present disclosure.

Although the present disclosure has been described based on the embodiments, it is understood that the present disclosure is not limited to the embodiments and structures. The present disclosure includes various modifications and modifications within the equivalent range. In addition, various combinations and forms, as well as other combinations and forms including only one element, more or less, are within the scope and spirit of the present disclosure.

Claims

An image processing device for a vehicle (4) that generates a vehicle peripheral image in which a periphery of the vehicle is viewed from above from a peripheral image captured by a plurality of photographing devices installed in the vehicle,
A front photographed image and a rear photographed image are acquired from each of a front photographing device (11) for continuously photographing the front region of the vehicle and a rear photographing device (12) for continuously photographing the rear region of the vehicle. A peripheral image acquisition unit (S30);
A calculation unit that calculates a moving distance of the vehicle using vehicle state information including information related to the rotation angle of the tire of the vehicle and parameter information related to the outer diameter of the tire stored in the storage unit (34). S80)
One moving image of the front photographed image and the rear photographed image showing the traveling direction of the vehicle is used as a history image, and the moving distance is used as the other image of the front photographed image and the rear photographed image. A synthesized image creation unit (S100, S110, S120) for creating the vehicle peripheral image by synthesizing the history image based on
A tire parameter updating unit (S250 to S360, which updates the parameter information stored in the storage unit so as to reduce a deviation between the history image synthesized based on the moving distance and the other image. S440, S480, S490, S730, S740).
A measurement object detection unit (S250 to S270, S290) that detects a predetermined measurement object from each of the plurality of front photographed images and the plurality of rear photographed images that are repeatedly acquired by the peripheral image acquisition unit. To S310),
The time difference between the times when the front photographed image and the rear photographed image in which the common measurement object is photographed is measured, and the travel distance traveled by the vehicle within the time difference is further measured. A measurement unit (S280, S290, S330, S340) for measuring based on the common measurement object shown in the image,
The tire parameter update unit (S250 to S360) calculates the parameter information for update based on the time difference and the travel distance measured by the measurement unit, and the rotation angle of the tire rotated within the time difference. The vehicle image processing device according to claim 1.
A measurement object detection unit (S250,) that detects a preset measurement object from each of the other image repeatedly acquired by the peripheral image acquisition unit and the history image stored in the storage unit. S420, S300, S470),
The tire parameter update unit (S440, S480, S490) updates the parameter information so that a deviation between the measurement object shown in the other image and the measurement object shown in the history image is reduced. The vehicle image processing device according to claim 1.
The vehicular image processing apparatus according to claim 3, wherein the storage unit includes a storage area for storing the history image obtained by photographing a range shown in the other image.
A straight traveling determination unit (S230) for determining whether or not the vehicle is in a straight traveling state;
The vehicle image processing device according to any one of claims 1 to 4, wherein the tire parameter update unit performs a process of updating the parameter information when it is determined that the vehicle is traveling straight by the straight traveling determination unit. .
A vehicle speed determination unit (S220) for determining whether the traveling speed of the vehicle is equal to or lower than a predetermined determination speed;
The tire parameter update unit performs a process of updating the parameter information when the vehicle speed determination unit determines that the traveling speed of the vehicle is equal to or lower than the determination speed. The image processing apparatus for vehicles described in 2.
A first movement speed detector (S680, S690) that detects a first movement speed that is a movement speed of an object that moves in the other image;
A second movement speed detector (S700, S710) that detects a second movement speed that is a movement speed of an object moving in the history image,
The vehicular image processing apparatus according to claim 1, wherein the tire parameter updating unit (S730, S740) calculates the parameter information for updating based on the first moving speed and the second moving speed.