WO2020079831A1

WO2020079831A1 - Rear image processing device and rear image display method

Info

Publication number: WO2020079831A1
Application number: PCT/JP2018/039010
Authority: WO
Inventors: 中村　好孝; 下谷　光生
Original assignee: 三菱電機株式会社
Priority date: 2018-10-19
Filing date: 2018-10-19
Publication date: 2020-04-23
Also published as: JP6833128B2; JPWO2020079831A1

Abstract

A rear image processing device (10) comprises: a rear image acquiring unit (11) which acquires a rear image which is an image depicting an environment to the rear of a vehicle; a distance acquiring unit (12) which acquires the distance from the vehicle to an object shown in the rear image; an image processing unit (13) which executes a screening process on the rear image; and a display processing unit (14) which displays the rear image after the screening process on a display device. The image processing unit (13), when the distance from the vehicle to the object is greater than a pre-determined threshold value, sets a screening coefficient to a fixed reference value and executes the screening process on the rear image, and, when the distance from the vehicle to the object is equal to or less than the threshold value, changes the screening coefficient such that the rear image reduces in size as the distance decreases, and executes the screening process on the rear image.

Description

Rear image processing device and rear image display method

The present invention relates to a rear image processing device that causes a display device to display an image of the rear of a vehicle.

In recent years, as an alternative to a physical mirror (optical mirror) of a vehicle, a camera monitoring system (CMS) that displays an image of the rear or side of the vehicle on a display device inside the vehicle, a so-called "electronic mirror" has been developed. There is. In addition, when displaying an image of the rear of the vehicle (rear image) captured by the camera for the electronic mirror on the screen of the electronic mirror, the rear image is enlarged according to the distance from the vehicle to the rear vehicle (following vehicle) or There is known a technique for suppressing a difference in sense of distance between an image of a rear vehicle displayed by an electronic mirror and a mirror image of the rear vehicle displayed on a physical mirror by performing a scaling process for reducing (for example, Patent Document below. 1).

JP, 2014-235640, A

In the technique of Patent Document 1, the scaling coefficient (enlargement ratio or reduction ratio) in the scaling process of the rear image is set according to the distance from the own vehicle to the rear vehicle. However, when the distance from the own vehicle to the rear vehicle is long, the influence of the distance on the sense of distance in the rear image is small. Therefore, when the technique of Patent Document 1 is applied even when the distance from the own vehicle to the rear vehicle is long (including the case where no rear vehicle exists), the image of the rear vehicle displayed by the electronic mirror and the rear image reflected on the physical mirror are displayed. It is possible that the difference in the sense of distance from the mirror image of the vehicle is amplified.

The present invention has been made to solve the above problems, and even when the distance from the own vehicle to the vehicle behind is long, the sense of distance between the image displayed by the electronic mirror and the mirror image displayed on the physical mirror is sensed. An object of the present invention is to provide an image processing device capable of suppressing the difference between

A rear image processing apparatus according to the present invention, a rear image acquisition unit that acquires a rear image that is an image of a landscape behind the vehicle, a distance acquisition unit that acquires the distance from the vehicle to the object in the rear image, When the distance from the vehicle to the object is larger than a predetermined threshold value, the scaling coefficient is set to a constant standard value to perform the scaling processing of the rear image. When the distance is less than or equal to the threshold value, the shorter the distance, the rearward. The image processing unit includes an image processing unit that performs scaling processing on the rear image by changing the scaling coefficient so that the image is reduced, and a display processing unit that displays the rear image after the scaling processing on the display device.

According to the rear image processing device of the present invention, when the distance from the vehicle to the object is equal to or less than the threshold value, the rear image is reduced as the distance becomes shorter, so that the sense of distance of the object displayed on the display device is reduced. It is equivalent to the sense of distance of the object reflected on the physical mirror. Further, when the distance from the vehicle to the object is larger than the threshold value, the scaling factor is set to a constant standard value so that the sense of distance of the object displayed on the display device is the same as that of the object reflected on the physical mirror. Maintained in the same condition as.

The objects, features, aspects and advantages of the present invention will become more apparent by the following detailed description and the accompanying drawings.

FIG. 3 is a block diagram showing the configuration of the electronic mirror system according to the first embodiment. FIG. 5 is a diagram for explaining a scaling coefficient used in the scaling process of the first embodiment. FIG. 5 is a diagram showing an example of a rear image displayed on the display device of the electronic mirror system according to the first embodiment and a mirror image reflected on a rear view mirror when the distance from the vehicle to the rear object is larger than a threshold value. It is a figure which shows the example of the mirror image reflected on a rear view mirror, when the distance from an own vehicle to a back object is smaller than a threshold value. It is a figure which shows the example of the back image displayed on the display apparatus of an electronic mirror system, when a scaling coefficient is set to a standard value, when the distance from a own vehicle to a back object is smaller than a threshold value. FIG. 5 is a diagram showing an example of a rear image displayed on the display device of the electronic mirror system according to the first embodiment when the distance from the own vehicle to the rear object is smaller than a threshold value. 3 is a flowchart showing the operation of the backward image processing apparatus according to the first embodiment. It is a figure which shows the example of the hardware constitutions of a back image processing apparatus. It is a figure which shows the example of the hardware constitutions of a back image processing apparatus. FIG. 9 is a diagram for explaining a scaling coefficient used in the scaling process of the second embodiment. FIG. 9 is a diagram for explaining a scaling coefficient used in the scaling process of the second embodiment. FIG. 8 is a diagram showing an example of a rear image displayed on the display device of the electronic mirror system according to the second embodiment when the distance from the own vehicle to the rear object is smaller than a threshold value. 9 is a flowchart showing the operation of the backward image processing apparatus according to the second embodiment. FIG. 9 is a diagram for explaining a problem that may occur in the backward image processing device according to the second embodiment. FIG. 9 is a block diagram showing a configuration of an electronic mirror system according to a third embodiment. FIG. 16 is a diagram for explaining a scaling coefficient used in the scaling process of the third embodiment. FIG. 16 is a diagram for explaining a scaling coefficient used in the scaling process of the third embodiment. FIG. 16 is a diagram for explaining a scaling coefficient used in the scaling process of the third embodiment. 9 is a flowchart showing the operation of the backward image processing device according to the third embodiment. FIG. 9 is a diagram for explaining a problem that may occur in the backward image processing device according to the second embodiment. FIG. 9 is a diagram for explaining a problem that may occur in the backward image processing device according to the second embodiment. FIG. 17 is a diagram for explaining a scaling coefficient used in the scaling process of the fourth embodiment. FIG. 17 is a diagram for explaining a scaling coefficient used in the scaling process of the fourth embodiment. FIG. 17 is a diagram for explaining a scaling coefficient used in the scaling process of the fourth embodiment. 9 is a flowchart showing the operation of the backward image processing device according to the fourth embodiment. FIG. 16 is a diagram showing an example of a rear image displayed on the display device of the electronic mirror system according to the fourth embodiment when the distance from the own vehicle to the rear object is smaller than a threshold value.

<Embodiment 1>
FIG. 1 is a block diagram showing the configuration of the electronic mirror system according to the first embodiment. The electronic mirror system according to the first embodiment provides an electronic mirror that is an alternative to a rear view mirror (also called a rearview mirror or a rearview mirror) of a vehicle. Hereinafter, a vehicle equipped with the electronic mirror system is referred to as "own vehicle".

As shown in FIG. 1, the electronic mirror system according to the first embodiment includes a rear image processing device 10, a rear image capturing device 21, a display device 22 and a distance measuring sensor 23 connected to the rear image processing device 10.

The rear image capturing device 21 is a camera that captures a landscape behind the vehicle. The display device 22 is for displaying an image output by the rear image processing device 10, and is configured by, for example, a liquid crystal display device.

The distance measuring sensor 23 is a sensor that measures a distance from the own vehicle to an object existing behind the own vehicle, and includes, for example, an infrared sensor, a millimeter wave radar, an ultrasonic radar, LiDAR (Light Detection and and Ranging), sonar. Composed of etc. The objects detected by the distance measuring sensor 23 include moving bodies such as vehicles, motorcycles, bicycles, and pedestrians, as well as stationary objects such as guardrails, walls, buildings, and utility poles. Hereinafter, an object existing behind the own vehicle is referred to as a "rear object". It should be noted that the distance measuring sensor 23 only needs to be able to measure at least the distance between the rear object existing within the photographing range of the rear image photographing device 21 and the vehicle.

The rear image processing apparatus 10 performs scaling processing for enlarging or reducing an image (hereinafter referred to as “rear image”) behind the vehicle captured by the rear image capturing apparatus 21, and displays the rear image after the scaling processing on the display device 22. Let The scaling coefficient (enlargement ratio or reduction ratio) used in the scaling process performed by the rear image processing device 10 is set based on the distance measured by the distance measuring sensor 23 from the own vehicle to the rear object.

As shown in FIG. 1, the backward image processing device 10 includes a backward image acquisition unit 11, a distance acquisition unit 12, an image processing unit 13, and a display processing unit 14.

The rear image acquisition unit 11 acquires the rear image captured by the rear image capturing device 21. The distance acquisition unit 12 acquires the distance measured by the distance measuring sensor 23 from the host vehicle to a rear object shown in the rear image (that is, a rear object existing within the photographing range of the rear image photographing device 21).

The image processing unit 13 performs scaling processing of the rear image acquired by the rear image acquisition unit 11. The display processing unit 14 trims a range according to the size of the screen of the display device 22 from the rear image after the scaling process, and displays the rear image in the trimmed range on the display device 22. The image processing unit 13 may perform the above trimming processing.

Here, the scaling process performed by the image processing unit 13 will be described. The scaling coefficient used in this scaling process is set based on the distance from the host vehicle to the rear object. Specifically, when the distance from the host vehicle to the rear object is larger than a predetermined threshold value, the image processing unit 13 sets the scaling coefficient to a constant standard value. When the distance from the host vehicle to the rear object is equal to or less than the threshold, the image processing unit 13 changes the scaling coefficient so that the rear image is reduced (zoomed out) as the distance becomes shorter. That is, the relationship between the distance from the host vehicle to the rear object and the scaling coefficient is shown in a graph as shown in FIG. The scaling factor corresponding to the vertical axis of the graph of FIG. 2 represents the enlargement ratio, and the rear image is reduced as the scaling factor decreases. In addition, in the first embodiment, the scaling coefficient is set to a uniform value over the entire rear image.

The image processing unit 13 may calculate the scaling coefficient by a calculation process based on the distance from the own vehicle to the rear object, or the scaling table in which the correspondence relationship between the distance from the own vehicle to the rear object and the scaling coefficient is described. (Data corresponding to the graph of FIG. 2) may be stored in advance.

The standard value of the scaling coefficient is that the field of view of the rear image displayed on the display device 22 of the electronic mirror system (the range reflected in the rear image) is equivalent to the field of view of the rear-view mirror as the physical mirror (range reflected in the rear-view mirror). Is determined to be. For example, when a mirror image of the field of view as shown in FIG. 3 is displayed on the rear view mirror, the standard value of the scaling coefficient is set so that the display device 22 of the electronic mirror system displays a rear image equivalent to that shown in FIG.

Therefore, when the distance from the own vehicle to the rear object is larger than the threshold value and the scaling coefficient is set to the standard value, both the rear image displayed on the display device 22 and the mirror image reflected on the rear view mirror are It is equivalent to 3. Therefore, there is no difference between the sense of distance of the rear object displayed on the display device 22 and the sense of distance of the rear object reflected on the rear view mirror.

In addition, when the distance from the host vehicle to the rear object is larger than the threshold value, the scaling coefficient is set to a constant standard value regardless of the value of the distance, so that the rear object displayed on the display device 22 is displayed. The sense of distance is maintained in the same state as the sense of distance of the rear object reflected on the rear view mirror.

On the other hand, when the distance from the host vehicle to the rear object is short, if the scaling coefficient is made constant, there is a difference between the distance feeling of the rear object displayed on the display device 22 and the distance feeling of the rear object reflected on the rear view mirror. Occurs. While the apparent distance from the driver to the rear object reflected on the rear view mirror is the distance from the driver's eyes to the rear object via the rear view mirror near the front window, the driver's electronic mirror system Since the apparent distance to the rear image displayed on the display device 22 corresponds to the distance between the rear image capturing device 21 and the rear object, there is a difference between the two and the distance from the own vehicle to the rear object is large. This is because when the distance is short, the difference greatly affects the sense of distance. Particularly, in an electronic mirror system that is an alternative to the rear-view mirror, the rear image capturing device 21 is installed in the rear part of the own vehicle in order to reduce the blind spot, so the difference is about 3 m.

Therefore, even when the distance from the host vehicle to the rear object is short, if the scaling coefficient is set to the standard value, for example, when a mirror image as shown in FIG. The rear image is displayed, and the sense of distance of the rear object (here, the rear vehicle) is different. That is, the rear object displayed on the display device 22 looks larger than the rear object reflected on the rear view mirror (occupies a larger proportion in the visual field). Therefore, the rear object displayed on the display device 22 appears to be closer than the rear object reflected in the rear view mirror.

On the other hand, in the electronic mirror system according to the first embodiment, as shown in FIG. 4, when the distance from the vehicle to the rear object becomes equal to or less than the threshold value, the image processing unit 13 reduces the rear image as the distance becomes shorter. Change the scaling factor as described above. Therefore, on the display device 22, the rear image as shown in FIG. 6 obtained by reducing the rear image of FIG. 5 is displayed. Since the rear object in the rear image of FIG. 6 has the same size as the rear object (FIG. 4) reflected in the rear view mirror, the difference in sense of distance between the two is suppressed.

As described above, according to the electronic mirror system of the first embodiment, when the distance from the host vehicle to the rear object is equal to or less than the threshold value, the rear image is reduced as the distance is reduced, so that the display device 22 displays the rear image. The sense of distance of the rear object can be made equal to the sense of distance of the rear object reflected on the rear view mirror (physical mirror). When the distance from the host vehicle to the rear object is larger than the threshold value, the scaling coefficient is set to a constant standard value, so that the physical feeling of the rear object displayed on the display device 22 is reflected. The state equivalent to the sense of distance of an object can be maintained.

FIG. 7 is a flowchart showing the operation of the backward image processing device 10 according to the first embodiment. Hereinafter, the operation of the backward image processing apparatus 10 according to the first embodiment will be described with reference to FIG. 7.

When the rear image processing device 10 is activated, first, the rear image acquisition unit 11 acquires the rear image captured by the rear image capturing device 21 (step S101). Next, the distance acquisition unit 12 acquires the distance from the vehicle measured by the distance measuring sensor 23 to the rear object shown in the rear image (step S102). Then, the image processing unit 13 confirms whether or not the distance from the own vehicle to the rear object is equal to or less than a predetermined threshold value (step S103).

If the distance from the host vehicle to the rear object is equal to or less than the threshold value (YES in step S103), the image processing unit 13 sets the scaling coefficient to a value according to the distance (step S104). Specifically, the image processing unit 13 changes the scaling coefficient according to the distance such that the rear image is reduced as the distance from the own vehicle to the rear object is shortened. Then, the image processing unit 13 uses the scaling coefficient set in step S104 to perform scaling processing on the rear image (step S105).

On the other hand, if the distance from the host vehicle to the rear object is larger than the threshold value (NO in step S103), the image processing unit 13 sets the scaling coefficient to the standard value (step S106). Then, the image processing unit 13 uses the scaling coefficient set in step S106 to perform scaling processing on the rear image (step S107).

When the scaling process of step S105 or S107 is completed, the display processing unit 14 trims a range corresponding to the size of the screen of the display device 22 from the rear image after the scaling process, and the display device 22 displays the rear image in the trimmed range. Is displayed (step S108). The rear image processing apparatus 10 repeatedly executes the above processing.

8 and 9 are diagrams each showing an example of the hardware configuration of the backward image processing apparatus 10. Each function of the constituent elements of the backward image processing apparatus 10 shown in FIG. 1 is realized by the processing circuit 50 shown in FIG. 8, for example. That is, the rear image processing device 10 acquires a rear image that is an image of a landscape behind the host vehicle, acquires a distance from the host vehicle to a rear object captured in the rear image, and determines the distance from the host vehicle to the rear object. When the distance is larger than a predetermined threshold value, the scaling coefficient is set to a constant standard value to perform the scaling process of the rear image, and when the distance from the own vehicle to the rear object is equal to or less than the threshold value, the distance becomes shorter. A processing circuit 50 is provided for changing the scaling coefficient so as to reduce the rear image, performing the rear image scaling process, and displaying the scaled rear image on the display device. The processing circuit 50 may be dedicated hardware, or may be a processor that executes a program stored in a memory (Central Processing Unit (CPU), processing device, arithmetic device, microprocessor, microcomputer, It may be configured using a DSP (also called a Digital Signal Processor)).

When the processing circuit 50 is dedicated hardware, the processing circuit 50 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable). Gate Array), or a combination of these. Each function of the constituent elements of the rear image processing apparatus 10 may be realized by an individual processing circuit, or those functions may be collectively realized by one processing circuit.

FIG. 9 shows an example of the hardware configuration of the rear image processing apparatus 10 when the processing circuit 50 is configured by using the processor 51 that executes a program. In this case, the functions of the components of the rear image processing apparatus 10 are realized by software (software, firmware, or a combination of software and firmware). The software and the like are described as a program and stored in the memory 52. The processor 51 realizes the function of each unit by reading and executing the program stored in the memory 52. That is, the rear image processing device 10, when executed by the processor 51, acquires a rear image that is an image of a landscape behind the host vehicle, and a distance from the host vehicle to a rear object included in the rear image. And when the distance from the host vehicle to the rear object is larger than a predetermined threshold value, the scaling coefficient is set to a constant standard value to perform the rear image scaling process, from the host vehicle to the rear object. When the distance is less than or equal to the threshold value, the scaling process is performed such that the scaling factor is changed so that the backward image is reduced as the distance becomes shorter, and the processing for displaying the backward image after the scaling process on the display device. And a memory 52 for storing a program that will eventually be executed. In other words, it can be said that this program causes a computer to execute the procedure and method of operation of the components of the rear image processing apparatus 10.

Here, the memory 52 is, for example, a nonvolatile memory such as a RAM (RandomAccessMemory), a ROM (ReadOnlyMemory), a flash memory, an EPROM (ErasableProgrammableReadOnlyMemory), and an EEPROM (ElectricallyErasableProgrammableReadOnlyMemory). Volatile semiconductor memory, HDD (Hard Disk Drive), magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disc) and its drive device, or any storage medium used in the future. May be.

The configuration of the functions of the constituent elements of the rear image processing apparatus 10 is realized by either hardware or software as described above. However, the configuration is not limited to this, and a configuration may be adopted in which some of the constituent elements of the rear image processing apparatus 10 are realized by dedicated hardware, and some of the other constituent elements are realized by software or the like. For example, for some of the constituent elements, the function is realized by the processing circuit 50 as dedicated hardware, and for some of the other constituent elements, the processing circuit 50 as the processor 51 executes the program stored in the memory 52. The function can be realized by reading and executing.

As described above, the rear image processing apparatus 10 can realize each of the above-mentioned functions by hardware, software, etc., or a combination thereof.

<Second Embodiment>
In the first embodiment, when the distance from the host vehicle to the rear object is equal to or less than the threshold value, scaling processing for reducing the rear image is performed, and the size of the rear object displayed on the display device 22 (see FIG. 6) is changed to the rear view. By making the size of the rear object reflected on the mirror (physical mirror) (see FIG. 4), the difference in the sense of distance between the two was suppressed.

However, in the first embodiment, the scaling coefficient is set to a uniform value over the entire rear image, and the background image of the rear object is also reduced in the same manner as the rear object, so that it is displayed on the display device 22. There is a difference between the field of view of the rear image and the field of view of the physical mirror. As a result, it becomes difficult to grasp the relative size of the rear object with respect to the background feature. The second embodiment shows a rear image processing apparatus 10 that can solve this problem.

The configuration of the electronic mirror system according to the second embodiment is similar to that shown in FIG. However, in the second embodiment, the image processing unit 13 can set the scaling coefficient for each position in the rear image.

When the distance from the host vehicle to the rear object is larger than the threshold value, the image processing unit 13 sets the scaling coefficient uniformly to the standard value over the entire rear image, as in the first embodiment. On the other hand, when the distance from the host vehicle to the rear object is equal to or less than the threshold value, the image processing unit 13 determines the scaling coefficient from the host vehicle to the rear object in the central portion of the rear image as in the first embodiment (FIG. 2). Although it is changed according to the distance, in a portion other than the center of the rear image, the scaling coefficient is gradually increased from the center of the rear image to the outside, as shown in FIG. Is set to a value close to the standard value.

That is, in the second embodiment, when the distance from the host vehicle to the rear object is equal to or less than the threshold value, the scaling coefficient (SF) is smaller than the standard value in the central portion of the rear image, as shown in FIG. It becomes a value close to the standard value in the outer peripheral part of. Further, as can be seen from FIG. 11, the scaling coefficient greatly changes at a specific position in the rear image. That is, the scaling coefficient changes non-linearly with respect to the distance from the center of the rear image. Hereinafter, scaling processing performed using a scaling coefficient that changes non-linearly on the rear image is referred to as "non-linear scaling processing".

For example, when the non-linear scaling process using the scaling coefficient as shown in FIG. 11 is performed on the back image shown in FIG. 5, the back image after the non-linear scaling process becomes as shown in FIG. In such a non-linear scaling process, the image of the rear object in the central portion of the rear image is reduced, so that the sense of distance of the rear object in the rear image after the non-linear scaling process becomes equivalent to the sense of distance of the physical mirror. Furthermore, since the scaling coefficient of the background image appearing on the outer periphery of the rear image is close to the standard value, reduction of the background image is suppressed, and the field of view of the rear image after the non-linear scaling processing becomes equivalent to the field of view of the physical mirror. As a result, compared to the first embodiment, in the rear image displayed on the display device 22, it becomes easier to understand the relative size of the rear object with respect to the background feature, and the above problem is solved.

FIG. 13 is a flowchart showing the operation of the backward image processing apparatus 10 according to the second embodiment. Hereinafter, the operation of the backward image processing apparatus 10 according to the second embodiment will be described with reference to FIG.

When the rear image processing device 10 is activated, first, the rear image acquisition unit 11 acquires the rear image captured by the rear image capturing device 21 (step S201). Next, the distance acquisition unit 12 acquires the distance from the host vehicle measured by the distance measuring sensor 23 to the rear object shown in the rear image (step S202). Then, the image processing unit 13 confirms whether or not the distance from the own vehicle to the rear object is equal to or less than a predetermined threshold value (step S203).

If the distance from the host vehicle to the rear object is equal to or less than the threshold value (YES in step S203), the image processing unit 13 sets the central scaling coefficient of the rear image to a value according to the distance (step S204). Specifically, the image processing unit 13 changes the scaling coefficient at the center of the rear image so that the rear image is reduced as the distance from the own vehicle to the rear object is shortened.

Subsequently, the image processing unit 13 sets the scaling coefficient at each position of the rear image (step S205). Specifically, the image processing unit 13 sets the scaling coefficient to a value closer to the standard value at positions farther from the center of the rear image.

Further, the image processing unit 13 performs the non-linear scaling process on the rear image using the scaling coefficient set in steps S204 and S205 (step S206).

On the other hand, if the distance from the host vehicle to the rear object is larger than the threshold value (NO in step S203), the image processing unit 13 sets the scaling coefficient of the entire rear image to the standard value (step S207). Then, the image processing unit 13 uses the scaling coefficient set in step S207 to perform scaling processing on the rear image (step S208).

When the scaling process of step S206 or S208 is completed, the display processing unit 14 trims a range corresponding to the screen size of the display device 22 from the rear image after the scaling process, and the display device 22 displays the rear image in the trimmed range. Is displayed (step S209). The rear image processing apparatus 10 repeatedly executes the above processing.

<Third Embodiment>
As shown in FIG. 11, in the second embodiment, the slope of the change of the scaling coefficient of the rear image becomes large at a specific position of the rear image. Further, when the distance from the host vehicle to the rear object becomes extremely short, the size of the rear object in the rear image becomes very large. At this time, if the position where the slope of the change of the scaling coefficient is large is inside the image of the rear object, only the image of the rear object is distorted as shown in FIG. 14, and the image of the rear object is not appropriately reduced. The third embodiment shows a rear image processing apparatus 10 that can solve this problem.

FIG. 15 is a block diagram showing the configuration of the electronic mirror system according to the third embodiment. The configuration of the electronic mirror system in FIG. 15 is obtained by adding an object image extraction unit 15 to the rear image processing apparatus 10 in addition to the configuration in FIG. The object image extraction unit 15 analyzes the rear image acquired by the rear image acquisition unit 11 and extracts an image of the rear object (hereinafter referred to as “rear object image”) from the rear image.

Further, in the third embodiment, when the distance from the host vehicle to the rear object is equal to or less than the threshold value, the image processing unit 13 determines the position where the slope of the change of the scaling coefficient becomes large as shown in FIG. It is changed according to the size of the rear object image extracted by 15. Specifically, the image processing unit 13 sets a position where the inclination of the change of the scaling coefficient becomes large in the peripheral portion of the rear object image. That is, when the size of the rear object image is small as shown in FIG. 17, the position where the slope of the change of the scaling coefficient becomes large is close to the center of the rear image, and when the size of the rear object image is large as shown in FIG. The position at which the slope of the change in the scaling coefficient becomes large moves away from the center of the rear image.

According to the third embodiment, it is possible to prevent the position where the slope of the change of the scaling coefficient is large from being inside the image of the rear object, and thus even if the size of the rear object shown in the rear image is very large. , The image of the rear object is appropriately reduced.

FIG. 19 is a flowchart showing the operation of the backward image processing apparatus 10 according to the third embodiment. The operation of the backward image processing apparatus 10 according to the third embodiment will be described below with reference to FIG.

When the rear image processing device 10 is activated, first, the rear image acquisition unit 11 acquires the rear image captured by the rear image capturing device 21 (step S301). Next, the distance acquisition unit 12 acquires the distance from the host vehicle measured by the distance measuring sensor 23 to the rear object shown in the rear image (step S302). Then, the image processing unit 13 confirms whether or not the distance from the own vehicle to the rear object is equal to or less than a predetermined threshold value (step S303).

If the distance from the vehicle to the rear object is equal to or less than the threshold value (YES in step S303), the image processing unit 13 sets the central scaling coefficient of the rear image to a value according to the distance (step S304). Specifically, the image processing unit 13 changes the scaling coefficient at the center of the rear image so that the rear image is reduced as the distance from the own vehicle to the rear object is shortened.

Subsequently, the object image extraction unit 15 extracts a rear object image from the rear image (step S305). The image processing unit 13 confirms the size of the rear object image extracted by the object image extraction unit 15, and sets the position where the inclination of the change of the scaling coefficient is increased according to the size (step S306). Specifically, the image processing unit 13 sets the position where the inclination of the change of the scaling coefficient is increased in the peripheral portion of the rear object image (outside the rear object image).

Further, the image processing unit 13 sets the scaling coefficient of each position of the rear image based on the processing results of steps S304 and S306 (step S307). Specifically, the image processing unit 13 sets the scaling coefficient to a value closer to the standard value at positions farther from the center of the rear image. Then, the image processing unit 13 uses the scaling coefficient set in steps S304 and S307 to perform nonlinear scaling processing on the rear image (step S308).

On the other hand, if the distance from the host vehicle to the rear object is larger than the threshold value (NO in step S303), the image processing unit 13 sets the scaling coefficient of the entire rear image to the standard value (step S309). Then, the image processing unit 13 uses the scaling coefficient set in step S309 to perform scaling processing on the rear image (step S310).

When the scaling process of step S308 or S310 is completed, the display processing unit 14 trims a range according to the size of the screen of the display device 22 from the rear image after the scaling process, and the display device 22 displays the rear image in the trimmed range. Is displayed (step S311). The rear image processing apparatus 10 repeatedly executes the above processing.

<Embodiment 4>
In the second embodiment, it is assumed that the rear object is reflected near the center of the rear image, but the rear object may be reflected at a position deviated from the center of the rear image as shown in FIG. 20, for example. . If the second embodiment is applied to the rear image as shown in FIG. 20 as it is, a part of the image of the rear object is reduced as shown in FIG. The fourth embodiment shows a rear image processing apparatus 10 that can solve this problem.

The configuration of the electronic mirror system according to the fourth embodiment is similar to that shown in FIG. That is, the rear image processing apparatus 10 according to the fourth embodiment includes the object image extraction unit 15 that extracts the rear object image from the rear image acquired by the rear image acquisition unit 11.

Further, in the fourth embodiment, when the distance from the host vehicle to the rear object is equal to or less than the threshold, the image processing unit 13 sets the scaling coefficient in the central portion of the rear object image as in the first embodiment (FIG. 2). Although it is changed according to the distance from the own vehicle to the rear object, the scaling coefficient is gradually increased from the center of the rear object image in the rear image to the outside as shown in FIG. Then, the scaling coefficient is set to a value closer to the standard value at a position farther from the rear object image. At this time, the image processing unit 13 changes the scaling coefficient in a non-linear manner with respect to the distance from the rear object image.

In the fourth embodiment, the position where the slope of the change in the scaling coefficient becomes large changes according to the position of the rear object image, as shown in FIGS. 23 and 24. For example, when the rear object image is located in the lower left part of the rear image as shown in FIG. 23, the position where the slope of the change in the scaling coefficient becomes large is when the rear object image is located in the center of the rear image (FIG. 11). Compared to the lower left. Further, when the rear object image is located in the upper right portion of the rear image as shown in FIG. 24, the position where the inclination of the change of the scaling coefficient becomes large is when the rear object image is located in the center of the rear image (FIG. 11). Compare to the upper right.

According to the fourth embodiment, the position where the slope of the change in the scaling coefficient is large moves in accordance with the position of the rear object image in the rear image, so that the rear object is displaced from the center of the rear image as shown in FIG. Even when the image is reflected in the image, the image of the rear object is appropriately reduced as shown in FIG.

FIG. 26 is a flowchart showing the operation of the backward image processing device 10 according to the fourth embodiment. Hereinafter, the operation of the backward image processing apparatus 10 according to the fourth embodiment will be described with reference to FIG.

When the rear image processing device 10 is activated, first, the rear image acquisition unit 11 acquires the rear image captured by the rear image capturing device 21 (step S401). Next, the distance acquisition unit 12 acquires the distance from the host vehicle measured by the distance measuring sensor 23 to the rear object shown in the rear image (step S402). Then, the image processing unit 13 confirms whether or not the distance from the own vehicle to the rear object is equal to or less than a predetermined threshold value (step S403).

If the distance from the vehicle to the rear object is equal to or less than the threshold value (YES in step S403), the object image extraction unit 15 extracts the rear object image from the rear image (step S404). Then, the image processing unit 13 confirms the position of the rear object image extracted by the object image extracting unit 15, and sets the scaling coefficient at the center of the rear object image in the rear image to a value according to the distance (step S405). .. Specifically, the image processing unit 13 changes the scaling coefficient at the center of the rear object image according to the distance such that the rear image is reduced as the distance from the own vehicle to the rear object becomes shorter.

The image processing unit 13 also sets a position at which the inclination of the change in the scaling coefficient is increased according to the position of the rear object image in the rear image (step S406). Specifically, the image processing unit 13 sets the position where the inclination of the change of the scaling coefficient is large in the peripheral portion of the rear object image.

Further, the image processing unit 13 sets the scaling coefficient at each position of the rear image based on the processing results of steps S405 and S406 (step S407). Specifically, the image processing unit 13 sets the scaling coefficient to a value closer to the standard value at a position farther from the rear object image. Then, the image processing unit 13 uses the scaling coefficient set in steps S405 and S407 to perform nonlinear scaling processing on the rear image (step S408).

On the other hand, if the distance from the host vehicle to the rear object is larger than the threshold value (NO in step S403), the image processing unit 13 sets the scaling coefficient of the entire rear image to the standard value (step S409). Then, the image processing unit 13 uses the scaling coefficient set in step S409 to perform scaling processing on the rear image (step S410).

When the scaling process of step S408 or S410 is completed, the display processing unit 14 trims a range corresponding to the screen size of the display device 22 from the rear image after the scaling process, and the display device 22 displays the rear image in the trimmed range. Is displayed (step S411). The rear image processing apparatus 10 repeatedly executes the above processing.

<Modification>
In the first to fourth embodiments, the rear image processing apparatus 10 is assumed to be mounted on the host vehicle, but the rear image processing apparatus 10 is provided with a communication means with an external server so that the functions of the rear image processing apparatus 10 can be improved. Some may be performed by the server. For example, if an external server performs processing with a high calculation load such as nonlinear scaling processing and image analysis processing for extracting a rear object image, the calculation load of the rear image processing apparatus 10 can be reduced.

Further, in the first to fourth embodiments, it is considered that the appropriate scaling coefficient strictly depends not only on the distance from the vehicle to the rear object but also on the actual size of the rear object. Therefore, the rear image processing apparatus 10 may recognize the size of the rear object and set the scaling coefficient in consideration of the size of the rear object.

When the rear object is a rear vehicle, the image processing unit 13 may determine the vehicle type of the rear vehicle and change the scaling coefficient according to the size known from the vehicle type of the rear vehicle. In this case, the rear image processing apparatus 10 may be provided with a database that stores a scaling table for each vehicle type. Alternatively, a database that stores a scaling table for each vehicle type may be built on an external server so that the image processing unit 13 can acquire the scaling table corresponding to the vehicle type of the rear vehicle from the server. In the system in which the database is built on the server, the administrator of the server may update the database when a new vehicle type is added, and the user of the rear image processing apparatus 10 does not need to update the database. The system is very convenient for the user.

Note that, in the present invention, it is possible to freely combine the embodiments or appropriately modify or omit the embodiments within the scope of the invention.

Although the present invention has been described in detail, the above description is illustrative in all aspects, and the present invention is not limited thereto. It is understood that innumerable variants not illustrated can be envisaged without departing from the scope of the invention.

10 rear image processing device, 11 rear image acquiring unit, 12 distance acquiring unit, 13 image processing unit, 14 display processing unit, 15 object image extracting unit, 21 rear image capturing device, 22 display device, 23 distance measuring sensor.

Claims

A rear image acquisition unit that acquires a rear image that is an image of a landscape behind the vehicle,
A distance acquisition unit for acquiring a distance from the vehicle to an object shown in the rear image,
When the distance from the vehicle to the object is larger than a predetermined threshold value, a scaling coefficient is set to a constant standard value to perform scaling processing of the rear image, and when the distance is equal to or less than the threshold value, the distance is set. An image processing unit that performs the scaling process of the rear image by changing the scaling coefficient so that the rear image is reduced as becomes shorter.
A display processing unit for displaying the rear image after the scaling processing on a display device;
A rear image processing apparatus including.
The backward image processing device according to claim 1, wherein the image processing unit sets the scaling coefficient to a uniform value over the entire backward image.
The rear according to claim 1, wherein the image processing unit sets the scaling coefficient for each position in the rear image, and sets the scaling coefficient closer to the standard value at a position farther from the center of the rear image. Image processing device.
The rear image processing device according to claim 3, wherein the image processing unit changes the scaling coefficient nonlinearly with respect to a distance from the center of the rear image.
Further comprising an object image extraction unit that extracts an object image that is an image of the object from the rear image,
The rear image processing apparatus according to claim 4, wherein the image processing unit changes a position at which a slope of a change in the scaling coefficient becomes large according to a size of the object image.
The rear image processing device according to claim 5, wherein the image processing unit sets a position where a slope of a change in the scaling coefficient becomes large in a peripheral portion of the object image.
Further comprising an object image extraction unit that extracts an object image that is an image of the object from the rear image,
The rear image processing according to claim 1, wherein the image processing unit sets the scaling coefficient for each position in the rear image, and sets the scaling coefficient closer to the standard value at a position farther from the object image. apparatus.
The rear image processing device according to claim 7, wherein the image processing unit changes the scaling coefficient in a non-linear manner with respect to a distance from the object image.
The rear image processing apparatus according to claim 8, wherein the image processing unit changes a position at which a slope of a change in the scaling coefficient becomes large according to a position of the object image.
The rear image processing apparatus according to claim 9, wherein the image processing unit sets a position where a slope of a change in the scaling coefficient becomes large in a peripheral portion of the object image.
A method of displaying a rear image, which is an image of a landscape behind a vehicle,
The rear image acquisition unit of the rear image processing device acquires the rear image,
The distance acquisition unit of the rear image processing device acquires a distance from the vehicle to an object shown in the rear image,
The image processing unit of the rear image processing device, when the distance from the vehicle to the object is larger than a predetermined threshold, sets a scaling coefficient to a constant standard value to perform scaling processing of the rear image, When the distance is equal to or less than the threshold value, the scaling process is performed on the rear image by changing the scaling coefficient so that the rear image is reduced as the distance becomes shorter,
The display processing unit of the rearward image processing device causes the rearward image after the scaling processing to be displayed on a display device,
Rear image display method.