JP7073237B2 - Image display device, image display method - Google Patents

Description

The present invention relates to an image display device and an image display method.

There is an attempt to replace the rearview mirror of a vehicle with a CMS (Camera Monitoring System). CMS is a technique for displaying an image captured by an image pickup device on a display. The rearview mirror in which the CMS is adopted is hereinafter referred to as an electronic rearview mirror. The electronic rear-view mirror is a conventional rear-view mirror with a built-in display. In the conventional rear-view mirror, the rear view may be obstructed by the occupants and luggage of the vehicle, but the electronic rear-view mirror displays the image of the image pickup device installed in the rear of the vehicle on the display inside the rear-view mirror, so that the rear view can be seen. It becomes easier to secure.

However, when the conventional rear-view mirror is replaced with an electronic rear-view mirror, the size of the rear-view object displayed on the electronic rear-view mirror is the same as the size of the conventional rear-view object reflected on the rear-view mirror due to the influence of the mounting position of the image pickup device and the focal length. It is known that it is difficult for the driver to grasp the sense of distance from the rear object because they are different. This phenomenon is particularly likely to occur due to the sense of distance from a rear object in the vicinity of the own vehicle.

In order to reduce this inconvenience, there is known a technique of preparing an image of the interior of a vehicle and fitting the image of the interior of the vehicle into an image of an image pickup device installed at the rear of the vehicle (see, for example, Patent Document 1). ). Patent Document 1 discloses a technique for making a vehicle interior image translucent and superimposing it on an image captured by an image pickup device installed at the rear of the vehicle.

Further, a technique has been devised in which the transparency of another image displayed together with the image of the image pickup device installed at the rear of the vehicle is changed according to the vehicle speed (see, for example, Patent Document 2). Patent Document 2 discloses a technique for displaying a symbol image of the own vehicle whose transparency is lowered as the vehicle speed is higher, below the image of the image pickup device installed at the rear of the vehicle.

Japanese Patent No. 5115136 Japanese Unexamined Patent Publication No. 2017-215446

However, in the conventional technique, when the vehicle speed is relatively high, there is a problem that the driver's field of view is narrowed by the image displayed together with the image captured by the image pickup device. This will be described with reference to FIG.

FIG. 1 is a diagram illustrating a problem when an image of a vehicle interior and an image captured by an image pickup device installed at the rear of the vehicle are combined. FIG. 1A is a diagram illustrating a combination of an image 9 in the vehicle interior and an image captured by an image pickup device 11 installed at the rear of the vehicle. The image captured by the image pickup device is hereinafter referred to as a captured image 8. The central portion of the vehicle interior image 9 is sufficiently transparent so that the vehicle interior image 9 can be visually recognized at least. For example, it is almost completely transparent. The region where the transparency is ensured in this way is referred to as the transparent portion 31 of the vehicle interior image 9. The shape of the transparent portion 31 is the same as or imitated by the shape of the rear glass of the vehicle. That is, in the vehicle interior image 9, the rear glass portion is the transparent portion 31.

As a result, when the vehicle interior image 9 is superimposed on the captured image 8, the captured image 8 is displayed on the transparent portion 31 and the vehicle interior image 9 is displayed around the captured image 8 as shown in FIG. 1 (b). ..

As shown in FIG. 1 (b), the captured image 8 and the vehicle interior image 9 are combined and displayed, so that the driver can easily grasp the size of the vehicle behind and can easily grasp the sense of distance.

However, as shown in FIG. 1 (c), when the vehicle is moving at high speed, the correlation between the size of the vehicle behind and the sense of distance is weakened, so that the captured image 8 and the vehicle interior image 9 are combined and displayed. The benefits of that are reduced. That is, when the vehicle speed is high, the driver should grasp a wider situation around the vehicle, but in the technique disclosed in Patent Document 1, the vehicle interior image 9 is always displayed semi-transparently regardless of the vehicle speed. , Always obstructs the view. Further, in the technique disclosed in Patent Document 2, the higher the vehicle speed, the darker the symbol image of the own vehicle (corresponding to the vehicle interior image 9) is displayed. Therefore, the higher the vehicle speed, the more obstructed the driver's view. As described above, in the conventional technique, it has not been considered to secure a wide field of view when the vehicle speed is high.

In view of the above problems, it is an object of the present invention to provide an image display device that suppresses the driver's field of view from being narrowed by an image displayed together with an image captured by the image pickup device.

In view of the above problems, the present invention has an imaging device that images the rear of the vehicle to generate an image, and an in-vehicle image acquisition means that acquires an image of the interior of the vehicle from the front to the rear of the vehicle. The speed detecting means for detecting the current traveling speed of the vehicle and the traveling speed detected by the speed detecting means increase the transparency of the region surrounding the central portion of the vehicle interior image and combine with the captured image. It has an image synthesizing means for generating a composite image and an image output means for outputting the composite image synthesized by the image synthesizing means to a display device, and the image synthesizing means can at least visually recognize the captured image. Provided is an image display device characterized in that the size of the central portion of the vehicle interior image whose transparency is secured regardless of the traveling speed is increased as the traveling speed is increased.

It is possible to provide an image display device that suppresses the driver's field of view from being narrowed by the image displayed together with the image captured by the image pickup device.

It is a figure explaining the problem when the image of the vehicle interior and the image taken by the image pickup device installed at the rear of the vehicle are combined. It is an example of the figure explaining the display control of the captured image and the vehicle interior image displayed by the display unit of the electronic rear-view mirror. It is a figure which shows the mounting example of the electronic rear-view mirror. This is an example of a schematic perspective view of a display unit. It is an example of the figure which shows the schematic structure of a display unit. This is an example of a functional block diagram illustrating the functions of the display unit. It is a figure explaining an example of a synthesis process. This is an example of a flowchart showing a procedure in which an electronic rear-view mirror displays an image on a display. This is an example of a functional block diagram illustrating the function of the display unit (Example 2). This is an example of a diagram for explaining the relationship between the distance to the vehicle behind and the α value. This is an example of a flowchart showing a procedure in which an electronic rear-view mirror displays an image on a display (Example 2). It is an example of the figure explaining the control of the transparency in the height direction of the vehicle interior image. It is an example of the figure explaining the method of determining the α value according to the height Y of the vehicle interior image. This is an example of a flowchart showing a procedure in which an electronic rear-view mirror displays an image on a display (Example 3). It is a figure explaining the image which shows the outline of the vehicle interior image. It is a figure which shows an example of the image displayed by an electronic rear-view mirror. This is an example of a functional block diagram illustrating the function of the display unit (Example 4). This is an example of a flowchart showing a procedure in which an electronic rear-view mirror displays an image on a display (Example 4). It is a figure which shows an example of the image displayed by an electronic rear-view mirror. This is an example of a flowchart showing a procedure in which an electronic rear-view mirror displays an image on a display (Example 5).

Hereinafter, as an example of an embodiment for carrying out the present invention, an electronic rear-view mirror and an image display method performed by the electronic rear-view mirror will be described with reference to the drawings with reference to examples.

<Outline of the electronic rear-view mirror of this embodiment>
FIG. 2 is an example of a diagram illustrating display control of the captured image 8 and the vehicle interior image 9 displayed by the display unit 12 of the electronic rear-view mirror of the present embodiment. The electronic rear-view mirror of the present embodiment changes the transparency of the vehicle interior image 9 according to the vehicle speed.
Vehicle speed: Medium to large → Transparency Low vehicle speed: Small to medium → Large transparency Fig. 2 (a) to Fig. 2 (c) show that the higher the vehicle speed, the greater the transparency of the vehicle interior image 9. That is, in FIG. 2A, since the vehicle speed is low, the vehicle interior image 9 is almost opaque. In FIG. 2B, since the vehicle speed is medium, the vehicle interior image 9 is translucent. In FIG. 2C, since the vehicle speed is high, the vehicle interior image 9 is almost transparent.

Therefore, since the vehicle interior image 9 is clearly displayed at low speed, the driver can easily grasp the sense of distance from the vehicle behind, and the vehicle interior image 9 becomes transparent as the vehicle speed increases, so that it is easy to secure the rear view. Become.

<Terminology>
The image display device is a device that displays image data around the vehicle captured by the image pickup device 11 installed in the vehicle on the display device. An electronic rear-view mirror will be described as an example of a device for displaying an captured image combined with a vehicle interior image, but the image pickup device 11 and the display device may be mounted on a moving body such as a vehicle. The image display device is not limited to the electronic rear-view mirror, as long as the captured image and some image are combined. The electronic rear-view mirror may be called an electronic rear-view mirror, an electronic mirror, or the like.

The image pickup device may be mounted on a moving body other than the vehicle. For example, it may be mounted on a motorcycle, a light vehicle such as a bicycle, a wheelchair, a robot, or the like.

<Configuration example>
FIG. 3 is a diagram showing an example of mounting the electronic rear-view mirror 10. FIG. 3A shows an example of the display unit 12 installed in the vehicle interior, and FIG. 3B is a diagram showing an arrangement example of the image pickup apparatus 11 and the display unit 12.

As shown in FIG. 3A, the display unit 12 of the electronic rear-view mirror 10 is located between the driver's seat and the passenger's seat and is fixed to the upper part inside the windshield, similarly to the conventional rear-view mirror. There are countries and regions where the mounting position of the display unit 12 is not clearly specified by law, and the position shown is only an example. In addition, in countries and regions where there is a clear designation by law, it will be installed in a position that complies with the law. The number of display units 12 is not limited to one, and a plurality of display units 12 may be mounted.

The image pickup device 11 is arranged behind the vehicle 101. It is preferable that the image pickup apparatus 11 takes an image in the direction of the optical axis close to the direction of the light incident on the conventional rear-view mirror. Since the conventional rear-view mirror is arranged above the windshield and reflects the rear of the vehicle on the mirror, it is often preferable that the image pickup device 11 is also arranged above the vehicle. For example, it may be installed on a roof or a roof spoiler. The horizontal direction of the image pickup apparatus 11 is preferably at or near the center of the vehicle. However, even if it is installed at the right end or the left end, it is possible to take a rear image. Image processing may be performed as if the image of the image pickup apparatus 11 installed at the right end or the left end was taken near the center.

The location of the image pickup device 11 may be different not only for each vehicle type but also for each individual vehicle. For example, it may be around the license plate, around the vehicle manufacturer emblem, or at an appropriate place on the rear bumper.

Further, conventionally, a back camera may be mounted on the vehicle 101. The back camera is a camera that displays the image behind when the car is moved backward (when the driver operates the shift lever in reverse), such as when entering the garage, and the captured image is displayed on a display such as a car navigation system. There are many. In the present embodiment, the image of the back camera may be displayed on the display unit 12, or the image captured by the image pickup device 11 of the electronic rearview mirror 10 may be displayed as an image of the back camera on a display such as a car navigation system. May be good.

Further, a relatively wide-angle camera such as a fisheye camera may be used for both the electronic room mirror 10 and the rear-view camera. That is, a part of the image captured by the wide-angle camera is displayed on the display unit 12, and another part (which may be duplicated) is displayed on a display such as a car navigation system.

The wiring from the image pickup device 11 to the display unit 12 may be laid on the ceiling or near the floor, and is determined in consideration of the wiring length and the wiring. Further, the image pickup apparatus 11 may transmit an image to, for example, the display unit 12 by wireless communication.

Further, a vehicle speed sensor 19 is connected to the display unit 12. The vehicle speed sensor 19 is a sensor that outputs a pulse signal according to the rotation speed of the wheels in order to determine the current traveling speed of the vehicle. The traveling speed detected by the vehicle speed sensor 19 is used for synthesizing the vehicle interior image 9 and the captured image 8.

Further, the display unit 12 may be connected to a power supply line, a reverse signal line, a winker signal line, a video input of another image pickup device (side camera, back camera, etc.), etc., but is not shown in FIG. It is omitted. For example, the display unit 12 can display the image of the back camera by the signal from the reverse signal line, and the display unit 12 can display the image of the side camera by the signal from the winker signal line.

FIG. 4 is an example of a schematic perspective view of the display unit 12. The display unit 12 has a display 15, four operation buttons 1 to 4, and a lever 17. The wiring interface and the like are omitted. In addition to the configuration shown in the figure, it may have a memory card insertion portion like a drive recorder, and the shape, the number of buttons, the position of the buttons, etc. are only examples.

The display 15 is a flat panel display such as a liquid crystal display or an organic EL display. It may be a rear projection type. As will be described later, a half mirror is fixed on the front surface (indoor side) of the display 15 so as to superimpose on the display 15.

The four operation buttons 1 to 4 are input devices for displaying the setting screen of the electronic rear-view mirror 10 and accepting operations from the user. OSD (On Screen Display) is known as a function for displaying a simple setting screen. However, how the user interface is displayed can be changed according to the resources of the display unit 12. For example, when an OS (Operation System) is installed on the board 14 described later, the screen display function supported by the OS can be used.

Further, an vehicle interior image pickup device 18 is installed around the display 15 (upper part in the figure). The vehicle interior image pickup device 18 is an image pickup device that captures an image of the vehicle interior behind the vehicle from the display unit 12 (front of the vehicle). Since the vehicle interior image pickup device 18 can image the vehicle interior in real time, the highly realistic vehicle interior image 9 can be displayed on the display 15. On the other hand, when there is an occupant in the rear seat of the passenger compartment or the like, it is not always necessary to take an image in real time because the view may be obstructed. It is sufficient to take an image of. In terms of capturing one or more vehicle interior images 9 before the occupant gets on board, the driver or the like may capture one or more vehicle interior images 9 with a portable camera. If the memory card storing the vehicle interior image 9 is attached to the display unit 12, the display unit 12 can synthesize the vehicle interior image 9 with the captured image 8. Further, the car navigation system may download the vehicle interior image 9 from the server. Therefore, the vehicle interior image pickup device 18 may not be necessary.

The function of each operation button is as follows, for example.
Operation button 1: Menu display (brightness, contrast, color temperature, language, display area settings, etc.)
Operation button 2: Select in a certain direction (move right, move up, rotate right, increase, etc.)
Operation button 3: Select another direction (move left, move down, rotate left, decrease, etc.)
Operation button 4: Confirmation The driver can set brightness, contrast, color temperature, language, etc., rotate the captured image, and move the display area.

The lever 17 is an input device for switching the power ON and the power OFF of the electronic rearview mirror 10. When the power is on, the display unit 12 outputs an image captured by the image pickup device 11, and when the power is off, the display unit 12 does not display the image captured by the image pickup device 11. Therefore, external light is reflected by the half mirror, and the display unit 12 becomes a mirror that reflects the room like a normal mirror. This will be described with reference to FIG.

FIG. 5 is an example of a diagram showing a schematic structure of the display unit 12. FIG. 5A shows the operation of the half mirror 16 in the power-on state, and FIG. 5B shows the operation of the half-mirror 16 in the power-off state. For example, assuming that the transparency of the half mirror 16 is T% (for example, 50 to 95%), the light of T% of the display 15 passes through the half mirror 16 when the power is turned on. Since the brightness of the display 15 is brighter than that in the vehicle interior even if only T% of light passes through, the driver can visually recognize the image displayed by the display 15.

Since the display 15 does not irradiate light when the power is off, the inside of the half mirror is dark. Since the light inside the vehicle interior is stronger than the inside of the half mirror, the intensity of the light reflected by the half mirror 16 is also high. As a result, the transmitted light from the inside of the half mirror 16 is squeezed by the bright reflected light on the vehicle interior side, so that it looks like a mirror from the vehicle interior side. Therefore, the driver can easily switch between the image displayed by the display 15 and the reflection of the room light by a normal mirror by swinging the lever forward or backward. For example, when the incident light from the rear of the vehicle is strong due to the west sun or the like and the visibility of the display 15 is deteriorated, it is possible to switch to a normal mirror.

As shown in FIG. 5, the display unit 12 has a substrate 14, and the substrate 14 controls the image pickup apparatus 11 and performs image processing such as cutting out a display area from the captured image 8. The substrate 14 has a general computer function, and has a CPU, RAM, ROM, a flash memory, an input / output I / F, and the like.

<Function of display unit>
The function of the display unit 12 will be described with reference to FIG. FIG. 6 is an example of a functional block diagram illustrating the function of the display unit 12. The display unit 12 includes an image acquisition unit 21, a vehicle interior image acquisition unit 22, an image cutting unit 23, an image composition unit 24, an image output unit 25, and a vehicle speed acquisition unit 30. Each function of the display unit 12 is a function or means realized by the CPU of the board 14 of the display unit 12 by executing a program.

The substrate 14 performs general control of the image pickup apparatus 11. For example, the substrate 14 performs exposure control that controls at least one of the shutter speed and the sensitivity (gain amount) so that the image has an appropriate brightness. In addition, by making the ratios of R, G, and B to the same level, which is called auto white balance control (AWB control), the colors of the white part and the colored part of the subject change depending on the color temperature of the light source. Perform processing to reduce. Further, a drive signal is generated so that the image pickup device can perform an image pickup operation at an appropriate timing, and a clock signal is supplied to the A / D converter. In addition, processing such as removing noise or adjusting the sharpness of an image by extracting and emphasizing only edge components may be performed.

The image acquisition unit 21 sequentially acquires image data from the image pickup device 11 at a predetermined frame rate. As described above, the image is assumed to be a moving image, but the image may be a still image. In this embodiment, this image data is referred to as a captured image 8.

The vehicle interior image acquisition unit 22 acquires the vehicle interior image 9 stored in the vehicle interior image storage unit 26 or the vehicle interior image 9 captured by the vehicle interior image pickup device 18. The vehicle interior image storage unit 26 is a flash memory included in the memory card or the substrate 14 described above.

The image cutting unit 23 cuts a display area from the captured image 8. The display area is an area of the image data displayed by the display 15. This is done because the size (number of pixels) of the captured image is larger than the size (number of pixels) of the display 15. The display area is image data having almost the same size as the size of the display 15. The cutting process may be called trimming.

The vehicle speed acquisition unit 30 periodically or repeatedly acquires the current traveling speed (vehicle speed) from the vehicle speed sensor 19. The vehicle speed sensor 19 may estimate the vehicle speed, and the vehicle speed acquisition unit 30 may estimate the vehicle speed from the difference in the position information detected by, for example, GPS (Global Positioning System).

The image synthesizing unit 24 synthesizes the vehicle interior image 9 and the captured image 8 according to the vehicle speed. That is, as the vehicle speed increases, the transparency of the vehicle interior image 9 increases and the image is combined with the captured image 8. An image in which the vehicle interior image 9 and the captured image 8 are combined is called a combined image.

The image output unit 25 sends the composite image to the display 15 and displays it on the display 15.

<About synthesis processing>
An example of the synthesis process will be described with reference to FIG. 7. FIG. 7A is an example of a diagram illustrating the composition of the vehicle interior image 9 and the captured image 8. The vehicle interior image 9 is superimposed on the front side, and the captured image 8 is superimposed on the back side in this order. The vehicle interior image 9 has a transparent portion 31 that is always transparent, and a transparency variable portion 32 that changes from almost completely opaque to almost completely transparent depending on the vehicle speed as a region surrounding the transparent portion 31.

When the vehicle interior image 9 is prepared in advance, a transparent portion 31 that imitates the shape of the rear glass of the vehicle is set in advance in the central portion of the vehicle interior image 9. That is, 0 is set to the α value of the pixel corresponding to the shape of the rear glass. The α value will be described below. When the vehicle interior image 9 is captured in real time, the pixels corresponding to the shape of the rear glass of the vehicle are predetermined, and the display unit 12 sets the α value of the pixel corresponding to the shape of the rear glass of the vehicle interior image 9 to 0. Set. The pixels corresponding to the shape of the rear glass of the vehicle may be arbitrarily set by the driver or the like, or may be set by the vehicle interior image acquisition unit 22 by utilizing the fact that the brightness of the rear glass is brighter than others. good.

When the vehicle interior image 9 is prepared in advance, it is preferable that the vehicle interior image 9 is an image captured by the same vehicle as the vehicle on which the display unit 12 is mounted, but the vehicle interior image 9 of a general vehicle is used. It may be prepared.

In the present embodiment, the transparency of the transparency variable portion 32 is represented by an alpha value (α value). The α value is the transparency information set for each pixel in the image data handled by the computer. As for the α value, 0 represents completely transparent (colorless) and 255 (in the case of 8 bits) represents completely opaque. Therefore, the α value of the transparent portion 31 is always 0 (or a small value that can be regarded as transparent), and the α value of the transparent variable portion 32 takes a value of 255 to 0.

FIG. 7B shows an example of the relationship between the α value of the transparency variable portion 32 and the vehicle speed. The α value is set so that the transparency variable portion 32 becomes completely transparent when the vehicle speed V = Vmax. Vmax may be preset as a vehicle speed in which the wide field of view is prioritized over the sense of distance. Further, the driver may set a desired vehicle speed (Vmax) from the OSD. For example, it is conceivable that Vmax = 30 km / h to 80 km / h, but the value is not limited to this value.

As shown in FIG. 7B, if the vehicle speed is determined, the α value is uniquely determined, so that the image synthesizing unit 24 can combine the captured image 8 and the vehicle interior image 9. The relationship between the vehicle speed and the α value may be a straight line 110, a curve 120, or a curve 130. Further, they may be associated with each other by a table. Further, the vehicle interior image 9 can be made slightly opaque instead of being completely transparent at the vehicle speed Vmax. Similarly, at a vehicle speed of 0, the vehicle interior image 9 can be made slightly transparent instead of being completely opaque.

The image synthesizing unit 24 synthesizes the captured image 8 and the vehicle interior image 9 as follows. Such synthesis using the α value is called an alpha blend. Combining is done pixel by pixel.
・ When α = 0, only the pixels of the captured image 8 are drawn. ・ When α = 255, only the vehicle interior image 9 is drawn. ・ When α = 254 to 1, the pixel values calculated by the following formula are drawn. Image 9 pixel value + (255-α) x captured image 8 pixel value} / 256 "... (1)
By synthesizing the captured image 8 and the vehicle interior image 9, the vehicle interior image 9 gradually becomes transparent as the vehicle speed increases, and the captured image 8 becomes visible.

<Operation procedure>
FIG. 8 is an example of a flowchart showing a procedure in which the electronic rear-view mirror displays an image on the display 15. The process of FIG. 8 is repeatedly executed while the power of the electronic rearview mirror is ON.

First, the image acquisition unit 21 acquires image data (captured image 8) from the image pickup device 11 (S10). Image pickup images 8 are transmitted one after another from the image pickup apparatus 11.

The image cutting unit 23 cuts a display area from the captured image 8 (S20). The cutout of the display area may be performed after composition. In this case, since it is necessary to perform a process of specifying the display area from the captured image 8, it is often preferable to cut out the display area before compositing.

Next, the vehicle interior image acquisition unit 22 acquires the vehicle interior image 9 (S30). The vehicle interior image 9 may be stored in the vehicle interior image storage unit 26 or may be captured by the vehicle interior image pickup device 18. The vehicle interior image 9 stored in the vehicle interior image storage unit 26 is preset with a region where the α value is 0 (transparency portion 31) and a region where the α value changes from 255 to 0 (transparency variable portion 32). When the vehicle interior image pickup device 18 takes an image, the vehicle interior image acquisition unit 22 sets a region where the α value is 0 and a region where the α value changes from 255 to 0 in the vehicle interior image 9 for each pixel of the vehicle interior image 9. ..

The vehicle speed acquisition unit 30 acquires the current vehicle speed from the vehicle speed sensor 19 (S40). Since the fluctuation of the vehicle speed directly affects the transparency of the vehicle interior image 9 and does not cause the vehicle interior image 9 to blink, the moving average of the vehicle speed may be used.

Next, the image synthesizing unit 24 determines the α value of the transparency variable unit 32 according to the vehicle speed (S50). The α value of the transparent portion 31 is 0 (constant).

The image synthesizing unit 24 combines the captured image 8 and the vehicle interior image 9 with the α value to generate a composite image (S60). The image output unit 25 displays the composite image on the display 15 (S70).

<Summary>
As described above, the electronic rear-view mirror 10 of the present embodiment clearly displays the vehicle interior image 9 at low speeds, so that the driver can easily grasp the sense of distance from the vehicle behind, and the vehicle interior image 9 is displayed as the vehicle speed increases. Since it is transparent, it is easy to secure the rear view.

In the first embodiment, the transparency is changed according to the vehicle speed, but in the present embodiment, the electronic rearview mirror that changes the transparency according to the distance to the rear-view vehicle will be described. Generally, when the distance to the rear vehicle is short, the vehicle speed is low, and when the distance to the rear vehicle is long, the vehicle speed is high. Therefore, the distance to the rear vehicle can be used instead of the vehicle speed.

<About the function of the display unit>
FIG. 9 is an example of a functional block diagram illustrating the function of the display unit 12 of this embodiment. With respect to the description of FIG. 9, since the components having the same reference numerals in FIG. 6 perform the same function, only the main components of the present embodiment may be mainly described.

The display unit 12 of this embodiment has a distance estimation unit 27 and does not have a vehicle speed acquisition unit 30. Therefore, the wiring for connecting the vehicle speed sensor 19 and the display unit 12 becomes unnecessary. The distance estimation unit 27 recognizes the rear vehicle from the captured image 8 and estimates the distance to the rear vehicle depending on how large the rear vehicle is imaged. For example, machine learning is used for recognizing a rear vehicle, and it is assumed that an identification model for identifying a rear vehicle is generated in advance by machine learning. It is also possible to judge the approximate vehicle type by machine learning. The distance estimation unit 27 has a table in which the image size of the rear vehicle (the length of the diagonal line of the extrinsic rectangle of the rear vehicle in the captured image, the area, etc.) and the distance are associated with each vehicle type, and the table is referred to. This estimates the distance to the vehicle behind. The distance information is sent to the image composition unit 24.

Instead of estimating the distance to the vehicle behind by such image processing, the distance to the vehicle behind may be measured by a millimeter wave radar or Lidar (Light Detection and Ranging, Laser Imaging Detection and Ranging).

The image synthesizing unit 24 determines the α value according to the distance information, and synthesizes the captured image 8 and the vehicle interior image 9. Other functions may be the same as in the first embodiment.

<Determining the α value according to the distance to the vehicle behind>
FIG. 10 is an example of a diagram illustrating the relationship between the distance to the vehicle behind and the α value. The α value is set so that the transparency variable portion 32 becomes completely transparent at a vehicle speed L = Lmax. Lmax may be preset as a distance in which the wide field of view is prioritized over the sense of distance. Further, the driver may set a desired distance from the OSD. For example, it is conceivable that Lmax = 30 m to 50 m, but the value is not limited to this.

As shown in FIG. 10, since the α value is uniquely determined when the distance is determined, the image synthesizing unit 24 can synthesize the captured image 8 and the vehicle interior image 9. The relationship between the distance and the α value may be a straight line 110, a curve 120, or a curve 130.

If the distance estimation unit 27 cannot detect the vehicle behind (when the vehicle behind does not exist), it is less necessary for the driver to grasp the sense of distance, so that the transparency variable unit 32 of the vehicle interior image 9 may be transparent. This makes it easier to secure a rear view when the vehicle behind is not detected.

<Operation procedure>
FIG. 11 is an example of a flowchart showing a procedure in which the electronic rear-view mirror displays an image on the display 15. In the description of FIG. 11, the difference from FIG. 8 will be mainly described. The processing of steps S110 to 130 may be the same as that of steps S10 to S30 of FIG.

When the vehicle interior image 9 is acquired in step S130, the distance estimation unit 27 processes the captured image 8 to estimate the distance to the vehicle behind (S140). The distance detection is executed periodically or repeatedly. When the distance estimation unit 27 detects a rear vehicle, it sends the distance to the rear vehicle to the image synthesis unit 24, and when it does not detect the rear vehicle, it sends a message to that effect to the image composition unit 24.

In the process of estimating the distance to the rear vehicle, the image synthesizing unit 24 determines whether or not the rear vehicle could be detected based on the notification from the distance estimation unit 27 (S150).

If the determination in step S150 is No, the image composition unit 24 sets the α value of the transparency variable unit 32 to 0 (S190). As a result, when the transparency variable portion 32 becomes transparent and the vehicle behind is not detected, it becomes easy to secure the rear view.

When the determination in step S150 is Yes, the image synthesizing unit 24 determines the α value of the transparency variable unit 32 according to the distance to the vehicle behind (S160). The processing of steps S170 to 180 may be the same as that of steps S60 to S70 of FIG.

<Summary>
In this embodiment, the same effect as that of the first embodiment can be obtained by image processing without using the vehicle speed sensor 19.

In Examples 1 and 2, the entire vehicle interior image 9 is made transparent according to the vehicle speed or the distance to the vehicle behind, but it is effective that the lower side of the vehicle interior image 9 is displayed in order to get a sense of distance. It is known to be. Therefore, in this embodiment, an electronic rearview mirror capable of controlling the transparency according to the height direction of the vehicle interior image 9 will be described.

<Display example of the electronic rear-view mirror of this embodiment>
FIG. 12 is an example of a diagram illustrating control of transparency in the height direction of the vehicle interior image 9. 12 (a) shows the transparency of the vehicle interior image 9 when the vehicle speed is low (for example, 0), and FIG. 12 (b) shows the transparency of the vehicle interior image 9 when the vehicle speed is medium. FIG. 12 (c). Indicates the transparency of the vehicle interior image 9 when the vehicle speed is high (Vmax). For convenience of drawing, in FIG. 12, the transparency of the transparency variable portion 32 is shown by the shaded density. That is, a high shading density means low transparency (large α value), and a low shading density means high transparency (small α value).

In the vehicle interior image 9 of FIG. 12A, since the vehicle speed is low, the transparency of the transparency variable portion 32 does not change in the height direction and is low (for example, the α value is 255 at a vehicle speed of 0).

In the vehicle interior image 9 of FIG. 12B, since the vehicle speed is medium, the transparency of the transparency variable portion 32 is higher toward the upper side and smaller toward the lower side. For example, the α value is small at the upper end, the α value is medium at the interruption, and the α value is large at the lower end. As described above, at the same vehicle speed, the transparency of the transparency variable portion 32 increases toward the upper side of the vehicle interior image 9.

In the vehicle interior image 9 of FIG. 12 (c), since the vehicle speed is high (Vmax), the upper part of the transparency variable portion 32 is almost transparent, and the vehicle interior image 9 remains only below. Further, the transparency of the lower vehicle interior image 9 is higher than that of the lower part of FIG. 12B (the α value is small).

By displaying the upper part of the vehicle interior image 9 with a greater transparency than the lower part with respect to the vehicle speed in this way, only the lower part of the vehicle interior image 9 is emphasized, and it becomes possible to secure a view while easily grasping a sense of distance. ..

<Method of determining the α value according to the height Y of the vehicle interior image>
Next, a method of determining the α value according to the height Y of the vehicle interior image 9 will be described with reference to FIG. Since the method of determining the α value is different in this embodiment, the functional block diagram of FIG. 6 will be used for description.

FIG. 13 is an example of a diagram illustrating a method of determining an α value according to the height Y of the vehicle interior image 9. FIG. 13A shows the relationship between the vertical direction and the height Y of the vehicle interior image 9. As shown in FIG. 13A, it is assumed that the height Y increases toward the upward direction of the vehicle interior image 9.

FIG. 13B is a diagram showing an example of the relationship between the coefficient K for obtaining the α value and the height Y. As shown by the straight line 150 in FIG. 13B, the coefficient K (absolute value) increases as the height Y increases. The coefficient K has an intercept so that Y = 0 and K = 0. Note that FIG. 13B is an example of correspondence information.

FIG. 13C is an example of a diagram illustrating the relationship between the vehicle speed V and the α value according to the height Y. In this embodiment, the α value is calculated as follows.
α = 255-K × V
That is, the value obtained by subtracting the product of the coefficient K and the vehicle speed V from the constant 255 is the α value. As described above, the coefficient K is a proportional coefficient between the vehicle speed V and the α value. The larger K, the smaller the α value when the vehicle speed increases. Since K is larger as the height Y is larger, it becomes transparent earlier as the vehicle interior image 9 is higher as the vehicle speed is increased.

The coefficient K is set so that the α value becomes 0 at the vehicle speed V = Vmax, for example, when the height Y = Ymax (K = 255 / Vmax). By doing so, the upper part of the vehicle interior image 9 becomes completely transparent at the vehicle speed V = Vmax, but the lower part of the vehicle interior image 9 is not completely transparent even at the vehicle speed V = Vmax, so that the driver can see even if the vehicle speed is high. It becomes easier to get a sense of distance.

As shown in FIG. 13B, the transparency can be adjusted above and below the vehicle interior image 9 depending on how the coefficient K is changed with respect to the height Y. For example, if it is desired to make the upper part of the vehicle interior image 9 transparent at a lower vehicle speed, the coefficient K may be obtained as in the dotted line 170 having a larger coefficient K with respect to the height Y. If it is desired to make the lower part of the vehicle interior image 9 opaque at a higher vehicle speed, the coefficient K may be obtained as in the dotted line 160 in which the coefficient K is smaller than the height Y. In addition, the coefficient K may be determined in any way.

<Operation procedure>
FIG. 14 is an example of a flowchart showing a procedure in which the electronic rear-view mirror displays an image on the display 15. In the description of FIG. 14, the difference from FIG. 8 will be mainly described. In FIG. 14, the process of step S50-2 is different from that in FIG.

In step S50-2, the image synthesizing unit 24 determines the coefficient K for each height Y. If the height Y is the same, the coefficient K is a constant (since it does not change), it may be determined in advance. Next, the image synthesizing unit 24 calculates an α value for each pixel of the transparency variable unit 32 based on the coefficient K and the vehicle speed V determined by the height Y of the pixels. Subsequent processing may be the same as in FIG.

<Summary>
As described above, the electronic rear-view mirror of the present embodiment makes the upper part of the vehicle interior image 9 transparent while displaying the lower part of the vehicle interior image 9 by making the upper part of the vehicle interior image 9 more transparent than the lower part at the same vehicle speed. Because it can be done, it becomes easier to get a sense of distance and secure visibility at the same time.

In this embodiment, the transparency is determined with respect to the vehicle speed, but the transparency may be determined with respect to the distance to the vehicle behind.

In Examples 1 to 3, the vehicle interior image 9 and the captured image 8 are combined, but in this embodiment, the electronic rearview mirror 10 that combines the image showing the outline of the vehicle interior image 9 and the captured image 8 will be described.

<Image showing the outline of the vehicle interior image>
An image showing the outline of the vehicle interior image 9 will be described with reference to FIG. An image showing the contour of the vehicle interior image 9 is hereinafter referred to as a contour image. FIG. 15A shows a vehicle interior image 9, and FIG. 15B shows a contour image. As shown in the figure, the contour image is an image in which the characteristic contour of the vehicle interior image 9 is represented only by the black-and-white contour line 180.

For example, only the position and shape of the rear glass and other characteristic shapes representing the structure of the vehicle interior are contour images drawn by the contour line 180. The portion without the contour line 180 is always transparent. That is, in this embodiment, the portion without the contour line 180 is the transparent portion 31. Further, since the transparency of the contour line 180 is changed according to the vehicle speed, it is the transparency variable portion 32 of this embodiment. Therefore, the driver can grasp the sense of distance from the vehicle behind by the contour image, and can secure the field of view by the transparent portion of the contour image.

<Example of image displayed by electronic rear-view mirror>
FIG. 16 shows an example of an image displayed by the electronic rear-view mirror of the present embodiment. The electronic rear-view mirror of the present embodiment changes the transparency of the contour image according to the vehicle speed in the same manner as in the first embodiment. The portion other than the contour line (transparent portion 31) is always transparent.
Vehicle speed: Medium to large → Transparency Low vehicle speed: Small to medium → Transparency large
16 (a) to 16 (c) show that the higher the vehicle speed, the higher the transparency of the contour image. That is, in FIG. 16A, since the vehicle speed is low, the contour line 180 of the contour image is almost opaque. In FIG. 16B, since the vehicle speed is medium, the contour line 180 of the contour image is translucent. In FIG. 16C, since the vehicle speed is high, the vehicle interior image 9 is almost transparent.

Therefore, although the reality as in the vehicle interior image 9 is reduced, it becomes easier to grasp the sense of distance from the vehicle behind while giving priority to ensuring the rear view.

The contour image may be switched instead of controlling the transparency. Several contour images having different thicknesses of the contour lines 180 are prepared in advance, and the image synthesizing unit 24 switches the contour images according to the vehicle speed. In this case, the contour image may be simply superimposed on the captured image 8 instead of being synthesized by the alpha blend. Therefore, the synthesis process becomes unnecessary, and the processing load of the CPU of the substrate 14 can be reduced.

<About the function of the display unit>
FIG. 17 is an example of a functional block diagram illustrating the function of the display unit 12 of this embodiment. In the description of FIG. 17, the difference from FIG. 6 will be mainly described. The display unit 12 of FIG. 17 has a contour image storage unit 29. The contour image storage unit 29 stores the above contour image in advance. However, as in the first embodiment, the vehicle interior image acquisition unit 22 may generate a contour image from the vehicle interior image 9 captured by the vehicle interior image pickup device 18. Since the contour line 180 corresponds to the edge of the image, the contour image can be dynamically generated by a process such as detecting the edge.

The image synthesizing unit 24 changes the α value of the contour line of the contour image according to the vehicle speed, as in the first embodiment. That is, as shown in FIG. 7B, the α value is determined according to the vehicle speed, and the pixels overlapping the contour line 180 in the captured image 8 are combined with the contour line 180 using the equation (1). Other functions may be the same as in FIG.

<Operation procedure>
FIG. 18 is an example of a flowchart showing a procedure in which the electronic rear-view mirror displays an image on the display 15. In the description of FIG. 18, the difference from FIG. 8 will be mainly described. In FIG. 18, the vehicle interior image 9 is changed to a contour image in steps S30 and S60. Other processing may be the same as in FIG.

<Summary>
As described above, since the electronic rear-view mirror of the present embodiment combines the contour image showing the contour of the vehicle interior image 9 with the captured image 8, the sense of distance from the vehicle behind is given priority while ensuring the rear view. It becomes easier to grasp.

In this embodiment, the transparency is determined with respect to the vehicle speed, but the transparency may be determined with respect to the distance to the vehicle behind.

Moreover, you may apply Example 3 to this Example. That is, when the vehicle speed increases, the upper part of the contour image becomes transparent faster than the lower part. By doing so, it becomes easier to secure a view while keeping a sense of distance from the vehicle behind.

In this embodiment, an electronic rearview mirror that enlarges the transparent portion 31 of the vehicle interior image 9 according to the vehicle speed will be described.

<Example of image displayed by electronic rear-view mirror>
FIG. 19 shows an example of an image displayed by the electronic rear-view mirror of the present embodiment. In the electronic rear-view mirror of this embodiment, the transparent portion 31 is always enlarged as the vehicle speed is increased.
Vehicle speed: Medium to large → Transparent part 31 Small Vehicle speed: Small to medium → Transparent part 31 Large
19 (a) to 19 (c) show that the higher the vehicle speed, the larger the transparent portion 31. That is, in FIG. 19A, the transparent portion 31 is small because the vehicle speed is small. In FIG. 19B, since the vehicle speed is medium, the transparent portion 31 is slightly large. In FIG. 19C, since the vehicle speed is high, the transparent portion 31 is maximized.

Therefore, when the vehicle speed is low, the vehicle interior image 9 occupies a large area, so that it is easy to grasp the distance to the vehicle behind. When the vehicle speed is high, the vehicle interior image 9 occupies a small area, so that the rear view is secured. It will be easier.

<Operation procedure>
FIG. 20 is an example of a flowchart showing a procedure in which the electronic rear-view mirror displays an image on the display 15. In the description of FIG. 20, the difference from FIG. 8 will be mainly described. In FIG. 20, in step S50-3, the image synthesizing unit 24 widens and narrows the transparent portion 31 of the vehicle interior image 9 according to the vehicle speed.

As a method of widening and narrowing the transparent portion 31 of the vehicle interior image 9, there are the following methods.
(i) Enlarge or reduce with reference to the center of the vehicle interior image 9. The enlargement ratio is determined according to the vehicle speed, and the part that exceeds the size of the display is deleted.
(ii) A table in which a threshold value is set in association with the pixels of the transparency variable portion 32 of the vehicle interior image 9 is prepared. The threshold value in this table is compared with the current vehicle speed for each pixel, and if the vehicle speed is larger than the threshold value, the α value of the pixel is set to 0, and if the vehicle speed is equal to or less than the threshold value, the α value of the pixel is set to 255. In this case, among the pixels of the transparency variable portion 32, the smaller the threshold value is set, the closer the pixel is closer to the transparent portion 31.
(iii) A plurality of vehicle interior images 9 having different sizes of the transparent portion 31 are prepared, and the vehicle interior image 9 to be combined with the captured image 8 is switched according to the vehicle speed.

By such a method, the transparent portion 31 can be increased as the vehicle speed increases. Other processing may be the same as in FIG.

<Summary>
According to this embodiment, by increasing the transparent portion 31 according to the vehicle speed, it is easy to get a sense of distance from the vehicle behind when the vehicle speed is low, and it is easy to secure a rear view when the vehicle speed is high. ..

Although the transparency is determined with respect to the vehicle speed in this embodiment, the size of the transparent portion 31 may be determined with respect to the distance from the vehicle behind.

Further, in combination with the first embodiment, the transparency of the transparency variable portion 32 may be changed as the vehicle speed increases. Further, it may be combined with Example 3.

<Other application examples>
Although the best mode for carrying out the present invention has been described above with reference to examples, the present invention is not limited to these examples, and various modifications are made without departing from the gist of the present invention. And substitutions can be made.

For example, the configuration example shown in FIG. 6 is divided according to the main functions in order to facilitate understanding of the processing by the electronic rear-view mirror 10. The present invention is not limited by the method of dividing the processing unit or the name. The processing of the electronic rearview mirror 10 can be further divided into more processing units depending on the processing content. Further, one processing unit can be divided so as to include more processing.

The vehicle interior image acquisition unit 22 is an example of an in-vehicle image acquisition means, the vehicle speed acquisition unit 30 is an example of a speed detection means, the image composition unit 24 is an example of an image composition means, and the image output unit 25 is an example. The image output means is an example, the distance estimation unit 27 is an example of a distance detection means, and the vehicle interior image pickup device 18 is an example of an image pickup means.

10 Electronic rear-view mirror 11 Imaging device 12 Display unit 14 Board 15 Display

Claims (8)

An image pickup device that captures the rear of the vehicle and generates an image.
An in-vehicle image acquisition means for acquiring an in-vehicle image of the vehicle interior taken from the front to the rear of the vehicle,
Speed detection means to detect the current running speed of the vehicle,
An image synthesizing means for generating a composite image by increasing the transparency of the region surrounding the central portion of the vehicle interior image and synthesizing the captured image as the traveling speed detected by the speed detecting means increases.
It has an image output means for outputting the composite image synthesized by the image compositing means to a display device.
The image synthesizing means is characterized in that the size of the central portion of the vehicle interior image in which at least the transparency in which the captured image can be visually recognized is secured regardless of the traveling speed is increased as the traveling speed is increased. Image display device. At least the transparency at which the captured image can be visually recognized is ensured in the central portion of the vehicle interior image regardless of the traveling speed.
The image display device according to claim 1, wherein the image synthesizing means increases the transparency of a region surrounding a central portion of the vehicle interior image as the traveling speed increases. The image display device according to claim 1 or 2, wherein the image synthesizing means increases the transparency toward the upper side of the vehicle interior image at the same traveling speed. The image synthesizing means determines the coefficient according to the height of the pixel of the vehicle interior image by referring to the corresponding information to which the coefficient is associated with the upper side of the vehicle interior image.
The image display device according to claim 3, wherein the transparency of the pixels of the vehicle interior image is determined based on a value obtained by multiplying the coefficient by the current traveling speed of the vehicle and subtracting the value from a constant. The image display device according to any one of claims 1 to 4, wherein the vehicle interior image is an image in which a characteristic contour of the vehicle interior image is represented only by a contour line. The image display device has an image pickup means for capturing an image of the vehicle interior, and has an image pickup means.
The image synthesizing means is characterized in that the transparency of the vehicle interior image captured by the imaging means is increased as the traveling speed is increased and combined with the captured image to generate the composite image. The image display device according to any one of 4. An image pickup device that captures the rear of the vehicle and generates an image.
An in-vehicle image acquisition means for acquiring an in-vehicle image of the vehicle interior taken from the front to the rear of the vehicle,
A distance detecting means for detecting the distance to the rear vehicle traveling behind the vehicle, and
As the distance detected by the distance detecting means increases, the transparency of the vehicle interior image is increased and combined with the captured image to generate a composite image.
It has an image output means for outputting the composite image synthesized by the image compositing means to a display device.
When the distance detecting means cannot detect the vehicle behind
The image synthesizing means is an image display device characterized in that the entire vehicle interior image is made transparent and combined with the captured image to generate a composite image. The step that the image pickup device captures the rear of the vehicle and generates the captured image,
A step in which the vehicle interior image acquisition means acquires an image of the vehicle interior in which the vehicle interior is captured from the front to the rear of the vehicle.
A step in which the speed detecting means detects the current traveling speed of the vehicle,
The step of generating a composite image by the image synthesizing means increasing the transparency of the region surrounding the central portion of the vehicle interior image as the traveling speed detected by the speed detecting means increases and synthesizing with the captured image.
The image output means has a step of outputting the composite image synthesized by the image compositing means to a display device.
The image synthesizing means is characterized in that the size of the central portion of the vehicle interior image in which at least the transparency in which the captured image can be visually recognized is secured regardless of the traveling speed is increased as the traveling speed is increased. Image display method.

Images