JP2020052143A - Image display apparatus and image display method - Google Patents

Abstract

To provide an image display apparatus in which driver's visual field is prevented from being narrowed due to an image displayed together with an image taken by an imaging device.SOLUTION: An image display apparatus 10 includes: an imaging device 11 for taking an image of a rear part of a vehicle to generate a captured image; in-vehicle image obtaining means 22 for obtaining an in-vehicle image in which the inside of the vehicle is captured from a front part to the rear part of the vehicle; speed detection means 30 for detecting a current travel speed of the vehicle; image synthesizing means 24 for synthesizing the captured image, by making transparency of the in-vehicle image increase as the travel speed detected by the speed detection means becomes greater, to generate a synthesized image; and image output means 25 for outputting, to a display device, the synthesized image synthesized by the image synthesizing means.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image display device and an image display method.

  There are attempts to replace the vehicle's rearview mirror with a CMS (camera monitoring system). CMS is a technology for displaying an image captured by an imaging device on a display. The room mirror employing the CMS is hereinafter referred to as an electronic room mirror. The electronic rearview mirror has a configuration in which a display is built in a conventional rearview mirror. Conventional rearview mirrors may obstruct the rear view due to the occupants and luggage of the vehicle, but the electronic rearview mirror displays the image of the imaging device installed behind the vehicle on the display inside the rearview mirror, so the rear view Is easy to secure.

  However, when the conventional rearview mirror is replaced with an electronic rearview mirror, the size of the rear object displayed on the electronic rearview mirror due to the influence of the mounting position and the focal length of the imaging device is smaller than the size of the conventional rear object reflected on the rearview mirror. It is known that it is difficult for the driver to grasp the sense of distance to the rear object because of the difference. This phenomenon is particularly likely to occur due to a sense of distance to a rear object near the host vehicle.

  In order to reduce this inconvenience, there is known a technique in which a vehicle interior image obtained by capturing an image of a vehicle interior is prepared, and the vehicle interior image is inserted into an image of an imaging device installed behind the vehicle (for example, see Patent Document 1). ). Patent Literature 1 discloses a technique in which an image inside a vehicle is made translucent and superimposed on an image captured by an imaging device installed behind the vehicle.

  Also, a technique has been devised in which the transparency of another image displayed together with the image of the imaging device installed behind the vehicle is changed in accordance with the vehicle speed (for example, see Patent Document 2). Patent Literature 2 discloses a technique for displaying, below an image of an imaging device installed behind a vehicle, a symbol image of the host vehicle whose transparency decreases as the vehicle speed increases.

Japanese Patent No. 5115136 JP 2017-215446 A

  However, in the related art, when the vehicle speed is relatively high, there is a problem that the field of view of the driver is narrowed by the image displayed together with the image captured by the imaging device. This will be described with reference to FIG.

  FIG. 1 is a diagram illustrating a problem that occurs when an image in a vehicle compartment and an image captured by an imaging device installed behind the vehicle are combined. FIG. 1A is a diagram illustrating the synthesis of the vehicle interior image 9 and the image captured by the imaging device 11 installed behind the vehicle. The image captured by the imaging device is hereinafter referred to as a captured image 8. The central part of the vehicle interior image 9 is ensured such that the vehicle interior image 9 can be visually recognized at least. For example, it is almost completely transparent. The area where the transparency is secured in this way is referred to as a transparent portion 31 of the vehicle interior image 9. The shape of the transparent portion 31 is the same as or imitated the shape of the rear glass of the vehicle. That is, the rear glass portion in the vehicle interior image 9 is the transparent portion 31.

  Thus, when the vehicle interior image 9 is superimposed on the captured image 8, the captured image 8 is displayed on the transparent portion 31 and the vehicle interior image 9 is displayed around the captured image 8, as shown in FIG. .

  As shown in FIG. 1B, the captured image 8 and the vehicle interior image 9 are combined and displayed, so that the driver can easily grasp the size of the vehicle behind and easily grasp the sense of distance.

  However, as shown in FIG. 1C, when the vehicle is moving at a high speed, the correlation between the size of the vehicle behind and the sense of distance is weakened, and the captured image 8 and the vehicle interior image 9 are combined and displayed. The benefits of things are reduced. That is, when the vehicle speed is high, the driver should grasp a wider situation around the vehicle. However, according to the technology disclosed in Patent Document 1, the vehicle interior image 9 is always displayed translucently regardless of the vehicle speed. , Always obstructs the view. Further, in the technology disclosed in Patent Document 2, the symbol image of the own vehicle (corresponding to the vehicle interior image 9) is displayed darker as the vehicle speed is higher. Therefore, the driver's view is hindered as the vehicle speed is higher. As described above, in the related art, it has not been considered to secure a wide field of view when the vehicle speed is high.

  In view of the above problems, an object of the present invention is to provide an image display device that suppresses a driver's view from being narrowed by an image displayed together with an image captured by an imaging device.

  The present invention has been made in consideration of the above problems, and has an image capturing apparatus that captures an image of a rear side of a vehicle to generate a captured image, and an in-vehicle image obtaining unit that obtains a vehicle interior image in which a vehicle interior is captured from the front to the rear of the vehicle. Speed detecting means for detecting the current running speed of the vehicle; and increasing the transparency of the vehicle interior image as the running speed detected by the speed detecting means increases, and synthesizing the captured image with the captured image to generate a synthesized image. An image display device comprising: an image synthesizing unit that performs the image synthesis; and an image output unit that outputs the synthesized image synthesized by the image synthesizing unit to a display device.

  It is possible to provide an image display device in which a driver's view is suppressed from being narrowed by an image displayed together with an image captured by the imaging device.

FIG. 3 is a diagram illustrating a problem when an image in a vehicle interior and an image captured by an imaging device installed behind the vehicle are combined. It is an example of a figure explaining display control of a picked-up image which a display unit of an electronic room mirror displays, and a cabin image. It is a figure which shows the example of mounting of an electronic room mirror. It is an example of a schematic perspective view of a display unit. It is an example of a figure showing the schematic structure of a display unit. FIG. 3 is an example of a functional block diagram illustrating functions of a display unit. FIG. 9 is a diagram illustrating an example of a combining process. It is an example of the flowchart figure which shows the procedure in which an electronic room mirror displays an image on a display. FIG. 9 is an example of a functional block diagram illustrating functions of a display unit (Example 2). FIG. 4 is an example of a diagram for explaining a relationship between a distance to a vehicle behind and an α value. FIG. 8 is an example of a flowchart illustrating a procedure in which an electronic room mirror displays an image on a display (second embodiment). It is an example of a figure explaining control of the transparency of the height direction of a vehicle interior picture. It is an example of a figure explaining the determination method of the alpha value according to height Y of a vehicle interior image. FIG. 13 is an example of a flowchart illustrating a procedure in which an electronic room mirror displays an image on a display (Example 3). It is a figure explaining the image showing the outline of a vehicle interior image. It is a figure showing an example of an image which an electronic room mirror displays. FIG. 14 is an example of a functional block diagram illustrating functions of a display unit (Example 4). FIG. 14 is an example of a flowchart illustrating a procedure in which an electronic room mirror displays an image on a display (Example 4). It is a figure showing an example of an image which an electronic room mirror displays. FIG. 15 is an example of a flowchart illustrating a procedure in which an electronic room mirror displays an image on a display (Example 5).

  Hereinafter, as an example of an embodiment for carrying out the present invention, an electronic room mirror and an image display method performed by the electronic room mirror will be described with reference to the drawings by way of examples.

<Outline of electronic room mirror of this embodiment>
FIG. 2 is an example of a diagram illustrating the display control of the captured image 8 and the vehicle interior image 9 displayed by the display unit 12 of the electronic room mirror according to the present embodiment. The electronic room mirror of the present embodiment changes the transparency of the vehicle interior image 9 in accordance with the vehicle speed.
Vehicle speed: Medium to large → Transparency Low vehicle speed: Small to medium → Transparency large FIGS. 2A to 2C show how the transparency of the vehicle interior image 9 increases as the vehicle speed increases. That is, in FIG. 2A, the vehicle interior image 9 is almost opaque because the vehicle speed is low. In FIG. 2B, the vehicle interior image 9 is translucent because the vehicle speed is medium. In FIG. 2C, since the vehicle speed is high, the vehicle interior image 9 is almost transparent.

  Therefore, at low speed, the vehicle interior image 9 is clearly displayed, so that the driver can easily grasp the sense of distance to the vehicle behind, and the vehicle interior image 9 becomes more transparent as the vehicle speed increases, so that it is easy to secure the rear view. Become.

<About terms>
The image display device is a device that displays, on a display device, image data around the vehicle captured using the imaging device 11 installed in the vehicle. Although an electronic room mirror will be described as an example of a device that displays a captured image synthesized with a vehicle interior image, the imaging device 11 and the display device may be mounted on a moving body such as a vehicle. The image display device is not limited to the electronic room mirror as long as the captured image and some image are combined. Note that the electronic room mirror may be called an electronic rearview mirror, an electronic mirror, or the like.

  The imaging device may be mounted on a moving object other than the vehicle. For example, it may be mounted on a light vehicle such as a motorcycle or a bicycle, a wheelchair, a robot, or the like.

<Example of configuration>
FIG. 3 is a diagram illustrating an example of mounting the electronic room mirror 10. FIG. 3A shows an example of the display unit 12 installed in the vehicle interior, and FIG. 3B shows an example of the arrangement of the imaging device 11 and the display unit 12.

  As shown in FIG. 3 (a), the display unit 12 of the electronic room mirror 10 is fixed between the driver's seat and the passenger's seat and in the upper portion inside the windshield similarly to the conventional room mirror. There are countries and regions where the display unit 12 is not clearly specified in terms of mounting position, and the position shown is merely an example. Also, in countries and regions where there is a clear designation in the law, it will be installed in a position that complies with the law. The number of display units 12 is not limited to one, and a plurality of display units 12 may be mounted.

  The imaging device 11 is disposed behind the vehicle 101. It is preferable that the imaging device 11 capture an image in an optical axis direction close to the direction of light incident on a conventional room mirror. Since the conventional room mirror is arranged above the windshield and reflects the rear of the vehicle on the mirror, it is often preferable that the imaging device 11 is also arranged above the vehicle. For example, it may be installed on a roof or a roof spoiler. The horizontal direction of the imaging device 11 is preferably at or near the center of the vehicle. However, even if it is installed at the right end or the left end, it is possible to image the rear. Image processing may be performed as if the image of the imaging device 11 installed at the right end or the left end was taken near the center.

  The location of the imaging device 11 is not limited to each type of vehicle and may be different for each individual vehicle. For example, it may be around a license plate, around a vehicle manufacturer emblem, or at an appropriate place for a rear bumper.

  Conventionally, a vehicle 101 may be equipped with a back camera. The back camera is a camera that displays the rear image when moving the car backward (eg, when the driver operates the shift lever in reverse), such as when entering a garage, and the captured image is displayed on a display such as a car navigation system. There are many. In the present embodiment, the image of the back camera may be displayed on the display unit 12, or the image captured by the imaging device 11 of the electronic room mirror 10 may be displayed on the display of a car navigation system or the like as the image of the back camera. Is also good.

  Also, a relatively wide-angle camera such as a fish-eye camera may be used as the electronic rearview mirror 10 and the rear camera. That is, a part of the image captured by the wide-angle camera is displayed on the display unit 12, and another part (which may be duplicated) is displayed on a display such as a car navigation system.

  The wiring from the imaging device 11 to the display unit 12 may be routed around the ceiling or the floor, and is determined in consideration of the wiring length and routing. Further, the imaging device 11 may transmit an image to the display unit 12 by wireless communication, for example.

  Further, a vehicle speed sensor 19 is connected to the display unit 12. The vehicle speed sensor 19 is a sensor that outputs a pulse signal according to the rotation speed of the wheels to determine the current traveling speed of the vehicle. The traveling speed detected by the vehicle speed sensor 19 is used for synthesizing the vehicle interior image 9 and the captured image 8.

  The display unit 12 may be connected to a power supply line, a reverse signal line, a blinker signal line, and a video input of another imaging device (such as a side camera or a back camera). Omitted. For example, the display unit 12 can display an image of the back camera by a signal from the reverse signal line, and the display unit 12 can display an image of the side camera by a signal from the blinker signal line.

  FIG. 4 is an example of a schematic perspective view of the display unit 12. The display unit 12 has a display 15, four operation buttons 1 to 4, and a lever 17. The wiring interface and the like are omitted. In addition to the configuration shown, a memory card insertion portion such as a drive recorder may be provided, and the shape, the number of buttons, the positions of the buttons, and the like are merely examples.

  The display 15 is a flat panel display such as a liquid crystal display or an organic EL display. Rear projection type may be used. As will be described later, a half mirror is fixed on the front surface (indoor side) of the display 15 so as to overlap the display 15.

  The four operation buttons 1 to 4 are input devices for displaying a setting screen of the electronic room mirror 10 and receiving an operation from the user. OSD (On Screen Display) is known as a function for displaying a simple setting screen. However, how to display the user interface can be changed according to the resources of the display unit 12. For example, when an OS (Operation System) is installed on the substrate 14 described later, a screen display function supported by the OS can be used.

  In addition, a vehicle interior imaging device 18 is installed around the display 15 (upper part in the figure). The vehicle interior image capturing device 18 is an image capturing device that captures an image of the vehicle interior behind the vehicle from the display unit 12 (front of the vehicle). Since the vehicle interior imaging device 18 can image the interior of the vehicle in real time, it is possible to display the highly realistic interior image 9 on the display 15. On the other hand, when there is an occupant in the rear seat of the passenger compartment, the visibility may be obstructed. Therefore, it is not always necessary to take an image in real time. May be imaged. In terms of capturing one or more vehicle interior images 9 before the occupant gets on the vehicle, a driver or the like may capture one or more vehicle interior images 9 with a portable camera. When the memory card storing the vehicle interior image 9 is inserted into the display unit 12, the display unit 12 can combine the vehicle interior image 9 with the captured image 8. Further, the car navigation system may download the vehicle interior image 9 from the server. Therefore, the vehicle interior imaging device 18 may not be provided.

The function of each operation button is as follows, for example.
Operation button 1: Menu display (brightness, contrast, color temperature, language, display area settings, etc.)
Operation button 2: Selection of a certain direction (move right, move up, rotate right, increase, etc.)
Operation button 3: Selection of another direction (move left, move down, rotate left, decrease, etc.)
Operation button 4: determination The driver can set luminance, contrast, color temperature, language, etc., rotate a captured image, move a display area, and the like.

  The lever 17 is an input device for switching the power of the electronic room mirror 10 between ON and OFF. When the power is on, the display unit 12 outputs the image captured by the imaging device 11, and when the power is off, the display unit 12 does not display the image captured by the imaging device 11. For this reason, outside light is reflected by the half mirror, and the display unit 12 becomes a mirror for projecting the interior of the room like a normal mirror. This will be described with reference to FIG.

  FIG. 5 is an example of a diagram showing a schematic structure of the display unit 12. FIG. 5A shows the operation of the half mirror 16 when the power is on, and FIG. 5B shows the operation of the half mirror 16 when the power is off. For example, assuming that the transparency of the half mirror 16 is T% (for example, 50 to 95%), T% light of the display 15 passes through the half mirror 16 in a power-on state. Even if only T% of light passes, the brightness of the display 15 is brighter than that of the vehicle interior, so that the driver can see the image displayed on the display 15.

  Since the display 15 does not emit light when the power is off, the inside of the half mirror is dark. Since the light in the vehicle interior is stronger than the inside of the half mirror, the intensity of the light reflected by the half mirror 16 is also higher. As a result, the transmitted light from the inside of the half mirror 16 is shaded by the bright reflected light on the passenger compartment side, and thus looks like a mirror from the passenger compartment side. Therefore, the driver can easily switch between the image displayed on the display 15 and the reflection of the room light by the ordinary mirror by the operation of swinging the lever forward or backward. For example, when the incident light from the rear of the vehicle is strong on a west day and the visibility of the display 15 is reduced, it is possible to switch to a normal mirror.

  As shown in FIG. 5, the display unit 12 has a substrate 14, which controls the imaging device 11 and performs image processing such as cutting out a display area from the captured image 8. The board 14 has a general computer function, and has a CPU, a RAM, a ROM, a flash memory, an input / output I / F, and the like.

<Display unit functions>
The function of the display unit 12 will be described with reference to FIG. FIG. 6 is an example of a functional block diagram illustrating functions of the display unit 12. The display unit 12 includes an image acquisition unit 21, a vehicle interior image acquisition unit 22, an image cutout unit 23, an image synthesis unit 24, an image output unit 25, and a vehicle speed acquisition unit 30. These functions of the display unit 12 are functions or means realized by the CPU of the substrate 14 of the display unit 12 executing a program.

  The board 14 performs general control of the imaging device 11. For example, the substrate 14 performs exposure control for controlling at least one of the shutter speed and the sensitivity (gain amount) so that the image has an appropriate brightness. In addition, by making the respective ratios of R, G, and B equal to each other, which is called auto white balance control (AWB control), the color of the white portion and the color portion of the subject changes depending on the color temperature of the light source. Is performed. In addition, a drive signal is generated so that the imaging device can perform an imaging operation at an appropriate timing, or a clock signal is supplied to an A / D converter. In addition, processing for removing noise, adjusting the sharpness of an image by extracting and enhancing only edge components, and the like may be performed.

  The image acquisition unit 21 sequentially acquires image data from the imaging device 11 at a predetermined frame rate. Thus, the image is assumed to be a moving image, but the image may be a still image. In the present embodiment, this image data is referred to as a captured image 8.

  The vehicle interior image acquisition unit 22 acquires the vehicle interior image 9 stored in the vehicle interior image storage unit 26 or the vehicle interior image 9 captured by the vehicle interior imaging device 18. The vehicle interior image storage unit 26 is the above-mentioned memory card or the flash memory included in the board 14.

  The image clipping unit 23 clips a display area from the captured image 8. The display area is an area of the image data that the display 15 displays. This is performed because the size (number of pixels) of the captured image is larger than the size (number of pixels) of the display 15. The display area is image data having substantially the same size as the display 15. The trimming process may be called trimming.

  The vehicle speed acquisition unit 30 periodically or repeatedly acquires the current traveling speed (vehicle speed) from the vehicle speed sensor 19. The vehicle speed sensor 19 only needs to be able to estimate the vehicle speed, and the vehicle speed acquisition unit 30 may estimate the vehicle speed from, for example, a difference in position information detected by a GPS (Global Positioning System).

  The image combining unit 24 combines the vehicle interior image 9 and the captured image 8 according to the vehicle speed. That is, as the vehicle speed increases, the transparency of the vehicle interior image 9 is increased and the image 9 is synthesized with the captured image 8. An image in which the vehicle interior image 9 and the captured image 8 are combined is referred to as a combined image.

  The image output unit 25 sends the composite image to the display 15 and causes the display 15 to display the composite image.

<About synthesis processing>
An example of the combining process will be described with reference to FIG. FIG. 7A is an example of a diagram illustrating the synthesis of the vehicle interior image 9 and the captured image 8. The vehicle interior image 9 is superimposed on the near side, and the captured image 8 is superimposed on the rear side in this order. The vehicle interior image 9 has a transparent portion 31 that is always transparent, and a transparency variable portion 32 that changes from almost completely opaque to almost completely transparent according to the vehicle speed as a region surrounding the transparent portion 31.

  When the vehicle interior image 9 is prepared in advance, a transparent portion 31 imitating the shape of the rear glass of the vehicle is set in the center of the vehicle interior image 9 in advance. That is, 0 is set to the α value of the pixel corresponding to the shape of the rear glass. The α value will be described later. When the vehicle interior image 9 is captured in real time, pixels corresponding to the shape of the rear glass of the vehicle are predetermined, and the display unit 12 sets the α value of the pixel corresponding to the shape of the rear glass in the vehicle interior image 9 to 0. Set. The pixel corresponding to the shape of the rear glass of the vehicle may be arbitrarily set by the driver or the like, or may be set by the vehicle interior image acquiring unit 22 by utilizing the fact that the brightness of the rear glass is brighter than others. Good.

  When the vehicle interior image 9 is prepared in advance, the vehicle interior image 9 is preferably an image captured by the same vehicle as the vehicle on which the display unit 12 is mounted. It may be prepared.

  In the present embodiment, the transparency of the transparency changing unit 32 is represented by an alpha value (α value). The α value is transparency information set for each pixel in image data handled by a computer. As the α value, 0 represents completely transparent (colorless), and 255 (in the case of 8 bits) represents completely opaque. Accordingly, the α value of the transparent portion 31 is always 0 (or a value small enough to be regarded as transparent), and the α value of the transparency varying portion 32 takes a value of 255 to 0.

  FIG. 7B shows an example of the relationship between the α value of the transparency varying unit 32 and the vehicle speed. The α value is set so that the transparency changing unit 32 becomes completely transparent at the vehicle speed V = Vmax. Vmax may be set in advance as a vehicle speed at which a wider field of view is prioritized over a sense of distance. The driver may set a desired vehicle speed (Vmax) from the OSD. For example, Vmax = 30 km / h to 80 km / h can be considered, but the value is not limited to this value.

  As shown in FIG. 7B, if the vehicle speed is determined, the α value is uniquely determined. Therefore, the image combining unit 24 can combine the captured image 8 and the vehicle interior image 9. The relationship between the vehicle speed and the α value may be a straight line 110, a curve 120, or a curve 130. Also, the correspondence may be established by a table. Further, instead of making the vehicle interior image 9 completely transparent at the vehicle speed Vmax, the vehicle interior image 9 can be made slightly opaque. Similarly, the vehicle interior image 9 at the vehicle speed 0 can be made slightly transparent instead of completely opaque.

The image combining unit 24 combines the captured image 8 and the vehicle interior image 9 as described below. Combining using the α value in this manner is called alpha blending. The combining is performed for each pixel.
When α = 0, only the pixels of the captured image 8 are drawn. When α = 255, only the vehicle interior image 9 is drawn. When α = 254 to 1, the pixel value calculated by the following equation is drawn. Pixel value of image 9+ (255−α) × pixel value of captured image 8/256 ”(1)
By synthesizing the captured image 8 and the vehicle interior image 9, the vehicle interior image 9 gradually becomes transparent and the captured image 8 becomes visible when the vehicle speed increases.

<Operation procedure>
FIG. 8 is an example of a flowchart illustrating a procedure in which the electronic room mirror displays an image on the display 15. 8 is repeatedly executed while the power of the electronic room mirror is ON.

  First, the image acquisition unit 21 acquires image data (captured image 8) from the imaging device 11 (S10). The captured images 8 are transmitted one after another from the imaging device 11.

  The image clipping unit 23 clips the display area from the captured image 8 (S20). The cutting of the display area may be performed after the composition. In this case, it is often preferable to cut out the display area before the composition because processing for specifying the display area from the captured image 8 is required.

  Next, the vehicle interior image acquisition unit 22 acquires the vehicle interior image 9 (S30). The vehicle interior image 9 may be stored in the vehicle interior image storage unit 26 or may be an image captured by the vehicle interior imaging device 18. In the vehicle interior image 9 stored in the vehicle interior image storage unit 26, an area where the α value is 0 (transparent part 31) and an area where the α value changes from 255 to 0 (transparency variable part 32) are set in advance. When the vehicle interior imaging device 18 captures an image, the vehicle interior image acquisition unit 22 sets, for each pixel of the vehicle interior image 9, an area where the α value is 0 and an area that changes from 255 to 0 to the vehicle interior image 9. .

  The vehicle speed acquisition unit 30 acquires the current vehicle speed from the vehicle speed sensor 19 (S40). The fluctuation of the vehicle speed directly affects the transparency of the vehicle interior image 9, and the moving average of the vehicle speed may be used to prevent the vehicle interior image 9 from blinking.

  Next, the image combining unit 24 determines the α value of the transparency varying unit 32 according to the vehicle speed (S50). The α value of the transparent portion 31 is 0 (constant).

  The image combining unit 24 combines the captured image 8 and the vehicle interior image 9 with the α value to generate a combined image (S60). The image output unit 25 displays the composite image on the display 15 (S70).

<Summary>
As described above, the electronic room mirror 10 of the present embodiment clearly displays the vehicle interior image 9 at low speeds, so that the driver can easily grasp the sense of distance to the vehicle behind, and the vehicle interior image 9 increases as the vehicle speed increases. Since it is transparent, it is easy to secure a rear view.

  In the first embodiment, the transparency is changed according to the vehicle speed. In the present embodiment, an electronic rearview mirror that changes the transparency according to the distance to the vehicle behind will be described. Generally, when the distance to the rear vehicle is short, the vehicle speed is low, and when the distance to the rear vehicle is long, the vehicle speed is high. Therefore, the distance to the rear vehicle can be used instead of the vehicle speed.

<About the functions of the display unit>
FIG. 9 is an example of a functional block diagram illustrating functions of the display unit 12 according to the present embodiment. Regarding the description of FIG. 9, components denoted by the same reference numerals in FIG. 6 perform similar functions, and thus only the main components of the present embodiment will be described in some cases.

  The display unit 12 of the present embodiment has the distance estimating unit 27 and does not have the vehicle speed obtaining unit 30. Therefore, wiring for connecting the vehicle speed sensor 19 and the display unit 12 becomes unnecessary. The distance estimation unit 27 recognizes the rear vehicle from the captured image 8 and estimates the distance to the rear vehicle based on how large the rear vehicle is being imaged. For example, machine learning is used for recognition of the rear vehicle, and an identification model for identifying the rear vehicle is generated in advance by machine learning. Approximate vehicle types can be determined by machine learning. The distance estimating unit 27 has a table in which the image pickup size of the rear vehicle (the length and area of the diagonal line of the circumscribed rectangle of the rear vehicle in the captured image, the area, and the like) and the distance are associated with each other, and refers to the table. This estimates the distance to the vehicle behind. The distance information is sent to the image combining unit 24.

  Instead of estimating the distance to the rear vehicle by such image processing, the distance to the rear vehicle may be measured by millimeter wave radar or lidar (Light Detection and Ranging, Laser Imaging Detection and Ranging).

  The image combining unit 24 determines an α value according to the distance information, and combines the captured image 8 and the vehicle interior image 9. Other functions may be the same as those in the first embodiment.

<Determining the α value according to the distance to the vehicle behind>
FIG. 10 is an example of a diagram for explaining the relationship between the distance to the vehicle behind and the α value. The α value is set so that the transparency changing unit 32 becomes completely transparent at the vehicle speed L = Lmax. Lmax may be set in advance as a distance in which the breadth of the field of view is prioritized over the sense of distance. Further, the driver may set a desired distance from the OSD. For example, it is conceivable that Lmax = 30 m to 50 m, but it is not limited to this value.

  As shown in FIG. 10, if the distance is determined, the α value is uniquely determined. Therefore, the image combining unit 24 can combine the captured image 8 and the vehicle interior image 9. The relationship between the distance and the α value may be a straight line 110, a curve 120, or a curve 130.

  When the distance estimating unit 27 cannot detect the rear vehicle (when there is no rear vehicle), it is not necessary for the driver to grasp the sense of distance, and the transparency changing unit 32 of the vehicle interior image 9 may be transparent. Thereby, when the rear vehicle is not detected, it becomes easy to secure the rear view.

<Operation procedure>
FIG. 11 is an example of a flowchart illustrating a procedure in which the electronic room mirror displays an image on the display 15. In the description of FIG. 11, differences from FIG. 8 will be mainly described. The processing in steps S110 to S130 may be the same as steps S10 to S30 in FIG.

  When the vehicle interior image 9 is acquired in step S130, the distance estimating unit 27 performs image processing on the captured image 8 to estimate the distance to the vehicle behind (S140). Note that the distance detection is performed periodically or repeatedly. The distance estimating unit 27 sends the distance to the rear vehicle to the image synthesizing unit 24 when a rear vehicle is detected, and sends the fact to the image synthesizing unit 24 when no rear vehicle is detected.

  In the process of estimating the distance to the rear vehicle, the image synthesis unit 24 determines whether or not the rear vehicle has been detected based on the notification from the distance estimation unit 27 (S150).

  If the determination in step S150 is No, the image composition unit 24 sets the α value of the transparency varying unit 32 to 0 (S190). As a result, the transparency changing section 32 becomes transparent, and when the rear vehicle is not detected, it is easy to secure the rear view.

  If the determination in step S150 is Yes, the image synthesizing unit 24 determines the α value of the transparency changing unit 32 according to the distance to the vehicle behind (S160). Steps S170 to S180 may be the same as steps S60 to S70 in FIG.

<Summary>
In the present embodiment, the same effects as in the first embodiment can be obtained by image processing without using the vehicle speed sensor 19.

  In the first and second embodiments, the entire vehicle interior image 9 is made transparent according to the vehicle speed or the distance to the vehicle behind, but it is effective to display the lower side of the vehicle interior image 9 to grasp the sense of distance. It is known that there is. Therefore, in this embodiment, an electronic room mirror that can control the transparency in accordance with the height direction of the vehicle interior image 9 will be described.

<Display example of the electronic room mirror of the present embodiment>
FIG. 12 is an example of a diagram for explaining the control of the transparency of the vehicle interior image 9 in the height direction. FIG. 12A shows the transparency of the vehicle interior image 9 when the vehicle speed is low (for example, 0), FIG. 12B shows the transparency of the vehicle interior image 9 when the vehicle speed is medium, and FIG. Indicates the transparency of the vehicle interior image 9 when the vehicle speed is high (Vmax). For convenience of drawing, FIG. 12 shows the transparency of the transparency varying unit 32 in shaded density. That is, a high density of shading means that the transparency is low (α value is large), and a low density of shading means that the transparency is high (low α value).

  In the vehicle interior image 9 in FIG. 12A, the vehicle speed is low, so that the transparency of the transparency changing unit 32 is low without change in the height direction (for example, the α value is 255 at vehicle speed 0).

  In the vehicle interior image 9 of FIG. 12B, the vehicle speed is medium, and the transparency of the transparency changing unit 32 is higher in the upper direction and lower in the lower direction. For example, the α value is small at the upper end, the α value is middle at the interruption, and the α value is large at the lower end. As described above, at the same vehicle speed, the transparency of the transparency varying unit 32 is higher in the upper part of the vehicle interior image 9.

  In the vehicle interior image 9 of FIG. 12C, the upper part of the transparency changing unit 32 is almost transparent because the vehicle speed is high (Vmax), and the vehicle interior image 9 remains only below. Further, the transparency of the lower vehicle interior image 9 is higher than that of the lower portion of FIG. 12B (the α value is smaller).

  In this way, by displaying the upper part of the vehicle interior image 9 with a greater transparency than the lower part thereof with respect to the vehicle speed, only the lower part of the vehicle interior image 9 is emphasized, and it is possible to secure a view while easily grasping the sense of distance. .

<Method of determining α value according to height Y of vehicle interior image>
Next, a method of determining an α value according to the height Y of the vehicle interior image 9 will be described with reference to FIG. In the present embodiment, since only the method of determining the α value is different, the description will be made with reference to the functional block diagram of FIG.

  FIG. 13 is an example of a diagram illustrating a method for determining an α value according to the height Y of the vehicle interior image 9. FIG. 13A shows the relationship between the vertical direction of the vehicle interior image 9 and the height Y. As shown in FIG. 13A, it is assumed that the height Y increases as going upward in the vehicle interior image 9.

  FIG. 13B is a diagram illustrating an example of the relationship between the coefficient K for obtaining the α value and the height Y. As shown by the straight line 150 in FIG. 13B, the coefficient K (absolute value) increases as the height Y increases. Note that the coefficient K has an intercept so that K = 0 when Y = 0. FIG. 13B shows an example of the correspondence information.

FIG. 13C is an example of a diagram for explaining the relationship between the vehicle speed V and the α value according to the height Y. In this embodiment, the α value is calculated as follows.
α = 255-K × V
That is, the value obtained by subtracting the product of the coefficient K and the vehicle speed V from the constant 255 is the α value. As described above, the coefficient K is a proportional coefficient between the vehicle speed V and the α value. As the K increases, the α value decreases when the vehicle speed increases. Since K increases as the height Y increases, the higher the vehicle speed, the faster the vehicle interior image 9 becomes transparent.

  The coefficient K is set such that, for example, when the height Y = Ymax, the α value becomes 0 at the vehicle speed V = Vmax (K = 255 / Vmax). In this case, the upper part of the vehicle interior image 9 is completely transparent at the vehicle speed V = Vmax, but the lower part of the vehicle interior image 9 is not completely transparent even at the vehicle speed V = Vmax. It is easier to get a sense of distance.

  In addition, as shown in FIG. 13B, the degree of transparency can be adjusted above and below the vehicle interior image 9 by changing the coefficient K with respect to the height Y. For example, to make the upper part of the vehicle interior image 9 transparent at a lower vehicle speed, the coefficient K may be obtained as indicated by a dotted line 170 where the coefficient K is larger than the height Y. If it is desired to make the lower part of the vehicle interior image 9 opaque at a higher vehicle speed, the coefficient K may be obtained as indicated by a dotted line 160 where the coefficient K is smaller than the height Y. In addition, the coefficient K may be determined in any manner.

<Operation procedure>
FIG. 14 is an example of a flowchart illustrating a procedure in which the electronic room mirror displays an image on the display 15. In the description of FIG. 14, differences from FIG. 8 will be mainly described. FIG. 14 differs from FIG. 8 in the processing of step S50-2.

  In step S50-2, the image combining unit 24 determines a coefficient K for each height Y. Note that if the height Y is the same, the coefficient K is a constant (because it does not change) and may be determined in advance. Next, the image combining unit 24 calculates an α value for each pixel of the transparency varying unit 32 based on the coefficient K and the vehicle speed V determined by the height Y of the pixel. Subsequent processing may be the same as in FIG.

<Summary>
As described above, the electronic room mirror of the present embodiment increases the transparency of the upper part of the interior image 9 from the lower part at the same vehicle speed, thereby making the upper part transparent while displaying the lower part of the interior image 9. Because it is possible, it is easy to achieve both a sense of distance and a secure view.

  In the present embodiment, the transparency is determined for the vehicle speed, but the transparency may be determined for the distance to the vehicle behind.

  In the first to third embodiments, the vehicle interior image 9 and the captured image 8 are combined. In the present embodiment, an electronic room mirror 10 that combines the image representing the contour of the vehicle interior image 9 and the captured image 8 will be described.

<Image representing the outline of the cabin image>
An image representing the outline of the vehicle interior image 9 will be described with reference to FIG. Hereinafter, an image representing the outline of the vehicle interior image 9 is referred to as an outline image. FIG. 15A shows a vehicle interior image 9 and FIG. 15B shows an outline image. As shown in the figure, the outline image is an image in which the outline that is a feature of the vehicle interior image 9 is represented only by the black and white outline 180.

  For example, the contour image in which only the position and shape of the rear glass and other characteristic shapes representing the structure of the vehicle interior are drawn by the contour line 180 is shown. The part without the contour 180 is always transparent. That is, in this embodiment, the portion without the contour 180 is the transparent portion 31. Since the transparency of the outline 180 changes according to the vehicle speed, the outline 180 is the transparency variable unit 32 of the present embodiment. Therefore, the driver can grasp the sense of distance to the vehicle behind by the contour image, and can secure a view by the transparent portion of the contour image.

<Example of image displayed by electronic room mirror>
FIG. 16 shows an example of an image displayed by the electronic room mirror of the present embodiment. The electronic room mirror according to the present embodiment changes the transparency of the contour image according to the vehicle speed as in the first embodiment. The portion other than the outline (transparent portion 31) is always transparent.
Vehicle speed: Medium to large → Transparency Low vehicle speed: Small to medium → Transparency large
FIGS. 16A to 16C show how the transparency of the contour image increases as the vehicle speed increases. That is, in FIG. 16A, since the vehicle speed is low, the outline 180 of the outline image is almost opaque. In FIG. 16B, since the vehicle speed is medium, the outline 180 of the outline image is translucent. In FIG. 16C, since the vehicle speed is high, the vehicle interior image 9 is almost transparent.

  Therefore, the reality as in the vehicle interior image 9 is reduced, but it is easy to grasp the sense of distance to the vehicle behind while giving priority to securing the rear view.

  Note that the outline image may be switched instead of controlling the transparency. Several outline images having different thicknesses of the outline 180 are prepared in advance, and the image combining unit 24 switches the outline images according to the vehicle speed. In this case, the outline image may be simply superimposed on the captured image 8 instead of being synthesized by the alpha blend. Therefore, the synthesizing process becomes unnecessary, and the processing load on the CPU of the substrate 14 can be reduced.

<About the functions of the display unit>
FIG. 17 is an example of a functional block diagram illustrating functions of the display unit 12 according to the present embodiment. In the description of FIG. 17, differences from FIG. 6 will be mainly described. The display unit 12 in FIG. 17 has a contour image storage unit 29. The outline image storage unit 29 stores the above outline image in advance. However, the vehicle interior image acquisition unit 22 may generate a contour image from the vehicle interior image 9 captured by the vehicle interior imaging device 18 as in the first embodiment. Since the contour 180 corresponds to the edge of the image, the contour image can be dynamically generated by processing such as edge detection.

  The image combining unit 24 changes the α value of the contour line of the contour image according to the vehicle speed, as in the first embodiment. That is, as shown in FIG. 7B, the α value is determined according to the vehicle speed, and a pixel overlapping the contour 180 in the captured image 8 is synthesized with the contour 180 using Expression (1). Other functions may be the same as those in FIG.

<Operation procedure>
FIG. 18 is an example of a flowchart illustrating a procedure in which the electronic room mirror displays an image on the display 15. In the description of FIG. 18, differences from FIG. 8 will be mainly described. In FIG. 18, the vehicle interior image 9 is changed to a contour image in steps S30 and S60. Other processes may be the same as those in FIG.

<Summary>
As described above, the electronic room mirror according to the present embodiment combines the outline image representing the outline of the vehicle interior image 9 with the captured image 8, so that priority is given to securing the rear view and the sense of distance to the rear vehicle is obtained. It becomes easy to grasp.

  In this embodiment, the transparency is determined for the vehicle speed, but the transparency may be determined for the distance to the vehicle behind.

  Further, the third embodiment may be applied to the present embodiment. That is, when the vehicle speed increases, the upper part of the contour image is made transparent faster than the lower part. By doing so, it becomes easier to secure the field of view while easily grasping the sense of distance to the vehicle behind.

  In the present embodiment, an electronic room mirror for increasing the size of the transparent portion 31 of the vehicle interior image 9 according to the vehicle speed will be described.

<Example of image displayed by electronic room mirror>
FIG. 19 shows an example of an image displayed by the electronic room mirror of the present embodiment. In the electronic rearview mirror of the present embodiment, the transparent portion 31 that is always transparent increases as the vehicle speed increases.
Vehicle speed: Medium to large → Transparent part 31 Small vehicle speed: Small to medium → Transparent part 31 Large
FIGS. 19A to 19C show that the transparent portion 31 becomes larger as the vehicle speed becomes higher. That is, in FIG. 19A, the transparent portion 31 is small because the vehicle speed is low. In FIG. 19B, since the vehicle speed is medium, the transparent portion 31 is slightly larger. In FIG. 19C, since the vehicle speed is high, the transparent portion 31 is maximized.

  Therefore, when the vehicle speed is low, the vehicle interior image 9 occupies a large area, so that it is easy to grasp the distance to the vehicle behind. When the vehicle speed is high, the area occupied by the vehicle interior image 9 decreases, so that the rear view is secured. It will be easier.

<Operation procedure>
FIG. 20 is an example of a flowchart illustrating a procedure in which the electronic room mirror displays an image on the display 15. In the description of FIG. 20, differences from FIG. 8 will be mainly described. In FIG. 20, in step S50-3, the image synthesis unit 24 widens and narrows the transparent portion 31 of the vehicle interior image 9 according to the vehicle speed.

As a method of widening or narrowing the transparent portion 31 of the vehicle interior image 9, there is the following method.
(i) Enlarge or reduce based on the center of the vehicle interior image 9. The enlargement ratio is determined according to the vehicle speed, and the portion exceeding the display size is deleted.
(ii) A table is prepared in which threshold values are set in association with the pixels of the transparency changing unit 32 of the vehicle interior image 9. The threshold value in this table is compared with the current vehicle speed for each pixel. If the vehicle speed is higher than the threshold value, the α value of the pixel is set to 0, and if the vehicle speed is equal to or less than the threshold value, the α value of the pixel is set to 255. In this case, a smaller threshold value is set for a pixel closer to the transparent portion 31 among the pixels of the transparency variable portion 32.
(iii) A plurality of vehicle interior images 9 having different sizes of the transparent portion 31 are prepared, and the vehicle interior image 9 to be combined with the captured image 8 is switched according to the vehicle speed.

  According to such a method, the transparent portion 31 can be enlarged as the vehicle speed increases. Other processes may be the same as those in FIG.

<Summary>
According to this embodiment, by increasing the size of the transparent portion 31 according to the vehicle speed, it is easy to grasp the sense of distance to the rear vehicle when the vehicle speed is low, and it is easy to secure the rear view when the vehicle speed is high. .

  In the present embodiment, the transparency is determined with respect to the vehicle speed, but the size of the transparent portion 31 may be determined with respect to the distance to the vehicle behind.

  Further, in combination with the first embodiment, the transparency of the transparency varying unit 32 may be changed as the vehicle speed increases. Further, it may be combined with the third embodiment.

<Other application examples>
As described above, the best mode for carrying out the present invention has been described using the embodiment. However, the present invention is not limited to such embodiment at all, and various modifications may be made without departing from the gist of the present invention. And substitutions can be added.

  For example, the configuration examples in FIG. 6 and the like are divided according to main functions in order to facilitate understanding of processing by the electronic room mirror 10. The present invention is not limited by the way of dividing the processing unit or the name. The processing of the electronic room mirror 10 can be divided into more processing units according to the processing contents. Further, it is also possible to divide the processing unit so that one processing unit includes more processing.

  In addition, the vehicle interior image acquiring unit 22 is an example of an in-vehicle image acquiring unit, the vehicle speed acquiring unit 30 is an example of a speed detecting unit, the image combining unit 24 is an example of an image combining unit, and the image output unit 25 is an example. This is an example of an image output unit, the distance estimating unit 27 is an example of a distance detection unit, and the vehicle interior imaging device 18 is an example of an imaging unit.

DESCRIPTION OF SYMBOLS 10 Electronic room mirror 11 Imaging device 12 Display unit 14 Substrate 15 Display

Claims (10)

An imaging device that images the rear of the vehicle to generate a captured image,
In-vehicle image acquisition means for acquiring a vehicle interior image in which the vehicle interior is imaged from the front to the rear of the vehicle,
Speed detection means for detecting the current traveling speed of the vehicle,
Image synthesizing means for increasing the transparency of the vehicle interior image as the traveling speed detected by the speed detecting means is increased and synthesizing with the captured image, and generating a synthesized image,
Image output means for outputting the synthesized image synthesized by the image synthesis means to a display device,
An image display device comprising: In the central portion of the vehicle interior image, at least transparency at which the captured image can be visually recognized is ensured regardless of the traveling speed,
The image display device according to claim 1, wherein the image combining unit increases the transparency of a region surrounding a central portion of the vehicle interior image as the traveling speed increases.   3. The image display device according to claim 1, wherein the image synthesizing unit increases the transparency as the position of the vehicle interior image increases at the same traveling speed. 4. The image synthesizing unit determines the coefficient according to the height of the pixel of the vehicle interior image with reference to the correspondence information in which the larger the coefficient is associated with the upper part of the vehicle interior image,
The image display device according to claim 3, wherein the transparency of the pixel of the vehicle interior image is determined based on a value obtained by subtracting a value obtained by multiplying the coefficient by a vehicle speed from a constant.   The image display device according to any one of claims 1 to 4, wherein the vehicle interior image is an image in which a contour characteristic of the vehicle interior image is represented only by a contour line. The display device has an imaging unit that captures the interior image of the vehicle,
The image synthesizing unit generates the synthesized image by increasing the transparency of the vehicle interior image captured by the imaging unit as the traveling speed increases, and synthesizing the image with the captured image. 5. The image display device according to any one of 4.   3. The image combining unit according to claim 2, wherein the image combining unit increases the size of the central portion of the vehicle interior image in which transparency is secured regardless of the traveling speed, as the traveling speed increases. 4. Image display device. Having distance detection means for detecting the distance to the rear vehicle running behind the vehicle,
8. The image combining unit according to claim 1, wherein the larger the distance detected by the distance detector, the greater the transparency of the interior image of the vehicle and the combined image with the captured image to generate a combined image. 9. The image display device according to any one of the above. When the distance detecting means cannot detect the rear vehicle,
9. The image display device according to claim 8, wherein the image synthesizing unit makes the entire vehicle interior image transparent and synthesizes the captured image with the captured image to generate a synthesized image. An imaging device for imaging the rear of the vehicle to generate a captured image;
A step in which the in-vehicle image obtaining means obtains an in-vehicle image in which the inside of the vehicle is imaged from the front to the rear of the vehicle;
Speed detecting means for detecting a current traveling speed of the vehicle;
Image synthesis means for synthesizing the captured image by increasing the transparency of the vehicle interior image as the traveling speed detected by the speed detection means is higher, and generating a synthesized image;
Image output means for outputting the synthesized image synthesized by the image synthesis means to a display device;
An image display method comprising: