JP2022048281A - Electronic mirror image synthesizing apparatus and image synthesizing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an electronic mirror for performing a process of replacing with an effective image (supplementary image) by making an image of an obstacle semitransparent even if there is an obstacle for blocking a part of a field of view.

SOLUTION: The image synthesizing device is configured to make an image of a blind spot of a side camera of a vehicle semi-transparent and perform projective transformation on upper and lower areas of a rear camera image into images of different virtual planes and synthesize them with the semi-transparent portion.

SELECTED DRAWING: Figure 3

COPYRIGHT: (C)2022,JPO&INPIT

Description

The embodiment relates to an electronic mirror image compositing device and an image compositing method.

For example, there is a so-called electronic mirror in which a plurality of cameras are mounted on a car and an image from the cameras can be displayed on a display instead of a conventional image of a rear-view mirror or a side mirror.

Japanese Patent No. 4883977 Japanese Patent No. 5500877 Japanese Unexamined Patent Publication No. 2011-21186 Japanese Unexamined Patent Publication No. 2014-197818

In the electronic mirror system, the plurality of cameras have different mounting positions with respect to one car. Therefore, when multiple cameras each shoot the scenery behind the car, even if it is an effective area that can be shot according to the field of view of one camera, a blind spot area that cannot be shot according to the field of view of the other camera. There is an (invalid area).

As described above, in the electronic mirror, when a plurality of cameras each capture a landscape behind a car, the cameras generate an image of an effective area and an image of an invalid area.

Therefore, in the present embodiment, even if there is a blocking object that blocks a part of the visual field, the image of the blocking object can be made translucent and replaced with an effective image (supplementary image) to obtain a natural composite image. , An object of the present invention is to provide a video compositing apparatus and a video compositing method for an electronic mirror.

According to one embodiment, in a video synthesizer of an electronic mirror mounted on a vehicle and displaying an image of the rear of the vehicle.
A rear camera that acquires the first image of the first landscape behind the vehicle from the first position by the first field of view.
From the second position, which is in front of the vehicle from the first position, the second field of view is such that a landscape blocker, which is a side portion of the own vehicle, is partially arranged toward the direction of the first landscape. With a side camera that captures images,
A semi-transparent processing unit that converts the image of the landscape blocker included in the second image into a semi-transparent image, and
An edge processing unit capable of adding a line to the contour of the translucent image in the second image, and an edge processing unit.
A projecting conversion unit that is a part of the first image, is projected to the viewpoint position of the side camera, and obtains a projecting converted image corresponding to the semi-transparent image portion of the second image.
The
The projective conversion image is
A part of the first projective conversion image in which the image of the viewpoint of the rear camera is projected and converted into the image of the viewpoint of the side camera based on the far plane position behind the vehicle by the first projective conversion unit. ,
The second projection conversion unit projected and converted the image of the viewpoint of the rear camera into the image of the viewpoint of the side camera based on the ground position closer to the shooting position of the side camera than the far plane position. Including a part of the second projective conversion image
When the horizontal line direction of the rear camera and the side camera is the horizontal direction and the vertical line direction is the vertical direction.
The upper region of the projective transformation image corresponding to the translucent image portion is the part of the first projective conversion image.
The lower region of the projective transformation image corresponding to the translucent image portion is the part of the second projective conversion image.
Further, in the first projective transformation or the second projective transformation, the distant plane position and the ground position are adjusted according to the settings according to the traveling speed of the vehicle.
An electronic mirror image synthesizer provided with an outlet for blowing compressed air in front of the lenses of the rear camera and the side camera is provided.

FIG. 1 is a plan view showing a car. FIG. 2 is a side view showing the vehicle of FIG. FIG. 3 is a block configuration diagram of an electronic mirror system to which one embodiment is applied. FIG. 4 is an image diagram shown for explaining a basic operation example of the electronic mirror system of FIG. FIG. 5 is an image diagram shown for explaining an operation example accompanied by a projective transformation of the electronic mirror system of FIG. FIG. 6 is an explanatory diagram shown to explain the basic principle for projective transformation. FIG. 7 is an explanatory diagram showing an example of a determinant for projective transformation. FIG. 8 is an explanatory diagram shown for explaining the luminance averaging process of the image of the vehicle rule surface. FIG. 9 is an explanatory diagram showing another example of generating an image synthesized in the complete blind spot region. FIG. 10 is an explanatory diagram showing an example in which a pseudo image is synthesized in a complete blind spot region. FIG. 11 is an explanatory diagram showing another example in which a pseudo image is synthesized in the complete blind spot region. FIG. 12 is an explanatory diagram showing an example in which a vehicle traveling on a road changes the distance from the vehicle to a reference plane position (distant plane position and ground position) according to the speed. FIG. 13 is a diagram showing an image example of an electronic mirror when the distance from the vehicle to the reference (virtual) plane position is extended to the distant plane position. FIG. 14 is a diagram showing an image example of an electronic mirror when the distance from the vehicle to the reference (virtual) plane position is shortened to the short-distance plane position.

Hereinafter, embodiments will be described with reference to the drawings. FIG. 1 shows the car 100 in a plane. FIG. 2 shows the car of FIG. 1 from the side.

In the car 100, the first camera 101 is arranged on the trunk room at the rear or near the license plate, and can photograph the scenery behind. Further, in the car 100, the second camera 102R and the third camera 102L are arranged at the positions of the side mirrors, respectively, and these can also photograph the scenery behind.

As described above, the plurality of cameras 101, 102R, and 102L have different mounting positions with respect to one vehicle 100. Therefore, when a plurality of cameras each photograph the scenery behind the car, even if the effective area can be photographed according to the field of view W1 of one camera 101, the field of view (for example, W2) of the other cameras 102R or 102L may be used. If so, there is a blind spot area (invalid area = field of view W3) that cannot be photographed.

That is, according to one embodiment of the image synthesizer of the present electronic mirror, the first camera 101 acquires the first image (rear image) of the first landscape from the first position (for example, the trunk room position) by the first field of view W1. do. The second camera 102R has a landscape blocker (side surface of the own vehicle) in the direction of the first landscape from the second position (lateral side position of the driver's seat, side door position) different from the first position (trunk room position). ) Is partially arranged in the second visual field W2, and the second image is acquired. For this reason, a part of the second image includes an image of the side surface of the own vehicle.

The video processing apparatus 200, which will be described later, connects a part of the first video as a supplementary video to a part of the second video. In this case, the image of the landscape barrier (own vehicle side) included in the second image is processed into a semi-transparent image, and the supplementary image is superimposed on the semi-transparent image and displayed. Further, a boundary between the second image and the supplementary image is provided at the outline of the semi-transparent image (that is, the outline of the side of the car), and the supplementary image is connected to the second image to obtain the third image.

The supplementary image superimposed on the translucent image area described above is connected to the one that is projected and converted based on the position of the distant plane and the one that is projected and converted based on the ground position that is closer to the shooting position than the distant plane. It is an image (or synthesized or spliced together).

Further, when the horizontal line direction of the first and second cameras is the left-right direction and the vertical line direction is the vertical direction, the upper region of the supplementary image is an image projected and converted based on the distant plane position, and the supplementary image The lower region is an image that has been projected and converted based on the ground position.

Furthermore, the image of the landscape block is translucently processed by averaging the brightness of a plurality of frames. As a result, it is possible to prevent a car running side by side, a car passing by, the light of a lamp, etc. from being reflected on the door of the car (having a Miller effect) and being imaged.

Further, in the lower region of the supplementary image, there is a filled image having a color approximated from the color of the surrounding image. As a result, the lower part (road) of the car is displayed by estimation, and a sense of stability as a whole screen can be obtained.

FIG. 3 shows an electrical processing block for performing the above processing. In FIG. 3, 101, 102R, and 102L are the first camera (may be referred to as a rear camera), the second camera 102R (may be referred to as a right side camera), and the third camera 10L (referred to as a left side camera), respectively. The shooting signals of the cameras 101, 102R, and 102L may be input to the image processing device 200.

That is, the shooting signal of the first camera 101 is input to the video cutting unit 221a of the first video processing unit 221. The shooting signal of the second camera 102R (which may be referred to as a right side camera) is input to the video cutting unit 222a of the second video processing unit 222, and the shooting signal of the third camera 102L (which may be referred to as a left side camera) is taken. The signal is input to the video cutting unit 223a of the third video processing unit 223.

The first camera 101, the second camera 102R, and the third camera 102L have acquired a wide-angle region image including an area wider than the image area to be used. Each of the cutout portions 221a, 222a, and 222b cut out the used image area to be used from the wide-angle area image. The cutout position and the cutout area may be changed according to the speed of the vehicle, the vibration of the vehicle, and the steering wheel operation direction. Further, the cutout position and / or the cutout area can be changed to the normal set position by the user operation. This is also effective when conducting video surveillance experiments and tests.

The video signal cut out by the video cutting unit 221a in the first video processing unit 221 is projected and converted by the projective conversion units 221b and 221c, and then input to the compositing unit 230. The projection conversion units 221b and 221c convert the viewpoint position of the first camera 101 to the viewpoint position of the second camera 102, although the reference positions for projection are different from each other.

That is, the projective conversion unit 221b and 221c can obtain an image projected and converted based on the position on the far plane and an image converted based on the ground position closer to the shooting position than the far plane. The video is output to the compositing unit 230 as a supplementary video.

On the other hand, in the second video processing unit 222, the video signal cut out by the video cutting unit 222a is projected and converted by the projective conversion unit 222b. The processing of the projection conversion unit 222b may be omitted. Further, also in the third video processing unit 223, the video signal cut out by the video cutting unit 223a is projected and converted by the projective conversion unit 223b. The processing of the projection conversion unit 223b may also be omitted. It is clear that the degree of conversion converted by the projective conversion unit 222b and 223b is smaller than the degree of conversion converted by the previous projective conversion unit 221b and 221c.

Homography is also called planar projective transformation technology or homography transformation technology, and transforms a plane figure of a virtual plane seen from one viewpoint (first viewpoint) into a plane figure of a virtual plane seen from another viewpoint (second viewpoint). It is a technology to convert.

The video signal acquired by the second video processing unit 222 described above includes a video of the side surface of the own vehicle (a video of the side surface of the right door taken toward the rear, that is, a landscape blockage video). Similarly, the video signal acquired by the third video processing unit 223 includes a video of the side surface of the own vehicle (a video of the side surface of the left door taken toward the rear, that is, a landscape blockage video).

Since the mounting position of the second camera (side camera) 102R with respect to the car is fixed, it is known in advance in which region of the angle of view the region of the landscape barrier image exists in the second image. Similarly, since the mounting position of the third camera (side camera) 102L with respect to the car is fixed, it is known in advance in which region of the angle of view the region of the landscape barrier image exists in the third image.

Therefore, the luminance averaging processing unit 222c performs the luminance averaging process of the landscape barrier image (the image on the right side of the own vehicle) included in a part of the second image. The reason for this averaging is to prevent cars running side by side, cars passing by, lamp light, etc. from being reflected on the door of the car (having the Miller effect) and being imaged. Similarly, the luminance averaging process is performed by the luminance averaging processing unit 223c for the landscape blockage image (the image on the left side surface of the own vehicle) included in a part of the third image.

Next, the landscape barrier image (the image on the right side of the own vehicle) included in a part of the second image whose brightness is averaged is translucently processed by the translucency processing unit 222d. Further, the landscape barrier image (the image on the left side of the own vehicle) included in a part of the third image whose brightness is averaged is also translucently processed by the translucency processing unit 223d. The degree of semi-transparency may be arbitrarily controlled by the information from the electronic controller 250.

Further, the second image output from the semi-transparency processing unit 222d is input to the edge processing unit 222e, and can be subjected to edge (contour) processing of the landscape barrier image. Similarly, the third image output from the translucency processing unit 223e is also input to the edge processing unit 223e and can be subjected to edge (contour) processing of the landscape barrier image. The edge processing is, for example, a processing (addition processing of an edge line) such as adding a line to an outline or adding a dotted line.

The color, thickness, darkness, and other elements of the edge line may be changed in various ways depending on the driving conditions of the vehicle and / or the external environment, and these elements may be adjusted by user operation. In some cases.

For example, when the vehicle is being driven backwards, the edges can be cleared to make it easier to see the distance between the obstacle and the side of the vehicle or the direction of travel of the vehicle.

The first video signal that received the above processing in the first video processing unit 221, the second video signal that received the above processing in the second video processing unit 222, and the third that received the above processing in the third video processing unit 223. The video signal is synthesized by the synthesis unit 230.

The electronic controller 250 includes vehicle speed information J1, various manual operation information J2, direction information J3 indicating the state of right turn / left turn / straight ahead / backward, time zone information J4, light lighting information, outside air temperature information, room temperature information, and the like. I know.

The speed information J1 is, for example, adjusted in the projective transformation unit 221b, 221c, 222b, 223b to automatically adjust the distance between the reference plane (reference image) which is a reference for changing the viewpoint position of the camera and the camera. It can be used as information.

The manual operation information J2 includes, for example, information for adjusting the reference position of the cutout area of the image, information for initial setting or adjusting the reference plane for performing the projective transformation, and the like.

Further, when the direction information J3 performs, for example, expansion processing of the cutout area of the image, enlargement processing, position correction processing of the cutout area, semi-transparency processing, change processing of edge thickness, color, density, and the like. It can be used as control information for. For example, when the car is moving forward, the transparency of the translucency is high (thin). Also, when the car is turning left or right, the edges become thicker and darker, making it easier for the driver to recognize the distance to obstacles located next to the car.

Further, the time zone information J4 can be used when adjusting the sensitivity of the camera. When the surroundings are dark, the detection sensitivity of the shooting signal of the camera can be increased, and when the surroundings are bright, the detection sensitivity can be suppressed.

Furthermore, there are various types of sensor information. There is also outside humidity information and outside air temperature information. By using these sensor information, a more appropriate image can be obtained from the camera. For example, if it is raining outside, a compressed air cleaner provided next to the camera may be used to remove water and dust around the lens. For this purpose, there may be an outlet for blowing compressed air in front of the lens of the camera. Furthermore, if snow or the like adheres to the front of the camera, it may freeze, so a heater may be further provided in front of the lens of the camera.

In this system, the display 226 is arranged in a position that is easy for the driver to see in the interior of the car. An indicator may be added in front of the rear seat or the passenger seat so that people other than the driver can see it.

Further, the inside or a part of the image processing device 200 of FIG. 3 can be realized by software. Further, the inside or a part thereof may be a processing device operated by software recorded on a storage medium. Further, the software is transmitted from the outside by a communication means, and the inside or a part thereof may be a device that receives and executes the software. Further, a recording / playback device may be connected to the video processing device 200, and a video signal may be recorded in the recording / playback device in response to, for example, a specially set running speed, vibration, sound, or the like.

FIG. 4 is an explanatory diagram for explaining the basic processing of video composition. The rear image 410 is a rear image cut out from the image captured by the rear camera 101. The side image 420 is a side image cut out from the image captured by the right side camera 102R.

FIG. 4 shows the rear image 410 and the side image 420 from the right side camera 102R, but in the actual device, the side image from the left side camera 102L on the left side is also used. Here, in order to simplify the explanation, an embodiment will be described in relation to the rear image 410 and the side image 420.

An image A0 of the side surface of the vehicle (which may be referred to as a side image of the own vehicle or an image of a landscape blocker) is reflected in a part (left side) of the above side image 420. This transfer region (region of the image A0) is a blind spot region for the rear view of the vehicle for the right side camera 102R.

Therefore, the area of the image A0 is watermarked (semi-transparent processing by the semi-transparency processing unit 222d), and a part of the image of the rear image 410 (the image of the area corresponding to the image A0) is superimposed, and the composition unit 230. Synthesize at.

Then, as shown in the composite image 430, a part of the rear image is superimposed on the translucent landscape blockage image. In the composite video 430, the edge line 435 represents a boundary (may be referred to as a connection position, a connection position, an outline, etc.) between the translucent video A0 and the video captured by the right side camera 102R. This edge line 435 is added in the edge processing unit 222e.

This edge line 435 is useful when the driver recognizes the width of the own vehicle when driving forward or backward. The line width, line color, brightness (darkness) and the like of the edge line 435 may be changed according to the surrounding conditions such as when the outdoors are dark or bright. This also makes it possible to call the driver's attention.

Further, in the composite video 430, a non-display line 432 is shown between the upper region A1 and the lower region A2 of the translucent video A0. This line 432 is not displayed as an image, but is shown for convenience of explanation.

As is the case with FIG. 4 or the drawings described later, a large number of diamond-shaped figures are shown, but these figures are not displayed in the actual image. This rhombus figure is shown to make it clear that the quadrangle has been transformed into a rhombus as a result of the viewpoint transformation. Many diamonds are not displayed in the actual image.

The images in the upper region A1 and the lower region A2 are images obtained by converting the rear image 410 from the viewpoint of the rear camera 101 into the image from the viewpoint of the right side camera 102R. In this case, the conversion reference position of the upper region A1 is the distant plane position 412, and the conversion reference position of the lower region A2 is the ground position 413 closer to the camera position than the distant plane position 412. The distant plane position 412 and the ground position 413 correspond to 422 and 423 shown in the side image 420 of the right side camera 102R.

FIG. 5 shows a first projective transformation image 410H in which the rear image 410 from the viewpoint of the rear camera 101 is projected and converted into the image from the viewpoint of the right side camera 102R based on the far plane position (FIG. 4). The conversion process is performed by the projective conversion unit 221b). Further, the rear image 410 from the viewpoint of the rear camera 101 shows the second projective conversion image 420H that has been projected and converted into the image from the viewpoint of the right side camera 102R based on the far-plane position (projection conversion unit in FIG. 3). Conversion processing is performed in 221c).

The above-mentioned distant plane position and distant plane position may be manually adjusted by the driver before driving. Further, as will be described later, a method may be adopted in which the vehicle automatically changes to a preset position according to the speed of the vehicle.

A part of the first projective conversion image 410H (periphery including P1) and a part of the second projective conversion image 420H (periphery including P2) are cut off to form a semi-transparent upper region A1 of the image A0, respectively. It is combined with the lower region A2. In this case, when the upper region A1 and the lower region A2 are cut out from the projected conversion image 410H and the projected conversion image 420H, the upper part is such that the images of each area and these images and the side image of the second camera are naturally connected. Region A1 and lower region A2 are cut out.

Further, in the side image 420 of the right side camera 102R, the blind spot area of the rear camera 101 also exists in the area of the image of the landscape blocker. That is, there is a blind spot area for both cameras.

This blind spot area is a complete blind spot area 433. This area corresponds to the area below the rear of the vehicle (usually the road surface). In FIG. 5, it is shown in the lower left of the composite image 420. Since there is no image in this complete blind spot area 433, there is a possibility that the image will be pitch black or pitch white if no processing is performed.

Therefore, in this system, for example, the color of the image around the complete blind spot area 433 or the image data of a similar color is filled and interpolated in this area.

That is, this system makes effective use of the images captured by the rear camera 101, the right side camera 102R, and the left side camera 102L.

FIG. 6 shows the way of thinking when changing the viewpoint. For example, it is assumed that the viewpoint C1 is the viewpoint of the first camera and the viewpoint C2 is the viewpoint of the second camera. First, the pixels A1-D1 of the reference plane P1 of the viewpoint C1 are temporarily replaced with the pixels AD of the virtual plane P. Next, the pixels AD of the virtual plane P are converted into the pixels A2-D2 of the virtual plane P2 of the second camera of the viewpoint C2.

Here, if the virtual planes P1 and P2 are at the same position (the previous ground position), the image of the ground captured by the first camera can be converted as the image captured by the second camera.

For example, when there are four points A to D on the plane P, each of these points is in A1-D1 on the virtual plane P1 in the image taken from the viewpoint C1 and in the virtual plane in the image taken from the viewpoint C2. Corresponds to A2-D2 on C2 respectively.

Here, once the correspondence between A1-D1 and A2-D2 is determined, the image captured by the first camera can be converted into the image captured by the second camera. This correspondence is called a projective transformation (or homography transformation) and can be expressed as a matrix.

FIG. 7 shows the transformation matrix H. Now, when the coordinates of A1-D1 on the virtual plane P1 are (xi, yi, 1), it is shown that the coordinates (ui, vi, 1) on the virtual plane P2 are converted by this transformation matrix H. ing. Here, i means 0, 1, 2, 3, and the coordinates are expressed in a three-dimensional homogeneous coordinate system in which one component is added to the two-dimensional plane.

That is, the coordinates of A1-D1 and A2-D2 correspond as follows.

A1: (x0, y0, 1), B1: (x1, y1, 1), C1: (x2, y2, 1), D1: (x3, y3, 1) A2: (u0, v0, 1), B2 : (u1, v1, 1), C2: (u2, v2, 1) D2: (u3, v3, 1),
In FIG. 7, if (xi, yi, 1) and (ui, vi, 1) are known, the matrix component h0-h7 can be obtained.

Since the component of the transformation matrix H is 8, h0-h7 can be obtained if the correspondence between the four points is known. The equation is expanded for i = 0,1,2,3, and h0-h7 is grouped out, which is shown in FIG. 7 (the determinant at the bottom of FIG. 7). From this formula, h0-h7 can be obtained.

With this transformation matrix H, it is possible to convert an arbitrary point on the virtual plane P1 seen from the viewpoint c1 into a point on the virtual plane P2 seen from the viewpoint c2.

FIG. 8 shows an example of an image whose brightness has been averaged by the brightness averaging units 222c and 223c shown in FIG. For example, as shown in image 510, it is assumed that the bus passes on the right side of the car. In this case, the side of the vehicle functions like a mirror and the bus is reflected. When this state is captured by the right side camera 102R as it is, the image B1 of the bus also exists in the image taken by the right side camera 102R (see images 510 and 511).

Since the car is running, the images reflected are various (buses, landscapes, houses, etc.). Therefore, in this system, the image B12 is obtained by averaging the brightness of the image B1 on the side surface of the car (see the image 512). When the image B12 whose brightness is averaged is semi-transparently processed, the reflected image has no great influence on the semi-transparent image as shown in the image 513. In the case of the image 511, the image is an image when the luminance averaging process is not performed, and an obstructive image is reflected in the semi-transparent image portion. However, according to this system, it is possible to make the image of the translucent part easier to see.

FIG. 9 is an explanatory diagram showing another embodiment when the complete blind spot region 433 is filled (or filled). If the car has a fourth camera that shoots the front, a part of the shot image 466 can be embedded in a part of the composite image 430. In this case, a cutting unit for cutting out the image captured by the front camera, a conversion unit for converting the front-back direction of the cut-out image, and a time adjusting unit are used. Then, the time-adjusted partial image 466 is sent to the synthesis unit 230 as a pseudo image and used as an image of the complete blind spot region 433.

FIG. 10 is an example in which the composite image 430 is displayed on the display unit 226 in an electronic mirror using a standard camera. Although the image on the right half side is shown in FIG. 10, the image on the left half side is also displayed on the display unit 226. In the image on the left half side, the supplementary image is superimposed on the semi-transparent image, and a line indicating the boundary between the image from the rear camera and the supplementary image is displayed at the outline of the semi-transparent image.

FIG. 11 shows an example of a composite image 430A obtained when the first, second, and third cameras capable of panoramic photography having a wider viewing angle than the standard are used. Since the complete blind spot region 431A is also generated in this composite image 430A, this portion is filled with a color or pattern similar to the surrounding color or pattern.

FIG. 12 shows the vehicles 502 and 502 traveling on the road in a plane. The vehicle 502 may automatically adjust the distance from the vehicle to the reference plane position (for example, the distant plane position shown in FIG. 4 and the ground position) according to the speed. When the speed is slow (for example, 0 Km / H-35 Km / H), pay attention to the following vehicle (for example, car 503) which is relatively close, and when the speed is medium speed (for example, 35 Km / H-65 Km / H). May want to pay attention to a vehicle that is farther away, and at higher speeds (for example, 65 km / H or more), to pay attention to a vehicle that is farther away.

In such a case, the distance from the vehicle to the reference plane position (for example, the distant plane position shown in FIG. 4 and the ground position) may be automatically adjusted by the user's setting.

FIG. 13 is an example of the composite video 430 when the reference plane position (virtual plane position) is far, and FIG. 14 is an example of the composite video 430 when the reference plane position is close. In the example of FIG. 13, when the reference plane position is far, the vertical line of the building far behind coincides with the first image and the second image. However, with respect to the image of a nearby car, a step is generated at the connection portion between the first image and the second image.

On the other hand, as shown in FIG. 14, when the reference plane position (virtual plane position) is close, the vertical line of the building far behind causes a deviation (step) between the first image and the second image. .. However, with respect to the image of a nearby car, the connection between the first image and the second image is smooth.

Therefore, by adjusting the virtual plane position according to the speed of the vehicle, it is possible to clarify the rear object at the distance that the driver wants to pay attention to.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are also included in the scope of the invention described in the claims and the equivalent scope thereof. It is also included in the present invention to separate or combine the elements described in the claims.

100 ... Car, 101 ... 1st camera, 102 ... 2nd camera ... 102R, 3rd camera ... 102L, 200 ... Video processing device 221a, 222a, 223a ... Image cutting unit, 221b, 221c, 222b, 223b ... Projection conversion unit, 222c, 223c ... Brightness averaging unit, 222d, 223d ... Semi-transparency processing unit, 222e, 223e ... Edge processing unit , 230 ... synthesis unit, 226 ... display unit, 250 ... electronic controller.

Claims (4)

In an electronic mirror image synthesizer that is mounted on a vehicle and displays an image behind the vehicle.
A rear camera that acquires the first image of the first landscape behind the vehicle from the first position by the first field of view.
From the second position, which is in front of the vehicle from the first position, the second field of view is such that a landscape blocker, which is a side portion of the own vehicle, is partially arranged toward the direction of the first landscape. With a side camera that captures images,
A semi-transparent processing unit that converts the image of the landscape blocker included in the second image into a semi-transparent image, and
An edge processing unit capable of adding a line to the contour of the translucent image in the second image, and an edge processing unit.
A projecting conversion unit that is a part of the first image, is projected to the viewpoint position of the side camera, and obtains a projecting converted image corresponding to the semi-transparent image portion of the second image.
The
The projective conversion image is
A part of the first projective conversion image in which the image of the viewpoint of the rear camera is projected and converted into the image of the viewpoint of the side camera based on the far plane position behind the vehicle by the first projective conversion unit. ,
The second projection conversion unit projected and converted the image of the viewpoint of the rear camera into the image of the viewpoint of the side camera based on the ground position closer to the shooting position of the side camera than the far plane position. Including a part of the second projective conversion image
When the horizontal line direction of the rear camera and the side camera is the horizontal direction and the vertical line direction is the vertical direction.
The upper region of the projective transformation image corresponding to the translucent image portion is the part of the first projective conversion image.
The lower region of the projective transformation image corresponding to the translucent image portion is the part of the second projective conversion image.
Further, in the first projective transformation or the second projective transformation, the distant plane position and the ground position are adjusted according to the settings according to the traveling speed of the vehicle.
In front of the lenses of the rear camera and the side camera, an outlet for blowing compressed air is provided.
Video synthesizer for electronic mirrors. The image synthesizer for an electronic mirror according to claim 1, further comprising a heater further provided in front of the lens and means for changing any of the thickness, color, and density of the line. In an electronic mirror mounted on a vehicle and displaying an image of the rear of the vehicle, a rear camera that acquires a first image of a first landscape behind the vehicle from a first position from a first position, and the first position. The second image is acquired from the second position, which is in front of the vehicle, from the second field of view in which the landscape barrier, which is a side portion of the own vehicle, is partially arranged toward the direction of the first landscape. In a method of video composition of an electronic mirror using a side camera and a video processing device that processes the first video and the second video from the rear camera and the side camera.
The image of the landscape blocker included in the second image is made into a semi-transparent image.
In the second image, a line is added to the outline of the semi-transparent image.
A method of projecting a part of the first image to the viewpoint position of the side camera to generate a projective conversion image, and synthesizing the projective conversion image with the semitransparent image portion of the second image. can be,
As the projective conversion image,
It has a part of the first projective conversion image in which the image of the viewpoint of the rear camera is projected and converted into the image of the viewpoint of the side camera based on the far plane position behind the vehicle by the first projective conversion unit.
Further, the second projection conversion unit projects and converts the image of the viewpoint of the rear camera into the image of the viewpoint of the side camera based on the ground position closer to the shooting position of the side camera than the far plane position. Has a part of the converted video,
When the horizontal line direction of the rear camera and the side camera is the horizontal direction and the vertical line direction is the vertical direction.
The upper region of the projective transformation image corresponding to the translucent image portion is the part of the first projective conversion image.
The lower region of the projective transformation image corresponding to the translucent image portion is the part of the second projective conversion image.
Further, in the first projective transformation or the second projective transformation, the distant plane position and the ground position are adjusted according to the settings according to the traveling speed of the vehicle.
Compressed air was blown in front of the lenses of the rear camera and the side camera.
Video composition method for electronic mirrors. The method for synthesizing an image of an electronic mirror according to claim 3, wherein the front of the lens can be heated by a heater, and any of the thickness, color, and density of the line can be changed.

Images