CN218877068U

CN218877068U - Digital rearview mirror with image synthesis function, vehicle-mounted unit system and vehicle

Info

Publication number: CN218877068U
Application number: CN202220969039.4U
Authority: CN
Inventors: S·格尔克
Original assignee: Mobility Asia Smart Technology Co Ltd
Current assignee: Mobility Asia Smart Technology Co Ltd
Priority date: 2022-04-25
Filing date: 2022-04-25
Publication date: 2023-04-18
Anticipated expiration: 2032-04-25

Abstract

The utility model relates to a digital rear-view mirror, car machine system and vehicle. This digital rear-view mirror includes: an image processing unit configured to connect at least two cameras disposed at different positions outside the vehicle, receive image data about an external environment of the vehicle collected by the at least two cameras, and perform image data processing on the received image data to obtain a composite image about the external environment; and a display located inside the vehicle and configured to connect the image processing unit and display the composite image. The utility model discloses an image to two at least cameras uptake carries out image data processing and can show the composite image on the display, is favorable to the driver to have a broader external environment and observes the field of vision, improves driving safety and drives experience.

Description

Digital rearview mirror with image synthesis function, vehicle-mounted unit system and vehicle

Technical Field

The utility model relates to a digital rear-view mirror, car machine system and vehicle with image synthesis function.

Background

The rear view mirror is a tool for directly obtaining external environment information at the back and side of the vehicle by a driver sitting on a cab seat, and comprises an inner rear view mirror and an outer rear view mirror. The inner rear-view mirror is arranged above the top of the head of a driver, so that the driver can confirm the scene right behind the vehicle without changing the forward sight line in driving too much. The outer rear view mirrors are generally installed at both sides of the vehicle body, and it is generally required to adjust the angle of the outer rear view mirrors in order to facilitate the driver's view. In the process of driving a vehicle, a driver needs to respectively observe the inner rear-view mirror, the left rear-view mirror or the right rear-view mirror according to the road and the external environment so as to check the scenes of different areas at the back and the side, and certain visual field blind areas exist in the inner rear-view mirror and the outer rear-view mirror.

There has been a digital rear view mirror including a camera provided at the rear and side of a vehicle for observing the rear and side of the vehicle, and a display screen for facilitating a driver to observe an image of a rear scene photographed by the camera. However, in the existing digital rearview mirror, the image shot by each camera is displayed by a single display screen, so that the driver still needs to observe the scene behind and beside the vehicle through a plurality of display screens, which is not convenient for observing and is also not beneficial for judging the relative position of the object appearing behind and beside the vehicle.

Disclosure of Invention

An object of the utility model is to provide a digital rear-view mirror, car machine system and vehicle with image synthesis function. The digital rearview mirror can process image data of images shot by at least two cameras arranged at the rear part and the side part of a vehicle and display a composite image on a single display screen, so that a driver can have a wider external environment observation visual field to improve driving safety and driving experience.

An embodiment of the utility model provides a digital rear-view mirror, digital rear-view mirror includes: the vehicle comprises an image processing unit, a display unit and a control unit, wherein the image processing unit is configured to connect at least two cameras arranged at different positions outside a vehicle, receive image data about the external environment of the vehicle collected by the at least two cameras, and perform image data processing on the received image data to obtain a composite image about the external environment; and a display located inside the vehicle and configured to connect the image processing unit and display the composite image.

The embodiment of the utility model provides a still provide a car machine system, car machine system includes: the vehicle comprises an image processing unit, a display unit and a control unit, wherein the image processing unit is configured to connect at least two cameras arranged at different positions outside a vehicle, receive image data about the external environment of the vehicle collected by the at least two cameras, and perform image data processing on the received image data to obtain a composite image about the external environment.

An embodiment of the utility model provides a vehicle is still provided, include: the digital rearview mirror according to the embodiment; or the vehicle machine system according to the above embodiment.

Drawings

Fig. 1 is a block diagram of a digital rear view mirror according to an embodiment of the present invention.

Fig. 2A shows a schematic position diagram of a vehicle and three cameras mounted thereon in an embodiment; fig. 2B shows a schematic view of the camera views of three cameras on a vehicle.

FIG. 3A shows an image captured by a rear camera in an embodiment; FIG. 3B shows an image captured by the right camera in an embodiment; FIG. 3C shows an image captured by the left camera in an embodiment; fig. 3D shows a schematic diagram of a composite image displayed on a display.

Fig. 4 is a block diagram of a vehicle machine system according to an embodiment of the present invention.

Detailed Description

Embodiments of the present invention are further described below with reference to the accompanying drawings.

Fig. 1 shows a structure of a digital rearview mirror according to an embodiment of the present invention. As shown in fig. 1, the digital rearview mirror 100 includes: an image processing unit 140 configured to connect at least two cameras (e.g., 110, 120, 130) disposed at different positions outside the vehicle, receive image data about an external environment of the vehicle captured by the at least two cameras, and perform image data processing on the received image data to obtain a composite image about the external environment; and a display 150 located inside the vehicle and configured to connect the image processing unit 140 and display the obtained composite image.

Utilize the utility model discloses a digital rear-view mirror, user (driver and passenger) can clearly observe the whole external environment that has the wider wide field of vision that the composite image of the image gathered by two at least cameras outside the vehicle appears to be favorable to the user to carry out the observation and the corresponding vehicle of vehicle outside and control, improve the security of vehicle driving and parking in-process, and improve user experience.

In the embodiment of the present invention, at least two cameras may have the shooting visual field ranges connected or overlapped with each other, but the present invention is not limited thereto. Further, the at least two cameras may also not be limited to shooting only toward the rear or the lateral rear of the vehicle. According to the embodiment of the utility model, the combination of the shooting visual field ranges of the cameras at different positions can be used for obtaining the corresponding composite image, so that the driver can observe the interested direction in a wider visual field range; by combining the camera with the adjustable shooting direction, the driver can observe the composite image with the adjustable visual field range in any direction on the display in the vehicle, so that the observation capability of the driver on the external environment is greatly enhanced, and the observation visual field is expanded. It will be appreciated by those skilled in the art that the composite image obtained in accordance with embodiments of the present invention is a composite video image.

In some embodiments, the at least two cameras disposed on the exterior of the vehicle comprise: a first rear-view camera 110 disposed on the left side of the vehicle; a second rear-view camera 120 disposed on a right side of the vehicle opposite the left side; and a third rear view camera 130 disposed at the top or rear of the vehicle. Through presenting the composite image that the image that gathers according to these three back vision cameras obtained, the user on the vehicle can observe the whole external environment in left side rear, right side rear and the dead astern of vehicle clearly, is favorable to the user to carry out the observation and corresponding vehicle control in vehicle side rear and rear.

In some embodiments, the image processing unit 140 is configured to perform the image data fusion process at least by performing the image stitching process and the image fusion process on the image data from at least two cameras (e.g., the first rear-view camera 110, the second rear-view camera 120, and the third rear-view camera 130), respectively. In one example, the image processing unit 140 includes an image stitching processing component 141 and an image fusion processing component 142 for performing image stitching and image fusion on the images acquired by the

cameras

110, 120, and 130, respectively, to obtain a composite image. The image stitching processing component 141 and the image fusion processing component 142 may be implemented by hardware, software, or a combination thereof. In another example, the image processing unit 140 comprises only the image stitching processing component 141 to obtain a stitched composite image with respect to the external environment by stitching the images acquired by the respective cameras.

In some embodiments, as shown in fig. 2A, the third rear-view camera 130 is located at a position on the longitudinal centerline 210 of the vehicle, and the first rear-view camera 110 and the second rear-view camera 120 are located at the left and right sides of the vehicle, respectively, and have symmetrical positions with respect to the longitudinal centerline 210 of the vehicle. The third rear-view camera 130 may be located on the top of the vehicle to have a rearward unobstructed camera field of view. Some vehicles have the greatest rear (aft) height, the third rear-view camera 130 may be mounted on the top edge of the rear of the vehicle. In another example, the third rear-view camera 130 is a camera positioned on the roof of the vehicle and has an adjustment mechanism that can adjust its camera view range to either camera forward of the vehicle or to camera rearward of the vehicle, so that the camera can be adjusted to camera both forward and rearward of the vehicle.

The left, right, and rear of the vehicle as described herein are relative to the direction of travel of the vehicle. Each camera in the embodiments may be an optical camera, an infrared camera, or the like, but the present invention is not limited to these kinds. It will be understood by those skilled in the art that the embodiment of the present invention is not limited to the use of three rear-view cameras, and other numbers of cameras may be used, for example, four or five cameras may be provided, and two cameras may be provided at the rear or a side of the vehicle.

As shown in fig. 2B, the camera 110 on the left side of the vehicle shoots towards the left rear of the vehicle, the camera 120 on the right side of the vehicle shoots towards the right rear of the vehicle, and the camera 130 on the rear or top of the vehicle shoots towards the right rear of the vehicle, the camera view ranges of the three cameras are respectively marked as 112, 122 and 132 in fig. 2B, and the

camera view ranges

112 and 122 in the figure adopt different area filling patterns from the camera view range 132 for distinction. The images captured by the three

cameras

110, 120, and 130 are subjected to image stitching processing and image fusion processing, so that a composite image having an entire imaging view range including 112, 122, and 132 can be obtained. Typically, the

cameras

110, 120 are positioned such that there is a range of overlap between their

camera view ranges

112, 122 and the camera view range 132 of the camera 130. For the same object in the overlapped range, due to the difference of the shooting angles and shooting distances of the

cameras

110, 120, and 130, the shape, color, brightness, and/or size of the shot object in the image may be different, and the object information included in the shot image may not be identical.

Splicing the source images shot by three different cameras into a synthesized panoramic image by image splicing; and the image splicing processing comprises image registration and image seamless connection. In general, the image stitching process may include: image preprocessing, including basic operations of digital image processing (such as denoising, edge extraction, histogram processing and the like), establishing a matching template of the images, performing certain transformation (such as Fourier transformation, wavelet transformation and the like) on the images and the like, determining the more accurate position of the superposition of two adjacent images, and laying a foundation for searching feature points; image registration, namely, adopting a certain matching strategy to find out the corresponding position of a template or a characteristic point in an image to be spliced in a reference image, finding out the characteristic point to be matched after determining the basic coincident position, and further determining the transformation relation between the two images; establishing a transformation model or a transformation matrix of the image according to the matching points, and calculating each parameter value in the mathematical model according to the corresponding relation between the templates or the image characteristics so as to establish a mathematical transformation model of the two images; unified coordinate transformation, namely transforming the images to be spliced into a coordinate system of a reference image according to the established mathematical transformation model to finish the unified coordinate transformation; performing seamless connection on images, namely performing seamless connection processing on a superposed region of the images to be spliced to obtain a spliced and reconstructed smooth and seamless panoramic image and realize the splicing of the images; and finally, smoothing the spliced image. The main objective of the image seamless joining processing is to enable the spliced synthetic image to be in smooth transition near the splicing seam and the whole color and the gray scale of the synthetic image to be harmonious through correcting the pixel value of each source image, and meanwhile, to ensure that the information quantity and the synthetic image which are as clear as possible are obtained from the source images to be spliced when the pixel value is corrected.

In the image fusion processing, image data which is collected by at least two (for example, three) cameras and relates to the same target can be subjected to image processing so as to extract target effective information in each collected image to the maximum extent, and finally, the target effective information is synthesized into a high-quality synthesized image, so that the utilization rate of image information is improved, the spatial resolution and the spectral resolution of an original image are improved, and the uncertainty and the redundancy of the output synthesized image are reduced; the time space information contained in the synthesized spliced image can be enlarged, the reliability of image information processing is increased, and the robustness of image processing is improved. The embodiment of the present invention can adopt more than three cameras, for example, two cameras are respectively disposed at the rear portion of the vehicle and/or two sides of the vehicle, and the definition of the composite image is improved by the image fusion processing.

In the example shown in fig. 3A-3D, fig. 3A shows images captured by a rear camera (i.e., third rear-view camera 130); FIG. 3B shows an image captured by the right side camera (i.e., the second rear view camera 120); FIG. 3C shows an image captured by the left-side camera (i.e., the first rear-view camera 110); fig. 3D shows a schematic diagram of a synthesized image displayed on the display 150, in which the scene images in fig. 3A (middle part), fig. 3B (left part), and fig. 3C (right part) are included. The driver can observe the composite image shown in fig. 3D through the display 150, can have a wider observation field of view, can clearly observe the external environments (including other vehicles traveling behind and behind) on both sides of the vehicle and directly behind, and is also advantageous in determining the relative positions of objects appearing directly behind and behind the vehicle, and the observation of which has no blind field of view.

In some embodiments, the display 150 is disposed at an interior rearview mirror location, such as to the upper right of a driver's seat in the vehicle, and the driver may view the digital rearview mirror 100 in a manner similar to viewing a conventional rearview mirror. In other embodiments, the display 150 may be a display of a vehicle-mounted system of a vehicle, such as a center control screen, on which a user may view the external environment behind and to the side of the vehicle by switching display functions. In other embodiments, the display 150 is a head-up display system (HUD) that maps the synthesized image obtained by the image processing unit 140 on, for example, a front windshield or a side windshield of the vehicle through a projection unit, so that the driver can easily see the entire external environment laterally and rearward of the vehicle without lowering his head. The head-up display system may also display a composite image of the entire external environment with respect to the lateral rear and the right rear of the vehicle, which is output from the image processing unit 140, while displaying parameters such as vehicle speed, oil amount, or electric quantity.

In other embodiments, the display 150 is a Head Mounted Display (HMD), and the composite image obtained by the image processing unit 140 is displayed on the HMD by using, for example, virtual Reality (VR), augmented Reality (AR), mixed Reality (MR) and other technologies, so that the driver can easily see the entire external environment behind and to the side of the vehicle at any time. The image processing unit 140 may be included in a vehicle-mounted system of the vehicle, that is, the image processing function of the image processing unit 140 is realized by the image processing unit in the vehicle-mounted system, and the obtained composite image is transmitted to the display 150 for display through a wired or wireless connection with the display 150.

In some embodiments, at least one of the first rear-view camera 110, the second rear-view camera 120, and the third rear-view camera 130 is mounted on an adjustment mechanism that can change its shooting field of view. In one example, the third rear-view camera 130 is disposed at the top or rear top of the vehicle (i.e., the highest portion on the rear of the vehicle), and is mounted on an adjustment mechanism that can change the photographing visual field range of the third rear-view camera 130. For example, the adjusting mechanism is a mechanical structure that can be automatically rotated or manually rotated, and by rotating the adjusting mechanism, the third rear-view camera 130 can be made to photograph in any one of the front, front-upper, left-rear, or right-rear directions of the vehicle, so that the driver can focus on the direction of interest or other blind areas. In addition, the first and second

rear view cameras

110 and 120 may be set in a state in which their photographing visual field ranges are adjustable (e.g., to photograph toward the front left and front right of the vehicle), and by adjusting the photographing visual field range of at least one of the three cameras, the composite image of the three

cameras

110, 120, and 130 may be enabled to present a direction desired by the driver or other visual field blind areas, for example, an external environment on a certain side of the vehicle needs to be carefully observed while the vehicle is traveling on the road, or an external environment in the front and upper of the vehicle needs to be carefully observed in a case where the roof of the vehicle may be collided with an external object. In another example, the three

cameras

110, 120, and 130 are infrared cameras and are respectively adjusted to face the front left, front right, and front right of the vehicle, and the driver can observe the external environment in front through a synthesized image at night or in a dark environment.

Utilize the utility model discloses an above-mentioned adjustment mechanism in the digital rear-view mirror can be adjusted digital rear-view mirror into any kind such as digital front-view mirror, side-view mirror, upward viewing mirror according to the actual needs of traveling.

In some embodiments, the image processing unit 140 is connected to a tachograph of the vehicle and transmits the composite image output therefrom to the tachograph. Therefore, the composite images showing the external environment (such as lateral rear and right rear) of the vehicle can be recorded in the driving recorder, so that the recorded content of the driving recorder is more complete, and the recording is not limited to the conventional driving recorder only recording the images collected by the front camera. For example, the drive recorder may record the entire road live and other vehicle driving conditions located laterally behind and directly behind the vehicle, and the sound information of the vehicle and the pedestrian located laterally behind and directly behind the vehicle.

Fig. 4 shows a block diagram of a vehicle machine system according to an embodiment of the present invention. As shown in fig. 4, the in-vehicle machine system 400 includes: an image processing unit 410 configured to connect at least two cameras disposed outside the vehicle, and receive image data about an external environment of the vehicle captured by the at least two cameras, and perform image data processing on the received image data to obtain a composite image about the external environment. The in-vehicle system 400 may be implemented by a conventional in-vehicle system configured with an image processor, and may not only implement various functions of the conventional in-vehicle system (such as navigation, positioning, music video playing, electronic games, radio, network communication, anti-theft monitoring, etc.) by including various functional units, but also implement a function of synthesizing images collected by a plurality of cameras.

In some embodiments, the at least two cameras disposed on the exterior of the vehicle comprise: a first rear-view camera 110 disposed on the left side of the vehicle; a second rear-view camera 120 disposed on a right side of the vehicle opposite the left side; and a third rear view camera 130 disposed at the top or rear of the vehicle. By displaying the composite image obtained from the images captured by the three rear-view cameras on the display 420, the user on the vehicle can clearly observe the overall external environment with a wider field of view behind the left side, behind the right side, and directly behind the vehicle, which is beneficial for the user to observe the rear side and behind the vehicle and to operate the vehicle accordingly.

In some embodiments, the image processing unit 410 is configured to obtain a composite image about the external environment at least by performing an image stitching process and an image fusion process on image data from at least two cameras (e.g., the first rear-view camera 110, the second rear-view camera 120, and the third rear-view camera 130), respectively. In one example, the image processing unit 410 includes an image stitching processing component 411 and an image fusion processing component 412 for performing image stitching and image fusion on the images acquired by the

cameras

110, 120, and 130, respectively, to obtain a composite image. The image stitching processing component 411 and the image fusion processing component 412 may be implemented by hardware, software, or a combination thereof.

In some embodiments, as shown by the dashed box in fig. 4, in-vehicle system 400 further includes a display 420 configured to connect to image processing unit 410 and to display the composite image output by image processing unit 410. The display 420 may be a separately provided display dedicated to rear view observation, or may be a display configured by a car machine system. In some embodiments, display 420 is a heads-up display system (HUD) or a head-mounted display screen (HMD).

In some embodiments, the image processing unit 410 is connected to a tachograph of the vehicle and transmits the output composite image to the tachograph for recording.

It will be understood by those skilled in the art that the image processing unit 410 and the display 420 shown in fig. 4 may be similar in structure and function to the image processing unit 140 and the display 150 shown in fig. 1, and thus may also implement the image processing and displaying functions of the embodiment described above with reference to fig. 1.

An embodiment of the utility model provides a vehicle is still provided, this vehicle includes: the digital rearview mirror according to the embodiment; or the vehicle machine system according to the above embodiment. In the embodiment of the present invention, the car machine system including the image processing unit can combine a plurality of cameras on the vehicle and the display (for example, the display screen dedicated for rear view observation, the center control screen, the HUD or the HMD) arranged inside the vehicle to realize the digital rearview mirror.

Those skilled in the art can understand that the image processing unit in the digital rearview mirror of the present invention can also be implemented by the image processing unit of other information processing devices (e.g. intelligent driving computing platform) of the vehicle, and the plurality of cameras on the vehicle can be directly connected to the intelligent driving computing platform to transmit the collected images of the external environment, and the image processing unit in the intelligent driving computing platform performs image processing and synthesis.

The above description is only a preferred embodiment of the present invention and should not be construed as limiting the invention in any way. Those skilled in the art who have the benefit of the teachings and principles of this invention may effect numerous modifications and changes in form and detail without departing from the principles of this invention, and such modifications and changes are intended to be included within the scope of this invention as defined in the following claims.

Claims

1. A digital rearview mirror, comprising:

the vehicle comprises an image processing unit, a display unit and a control unit, wherein the image processing unit is configured to connect at least two cameras arranged at different positions outside a vehicle, receive image data about the external environment of the vehicle collected by the at least two cameras, and perform image data processing on the received image data to obtain a composite image about the external environment; and

a display located inside the vehicle and configured to connect the image processing unit and display the composite image.

2. The digital rearview mirror of claim 1, wherein said at least two cameras comprise:

a first rear-view camera disposed on a left side of the vehicle;

a second rear-view camera disposed on a right side of the vehicle opposite the left side; and

a third rear view camera disposed at a top or rear of the vehicle.

3. The digital rearview mirror according to claim 1 or 2, wherein the image processing unit is configured to obtain a composite image about the external environment at least by performing image stitching processing and image fusion processing on the image data from the at least two cameras, respectively.

4. The digital rearview mirror of claim 2, wherein the third rearview camera is located at a position on a longitudinal centerline of the vehicle, and the first and second rearview cameras are located at left and right sides of the vehicle, respectively, and have symmetrical positions with respect to the longitudinal centerline.

5. Digital rearview mirror according to claim 1, wherein said display is a head-up display system (HUD) or a head-mounted display screen (HMD).

6. The digital rearview mirror according to claim 1 or 2, wherein at least one of the at least two cameras is mounted on an adjustment mechanism that can change its shooting view range.

7. The digital rearview mirror of claim 1, wherein the image processing unit is connected to a tachograph of the vehicle and transmits the composite image to the tachograph.

8. The utility model provides a car machine system, its characterized in that, car machine system includes:

the vehicle comprises an image processing unit, a display unit and a control unit, wherein the image processing unit is configured to connect at least two cameras arranged at different positions outside a vehicle, receive image data about the external environment of the vehicle collected by the at least two cameras, and perform image data processing on the received image data to obtain a composite image about the external environment.

9. The in-vehicle machine system of claim 8, wherein the at least two cameras comprise:

a first rear-view camera disposed on a left side of the vehicle;

a third rear view camera disposed at a top or rear of the vehicle.

10. The in-vehicle machine system according to claim 8 or 9, wherein the image processing unit is configured to obtain a composite image about the external environment at least by performing an image stitching process and an image fusion process on the image data from the at least two cameras, respectively.

11. The vehicle machine system according to claim 8, further comprising:

a display configured to connect the image processing unit and display the composite image.

12. The in-vehicle system of claim 11, wherein the display is a head-up display system (HUD) or a head-mounted display screen (HMD).

13. The in-vehicle system of claim 8, wherein the image processing unit is connected to a tachograph of the vehicle, and transmits the composite image to the tachograph.

14. A vehicle, characterized in that the vehicle comprises:

the digital rearview mirror of any one of claims 1-7; or

The in-vehicle machine system according to any one of claims 8 to 13.