TWI622297B - Display method capable of simultaneously displaying rear panorama and turning picture when the vehicle turns - Google Patents

Abstract

The invention provides a display method capable of synchronously displaying a rear panorama and a turning picture when a vehicle turns, and is applied to a line of vehicle display system, which includes a control unit, a photographing module, and a display screen, wherein: The angle of view of a lens of the photographing module is at least greater than 140 degrees. In addition, after the control unit receives a turning signal, it will receive the distorted image from the photographing module, and intercept the distorted image from the distorted image according to the turning signal. Corresponds to a partial image, and performs a distortion correction procedure on the deformed image and the partial image to form a panoramic image and a turning image. After that, the control unit displays the panoramic image and the turning image on the display screen at the same time. .

Description

Display method capable of simultaneously displaying rear panorama and turning picture when the vehicle turns

The invention relates to a display method when a vehicle is moving, and in particular to a method that can use only a single lens when a user is turning, that is, the image screen of the rear and the turning direction can be displayed on the display screen at the same time to improve the driving safety of the user Sexual approach.

Generally speaking, for the driver, because of his limited field of vision to look behind the vehicle, when the driver is reversing, he often uses the assistance of "Parking sensors" to confirm whether there is an obstacle behind the vehicle body. The presence of objects and the distance between the vehicle body and obstacles to improve safety when reversing.

Check that most of the existing "reversing radars" use the ultrasonic ranging principle to complete the detection of obstacles. It is mainly provided with a plurality of ultrasonic sensors on the rear of the vehicle. These ultrasonic sensors can be used to emit ultrasonic waves. Ultrasonic waves will generate reflected waves after encountering obstacles. When the ultrasonic sensors receive the reflected waves, they will send a sensing signal to a controller so that the controller can calculate the vehicle body and obstacles. The distance between the objects and the position of the obstacles, according to which the buzzer is driven to issue a warning sound. Because the ultrasonic sensor can only receive ultrasonic waves in a certain angle range, the aforementioned angle range is called "detection angle". Generally speaking, the "detection angle" of an ultrasonic sensor is usually 90 degrees to 120 degrees in the horizontal direction and vertical. The direction is 60 degrees to 80 degrees. Therefore, in order to reduce the detection blind zone, more than two ultrasonic sensors are generally installed on the vehicle to improve the safety in use. Fullness.

However, for the user, by listening to the warning sound, it is not possible to grasp the real situation behind the car body. After all, the intensity of the reflected wave is related to the propagation characteristics of the ultrasonic wave. For example, if the obstacle reflects the surface (that is, facing Ultrasonic sensor) has a small area, the surface of the obstacle is smooth, the direction of the obstacle deviates from the ultrasonic sensor, the material of the obstacle is a loose, porous object, etc. The device only receives a small amount of reflected waves, or even cannot receive ultrasonic waves, causing the "reversing radar" to not be able to determine the existence of obstacles behind the vehicle. In addition, factors such as ambient temperature, air humidity, and air pressure will affect the reflected waves. This reduces the detection effect of the "reversing radar".

In order to solve the aforementioned problems, the operator will install a photographic lens on the vehicle body so that the user can view the picture transmitted by the photographic lens when reversing the vehicle. However, the applicant found that if there is only a single photograph on the vehicle body In the case of a camera, the driver can only see the picture transmitted by the camera lens when reversing. At this time, if the driver needs to turn left or right, he must use the left side mirror or right of the car body. The sight glass, even when you turn your head, you can see the scene at the left or right rear of the car body, which causes inconvenience in use. In addition, industry owners will also install multiple photographic lenses on the car body to enable shooting in multiple directions. However, this will not only cause the driver to spend too much money, but also need to switch the screen of each lens while reversing, which undoubtedly increases the risk of the driver reversing and loses the assistance of the photographic lens. Good intentions.

In addition, the applicant has also found that when a vehicle needs to make a turn while the vehicle is traveling, the driver can usually only look at the rear scene through the rear-view mirror, left-view mirror, and right-view mirror, in order to avoid the possibility that Pay attention to the situation of collision, but the foregoing situation will also cause the driver to turn his head multiple times to be able to view the pictures reflected by the rear-view mirror, left-view mirror and right-view mirror, respectively. As a result, the driver is troubled by driving. Therefore, how to improve the foregoing problems has become an important issue to be solved by the present invention.

In view of the fact that the auxiliary system of the conventional vehicle traveling or reversing still has the aforementioned deficiency, which causes troubles in use, the inventor finally developed the present invention after repeated research and testing based on his many years of practical experience in serving related industries. A display method capable of synchronously displaying the rear panorama and the turning picture when the vehicle turns is expected to provide users with a better experience in using the method to obtain the user's favor.

An object of the present invention is to provide a display method capable of synchronously displaying a rear panorama and a turning screen when a vehicle turns, and the method is applied to a vehicle display system that can be assembled on a vehicle body and includes a control unit. A photographic module and a display screen, wherein a lens of the photographic module is assembled to the rear of the vehicle body, and the angle of view of the lens is at least 140 degrees, and the display screen is provided in the vehicle body, on which The panoramic picture area and the turning picture area are divided. The control unit will be electrically connected to the photographic module and the display screen. When the control unit receives a turn signal, it will receive the photographic module in order. A plurality of distorted images, and then, the control unit will intercept a corresponding partial image from the distorted image according to the turn signal, and respectively perform a distortion correction procedure on the distorted images and the partial images to sequentially form a The panoramic image and a turning image. Finally, the control unit sends the panoramic image and the turning image to the display screen, wherein the panoramic image The field will display the panoramic image, and the turning screen area will display the turning image. In this way, the driving display system only needs a single lens to display the panoramic image and the turning image at the same time, and the driver does not need to switch screens. The operation with the lens greatly improves the safety in use.

Another object of the present invention is the aforementioned method. After receiving the distorted image, the control unit first executes the distortion correction procedure on the distorted image to form the panoramic image. After that, the control unit automatically calculates the distortion image based on the turn signal. In the panoramic image, a corresponding partial image is captured to form the turning image. In this way, the control unit only needs to perform a distortion correction procedure for a single deformed image, so the operation load of the control unit can be effectively reduced.

In order that the review committee can further understand and understand the purpose, technical features and effects of the present invention, the embodiments are described in detail with the drawings, as follows:

[Learning]

no

〔this invention〕

1‧‧‧ Driving Display System

10‧‧‧Control unit

11‧‧‧Photographic Module

111‧‧‧ lens

13‧‧‧display

131‧‧‧Panoramic picture area

133‧‧‧turn picture area

15‧‧‧Warning device

201 ~ 206‧‧‧ steps

H‧‧‧body

A‧‧‧ distorted image

R ’, L’ ‧‧‧ partial image

L1‧‧‧First distance value

L2‧‧‧Second distance value

T‧‧‧ road marking

M‧‧‧ obstacle

FIG. 1 is a schematic diagram of a vehicle body and a lens of the present invention; FIG. 2 is a schematic diagram of a hardware block of a driving display system of the present invention; FIG. 3 is a schematic view of a panoramic picture area and a turning picture area of a display screen of the present invention; Figure 4 is a schematic diagram of a right-turned deformed image of the lens of the present invention; Figure 4B is a schematic diagram of a left-turned deformed image of the lens of the present invention; Figure 5 is a flowchart of the present invention; and Figure 6A is a three-dimensional rear view of the vehicle body of the present invention Figure 6B is a schematic view of the panoramic image of Figure 6A; Figure 7A is a schematic view of the three-dimensional object behind the vehicle body of the present invention; Figure 7B is a schematic view of the near-end panoramic image of Figure 7A; FIG. 8A is another schematic view of a three-dimensional object behind the vehicle body of the present invention; and FIG. 8B is a schematic top view panoramic image of FIG. 7A.

The present invention is a display method capable of synchronously displaying a rear panorama and a turning picture when a vehicle turns, and is applied to a one-line vehicle display system 1. Please refer to FIGS. 1 and 2, which can be assembled into one vehicle. The body H includes a control unit 10, a photographing module 11 and a display screen 13. In one embodiment, the control unit 10 is electrically connected to the photographing module 11 and the display screen 13, respectively, so as to be able to receive the photograph. The image from the module 11 sends a message to the display screen 13, and a lens 111 of the photographic module 11 is assembled to the rear outside the vehicle body H, and the angle of view of the lens 111 is At least greater than 140 degrees. In this embodiment, the lens 111 can be a fisheye lens, mainly because the angle of view of the fisheye lens is close to 180 degrees. However, in other embodiments of the present invention, the industry can also A wide-angle lens or other kind of lens 111 is used, as long as the angle of view of the lens 111 can exceed at least 140 degrees, it is the lens 111 according to the present invention. In addition, the photography module 11 can include at least the lens 111 and the photosensitive coupling in practice. Components, image processing And other electronic components, and the aforementioned image processor can also be integrated into the control unit 10.

Please refer to FIG. 1 to FIG. 3, the display screen 13 is disposed in the vehicle body H, and a panoramic picture area 131 and a turning picture area 133 are divided thereon. After receiving a message from the control unit 10, The image included in the message can be displayed. In this embodiment, the display screen 13 can be an independent device. However, in other embodiments of the present invention, the display screen 13 can also be connected with a rear view mirror, Integrated electronic dashboard or car computer. In addition, when the user wants to make a right turn in reverse, the shift lever is moved to the R position, and at the same time, the direction light lever is moved to the position of the right turn light. At this time, the control unit 10 will receive a reverse signal and Upon a turn signal, the control unit 10 will receive a plurality of distorted images sequentially transmitted by the photographing module 11, mainly because the angle of view of the lens 111 is large, so the images taken by it will show a large distortion. In this embodiment, the lens 111 is a circular fisheye (also known as a round fisheye) type. Therefore, The captured image will be circular (as in the deformed image A shown in Fig. 4); however, in other embodiments of the present invention, as the structure type of the lens is different, the deformation method of the displayed image is also different. For example, the picture taken by a diagonal fisheye (Diagonal fisheye or Full-frame fisheye) is square; therefore, no matter how the picture taken by the lens 111 is deformed, it is a deformed image A in the present invention.

Please refer to FIG. 2 and FIG. 4A again, the control unit 10 will intercept the partial image R ′ on the right from the deformed image A according to the turn signal (eg, turn right), and then the control unit 10 will Distorted image A and local image R 'execute a distortion correction program. It is stated here that the distortion correction program can be interpolation method, polynomial function curve fitting method, etc., whose main function is to make the image taken by a fisheye lens. The correction is performed to form a perspective projection image used by the human eye. Since the aforementioned interpolation method and polynomial function curve fitting method are already known techniques, they are not described in detail, as long as the deformed image A undergoes a distortion correction program, it can form a person. The perspective projection image used by the eyes is the distortion correction procedure described in the present invention, which will be described first.

In addition, please refer to FIGS. 1 to 3 and 4A. After the distortion image A undergoes the distortion correction procedure, a panoramic image is formed. After the partial image R ′ undergoes the distortion correction procedure, a turning image is formed. Among them, the panoramic image is the scenery behind the car body, and the turning image is only the scenery behind the right side of the car body. Alas, the control unit 10 will transmit the panorama image and the turning image to the display screen 13, Make the panoramic image area 131 of the display screen 13 display the panoramic image, and the turning image area 133 display the turning image, wherein the panoramic image will also include the content of the turning image; similarly, please refer to sections 1 to 3 and As shown in Fig. 4B, when the turn signal is a left turn, the control unit 10 will capture the left partial image L 'from the deformed image A. After that, the control unit 10 will take the deformed image A and the partial image L. 'Perform a distortion correction procedure to form pairs Ying's panoramic image and turning image, where the panoramic image is the scenery behind the car body, the turning image is only the scenery behind the left side of the car body, and the control unit 10 transmits the panorama image and the turning image Go to the display screen 13 so that the panoramic image area 131 of the display screen 13 displays the panoramic image, and the turning screen area 133 displays the turning image. In addition, although the reverse turning is taken as an example in the foregoing embodiment, In other embodiments of the present invention, please refer to FIG. 1 to FIG. 3 again. As long as the control unit 10 receives the turn signal (such as right turn or left turn), the foregoing display method is performed, and The picture area 131 displays the panoramic image, and the turning picture area 133 displays the turning image for the driver to view the situation behind the vehicle body H.

In this way, please refer to FIG. 1 to FIG. 3 again. With the method of the present invention, the following effects can be achieved: (1) Although limited by the space inside the vehicle body H, the size of the display screen 13 is usually small, but When the user makes a forward turn or a reverse turn, in addition to being able to see the scenery behind the vehicle body H through the panoramic image, the user can also use the turning screen area 133 to particularly see the scene at the right and rear of the vehicle body H or borrow. From the turning screen area 133, the scene at the left and rear of the vehicle body H is specifically observed to avoid the problem of collision with obstacles; (2) The driving display system 1 can use only a single lens 111 to simultaneously display on the display screen 13 The panoramic image and the turning image are displayed. Compared with the conventional reversing device using multiple photographic lenses, the driving display system 1 of the present invention can obviously reduce the money that the user spends on hardware; (3) when the user When turning forward or turning in reverse, the display screen 13 will simultaneously display the panoramic image and the turning image, without the need for the user to distract manually, so it can greatly improve the safety of the user traveling forward or reversing.

In order to clearly disclose the method of the present invention, only the control unit of the present invention is described below. The processing flow of 10 is explained. Please refer to Figures 1 to 5. After receiving the turn signal, the control unit 10 executes the following steps: (201) The control unit 10 receives the photographing module 11 and transmits it in order. The plurality of deformed images come to step (202); (202) The control unit 10 judges that the turn signal is a right turn or a left turn. If it is a right turn, it goes to step (203). If it is a left turn, it goes to step. (204); (203) The control unit 10 intercepts the right partial image of the deformed image and proceeds to step (205); (204) The control unit 10 intercepts the left partial image of the deformed image and proceeds to step (205); (205) the The control unit 10 performs a distortion correction procedure on the deformed image and the partial image to form a panoramic image and a turning image, respectively, and proceeds to step (206); (206) The control unit 10 transmits the panoramic image and the turning image to the The display screen 13 is configured such that the display screen 13 displays the panoramic image in the panoramic screen area 131 and the turning screen area 133 displays the turning image.

In summary, please refer to Figures 1 to 4B. The driving display system 1 only needs a single lens 111 to display the panoramic image and the turning image on the display screen 13 at the same time. In another embodiment of the invention, after receiving the deformed image A, the control unit 10 can first execute the distortion correction procedure on the deformed image A to form a panoramic image. After that, the control unit 10 will perform the turning signal according to the turn signal ( For example: turn left or turn right) from the panoramic image, a corresponding partial image (such as the left partial image or the right partial image) is captured to form the turning image, that is, the control unit 10 performs a single deformation image A only needs to perform the distortion correction procedure once, and does not need to perform the secondary distortion correction procedure for a single deformed image A as in the previous embodiment (that is, the deformed image A and the partial images R ', L' are respectively performed). Distortion correction program), so this control can be reduced The operational burden of the unit 10.

In addition, please refer to FIG. 1 and FIG. 2. When the vehicle body H is in the reverse state, in order to remind the driver to pay attention to the rear objects on the screen, a warning device 15 is still installed in the vehicle body H. The alarm device 15 can It is a buzzer, a light or a voice unit of a vehicle-mounted computer, which can be electrically connected to the control unit 10 and can generate a warning effect, such as sound, light, etc. When the control unit 10 receives a reversing signal, and After the panoramic image is transmitted to the display screen 13, the control unit 10 executes an optical flow estimation procedure on the panoramic image and the turning image to determine the moving direction of at least one object in the panoramic image. It is mentioned that the optical flow (optical flow or optical flow) is used to describe the surface or edge changes caused by the objects in the field of view in motion. It is a known technology and will not be described in detail. In this embodiment, The optical flow estimation program can be a least square error estimation method. However, in other embodiments of the present invention, the operator can adopt a different optical flow estimation program, as long as the control unit 10 can estimate the optical flow using the optical flow estimation method. Sequence and that the moving direction of the object in the panorama image, that is, the optical flow estimation procedure of the present invention.

Please refer to FIG. 1 and FIG. 2 again. After the control unit 10 learns the moving direction of the objects in the panoramic images, it will obtain the horizontal and vertical edges of each of the objects in the panoramic images. The object is a three-dimensional object or a plane marking. After the control unit 10 determines that each of the objects is a three-dimensional object, it calculates the distance between each object and the vehicle body H, and then calculates the distance between each object and the vehicle body H. After the distance is equal to or less than a warning distance value (for example, 100 cm), a first warning message is generated and transmitted to the warning device 15 to cause the warning device 15 to produce the warning effect. However, in other aspects of the present invention, In the embodiment, the operator may also set multiple warning distance values in the control unit 10, and set corresponding warning effects according to different warning distance values. For example, if the warning distance value is 100 cm, the warning effect is generated every 2 seconds. A warning tone; the warning distance value is 50 cm, then The warning effect is to generate a warning sound every 1 second; the warning distance value is 20 cm, the warning effect is to generate a warning sound every 0.5 seconds, and the lights are flashing ... etc.

Moreover, when a three-dimensional object appears behind the vehicle body H, in order to make the driver pay attention to the distance between the vehicle body H and the three-dimensional object, the control unit 10 will adjust the display area of the panoramic image accordingly. For example, please Refer to Figures 2, 3, 6A, and 6B. When there is no three-dimensional object behind the vehicle body H, or the distance between the three-dimensional object and the vehicle body H is greater than a first distance value L1 (eg, 100 cm), it means that When there is no three-dimensional object within the range of the first distance value L1, the control unit 10 causes the panoramic image area 131 of the display screen to display the panoramic image, wherein the panoramic image of FIG. 6B includes a planar road marking T; see As shown in Figures 2, 3, 7A, and 7B, when the first distance value L1 behind the vehicle body H has a three-dimensional object (hereinafter referred to as an obstacle M), and the distance between the obstacle M and the vehicle body H is greater than a second When the distance value is L2 (for example: 30 cm), the control unit 10 will capture a partial local image from the panoramic image, such as a local image with a horizontal viewing angle between 100 degrees and 140 degrees to form a near-end panoramic image. After that, the control unit 10 transmits the near-end panoramic image to the display screen 13 so that The screen area 131 displays the near-end panoramic image. As shown in FIG. 7B, it can be found from the position of the road marking T on FIG. 7B that the near-end panoramic image is compared with the panoramic image of FIG. 6B. The intercepted horizontal angle of view is really small; please refer to Figures 2, 3 and 8A, 8B, when the distance between the obstacle M and the vehicle body H is equal to or less than the second distance value L2 (such as: 30 cm) The control unit 10 intercepts the lower part of the panoramic image from the panoramic image, such as a local image with a horizontal viewing angle of 60 degrees to 90 degrees to form a top panoramic image. After that, the control unit 10 will generate the top panoramic image. The image is transmitted to the display screen 13 to display the top view panoramic image in the panoramic screen area 131. As shown in FIG. 8B, there is no road marking T thereon, so it can be known that the top view panoramic image is compared with As far as the near-end panoramic image in FIG. 7B is concerned, the horizontal angle of view is indeed smaller. In addition, here is a special mention, The control unit 10 uses the distance closest to the vehicle body H in each of the three-dimensional objects (ie, the obstacle M) as the reference for adjusting the panoramic image.

According to the above, it is only the preferred embodiment of the present invention, but the scope of the rights claimed by the present invention is not limited to this. According to the technical content disclosed by those skilled in the art, Equivalent changes that can be easily considered should all belong to the protection scope of the present invention.

Claims (8)

A display method capable of simultaneously displaying a rear panorama and a turning screen when a vehicle turns, which is applied to a line of vehicle display system, which can be assembled on a vehicle body, and includes a control unit, a photographing module, and a display. Screen and a warning device, wherein a lens of the photographic module is assembled to the rear of the vehicle body, and the angle of view of the lens is at least 140 degrees, the display screen is set in the vehicle body, and a panoramic picture is divided thereon Area and a turning picture area, the control unit is electrically connected to the camera module and the display screen, respectively, so as to be able to receive the image from the camera module and send a message to the display screen, the warning device is connected with the control The units are electrically connected and set in the vehicle body, and can produce a warning effect. After receiving a turn signal, the control unit will perform the following steps: receive a plurality of distorted images sequentially transmitted by the camera module; according to The turn signal intercepts a corresponding partial image from the distorted image; performs a distortion correction procedure on the distorted image and the partial images to separate Forming a panoramic image and a turning image in sequence; transmitting the panoramic image and the turning image to the display screen, wherein the panoramic image area will display the panoramic image, and the turning image area will display the turning image; and The control unit receives a reversing signal and transmits the panoramic images to the display screen. The control unit performs an optical flow estimation procedure on the panoramic images to determine the movement of at least one object in the panoramic images. Direction, and obtain the horizontal and vertical edges of at least one object in the panoramic images to determine that each object is a three-dimensional object or a plane graticule, and after determining that each object is a three-dimensional object, calculate each object The distance from the vehicle body, in the case that the distance between the object and the vehicle body is equal to or less than a warning distance value, a first warning message is generated and transmitted to the warning device, so that the warning device generates the Warning effect. The display method according to claim 1, wherein the control unit is determining the panoramic images When each of the objects is a three-dimensional object and the distance between any three-dimensional object and the vehicle body is greater than a first distance value, the control unit causes the panoramic image area of the display screen to display the panoramic image. The display method according to claim 2, wherein the control unit determines that each of the objects in the panoramic images is a three-dimensional object, and the distance between any three-dimensional object and the vehicle body is less than the first distance value, and In the case of a value greater than a second distance, the control unit still performs the following steps: intercepting a partial local image from the panoramic image to form a near-end panoramic image; and transmitting the near-end panoramic image to the display screen To display the near-end panoramic image in the panoramic picture area. The display method according to claim 3, wherein the control unit determines that each of the objects in the panoramic images is a three-dimensional object, and the distance between any three-dimensional object and the vehicle body is equal to or less than the second distance value In the case of the control unit, the control unit still executes the following steps: from the panoramic image, the lower partial image is captured to form a top panoramic image; and the top panoramic image is transmitted to the display screen to be displayed in the panoramic screen area The top panoramic image. A display method capable of simultaneously displaying a rear panorama and a turning screen when a vehicle turns, which is applied to a line of vehicle display system, which can be assembled on a vehicle body, and includes a control unit, a photographing module, and a display. Screen and a warning device, wherein a lens of the photographic module is assembled to the rear of the vehicle body, and the angle of view of the lens is at least 140 degrees, the display screen is set in the vehicle body, and a panoramic picture is divided thereon Area and a turning picture area, the control unit is electrically connected to the camera module and the display screen, respectively, so as to be able to receive the image from the camera module and send a message to the display screen, the warning device is connected with the control The unit phases are electrically connected and set in the vehicle body, and can produce a warning effect. After receiving a turn signal, the control unit will perform the following steps: Receiving a plurality of distorted images sequentially transmitted by the photographic module; performing a distortion correction procedure on the distorted images to sequentially form a panoramic image; and capturing a corresponding part from the panoramic image according to the turn signal The image to form a turning image; transmitting the panoramic image and the turning image to the display screen, wherein the panoramic image area will display the panoramic image, and the turning image area will display the turning image; and receiving at the control unit When a reversing signal is transmitted and the panoramic images are transmitted to the display screen, the control unit performs an optical flow estimation procedure on the panoramic images to determine the moving direction of at least one object in the panoramic images, and Obtain the horizontal and vertical sides of at least one object in the panoramic images to determine whether each object is a three-dimensional object or a plane graticule, and after determining that each object is a three-dimensional object, calculate each object and vehicle body The distance between each object to generate a first warning message when the distance between the object and the vehicle body is equal to or less than a warning distance value, and transmit The alarm device, enabling the alarm device generating the alarm effect. The display method according to claim 5, wherein the control unit determines that each of the objects in the panoramic images is a three-dimensional object, and a distance between any three-dimensional object and the vehicle body is greater than a first distance value Next, the control unit causes the panoramic image area of the display screen to display the panoramic image. The display method according to claim 6, wherein the control unit determines that each of the objects in the panoramic images is a three-dimensional object, and the distance between any three-dimensional object and the vehicle body is less than the first distance value, and In the case of a value greater than a second distance, the control unit still performs the following steps: intercepting a partial local image from the panoramic image to form a near-end panoramic image; and transmitting the near-end panoramic image to the display screen To display the near-end panoramic image in the panoramic picture area. The display method according to claim 7, wherein the control unit is determining the panoramic images In the case where each of the objects is a three-dimensional object, and the distance between any three-dimensional object and the vehicle body is equal to or less than the second distance value, the control unit still performs the following steps: from the panoramic image, the lower part is captured Image to form a top panoramic image; and transmitting the top panoramic image to the display screen to display the top panoramic image in the panoramic screen area.