KR20240041373A - Indication system and method for compensating for blind spots in tractor-trailer rearview mirrors - Google Patents

Abstract

The display system for compensating for the blind spot of the tractor-trailer rearview mirror of the present invention includes a collection device, a detection device, a display control device, and a display device, and the collection devices include a left front camera, a right front camera, a left rear camera, and a right front camera. It includes a rear camera, the detection device is an angle detection sensor disposed on the tractor-trailer, the angle detection sensor is used to obtain the angle at which the axis of the trailer deviates from the axis of the head of the vehicle, and the display control device is a collection device and a detection device. The device is connected, and the display device is placed on the display in the driver's cab and used to display the image output by the display control device to the driver. The present invention performs screen compensation for the area obscured by the trailer in the rearview mirror during the rotation process through the arrangement of multiple cameras, effectively solves the problem of the trailer blocking the view, and creates a panorama by splicing a general fisheye lens. Compared to images, drivers more intuitively observe where dangerous elements exist, thus reducing thinking time and reducing safety accidents.

Description

Indication system and method for compensating for blind spots in tractor-trailer rearview mirrors

This application has the priority and title of a Chinese patent application (application number: 202110901387.8) titled “Indication system and display method for compensation of blind spots in the field of view of tractor-trailer rearview mirrors” and titled “Automatic exposure adjustment method and automatic exposure adjustment device.” Priority is claimed on the Chinese patent application (application number: 202210100871.5).

The present invention relates to the technical field of vehicle-mounted electronic devices, and specifically to a display system and method for compensating for blind spots in a tractor-trailer rearview mirror.

Rearview mirrors and electronic rearview mirrors are important parts to ensure driving safety when driving a vehicle. In GB-15084, the performance standard for automobile rearview mirrors, the observation range of a vehicle rearview mirror is the viewing range when the vehicle is in a normal stationary state, and observation of both sides of the field of view is required. The range varies within 25 to 45 degrees outside the vehicle's central axis (formed by different observation points), but the observation range of the vehicle's field of view in any specific environment is not specified.

In a tow vehicle or winch vehicle, the head and the trailer are fixed or hinged in the axial direction, so when the vehicle moves backwards or turns, the driver observes the images on both sides of the trailer and the front and rear to assist the driver. However, there are situations where the separation angle of the head of the vehicle is excessive and the view of the rear-view mirror or electronic rear-view mirror on one side is blocked by the trailer depending on the rotation angle. Drivers can only observe the condition of one side of the vehicle, and single-side inner-wheel vehicle accidents and rear-end collision accidents are likely to occur in complex and unfamiliar environments.

In the prior art, for example, in the Chinese patent with application number 201811278439.5, image information is collected and acquired through fisheye cameras placed in various places on the vehicle body, and pre-processing and splicing are performed on the image information; The image after splicing is shown to the driver on a display, allowing the driver to obtain a 360-degree view of the vehicle. The 360-degree view is obtained by splicing a panoramic view with 4 to 6 cameras, and the cameras must be fixed at a certain position. It must not have deflection movement and is suitable for use at low speeds in buses or integrated trucks. The tow vehicle can be moved by the trailer at any time, and the panorama cannot be spliced into a complete image as the video collection range changes.

Panoramic cameras have large fisheye image distortions, and because they are driving assistance products, there are no legal requirements for depth of subject, object distance, etc., and they can only be used as assistance-type products. Since the panoramic camera is mounted to survey downward, it can only see the area near the vehicle depending on the camera survey angle, and cannot see the field of view far from the vehicle.

The present invention is used as a product regulated by mandatory laws to replace rear-view mirrors, and the laws require a clear observation range. In addition, the camera performance tests, usage conditions, viewing area required by law, and safety requirements are not equivalent for the products.

The purpose of the present invention is to solve the marking of blind spots obscured by a single-side trailer that occurs when the head of the vehicle is shifted when turning or reversing the vehicle in light of the prior art, and to prevent the driver from detecting the possibility of danger occurring in the blind spot. To provide a display system and display method for compensating for blind spots in the field of view of a tractor-trailer rearview mirror that quickly determines the position, and in a preferred embodiment, the present invention provides a tilt-shift arrangement method and a light sensitivity for the camera of the collection device. A shooting adjustment method based on the present invention is added, and the performance of the present invention is further improved.

Specifically, it is a display system for compensating for blind spots in the field of view of a tractor-trailer rearview mirror. The display system includes a collection device, a detection device, a display control device, and a display device, and the collection device includes a left front camera and a right front camera. , including a left rear camera and a right rear camera, wherein the left front camera and the right front camera are respectively disposed on both left and right sides outside the cab, and the left rear camera and the right rear camera are disposed on both left and right sides of the trailer, respectively, and the detection The device is an angle detection sensor disposed on a tractor-trailer, the angle detection sensor is used to obtain the angle at which the axis of the trailer deviates from the axis of the head of the car, and the display control device is connected to the collection device and the detection device. It includes a processor, wherein the processor obtains an angle at which the axis of the trailer is deviated from the axis of the car head through the sensing device, and directly transmits and/or cuts and synthesizes the screen obtained by the collection device to the display device. and then used to transmit to the display device. The display device is placed on the display in the driver's cab, and the display device is connected to the display control device to display the image output by the display control device to the driver. It is used.

In addition, the field of view of the screen collected by the left front camera and the right front camera conforms to the class II field of view and/or class IV field of view specified in the standard, and the left rear camera, right rear camera, left front camera, and right front camera The screen collected includes the trailer body on the side where the camera is located.

In addition, the display device includes a left display unit and a right display unit, and the left display unit is for displaying a screen collected by the left front camera or a screen synthesized by the left front camera and the left rear camera, The right display unit is for displaying a screen collected by the right front camera or a composite screen between the right front camera and the right rear camera.

In addition, in the process of the processor cutting and compositing the screen, the screen of a preset width or fixed width on the side obscured by the trailer of the front camera in the direction opposite to and/or in the rotation direction of the car head is removed, and the screen of the preset width or fixed width is removed when the car head rotates. A screen of a preset width or fixed width on the trailer side of the rear camera in the opposite direction and/or rotation direction is reserved, and a new screen is synthesized by placing the reserved screen in the area from which the screen was removed.

In addition, when the angle detection sensor detects that the angle between the head of the vehicle and the central axis of the trailer exceeds the irradiation angle of the front camera on the rotating side, the processor uses the fixed-width screen for cropping and compositing. , the fixed width is the width occupied by the trailer on the screen of the front camera on the rotation side.

In addition, the display control device includes a storage unit, and the storage unit stores display blocks The processor performs calculations and judgments according to the detection of the sensing device, and the pre-stored compensation blocks, Ya and compensation blocks that match the preset width in the Y set, also have a fixed width or preset width in the Z set. The width compensation block also finds Za, and the processor simultaneously cuts the screen of a preset width on the trailer side of the rear camera in the opposite direction of rotation of the car head, and also projects the compensation block that finds the processed screen onto Ya. The processor cuts a fixed-width screen on the trailer side of the rear camera in the direction of car head rotation, and also projects the compensation block found on the processed screen onto Za.

In addition, the preset width is set according to the angle between the head of the vehicle and the central axis line of the trailer, and when the angle between the head of the vehicle and the central axis line of the trailer is 0 degrees to 15 degrees, the preset width is 0, and the preset width is 0. If the angle between the head of the vehicle and the central axis of the trailer is 15 degrees to 30 degrees, the preset width is 1/3 of the screen width, and if the angle between the head of the vehicle and the central axis of the trailer is 30 to 45 degrees. , the preset width is 2/3 of the screen width, and when the angle between the head of the car and the central axis of the trailer is more than 45 degrees, the preset width is the width of the entire screen.

Preferably, one or more of the left front camera, right front camera, left rear camera, and right rear camera uses a tilt-shift camera, and the tilt-shift camera includes a camera body, an image collection unit, and an imaging lens. And the imaging lens and the image collection unit are respectively mounted on the front end and the rear end of the camera body, and the camera body has a first angle θ inclined in a first direction relative to the reference direction, and the image collection unit The unit and the imaging lens have a second angle α and a third angle β inclined in a direction opposite to the inclination direction of the camera body, respectively, relative to the main axis of the camera body, and the second angle α and the third angle β are They are not equal to each other, and preferably, the second angle α is greater than the third angle β, so that the planes where the image collection unit and the imaging lens are located intersect with each other to form an acute angle.

A display method for compensating for blind spots in the field of view of a tractor-trailer rearview mirror is also provided. The display method includes steps 1 in which a processor acquires video images collected by a plurality of cameras in real time; Step 2, where the processor acquires the operating state of the tractor-trailer; and step 3, wherein the processor generates a screen of an optimally unobscured, minimal blind spot plan based on the operating state of the tractor-trailer, wherein the operating state includes a trailer state and a driving state, and the trailer state includes: It includes a trailer mode and a trailerless mode, and the driving state includes straight forward, turning, and reverse, and the plurality of cameras include a left front camera, a right front camera, a left rear camera, and a right rear camera, and the left front camera The camera and the right front camera are respectively placed on both left and right sides outside the cab, and the left rear camera and right rear camera are placed on both left and right sides of the trailer, respectively.

Additionally, in step 3, under the trailerless mode, the processor provides the screens of the left front camera and the right front camera on the display device, and when the driving state is straight under the trailerless mode, the processor provides the left front camera screen on the display device. It provides screens of the front camera and the right front camera, and when the driving state is turning in the trailer mode, the processor provides a screen to the display device based on the angle between the head of the vehicle and the collision axis line of the trailer, and the head of the vehicle In the cropping and compositing of the screen in the direction opposite to the rotation of the head of the car, the screen of the preset width is removed from the side obscured by the trailer of the front camera in the direction opposite to the rotation of the head of the car, and the trailer side of the rear camera in the direction opposite to the rotation of the head of the car is removed. A screen of a preset width is reserved, a new screen is synthesized by placing the reserved screen in the area from which the screen was removed, and in the cutting and compositing of the screen in the rotation direction of the vehicle head, between the vehicle head and the central axis line of the trailer When the angle exceeds the irradiation angle of the front camera on the rotation side, the screen with a fixed width on the rotation side of the front camera in the direction of rotation of the vehicle head is removed, and the fixed width screen on the trailer side of the rear camera in the rotation direction of the vehicle head is removed. The screen is shelved and a new screen is synthesized by placing the shelved screen in the area from which the screen was removed, and the fixed width is the width occupied by the trailer on the screen of the front camera on the rotating side of the car head.

In addition, the preset width is set according to the angle between the head of the vehicle and the central axis line of the trailer, and when the angle between the head of the vehicle and the central axis line of the trailer is 0 degrees to 15 degrees, the preset width is 0, and the preset width is 0. If the angle between the head of the vehicle and the central axis of the trailer is 15 degrees to 30 degrees, the preset width is 1/3 of the screen width, and if the angle between the head of the vehicle and the central axis of the trailer is 30 to 45 degrees. , the preset width is 2/3 of the screen width, and when the angle between the head of the car and the central axis of the trailer is more than 45 degrees, the preset width is the width of the entire screen.

The present invention described above has the following effects.

The present invention, through the arrangement of a plurality of cameras, performs screen compensation for the area obscured by the trailer in the rearview mirror during the rotation process, effectively solves the problem of the trailer blocking the field of view, and also provides Compared to panoramic images, drivers can more intuitively observe where dangerous elements exist, thus reducing thinking time and effectively reducing safety accidents.

In the present invention, the camera placed at the front can satisfy the class II field of view of the GB-15084 standard and has the ability to replace a traditional rear-view mirror, and at the same time, combined with the camera placed at the rear of the trailer, it can provide a blind spot for the entire vehicle. Implement surveillance.

In the present invention, the field of view of the camera placed at the rear of the trailer must include the trailer itself, so when compensating for the screen, the driver can intuitively observe the positional relationship between dangerous elements and the trailer, thereby allowing the driver to It shortens the time to judge the content on the screen and allows a quick reaction to dangerous scenes.

The present invention confirms the size of the area requiring compensation using the vehicle's turning width and can implement mutual mapping between the compensation area and the obscured area, allowing the driver to continuously obtain a more intuitive view within the blind spot.

Figure 1 is a structural schematic diagram of a display system for compensating for blind spots in a tractor-trailer rearview mirror provided in an embodiment of the present invention.
Figure 2 is a schematic diagram of the viewing range requirements that must be satisfied by the left front camera and the right front camera in the display system for compensating for blind spots in the field of view of a tractor-trailer rearview mirror provided in an embodiment of the present invention.
Figure 3 is a schematic diagram of the viewing range requirements that the left rear camera and the right rear camera must satisfy in the display system for compensating for blind spots in the field of view of a tractor-trailer rearview mirror provided in an embodiment of the present invention.
Figure 4 is a schematic diagram of the distribution of cameras in a display system for compensating for blind spots in a tractor-trailer rearview mirror provided in an embodiment of the present invention.
Figure 5 is a schematic diagram of the field of view of each camera under the rotation state of the tractor-trailer in the display system for compensating for blind spots in the field of view of the rearview mirror of the tractor-trailer provided in an embodiment of the present invention.
Figure 6 is a schematic diagram of a display device on the rotating side providing an uncompensated screen under a rotating state of the tractor-trailer in a display system for compensating blind spots in the rearview mirror of a tractor-trailer provided in an embodiment of the present invention.
Figure 7 is a schematic diagram of a display device on the opposite side of rotation providing an uncompensated screen under a rotation state of the tractor-trailer in a display system for compensating for blind spots in the field of view of a tractor-trailer rearview mirror provided in an embodiment of the present invention.
Figure 8 is a schematic diagram of a display device on the rotating side providing a screen after compensation under a rotating state of the tractor-trailer in a display system for compensating blind spots in the rearview mirror of a tractor-trailer provided in an embodiment of the present invention.
Figure 9 is a schematic diagram in which, in a display system for compensating blind spots in a tractor-trailer rearview mirror provided in an embodiment of the present invention, a display device on the opposite side of the rotation provides a screen after compensation under a rotation state of the tractor-trailer.
Figure 10 is an evolution view in which each part of the camera is disposed obliquely in a preferred embodiment, where Figures A to C are side views and Figures D and E are plan views.
FIG. 11 is a side view of the angular relationship between the imaging lens 100, the image collection unit 200, and the camera body when taking downward tilt photography.
FIG. 12 is a plan view of the angle and displacement relationship of the imaging lens 100 relative to the main axis of the camera body when shooting at a downward angle.
Figure 13 is an image taken when the imaging lens 100 and the image collection unit 200 are arranged in parallel, although the camera is slightly inclined downward and toward the outside of the vehicle body.
Figure 14 shows that when the camera is mounted on a vehicle, the camera body takes pictures toward the rear of the vehicle and is tilted downward, and both the imaging lens 100 and the image collection unit 200 are relative to the main axis (middle axis of the lens barrel) of the camera body. This is an image taken tilted at a different angle in the opposite direction.
Figure 15 is a flowchart of a display method for compensating blind spots in a tractor-trailer rearview mirror provided in an embodiment of the present invention.

Below, the technical solutions of the present invention are combined and described in more detail, and the present invention includes, but is not limited to, the following embodiments.

As shown in Figure 1, the display system for compensating for blind spots in the tractor-trailer rearview mirror of the present invention includes a collection device, a detection device, a display control device, and a display device.

As shown in Figures 2 to 4, the collection device includes a left front camera, a right front camera, a left rear camera, and a right rear camera. A left front camera and a right front camera are placed on both left and right sides outside the cab of the tractor-trailer, respectively. The lenses of the left front camera and the right front camera are directed toward the rear side of the side on which they are located, and the lenses of the left front camera and the right front camera are located on both sides outside the cab of the tractor-trailer. This is to collect real-time screens of the rear side. The left front camera and right front camera are intended to collect class II and IV fields of view that meet the provisions of the GB-15084 standard. A left rear camera and a right rear camera are disposed on both left and right sides near one end of the cab of the tractor-trailer trailer, respectively, and the lenses of the left rear camera and right rear camera are directed toward the rear of the side on which they are located, It is to collect real-time screens of the rear side of both sides of the trailer, the lenses of the left rear camera and right rear camera and the mounting height and angle of the left front camera and right front camera are close, and the lenses of the left rear camera and right rear camera are close, and the mounting height and angle of the left rear camera and right rear camera The swing angle of the lens must match that of the trailer, and the screen collected by the lens must include the trailer body on the side where the camera is located.

The detection device is an angle detection sensor placed on the tractor-trailer, and the angle detection sensor is used to obtain the angle between the head of the tractor-trailer and the trailer in real time. In another embodiment, the detection device is implemented with software integrated in the processing device, and monitors the real-time screen acquired by the collection device, and also monitors the screen collected by the camera placed on the head of the vehicle and the trailer through various algorithms. By analyzing the screen collected by the deployed camera, the angle between the head of the car and the trailer is determined from the image.

The display control device is connected to the collection device and the detection device, respectively, the display control device includes a processor and a storage unit, the processor and the storage unit are connected to each other, and the processor provides the angle data between the vehicle head and the trailer obtained by the detection device. Using , the corresponding collection device on the trailer is started to cut and combine the real-time screen acquired by the collection device, and the storage unit is used to store the screen acquired by the collection device and the screen cut by the processor.

As shown in Figures 5 to 9, the screen collected by the left front camera is screen C, the screen collected by the right front camera is screen D, the screen collected by the left rear camera is screen E, and the screen collected by the right rear camera is screen E. The collected screen is screen F, and in the composite image, when the vehicle turns left or reverses to the left as shown in Figure 5, the screen observed by the driver on the left is screen C shown in Figure 6, and the driver on the right shows The observed screen is D shown in Figure 7. Since a large area of the left field of view is filled with the trailer and there is a relatively large blind spot in the right field of view, both sides need to perform a certain degree of screen compensation. , After compensation, the screen observed by the driver on the left is C+E shown in FIG. 8, and the screen observed by the driver on the right is D+F shown in FIG. 9.

Specifically, the compensation plan on the screen is as follows. First, the processor determines the width of the right compensation screen based on the angle at which the axis of the trailer deviates from the axis of the head of the vehicle, and then the corresponding left side closer to the trailer side in the right rear camera screen based on the width of the compensation screen. Obtain a screen width, and then, based on the width of the compensation screen, cut out a screen with a width corresponding to the part obscured by the left trailer side from the right front camera screen, and finally, change the right rear camera screen to the right front. By filling in the cropped position on the camera screen, an image with blind spot compensation is synthesized. For the left compensation screen, first, the processor determines the rotation angle of the vehicle based on the angle by which the axis of the trailer deviates from the axis of the head of the vehicle, and then the rotation angle exceeds the cover angle of the left front camera (the cover angle above). When the angle is a narrow angle between the edge of the field of view of the left front camera and the side wall of the head of the vehicle, a screen of a specified width on the side closest to the right trailer among the left rear camera screens is obtained based on the width of the compensation screen, and then Based on the width of the compensation screen, the screen with a specified width on the right is cut from the left front camera screen, and finally, the left rear camera screen is filled in the cut position in the left front camera screen to create an image with blind spot compensation. When the rotation angle is less than the irradiation angle of the left front camera, compensation is not performed and only the screen collected by the left front camera is displayed.

Similarly, when the vehicle turns right or reverses to the right, the images of both front and rear cameras are composited, the two rear cameras can always be in an operational state, and after determining whether the vehicle is turning right or reversing to the right, the two rear cameras may be activated, and for the left compensation screen, first, the processor determines the width of the compensation screen based on the angle at which the axis of the trailer deviates from the axis of the car head, and then, based on the width of the compensation screen, the processor determines the width of the compensation screen on the left. Obtain a screen with a corresponding width on the right side close to the trailer side from the rear camera screen, and then, based on the width of the compensation screen, obtain a screen with a corresponding width on the right side of the part obscured by the trailer side on the left front camera screen. Cutting, and finally, the left rear camera screen is filled in the cut position of the left front camera screen, and an image with blind spot compensation is synthesized. For the right compensation screen, first, the processor determines the turning angle of the vehicle based on the angle by which the trailer's axis deviates from the head's axis, and then determines if the turning angle exceeds the cover angle of the right front camera (above). The cover angle is the narrow angle between the edge of the field of view of the right front camera and the head side wall), based on the width of the compensation screen, obtain a screen of a specified width on the left side closest to the trailer side from the right rear camera screen, and then, Based on the width of the compensation screen, the screen with a specified width on the left is cut from the right front camera screen, and finally, the right rear camera screen is filled in the cut position on the right front camera screen to create an image with blind spot compensation. When the rotation angle is less than the irradiation angle of the right front camera, compensation is not performed and only the screen collected by the right front camera is displayed.

In one embodiment, in compensation for the screen on the opposite side of the rotation direction of the head of the tractor-trailer, a set of pre-produced display blocks X and compensation blocks Y used for screen compensation are stored in the storage device, , the resolution of the display block diagram X and the resolution provided by the corresponding display device correspond to each other, the compensation block Ya is overlapped within the display block diagram The width of the compensation block diagram Ya is determined based on the angle at which the trailer axis deviates from the vehicle head axis, and when the angle at which the trailer axis deviates from the vehicle head axis reaches a preset range, the corresponding width is determined. The compensation block also automatically finds Ya, and the compensation block Ya can be rectangular or a special shape established according to the blind spot of specific tractor-trailer and trailer specifications, and the compensation block in the left field of view is also Y and the compensation block in the right field of view. A symmetrical design is used with respect to Figure Y, and the compensation block diagram Y in the left field of view is placed on the right side of the display block diagram is placed in

When the angle at which the axis of the trailer deviates from the axis of the head of the vehicle reaches a preset range, the compensation block diagram Ya of the corresponding size is based on the angle at which the axis of the trailer deviates from the axis of the head of the vehicle. Find it. For the display block diagram X on one side that needs to be compensated, the processor fills the screen of the front camera on that side into the display block diagram charge within the figure, cut out the screen corresponding to the position of the compensation block Ya, and project the cut screen onto the position corresponding to the compensation block Ya within the display block diagram X, and display the compensation block Ya on the front camera screen. By covering the screen at the corresponding position, a screen with a compensated field of view is created through cutting and compositing.

In the compensation of the screen on the rotation side of the tractor-trailer, the storage device stores a pre-made compensation block Z set used for screen compensation, and the compensation block Za is overlapped with the display block X, and the compensation block is also Za The compensation block is also an element in the Z set, and the specified width of Za in the compensation block can use a fixed width, and can be determined based on the angle at which the axis of the trailer deviates from the axis of the head of the vehicle. When the deviation angle from the axis of the vehicle head reaches the preset range, the compensation block of the corresponding width also automatically finds Za, and the compensation block Za is also rectangular or in the blind spot of specific tractor-trailer and trailer specifications. It may be a special shape established accordingly, and the compensation block Z in the left field of view and the compensation block Z in the right field of view use a symmetrical design, and the compensation block Z in the left field of view is placed to the left of X. And, the compensation block Z in the right field of view is also placed to the right of X in the right field of view.

The compensation block diagram Za finds the corresponding compensation block diagram Za based on the angle at which the axis of the trailer deviates from the axis line of the car head. The size of the compensation block diagram Za can be the width occupied by the trailer on the screen of the front camera, and the trailer The axis of may be a corresponding size generated based on the angle offset from the axis of the vehicle head. In the display block diagram X on one side that needs compensation, the processor fills the screen of the front camera on the side into the display block diagram charge within the screen, cut out the screen corresponding to the position of Za in the compensation block, and project the cut screen onto the position corresponding to Za in the compensation block in the display block X, so that the compensation block also corresponds to Za in the front camera screen. Covers the screen and creates a screen with a compensated field of view through cutting and compositing.

The display device is placed on the display in the driver's cab, and the display device is connected to the display control device to show the driver a composite image output from the display control device. The display device includes a left display unit and a right display unit, The left display unit is for displaying the screen collected by the left front camera or a composite screen between the left front camera and the left rear camera, and the right display unit is for displaying the screen collected by the right front camera or a composite screen between the right front camera and the right rear camera. It is one screen, and the two display units on the left and right can be integrated into one display and displayed for each area, or each can be an independent display. In addition to being connected to the display control device, the display device can also be connected to the vehicle's control bus or other vehicle control units to show the driver each status parameter of the vehicle. The display of the display device can be integrated in the central control area or the electronic rearview mirror, and the display device of the electronic rearview mirror can be placed on the A pillar on both sides of the cab.

In one embodiment, while the tractor-trailer is traveling in a straight line, the processor acquires the video screens of the left front camera and the right front camera in real time, directly transmits the video screens to the display device without processing them, and sends the video screens to the display device. Displays the rear screen on both left and right sides of the tractor-trailer. When the tractor-trailer starts to turn or reverse during the driving process, the sensing device transmits the detected angle between the head of the vehicle and the central axis of the trailer to the processor in real time, and the processor calculates the angle between the head of the vehicle and the central axis of the trailer. Based on this, the video screens collected by the front and rear cameras on both sides are cut out to a size corresponding to the angle, and in the process of cutting, the original video screen and the video screen after cutting are all cached through the storage unit, and the processor makes the final decision. The cut video screens are composited, and the composited video screens on both sides are transmitted and displayed in the corresponding area of the display device. Of course, the driver manually controls the display device to display the vehicle Class II mirror display mode provided by the left front camera and the right front camera, and the manual control mode has priority over the display screen provided by the present invention.

Specifically, in the screen compensation on the opposite side of the tractor-trailer rotation direction, the left display unit and the right display unit are arranged on two displays on the A pillars on both left and right sides of the cab, respectively. The screen width of each display is 109.5 mm, and the resolution is 1920. *720 pixels; When the angle between the head of the car and the central axis of the trailer is 15, the display width of the compensation screen on the display is 36.5mm; When the angle between the car head and the trailer's central axis is 30, the display width of the compensation screen is 73 mm; When the angle between the vehicle head and the trailer's central axis is 45 degrees, the display shows the full compensation screen. In screen compensation on the turning side of the tractor-trailer, if the angle between the head of the vehicle and the central axis of the trailer is greater than the angle of the front camera's field of view, the display compensates and displays a screen with a fixed width, and the fixed width is collected by the front camera. In one screen, the trailer occupies a fixed width on the screen, and a compensation strategy of the same angle and width as the side opposite to the turning direction of the tractor-trailer can be used.

Preferably, one or more of the left front camera, right front camera, left rear camera, and right rear camera uses a tilt-shift camera. As shown in FIG. 10, each tilt-shift camera includes an imaging lens 100 and an image collection unit 200, and general structures such as the camera case and control chip will not be described in detail here. As shown in FIG. 11, both the imaging lens 100 and the image collection unit 200 are inclined in opposite directions relative to the main axis of the camera body (the middle axis of the lens barrel), and the imaging lens (relative to the main axis of the camera body) 100) and the tilt direction of the main axis of the camera body relative to the horizontal direction and the tilt direction of the image collection unit 200 are opposite, the tilt angles are the second angle α and the third angle β, respectively, 10 is the horizontal direction, and 20 , 30 indicate normal lines of the imaging lens 100 and the image collection unit 200, respectively. Preferably, the third angle β at which the image collection unit 200 is inclined is greater than the second angle α at which the imaging lens 100 is inclined, and the second angle α and the third angle β are not the same.

As shown in the plan views of FIGS. 10D and 10E and in FIG. 12, the imaging lens 100 is shifted relative to the main axis of the camera body (parallel movement), the shift distance is d, and the shift distance is approximately tan8°~ It is equal to the product of tan15° and the distance L between the imaging lens 100 and the image collection unit 200, that is, based on the center of the image collection unit, the center of the imaging lens is approximately 8 to 15 degrees relative to the main axis of the camera. The distance corresponding to is shifted, and the shift direction is a direction away from the vehicle body (this shift can be implemented by moving the image collection unit 200 in parallel toward the direction of the vehicle body relative to the main axis of the camera body), and at the same time, Both the imaging lens 100 and the image collection unit 200 rotate relative to their own vertical center line, and the rotation direction is away from the vehicle body. More specifically, because the camera shoots toward the rear of the vehicle body. , the imaging lens 100 and the image collection unit 200 both surround their respective vertical central axes and extend at a certain angle toward the outside of the rear of the car. rotates, preferably, the angle is between 8 degrees and 16 degrees, more preferably, the angle is and the distance at which the imaging lens 100 is relatively offset from the main axis of the camera body have a positive correlation with each other. Preferably, the rotation angles of the imaging lens 100 and the image collection unit 200 are the same, but the case where there is a slight difference in the rotation angles of the two is not excluded. From Figures 13 and 14, it can be seen that there is minimal deformation in the images captured by the tilt-shift camera.

As shown in FIG. 15, the present invention provides a display method for compensating for blind spots in the field of view of a tractor-trailer rearview mirror, including the following steps.

Step 1: The processor acquires video images collected by multiple cameras in real time.

Here, the plurality of cameras includes a left front camera and a right front camera disposed on both left and right sides outside the cab of the tractor-trailer, and a left rear camera and a right rear camera disposed on both left and right sides of one end of the trailer close to the cab. The left front camera and right front camera are used to collect class II and IV views that meet the provisions of the GB-15084 standard, and the left rear camera and right rear camera are used to collect the horizontal angles behind both trailers of the tractor-trailer. It is used to collect a 50-degree field of view, and the field of view must include the trailer body on the side where the camera is located.

Step 2: The processor acquires the operating status of the tractor-trailer.

Here, the operating state of the tractor-trailer includes a trailer state and a driving state, the trailer state includes a trailer mode and a non-trailer mode, and the driving state includes straight driving and turning. In the detection of the trailer status, the processor and the vehicle control bus are data connected to detect whether the trailer is mounted on the tractor-trailer through the vehicle control bus. The driving status is collected by the detection device, and the detection device is connected to the tractor-trailer. Using a deployed angle detection sensor, the angle at which the trailer axis deviates from the head axis is obtained using the angle detection sensor, and at the same time, using a software algorithm mounted on the processor, the trailer axis is determined through video images from each camera. The angle of deviation from the axis of the head can be determined.

Step 3: The processor generates a screen with optimal unobstructed and minimal blind spot options based on the operating status of the tractor-trailer.

In step 3, the processor provides various types of screens to the display device depending on the operating status of the tractor-trailer.

Specifically, when the trailer state is trailerless mode, the trailer is not mounted on the tractor-trailer, the processor does not further monitor the angle at which the axis of the trailer deviates from the axis of the head of the vehicle, and at the same time, because the trailer is not mounted, The processor also cannot acquire the image signals collected by the left rear camera and the right rear camera. At this time, the processor directly acquires the images from the left front camera and the right front camera and sends the acquired images to the display device to inform the driver of left and right rear cameras. It provides images corresponding to the field of view of the main outside rearview mirror and the left and right wide-angle outside rearview mirrors.

When the trailer status is in the trailer mode, the trailer is mounted on the tractor-trailer, and the processor determines the driving state of the tractor-trailer through the sensing device, and determines whether the tractor-trailer is going straight or the sensing device is off the axis of the trailer. If the processor detects that the angle of deviation from the head axis is less than 15 degrees, the processor determines that the driving state of the tractor-trailer is straight, and the detection device determines that the angle by which the trailer axis is offset from the head axis is less than 15 degrees. If it detects that the speed is above 100 degrees, the processor determines that the driving state of the tractor-trailer is rotating.

When the processor determines that the driving state of the tractor-trailer is straight, the processor directly acquires images from the left front camera and the right front camera and sends the images to the display device, showing the driver the left and right main outside rearview mirrors and the left and right wide angle outside rearview mirrors. Provides images corresponding to .

When the processor determines that the driving state of the tractor-trailer is rotating, it continuously detects the angle at which the axis of the trailer deviates from the axis of the head of the vehicle, and controls the screen displayed on the display device based on the detected angle.

Specifically, for example, in the case where the video screen of the left display unit is generated while a tractor-trailer is turning right or reversing to the right, the angle at which the axis of the trailer deviates from the axis of the head of the vehicle is detected to be 15 to 30 degrees. In this case, first, the processor acquires the video footage collected by the left front camera and the left rear camera, transfers the video footage to the storage unit for caching, and then the processor determines the angle at which the axis of the trailer deviates from the axis of the head of the car. Based on this, cut the video screen collected by the left front camera and the left rear camera, respectively, and cut out the 1/3 width of the video screen collected by the left front camera, which was obscured by the trailer on the right, forming a blind spot. In the video screen collected by the left rear camera, only the 1/3 width of the video screen near the trailer on the right is held, the cropped and held screen is cached through the storage unit, and then the held screen of the left rear camera is stored. The video screen is supplemented at the location where the video screen of the left front camera was cut and composited into the same video screen. Finally, the composite video screen of the left front camera and left rear camera is transmitted to the display system to display the left video screen portion. .

When detecting that the angle at which the trailer's axis deviates from the head's axis is 30 to 45 degrees, the processor first acquires the video screens collected by the left front camera and left rear camera, and transmits the video screens to the storage unit. Then, the processor cuts the video screens collected by the left front camera and the left rear camera based on the angle at which the axis of the trailer deviates from the axis of the car head, respectively. The video screens collected by the left front camera are The 2/3-width video screen that is obscured by the trailer on the right, forming a blind spot, is cut out, and the video screen collected by the left rear camera only holds the 2/3-width video screen close to the trailer on the right, and the video screen collected by the left rear camera is cut out. Then, the reserved screen is cached through the storage unit, and then the reserved video screen of the left rear camera is supplemented at the position where the video screen of the left front camera was cut out to composite the same video screen, and finally, the left front camera and the left front camera are cached. The composite video screen of the left rear camera is transmitted to the display system to display the left video screen area.

When it detects that the angle at which the trailer's axis deviates from the head's axis is more than 45 degrees, the processor directly acquires the image of the left rear camera and transmits the acquired image to the display system to display the left video screen area.

There are two cases in which the video screen of the right display unit is generated while the tractor-trailer is turning right or reversing to the right. In the first case, a compensation screen of a fixed size is used. First, if the processor determines that the rotation angle of the vehicle exceeds the irradiation angle of the right front camera based on the angle at which the axis of the trailer deviates from the axis of the head of the vehicle, The processor acquires the video screens collected by the right front camera and the right rear camera, transfers the video screens to the storage unit for caching, and then the processor cuts the video screens collected by the right front camera and the right rear camera, respectively. , from the video screen collected by the right front camera, the video screen the width occupied by the trailer on the right screen is cut out, and the video screen collected by the right rear camera only holds the video screen the width occupied by the trailer on the right screen, and is cut out. Then, the reserved screen is cached through the storage unit, and then the reserved video screen of the right rear camera is supplemented at the position where the video screen of the right front camera was cut to composite the same video screen, and finally, the right front camera and the right front camera are cached. The composite video screen of the right rear camera is transmitted to the display system to display the right video screen area. When the processor determines that the rotation angle of the vehicle does not exceed the irradiation angle of the right front camera based on the angle at which the trailer axis deviates from the vehicle head axis, the processor does not compensate and displays the video screen collected by the right front camera. It is sent directly to the display system to display the right video screen area.

In the second case, a method in which the compensation size and rotation angle are adapted to each other is used, and the specific compensation plan and the video screen generation method of the left display unit correspond to each other.

The process of a tractor-trailer turning left or reversing to the left corresponds to the situation described above.

In one preferred embodiment, each photographing device performs exposure adjustment using an automatic exposure adjustment method when taking a video, and the adjustment method includes the following steps.

Step 1: Collect data with a photosensitive sensor and generate photosensitive parameters at the time.

Step 2: Adjust the exposure mode through the relationship between the photosensitive parameter at the time and the first setting value.

Step 3: In fixed exposure mode, adjust the state of the counter based on the comparison value of the photosensitive parameter at the time and the second setting value.

Step 4: Adjust the counter status and exposure mode based on the counting value and the third setting value.

Specifically, first, the range in which the photosensitive parameters at the time are mapped is determined, the data collected by the photosensitive sensor is calculated by the control unit to obtain the average value of the current photosensitive values, and the ADC value mapped at the time is converted through analog-to-digital conversion/digital-to-analog conversion. confirm.

The first setting value is used to determine whether image collection should be performed using the automatic exposure mode or fixed exposure mode along with the ADC value mapped at the time. The exposure mode uses automatic exposure mode in the default state, and if the ratio of the ADC value mapped at the time to the mapping range is smaller than the first setting value, the fixed exposure mode is turned on, and the ratio of the ADC value mapped at the time to the mapping range is set. If it is greater than or equal to this first set value, the automatic exposure mode is maintained. In general, when the first setting value is 28% to 33%, automatic exposure adjusts brightness based on mid-gradation, so if the proportion of the mapped ADC value in the mapping range is less than 33%, the automatic exposure state The screen display is relatively dark. In fixed exposure mode, an exposure time of 9ms to 20ms is generally used.

When the fixed exposure mode is already turned on, the relationship between the ADC value and the second set value is continuously monitored, and if the proportion of the ADC value in the mapping range is greater than or equal to the second set value, 1 is added to the counter value, and the ADC value is If the proportion of the mapping range is smaller than the second set value, the counter value is reset.

The coefficient value is compared in size with the third set value, and when the count value reaches the third set value, the counter is reset, the fixed exposure mode is turned off, the automatic exposure mode is used, and if the count value falls below the third set value, the counter is reset. , duplicate the above operation without performing any operation.

The vehicle is outdoors, and automatically turns on the automatic exposure mode when starting. When the vehicle enters a dark environment from a bright ground, the brightness of the light collected by the photosensitive sensor goes through analog-to-digital and digital-to-analog conversion, and the ADC value at the time is When 10% of the total mapping range is lower than the first set value, the range of the first set value is 28%-33%, can take 28%, 30% or 33%, the first set value is the ADC at the time It is the ratio of the value and the mapping range, the control unit sends a command to the image acquisition device to turn on the fixed exposure mode and turn off the automatic exposure mode, the exposure time is 11ms, the exposure time can take 9ms~20ms.

When a vehicle passes through a series of tunnels from a bright area, the interval between tunnels is relatively short, and the inside of the tunnel is relatively dark, when the vehicle enters the dark tunnel, the ADC value collected by the photosensitive sensor is 9% of the mapping range. is smaller than the first set value, the control unit sends a command to the image collection device to turn on the fixed exposure mode and turn off the automatic exposure mode, the exposure time is 11 ms, and when the vehicle leaves the tunnel and enters another tunnel, At that time, the fixed exposure mode was still turned on, and the three consecutive ADC values collected by the photosensitive sensor accounted for 61%, 67%, and 33% of the mapping range, respectively, where the occupied percentage value of the previous two times was set as the second setting. is greater than 60% of the value, the counter continues to count, but the third set value still does not reach the specified three times, and the third time is less than 60% of the second set value, the counter is reset, and the counter still counts. This is a fixed exposure mode.

When the vehicle returns from a dark environment to a bright ground, the fixed exposure mode is turned on, and the ADC value collected by the photosensitive sensor is 61% of the mapping range, which is higher than the second set value of 60%, and the second set value is 60%. The range is 54% to 60%, 55%, 58%, 60%, the counter counts, and when the vehicle returns to the bright ground and a certain period of time has passed. That is, if the counting value of the counter is 3 of the third setting value, the third setting value can take 2, 3, or 4, the counter is reset, and the control unit sends a command to the image collection device to turn off the fixed exposure mode. , you can turn on the automatic exposure mode.

The path between two similar tunnels has a relatively high brightness, and due to the relationship of the tunnel, the brightness when entering and leaving the tunnel is one transitional brightness, which is lower than the brightness outside and higher than the brightness inside the tunnel. , generally, the proportion of the mapping range of the ADC value mapped to the collected brightness is 38% to 50%, the second setting value is set to 38%, and the mapped ADC value corresponding to the continuously collected brightness is mapped. When the percentage of the range is 28%, 30%, 40%, 61%, 67%, 44%, 30%, and 27%, respectively, the camera comes out of the tunnel, turns on automatic exposure mode for a moment, and then returns to fixed exposure mode. converted.

When a vehicle that has recovered the automatic exposure mode passes through a relatively dark tunnel, the ADC value at the time is 33% of the mapping range, which is higher than 30%, and the control unit sends a command to the image collection device to use the automatic exposure mode.

The present invention is not limited to the specific embodiments described above, and those skilled in the art can practice the present invention using various other specific embodiments based on the disclosure of the examples and drawings, and accordingly, Any technical proposal that uses the design structure and idea of the present invention and is simply converted or modified falls within the scope of protection of the present invention.

100: imaging lens 200: Collection unit

Claims (11)

A display system for compensating for blind spots in the tractor-trailer rearview mirror,
The display system includes a collection device, a detection device, a display control device, and a display device,
The collection device includes a left front camera, a right front camera, a left rear camera, and a right rear camera. The left front camera and the right front camera are respectively disposed on both left and right sides outside the cab, and the left rear camera and the right rear camera are Located on both left and right sides of the trailer,
The detection device is an angle detection sensor disposed on a tractor-trailer, and the angle detection sensor is used to obtain the angle at which the axis of the trailer deviates from the axis of the head of the car,
The display control device is connected to the collection device and the detection device and includes a processor, wherein the processor obtains an angle at which the axis of the trailer is deviated from the axis of the head of the vehicle through the detection device, and the It is used to transmit the screen directly to the display device and/or cut and composite it and then transmit it to the display device,
The display device is disposed on a display in the cab, and the display device is connected to the display control device, and is used to display an image output by the display control device to the driver to compensate for blind spots in the field of view of the tractor-trailer rearview mirror. A display system for.
According to claim 1,
The field of view of the screen collected by the left front camera and the right front camera must conform to the class II field of view and/or class IV field of view specified in the standard, and the camera is located within the screen collected by the left rear camera and the right rear camera. A display system for compensating for blind spots in the view of the tractor-trailer rearview mirror, which must include the trailer body on the side.
According to claim 1,
The display device includes a left display unit and a right display unit, wherein the left display unit is for displaying a screen collected by the left front camera or a composite screen between the left front camera and the left rear camera, and the right display unit. A display system for compensating blind spots in the view of a tractor-trailer rearview mirror, wherein the unit displays a screen collected by the right front camera or a composite screen of the right front camera and the right rear camera.
According to claim 1,
In the process of the processor cutting and compositing the screen, the screen of a preset width or fixed width on the side obscured by the trailer of the front camera in the opposite direction and/or direction of rotation is removed, A display system for compensating for blind spots in the view of a tractor-trailer rearview mirror that holds back a screen of a preset width or fixed width on the trailer side of the rear camera and places the held back screen in the area from which the screen was removed to create a new screen.
According to claim 4,
When the angle detection sensor detects that the angle between the head of the vehicle and the central axis of the trailer exceeds the irradiation angle of the front camera on the rotation side, the processor uses the fixed-width screen for cropping and compositing,
The fixed width is the width occupied by the trailer on the screen of the front camera on the rotation side. A display system for compensating for blind spots in the field of view of a tractor-trailer rearview mirror.
According to claim 1,
The display control device includes a storage unit, and the storage unit stores display blocks The processor performs calculations and judgments according to the detection of the sensing device, and the pre-stored compensation blocks match the preset width in the Y set, and the Ya and compensation blocks also have a fixed width or preset width in the Z set. The reward block also finds Za,
The processor cuts a screen of a preset width on the trailer side of the rear camera in the opposite direction of rotation, and also projects the compensation block found on the processed screen onto Ya,
The processor cuts a fixed-width screen on the trailer side of the rear camera in the direction of rotation, and also projects a compensation block that finds the processed screen onto Za. A display system for compensating blind spots in the field of view of a tractor-trailer rearview mirror.
According to claim 4 or 6,
The preset width is set according to the angle between the head of the vehicle and the central axis of the trailer,
When the angle between the head of the car and the central axis of the trailer is 0 degrees to 15 degrees, the preset width is 0,
When the angle between the head of the vehicle and the central axis of the trailer is 15 to 30 degrees, the preset width is 1/3 of the screen width,
When the angle between the head of the car and the central axis of the trailer is 30 to 45 degrees, the preset width is 2/3 of the screen width,
When the angle between the head of the vehicle and the central axis of the trailer is greater than 45 degrees, the preset width is the width of the entire screen. A display system for compensating for blind spots in the field of view of a tractor-trailer rearview mirror.
According to claim 1,
One or more cameras among the left front camera, right front camera, left rear camera, and right rear camera use a tilt-shift camera, and the tilt-shift camera includes a camera body, an image collection unit, and an imaging lens, and the image capture A lens and the image collection unit are respectively mounted on a front end and a rear end of the camera body, the camera body has a first angle θ inclined in a first direction relative to a reference direction, and the image collection unit and the image capture unit The lens has a second angle α and a third angle β that are inclined in directions opposite to the inclination direction of the camera body, respectively, relative to the main axis of the camera body, and the second angle α and the third angle β are not equal to each other. , Preferably, the second angle α is greater than the third angle β, so that the plane where the image collection unit and the imaging lens are located intersect each other to form an acute angle. A marking system for compensation of rent.
A display method for compensating for blind spots in the field of view of a tractor-trailer rearview mirror of the display system of any one of claims 1 to 8, comprising:
Step 1, where the processor acquires video images collected by a plurality of cameras in real time;
Step 2, where the processor acquires the operating state of the tractor-trailer; and
Step 3, wherein the processor generates a screen of an optimally unobscured, minimal blind spot solution based on the operating state of the tractor-trailer,
The operating state includes a trailer state and a driving state, the trailer state includes a trailer mode and a non-trailer mode, and the driving state includes straight forward, turning, and reverse,
The plurality of cameras include a left front camera, a right front camera, a left rear camera, and a right rear camera, the left front camera and the right front camera are respectively disposed on both left and right sides outside the cab, and the left rear camera and the right rear camera is a display method for compensating for the blind spot of the tractor-trailer rearview mirror placed on both left and right sides of the trailer.
According to clause 9,
In step 3, under the trailerless mode, the processor provides a screen to the display device,
When the driving state is straight under the trailer mode, the processor provides screens of the left front camera and the right front camera on the display device,
When the driving state in the trailer mode is turning or reversing, the processor provides a front camera screen on the turning side and a cut and composite screen on the opposite side to the display device based on the angle between the head of the car and the central axis of the trailer, ,
In the cropping and compositing of the screen in the opposite direction of rotation, the screen of the preset width on the side obscured by the trailer of the front camera in the opposite direction of rotation is removed, and the screen of the preset width on the side of the trailer of the rear camera in the opposite direction of rotation is removed. Putting the screen on hold and placing the put on hold screen in the area where the screen was removed to create a new screen,
In cropping and compositing of the screen in the rotation direction, when the angle between the head of the car and the central axis of the trailer exceeds the irradiation angle of the front camera on the rotation side, a fixed-width screen on the rotation side of the front camera in the rotation direction is selected. remove it, reserve the fixed-width screen on the trailer side of the rear camera in the direction of rotation, place the reserved screen in the area where the screen was removed, and composite a new screen;
The fixed width is the width occupied by the trailer on the screen of the front camera on the rotation side. A display method for compensating for blind spots in the field of view of a tractor-trailer rearview mirror.
According to claim 10,
The preset width is set according to the angle between the head of the vehicle and the central axis of the trailer,
When the angle between the head of the car and the mid-axis line of the trailer is 0 degrees to 15 degrees, the preset width is 0, and when the angle between the head of the car and the mid-axis line of the trailer is 15 degrees to 30 degrees, the preset width is is a width of 1/3 of the screen width, and when the angle between the head of the car and the mid-axis of the trailer is 30 degrees to 45 degrees, the preset width is a width of 2/3 of the width of the screen, and the mid-axis of the car head and the trailer When the angle between the lines is more than 45 degrees, the preset width is the width of the entire screen. A display method for compensating for blind spots in the field of view of a tractor-trailer rearview mirror.