KR20240054062A - Image correction system and method based on vehicle inclination information - Google Patents

Abstract

A vehicle image correction system and method are disclosed. The image correction method of a vehicle according to an embodiment of the present invention includes the steps of an image processing unit acquiring surrounding images and tilt information of the vehicle, and determining, by the image processing unit, whether the tilt of the vehicle exceeds a threshold angle. and correcting, by the image processing unit, the surrounding image or visual information overlaid on the surrounding image when the inclination of the vehicle exceeds a critical angle.

Description

Vehicle image correction system and method {IMAGE CORRECTION SYSTEM AND METHOD BASED ON VEHICLE INCLINATION INFORMATION}

The present invention relates to an image correction system and method, and more specifically, to a technology for correcting images captured from a camera based on tilt information of a vehicle.

Recently, in the automotive industry, technology development related to intelligent driver assistance systems (ADAS) to supplement drivers' driving abilities has been active.

Representative examples include Surround View Monitoring System (SVM), Audio Video Navigation (AVN), Forward Collision Warning (FCW), and Lane Departure Warning System (LDWS). etc.

In particular, technologies for capturing images using cameras have been developed to ensure a wide field of view when parking a car.

In addition, in the case of a vehicle's headlamp, when loading cargo in the vehicle's trunk, the headlamp irradiation angle increases as the vehicle tilts back and forth, preventing discomfort in the view of passers-by on the road or drivers of other vehicles. To achieve this, the irradiation angle of the headlamp light source was rotated according to the degree to which the vehicle was tilted.

Meanwhile, in the case of a rear camera or SVM camera, even when a load is loaded into the trunk of a vehicle, the camera's angle of view rotates as the vehicle tilts back and forth, and a different image from before the load is captured is captured and displayed on the screen.

There is an Auto Offset Control (AOC) function in conventional vehicle SVMs, etc., but this only corrects errors in the empty vehicle state and displays images in real time according to the inclination of the vehicle, which can change at any time due to the weight of the trunk, etc. It is inconvenient to correct.

Therefore, in this technical field, there is a demand for technology to correct images in real time so that images close to those in an empty state can be output even if the weight of the vehicle load changes.

Korean Patent No. 10-2371618, registered on March 2, 2022 (Name: Camera system and image correction method of camera system) Korean Patent Publication No. 10-2016-0092405, published on August 4, 2016 (Name: System and method for image correction of vehicle camera using portable terminal)

The technical task of the present invention is to provide a technology for correcting images in real time so that an image close to the image in a horizontal state can be output even if the vehicle is tilted back and forth due to loading in the trunk of the vehicle.

An image correction method according to an embodiment of the present invention for realizing the above-described problem includes the steps of an image processing unit acquiring surrounding images and tilt information of the vehicle, wherein the image processing unit determines that the tilt of the vehicle exceeds a critical angle. A step of determining whether the vehicle is tilted or not, and a step of correcting the surrounding image or visual information overlaid on the surrounding image by the image processing unit when the inclination of the vehicle exceeds a critical angle.

At this time, correcting the surrounding image or visual information overlaid on the surrounding image may include cropping the image based on the tilt information.

At this time, the step of correcting the surrounding image or visual information overlaid on the surrounding image may further include rotating and resizing the cropped image to fit the display screen.

At this time, the overlaid visual information may include guidelines.

In this case, correcting the surrounding image or visual information overlaid on the surrounding image may include resetting the interval of the guideline based on the tilt information.

At this time, the surrounding image may be one of a front image, a rear image, or a top view image of the vehicle.

At this time, the image correction method may further include displaying the corrected image on a display screen in the vehicle.

At this time, the tilt information of the vehicle may include at least one of a height sensor, a weight sensor, a gyro sensor, or a combination thereof.

At this time, the tilt information may be transmitted from the sensor unit to the image processing unit via the lamp control unit.

At this time, the tilt information may be transmitted from the sensor unit to the lamp control unit in the form of a PWM signal, converted into tilt information by the lamp control unit, and transmitted to the image processing unit.

Meanwhile, an image correction system according to an embodiment of the present invention includes a camera that captures images around a vehicle, a sensor unit that measures tilt information of the vehicle, and an image captured by the camera and the tilt information, and an image processing unit that determines whether the inclination of the vehicle exceeds a critical angle and, when the inclination of the vehicle exceeds the critical angle, corrects the surrounding image or visual information overlaid on the surrounding image.

At this time, the image processor may crop the image based on the tilt information.

At this time, the image processing unit may rotate and resize the cropped image to fit the display screen.

At this time, the overlaid visual information may include guidelines.

At this time, the image processor may reset the interval of the guideline based on the tilt information.

At this time, the surrounding image may be one of a front image, a rear image, or a top view image of the vehicle.

At this time, the image correction system may further include a display unit that displays the corrected image on a display screen in the vehicle.

At this time, the tilt information of the vehicle may include at least one of a height sensor, a weight sensor, a gyro sensor, or a combination thereof.

At this time, the tilt information may be transmitted from the sensor unit to the image processing unit via the lamp control unit.

At this time, the tilt information may be received from the sensor unit to the lamp control unit in the form of a PWM signal, converted into tilt information by the lamp control unit, and transmitted to the image processing unit.

According to various embodiments of the present invention as described above, the image is corrected in real time so that an image close to the image in the empty state can be output even if the weight of the load in the vehicle trunk changes.

The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below. will be.

Figure 1 shows the structure of an image correction system according to an embodiment of the present invention.
Figure 2 exemplarily shows the principle of correcting the headlamp angle of a vehicle according to an embodiment of the present invention.
3(a) to 3(c) exemplarily show that the image capture range changes as the angle of the vehicle is tilted according to an embodiment of the present invention.
Figure 4 exemplarily shows that the rear image is displayed distorted as the angle of the vehicle is tilted according to an embodiment of the present invention.
Figure 5 shows a method for correcting an image of a vehicle according to an embodiment of the present invention.
Figure 6 shows an example of the range in which image cropping is performed according to the image correction method of Figure 5.
Figure 7 shows a method for correcting an image of a vehicle according to another embodiment of the present invention.
FIGS. 8(a) and 8(b) show an example in which the spacing of guidelines within an image is reset according to the image correction method of FIG. 7.

Hereinafter, embodiments disclosed in the present specification will be described in detail with reference to the attached drawings. However, identical or similar components will be assigned the same reference numbers regardless of reference numerals, and duplicate descriptions thereof will be omitted. The suffixes “module” and “part” for components used in the following description are given or used interchangeably only for the ease of preparing the specification, and do not have distinct meanings or roles in themselves. Additionally, in describing the embodiments disclosed in this specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed descriptions will be omitted. In addition, the attached drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical idea disclosed in this specification is not limited by the attached drawings, and all changes included in the spirit and technical scope of the present invention are not limited. , should be understood to include equivalents or substitutes.

Terms containing ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this specification, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that it does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Figure 1 shows the structure of an image correction system 100 according to an embodiment of the present invention.

Referring to FIG. 1, the image correction system 100 according to this embodiment includes a sensor unit 110, a lamp control unit 120, an image capture unit 130, an image processor 140, and a display unit 150. do.

The sensor unit 110 includes a height sensor, a weight sensor, a gyro sensor, etc., and this is only an example to help understand the present invention, and is not limited thereto.

At this time, the height sensor is connected to the lower control arm and the sensor body by a lever and a rod in the electronically controlled suspension system, and is installed one each at the front and rear of the vehicle. The rotation amount of the lever is transmitted to the sensor to change the height of the vehicle. Detects the position of the axle and body according to the

At this time, the weight sensor is installed on the bottom of the trunk of the vehicle and measures the weight of the load mounted in the trunk.

At this time, the gyro sensor measures the vehicle's tilt angle using the dynamic motion of a rotating object.

Additionally, information measured from the height sensor, weight sensor, or gyro sensor may be used when setting the headlamp irradiation direction of the lamp control unit 120 and correcting the image of the image processing unit 140. For example, when a load is placed in the trunk of a vehicle in the state shown in Figure 2(a), the center of gravity of the vehicle moves, causing the vehicle to tilt backwards, and the illumination angle of the vehicle's headlamp increases as shown in Figure 2(b). It may cause glare and discomfort to other drivers or passers-by. At this time, when the inclination of the vehicle or the weight of the trunk load is measured by the sensor unit 110, the lamp control unit 120 can correct the irradiation angle of the headlamp as shown in FIG. 2(c) based on the measured information. there is.

The lamp control unit 120 controls the irradiation brightness and irradiation angle of the headlamp.

At this time, the lamp control unit 120 may control the irradiation angle using information measured from the sensor unit 110. For example, the lamp control unit 120 may receive information about the vehicle inclination from the sensor unit 110.

At this time, the lamp control unit 120 may be either an Integrated Lighting Control Unit (ILCU) or an Adaptive Front Lighting System (AFLS) Electronic Control Unit (ECU).

At this time, information about the vehicle tilt received from the sensor unit 110 by the lamp control unit 120 may be a PWM (Pulse Width Modulation) signal.

At this time, the lamp control unit 120 converts the received PWM signal into vehicle tilt information, determines the required correction angle of the headlamp based on this, and adjusts the angle of the headlamp.

At this time, the angle adjustment of the headlamp can be performed by controlling the HLLD (Head Lamp Leveling Device) motor.

At this time, the vehicle tilt information converted by the lamp control unit 120 may be transmitted to the image processing unit 140.

The image capturing unit 130 captures images of the exterior surroundings of the vehicle and transmits them to the image processing unit 140.

At this time, the image capture unit 130 may be an AVM (Around View Monitor) camera. For example, the image capture unit 130 may be installed on the front, rear, and both side mirrors to capture images of the surroundings of the front, left, right, and rear of the vehicle, and may capture the space around the vehicle. The cameras constituting the image capture unit 130 can be defined as AVM (Around View Monitor) cameras, and the AVM cameras transmit captured images to the image processing unit 140 using CAN (Controller Area Network) communication. You can.

The image processing unit 140 receives vehicle tilt information from the lamp control unit 120, receives the captured image from the image capture unit 130, and images the vehicle based on the vehicle tilt information received from the lamp control unit 120. The images received from the photographing unit 130 are combined, corrected, and displayed on the display unit 150.

For example, when a load is placed in the trunk of a vehicle in the state shown in FIG. 3(a), the center of gravity of the vehicle moves and the vehicle tilts backwards, and the shooting range of the camera is raised as shown in FIG. 3(b), so that the display screen The position of the object in the image is displayed distorted.

For example, referring to FIG. 4, in an image captured by a rear camera, an object that should be displayed at the first point 410 may be displayed at the second point 430, which allows the driver to stay close to the vehicle. This can cause a contact accident by causing the illusion that an object is far away.

Therefore, in order to prevent such distortion, it is necessary to correct the shooting range as shown in FIG. 3(c).

At this time, the image processing unit 140 corrects the image based on the tilt information of the vehicle received from the lamp control unit 120 to change the position of the object displayed in the image of the tilted vehicle to the actual position, that is, that of the vehicle being horizontal. It can be corrected to be close to the actual location of the object displayed in the video.

At this time, the image processing unit 140 may be one of the Advanced Driver Assistance System (ADAS) controllers, for example, an ADAS_PRK controller.

The display unit 150 receives the corrected image from the image processing unit 140 and displays it on the display screen.

At this time, the display unit 150 may be a head unit installed in the vehicle or a display screen of the vehicle's AVN (Audio Video Navigation).

Figure 5 shows an image correction method of an image correction system according to an embodiment of the present invention.

Referring to FIG. 5, the image correction method according to this embodiment receives the image captured from the image capture unit 130 by the image processing unit 140 (S510), and calculates the tilt of the vehicle measured from the sensor unit 110. Receive information (S520).

At this time, the tilt information of the vehicle may be transmitted from the sensor unit 110 to the image processing unit 140 via the lamp control unit 120.

At this time, information about the vehicle inclination received from the sensor unit 110 by the lamp control unit 120 may be in the form of a PWM (Pulse Width Modulation) signal.

At this time, the lamp control unit 120 may convert the received PWM signal into vehicle tilt information and transmit it to the image processing unit 140.

Meanwhile, the image processing unit 140 determines whether the inclination of the vehicle exceeds the threshold angle based on the inclination information of the vehicle (S530), and if the inclination of the vehicle exceeds the threshold angle, the image processing unit 140 receives the information based on the inclination information of the vehicle. Correct the image (S540).

At this time, correction of the image may include the process of cropping the image and rotating and resizing it to fit the display screen.

At this time, the correction of the image can be applied not only to the image of the rear of the vehicle, but also to the top view image created by combining images from multiple cameras.

Meanwhile, the display unit 150 may receive the corrected image from the image processing unit 140 and display the received image on the display screen (S550).

Figure 6 shows an example of the range in which image cropping is performed according to the image correction method of Figure 5. Referring to Figure 6, the shooting range 610 of the tilted camera faces downward compared to the shooting range 620 of the normal camera. To correct this, the lower part of the image is removed and cropped to the output range 630. Only the image 640 can be left. The cropped image 640 in this way can be rotated and resized to be displayed on the screen similar to the normal image.

Figure 7 shows a method for correcting an image of a vehicle according to another embodiment of the present invention.

Referring to FIG. 7, the image correction method according to this embodiment receives the image captured from the image capture unit 130 by the image processing unit 140 (S710), and calculates the tilt of the vehicle measured by the sensor unit 110. Receive information (S720).

At this time, the tilt information of the vehicle may be transmitted from the sensor unit 110 to the image processing unit 140 via the lamp control unit 120.

At this time, information about the vehicle inclination received from the sensor unit 110 by the lamp control unit 120 may be in the form of a PWM (Pulse Width Modulation) signal.

At this time, the lamp control unit 120 may convert the received PWM signal into vehicle tilt information and transmit it to the image processing unit 140.

Meanwhile, the image processing unit 140 determines whether the inclination of the vehicle exceeds the critical angle based on the inclination information of the vehicle (S730), and if the inclination of the vehicle exceeds the critical angle, the image processing unit 140 determines whether the inclination of the vehicle exceeds the critical angle. The visual information overlaid is corrected (S740).

At this time, the visual information overlaid on the image may be a guideline indicating the distance from the vehicle.

At this time, the image processor 140 may reset the spacing of the guidelines in the image based on the vehicle's tilt information.

For example, three guidelines may be displayed on the front or rear image of a vehicle, and in the case of the rear image, the first line is 0.5M away from the vehicle, the second line is 1M away, and the second line is 3M away. A third line may be displayed. Additionally, in the case of a front image of a vehicle, a first line may be displayed at a location 0.5M away from the vehicle, a second line may be displayed at a location 3M away, and a third line may be displayed at a location 6M away from the vehicle.

Figure 8 shows an example in which the spacing of guidelines within an image is reset according to the image correction method of Figure 7. Figure 8(a) shows the guideline before the interval is reset, and Figure 8(b) shows the guideline after the interval is reset.

Referring to FIGS. 8(a) and 8(b), when a load is placed in the trunk of a vehicle and the vehicle tilts, nearby objects are distorted to appear far away, making the guideline interval shorter than it actually is. Accordingly, the first to third guidelines 810, 820, and 830 shown in 8(a) indicate planes or objects that are closer than the originally intended distances of 0.5M, 1M, and 3M. Accordingly, by correcting this, the spacing of the guidelines can be increased, such as the first to third guidelines 860, 870, and 880 in 8(b).

Meanwhile, the display unit 150 may receive an image in which the overlaid visual information has been corrected from the image processing unit 140 and display the received image on the display screen (S750).

According to the embodiments of the present invention described so far, the image is corrected in real time so that an image close to the image in an empty state can be output even if the weight of the load in the vehicle trunk changes.

Claims (20)

An image processing unit acquiring surrounding images and tilt information of the vehicle;
determining, by the image processing unit, whether the inclination of the vehicle exceeds a critical angle; and
Correcting, by the image processing unit, the surrounding image or visual information overlaid on the surrounding image when the inclination of the vehicle exceeds a threshold angle.
An image correction method comprising: The method of claim 1, wherein correcting the surrounding image or visual information overlaid on the surrounding image comprises:
An image correction method comprising cropping the surrounding image based on the tilt information. The method of claim 1, wherein correcting the surrounding image or visual information overlaid on the surrounding image comprises:
An image correction method further comprising rotating and resizing the cropped image to fit the display screen. The image correction method of claim 1, wherein the overlaid visual information includes guidelines. The method of claim 4, wherein correcting the surrounding image or visual information overlaid on the surrounding image comprises:
An image correction method comprising resetting the interval of the guideline based on the tilt information. According to paragraph 1,
An image correction method, wherein the surrounding image is one of a front image, a rear image, or a top view image of the vehicle. The method of claim 1, wherein the image correction method:
Displaying the corrected image on a display screen in the vehicle
An image correction method further comprising: The image correction method of claim 1, wherein the tilt information of the vehicle includes at least one of a height sensor, a weight sensor, a gyro sensor, or a combination thereof. The image correction method of claim 1, wherein the tilt information is transmitted from the sensor unit to the image processing unit via the lamp control unit. The image correction method of claim 9, wherein the tilt information is transmitted from the sensor unit to the lamp control unit in the form of a PWM signal, converted into tilt information by the lamp control unit, and transmitted to the image processing unit. A camera that captures images of the vehicle's surroundings;
A sensor unit that measures tilt information of the vehicle; and
Receive the image captured by the camera and the tilt information, determine whether the tilt of the vehicle exceeds a threshold angle, and overlay the surrounding image or the surrounding image if the tilt of the vehicle exceeds the threshold angle. Image processing unit that corrects visual information
An image correction system comprising: The method of claim 11, wherein the image processing unit,
An image correction system characterized in that the image is cropped based on the tilt information. The method of claim 11, wherein the image processing unit,
An image correction system characterized by rotating and resizing the cropped image to fit the display screen. The image correction system of claim 11, wherein the overlaid visual information includes guidelines. The method of claim 14, wherein the image processing unit,
An image correction system characterized in that the interval of the guideline is reset based on the tilt information. According to clause 11,
An image correction system, wherein the surrounding image is one of a front image, a rear image, or a top view image of the vehicle. The method of claim 11, wherein the image correction system:
A display unit that displays the corrected image on a display screen in the vehicle
An image correction system further comprising: The image correction system of claim 11, wherein the vehicle tilt information includes at least one of a height sensor, a weight sensor, a gyro sensor, or a combination thereof. The image correction system of claim 11, wherein the tilt information is transmitted from the sensor unit to the image processing unit via the lamp control unit. The image correction system of claim 19, wherein the tilt information is received from the sensor unit to the lamp control unit in the form of a PWM signal, converted into tilt information by the lamp control unit, and transmitted to the image processing unit.

Images