KR20200002514A - Method and apparatus for correcting a position of ADAS camera during driving - Google Patents

Abstract

An embodiment of the present invention relates to a method for correcting a position of an ADAS camera during driving which photographs a road image using the ADAS camera during driving of a vehicle, acquires the horizontal data of the road from the photographed road image, acquires the inclination data by measuring an inclination of the vehicle body using a sensor of the vehicle during driving of the vehicle and corrects a position of the ADAS camera based on the acquired horizontal data and inclination data, thereby a lowering problem of detection performance caused by a mismatch of a basic setting value of an ADAS during driving of the vehicle.

Description

Method and apparatus for correcting a position of ADAS camera during driving}

Embodiments relate to a method and an apparatus for correcting a position of an ADAS camera while a vehicle is driving.

ADAS (Advanced Driver Assistance Systems) is also called Advanced Driver Assistance System, which allows the vehicle to respond to situations with faster response than the driver, providing convenience and safety to the driver. The goal is to reduce accidents on the road, and most importantly, most of the ADAS technology is done through sensors, and sensors are very important, and each of ADAS technologies will be applied to autonomous vehicles in the future. Also called.

ADAS sensor can be divided into camera, radar and lidar.

Radar (Radio Detection And Ranging, Radar) is a system that detects the direction, distance and speed of a detected object by shooting electromagnetic waves in the air and then measuring the reflected wave that hits an object. Using radio waves, distance can be measured stably without distinguishing between weather and day and night, which complements the camera. Radar is difficult to increase the measurement distance and measurement angle at the same time, so it is divided into long range radar and medium and short range radar according to ADAS function. In terms of technology, automotive radar is evolving with the aim of increasing measurement distance, angle of measurement, and frequency bandwidth of radio waves in order to improve data accuracy, and efforts for light weight, miniaturization, and low cost are continuing.

LiDAR (Light Detection And Ranging, LiDAR) has the same basic principle as radar, but differs in that it uses a high power pulsed laser to obtain distance information. This is a method of measuring the speed at which lidar sensors constantly shoot rays back. Since it shoots millions of rays per second to calculate the actual distance, it is possible to add them all together to reconstruct the visual information in 3D. It is necessary to prevent collision accidents that may occur due to intentional or negligence of others by judging not only the forward situation of the vehicle driving direction but also the rear situation.

The camera is the most basic in ADAS system because it can grasp precise shape information that cannot be understood through radar and lidar. Accurate data analysis by camera is essential for traffic sign recognition, blind spot detection and lane departure. However, the road on which the vehicle travels is not a straight horizontal line, and thus the basic setting value of the camera of the ADAS is changed according to the situation of the road.

[Prior art document number]

Prior Art 1: Korean Patent Publication No. 2018-0052415

Prior Art 2: Korean Patent Registration No. 10-0162342

Prior Art 3: Korean Registered Patent No. 10-0820459

Embodiments provide a method and apparatus for correcting the position of an ADAS camera while driving a vehicle. In addition, the present invention provides a computer-readable recording medium having recorded thereon a program for executing the method on a computer. The technical problem to be solved is not limited to the above technical problems, and other technical problems may exist.

As a technical means for achieving the above-described technical problem, in the method for correcting the position of the ADAS camera while driving in accordance with an embodiment, the road image is taken by using the ADAS camera while driving the vehicle, and from the captured road image Obtaining horizontal data of the roadway; Acquiring inclination data obtained by measuring a vehicle body inclination using a sensor of the vehicle while the vehicle is driving; And correcting the position of the ADAS camera based on the obtained horizontal data and the inclination data.

In the correcting step, the setting value of the ADAS camera may be corrected to correspond to a previous setting value.

The sensor is characterized in that it comprises a tilt sensor or a gyro sensor.

The position correction method may be configured to obtain up, down, left, and right tilt data of the vehicle being driven by using the tilt sensor or the gyro sensor.

According to another embodiment, a method of correcting a position of a camera mounted on a driving vehicle, the method comprising: obtaining horizontal data from a road image photographed while driving of a vehicle; Measuring an inclination of the inclined vehicle according to a state of a road while driving of the vehicle; And correcting the current position of the camera based on the obtained horizontal data and the measured tilt.

An apparatus for calibrating the position of an ADAS camera during driving, the apparatus comprising: a memory in which at least one program is stored; And a processor for driving the position correction device by executing the at least one program.

The processor is configured to capture a road image using an ADAS camera while driving the vehicle, obtain horizontal data of the road from the captured road image, and measure tilt data of the vehicle body using a sensor of the vehicle while driving the vehicle. And correct the position of the ADAS camera based on the obtained horizontal data and the tilt data.

An apparatus for correcting a position of a camera while driving according to another embodiment, the apparatus comprising: a camera photographing a road image while a vehicle is traveling; A tilt sensor for measuring a tilt of a vehicle inclined according to a state of a road while driving of the vehicle; And a controller for acquiring horizontal data from the photographed road image and correcting a current position of the camera based on the obtained horizontal data and the measured slope.

Another embodiment includes a recording medium on which a program for executing the position correction method on a computer is recorded.

1 is an internal block diagram of a vehicle according to an exemplary embodiment.
FIG. 2 is a block diagram of a vehicle driving assistance system (ADAS) shown in FIG. 1.
3 is a schematic diagram of a camera position correcting apparatus capable of correcting a position of a camera while driving a vehicle according to another exemplary embodiment.
4 and 5 are exemplary diagrams for explaining the inclination of the road on which the vehicle travels.
6 is an exemplary view of camera position correction according to another embodiment.
7 is a flowchart illustrating a camera position correction method according to another embodiment.

The phrases “in some embodiments” or “in one embodiment” appearing in various places in the specification are not necessarily all referring to the same embodiment.

Some embodiments of the present disclosure can be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented in various numbers of hardware and / or software configurations that perform particular functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors or by circuit configurations for a given function. In addition, for example, the functional blocks of the present disclosure may be implemented in various programming or scripting languages. The functional blocks may be implemented in algorithms running on one or more processors. In addition, the present disclosure may employ the prior art for electronic configuration, signal processing, and / or data processing. Terms such as "mechanism", "element", "means" and "configuration" may be widely used and are not limited to mechanical and physical configurations.

In addition, the connecting lines or connecting members between the components shown in the drawings are merely illustrative of functional connections and / or physical or circuit connections. In an actual device, the connections between components may be represented by various functional connections, physical connections, or circuit connections that are replaceable or added.

Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating a vehicle according to an exemplary embodiment.

Referring to FIG. 1, the vehicle 100 includes a communication unit 110, an input unit 120, a sensing unit 125, a memory 130, an output unit 140, a vehicle driving unit 150, and a vehicle driving assistance system (ADAS). , 160, a controller 170, an interface 180, and a power supply 190.

The communication unit 110 may include a short range communication module, a location information module, an optical communication module, and a V2X communication module.

The input unit 120 may include a driving manipulation unit, a camera, a microphone, and a user input unit.

The driving manipulation means receives a user input for driving the vehicle 100. The driving manipulation means may include a steering input means, a shift input means, an acceleration input means, a brake input means.

The acceleration input means receives an input for accelerating the vehicle 100 from the user. The brake input means receives an input for deceleration of the vehicle 100 from a user.

The camera may include an image sensor and an image processing module. The camera may process still images or moving images obtained by an image sensor (for example, CMOS or CCD). The image processing module may process the still image or the video obtained through the image sensor, extract necessary information, and transfer the extracted information to the controller 170.

Meanwhile, the vehicle 100 may include a front camera for photographing a vehicle front image, an around view camera for photographing a vehicle surrounding image, and a rear camera for photographing a rear image of the vehicle. Each camera may include a lens, an image sensor, and a processor. The processor may computer-process the captured image to generate data or information, and transmit the generated data or information to the controller 170. The processor included in the camera may be controlled by the controller 170.

The camera may comprise a stereo camera. In this case, the processor of the camera may detect the distance to the object, the relative speed to the object detected in the image, and the distance between the plurality of objects by using the disparity difference detected in the stereo image.

The camera may include a time of flight (TOF) camera. In this case, the camera may include a light source (for example, an infrared ray or a laser) and a receiver. In this case, the processor of the camera detects the distance to the object, the relative speed to the object, and the distance between the plurality of objects based on a time (TOF) until infrared rays or laser beams emitted from the light source are reflected by the object and received. can do.

Meanwhile, the rear camera may be disposed near the rear license plate or the trunk or the tail gate switch, but the position at which the rear camera is disposed is not limited thereto.

Each image photographed by the plurality of cameras is transferred to a processor of the camera, and the processor may synthesize the respective images to generate a vehicle surrounding image. In this case, the vehicle surrounding image may be displayed through the display unit as a top view image or a bird eye image.

The sensing unit 125 senses various situations of the vehicle 100. To this end, the sensing unit 125 may include a collision sensor, a wheel sensor, a speed sensor, an inclination sensor, a weight sensor, a heading sensor, a yaw sensor, a gyro sensor. Position module, vehicle forward / reverse sensor, battery sensor, fuel sensor, tire sensor, steering sensor by steering wheel rotation, interior temperature sensor, interior humidity sensor, ultrasonic sensor, illumination sensor, radar, rider, etc. It may include.

Accordingly, the sensing unit 125 includes vehicle collision information, vehicle direction information, vehicle position information (GPS information), vehicle angle information, vehicle speed information, vehicle acceleration information, vehicle tilt information, vehicle forward / reverse information, and battery information. The sensing signal may be acquired about fuel information, tire information, vehicle lamp information, vehicle internal temperature information, vehicle internal humidity information, steering wheel rotation angle, and vehicle external illumination.

The memory 130 is electrically connected to the controller 170. The memory 130 may store basic data for the unit, control data for controlling the operation of the unit, and input / output data. The memory 130 may be various storage devices such as a ROM, a RAM, an EPROM, a flash drive, a hard drive, and the like, in hardware. The memory 130 may store various data for overall operation of the vehicle 100, such as a program for processing or controlling the controller 170.

The output unit 140 outputs the information processed by the controller 170 and may include a display unit, a sound output unit, and a haptic output unit.

The display unit may display information processed by the controller 170. For example, the display unit may display vehicle related information. Here, the vehicle related information may include vehicle state information indicating a current state of a vehicle or vehicle driving information related to driving of the vehicle. For example, the display unit may display speed (or speed) of the current vehicle, speed (or speed) of the surrounding vehicle, and distance information between the current vehicle and the surrounding vehicle.

The display unit may include a cluster so that the driver can check the vehicle status information or the vehicle driving information while driving. The cluster can be located on the dashboard. In this case, the driver can check the information displayed on the cluster while keeping the gaze in front of the vehicle.

The sound output unit converts an electrical signal from the controller 170 into an audio signal and outputs the audio signal. To this end, the sound output unit may include a speaker.

The vehicle driver 150 may control operations of various vehicles. The vehicle driver 150 may include a power source driver, a steering driver, a brake driver, a lamp driver, an air conditioner driver, a window driver, an airbag driver, a sunroof driver and a suspension driver.

The power source driver may perform electronic control of the power source in the vehicle 100. For example, when a fossil fuel based engine (not shown) is a power source, the power source driver may perform electronic control of the engine. Thereby, the output torque of an engine, etc. can be controlled. When the power source driver is an engine, the speed of the vehicle may be limited by limiting the engine output torque under the control of the controller 170.

The brake driver may perform electronic control of a brake apparatus (not shown) in the vehicle 100. For example, the speed of the vehicle 100 may be reduced by controlling the operation of the brake disposed on the wheel. As another example, by varying the operation of the brakes disposed on the left wheels and the right wheels, the traveling direction of the vehicle 100 may be adjusted to the left or the right.

The lamp driver may control turn on / off of lamps disposed in and outside the vehicle. In addition, it is possible to control the intensity, direction, etc. of the light of the lamp. For example, control of a direction indicator lamp, a brake lamp, and the like can be performed.

The controller 170 may control the overall operation of each unit in the vehicle 100. The controller 170 may be referred to as an electronic control unit (ECU). The collision preventing device described above may be driven by the controller 170.

The controller 170, in hardware, may include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), and processors ( It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing other functions.

According to an embodiment, the controller 170 captures a road image using an ADAS camera while driving the vehicle, obtains horizontal data of the road from the captured road image, and tilts the vehicle body using a sensor of the vehicle while driving the vehicle. The measured tilt data is acquired, and the position of the ADAS camera is corrected based on the obtained horizontal data and the tilt data.

The interface unit 180 may serve as a path to various types of external devices connected to the vehicle 100. For example, the interface unit 180 may include a port that can be connected to a mobile terminal (not shown), and can be connected to the mobile terminal (not shown) through the port. In this case, the interface unit 180 may exchange data with the mobile terminal.

The power supply unit 190 may supply power required for the operation of each component under the control of the controller 170. In particular, the power supply unit 190 may receive power from a battery (not shown) in the vehicle.

FIG. 2 is a block diagram of a vehicle driving assistance system (ADAS) shown in FIG. 1.

The ADAS 200 is a vehicle driving assistance system that assists the driver for providing convenience and safety.

The ADAS 200 includes an automatic emergency braking module (hereinafter referred to as AEB: Autonomous Emergency Braking (210), a forward collision avoidance module (hereinafter referred to as FCW: Foward Collision Warning) 211, a lane departure warning module (hereinafter referred to as LDW: Lane). Departure Warning (212), Lane Keeping Assist (LKA) (213), Speed Assist System (hereinafter SAS) (214), Traffic Signal Detection Module (TSR) Recognition (215), adaptive high beam control module (hereinafter referred to as HBA: High Beam Assist) 216, blind spot monitoring module (hereinafter referred to as BSD: Blind Spot Detection) (217), automatic emergency steering module (hereinafter referred to as AES: Autonomous Emergency Steering (218), Curve Speed Warning System Module (hereinafter referred to as CSWS: Curve Speed Warning System) (219), Adaptive Forward Control Module (hereinafter referred to as ACC: Adaptive Cruise Control) 220, Smart Parking System Module (hereinafter referred to as , Smart Parking Assist System (SPAS) (221), Traffic Jam Assist (TSA) (222) and Around View Mode It may include a monitor module (hereinafter, AVM: Around View Monitor) 223.

Each of the ADAS modules 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, and 223 may include a processor for controlling a vehicle driving assistance function.

The processor included in each of the ADAS modules 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, and 223 may be controlled by the controller 270. .

Each ADAS module (210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223) processor, in hardware, ASICs (application specific integrated circuits), DSPs (digital signal processors), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, micro-controllers, microprocessors ( microprocessors) and electrical units for performing other functions.

The AEB 210 is a module that controls automatic braking to prevent collision with the detected object. The FCW 211 is a module that controls the warning to be output in order to prevent a collision with the vehicle front object. The LDW 212 is a module that controls a warning to be output to prevent lane departure during driving. The LKA 213 is a module that controls the vehicle to maintain a traveling lane while driving. The SAS 214 is a module which controls to travel below a set speed. The TSR 215 is a module that detects a traffic signal while driving and provides information based on the detected traffic signal. The HBA 216 is a module that controls the irradiation range or the irradiation amount of the upward lamp according to the driving situation. The BSD 217 is a module that detects an object outside the driver's field of view and provides detection information while driving. AES 218 is a module that automatically performs emergency steering. The CSWS 219 is a module that controls to output a route when driving at a speed higher than a predetermined speed during curve driving. The ACC 220 is a module that controls the vehicle to follow the preceding vehicle. SPAS 221 is a module which detects a parking space and controls it to park in a parking space. The TJA 222 is a module for controlling the vehicle to automatically operate when the traffic is congested. The AVM 223 is a module that provides a vehicle surrounding image and controls to monitor the vehicle surroundings.

The ADAS 200 provides a control signal for performing each vehicle driving assistance function to the output unit 250 or the vehicle driver 260 based on the data acquired by the input unit 230 or the sensing unit 235. can do. The ADAS 200 may directly output the control signal to the output unit 250 or the vehicle driver 260 through in-vehicle network communication (eg, CAN). Alternatively, the ADAS 200 may output the control signal to the output unit 250 or the vehicle driver 260 via the control unit 270.

3 is a schematic diagram of a camera position correcting apparatus capable of correcting a position of a camera while driving a vehicle according to another exemplary embodiment.

Referring to FIG. 3, the position correcting apparatus includes a controller 300, a camera 310, a vehicle tilt detection sensor 320, and a display unit 330. Here, although illustrated as including the display unit 330, the control unit 300 may be configured to exclude the case where the camera automatically corrects the set value to the previous set value.

The camera 310 may include an image sensor and an image processing module. The camera may process still images or moving images obtained by an image sensor (for example, CMOS or CCD). The image processing module may process the still image or the video obtained through the image sensor, extract necessary information, and transfer the extracted information to the controller 300. Meanwhile, the vehicle 100 may include a front camera for photographing a vehicle front image, an around view camera for photographing a vehicle surrounding image, and a rear camera for photographing a rear image of the vehicle. Each camera may include a lens, an image sensor, and a processor. The processor may computer-process the captured image to generate data or information, and transmit the generated data or information to the controller 300. The processor included in the camera may be controlled by the controller 300.

The camera may comprise a stereo camera. In this case, the processor of the camera may detect the distance to the object, the relative speed to the object detected in the image, and the distance between the plurality of objects by using the disparity difference detected in the stereo image. The camera may include a time of flight (TOF) camera. In this case, the camera may include a light source (for example, an infrared ray or a laser) and a receiver. In this case, the processor of the camera detects the distance to the object, the relative speed to the object, and the distance between the plurality of objects based on a time (TOF) until infrared rays or laser beams emitted from the light source are reflected by the object and received. can do.

Meanwhile, the rear camera may be disposed near the rear license plate or the trunk or the tail gate switch, but the position at which the rear camera is disposed is not limited thereto.

Each image photographed by the plurality of cameras is transferred to a processor of the camera, and the processor may synthesize the respective images to generate a vehicle surrounding image. In this case, the vehicle surrounding image may be displayed through the display unit as a top view image or a bird eye image.

The camera 310 photographs a road image while the vehicle is driving. The photographing operation of the camera 310 may be started under the control of the controller 300 or with the start and driving of the vehicle. The road image photographed by the camera 310 is provided to the controller 300.

The controller 300 obtains horizontal data from the road image. Although the road image photographed while driving on a curved road, for example, an uphill road or a downhill road, is not horizontal, horizontal data may be acquired through algorithms such as edge detection and horizontal line detection. As shown in FIG. 5, the horizontal data of the road is data including all the uphill, the downhill, the left and the right.

The vehicle tilt detection sensor 320 measures the vehicle body tilt. The vehicle tilt detection sensor 320 may be a tilt sensor or a gyro sensor. The controller 300 receives the inclination data measured by the vehicle inclination detection sensor 320. Here, the inclination includes up, down, left, and right inclinations of the vehicle being driven.

The controller 300 corrects the position of the camera based on the horizontal data and the tilt data. For example, as shown in Figure 4, when the change from the uphill to the downhill, since the horizontal value is changed, the position of the camera is automatically changed and corrected. Here, the object to be corrected is a setting value of a camera or a camera lens, and means a setting value in a state where the previous setting value, the road is horizontal, and the vehicle body is not tilted.

The display unit 330 may display a result of the position correction of the camera under the control of the controller 300. For example, as shown in FIG. 6, inconsistency information of a camera may be displayed and displayed in a form of fitting horizontal data. 6 illustrates an example of adjusting the horizontal position of the camera when the ADAS is initially installed. The horizontal position of the ADAS camera thus adjusted may change the position of the camera while driving on road conditions such as uphill and downhill driving. The controller 300 may correct the camera position to correspond to the initially set camera setting value, as shown in FIG. 6, based on the horizontal data and the tilt data photographed by the camera.

7 is a flowchart illustrating a camera position correction method according to another embodiment.

Referring to FIG. 7, in step 700, road images are acquired from a camera, and in step 702, horizontal data of a road is calculated.

In step 704, the vehicle body tilt data of the vehicle traveling on the road is obtained from the tilt sensor.

In step 706, the camera position is corrected based on the horizontal data and the tilt data. Here, the camera position means that the camera position is automatically corrected to the previous setting value, for example, the camera setting value suitable for the horizontal data adjusted during the initial installation.

Through the position correction method according to the embodiment, it is possible to solve the problem that the detection performance is reduced by changing the basic setting value of the ADAS according to the situation of the road while the vehicle is driving.

The embodiments may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by the computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, computer readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures, other data in modulated data signals such as program modules, or other transmission mechanisms, and includes any information delivery media.

Further, in this specification, “unit” may be a hardware component such as a processor or a circuit, and / or a software component executed by a hardware component such as a processor.

The foregoing description of the specification is intended to be illustrative, and one of ordinary skill in the art may understand that the present invention may be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. There will be. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present embodiment is shown by the following claims rather than the detailed description, and should be construed as including all changes or modifications derived from the meaning and scope of the claims and their equivalents.

Claims (8)

In the method of correcting the position of the ADAS camera while driving,
Photographing a road image using an ADAS camera while driving the vehicle, and obtaining horizontal data of the road from the photographed road image;
Acquiring inclination data obtained by measuring a vehicle body inclination using a sensor of the vehicle while the vehicle is driving; And
Correcting the position of the ADAS camera based on the obtained horizontal data and the tilt data. The method of claim 1,
The correction step,
And correcting a setting value of the ADAS camera so as to correspond to a previous setting value. The method of claim 1,
Wherein the sensor comprises a tilt sensor or a gyro sensor. The method of claim 3, wherein
Position up and down, left and right and front and rear inclination data of the driving vehicle using the inclination sensor or the gyro sensor. In the method for correcting the position of the camera mounted on the driving vehicle,
Obtaining horizontal data from a road image photographed while driving the vehicle;
Measuring an inclination of the inclined vehicle according to a state of a road while driving of the vehicle; And
Correcting the current position of the camera based on the obtained horizontal data and the measured tilt. A recording medium having recorded thereon a program for executing a method according to any one of claims 1 to 5 on a computer. In the device for correcting the position of the ADAS camera while driving,
A memory in which at least one program is stored; And
A processor for driving the position correction device by executing the at least one program,
The processor,
Take a road image using the ADAS camera while driving the vehicle, obtain horizontal data of the road from the recorded road image,
Acquisition of the inclination data measured the inclination of the vehicle body using the sensor of the vehicle while driving the vehicle,
And correcting the position of the ADAS camera based on the obtained horizontal data and the tilt data. In the device for correcting the position of the camera while driving,
A camera for photographing a road image while the vehicle is driving;
A tilt sensor for measuring a tilt of a vehicle inclined according to a state of a road while driving of the vehicle; And
And a controller configured to acquire horizontal data from the photographed road image and correct the current position of the camera based on the obtained horizontal data and the measured slope.

Images