KR20210069847A - System and method for controlling power tailgate based on 3d sensor - Google Patents

Abstract

According to an embodiment of the present invention, a three-dimensional (3D) sensor-based power tailgate control system may comprise: an input unit into which vehicle user's preset information for a power tailgate of a vehicle is input; a sensing unit for sensing a rear side of the vehicle or the power tailgate and acquiring 3D data of an object including an obstacle located at the rear side of the vehicle or the power tailgate; a signal/image processing unit for processing the 3D data of the object obtained by the sensing unit to extract 3D information of the object; and a control unit configured to control an operation of the power tailgate of the vehicle using information received from the input unit, the sensing unit, and the signal/image processing unit.

Description

SYSTEM AND METHOD FOR CONTROLLING POWER TAILGATE BASED ON 3D SENSOR

The present invention relates to a three-dimensional sensor-based power tailgate control system and method, and more particularly, extracts three-dimensional information (distance, position, shape, etc.) of an object from a signal received based on a three-dimensional sensor and detects an obstacle The present invention relates to a three-dimensional sensor-based power tailgate control system and method that can improve safety and convenience behind a vehicle by controlling operations including opening and closing of the power tailgate.

The present invention is the result of "2018 Auto Parts Company Supply Ecosystem Advancement Technology Development Project (Project Unique No.: P039100053)" carried out with the support of the Ministry of Trade, Industry and Energy and the Korea Institute of Industrial Technology Promotion.

The demand for a power tailgate opening/closing control system for the convenience of vehicle users, such as the conventional button/remote control sensing type power tailgate or ultrasonic sensor based foot motion and object sensing type smart power tailgate, is increasing.

Most power tailgate opening/closing control methods using conventional video cameras, infrared and ultrasonic methods have limitations in sensing performance depending on the external environment (night, rain, fog, etc.), and 3D information (distance, location, shape, etc.) is limited.

Ultrasonic sensors used as conventional obstacle detection sensors are sensitive to weather conditions and have limitations in detecting obstacles due to diffuse reflection, infrared sensors have a problem that signal characteristics change depending on the surrounding environment, and video cameras are vulnerable to low-light environments. have. In addition, existing sensors are limited in supplying three-dimensional information (distance, position, shape, etc.) of an object, and there is a difficulty in precise object recognition for safe power tailgate control.

Therefore, power tailgate through effective obstacle detection by acquiring more accurate three-dimensional information to improve vehicle rear safety and convenience using a three-dimensional sensor-based technology that can obtain information day and night without being affected by ambient illumination You need a control system.

An embodiment of the present invention provides safety and convenience through robust and reliable obstacle detection in various types of outdoor environments (night, rain, fog, etc.) using a three-dimensional sensing technology that can acquire three-dimensional information with high accuracy. An improved three-dimensional sensor-based power tailgate control system and method are provided.

An embodiment of the present invention extracts three-dimensional information (distance, position, shape, etc.) of an object from a signal received based on a three-dimensional sensor, detects an obstacle, and performs operation control including a power tailgate opening/closing function. A three-dimensional sensor-based power tailgate control system and method capable of improving safety and convenience are provided.

The problem to be solved by the present invention is not limited to the problem(s) mentioned above, and another problem(s) not mentioned will be clearly understood by those skilled in the art from the following description.

A three-dimensional sensor-based power tailgate control system according to an embodiment of the present invention includes an input unit to which preset information of a vehicle user for a power tailgate of a vehicle is input; a sensing unit sensing a rear side of the vehicle or the power tailgate and acquiring 3D data of an object including an obstacle positioned at the rear side of the vehicle or the power tailgate; a signal/image processing unit for processing 3D data of the object obtained by the sensing unit to extract 3D information of the object; and a control unit configured to control the operation of the power tailgate of the vehicle by using the information received from the input unit, the sensing unit, and the signal/image processing unit.

The sensing unit includes a three-dimensional camera or an image camera for photographing the rear of the vehicle or the power tailgate, and an image sensor for obtaining three-dimensional data of a photographed object, wherein the image sensor transmits the three-dimensional data to the signal /Can be sent to the image processing unit.

The sensing unit may further include at least one of an inertial measurement sensor, a temperature sensor, a pressure sensor, a ToF sensor, a radar, a lidar, or a biometric information detection sensor.

and a power tailgate microcontrol unit (MCU) receiving the opening/closing control information or the object information of the power tailgate from the control unit, wherein the control unit receives the status information of the power tailgate and inputted to the input unit It is possible to transmit/receive preset input information of a vehicle user.

The signal/image processing unit receives a control signal from the control unit, sensor information from the sensing unit, and preset input information of a vehicle user from the input unit, and obtains 3D data of the object from the sensing unit to control the control unit. through the power to the tailgate microcontrol unit.

The signal/image processing unit may use the 3D data received from the sensing unit as input information to detect an object and perform an algorithm for extracting 3D information of the object.

The signal/image processing unit may obtain the height of the object by converting the 3D coordinate information value according to the inclination of the sensing unit generated when the sensing unit is mounted on the rear of the vehicle.

In addition, the present invention provides a method for controlling a three-dimensional sensor-based power tailgate using the three-dimensional sensor-based power tailgate control system, the method comprising: inputting preset information of a vehicle user through the input unit; obtaining three-dimensional data of an object by the sensing unit; processing the 3D data of the object received from the sensing unit to extract and calculate 3D information of the object; and detecting the object using the extracted 3D information of the object and controlling the operation of the vehicle power tailgate.

In addition, the present invention provides a three-dimensional sensor-based power tailgate control method using the three-dimensional sensor-based power tailgate control system, comprising: receiving operation information of the sensing unit from the control unit and setting a sensor mode of the sensing unit; step; performing three-dimensional coordinate transformation on the three-dimensional data of the object received by the sensing unit; removing the floor and ceiling of the vehicle in the region of interest in consideration of the height setting information of the sensing unit; binarizing the 3D result data with the floor and ceiling removed; dividing an image by separating a background and a foreground with respect to the 3D result data; tracking the object by performing an object tracking algorithm; extracting object information including 3D distance and height information of the nearest object; and controlling the operation of the power tailgate by transmitting the extracted object information to the power tailgate microcontrol unit through the control unit.

In addition, the present invention provides a three-dimensional sensor-based power tailgate control method using the three-dimensional sensor-based power tailgate control system, comprising: receiving operation information of the sensing unit from the control unit and setting a sensor mode of the sensing unit; step; performing three-dimensional coordinate transformation on the three-dimensional data of the object received by the sensing unit; generating a LUT for the maximum distance of the power tailgate by assuming an opening/closing path of the power tailgate and calculating an angle for each pixel; Initializing the 3D detection result of the obstacle; setting an obstacle detection area; detecting an obstacle; binarizing the detected obstacle area; removing noise from the detected obstacle area; extracting distance information for each obstacle; and controlling the operation of the power tailgate according to obstacle detection.

In addition, the present invention provides a three-dimensional sensor-based power tailgate control method using the three-dimensional sensor-based power tailgate control system, comprising: receiving operation information of the sensing unit from the control unit and setting a sensor mode of the sensing unit; step; performing three-dimensional coordinate transformation on the three-dimensional data of the object received by the sensing unit; generating a LUT for the maximum distance of the power tailgate by assuming an opening/closing path of the power tailgate and calculating an angle for each pixel; setting a region of interest for detecting the vehicle rear floor; detecting the vehicle rear floor; detecting an obstacle area; binarizing the detected obstacle area; removing noise from the detected obstacle area; extracting distance information for each obstacle; and controlling an operation of the power tailgate according to the detection of an obstacle. It is possible to provide a three-dimensional sensor-based power tailgate control method comprising a.

The details of other embodiments are included in the detailed description and accompanying drawings.

According to an embodiment of the present invention, high-precision obstacle detection robust to the external environment (night, rain, fog, etc.) using three-dimensional sensing technology and safer and more convenient power using three-dimensional information of the obstacle A tailgate control system may be provided.

According to an embodiment of the present invention, it can be extended to a rear detection system applicable to a driver assistance system, an autonomous driving system, and an automatic parking system in conjunction with a vehicle rear control function.

1 is a diagram showing the configuration of a three-dimensional sensor-based power tailgate control system according to an embodiment of the present invention.
2 is a diagram illustrating an application example of a three-dimensional sensor-based power tailgate control system according to an embodiment of the present invention.
FIG. 3 is a diagram illustrating a principle of calculating the height of a detected object according to the inclination of a sensor in the system according to FIG. 1 .
4 to 7 are flowcharts for explaining a 3D sensor-based power tailgate control method using the system according to FIG. 1 .
FIG. 8 is a diagram exemplarily showing an object detection result for controlling a power tailgate opening angle in the system according to FIG. 1 .
9 is a diagram exemplarily showing an obstacle detection result when the power tailgate is opened in the system according to FIG. 1 .
FIG. 10 is a diagram exemplarily showing an obstacle detection result when the power tailgate is closed in the system according to FIG. 1 .

Advantages and/or features of the present invention, and methods of achieving them, will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the art to which the present invention pertains It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout.

In addition, the preferred embodiment of the present invention to be implemented below is already provided in each system functional configuration in order to efficiently describe the technical components constituting the present invention, or system functions normally provided in the technical field to which the present invention belongs The configuration is omitted as much as possible, and the functional configuration to be additionally provided for the present invention will be mainly described. If a person of ordinary skill in the art to which the present invention pertains, it will be possible to easily understand the functions of the conventionally used components among the functions omitted not shown below, and the components omitted as described above. Relationships between elements and components added for purposes of the present invention will also be clearly understood.

In addition, in the following description, the term "transmission", "communication", "transmission", "reception" and other similar meanings of a signal or information means that a signal or information is directly transmitted from one component to another component as well as passing through other components. In particular, "transmitting" or "transmitting" a signal or information to a component indicates the final destination of the signal or information and does not imply a direct destination. The same is true for "reception" of signals or information.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram showing the configuration of a three-dimensional sensor-based power tailgate control system according to an embodiment of the present invention, FIG. 2 is an application example of the three-dimensional sensor-based power tailgate control system according to an embodiment of the present invention 3 is a view showing the principle of calculating the height of a detected object according to the inclination of the sensor in the system according to FIG. 1, and FIGS. 4 to 7 are a three-dimensional sensor-based power tailgate using the system according to FIG. A flowchart for explaining a control method, FIG. 8 is a view exemplarily showing an object detection result for controlling the power tailgate opening angle in the system according to FIG. 1 , and FIG. 9 is an obstacle when the power tailgate is opened in the system according to FIG. FIG. 10 is a view exemplarily showing the detection result of the obstacle when the power tailgate is closed in the system according to FIG. 1 .

1 and 2 , in the three-dimensional sensor-based power tailgate control system 100 according to an embodiment of the present invention, the user's preset information for the power tailgate 210 of the vehicle 200 is input. an input unit 110; a sensing unit 140 sensing the rear of the vehicle 200 or the power tailgate 210 and acquiring 3D data of an object (or obstacle); a signal/image processing unit 120 for processing 3D data of the object obtained by the sensing unit 140 to extract 3D information of the object; and a control unit 130 for controlling the operation of the power tailgate 210 of the vehicle 200 using the information received from the input unit 110 , the sensing unit 140 , and the signal/image processing unit 120 . may include.

A three-dimensional sensor-based power tailgate control system 100 (hereinafter, referred to as a “power tailgate control system”) according to an embodiment of the present invention is a system for controlling the operation of the power tailgate 210 of a vehicle 200 . As such, it can be applied to the trunk door of all vehicles such as a sedan as well as the SUV shown in FIG. 2 .

In FIG. 1 , reference numeral 160 denotes an interface unit. The interface unit 160 may be simultaneously connected to the input unit 110 , the signal/image processing unit 120 , the control unit 130 , and the sensing unit 140 to transmit/receive signals or information.

The power tailgate control system 100 according to an embodiment of the present invention having a configuration as shown in FIG. 1 uses a three-dimensional sensor technology to sense a signal reflected from an object and received by the image of the sensing unit 140 . An object can be detected by acquiring 3D data from a sensor, extracting 3D information of an object (object), and segmenting an image. Safety and convenience can be provided to vehicle users by controlling the operation of the power tailgate 210 in real time, including the opening/closing function, using three-dimensional information (eg, distance, location, shape, etc.) of the detected object. have.

As shown in FIG. 1 , the power tailgate control system 100 according to an embodiment of the present invention includes an input unit 110 that receives input information from a vehicle user, and a power tailgate 210 of a vehicle 200 . The control unit 130 for controlling the operation, the sensing unit 140 for sensing the rear surrounding environment of the vehicle 200 or the power tailgate 210, and processing a signal or image data obtained from the sensing unit 140 to detect an object It may include a signal/image processing unit 120 for detecting and extracting 3D information of the object.

The user's preset information for the power tailgate 210 of the vehicle 200 may be input to the input unit 110 . The user's preset information is a user input setting profile for the power tailgate 210 , and the sensor inclination of the sensing unit 140 , the sensor height, the width of the power tailgate, the length of the power tailgate, and the total movement angle of the power tailgate. , a region of interest, an obstacle recognition size, and the like.

That is, the input unit 110 is a device for receiving information from the user of the vehicle 200 , and the input information may include a touch input device, a steering wheel input device, a smart key input device, and a setting file. In addition, user input information may be received from the sensor device of the sensing unit 140 . The user input information input to the input unit 110 may be transmitted to the control unit 130 to control the operation including the opening/closing function of the power tailgate 210 , and transmitted to the sensing unit 140 to control the operation of the sensor. may be

The sensing unit 140 may include a three-dimensional camera or an image camera for photographing the rear of the vehicle 200 or the power tailgate 210 , and an image sensor for obtaining three-dimensional data of the photographed object. may include.

The image sensor may acquire 3D data of an object from an image captured by the camera and transmit it to the signal/image processing unit 120 .

Additionally, the sensing unit 140 may further include at least one of an inertial measurement sensor, a temperature sensor, a pressure sensor, a Time of Flight (ToF) sensor, a radar, a lidar, or a biometric information detection sensor. Here, the biometric information may include fingerprint recognition, hand shape recognition, face recognition, voice recognition information, and the like.

The power tailgate control system 100 according to an embodiment of the present invention includes a power tailgate micro control unit (MCU) that receives the opening/closing control information or the object information of the power tailgate 210 from the controller 130 . Controller Unit, 150) may be further included.

Here, the power tailgate microcontrol unit 150 may be provided separately from the control unit 130 or may be included in the control unit 130 .

The control unit 130 may transmit open/close control information or detected object information of the power tailgate 210 to the power tailgate microcontrol unit 150 and receive status information of the power tailgate 210, and an input unit ( 110) may transmit/receive vehicle user input information.

The signal/image processing unit 120 receives a control signal from the control unit 130 , sensor information from the sensing unit 140 , and vehicle user input information from the input unit 110 , and 3 of the object from the sensing unit 140 . The dimensional data may be acquired and transmitted to the power tailgate microcontrol unit 150 through the control unit 130 . In other words, the signal/image processing unit 120 receives control information from the control unit 130 , sensor information from the sensing unit 140 , and vehicle user input information from the input unit 110 , and detects the object and selects three of the detected objects. In order to obtain dimensional information (distance, position, shape, etc.) and control the operation of the power tailgate 210 , the control unit 130 transmits it to the power tailgate microcontrol unit 150 .

Here, the signal/image processing unit 120 is a device that is performed on an application processor (AP) with high CPU (Central Processing Unit) performance, and a graphics processing unit (GPU) or digital signal signal (DSP) (DSP) to accelerate data processing speed. Processing) can be used for parallel processing. The AP may be designed together with an existing MCU including a GPU or DSP to become an integrated Electronic Control Unit (ECU) device for controlling the operation of the power tailgate.

The signal/image processing unit 120 may use the 3D data received from the sensing unit 140 as input information to detect an object and perform an algorithm for extracting 3D information of the object.

The signal/image processing unit 120 may perform an algorithm for detecting an object and extracting 3D information (distance, location, shape, etc.) of the object for controlling the operation of the power tailgate 210 of the vehicle 200 . have.

Meanwhile, some of the sensors of the sensing unit 140 may be installed at the rear of the vehicle 200 . Referring to FIG. 2 , the 3D sensor 141 may be installed at the lower end of the power tailgate 210 . In this case, the inclination of the 3D sensor 141 also changes according to the opening angle of the power tailgate 210 . Accordingly, the coordinate value of the 3D information on the obstacle (object) detected from the rear of the vehicle 200 may vary according to the inclination of the sensing unit 140 or the sensor mounted on the rear of the vehicle 200 . In consideration of this point, the power tailgate control system 100 according to an embodiment of the present invention provides three-dimensional coordinate information according to the inclination of the sensor generated when the three-dimensional sensors 140 and 141 are mounted on the rear of the vehicle 200 . The conversion calculation is considered to obtain the height of the object because the sensor origin and sensor height of the three-dimensional Y-coordinate change according to the inclination of the sensor while converting the value.

The signal/image processing unit 120 determines the sensor origin of the three-dimensional Y coordinate and the sensor height according to the inclination of the sensing units 140 and 141 generated when the sensing units 140 and 141 are mounted on the rear of the vehicle 200 depending on the distance. Since it changes, the height of the object can be obtained using the following [Equation 1] to [Equation 5].

In [Equation 1] to [Equation 5], θ is the inclination of the sensor, z is the Z coordinate received from the tilted sensor, and y is the Y coordinate received from the tilted sensor. Also, FIG. 3 shows a method of calculating the height of an object according to a three-dimensional sensor inclination using [Equation 1] to [Equation 5].

3 and [Equation 1] to [Equation 5], the detected height of the object may be used as control information of the opening angle of the power tailgate 210 to improve the convenience of the vehicle user.

On the other hand, the power tailgate control system 100 according to an embodiment of the present invention detects an obstacle in the rear of the vehicle 200 or the power tailgate 210 in real time when the power tailgate 210 is opened and closed to be safely secured. An obstacle detection algorithm may be performed to control the operation of the power tailgate 210 .

The sensing unit 140 may include a three-dimensional sensor 141 to detect an obstacle. The three-dimensional sensor 141 may be equipped with two sensors for effective real-time obstacle detection when the power tailgate 210 is opened and closed. Any one of these sensors 141 may be mounted on the upper end of the power tailgate 210 and used to detect an obstacle when the power tailgate 210 is opened, and the other sensor 141 is located at the rear of the vehicle 200 . It can be used for detecting an obstacle when the power tailgate 210 is closed by being mounted on the inner ceiling or inside the power tailgate 210 .

If the 360 degree rotation motor method is applied to the 3D sensor, one 3D sensor may be used to detect obstacles in real time when the power tailgate 210 is opened and closed. Since the sensing direction of the three-dimensional sensor can be changed by the 360-degree rotation motor, only one sensor can detect obstacles when the power tailgate 210 is opened and obstacles are detected when the power tailgate 210 is closed.

The system described above may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

Software may comprise a computer program, code, instructions, or a combination of one or more of these, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems, and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

Hereinafter, a three-dimensional sensor-based power tailgate control method according to an embodiment of the present invention will be described. The three-dimensional sensor-based power tailgate control method (hereinafter, referred to as a “power tailgate control method”) according to an embodiment of the present invention may be performed using the power tailgate control system 100 described above.

First, referring to FIG. 4 , in the present invention, in the three-dimensional sensor-based power tailgate control method using the three-dimensional sensor-based power tailgate control system 100 , user information is inputted by the input unit 110 . inputting (1100); obtaining 3D data of an object by the sensing unit 140 (1200); processing the 3D data of the object received from the sensing unit 140 to extract and calculate 3D information of the object (1300); and detecting the object using the extracted 3D information of the object and controlling the operation of the vehicle power tailgate 210 ( 1400 ).

In the power tailgate control method according to an embodiment of the present invention, the step of applying user input information to the input unit 110 ( 1100 ), the 3D data using the high accuracy 3D sensing unit 140 robust to the external environment Step 1200 of obtaining a, processing the data received from the three-dimensional image sensor of the sensing unit 140 to extract and calculate three-dimensional information (distance, location, shape, etc.) of the object (1300) and the extracted The method may include detecting an object using 3D information of the object and controlling an operation including an opening/closing function of the power tailgate 210 of the vehicle 200 ( 1400 ).

Here, the object means an obstacle or an object located or existing in the rear of the vehicle 200 or the power tailgate 210 , and is a concept meaning a vehicle driver or vehicle occupant.

5 is a flowchart illustrating a method of controlling the opening angle of the power tailgate 210 using the power tailgate control system 100 according to an embodiment of the present invention.

Referring to FIG. 5 , in the present invention, in the three-dimensional sensor-based power tailgate control method using the three-dimensional sensor-based power tailgate control system 100 , the sensing unit 140 is receiving operation information of and setting a sensor mode of the sensing unit 140 (2100); performing 3D coordinate transformation on the 3D data of the object received by the sensing unit 140 (2200); removing the floor and ceiling in the region of interest (2300); binarizing the 3D result data (2400); dividing the image by separating the background and the foreground (2500); performing an object tracking algorithm to track the object (2600); extracting object information including 3D distance and height information of the nearest object (2700); and transmitting the extracted object information to the power tailgate microcontrol unit 150 through the control unit 130 to control the operation of the power tailgate 210 ( 2800 ). A gate control method may be provided.

Here, a distance-based morphology step may be performed between the step 2300 of removing the floor and the ceiling and the step 2400 of binarizing the 3D result data.

A method of controlling the opening angle of the power tailgate 210 will be described in detail with reference to FIG. 5 . First, in order to control the opening angle of the three-dimensional sensor-based power tailgate 210, the controller 130 receives the sensor operation information of the sensing unit 140 from the power tailgate microcontrol unit 150, and the sensing unit ( 140) sets the sensor mode to the power tailgate opening angle control mode (2100).

In the 3D data of the object received from the sensing unit 140, the user input setting profile (sensor tilt, sensor height, power tailgate width, power tailgate length, power) received from the input unit 110 or the control unit 130 3D coordinate transformation is calculated and transformed using Figure 3 and [Equation 1] to [Equation 5] by applying the entire tailgate movement angle, region of interest, obstacle recognition size, etc.), and 3D using region of interest information A pass-through algorithm is applied to the coordinate information ( 2200 ).

In order to effectively perform object detection, a floor and ceiling removal algorithm in the region of interest is performed using the floor 220 (refer to FIG. 2 ) and ceiling characteristics in consideration of the three-dimensional Y-coordinate, the sensor tilt, and the sensor height setting information ( 2300 ).

The three-dimensional result data obtained by performing the floor and ceiling removal algorithm is binarized (2400), the background and foreground are separated (2500), an object is detected through labeling, and an object tracking algorithm is performed (2600). In addition, after extracting (2700) the 3D distance and height information of the nearest object, the object information is transmitted to the power tailgate microcontrol unit 150 through the control unit 130 to extract when the power tailgate 210 is opened. The height information is regarded as the height of the vehicle user and the opening angle of the power tailgate 210 is controlled so that the power tailgate 210 can be stopped at a height at which the vehicle user can easily open and close the power tailgate 210 . to perform (2800).

8 is a diagram exemplarily showing an object detection result for controlling a three-dimensional sensor-based power tailgate opening angle.

6 is a flowchart illustrating a method of controlling the operation of the power tailgate 210 according to the detection of an obstacle when the power tailgate 210 is opened using the power tailgate control system 100 according to an embodiment of the present invention. is shown.

Referring to FIG. 6 , in the present invention, in the three-dimensional sensor-based power tailgate control method using the three-dimensional sensor-based power tailgate control system 100 , the sensing unit 140 is receiving operation information of and setting a sensor mode of the sensing unit 140 (3100); performing 3D coordinate transformation on the 3D data of the object received by the sensing unit 140 (3200); generating a LUT for the maximum distance of the power tailgate 210 ( 3300 ); Initializing the obstacle detection result (3400); setting an obstacle detection area (3500); detecting an obstacle (3600); binarizing the detected obstacle area (3700); removing noise from the detected obstacle area (3800); extracting distance information for each obstacle ( 3900 ); and controlling the operation of the power tailgate 210 according to the obstacle detection ( 3950 ).

A method of controlling the operation of the power tailgate according to the detection of an obstacle when the power tailgate 210 is opened will be described in detail with reference to FIG. 6 . First, in order to control the operation of the power tailgate according to the detection of an obstacle when the three-dimensional sensor-based power tailgate is opened, the controller 130 receives sensor operation information from the power tailgate microcontrol unit 150 and the sensing unit 140 . The sensor mode of ' is set to the obstacle detection mode when the power tailgate is opened ( 3100 ).

In the 3D data received from the sensing unit 140, the user input setting profile (sensor tilt, sensor height, power tailgate width, power tailgate length, power tailgate) received from the input unit 110 or the control unit 130 3D coordinate transformation is calculated and transformed using Figure 3 and [Equation 1] to [Equation 5] by applying the total motion angle, region of interest, obstacle recognition size, etc.), and 3D coordinate information using region of interest information A pass-through algorithm is applied to (3200).

Assuming that the movement of opening the power tailgate 210 in the 3D image draws a circle, calculate the angle for each pixel, and read a lookup table (LUT) for the maximum distance of the power tailgate 210 first once Create (3300).

The 3D information of the obstacle is initialized ( 3400 ), and an obstacle detection area is set ( 3500 ) to detect the obstacle ( 3600 ). The detected obstacle area is binarized ( 3700 ), noise is removed from the detected obstacle area by morphological filtering ( 3800 ), and 3D distance information for each detected obstacle is extracted by performing labeling ( 3900 ). The controller 130 transmits the detected obstacle information to the power tailgate microcontrol unit 150 to perform power tailgate opening stop control with improved safety for the opening operation of the power tailgate 210 according to the detection of the obstacle ( 3950).

9 is a diagram exemplarily showing an obstacle detection result for detecting an obstacle when the three-dimensional sensor-based power tailgate is opened.

7 is a flowchart illustrating a method of controlling the operation of the power tailgate 210 according to the detection of an obstacle when the power tailgate 210 is closed using the power tailgate control system 100 according to an embodiment of the present invention. is shown.

Referring to FIG. 7 , in the present invention, in the three-dimensional sensor-based power tailgate control method using the three-dimensional sensor-based power tailgate control system 100 , the sensing unit 140 from the controller 130 receiving operation information of and setting a sensor mode of the sensing unit 140 (4100); performing 3D coordinate transformation on the 3D data of the object received by the sensing unit 140 (4200); generating (4300) a LUT for the maximum distance of the power tailgate (210); setting a region of interest for detecting the vehicle rear floor (4400); detecting the vehicle rear floor (4500); detecting an obstacle area (4600); binarizing the detected obstacle area (4700); removing noise from the detected obstacle area (4800); extracting distance information for each obstacle ( 4900 ); and controlling the operation of the power tailgate according to the detection of an obstacle ( 4950 ).

A method of controlling the operation of the power tailgate according to the detection of an obstacle when the power tailgate 210 is closed will be described in detail with reference to FIG. 7 . First, in order to control the operation of the power tailgate according to the detection of an obstacle when the three-dimensional sensor-based power tailgate is closed, the controller 130 receives sensor operation information from the power tailgate microcontrol unit 150 and the sensing unit 140 . Set the sensor mode of , to the obstacle detection mode when the power tailgate is closed ( 4100 ).

In the 3D data received from the sensing unit 140, the user input setting profile (sensor tilt, sensor height, power tailgate width, power tailgate length, power tailgate) received from the input unit 110 or the control unit 130 3D coordinate transformation is calculated and transformed using Figure 3 and [Equation 1] to [Equation 5] by applying the total motion angle, region of interest, obstacle recognition size, etc.), and 3D coordinate information using region of interest information A pass-through algorithm is applied to (4200).

Assuming that the movement of opening the power tailgate 210 in the 3D image draws a circle, calculate the angle for each pixel, and read a lookup table (LUT) for the maximum distance of the power tailgate 210 first once Create (4300).

In addition, when the sensor mode is changed or when the algorithm is initially executed, a region of interest for detecting the vehicle rear floor (vehicle rear bumper or interior floor connected to the bumper) is set ( 4400 ) and the vehicle rear floor is detected ( 4500 ).

The noise of the received 3D data is removed using morphological filtering. The y-coordinate of the starting point of the rear floor of the vehicle is obtained from the initially set vehicle rear floor of interest region, and if detection is not possible, the previously detected vehicle floor detection result is used.

The detection of the obstacle area is performed except for the power tailgate detection area and the vehicle rear floor detection area ( 4600 ). As a result, the image is subjected to labeling through binarization (4700) and noise removal through morphological filtering (4800) to extract 3D distance information for each detected obstacle (4900). The controller 130 transmits the detected obstacle information to the power tailgate microcontrol unit 150 to perform power tailgate closing and stopping control with improved safety for the closing operation of the power tailgate 210 according to the detection of the obstacle ( 4950).

10 is a diagram exemplarily showing an obstacle detection result for detecting an obstacle when the three-dimensional sensor-based power tailgate is closed.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CDROMs and DVDs, magneto-optical disks such as floppy disks. hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

The three-dimensional sensor-based power tailgate control system and method according to an embodiment of the present invention described above extracts distance and height information of the nearest object based on the three-dimensional sensor, and the extracted height information is the height of the vehicle user. Considering this, it is possible to provide a power tailgate control system and method with improved convenience for controlling the power tailgate opening angle so that the power tailgate can be opened and stopped at a height at which the vehicle user can easily open and close the power tailgate.

In addition, it is possible to provide a power tailgate control system and method with improved safety for extracting the distance of the nearest object based on a three-dimensional sensor and recognizing a danger in real time when the power tailgate is opened and closed to stop the operation of the power tailgate.

In addition, when extracting information on a nearby object, it effectively extracts 3D information (distance, height, location, shape, etc.) of a nearby object through floor and ceiling removal using the characteristics of the floor and ceiling in consideration of the inclination and height information of the sensor. A power tailgate control system and method may be provided.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible from the above description by those skilled in the art. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

100: 3D sensor-based power tailgate control system
110: input unit
120: signal / image processing unit
130: control unit
140: sensing unit
150: power tailgate microcontrol unit
160: interface unit

Claims (11)

an input unit into which vehicle user's preset information for a power tailgate of the vehicle is input;
a sensing unit sensing a rear side of the vehicle or the power tailgate and acquiring 3D data of an object including an obstacle positioned at the rear side of the vehicle or the power tailgate;
a signal/image processing unit for processing 3D data of the object obtained by the sensing unit to extract 3D information of the object; and
a controller configured to control an operation of the power tailgate of the vehicle using information received from the input unit, the sensing unit, and the signal/image processing unit;
A three-dimensional sensor-based power tailgate control system comprising a.
According to claim 1,
The sensing unit,
A three-dimensional camera or video camera for photographing the rear of the vehicle or the power tailgate, and an image sensor for obtaining three-dimensional data of the photographed object,
The three-dimensional sensor-based power tailgate control system, characterized in that the image sensor transmits the three-dimensional data to the signal/image processing unit.
3. The method of claim 2,
The sensing unit,
3D sensor-based power tailgate control system, characterized in that it further comprises at least one of an inertial measurement sensor, a temperature sensor, a pressure sensor, a ToF sensor, a radar, a lidar, or a biometric information detection sensor.
3. The method of claim 2,
Further comprising a power tailgate microcontrol unit (MCU) receiving the opening/closing control information or the object information of the power tailgate from the control unit,
The three-dimensional sensor-based power tailgate control system, wherein the control unit receives the state information of the power tailgate and transmits and receives preset input information of a vehicle user input to the input unit.
5. The method of claim 4,
The signal / image processing unit,
Receives a control signal from the control unit, sensor information from the sensing unit, and preset input information of a vehicle user from the input unit, obtains 3D data of the object from the sensing unit, and controls the power tailgate microcontroller through the control unit A three-dimensional sensor-based power tailgate control system, characterized in that it is delivered to the unit.
6. The method of claim 5,
3D sensor-based power tailgate control system, characterized in that the signal/image processing unit detects an object using the 3D data received from the sensing unit as input information and performs an algorithm for extracting 3D information of the object .
7. The method of claim 6,
3D sensor-based power tailgate control, characterized in that the signal/image processing unit converts the 3D coordinate information value according to the inclination of the sensing unit generated when the sensing unit is mounted on the rear of the vehicle to obtain the height of the object system.
In the three-dimensional sensor-based power tailgate control method using the three-dimensional sensor-based power tailgate control system according to any one of claims 1 to 7,
inputting preset information of a vehicle user by the input unit;
obtaining three-dimensional data of an object by the sensing unit;
processing the 3D data of the object received from the sensing unit to extract and calculate 3D information of the object; and
3D sensor-based power tailgate control method comprising: detecting an object using the extracted 3D information of the object and controlling the operation of the vehicle power tailgate.
In the three-dimensional sensor-based power tailgate control method using the three-dimensional sensor-based power tailgate control system according to any one of claims 1 to 7,
receiving operation information of the sensing unit from the control unit and setting a sensor mode of the sensing unit;
performing three-dimensional coordinate transformation on the three-dimensional data of the object received by the sensing unit;
removing the floor and ceiling of the vehicle in the region of interest in consideration of the height setting information of the sensing unit;
binarizing the 3D result data with the floor and ceiling removed;
dividing an image by separating a background and a foreground with respect to the 3D result data;
tracking the object by performing an object tracking algorithm;
extracting object information including 3D distance and height information of the nearest object; and
controlling the operation of the power tailgate by transmitting the extracted object information to the power tailgate microcontrol unit through the control unit;
A three-dimensional sensor-based power tailgate control method comprising a.
In the three-dimensional sensor-based power tailgate control method using the three-dimensional sensor-based power tailgate control system according to any one of claims 1 to 7,
receiving operation information of the sensing unit from the control unit and setting a sensor mode of the sensing unit;
performing three-dimensional coordinate transformation on the three-dimensional data of the object received by the sensing unit;
generating a LUT for the maximum distance of the power tailgate by assuming an opening/closing path of the power tailgate and calculating an angle for each pixel;
Initializing the 3D detection result of the obstacle;
setting an obstacle detection area;
detecting an obstacle;
binarizing the detected obstacle area;
removing noise from the detected obstacle area;
extracting distance information for each obstacle; and
controlling an operation of the power tailgate according to the detection of an obstacle;
A three-dimensional sensor-based power tailgate control method comprising a.
In the three-dimensional sensor-based power tailgate control method using the three-dimensional sensor-based power tailgate control system according to any one of claims 1 to 7,
receiving operation information of the sensing unit from the control unit and setting a sensor mode of the sensing unit;
performing three-dimensional coordinate transformation on the three-dimensional data of the object received by the sensing unit;
generating a LUT for the maximum distance of the power tailgate by assuming an opening/closing path of the power tailgate and calculating an angle for each pixel;
setting a region of interest for detecting the vehicle rear floor;
detecting the vehicle rear floor;
detecting an obstacle area;
binarizing the detected obstacle area;
removing noise from the detected obstacle area;
extracting distance information for each obstacle; and
controlling an operation of the power tailgate according to the detection of an obstacle;
A three-dimensional sensor-based power tailgate control method comprising a.

Images