KR101886416B1 - System for controlling door of vehicle and method thereof - Google Patents

Abstract

The present invention relates to a system for controlling a vehicle door and, more specifically, to a system and a method for automatically opening a passenger seat door in accordance with types of passengers without a direct operation of a driver. Accordingly, a photographing means for photographing around a vehicle is installed on one side of the vehicle, and if it is determined that an object satisfying a certain condition approaches by analyzing an image obtained from the photographing means, the vehicle door can be opened so as to increase user convenience of getting in a vehicle.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a vehicle door control system,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle door control system, and more particularly, to a vehicle control system and method for automatically opening a door of a passenger seat or a passenger seat in accordance with a boarding target even if the driver does not directly operate the vehicle.

2. Description of the Related Art [0002] Vehicle manufacturers are currently equipped with various electronic control devices such as an engine control device, a transmission control device, a suspension control device, and a braking control device in a vehicle according to the development of vehicle manufacturing technology. The function is implemented so that it can be automatically recognized and controlled by the sensor, rather than being operated and controlled.

Japanese Patent Laid-Open Publication No. 10-2017-0045591 is an example of applying the above-described vehicle automation technology. In the case where a driver or a passenger tries to ride a vehicle, it is inconvenient to open the vehicle door due to heavy load or unfavorable conditions And suggest a technique to cope with it.

In the above-described conventional technique, an AVM module (Around View Monitoring Module) is mounted on a vehicle, and a door opening / closing part located on one side of the vehicle door is provided, so that a driver or a passenger can automatically open / close the door without using a handle, There is an advantage in that it can be increased.

However, according to the prior art, only a user having a specific identification means such as a remote controller is set as a target, and a user who does not have the identification means can not provide a door opening function for accessing the vehicle.

Japanese Patent Application Laid-Open No. 10-2017-0045591 (published on Apr. 27, 2017). Patent Registration No. 10-1317582 (Publication Date: Oct. 11, 2013)

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and it is an object of the present invention to provide a system for automatically opening a vehicle door according to a predetermined condition when an unspecified person who does not possess a specific identification means approaches the vehicle have.

In order to solve the above problems, a vehicle door control system according to an embodiment of the present invention includes a camera module installed in any one of a vehicle door and capturing a video signal in one direction in real time to acquire a video signal, A control unit for generating a control signal for controlling the opening and closing of the vehicle door when the object displayed in the image meets predetermined conditions, and a door open / close module for opening the vehicle door in response to the control signal .

According to another aspect of the present invention, there is provided a method for controlling a vehicle door using a system as described above, Generating a control signal for controlling opening or closing of a vehicle door when the object displayed in the image satisfies predetermined conditions by analyzing the video signal; And opening the vehicle door.

According to the embodiment of the present invention, a photographing means for photographing the periphery is installed on one side of the vehicle, and when the image obtained from the photographing means is analyzed and it is judged that the target person satisfying the specific condition approaches, This has the effect of increasing the convenience of the subject.

1 is a diagram illustrating an example in which a vehicle door control system according to an embodiment of the present invention is applied.
2 is a view showing a structure of a vehicle door control system according to an embodiment of the present invention.
3 is a diagram showing a structure of a program of a vehicle door control system according to an embodiment of the present invention.
4 is a view illustrating a vehicle door control method according to an embodiment of the present invention.
5A to 5C are views showing examples of images derived in the image processing process of the control door control system of the present invention.

Prior to the description, when an element is referred to as being "comprising " or" comprising "any element throughout the specification, it is to be understood that the element It means that you can.

The term "embodiment" is used herein to mean serving as an example, instance, or illustration, although the subject matter of the invention is not limited by such example. It is also to be understood that the word "comprising, "" having "and other similar terms are used, unless the context clearly dictates otherwise. &Quot;Quot; is < / RTI > used in a manner similar to the term " Comprising ".

The terms "unit," "device," and "system" are used throughout this specification to describe one or more combined operations , Which may be implemented in hardware, software, or a combination of hardware and software.

The various techniques described herein may be implemented with hardware or software, or may be implemented with a combination of both, where appropriate. As used herein, terms such as "part," "device," and " system "are likewise equally applicable to computer-related entities such as a combination of hardware, hardware and software, . Further, in the present invention, an application program executed in a user terminal may be constituted by "sub-units ", and may be recorded in one physical memory in a form readable, writable and erasable, or distributed among two or more memories or recording media Lt; / RTI >

Hereinafter, a vehicle door control system and method according to a preferred embodiment of the present invention will be described with reference to the drawings.

1 is a diagram illustrating an example in which a vehicle door control system according to an embodiment of the present invention is applied.

Referring to FIG. 1, a vehicle door control system according to an embodiment of the present invention may include a camera module 16 installed on one side of a vehicle 10 and photographing the periphery thereof, and a video signal obtained therefrom The vehicle door 14 in the corresponding direction can be opened by analyzing it through a control unit (not shown) installed in the vehicle.

The camera module 16 described above is installed in the vehicle 10 or the right vehicle door 14 so that the camera module 16 can photograph the surroundings in the same elevation direction 12 opposite to the driver's seat direction 11 in the two- .

Alternatively, the camera module 16 may be substituted for the case where the black box 17 for photographing the forward and backward directions installed in the vehicle 10 captures 360 degrees all directions.

Particularly, in the image obtained by the camera module 16, objects other than the object may appear, and in the vehicle door control system according to the embodiment of the present invention, objects excluding the person in the image signal are filtered, It is possible to open the vehicle door 14 by generating and providing a control signal of the opening and closing means provided on the vehicle door 14 when there is a person who meets a predetermined condition and approaches a predetermined distance or more.

In this case, the target person may be a woman who is expected to be a passenger according to the setting of the system configurator. Accordingly, the vehicle door control system of the present invention removes objects in the image and displays the gender of the target object in an image processing method . Here, in discriminating the gender of the subject, consideration is given not only to a face region but also to at least one region capable of discriminating gender of other subjects, and by setting a weight that can yield a best result for each region , Providing high accuracy gender recognition results.

 Hereinafter, the structure of a vehicle door control system according to an embodiment of the present invention will be described in detail with reference to the drawings.

2 is a view showing a structure of a vehicle door control system according to an embodiment of the present invention.

Referring to FIG. 2, the vehicle door control system 100 according to the embodiment of the present invention includes a camera module 110 installed in one of the vehicle doors and capturing a video signal in one direction in real time, A controller 120 for generating a control signal for controlling the opening and closing of the vehicle door when the object displayed in the image corresponds to a predetermined condition by analyzing the signal and a door open / (130).

The camera module 110 may be installed in a vehicle or on a vehicle door so as to acquire a video signal for a surrounding situation by photographing one direction in real time. The camera module 110 may be a wide-angle lens or a fisheye lens having a wide field of view and a high depth of field.

The control unit 120 receives the image signal acquired by the camera module 110, identifies the object using an image processing technique, and generates a control signal for controlling the opening and closing of the vehicle door according to the set conditions.

The control unit may be a microcomputer including a processor 121 and a memory 122. A program for implementing an image processing technique in the mounted memory 122 may be recorded, So that the object in the video signal is identified and a control signal is generated.

The above-mentioned program includes a function of preprocessing a video signal, a function of extracting an object appearing in an image, a function of setting a region of interest (ROI) area for analysis, a function of discriminating and determining a sex of a subject, And a plurality of functions implemented through a programming language.

The door opening / closing module 130 is installed in the vehicle door, and can release the locked state of the vehicle door according to the received control signal. The door opening / closing module 130 may be connected to at least one actuator installed on the locking device and the hinge of the vehicle door, and may open the vehicle door by driving the actuator according to the command for opening the control signal.

Here, the control signal can determine not only the opening and closing operation of the vehicle door but also the opening width of the vehicle door.

The vehicle door control system 100 according to an embodiment of the present invention includes a communication unit 140 that is connected to an external system and collects and provides a plurality of samples to a learning device to improve the accuracy of object recognition results, And an infrared sensor 150 for measuring a distance between a person approaching the vehicle and a vehicle parked adjacent to the vehicle.

In the embodiment of the present invention, a machine learning method may be applied to the program installed in the controller 120. Machine learning acquires results by repeatedly performing a series of information acquisition processes using a plurality of samples. As the number of samples used is increased, good results can be obtained, and a plurality of images related to object recognition can be obtained (Not shown) that distributes the sample to be distributed to receive the sample therefrom and provide it to the program.

In addition, the communication unit 140 may further perform a function of updating a program installed in the control unit 120 and connecting to an external interface for changing a set value.

The infrared sensor 150 may be installed on a vehicle door or the like, and may measure a distance to an object appearing within the sensing range and report the distance to the control unit 120. The distance measured by the infrared sensor 150 may be used to calculate the distance between the vehicle door and the subject through the program, and may be used to determine the time of the vehicle door opening drive when the subject is approaching the condition have.

In addition, the infrared sensor 150 can detect the distance from the parked vehicle to the neighboring vehicle, and the detection result can be utilized to control the opening width of the vehicle door when it is opened.

The main system 300 includes at least one ECU (Electronic Control Unit), a head unit, and the like mounted on a vehicle for driving the vehicle itself, and may transmit and receive data to and from the control unit 120 when necessary. As an example, when the vehicle door is opened, the control unit 120 notifies the main system 300 of the current state of the vehicle door so that it can be displayed on the vehicle instrument panel.

Meanwhile, the respective components of the vehicle door control system 100 and the main system 300 of the vehicle may be connected to each other through a predetermined bus. A bus is a communication path connecting an ECU (Electronic Control Unit), a node, and a module in the system. The bus includes an address bus, a data bus, a control bus, a field bus, And may be implemented in a network structure such as a star type, a mesh type, a ring type, a mesh type, and a tree type.

In addition, the bus can be implemented by a bus system such as a CAN (Controller Area Network), an RS485 bus, a Versa Module Eurocard (VME) bus, a Multibus, and various network structures of wired and wireless.

According to the above-described structure, the vehicle door control system according to the embodiment of the present invention can recognize and identify the object approaching the vehicle, and automatically open the vehicle door when the object satisfies a specific condition have.

Hereinafter, a structure of a vehicle control program installed in a vehicle door control system according to an embodiment of the present invention will be described with reference to the drawings.

3 is a diagram showing a structure of a program of a vehicle door control system according to an embodiment of the present invention. In the following description, each constituent part constituting the program can be embodied as one or more program functions, and can be recorded in a memory or a recording medium in a form executable by the processor.

In the embodiment of the present invention, OpenCV can be used as a known library for implementing image processing functions.

3, the control program 200 according to the embodiment of the present invention includes a preprocessor 210 for pre-processing the collected video signals, an object extractor 220 for extracting objects appearing in the preprocessed video, One or more discriminators 240 and 250 for discriminating the gender of the object based on the ROI region, and a weighting unit 230 for reflecting the predetermined weight in the discrimination result, A sex determination unit 260 for determining a sex, and a signal generating unit 270 for generating the control signal according to sex of a determined object.

The preprocessing unit 210 may perform predetermined preprocessing on the original image signal to improve the accuracy of the image analysis process. The preprocessing unit 210 calculates a gray value for each pixel of the video signal and emphasizes the edges of the objects appearing on the screen in accordance with the change of the gray value, .

In addition, we use a binarization technique such as the Otsu Algorithm to separate the objects from the background.

In addition, the preprocessing unit 210 converts the resolution of the video signal to reflect the distance between the vehicle and the subject measured by the infrared sensor installed in the vehicle door, And a function of setting an analysis target area.

As the distance between the vehicle and the subject gets closer, the size of the subject in the image signal becomes larger and the area to be analyzed is limited. Therefore, even if the resolution of the image is lowered to a certain level, there is almost no change in the accuracy of object identification. The smaller the area, the smaller the number of pixels of the image and the smaller the data size, thereby improving the image processing speed.

Accordingly, the preprocessing unit 210 can improve the processing speed by narrowly converting the resolution of the image and the region to be analyzed in the preprocessing process.

The object extracting unit 220 can process the background on the screen where the objects are separated and extract the object corresponding to the person. An object can be designated as an area having a certain size in an image, and the remaining area can be designated as a background even if it is actually a person or an object.

The background can be removed using the binarized image by the preprocessor, and the remaining objects can be divided into objects and people. Here, the object corresponding to a person includes a head, a body, an arm, and a leg that can be divided into outlines, and an object that is not distinguished is defined as a thing and removed, and only an object determined as a person is extracted.

The region setting unit 230 may set one or more regions in the extracted human object as ROI regions. The ROI region may be a plurality of ROI regions, and at least a ROI region may be included in a face. In addition, a separate ROI area can be set for an accessory, bag, shoes, etc. worn by a person.

The determining unit 240 or 250 may determine the presence or absence of a person from the set ROI region to determine the sex of the object.

In particular, the determination unit 240 or 250 according to the embodiment of the present invention may include a face discrimination unit 240 for discriminating the sex of an object based on an area corresponding to a face among ROI areas, And an auxiliary discriminating unit 250 for supplementing the accuracy of the discrimination of the gender of the object based on the discrimination result.

The face discrimination unit 240 recognizes that the average image of the male and female is different and that the facial features that help distinguish only the male face in facial recognition are different from the facial features that play a major role in distinguishing only the female face The face ROI region and the sample image collected in advance are compared with each other and the gender of the face of the sample image having a degree of similarity higher than a predetermined level is judged as the sex of the object .

The auxiliary discriminating unit 250 uses the fact that men and women are different from each other in possession or wearing. Such a commodity may be a briefcase, a tie, or a shoe in the case of a male, a handbag , Necklaces, and high heels. The auxiliary discriminating unit 250 may compare the sample ROI region with the sample ROI region to determine the gender of the object according to the similarity.

In order to realize the above function, the face discrimination unit 240 and the auxiliary discrimination unit 250 can calculate the similarity by calculating and comparing the histograms of the ROI region and the sample image to be compared, which is a function provided by OpenCV Can be implemented.

In the present invention, 'cv :: calcHist ()' can be used to store each pixel of an image in one or more arrays, and 'cv :: compareHist ()' to compare the histograms of two images, Can be used. Here, the codes for comparing codes of two images using 'cv :: compareHist ()' are shown in Table 1.

double cv: compareHist (InputArray Img1, InputArray Img2, int method)

In the above code, the return value of the function is declared as a double type, and a method in which the accuracy of the result is high can be applied as the method is selectively set. Various methods can be applied to these methods based on OpenCv2.4.

Preferably, as a correlation method, a method of calculating a difference between each grade value and a histogram average for a histogram of two images is used to calculate a comparison value between two images in a vertical direction of the cross correlation function, and the similarity is returned. As the return value of the correlation method approaches 1, the two images have a high degree of similarity. As the closer to -1, the two images have a low degree of similarity. It is determined by the corresponding sex.

The gender determining unit 260 can determine the gender of the object identified by the determining unit 240 or 250. The discriminators 240 and 250 discriminate the gender of the object according to the degree of similarity between the image and the sample, and the gender determining unit 260 determines the gender of the object based on the result of the discrimination.

Particularly, in the embodiment of the present invention, the gender of the object can be determined by dividing the face region and the other product region with respect to one object, and the accuracy of gender determination can be improved by setting a weight on each determination result.

For this, the control program 200 according to the embodiment of the present invention may further include a weight setting unit 280 for setting weights differently for the first ROI region and the second ROI region.

That is, when the weight of the result is relatively high (60% or more) and the weight of the result is set low (40% or less) in the face area with relatively high accuracy, The sex can be determined.

Or when the judgment results of the two regions are different, the sex having the high weight can be reliably determined as it is.

The signal generator 270 may generate a control signal according to the sex of the object determined by the sex determiner 260. [ Particularly, according to the present invention, the object to which the vehicle door is opened may be a woman, and the signal generating unit 270 may be configured to maintain the lock state of the vehicle door when the object is judged to be male, It is possible to generate a control signal corresponding to the control signal. Also, the signal generator 270 may generate a control signal corresponding to an instruction to open the vehicle door when the object is determined to be female.

In addition, when a parked vehicle is detected through the infrared sensor installed in the vehicle, the signal generator 270 measures the distance to the vehicle and adjusts the opening width of the vehicle door to generate a control signal, It is possible to prevent a collision with a neighboring vehicle when the door is opened.

The control program 200 according to the embodiment of the present invention may further include a learning unit 290 that provides learning results using a Harr-based cascade classifier for a plurality of collected samples to the face discrimination unit 240 ).

The cv :: Algorithm :: create () function is used to perform a machine learning using a sample, which is an image for a plurality of collected learning operations, and provide the result to the face discrimination unit 240. [ 'And' cv :: FaceRecognizer :: train () '. Table 2 shows the code for implementing machine learning using this function.

void FaceRecognizer :: train (InputArrayOfArrays sample, InputArray labels

Here, the sample parameter is a sample image declared in a Mat structure, and the parameters parameter is a recognition value for a face. For learning, we create an instance through the 'cv :: Algorithm :: create ()' function, and perform the learning through the above code.

According to the above-described structure, the control program of the present invention identifies an object appearing in an image and generates a control signal for opening the vehicle door as a subject when the identified object is female.

Hereinafter, a vehicle door control method by a vehicle door control system according to an embodiment of the present invention will be described with reference to the drawings.

4 is a view illustrating a vehicle door control method according to an embodiment of the present invention. Although there is no description in the following description, the execution subject for each step is the above-described vehicle door control system and each component included therein.

Referring to FIG. 4, the method for controlling a vehicle door according to the present invention includes the steps of capturing a video signal in one direction in real time in a vehicle door (S100), analyzing the video signal, (S110) of generating a control signal for controlling the opening or closing of the vehicle door, and opening the vehicle door in response to the control signal (S120), in the case of a matching subject.

The step S100 of photographing the vehicle door in one direction in real time to acquire a video signal is performed by the camera module installed in the vehicle or the door of the vehicle photographing the surroundings of the vehicle in real time, And acquiring the video signal in real time.

Next, a step S110 of generating a control signal for controlling the opening or closing of the vehicle door in the case that the object displayed in the image satisfies a predetermined condition by analyzing the video signal, As the program is executed, the image signal is analyzed using an image processing technique and a control signal is generated.

More specifically, in operation S110, the preprocessing unit of the control program preprocesses the video signal, extracts an object appearing in the preprocessed image of the object extracting unit, and sets one or more ROI regions Determining a gender of the object based on the ROI region, determining a gender of the object by reflecting a predetermined weight in a result of the gender determining unit, And generating the control signal according to the control signal.

Particularly, according to an embodiment of the present invention, the step of setting the one or more ROI regions includes a first ROI region corresponding to a face of the subject and a second ROI region corresponding to one or more objects worn or held by the subject The first ROI region and the second ROI region may be set to have different weights, respectively, so that the accuracy of the discrimination result can be improved by using the weights.

And opening the vehicle door in response to the control signal (S120). In particular, when the subject is judged to be a female, the control signal corresponds to an open command, and the door opening / closing module installed on the vehicle door opens the corresponding vehicle door according to the control signal.

Hereinafter, a technical configuration of the vehicle door control system of the present invention will be described with reference to an example of an image according to an image processing process.

5A to 5C are views showing examples of images derived in the image processing process of the control door control system of the present invention.

Referring to FIG. 5A, when a plurality of people and objects appear in an image signal obtained by a camera module, a first object o1 corresponding to one person and a second object o1 corresponding to an object A second object o2 and a third object o3 may be designated. The remaining part can be designated as background (bg).

Next, referring to FIG. 5B, the background bg is removed by the dark region processing by the object extraction unit in the image, and the object o1 corresponding to the person is left as the object determined as the object is also removed. Here, the control program designates the face portion of the object as a human ROI region ROI1, and determines the gender of the object through the face recognition process.

Here, if the object is determined to be female, the control program generates a control signal including an open command and transmits the control signal to the open / close module so as to open the vehicle door.

5B, in order to improve the accuracy of the gender discrimination of the object, in addition to the first ROI region ROI1 corresponding to the face, at least one second ROI region ROI1 corresponding to a specific object worn or held by the object ROI2, ROI3) may be further specified and used for gender discrimination separately from the first ROI region ROI1. Thereafter, weights are set differently for each ROI area, or the results are used to identify more accurate object gender.

While a great many are described in the foregoing description, it should be construed as an example of preferred embodiments rather than limiting the scope of the invention. Accordingly, the invention is not to be determined by the embodiments described, but should be determined by equivalents to the claims and the appended claims.

100: vehicle door control system 110: camera module
120: control unit 121:
121: memory 130: door opening / closing module
140: communication unit 150: sensor
200: control program 210: preprocessing section
220: object extracting unit 230: area setting unit
240: face discrimination unit 250: auxiliary discrimination unit
260: sex determination unit 270: signal generation unit
280: weight setting unit 290:
300: Main system

Claims (11)

A camera module installed in any one of the vehicle doors and capturing a video signal in one direction in real time;
And a controller for analyzing the video signal and generating a control signal for controlling the opening and closing of the vehicle door when the object appeared in the video meets predetermined conditions, ; And
And a door opening / closing module for opening the vehicle door in response to the control signal,
The program includes:
A preprocessor for preprocessing the video signal;
An object extraction unit for extracting an object appearing in a preprocessed image;
A region setting unit for setting a plurality of ROI regions in the extracted object and setting a first ROI region corresponding to a face of the subject and a second ROI region corresponding to one or more articles worn or held by the subject;
At least one discriminator for discriminating the gender of the object based on the ROI region;
A weight setting unit that sets weights for the first ROI region and the second ROI region differently from each other and sets a weight for the first ROI region to be higher than a weight for the second ROI region;
A gender determining unit for determining a gender of the object by reflecting a predetermined weight in a result of the determination; And
And a signal generator for generating the control signal according to the gender of the determined object. delete delete The method according to claim 1,
Wherein,
And a face discrimination unit for recognizing a face, which is a characteristic of a person, as the first ROI area,
The program includes:
A learning unit for providing learning results using a Harr-based cascade classifier for a plurality of collected samples to the face discrimination unit;
The vehicle door control system further comprising: 5. The method of claim 4,
A communication unit connected to an external system to collect the plurality of samples and provide the collected samples to the learning unit;
The vehicle door control system further comprising: The method according to claim 1,
An infrared sensor for measuring a first distance between the vehicle approaching the vehicle door and the vehicle or a second distance from the vehicle parked adjacent to the vehicle;
The vehicle door control system comprising: The method according to claim 6,
The pre-
And converting the resolution of the video signal according to the first separation distance to set an analysis target area. The method according to claim 6,
Wherein the signal generator comprises:
And adjusts the opening width of the vehicle door according to the second distance. A vehicle door control method using the system according to claim 1,
Capturing a video signal by photographing one direction from a vehicle door in real time;
Analyzing the image signal to generate a control signal for controlling opening or closing of the vehicle door when the object displayed in the image satisfies predetermined conditions; And
Opening the vehicle door in response to the control signal,
Wherein the step of generating the control signal comprises:
Pre-processing the image signal;
Extracting an object appearing in the preprocessed image;
Setting a plurality of ROI regions in the extracted object as a first ROI region corresponding to a face of the subject and a second ROI region corresponding to one or more articles worn or held by the subject;
Determining a gender of the object based on the ROI region;
Determining a gender of the object by reflecting a predetermined weight in a result of the determination; And
And generating the control signal according to the gender of the determined object,
Wherein the setting of the one or more ROI regions comprises:
A first ROI region corresponding to a face of the subject and a second ROI region corresponding to one or more objects worn or held by the subject,
The first ROI region and the second ROI region may include:
Wherein the weights for the first ROI region are set to be higher than the weights for the second ROI region. delete delete

Images