KR102443980B1 - Vehicle control method - Google Patents

Abstract

The present invention relates to a vehicle control method.
According to an embodiment of the present invention, there is provided a vehicle control method of a vehicle control apparatus mounted on a vehicle equipped with a sensor, the method comprising: receiving sensing information from the sensor sensing the interior of the vehicle; analyzing the occupants of the vehicle by using the sensing information; searching for whether the passenger performs at least one of preset gestures defined as driving; and when at least one of the gestures is detected, determining the occupant who performed the detected gesture as a driver.

Description

Vehicle control method {VEHICLE CONTROL METHOD}

The present invention relates to a vehicle control method of a vehicle control device mounted in a vehicle.

A vehicle refers to a means of transportation capable of moving a person or a load by using kinetic energy. Representative examples of vehicles include automobiles and motorcycles.

For the safety and convenience of users using the vehicle, various sensors and devices are provided in the vehicle, and the functions of the vehicle are being diversified.

The function of the vehicle may be divided into a convenience function for promoting the convenience of the driver and a safety function for promoting the safety of the driver and/or pedestrian.

First, convenience functions have a motive for development related to driver convenience, such as giving infotainment (information + entertainment) functions to vehicles, supporting partial autonomous driving functions, or helping drivers to secure their vision, such as night vision or blind spots. . For example, adaptive cruise control (ACC), smart parking assist system (SPAS), night vision (NV), head up display (HUD), around view monitor (around view monitor, AVM) and adaptive headlight system (AHS) functions.

The safety function is a technology that secures driver safety and/or pedestrian safety. Lane departure warning system (LDWS), lane keeping assist system (LKAS), automatic emergency braking (autonomous emergency) braking, AEB) functions, etc.

Most of the current traffic accidents are caused by the driver's negligence, and although autonomous driving has been developed to compensate for the driver's carelessness, there are still frequent cases in which the driver cannot concentrate on driving. Therefore, it is necessary to develop a vehicle control method that allows the driver to concentrate again by giving a notification when the driver is careless, such as when the driver is drowsy while driving.

An object of the present invention is to provide a vehicle control method. SUMMARY OF THE INVENTION An object of the present invention is to provide a charang control method that provides an alarm when a driver fails to concentrate on driving, such as drowsy driving, so that the driver can concentrate again.

Another problem to be solved by the present invention is to provide a charang control method for determining a driver among occupants and analyzing whether or not the driver is focused, when there are a plurality of occupants in a vehicle.

Receiving sensing information from the sensor for sensing the interior of the vehicle of a vehicle control device mounted on a vehicle equipped with a sensor according to an example of the present invention; analyzing the occupants of the vehicle by using the sensing information; searching for whether the passenger performs at least one of preset gestures defined as driving; and when at least one of the gestures is detected, determining the occupant who performed the detected gesture as a driver.

In one embodiment of the present invention, the sensor may include a camera, and the sensing information may be image information measured by the camera.

In one embodiment of the present invention, the camera may be capable of taking 360 °.

In one embodiment of the present invention, the location of the driver may be analyzed and compared with traffic rules of a region in which the vehicle is driving.

In one embodiment of the present invention, when the location of the driver is different from the local traffic rules, an alarm may be provided to the driver.

In an embodiment of the present invention, the method may further include tracking the driver's face by the sensor.

In an embodiment of the present invention, the tracking may be performed by a pre-trained CNN model.

In one embodiment of the present invention, the tracking of the driver's face may include tracking at least one of an eye movement and a mouth movement of the driver.

In one embodiment of the present invention, tracking the eye movement of the driver may analyze the degree of blinking of the eyes and compare the degree of blinking with a preset reference.

In one embodiment of the present invention, the driver's state may be determined as any one of a predetermined state based on the comparison result.

In one embodiment of the present invention, tracking the movement of the driver's mouth may analyze the degree of mouth gaping and compare the degree of mouth gaping with a preset standard.

In one embodiment of the present invention, the driver's state may be determined as any one of a predetermined state based on the comparison result.

An object of the present invention is to solve the following problems.

SUMMARY OF THE INVENTION One object of the present invention is to provide a vehicle control method for determining the location of a driver so that the driver can be analyzed for safe driving of the vehicle.

Another object of the present invention is to provide a vehicle control method for concentrating on driving by giving an alarm to a driver who is inattentive in driving because he is concentrating on another action.

1 is a view showing the exterior of a vehicle according to an embodiment of the present invention.
2 is a view showing the interior of a vehicle according to an embodiment of the present invention.
3 is an exemplary view for explaining a vehicle control method according to an embodiment of the present invention.
4 is a flowchart illustrating a vehicle control method according to an embodiment of the present invention.
5 is a view for explaining a specific example of a vehicle control method according to an embodiment of the present invention.
6 is a view for explaining a specific example of a vehicle control method according to an embodiment of the present invention.
7 is a view for explaining a specific example of a vehicle control method according to an embodiment of the present invention.
8 is a view for explaining a specific example of a vehicle control method according to an embodiment of the present invention.
9 is a view for explaining a specific example of a vehicle control method according to an embodiment of the present invention.

Hereinafter, the embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "part" for components used in the following description are given or mixed in consideration of only the ease of writing the specification, and do not have distinct meanings or roles by themselves. In addition, in describing the embodiments disclosed in the present specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical idea disclosed herein is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present invention , should be understood to include equivalents or substitutes.

Terms including ordinal numbers such as first, second, etc. may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but it is understood that other components may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The singular expression includes the plural expression unless the context clearly dictates otherwise.

In the present application, terms such as “comprises” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that this does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

In the present specification, the communication method of the network is not limited, and the connection between each component may not be connected by the same network method. The network may include not only a communication method using a communication network (eg, a mobile communication network, a wired Internet, a wireless Internet, a broadcasting network, a satellite network, etc.) but also short-range wireless communication between devices. For example, the network may include all communication methods through which an object and an object can network, and is not limited to wired communication, wireless communication, 3G, 4G, 5G, or other methods. For example, a wired and/or network may be a Local Area Network (LAN), a Metropolitan Area Network (MAN), a Global System for Mobile Network (GSM), an Enhanced Data GSM Environment (EDGE), a High Speed Downlink Packet Access (HSDPA), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Zigbee, Wi-Fi, Voice over VoIP Internet Protocol), LTE Advanced, IEEE802.16m, Wireless MAN-Advanced, HSPA+, 3GPP Long Term Evolution (LTE), Mobile WiMAX (IEEE 802.16e), UMB (formerly EV-DO Rev. C), Flash-OFDM, iBurst and MBWA (IEEE 802.20) systems, HIPERMAN, Beam-Division Multiple Access (BDMA), Wi-MAX (World Interoperability for Microwave Access), and IR-based communication to refer to a communication network by one or more communication methods selected from the group consisting of However, the present invention is not limited thereto.

The vehicle described in this specification may be a concept including an automobile and a motorcycle. Hereinafter, the vehicle will be mainly described with respect to the vehicle.

The vehicle described herein may be a concept including both an internal combustion engine vehicle having an engine as a power source, a hybrid vehicle having an engine and an electric motor as a power source, and an electric vehicle having an electric motor as a power source.

In the following description, the left side of the vehicle means the left side of the driving direction of the vehicle, and the right side of the vehicle means the right side of the driving direction of the vehicle.

Hereinafter, a vehicle control method according to the present invention will be described with reference to the accompanying drawings.

1 is a view showing the exterior of a vehicle according to an embodiment of the present invention.

2 is a view showing the interior of a vehicle according to an embodiment of the present invention.

1 to 2 , the vehicle 100 may include wheels rotated by a power source and a steering input device 120 for controlling the traveling direction of the vehicle 100 .

The vehicle 100 may be an autonomous driving vehicle.

Here, autonomous driving is defined as controlling at least one of acceleration, deceleration, and driving direction based on a preset algorithm. In other words, even if a user input is not input to the driving manipulation device, the driving manipulation device is automatically operated.

The steering input device 120 may receive a driving direction input of the vehicle 100 from the user. The steering input device 120 may be formed in a wheel shape to enable steering input by rotation. According to an embodiment, the steering input device may be formed in the form of a touch screen, a touch pad, or a button.

The vehicle 100 may include a sensor.

The sensor is a device for detecting an object located inside or outside the vehicle 100 .

The sensor may include a camera 110 , a radar, a lidar, an ultrasonic sensor, an infrared sensor, and the like.

Depending on the embodiment, the sensor may further include other components in addition to the described components, or may not include some of the described components.

The camera 110 may be located at an appropriate place inside or outside the vehicle in order to acquire an image inside or outside the vehicle. The camera 110 may be a mono camera, a stereo camera, an AVM (Around View Monitoring) camera, an internal camera, or a rotating camera capable of capturing 360 degrees.

For example, the camera 110 may be disposed adjacent to the front windshield in the interior of the vehicle in order to acquire both an external image and an internal image of the front of the vehicle. Alternatively, the camera 110 may be disposed around the front bumper or the like.

The internal camera may acquire an image inside the vehicle. The vehicle control device may detect the user's state based on the image inside the vehicle. The vehicle control apparatus may obtain the user's gaze information from the image inside the vehicle. The vehicle control device may detect the user's gesture from the image inside the vehicle.

The vehicle 100 may include an output device.

The output device is for generating an output related to visual, auditory or tactile sense. In an embodiment, the output device may include a display unit capable of displaying graphic objects corresponding to various pieces of information.

The vehicle control device may output information such as an alarm through the display unit.

The display unit includes a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display, 3 It may include at least one of a 3D display and an e-ink display.

The display unit is implemented in one region of the steering wheel, one region 240 of the panel, one region 210 of the vehicle front windshield, one region of the seat, one region of the console 220, one region 230 of the window, and the like. can be

3 is an exemplary view for explaining a vehicle control method according to an embodiment of the present invention.

Referring to FIG. 3 , several occupants may be boarded in the vehicle ( 310 ). In general, one of several occupants is a driver, and all occupants other than the driver are general occupants. In order to carry out the object of the present invention, it is necessary to distinguish a driver from among vehicle occupants. An image inside the vehicle may be acquired through the above-described sensor, and a user's state may be detected based on the image inside the vehicle.

A detailed method of determining a driver will be described with reference to FIG. 4 .

4 is a flowchart illustrating a vehicle control method according to an embodiment of the present invention.

A vehicle control device mounted on a vehicle equipped with a sensor may receive sensing information from a sensor that senses the interior of the vehicle ( S410 ). In an embodiment, the sensor may be an optical camera and the sensing information may be image information.

The vehicle control apparatus may analyze ( S420 ) the occupants of the vehicle by using the sensing information. In one embodiment, as described above, when the sensing information is image information, analyzing the occupant of the vehicle may be analyzing the position of the occupant inside the vehicle. Specifically, the camera may detect a person riding in the vehicle through image sensing.

Thereafter, it may be searched whether the passenger performs at least one of preset gestures defined as driving ( S430 ). The preset gesture may be formed in various forms such as action, name, and gaze, and the gesture may be added or subdivided by learning. As an embodiment, referring to FIG. 3 , a preset gesture defined as driving may appear as an action of controlling the steering input device ( 320 ). Alternatively, in another embodiment, it may appear in a manner of touching the steering system of the vehicle. As another example, it may appear that the gaze of the occupant continues to face the front part of the vehicle. The present invention is not limited thereto, and various preset gestures may exist, and the vehicle control device may search whether the passenger performs a combination of preset gestures.

When at least one of the gestures is detected, a driver who performed the searched gesture may be determined (S440). Generally, there will be one driver among the occupants inside the vehicle. However, when a plurality of passengers perform at least one of the preset gestures defined as driving, the vehicle control device may determine the passenger who continuously performs the gesture as the driver. However, the present invention is not limited thereto, and one of the occupants in the vehicle may be determined as the driver.

5 is a view for explaining a specific example of a vehicle control method according to an embodiment of the present invention.

Referring to FIG. 5 , the vehicle control device may analyze the driver's location and compare it with the traffic rules of the area in which the vehicle is driving ( S510 ).

In general, the position of the driver's seat in a vehicle varies from country to country. In a normal vehicle, a driver's seat and a passenger seat are located in the front of the vehicle. Based on the direction from the inside of the vehicle to the front, in countries such as Korea and the United States, the driver's seat is on the left. On the other hand, in countries such as the UK and Japan, the driver's seat is on the right side. This depends on the traffic rules of each country, and the sensor cannot determine the driver's location in a batch after analyzing the inside of the vehicle. Also, in the case of Korea, a vehicle with a driver's seat on the right may not be illegal depending on circumstances, such as a vehicle imported from a country with a driver's seat on the right, such as Japan.

Therefore, after the vehicle control device determines the driver among the occupants of the vehicle ( S440 ), it is necessary to analyze the driver's position.

Specifically, since the position of the occupant is analyzed before the step of determining the driver, the vehicle control device can specify the position of the driver. Thereafter, the vehicle control device may compare whether or not the driver's location matches the traffic rules by comparing it with the traffic rules of the region in which the corresponding vehicle is traveling.

Thereafter, when the location of the driver is different from the traffic rules, an alarm may be provided to the driver ( S520 ). In one embodiment, the alarm may be provided to the driver through the above-described display unit.

When the driver's position is different from the traffic rules, in one embodiment, when a vehicle with the driver's seat on the right is running in Korea where the driver's seat is on the left according to the traffic rules, the vehicle control device determines that the driving vehicle does not follow the traffic rules can tell you Since all traffic rules are set according to the location of the driver's seat in traffic rules, if the driver's seat of a vehicle in motion does not follow the traffic rules, the risk of an accident may increase. Therefore, the risk of an accident can be prevented by allowing the driver to take action through the alarm.

6 is a view for explaining a specific example of a vehicle control method according to an embodiment of the present invention.

Referring to FIG. 6 , when a driver in a vehicle is determined through the above-described embodiments, the driver may be analyzed. Specifically, analyzing the driver may be a sensor tracking the driver's face. In one embodiment, tracking the driver's face may be tracked by a pre-trained Convolutional Neural Network (CNN) model after acquiring image information through a camera.

Tracking the driver's face may include tracking at least one of the driver's eye movement 610 and mouth movement 620 . However, the present invention is not limited thereto, and it is possible to track the driver's face through the movement of the nose. Specifically, when the driver's nose moves left/right or up/down through the image information acquired by the camera, the vehicle control device detects that the driver eats food, talks, uses a mobile phone, or feels drowsy while driving. It can be judged to be in a state of distraction, such as More specifically, when it is determined that the driver is in the distracted state as described above for 1 second or more, the vehicle control device may provide an alarm to the driver so that the driver can concentrate on driving again.

7 is a view for explaining a specific example of a vehicle control method according to an embodiment of the present invention.

In another embodiment, the tracking of the driver's face may be tracking the movement of the driver's eyes.

Referring to FIG. 7 , the vehicle control device may analyze the degree of blinking of the driver's eyes ( S710 ). Also, the degree of eye blinking may be compared with a preset reference (S720). Also, based on the comparison result, the driver's state may be determined as one of predetermined states ( S730 ). Also, when it is determined that the driver's state is not concentrating on driving, an alarm may be provided to the driver ( S740 ).

Specifically, the degree of eye blinking may be analyzed as the number of blinks at a preset time. In addition, whether the eyes blink may be determined by analyzing the size of the eyes.

<Figure 1>

More specifically, according to <Figure 1>, the distance between A-B can be defined as D1, the distance between C-D as D2, and the distance between E-F as D3. In this case, the eye aspect ratio may be defined as (D1+D2)/(2*D3). The vehicle control device may determine whether the eye blinks by comparing the eye aspect ratio with a threshold value. For example, when the threshold value is 0.2, when the aspect ratio of the eye is less than 0.2, the blinking state may be determined, and when the eye aspect ratio is 0.2 or more, the blinking state may be determined.

The vehicle control device may analyze the degree of blinking as the number of blinks at a preset time. As an embodiment, if the eye blinks 25 times or more in 1,500 milliseconds, the vehicle control device may determine that the driver feels drowsy. Based on the comparison result, the vehicle control device may determine the driver's state as any one of the predetermined states. You can focus on driving.

8 is a view for explaining a specific example of a vehicle control method according to an embodiment of the present invention.

Referring to FIG. 8 , the vehicle control apparatus may analyze the degree of opening of the driver's mouth ( S810 ). Also, the degree of opening of the mouth may be compared with a preset standard ( S820 ). Also, based on the comparison result, the driver's state may be determined as any one of predetermined states ( S830 ). Also, when it is determined that the driver's state is not concentrating on driving, an alarm may be provided to the driver ( S840 ).

Specifically, the degree of opening of the mouth may be analyzed by the number of times the eyes are opened at a preset time. In addition, whether to open the mouth may be determined by analyzing the size of the mouth.

<Figure 2>

More specifically, according to <Figure 2>, the distance between A-E can be defined as D1, the distance between B-D as D2, and the distance between C-F as D3. In this case, the mouth aspect ratio may be defined as (D1+D2)/(2*D3). The vehicle control device may determine whether the mouth is open by comparing the aspect ratio of the mouth to a threshold value. For example, when the threshold value is 0.6, when the aspect ratio of the mouth is 0.6 or less, the mouth may be determined as an open state, and when the aspect ratio of the mouth is greater than 0.6, the mouth may be determined as an open state.

The vehicle control device may analyze the degree of mouth opening as the number of mouth openings at a preset time. As an embodiment, if the mouth is opened more than 55 times in 4 seconds, the vehicle control device may determine that the driver is yawning and feels drowsy. Based on the comparison result, the vehicle control device may determine the driver's state as any one of the predetermined states. You can focus on driving.

9 is a view for explaining a specific example of a vehicle control method according to an embodiment of the present invention.

Referring to FIG. 9 , before determining the location of the driver by the above-described method, the location of the driver may be determined in advance based on information on a region in which the vehicle is driven. Specifically, the vehicle control device may receive a database including driver seat information for each region. Alternatively, the database may be included in the vehicle control device. As an embodiment of the database, driver seat information for each country may be included. As described above, in countries such as Korea and the United States, the driver's seat is located on the left side of the vehicle in a direction facing the front from the inside. On the other hand, in countries such as the UK and Japan, the driver's seat is on the right side. In this case, regional information of Korea and driver's seat information indicating that the driver's seat is on the left may be matched to exist in the database.

After the vehicle control device receives data from the database, it is possible to determine an area in which the vehicle is located, and determine driver's seat information matched with the area in which the vehicle is located in the database. Through this method, it is possible to automatically determine the occupant in the seat according to driver seat information corresponding to the region where the vehicle is located in the database as the driver.

The present invention described above can be implemented as computer-readable code (or application or software) on a medium in which a program is recorded. The above-described method for controlling the autonomous vehicle may be realized by a code stored in a memory or the like.

The computer-readable medium includes all kinds of recording devices in which data readable by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. There is also a carrier wave (eg, transmission over the Internet) that includes implementation in the form of. In addition, the computer may include a processor or a control unit. Accordingly, the above detailed description should not be construed as restrictive in all respects but as exemplary. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are included in the scope of the present invention.

The technical features disclosed in each embodiment of the present invention are not limited to the embodiment, and unless they are mutually incompatible, the technical features disclosed in each embodiment may be combined and applied to different embodiments.

In the above, embodiments of the present invention have been described. The present invention is not limited to the above-described embodiments and the accompanying drawings, and various modifications and variations will be possible from the point of view of those of ordinary skill in the art to which the present invention pertains. Accordingly, the scope of the present invention should be defined not only by the claims of the present specification, but also by those claims and their equivalents.

100: vehicle
110: camera
120: steering input device
210: front windshield
220: console
230: window
240: panel

Claims (13)

A vehicle control method of a vehicle control device mounted on a vehicle in which a sensor including a camera capable of 360° shooting is disposed close to a front windshield inside the vehicle,
receiving sensing information including image information measured by the camera from the sensor sensing the interior of the vehicle;
analyzing the occupants of the vehicle by using the sensing information;
Detecting whether the occupant performs at least one of preset gestures defined as driving, wherein the preset gesture includes an action of adjusting a steering input device, an action of touching the steering system, and the occupant's gaze continuously Including forward-facing actions, etc. -; and
when at least one of the gestures is detected, determining a passenger who performed the detected gesture as a driver;
Analyze the location of the driver and provide an alarm to the driver when the analyzed location of the driver is different from the location of the driver according to the traffic rules of the area compared with the traffic rules of the area in which the vehicle is driving - in the area According to traffic rules, the driver's position is related to whether the driver's seat is on the left or right side -
How to control the vehicle. delete delete delete delete The method of claim 1,
The method of controlling a vehicle further comprising the step of the sensor tracking the driver's face. 7. The method of claim 6,
The tracking step is a vehicle control method made by a pre-trained CNN model. 7. The method of claim 6,
The vehicle control method, wherein the tracking of the driver's face is tracking at least one of eye movement and mouth movement of the driver. 9. The method of claim 8,
The tracking of the eye movement of the driver is a vehicle control method of analyzing the degree of blinking of the eyes and comparing the degree of blinking with a preset reference. 10. The method of claim 9,
A vehicle control method for determining the driver's state as one of predetermined states based on the comparison result. 9. The method of claim 8,
The tracking of the movement of the driver's mouth includes analyzing a degree of mouth opening and comparing the degree of mouth opening with a preset standard. 12. The method of claim 11,
A vehicle control method for determining the driver's state as one of predetermined states based on the comparison result. The method of claim 1,
receiving, by the vehicle control device, data from a database including driver seat information for each region; and
Further comprising the step of determining an area in which the vehicle is located,
analyzing the occupants of the vehicle.
Automatically determining a occupant in a seat according to driver seat information corresponding to an area where the vehicle is located in the database as a driver
How to control the vehicle.

Images