KR20190143074A - Apparatus and method for monitoring vehicle operation - Google Patents

Abstract

Disclosed are an apparatus and a method for monitoring vehicle driving, capable of monitoring lane departure deviation, forward collision warning, drowsy driving warning, and the like by monitoring a driving state and a driver state of a driving vehicle. The disclosed apparatus includes: a front photographing unit for photographing the front of a driving vehicle; a driver photographing unit which photographs a driver′s face; an image processing unit which outputs information for determining whether the vehicle is out of lane, whether a head collision occurs, and whether the driver is sleepy by processing the image data from the front photographing unit and the driver photographing unit; a control unit which determines whether the vehicle is out of lane, whether a head collision occurs, and whether the driver is sleepy based on the information from the image processing unit, and controls an alarm operation according to the determination result; and an alarm unit for generating an alarm under control of the control unit.

Description

Apparatus and method for monitoring vehicle operation

The present invention relates to a vehicle driving monitoring apparatus and method, and more particularly, to a vehicle driving monitoring apparatus and method that can alert by monitoring the driving state and the driver's state of a running vehicle.

Recently, as traffic accidents caused by drunk driving, drowsy driving, using a mobile phone while driving, and the like, various methods for preventing these accidents have been sought.

In particular, traffic accidents caused by drowsy driving are not only the types of accidents with the highest accident rate, but also most of the situations where the driver is not aware of the accidents at the time of the accident, and thus the probability of a large accident is higher.

Various techniques for preventing such drowsiness driving have been proposed, and many of the proposed techniques directly detect a driver's living state and warn the driver according to the detection result.

However, the method of detecting the driver's living state has a problem in that an expensive high-precision device is provided inside or outside the vehicle, or requires direct or indirect contact with the driver's body, thereby reducing the driver's driving convenience.

Meanwhile, Intelligent Transportation Systems (ITS) are rapidly being applied to roads across the country, enabling many drivers to check vehicle politics and delays on each road in real time. ITS acquires information about vehicles on the road through the Vehicle Detection System (VDS), and uses the acquired information by using Dedicated Short Range Communication (DSRC). Send it to the OBU, allowing the driver to recognize the road situation.

However, although the intelligent traffic system has been applied to the road, traffic accidents caused by overspeed and drowsy driving still occur frequently on the highway, which is one of the highest accident types among accidents. This is because, until now, there is no way to effectively utilize the intelligent transportation system to prevent drowsy driving.

Drowsiness or careless driving is a dangerous act that is more likely to cause serious accidents than drunk driving. Accordingly, it is necessary to develop a response system that comprehensively judges the driver's condition and performs more active and immediate actions beyond simply recommending rest according to the judgment result.

Prior Art 1: Republic of Korea Patent No. 10-1694785 (Drowsy driving prevention system) Prior Art 2: Korean Patent Publication No. 10-2017-0002166 (vehicle driving assistance method and vehicle) Prior Art 3: Republic of Korea Patent Publication No. 10-2016-0011989 (lane departure warning system) Prior Art 4: Republic of Korea Patent Publication No. 10-2018-0042742 (driver monitoring apparatus and method)

The present invention has been proposed to solve the above-mentioned conventional problems, the vehicle driving monitoring apparatus that can perform a lane departure warning, forward collision warning, drowsiness driving warning and the like by monitoring the driving state and the driver state of the driving vehicle and the like; The purpose is to provide a method.

Another object of the present invention is to provide a vehicle driving monitoring apparatus and method for transmitting a driver's state, driving information, vehicle information, and the like of a driving vehicle to an external management system.

In order to achieve the above object, a vehicle driving monitoring apparatus according to a preferred embodiment of the present invention, the front photographing unit for photographing the front of the driving vehicle; A driver photographing unit which photographs a face of a driver of the vehicle; An image processor configured to process the image data from the front photographing unit and the driver photographing unit to output information for determining whether the vehicle is out of lane, whether it is a front collision, or whether the driver is drowsy; A controller which determines whether the vehicle is out of lane, whether it is a front collision, whether the driver is drowsy, and controls an alarm operation based on the determination result based on the information from the image processor; And an alarm unit for generating an alarm under the control of the controller.

And a storage unit for storing image data from the front photographing unit and the driver photographing unit, wherein the image processing unit processes the image data stored in the storage unit to determine lane departure, front collision, and the like. Information for determining whether or not sleepiness may be output.

The controller detects the current lane in which the driver is traveling based on the lane detection image and the edge extraction image in the image processor based on the input image from the front photographing unit, detects the front vehicle, and detects the lane. Thereafter, the lane intersection point is detected based on the lane intersection detection image from the image processor, and after detecting the lane intersection, the optimum lane is recognized based on the optimal lane recognition image from the image processor, and the front vehicle After the detection, the light and the bumper of the front vehicle are recognized based on the light and the bumper recognition image of the front vehicle from the image processor, and whether the lane departure is expected based on the light and the bumper of the optimal lane and the front vehicle. Determine if forward collisions are expected.

The controller detects a face region of the driver based on a face detection image in the image processor based on an input image from the driver photographing unit, and detects the head angle measurement image from the image processing unit after detecting the face region. The driver measures the head angle of the driver based on the driver's head angle, detects the eyes in the face region, classifies the driver as closed eyes and open eyes, and determines whether the driver is sleepy based on the measured driver's head angle and the classified closed eyes and open eyes. You can judge.

The display unit may further display a warning message related to the lane departure, the front collision, and drowsiness, and the display unit may operate together with the alarm unit.

The controller may receive vehicle information from at least one sensor installed in the vehicle, and transmit the determination result and the vehicle information to an external driving information management system.

On the other hand, the vehicle driving monitoring method according to a preferred embodiment of the present invention, the photographing unit, the front of the driving vehicle and the face of the driver of the vehicle; Image processing by the image processor to process the image data from the photographing unit to output information for judging whether the lane is out of front, whether there is a collision, or whether the driver is drowsy; Determining, by the controller, whether the vehicle is out of lane, whether it is a front collision, or whether the driver is sleepy based on the information from the image processor; And controlling, by the controller, an alarm operation according to the result of the determining.

According to the present invention having such a configuration, it is possible to monitor the driving information in real time by fusing the image of the front-facing camera, the image of the driver-taking camera, the vehicle information, and the sensor information.

In addition, based on the image of the front camera, the image of the driver camera, vehicle information, and sensor information, it not only alerts the driver of lane departures, forward collisions, driver drowsiness, etc. can do.

1 is a conceptual diagram of a vehicle driving monitoring apparatus according to an embodiment of the present invention.
2 is a block diagram illustrating a vehicle driving monitoring apparatus according to an exemplary embodiment of the present invention.
3 is a diagram illustrating a software architecture of the controller illustrated in FIG. 2.
4 is a configuration diagram of the driving information management system shown in FIG. 2.
FIG. 5 is a diagram for explaining a lane departure warning and a forward collision warning process according to an exemplary embodiment of the present invention.
6 is a view for explaining a process of detecting a driver state in an embodiment of the present invention.
7 is a flowchart illustrating a vehicle driving monitoring method according to an embodiment of the present invention.

As the inventive concept allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description.

However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art, and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

Hereinafter, with reference to the accompanying drawings, it will be described in detail a preferred embodiment of the present invention. In the following description of the present invention, the same reference numerals are used for the same elements in the drawings and redundant descriptions of the same elements will be omitted.

1 is a conceptual diagram of a vehicle driving monitoring apparatus according to an embodiment of the present invention.

A vehicle driving monitoring apparatus according to an embodiment of the present invention is a control unit for the fusion of the camera (10a) for recognizing the front of the vehicle and the camera (12a) for recognizing the driver's state (eg, drowsiness) ( 32).

The camera 10a may detect forward and lane detection and detect a traffic sign. The camera 12a may acquire face information of the driver.

The controller 32 may perform image recognition fusion, external communication, vehicle communication, and the like.

In other words, the control unit 32 may fuse with the cameras 10a and 12a to communicate the driver status, driving information, and vehicle information with the driving information management system 34 through 4G or 5G wireless communication.

The driving information management system 34 may perform various management services based on the information from the controller 32.

FIG. 2 is a block diagram illustrating a vehicle driving monitoring apparatus according to an exemplary embodiment of the present invention. FIG. 3 is a diagram illustrating a software architecture of the controller shown in FIG. 2, and FIG. 4 is driving information illustrated in FIG. 2. It is a block diagram of the management system.

The vehicle driving monitoring apparatus according to the embodiment of the present invention, the front photographing unit 10, the driver photographing unit 12, the sensor unit 14, the serial and parallel converters 16 and 18, the lighting driver 20, storage The unit 22 may include an image processor 24, an alarm unit 26, a display unit 28, a communication unit 30, and a controller 32.

The front photographing unit 10 photographs the front of the vehicle in driving (that is, including other vehicles and lanes, etc.).

The front photographing unit 10 may include a camera 10a and a serial converter 10b.

The camera 10a may be mounted at the front of the vehicle to acquire driving information in front of the vehicle.

For example, the camera 10a may include an HD sensor or higher. Preferably, the camera 10a may be a wide angle camera having an angle of view of about 110 degrees.

The serializer 10b (serializer) performs a serial transfer process to enable a high speed interface when transmitting the driving information (image information) obtained from the camera 10a to the controller 32. For example, the serial converter 10b may be a specification of an HSD output of 1.5Gbps or more.

The driver photographing unit 12 may photograph a face of a driver in a vehicle.

The driver photographing unit 12 may include a camera 12a, an illumination unit 12b, and a serial converter 12c.

The camera 12a may be mounted to face the driver in the dashboard of the vehicle, and may include an image sensor having an angle of view of about 40 degrees. Here, the image sensor minimizes image distortion by applying a global shutter. Of course, the camera 12a may be mounted on the front end area of the roof panel of the vehicle, the glass upper area of the front surface of the vehicle, near the room mirror, etc., in addition to the dashboard.

The lighting unit 12b provides illumination so that the camera 12a can properly photograph the driver's face even at low light. The lighting unit 12a is installed inside the vehicle so that near infrared rays can be projected onto the face portion including at least the pupil of the driver.

The lighting unit 12b includes a light emitting element that emits near infrared rays.

Meanwhile, the lighting unit 12b may further include an illumination lens, a light diffusion plate, a light diffusion film, a reflector, and the like for the light uniformity of the projection light.

In FIG. 2, the camera 12a and the lighting unit 12b are configured independently, but the camera 12a and the lighting unit 12b may be integrally formed.

The serializer 12c (serializer) performs a serial transfer process to enable a high speed interface when transmitting the driver's face image information acquired by the camera 12a to the controller 32. For example, the serial converter 12c may be a HSD output 1.5Gbps or more.

If necessary, the front photographing unit 10 and the driver photographing unit 12 may be collectively referred to as a photographing unit.

The sensor unit 14 may sense the acceleration direction of each vehicle's collision direction and three axes (X, Y, and Z axes) of the current vehicle.

For example, the sensor unit 14 may include a gravity sensor and a motion sensor. The gravity sensor may sense the collision direction when the vehicle crashes. The motion sensor is installed in the vehicle, and the X-axis direction, which is the current direction of the vehicle, and the 90-degree angle with the X-axis. Each piece of acceleration information obtained in the Z-axis direction with respect to the upper side or the lower side can be obtained. In general, since the vehicle is driven in a two-dimensional plane, Z-axis acceleration information is not necessary, but may be used to utilize driver's comfort information in the future.

The sensor unit 14 described above may be an optional component.

If necessary, the information obtained by the sensor unit 14 may be included in the vehicle information described later.

Vehicle information applied to the control unit 32 shown in FIG. 2 may be sensing information of various sensors typically installed in the vehicle, in addition to the detection information of the sensor unit 14. Here, a variety of sensors commonly installed are wheel sensors, GPS modules, speed sensors, tilt sensors, vehicle forward / reverse sensors, battery sensors, fuel sensors, tire sensors, steering sensors by steering wheel steering, and vehicle interiors. It may include a temperature sensor, a vehicle interior humidity sensor, an ultrasonic sensor, an illumination sensor.

Accordingly, the vehicle information includes vehicle direction information, vehicle position information (GPS information), vehicle angle information, vehicle speed information, vehicle tilt information, vehicle forward / reverse information, battery information, fuel information, tire information, steering wheel rotation angle, It may be a sensing signal for vehicle interior temperature information, vehicle interior humidity information, illuminance around the vehicle, and the like.

On the other hand, the above-described various types of sensors that are usually installed, such as accelerator pedal sensor, pressure sensor, engine speed sensor (engine speed sensor), air flow sensor (AFS), intake temperature sensor (ATS), water temperature sensor (WTS), throttle Position sensor (TPS), TDC sensor, crank angle sensor (CAS) may be further included.

The vehicle information as described above may be applied to the controller 32 through a controller area network (CAN) communication module (not shown).

The deserializer 16 converts the serial data from the serial converter 10b of the front photographing unit 10 into parallel data and applies it to the controller 32. For example, the serial-to-parallel converter 16 may have a specification of an HSD input of 1.5Gbps or more.

The deserializer 18 converts the serial data from the serial converter 12c of the driver photographing unit 12 into parallel data and applies it to the controller 32. For example, the serial-to-parallel converter 18 may be a HSD input of 1.5 Gbps or more.

The lighting driver 20 may drive the lighting unit 12b according to a control signal from the control unit 32.

The storage unit 22 stores the camera data of the front photographing unit 10 (that is, the image in front of the vehicle) through the serial / parallel conversion unit 16, and the driver photographing unit through the serial / parallel conversion unit 18 ( 12) camera data (ie, driver's face image). This is so that the image processing in the image processing unit 24 can be easily performed.

That is, the storage unit 22 stores the camera data of the front photographing unit 10 and the driver photographing unit 12 under the control of the control unit 32. The storage unit 22 may be referred to as a video buffer.

The image processor 24 processes the camera data of the front photographing unit 10 (that is, the image in front of the vehicle) stored in the storage unit 22 and provides the image data to the controller 32 so that the controller 32 receives the image data. It allows you to determine whether you have left the lane and whether you are in front of you.

Meanwhile, the image processor 24 processes the camera data (ie, the driver's face image) of the driver photographing unit 12 stored in the storage unit 22 and provides the image to the controller 32, thereby controlling the controller 32. Allows the driver to determine whether the driver is drowsy.

That is, the image processor 24 outputs information for determining whether the lane is out of line, whether it is a front collision, or whether the driver is drowsy based on the camera data of the front photographing unit 10 and the camera data of the driver photographing unit 12. can do.

The alarm unit 26 warns of lane departure, frontal collision, and drowsiness of the driver.

For example, the alarm unit 26 may include a speaker or a buzzer that outputs a predetermined sound.

On the other hand, the alarm unit 26 may generate a predetermined tactile output. For example, the alarm unit 26 may operate by vibrating a steering wheel, a seat belt, a seat, or the like so that a driver may recognize an output.

The display unit 28 displays a predetermined warning message related to lane departure, forward collision, and drowsiness. For example, the display unit 28 may display a warning message when a lane departure is expected, a forward collision is expected, or the driver is determined to be asleep, so that the driver may check.

For example, the display unit 28 may include a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display. The display device may include at least one of a flexible display, a 3D display, and an e-ink display.

It is preferable that the above-mentioned alarm part 26 and the display part 28 operate | move together. That is, when an alarm is issued by the alarm unit 26, the display unit 28 displays a warning message, which not only makes the driver aware of what state he is currently in, but also maximizes the effect of the alarm / warning.

The communication unit 30 may serve as a path to an external driving information management system 34 that may be connected to the vehicle driving monitoring apparatus according to the exemplary embodiment of the present invention.

The communication unit 30 may be configured as a short range communication module, a V2X communication module, or the like.

The near field communication module is for short range communication, and includes Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and Near Field Communication (NFC). At least one of Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless Universal Serial Bus (Wireless USB) technologies may be used to support short-range communication, which may include a wireless area network. In such a manner, short-range wireless communication between the vehicle driving monitoring apparatus and the driving information management system 34 according to an embodiment of the present invention may be performed.

The V2X communication module performs wireless communication with the driving information management system 34 or another vehicle. The V2X communication module may include a module capable of implementing inter-vehicle communication (V2V) or inter-vehicle communication (V2I) protocol.

The control unit 32 controls the overall operation of the vehicle driving monitoring apparatus according to the embodiment of the present invention.

In particular, the controller 32 may determine whether the vehicle is out of lane, whether it is a front collision, or whether the driver is sleepy based on the information from the image processor 24. The controller 32 may control the alarm and the display operation according to the determination result.

Meanwhile, the control unit 32 transmits the information from the sensor unit 14 and various vehicle information together with the above-described determination result (that is, lane departure determination information, forward collision determination information, and driver drowsiness determination information). 30 may be sent to the driving information management system 34.

In FIG. 2, the image processor 24 and the controller 32 are shown in separate configurations, but the image processor 24 may be included in the controller 32.

Accordingly, the control unit 32 can be said to have a software architecture as shown in FIG. According to FIG. 3, the software architecture of the controller 32 may include an application processor (AP) software and a microcomputer (MCU) software. The application processor (AP) software may be responsible for driver recognition and forward recognition programs, communications, and vehicle driving information management programs. Microcontroller (MCU) software can handle vehicle CAN communication, power control and system diagnostics.

The application processor (AP) software and the microcomputer software may be divided into a hardware layer, an operating system layer, a device layer, a service layer, an algorithm layer, and a higher level application layer as shown in FIG. 3.

Meanwhile, the driving information management system 34 associates the driver status, driving information, and vehicle information from the vehicle driving monitoring apparatus according to the embodiment of the present invention with the backend service as shown in FIG. 4, and manages / controls services and service apps. Through the information can be provided in real time.

FIG. 5 is a diagram for explaining a lane departure warning and a forward collision warning process according to an exemplary embodiment of the present invention.

When the front photographing unit 10 photographs the front of the vehicle, an input image as shown in FIG. 5A may be stored in the storage unit 22 and then applied to the image processing unit 24.

The image processor 24 performs image processing for detecting the current driving lane of the driver based on the input image. Accordingly, the image processor 24 may generate a lane detection image as shown in FIG. 5B so that the controller 32 may detect the lane in which the driver is currently driving.

After lane detection, the image processing unit 24 performs image processing for lane crossing detection. Accordingly, the image processor 24 may generate a lane intersection detection image as shown in FIG. 5C, and the controller 32 may detect a point where the lanes cross based on the lane intersection detection image.

After lane crossing detection, the image processing unit 24 performs image processing for optimal lane recognition. Accordingly, the image processor 24 may generate the optimal lane recognition image as shown in FIG. 5D, so that the controller 32 may recognize the optimal lane based on the image.

On the other hand, the image processor 24 performs image processing for extracting the edge of the front vehicle based on the input image. Accordingly, the image processor 24 may generate an edge extraction image as shown in FIG. 5E, so that the controller 32 may detect the front vehicle based on the image.

After the front vehicle detection, the image processing unit 24 performs image processing for recognizing the lights, bumpers, and the like of the front vehicle. Accordingly, the image processor 24 may generate the light and the bumper recognition image of the front vehicle as shown in FIG. 5F, so that the control unit 32 may recognize the light and the bumper of the front vehicle.

Finally, since the image processor 24 may output the final image as shown in (g) of FIG. 5, the controller 32 may determine whether a lane departure is expected or a forward collision is expected based on this. .

In other words, after detecting the lane, the controller 32 may calculate the lane departure estimated time and the lane departure estimated distance, and may alert the driver. In addition, after detecting the vehicle ahead, the controller 32 may calculate an estimated collision time and a vehicle distance and alert the driver.

6 is a view for explaining a process of detecting a driver state in an embodiment of the present invention.

When the driver photographs the driver 12, the input image as shown in FIG. 6A may be stored in the storage 22 and then applied to the image processor 24.

The image processor 24 performs image processing for detecting the face of the current driver based on the input image. Accordingly, the image processor 24 may generate the face detection image as shown in FIG. 6B so that the controller 32 may detect the face region of the current driver based on the face detection image.

After face region detection, the image processing unit 24 performs image processing for head angle measurement (eye tracking) of the driver. Accordingly, the image processor 24 generates the head angle measurement image as shown in FIG. 6C, so that the controller 32 can measure the head angle of the driver (ie, track the gaze) based on the image. have.

In addition, after detecting the face region described above, the image processor 24 detects the eye region of the driver by using the feature point extraction technique as shown in FIG. 6 (d) and closes the closed eyes existing in the detected eye region. eye and / or open eye.

Finally, the image processor 24 may output a final image (ie, an image capable of grasping a face area, a head angle, an eye, etc.) as shown in FIG. The driver's state (ie, sleepiness) can be determined.

In other words, the controller 32 may determine whether the driver is dozing by face detection, head angle measurement, and eye detection to alert the driver.

7 is a flowchart illustrating a vehicle driving monitoring method according to an embodiment of the present invention.

When the vehicle is driven (S10), the front photographing unit 10 photographs an image in front of the vehicle (that is, including other vehicles and lanes), and the driver photographing unit 12 photographs the face of the driver in the vehicle. (S20).

Accordingly, the controller 32 determines the driving state and the driver state based on the image photographed by the linkage with the image processing unit 24 (S30).

That is, in order to grasp the driving state, as shown in FIG. 5, the control unit 32 detects the current lane in which the vehicle is driving based on the image in front of the vehicle by linking with the image processing unit 24. The estimated lane departure time and the estimated lane departure distance are calculated. The controller 32 detects a vehicle present in front of the vehicle based on an image in front of the vehicle by linking with the image processor 24 and calculates an estimated collision time and a vehicle distance.

In addition, in order to grasp the driver's condition, the controller 32 performs face detection, head angle measurement, and eye detection of the driver in association with the image processing unit 24 as described above.

As a result, the controller 32 determines whether lane departure or forward collision is expected based on the calculated information (S40), and if the lane departure and forward collision is not expected (S50). Here, it is described as determining whether the driver is asleep after determining whether a lane departure or forward collision is expected, but the order may be changed. Further, judging whether a lane departure or forward collision is expected and whether the driver is asleep may be performed simultaneously.

If a lane departure or a forward collision is expected (Yes in S40) or the driver is determined to be dozing (Yes in S50), the control unit 32 alerts through the alarm unit 26 and displays the unit. The current state is displayed on (28) (S60). For example, a predetermined sound may be output or a buzzer may sound through the speaker. Alternatively, vibrations can be generated in the steering wheel, seat belts, seats, etc. so that the driver can recognize the output. In addition, a message may be displayed on the display unit 28 warning that lane departure is expected, a forward collision is expected, or that the driver is dozing.

The control unit 32 transmits the information from the sensor unit 14 and various vehicle information together with the above-described determination result (that is, lane departure determination information, forward collision determination information, and driver drowsiness determination information). 30 and transmits to the driving information management system 34 (S70).

In addition, the vehicle driving monitoring method according to the above-described embodiment of the present invention may be implemented as computer readable codes on a computer readable recording medium. Computer-readable recording media include all kinds of recording devices that store data that can be read by a computer system. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disks, optical data storage devices, and the like. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. In addition, functional programs, codes, and code segments for implementing the method can be easily inferred by programmers in the art to which the present invention belongs.

As described above, the best embodiment has been disclosed in the drawings and the specification. Although specific terms have been used herein, they are used only for the purpose of illustrating the present invention and are not intended to limit the scope of the invention as defined in the meaning or claims. Therefore, those skilled in the art will understand that various modifications and equivalent other embodiments are possible from this. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

10: front photographing unit 12: driver photographing unit
14 sensor 16, 18 serial-to-parallel converter
20: light driving unit 22: storage unit
24: image processing unit 26: alarm unit
28: display unit 30: communication unit
32: control unit 34: driving information management system

Claims (12)

A front photographing unit which photographs a front side of a driving vehicle;
A driver photographing unit which photographs a face of a driver of the vehicle;
An image processor configured to process the image data from the front photographing unit and the driver photographing unit to output information for determining whether the vehicle is out of lane, whether it is a front collision, or whether the driver is drowsy;
A controller which determines whether the vehicle is out of lane, whether it is a front collision, whether the driver is drowsy, and controls an alarm operation based on the determination result based on the information from the image processor; And
And an alarm unit for generating an alarm under the control of the controller. The method according to claim 1,
And a storage unit which stores image data from the front photographing unit and the driver photographing unit.
And the image processing unit processes the image data stored in the storage unit to output information for determining whether the lane is out of line, whether it is a front collision, or whether the driver is drowsy. The method according to claim 1,
The control unit,
Based on the lane detection image and the edge extraction image in the image processing unit based on the input image from the front photographing unit, the driver detects the current lane in which the driver is traveling and detects the vehicle ahead, and the image after the lane detection. Detecting the intersection of the lanes based on the lane crossing detection image from the processing unit, and after detecting the lane intersection point, recognizes the optimum lane based on the optimal lane recognition image from the image processing unit, and after detecting the front vehicle Recognizes the lights and bumpers of the front vehicle based on the light and bumper recognition image of the front vehicle from the image processing unit, and predicts the lane departure and the forward collision based on the recognized optimal lanes and the lights and bumpers of the front vehicle. Vehicle operation, characterized in that determining whether Monitoring device. The method according to claim 1,
The control unit,
The driver detects a face region of the driver based on a face detection image in the image processor based on an input image from the driver photographing unit, and after detecting the face region, the corresponding driver based on the head angle measurement image from the image processor. Measuring the head angle of the eye and detecting the eyes in the face region and classifying them into closed eyes and open eyes, and determining whether the driver is sleepy based on the measured head angle of the driver and the classified closed eyes and open eyes. Vehicle running monitoring device. The method according to claim 1,
And a display unit for displaying a warning message related to the lane departure, the front collision, and the drowsiness.
And the display unit is operated together with the alarm unit. The method according to claim 1,
The control unit,
Receiving vehicle information from at least one sensor installed inside the vehicle,
And transmitting the determination result and the vehicle information to an external driving information management system. Photographing unit, photographing the front of the driving vehicle and the face of the driver of the vehicle;
Image processing by the image processor to process the image data from the photographing unit to output information for judging whether the lane is out of front, whether there is a collision, or whether the driver is drowsy;
Determining, by the controller, whether the vehicle is out of lane, whether it is a front collision, or whether the driver is sleepy based on the information from the image processor; And
And controlling, by the controller, an alarm operation according to a result of the determining. The method according to claim 7,
And storing, by the control unit, image data from the photographing unit in a storage unit.
The outputting of the information may include image processing the image data stored in the storage unit to output information for determining whether the lane is out of line, whether it is a front collision, or whether the driver is drowsy. The method according to claim 7,
The determining step,
Detecting a current lane in which the driver is driving based on a lane detection image and an edge extraction image in the image processing unit based on an image photographing the front of the driving vehicle and detecting a front vehicle;
Detecting the intersection of the lanes based on the lane intersection detection image from the image processing unit after the lane detection;
Recognizing an optimal lane based on an optimal lane recognition image from the image processor after the lane intersection detection;
Recognizing a light and a bumper of the front vehicle based on the light and the bumper recognition image of the front vehicle from the image processor after detecting the front vehicle; And
And determining whether a lane departure is expected or a forward collision is predicted based on the recognized optimal lane and the lights and bumpers of the front vehicle. The method according to claim 7,
The determining step,
Detecting a face region of a driver based on a face detection image of the image processor based on an image of the face of the driver of the vehicle;
Measuring the head angle of the driver based on the head angle measurement image from the image processing unit after detecting the face region, and detecting the eyes in the face region and classifying them into closed eyes and open eyes; And
And determining whether the driver is sleepy based on the measured head angle of the driver and the classified closed eyes and the open eyes. The method according to claim 7,
And displaying a warning message related to the lane departure, frontal collision, and drowsiness on a display unit.
And the display by the displaying step is performed together with the alarm by controlling the alarm operation. The method according to claim 7,
And transmitting the vehicle information from the one or more sensors installed in the vehicle and the result of the determining to an external driving information management system.

Images