KR102365361B1 - An apparatus and method for determining a driving state of a vehicle - Google Patents

Abstract

The present invention relates to an apparatus and a method for determining a driving state of a vehicle. The apparatus for determining a driving status of a vehicle of the present invention comprises: a camera; and a processor which obtains a driving image of the vehicle by the camera, obtains road coordinate information with respect to the vehicle, obtains information of a vehicle driving lane, and sets at least one section on the driving image to determine whether the vehicle is out of the lane based on the road coordinate information and the lane information. Therefore, according to the present invention, the driving state of the vehicle is accurately determined.

Description

Apparatus and method for determining the driving state of a vehicle

The present invention relates to an apparatus and method for determining a driving state of a vehicle.

In general, a vehicle may be equipped with a device (eg, a black box) that records and stores a driving state. Such a device (eg, a black box) may be released by a vehicle manufacturer embedded in the vehicle, or may be separately purchased by a driver and installed in the vehicle.

In addition, commercial vehicles such as trucks, buses, and taxis can store images in a black box. In addition, the business vehicle transmits the stored image to the server managing the business vehicle through broadband network communication (eg, LTE, 4G, 5G, etc.) can be sent to the server. In this case, the operator who manages the vehicle can secure data on the occurrence of an accident, driving violation, etc. through the captured image.

As such, since the operator managing the vehicle secures the image of the driving situation, the operator can accurately determine the cause of the accident or utilize the image as data for evaluating and educating a careless driver.

However, when the driver of the vehicle violates a signal or a lane, it may take a lot of time for the operator to analyze all the images taken from the vehicle in order to confirm this, and may result in an increase in labor costs.

And, the prior art (Registered Patent Publication No. 10-1840974) relates to a lane identification system for autonomous driving, and discloses content for identifying a lane.

However, the prior art only identifies the lane by deleting unnecessary pixels from the image captured by the camera, and does not determine the overall driving state as to whether the vehicle is driving while touching the lane or driving across the lane.

Accordingly, there is a need to automatically identify a lane while driving in order to determine the overall driving state as to whether the vehicle travels while touching the lane or driving across the lane.

Registered Patent Publication No. 10-1840974

Accordingly, it is an object of the present invention to provide an apparatus and method for determining a driving state of a vehicle.

Another object of the present invention is to provide an apparatus and method for setting, in a driving image, at least one section for determining whether to depart from a lane by using vehicle-based road coordinate information and lane information.

The objects of the present invention are not limited to the objects mentioned above, and other objects and advantages of the present invention not mentioned may be understood by the following description, and will be more clearly understood by the examples of the present invention. It will also be readily apparent that the objects and advantages of the present invention may be realized by the means and combinations thereof indicated in the appended claims.

In order to achieve this object, the present invention provides an apparatus for determining the driving state of a vehicle, a camera, and a driving image of the vehicle through the camera, obtains road coordinate information based on the vehicle, and a lane in which the vehicle travels and a processor for acquiring information and setting, in the driving image, at least one section for determining whether the vehicle departs from a lane based on the coordinate information and the lane information.

In addition, the present invention provides a method for determining the driving state of a vehicle, comprising: acquiring a driving image of the vehicle through a camera; acquiring road coordinate information based on the vehicle; The method may include setting, in the driving image, at least one section for determining whether the vehicle departs from a lane based on the process, the road coordinate information and the lane information.

The present invention can accurately determine the driving state of the vehicle by obtaining the vehicle reference road coordinate information and lane information of the vehicle.

In addition, the present invention can determine the current driving state of the vehicle as well as determine the current driving state by setting at least one section for determining whether the vehicle departs from the lane based on the vehicle reference road coordinate information and lane information in the driving image. , can be applied to autonomous vehicles.

In addition, the present invention sets a margin to the left and right of a vanishing point in the driving image, sets a midpoint of the width of the vehicle at the lower end of the driving image as a reference position, and sets a point meeting the horizon in the driving image and the driving By setting in the driving image at least one section for determining whether or not lane departure is made through a point where the lower end of the image intersects, more accurate data on lane departure may be secured.

In addition, the present invention can be applied to various road environments by determining the margin set to the left and right of the vanishing point based on the weight of an occupant of the vehicle, the bending state of the lane, and the shaking of the vehicle.

In addition, according to the present invention, by setting the complete invasion section, the temporary invasion section, and the normal section based on the reference position, it is possible to not only determine a more accurate driving state but also analyze the driver's driving habits.

In addition to the above-described effects, the specific effects of the present invention will be described together while describing specific details for carrying out the invention below.

1A is an exemplary diagram illustrating a case in which a center lane is a dotted line according to an embodiment of the present invention.
1B is an exemplary view illustrating a case in which a center lane is a solid line according to an embodiment of the present invention.
1A is an exemplary diagram illustrating a case in which a center lane is a double track according to an embodiment of the present invention.
2 is a block diagram of an apparatus for determining a driving state of a vehicle according to an embodiment of the present invention.
3 is a flowchart illustrating a method of setting at least one section for determining a driving state of a vehicle in a driving image according to an embodiment of the present invention.
4 is an exemplary diagram illustrating a relationship between actual road coordinates for a lane and coordinates of a lane in an image of a road projected by a camera according to an embodiment of the present invention.
5A is an exemplary view in which a vehicle travels in the center of a lane according to an embodiment of the present invention.
5B is an exemplary diagram illustrating a vanishing point in FIG. 5A .
6A is an exemplary view in which a vehicle travels while touching a left lane according to an embodiment of the present invention.
FIG. 6B is an exemplary view illustrating a vanishing point in FIG. 6A .
7A is an exemplary view in which a vehicle travels while touching a right lane according to an embodiment of the present invention.
7B is an exemplary view showing the vanishing point in FIG. 7A.
8A is an exemplary view in which a vehicle travels while crossing a left lane according to an embodiment of the present invention.
8B is an exemplary diagram illustrating a vanishing point in FIG. 8A .
9 is an exemplary view illustrating at least one section set in a driving image to determine whether a vehicle departs from a lane according to an embodiment of the present invention.
10 is a flowchart illustrating a process of storing and transmitting a driving image by determining whether a vehicle departs from a lane according to an embodiment of the present invention.
11 is an exemplary view illustrating normal driving in a lane according to an embodiment of the present invention.
12A is an exemplary diagram of driving while temporarily infiltrating a left lane according to an embodiment of the present invention.
12B is an exemplary diagram of driving while temporarily infiltrating a right lane according to an embodiment of the present invention.
13A is an exemplary view of driving while completely encroaching on the left lane according to an embodiment of the present invention.
13B is an exemplary view of driving while completely encroaching on the right lane according to an embodiment of the present invention.
14A is an exemplary diagram illustrating abnormal driving by temporarily infiltrating a left lane according to an embodiment of the present invention.
14B is an exemplary diagram illustrating abnormal driving by temporarily infiltrating a right lane according to an embodiment of the present invention.
15A is an exemplary diagram illustrating abnormal driving by completely encroaching on the left lane according to an embodiment of the present invention.
15B is an exemplary diagram illustrating abnormal driving by temporarily infiltrating a right lane according to an embodiment of the present invention.
16 is an exemplary diagram illustrating a change in a GPS travel direction when a vehicle changes to a left lane according to an embodiment of the present invention.
17 is an exemplary diagram illustrating a change in a GPS traveling direction when a vehicle makes a U-turn according to an embodiment of the present invention.
18 is an exemplary diagram illustrating a format of a GPRMC message and definitions of each field according to an embodiment of the present invention.
19 is an exemplary view for identifying the rotation of a vehicle according to an embodiment of the present invention.
20 is an exemplary diagram illustrating a rate change when a vehicle changes a left lane according to an embodiment of the present invention.
21 is an exemplary diagram illustrating a rate change when the vehicle makes a U-turn according to an embodiment of the present invention.
22 is a flowchart illustrating a process of identifying a vehicle intrusion into a center lane according to an embodiment of the present invention.
23 is an exemplary diagram illustrating a process of identifying a vehicle intrusion into a center lane according to an embodiment of the present invention.
24 is a flowchart illustrating a process of identifying driving in an opposite lane of a vehicle according to an embodiment of the present invention.
25 is an exemplary diagram illustrating a process of identifying driving in an opposite lane of a vehicle according to an embodiment of the present invention.
26 is a flowchart illustrating a process of identifying driving in an opposite lane of a vehicle according to another embodiment of the present invention.
27 is an exemplary diagram illustrating a process of identifying driving in an opposite lane of a vehicle according to another embodiment of the present invention.
28 is a flowchart illustrating a process of identifying driving in an opposite lane of a vehicle according to another embodiment of the present invention.
29 is an exemplary diagram illustrating a process of identifying driving in an opposite lane of a vehicle according to another embodiment of the present invention.
30 is a flowchart illustrating a process of identifying that a vehicle makes a U-turn in a prohibited section according to an embodiment of the present invention.
31A and 31B are exemplary views illustrating a process in which a vehicle makes a U-turn in a prohibited section according to an embodiment of the present invention.
32 is a flowchart illustrating a process of identifying that a vehicle makes a U-turn in a prohibited section according to another embodiment of the present invention.
33 is an exemplary diagram illustrating a process in which a vehicle makes a U-turn in a prohibited section according to another embodiment of the present invention.

The above-described objects, features and advantages will be described below in detail with reference to the accompanying drawings, and accordingly, those of ordinary skill in the art to which the present invention pertains will be able to easily implement the technical idea of the present invention. In describing the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to indicate the same or similar components.

Although the first, second, etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from other components, and unless otherwise stated, it goes without saying that the first component may be the second component.

In the following, that an arbitrary component is disposed on the "upper (or lower)" of a component or "upper (or below)" of a component means that any component is disposed in contact with the upper surface (or lower surface) of the component. Furthermore, it may mean that other components may be interposed between the component and any component disposed on (or under) the component.

In addition, when it is described that a component is “connected”, “coupled” or “connected” to another component, the components may be directly connected or connected to each other, but other components are “interposed” between each component. It should be understood that “or, each component may be “connected,” “coupled,” or “connected,” through another component.

Throughout the specification, unless otherwise stated, each element may be singular or plural.

As used herein, the singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “consisting of” or “comprising” should not be construed as necessarily including all of the various components or various steps described in the specification, some of which components or some steps are It should be construed that it may not include, or may further include additional components or steps.

Throughout the specification, when “A and/or B” is used, it means A, B or A and B, unless otherwise stated, and when “C to D” is used, it means that there is no specific opposite description. Unless otherwise specified, it means that it is greater than or equal to C and less than or equal to D.

Hereinafter, an apparatus and method for determining a driving state of a vehicle according to some embodiments of the present invention will be described.

1A is an exemplary diagram illustrating a case in which a center lane is a dotted line according to an embodiment of the present invention. 1B is an exemplary view illustrating a case in which a center lane is a solid line according to an embodiment of the present invention. 1A is an exemplary diagram illustrating a case in which a center lane is a double track according to an embodiment of the present invention.

In general, whether vehicles drive to the right or to the left differs from country to country. In this specification, a case in which the vehicle travels to the right is described, but it may also be applied to a case in which the vehicle travels to the left.

Referring to FIG. 1A , the vehicle 101 may drive 110 to the right. In addition, the center lane 111 that separates the driving direction and the reverse direction may be a single yellow dotted line. A case in which the center lane 111 is a single yellow dotted line may correspond to a case in which the road is a one-lane one-way road, which means that lane intrusion is temporarily permitted. As such, when the vehicle 101 travels on the right road, the center lane 111 is located on the left side of the vehicle 101 .

Referring to FIG. 1B , the vehicle 101 may drive 120 to the right. In addition, the center lane 121 that separates the driving direction and the reverse direction may be a single yellow solid line. When the center lane 121 is a solid yellow line of a single lane, it means that the lane intrusion is prohibited. As such, when the vehicle 101 travels on the right road, the center lane 121 is located on the left side of the vehicle 101 .

Referring to FIG. 1C , the vehicle 101 may drive 130 to the right. In addition, the center lane 131 that separates the driving direction and the reverse direction may be a double-track yellow solid line. When the center lane 121 is a solid yellow line of a double-track, it means that the intrusion of the lane is absolutely prohibited. As such, when the vehicle 101 travels on the right road, the center lane 121 is located on the left side of the vehicle 101 .

The vehicle 101 traveling in the lane may be driven while contacting the inner reference line 132 of the center lane 131 or may be driven by invading the center lane 131 depending on the driver's inexperienced driving or surrounding circumstances. The present invention is such that, in order to automatically identify whether the vehicle is driven in contact with the center lane or invades the center lane 131, it is determined whether the vehicle departs from the lane based on the vehicle standard road coordinate information and lane information. to set at least one section for the driving image.

Hereinafter, a specific technique for setting at least one section for determining whether to depart a lane based on lane information of an image obtained from a camera mounted on a vehicle in a driving image will be described.

2 is a block diagram of an apparatus for determining a driving state of a vehicle according to an embodiment of the present invention.

Referring to FIG. 2 , an apparatus 200 for determining a driving state of a vehicle according to an embodiment of the present invention includes a camera 210 , a sensor unit 220 , a communication unit 230 , a storage unit 240 , and a display unit ( 250 ), and a processor 260 .

The configuration of the device 200 for determining the driving state of the vehicle shown in FIG. 2 is according to an embodiment, and the components of the device 200 for determining the driving state of the vehicle are limited to the embodiment shown in FIG. 2 . Not necessarily, and some components may be added, changed, or deleted as necessary.

According to an embodiment, the camera 210 may include at least one image sensor. The camera 210 may acquire an image of the front or rear of the vehicle 101 being driven. The camera unit 250 may include an image intelligence camera (or smart image intelligence camera) with an embedded artificial intelligence chip. Objects, people, lanes, crosswalks, intersections, etc. in front or behind the vehicle 101 can be recognized through the image intelligence camera.

According to an embodiment, the camera 210 is a camera capable of acquiring an image of the front or rear of the vehicle 101 even in a situation in which illumination is high due to lighting (eg, a Red Green Blue (RGB) camera); It may include an IR (infrared) camera capable of acquiring an image in a low-illuminance situation. The camera 210 may include a device (eg, LED, IR pattern lighting) that emits light necessary to acquire an image. The camera 210 may be included in the device 200 or may be separated from the device 200 . For example, when the camera 210 is separated from the device 200 , the camera 210 and the device 200 may transmit/receive images, images, or data via wired or wireless.

According to an embodiment, the sensor unit 220 includes a GPS sensor (not shown) that detects a driving direction, lane change, U-turn, etc. of the vehicle 101 based on a Global Positioning System (GPS) signal, and the vehicle 101 . ) to the left or right (eg, in the X-axis direction), and a roll sensor that detects the amount of rotation change, forward or backward of the vehicle 101 (eg, in the Y-axis direction) and shake At least one of a pitch sensor that detects a change in can The sensor unit 220 may, for example, transmit information about a change in the traveling direction of the vehicle obtained from GPS and information about a rotation angle of the vehicle obtained from a yaw sensor to the processor 260 . there is.

According to an embodiment, the sensor unit 220 may include at least one sensor capable of detecting a distance of a pedestrian approaching the vehicle 101 , a movement of a pedestrian, or brightness around the vehicle 101 . there is. For example, the sensor unit 270 may include a distance measuring sensor (not shown), a motion detection sensor (not shown), and an illuminance sensor (not shown).

According to an embodiment, the communication unit 230 includes at least one component (eg, a camera 210 , a sensor unit 220 , and a storage unit 240 ) included in the device 200 for determining the driving state of the vehicle. , the display unit 250 , and the processor 260) and at least one circuit capable of transmitting and receiving at least one signal or information through wired communication or wireless communication.

According to an embodiment, the communication unit 230 is at least one circuit capable of transmitting and receiving at least one signal or information through wireless communication to an external electronic device (not shown) (eg, a server for collecting vehicle driving information). may include For example, the communication unit 230 transmits, under the control of the processor 260 , photos, videos, and data related to driving of the vehicle 101 stored in the storage unit 240 to the external electronic device (not shown). It can be transmitted through wireless communication to (eg, a server that collects vehicle driving information).

According to an embodiment, the storage unit 240 may include a volatile memory or a non-volatile memory. For example, the storage unit 240 may store information, data, and programs necessary for the operation of the device 200 for determining the driving state of the vehicle 101 . Accordingly, the processor 260 may perform a control operation to be described later with reference to the information stored in the storage unit 240 . The storage unit 240 may store various platforms. The storage unit 240 is, for example, a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg, SD or XD memory, etc.), RAM (RAM), ROM (EEPROM, etc.), and may include at least one type of storage medium among USB memory.

According to an embodiment, the storage unit 240 includes at least one component (eg, a camera 210 , a sensor unit 220 , and a communication unit 200 ) of the device 200 for determining the driving state of the vehicle 101 . 230), the display unit 250, and the processor 260) may store various data acquired or used (eg, software, applications, acquired information, measured information, control signals, etc.), and instructions related thereto. there is. For example, the storage unit 240 may store photos, videos, and data regarding the driving state of the vehicle 101 acquired through the camera 210 . Also, the storage unit 240 may store data, information, or signals acquired by at least one sensor included in the sensor unit 220 .

According to an embodiment, the display unit 250 may display various types of information (eg, multimedia data or text data). The display unit 250 may display a result processed, in process, or to be processed by the processor 260 . The display unit 260 may visually display photos, videos, or data acquired through the camera 210 under the control of the processor 260, and may include a control circuit for displaying such various information. there is. For example, the display unit 260 may display a moving picture related to the driving of the vehicle 101 acquired through the camera 210 in real time.

According to an embodiment, the display unit 230 may display an operation state, a communication state, a GPS reception state, and the like of a device for determining the driving state of the vehicle.

According to an embodiment, the display unit 230 may include a touch circuitry for sensing a touch or a pressure sensor capable of measuring the intensity of the pressure applied to the touch.

According to an embodiment, the processor 260 drives software to operate at least one component connected to the processor 260 (eg, a camera 210, a sensor unit 220, a communication unit 230, a storage unit ( 240), and the display unit 250) may be controlled based on wired communication or wireless communication. In addition, the processor 260 may perform various data processing and operations based on the wired communication or the wireless communication.

According to an embodiment, the processor 260 stores commands or data received from the camera 210 , the sensor unit 220 , the communication unit 230 , the storage unit 240 , and the display unit 250 , etc. It may be loaded and processed in 240 , and the processed data may be stored in the storage unit 240 . Alternatively, the processor 260 may display the processed data through the display unit 250 or output it through a speaker (not shown).

According to an embodiment, the processor 260 may acquire an image or an image regarding the driving state of the vehicle 101 through the camera 210 . The processor 260 may acquire an image of a road on which the vehicle 101 is traveling in real time through a camera 210 disposed in front and/or rear of the vehicle 101 . In addition, the processor 260 may display an image of a road on which the vehicle 101 is traveling in real time through the display unit 250 .

According to an embodiment, the processor 260 may obtain road coordinate information (eg, coordinate information) based on the vehicle 101 . The processor 260 determines the actual coordinates (eg, X, Y, Z) (or (X, 0, Z)) of the lane on which the vehicle 101 is traveling, Coordinate information including at least a part of coordinates (eg, x, y), a focal length of the camera 210 , and a mounting height h of the camera 210 in the lane may be acquired. The processor 260 may obtain road coordinate information based on the vehicle 101 through the sensor unit 220 and the camera 210 .

According to an embodiment, the processor 260 may obtain lane information on a lane of a road on which the vehicle 101 travels. In the driving image obtained through the camera 210 , the processor 260 determines a first inner reference line of a left lane with respect to the vehicle 101 (eg, a line closest to the left side of the vehicle 101 ); A second inner reference line of the right lane (eg, the line closest to the right side of the vehicle 101 ), a vanishing point at which the first inner reference line and the second inner reference line meet, and the first inner reference line and the second inner reference line It is possible to obtain lane information including at least a portion of a lane width between the lanes, a ratio of a vehicle width crossing the lane, coordinates of the vanishing point, and a left and right margin of the vanishing point. The processor 260 may obtain lane information on a lane of a road on which the vehicle 101 travels through the sensor unit 220 and the camera 210 .

According to an embodiment, the processor 260 performs at least one section for determining whether the vehicle 101 departs from a lane based on the road coordinate information of the vehicle 101 and the lane information as a driving image. (eg coordinates) can be set. At least one section set in the driving image may or may not be displayed on the display unit 250 . The processor 260 may determine whether or not the vehicle 101 deviated from the lane according to whether or not the lane exists within the set section.

According to an embodiment, the processor 260 may identify a vanishing point where a left lane and a right lane of the vehicle 101 contact each other in the driving image acquired through the camera 210 . In general, information far away in the front direction (eg, Z direction) of the vehicle 101 is collected at a point on the screen, that is, a vanishing point. The processor 260 may set screen coordinates in the driving image based on the vanishing point. In addition, the processor 260 may set a margin (or a predetermined interval) to the left and right of the vanishing point in the driving image.

According to an embodiment, the processor 260 may determine the margin by identifying the weight of the occupant of the vehicle 101 , the curvature of the lane, and the shaking of the vehicle 10 .

According to an embodiment, the processor 260 is configured to configure the width (eg: A midpoint (eg, the position of the camera 210 ) of the vehicle 101 ) may be set as a reference position. The processor 260 may set a section having a first predetermined interval to the left and right with respect to the reference position as a complete invasion section. The first predetermined interval corresponds to the width of the vehicle 101 at a point corresponding to 1/2 of the width of the vehicle 101 with respect to the reference position and the width of the vehicle 101 that invades the lane. It may be an interval obtained by subtracting a value multiplied by a ratio. The first predetermined interval may be variably adjusted according to the size, driving direction, height, and width of the vehicle. For example, if a vehicle (eg, the vehicle's left wheel) travels beyond the center lane, this may be considered complete intrusion.

According to an embodiment, the processor 260 may set a point where each of the first inner reference line and the second inner reference line meet a horizon in the obtained driving image as a first horizontal coordinate. The processor 260 may set a section having a second predetermined interval to the left and right of the complete invasion section as a temporary invasion section. The second predetermined interval may be an interval obtained by subtracting the first predetermined interval in the direction of the reference position from a point corresponding to 1/2 of the width of the vehicle. The second predetermined interval may be variably adjusted according to the size, driving direction, height, and width of the vehicle. For example, when the left wheel of the vehicle travels while touching the center lane, this may be referred to as a temporary intrusion. The complete encroachment and the temporary encroachment may be classified according to a difference in the degree to which the wheel of the vehicle travels over a lane.

For example, driving with the left wheel of a vehicle in contact with a lane is referred to as temporary intrusion, and driving with the left wheel of the vehicle crossing the lane may be referred to as complete intrusion.

For example, the temporary encroachment and the complete encroachment are distinguished by a ratio of vehicle wheels crossing an inner reference line of a lane, and this ratio may be variably adjusted.

According to an embodiment, the processor 260 may set a point where each of the first inner reference line and the second inner reference line and the lower end of the obtained driving image intersect as the second horizontal coordinates. The processor 260 may set a section that deviates from the temporary intrusion section to the left and right as a normal section.

According to an embodiment, the processor 260 may determine whether the vehicle departs from a lane based on the set at least one section.

According to an embodiment, the processor 260 identifies a vanishing point where the first inner reference line of the left lane with respect to the vehicle and the second inner reference line of the right lane are in contact in the driving image, and to the left and right of the identified vanishing point. You can set margins. In addition, the processor 260 may set a midpoint of the width of the vehicle at the lower end of the obtained driving image as a reference position.

Also, the processor 260 may set a point where each of the first inner reference line and the second inner reference line meet a horizon in the obtained driving image as a first horizontal coordinate.

Also, the processor 260 may set a point where each of the first inner reference line and the second inner reference line and the lower end of the obtained driving image intersect as second horizontal coordinates.

According to an embodiment, the processor 260 sets a first section having a predetermined interval to the left and right of the reference position as a complete invasion section, and temporarily invades a second section having a predetermined interval to the left and right of the complete invasion section It can be set as a section. In addition, the processor 260 may set the third section that deviates from the temporary intrusion section to the left and right as the normal section.

According to an embodiment, when the vanishing point is located within the set margin and the left or right coordinates of the set second horizontal coordinates exist in the first section, the processor 260 is configured to move the vehicle to the first inner side. It may be determined that the reference line or the second inner reference line is completely violated.

According to an embodiment, when the vanishing point is located within the set margin and the left or right coordinates of the set second horizontal coordinates exist in the second section, the processor 260 is configured to control the vehicle in the first section. It may be determined that the inner reference line or the second inner reference line is temporarily violated.

According to an embodiment, when the vanishing point is located within the set margin and the set left or right coordinates of the second horizontal coordinates exist in the third section, the processor 260 determines that the vehicle is running normally. can be judged as

According to an embodiment, when the vanishing point is not located within the set margin and the left coordinate or right coordinate of the set second horizontal coordinate exists in the first section, the processor 260 determines that the vehicle The first inner reference line or the second inner reference line may be completely violated and it may be determined that the vehicle is abnormally driven.

According to an embodiment, when the vanishing point is not located within the set margin and the left or right coordinates of the set second horizontal coordinates exist in the second section, the processor 260 determines that the vehicle The first inner reference line or the second inner reference line may be temporarily violated and it may be determined that the vehicle is driving abnormally.

According to an embodiment, the processor 260 acquires a driving image of the vehicle through the camera 210, and based on at least one section set in the acquired driving image, the left side of the lane in which the vehicle is driving It can determine whether the lane is a center lane or a general lane (eg, a white line that separates lanes).

According to an embodiment, when the left lane is the center lane, the processor 260 may determine that the vehicle is normally driving within the lane. In the normal driving, the coordinates of the vanishing points of the left and right lanes are within the margin while the vehicle is driving in the lane, and the left and right lanes (eg, the left coordinates or the right coordinates of the second horizontal coordinates) are the third When it exists in the section, it may be determined that the vehicle is driving normally.

According to an embodiment, the processor 260 identifies whether the vehicle is driving completely encroaching on the left lane, and after the vehicle has completely encroached on the vehicle, when it is identified that the right lane of the vehicle is the center lane, It may be determined that the vehicle is traveling in the opposite lane. The processor 260 determines that the vehicle is currently traveling in the opposite lane when the center lane located on the right side of the vehicle is identified while the vehicle is driving in the opposite lane after the vehicle completely invades the left lane. can In this case, the driver may intentionally cross the center lane.

According to an embodiment, the processor 260 acquires a driving image of the vehicle through the camera 210, and based on at least one section set in the acquired driving image, both sides of the lane in which the vehicle is driving It can be determined whether the lane is a normal lane. For example, when the both lanes are normal lanes, the processor 260 may determine that the vehicle is driving normally in the lane.

According to an embodiment, the processor 260 identifies whether the vehicle is driving completely encroaching on the right lane among both lanes, and after the vehicle is completely encroached on, the right lane of the vehicle is identified as the center lane. , it may be determined that the vehicle is traveling in the opposite lane. In this case, it may be a case in which the driver crosses the center lane due to inability to distinguish directions.

According to an embodiment, the processor 260 may identify whether a lane is recognized or not recognized in the driving image obtained through the camera 210 . After determining that the lane is not recognized, the processor 260 may determine whether a right lane of the vehicle is identified as a center lane.

According to an embodiment, when the right lane is identified as the center lane, the processor 260 may determine that the vehicle is traveling in the opposite lane. In this case, a foreign driver (eg, a Japanese driver) who is not familiar with right-hand traffic may enter the road in the opposite direction.

In this way, after it is identified that the vehicle is currently traveling in the opposite lane, the processor 260 determines that the vehicle is traveling in the opposite lane for a predetermined period of time (eg, the vehicle is traveling in the opposite lane). A driving image corresponding to about 30 seconds before and after one time) may be stored in the storage unit 240 .

According to an embodiment, the processor 260 determines that, based on at least one section set in the driving image acquired through the camera 210 , when the left lane of the lane in which the vehicle is traveling is the center lane, the vehicle It can be determined that the vehicle is driving normally in the vehicle.

In addition, the processor 260 may identify whether the vehicle completely invades the left lane. Then, after the vehicle completely invades the left lane and drives, the processor 260 determines a general lane (eg, a white line dividing lanes) of an opposite lane (eg, a lane in which the vehicle travels in the opposite direction). It can be determined whether there is a complete invasion of

In this way, after the vehicle completely invades the left lane (eg, the center lane) and then completely invades the general lane (eg, the white lane) of the opposite lane, it may be referred to as abnormal driving.

According to an embodiment, when a center lane is identified based on the abnormal driving, the processor 260 may determine that the vehicle is driving by making a U-turn in the prohibited section. In general, the section where U-turns can be made is the section where the center lane changes from a double solid yellow line to a dotted white line. A U-turn in the prohibited section means making a U-turn on a double solid yellow line, not on a white dotted line.

According to an embodiment, after the vehicle completely invades the left lane (eg, the center lane), the processor 260 determines whether a general lane (eg, a white lane) on the right side of the vehicle is recognized in the opposite lane. can In addition, when the general lane is recognized, the processor 260 may identify whether the vehicle is driving normally by completely encroaching on the general lane.

According to an embodiment, if the general lane of the opposite lane is not recognized after the vehicle has completely invaded the vehicle, the processor 260 provides information on the change in the traveling direction of the vehicle and the rotation angle through the sensor unit 220 . At least some of the information on

For example, the processor 260 may obtain information about a change in the traveling direction of the vehicle from the GPS in the sensor unit 220, and obtain information about the rotation angle of the vehicle from a yaw sensor (not shown). there is.

According to an embodiment, the processor 260 may determine that the vehicle is driving abnormally because it completely invades the general lane based on the at least part of the obtained information.

According to an embodiment, the processor 260 determines that the left lane of the vehicle is the center lane, and when it is determined that the vehicle is driving normally in the opposite lane, it is determined that the vehicle is driving by making a U-turn in the prohibited section can do.

According to an embodiment, after the processor 260 identifies that the left lane of the vehicle is the center lane, if the left lane of the vehicle is maintained as the center lane for a predetermined time, the vehicle makes a U-turn in the prohibited section Thus, it can be judged that it is driving.

As such, when it is identified that the vehicle is driving by making a U-turn in the prohibited section, the processor 260 determines that the vehicle is driving after making a U-turn in the prohibited section for a predetermined time (eg, the vehicle is in the prohibited section). The driving image corresponding to the driving time (for about 30 seconds before and after the U-turn in the ) may be stored in the storage unit 240 .

According to an embodiment, the processor 260 may transmit the driving image stored in the storage unit 240 to the server through the communication unit 230 . For example, the processor 260 may generate a driving image corresponding to a predetermined time based on the time at which the lane is invaded, a driving image corresponding to a predetermined time based on the time when the vehicle invades the center lane, and the vehicle moving into the opposite lane. At least one of a driving image corresponding to a predetermined time based on the driving time and a driving image corresponding to a predetermined time based on the time when the vehicle makes a U-turn in the prohibited section may be transmitted through the communication unit 230 .

According to an embodiment, the processor 260 may periodically identify whether a communication channel with an external electronic device (not shown) (eg, a server collecting vehicle driving information) is set up. For example, when the communication channel is set up, the processor 260 stores a driving image (eg, a lane) stored in the storage unit 240 through the set communication channel (eg, 3G, 4G, WiFi, Bluetooth, etc.) An image indicating violation, reverse driving, center line violation, U-turn in a prohibited section, etc.) may be transmitted to the external electronic device.

According to an embodiment, the processor 260 may sequentially analyze in real time images continuously acquired from the camera 210 mounted on the front of the vehicle. The processor 260 can consistently recognize and track a lane. The tracking means recognizing that the lane recognized in the previous frame is the same lane even if the positions of the lane recognized in the current frame are different.

According to an embodiment, when it is determined that the vehicle invades the center lane, the processor 260 stores a driving image corresponding to a predetermined time (eg, 15 seconds) before and after the time when the vehicle invades the center lane. ) can be stored in The stored driving image may be stored in an area where overwriting is prohibited and then transmitted to the server.

3 is a flowchart illustrating a method of setting at least one section for determining a driving state of a vehicle in a driving image according to an embodiment of the present invention. 4 is an exemplary diagram illustrating a relationship between actual road coordinates for a lane and coordinates of a lane in an image of a road projected by a camera according to an embodiment of the present invention. 5A is an exemplary view in which a vehicle travels in the center of a lane according to an embodiment of the present invention. 5B is an exemplary diagram illustrating a vanishing point in FIG. 5A . 6A is an exemplary view in which a vehicle travels while touching a left lane according to an embodiment of the present invention. FIG. 6B is an exemplary view illustrating a vanishing point in FIG. 6A . 7A is an exemplary view in which a vehicle travels while touching a right lane according to an embodiment of the present invention. 7B is an exemplary view showing the vanishing point in FIG. 7A. 8A is an exemplary view in which a vehicle travels while crossing a left lane according to an embodiment of the present invention. 8B is an exemplary diagram illustrating a vanishing point in FIG. 8A . 9 is an exemplary view illustrating at least one section set in a driving image to determine whether a vehicle departs from a lane according to an embodiment of the present invention.

Hereinafter, a method of setting at least one section for determining a driving state of a vehicle in a driving image according to an embodiment of the present invention will be described in detail with reference to FIGS. 3 to 8B .

According to an embodiment, the device 200 (eg, the processor 260 ) may identify whether a driving image of the vehicle is acquired ( S310 ). The device 200 (eg, the processor 260 ) determines whether an image of the front and/or rear of the vehicle 101 captured by at least one camera 210 included in the device 200 is acquired can do.

According to an embodiment, the device 200 (eg, the processor 260 ) may acquire an image or an image regarding the driving state of the vehicle 101 through the camera 210 . The device 200 (eg, the processor 260 ) captures an image of a lane of the road on which the vehicle 101 is traveling through the camera 210 disposed in front and/or rear of the vehicle 101 . It can be obtained in real time.

According to an embodiment, the device 200 (eg, the processor 260 ) may obtain information on the road coordinates based on the vehicle 101 ( S312 ). The device 200 (eg, the processor 260 ) determines the actual coordinates (eg, X, Y, Z) (or (X, 0, Z)) of the lane on which the vehicle 101 is currently traveling, the At least part of the coordinates (x, y) of the lane for the image projected through the camera, the focal length of the camera 210, and the distance between the lane and the camera 210 (eg, the height of the camera in the lane) coordinate information to be identified or obtained.

Also, the coordinates of a point on the road can be expressed as (X, 0, Z) because the Y-axis coordinate value is 0. The two-dimensional coordinates (X, Z) of the road may be expressed as two-dimensional coordinates (x, y) on the image projected by the camera, and accordingly, the location of the actual lane may be reversely known through the image.

[Vehicle-based road coordinate information]

Referring to FIG. 4 , the vehicle 101 is traveling in the Z-axis direction. There are two solid lines 131 on the left side of the vehicle 101 and one solid line 132 on the right side of the vehicle 101 . In this case, the device 200 (eg, the processor 260 ) may acquire an image about the road through the camera 210 . Then, the device 200 (eg, the processor 260 ) analyzes the lane coordinates (x, y) of the obtained image, and the actual coordinates (X, 0, Z) of the lane on which the vehicle 101 is traveling ) can be obtained. Also, the device 200 (eg, the processor 260 ) may identify the height h of the camera 210 disposed inside the vehicle 101 . The height of the camera 210 may be a value input by measuring the actual height after mounting the camera 210, or a value obtained and stored through a correction procedure after installing a prescribed correction plate according to a predetermined procedure in front of the vehicle there is. Alternatively, the height of the camera 210 may be obtained through a value in a table made by accumulating previous installation experience according to a vehicle type and a mounting position. For example, when the vehicle 101 is traveling in the Z-axis direction, the camera coordinates may be (0, h, 0).

Also, the coordinates of a point on the road can be expressed as (X, 0, Z) because the Y-axis coordinate value is 0. The two-dimensional coordinates (X, Z) of the road may be expressed as (x, y) in the image projected through the camera, and accordingly, the location of the actual lane may be reversed through the image.

According to an embodiment, the device 200 (eg, the processor 260 ) may obtain the coordinates of the lane shown in the image in which the road is projected by the camera through [Equation 1] below.

는 카메라(210)의 초점 거리를 나타내고, h는 차로에서 카메라(210)의 높이를 나타낸다.In [Equation 1], x and y represent the coordinate values of the lane in the image where the road is projected to the camera, and X and Z represent the actual coordinate values of the lane on which the vehicle 210 is traveling,

denotes a focal length of the camera 210 , and h denotes a height of the camera 210 in the lane.

Referring back to FIG. 3 , the device 200 (eg, the processor 260 ) may obtain information on a lane in which the vehicle travels ( S314 ). The device 200 (eg, the processor 260 ) acquires the driving image captured by the camera 210 , and in the acquired driving image, the first inner reference line of the left lane with respect to the vehicle 101 . (eg, the line closest to the left side of the vehicle 101), the second inner reference line of the right lane (eg, the line closest to the right side of the vehicle 101), the first inner reference line and the second inner reference line Lane information including at least a portion of the vanishing point, the lane width between the first inner reference line and the second inner reference line, the ratio of the vehicle width crossing the lane, the coordinates of the vanishing point, and the left and right margins of the vanishing point can be obtained there is. The device 200 (eg, the processor 260 ) may obtain lane information on a lane of a road in which the vehicle 101 travels through the sensor unit 220 and the camera 210 .

[Central driving]

According to an embodiment, the device 200 (eg, the processor 260 ) may identify or obtain information on a lane of a road on which the vehicle 101 is currently traveling.

5A and 5B , in a state 510 in which the vehicle 101 is driving in the middle of a lane, the camera 210 acquires the road at a location 511 of the vehicle 101 . Let Z_bottom be the distance to the points 512 on the nearest road displayed in the image projected by the camera 210, and the distance from the vanishing point 521 to the bottom 522 of the screen 520 is y_bottom. In this case, the following [Equation 2] may be established.

는 카메라(210)의 초점 거리를 나타내고, h는 차로에서 카메라(210)의 높이를 나타낸다. 상기 장치(200)(예: 프로세서(260))는 상기 [수학식 2]를 통해 도로가 카메라로 투영된 영상에 표시되는 가장 가까운 도로 위 지점들(512)까지의 거리(Z_bottom)와 상기 소실점(521)에서 화면(520) 하단(522)까지의 거리(y_bottom)를 획득할 수 있다.In the above [Equation 2]

denotes a focal length of the camera 210 , and h denotes a height of the camera 210 in the lane. The device 200 (eg, the processor 260 ) calculates the distance Z_bottom and the vanishing point to the nearest points 512 on the road displayed in the image projected with the camera through Equation 2 above [Equation 2] A distance (y_bottom) from 521 to the bottom 522 of the screen 520 may be obtained.

In addition, the length (the sum of 523 and 524) (w_lane) between two points where two parallel lanes 525 and 526 intersect with the lower end of the screen 520 is proportional to the actual distance between the two lanes on the road (W_lane) could be a relationship.

For example, when the camera 210 is installed in a vehicle, the distance on the road at which the road is formed at the lower end of the image projected by the camera 210 may be fixed. Accordingly, when the vehicle 101 drives in a lane with a constant width, the length w between two points where the inner reference lines of both lanes 525 and 526 intersect the lower end of the image is constant, and this is width parameter). In the device 200 (eg, the processor 260 ), such a _lane width index may be obtained through Equation 3 below.

[Drive while touching the left lane]

Referring to FIGS. 6A and 6B , when the vehicle 101 is driving in parallel with the lane 611 in contact with any one of the left and right lanes (eg, the left lane 611) (610), two Extensions of lanes 611 , 612 , or 625 , 626 still meet at vanishing point 621 , as shown in FIG. 6B 820 . Since the camera 210 is located in the center of the inside of the vehicle 101, the point where the inner reference line of the lane the vehicle 101 touches and the lower end of the image 622 intersect is the vehicle 101 based on the vanishing point 621. ) is half (1/2) of the width.

As such, when the vehicle 101 travels in contact with the left lane 611 , the length 623 corresponding to the width (1/2) of the vehicle in the image projected by the camera is w_vehicle, left/2 can be expressed as

[Drive while touching the right lane]

Referring to FIGS. 7A and 7B , when the vehicle 101 is driving in parallel with the lane 712 while in contact with any one of the left and right lanes (eg, the right lane 712) (710), two Extensions of lanes 711 , 712 , or 725 , 726 still meet at vanishing point 721 as shown in FIG. 7B 720 . Since the camera 210 is located in the center inside the vehicle 101 , the point where the inner reference line of the lane that the vehicle 101 touches and the lower end of the image 722 intersect is the vehicle 101 based on the vanishing point 721 . ) is half (1/2) of the width.

As such, when the vehicle 101 travels in contact with the right lane 712, the length 724 corresponding to the width (1/2) of the vehicle in the image projected by the camera 210 is w_vehicle, It can be expressed as right/2.

The length corresponding to the width of the vehicle 101 in the image may be obtained through [Equation 4] below, and the width of the vehicle may be referred to as a vehicle width parameter. And, the sum of w_vehicle, left/2 and w_vehicle, right/2 can be called w_vehicle. In this way, the device 200 (eg, the processor 260 ) may calculate the vehicle width w _vehicle through Equation 4 below.

[Driving over the left lane]

8A and 8B, when the vehicle 101 crosses the left lane 811 and travels side by side with the lane 810, the extension line of the two lanes 811, 812, or 825, 826 is shown in FIG. 8B ( 820 ) still meet at the vanishing point 821 . And, if the ratio to the width of the vehicle 101 crossing the left lane 811 is α (the ratio to the width of the vehicle 101 crossing the right lane 812 is referred to as β), the left lane ( The point at which 811 intersects the lower end 822 of the image is closer to the vanishing point 821 by _{αw_vehicle} 823 than when the vehicle 101 drives while touching the left lane 811 . For example, when 20% of the vehicle 101 crosses the left lane 811 , α is 0.2.

Also, lengths 823 and 824 corresponding to the width 1/2 of the vehicle in the image of the road projected by the camera 210 may be expressed as w_vehicle, left/2 and w_vehicle and right/2, respectively.

In this way, the device 200 (eg, the processor 260 ) may acquire information on the lane in which the vehicle 101 travels through the above description. The lane information includes, in the obtained driving image, a vanishing point where a first inner reference line of a left lane, a second inner reference line of a right lane, and the first inner reference line and the second inner reference line are in contact with the vehicle 101, and a lane width between the first inner reference line and the second inner reference line, a ratio of a vehicle width crossing the lane, coordinates of the vanishing point, and a left and right margin of the vanishing point.

Referring back to FIG. 3 , the device 200 (eg, the processor 260 ) determines that the vehicle 101 departs from a lane based on the obtained vehicle 101 reference road coordinate information and the lane information. At least one section for determining whether or not there is may be set in the driving image (S316).

According to an embodiment, the device 200 (eg, the processor 260 ) may set a margin based on a vanishing point in the driving image acquired in the process S310 . The margin may be set based on at least one of a weight of an occupant of the vehicle 101 , a bending state of a lane, and shaking of the vehicle 101 .

According to an embodiment, the device 200 (eg, the processor 260 ) is based on a midpoint of the width w of the vehicle 101 at the lower end of the driving image acquired through the camera 210 . position can be set. The device 200 (eg, the processor 260 ) completely invades a section having a first predetermined interval to the left and right with respect to the reference position (ie, the left wheel or the right wheel of the vehicle 101 completely covers the lane). section) to identify what has been crossed can be set in the driving video. The first predetermined interval is the width w of the vehicle 101 at a point corresponding to 1/2 of the width of the vehicle 101 with respect to the reference position and the width w of the vehicle 101 that invades the lane. It is the interval minus the value multiplied by the ratio to the width.

According to an embodiment, the device 200 (eg, the processor 260 ), with respect to the vehicle, for each of the first inner reference line of the left lane and the second inner reference line of the right lane with respect to the vehicle, in the obtained driving image, A point that meets the horizon (or vanishing point) of may be set as the first horizontal coordinate. The device 200 (eg, the processor 260 ) sets a section having a second predetermined interval to the left and right of the complete intrusion section in a temporary intrusion section (ie, while the left wheel or right wheel of the vehicle 101 is in contact with the lane) section to identify driving) can be set in the driving image. The second predetermined interval is an interval obtained by subtracting the first predetermined interval in the direction of the reference position from a point corresponding to 1/2 of the width of the vehicle 101 .

According to an embodiment, the device 200 (eg, the processor 260 ) sets a point where each of the first inner reference line and the second inner reference line and the lower end of the acquired driving image intersect in the second horizontal direction. It can be set by coordinates. The device 200 (eg, the processor 260 ) determines that the section deviating from the temporary intrusion section to the left and right is a normal section (that is, the section in which the vehicle 101 is driving in the center of the lane without touching the lane) can be set in the driving video.

Referring to FIG. 9 , the device 200 (eg, the processor 260 ) may map a relationship between a lane and a vehicle using image coordinates. The device 200 (eg, the processor 260) sets a reference position 922 corresponding to 1/2 of the vehicle width at the lower end of the image based on the coordinates of the vanishing point 921, and the reference position 922 Temporary encroachment sections 914 and 915 based on the vehicle width w _vehicle in the direction of the vanishing point 921 based on can be set in the driving image. The device 200 (eg, the processor 260 ) may differently set the temporary intrusion sections 914 and 915 according to the left or right side based on the type of lane or the vehicle.

Then, the device 200 (eg, the processor 260) identifies the weight of the occupant of the vehicle, the curvature state of the lane, and the shaking of the vehicle, the weight of the occupant of the vehicle, the curvature state of the lane, and based on at least one of the shaking of the vehicle, a margin 918 (w _vp ) may be set in the driving image around the vanishing point 921 . And, since the distance 911 (w _lane ) between the two lanes is greater than the vehicle width (w _vehicle ), the two lanes cannot be in the encroaching section at the same time.

For example, if the horizontal coordinate of a point where the extension line of two lanes meets the horizon is x_t(top), and the horizontal coordinate of the point where each lane intersects the bottom of the image is x_b(bottom), the device 200 ) (eg, the processor 260 ) may define the position and driving state of the vehicle and the lane (eg, the center lane) using x_t and x_b.

In this way, the device 200 (eg, the processor 260) determines the width ( For example, a midpoint (eg, the position of the camera 210 ) of the width of the vehicle 101 may be set as the reference position. Then, the processor 260 determines the width of the vehicle 101 at a point corresponding to 1/2 of the width of the vehicle 101 with respect to the reference position and the width of the vehicle 101 that invades the lane. A section having an interval obtained by subtracting a value multiplied by a ratio of the corresponding values may be set in the driving image as the complete intrusion sections 912 and 913 .

In addition, the device 200 (eg, the processor 260 ) sets a point where each of the first inner reference line and the second inner reference line and the horizon in the obtained driving image meet as the first horizontal coordinates. can The device 200 (eg, the processor 260 ) temporarily invades a section 914 having an interval obtained by subtracting the first predetermined interval from a point corresponding to 1/2 of the width of the vehicle in the direction of the reference position. , 915) can be set in the driving video.

Also, the device 200 (eg, the processor 260 ) may set a point where each of the first inner reference line and the second inner reference line and the lower end of the obtained driving image intersect as the second horizontal coordinates. there is. The device 200 (eg, the processor 260 ) may set a section that deviates from the temporary intrusion sections 914 and 915 to the left and right as the normal sections 916 and 917 in the driving image.

10 is a flowchart illustrating a process of storing and transmitting a driving image by determining whether a vehicle departs from a lane according to an embodiment of the present invention. 11 is an exemplary view illustrating normal driving in a lane according to an embodiment of the present invention. 12A is an exemplary diagram of driving while temporarily infiltrating a left lane according to an embodiment of the present invention. 12B is an exemplary diagram of driving while temporarily infiltrating a right lane according to an embodiment of the present invention. 13A is an exemplary view of driving while completely encroaching on the left lane according to an embodiment of the present invention. 13B is an exemplary view of driving while completely encroaching on the right lane according to an embodiment of the present invention. 14A is an exemplary diagram illustrating abnormal driving by temporarily infiltrating a left lane according to an embodiment of the present invention. 14B is an exemplary diagram illustrating abnormal driving by temporarily infiltrating a right lane according to an embodiment of the present invention. 15A is an exemplary diagram illustrating abnormal driving by completely encroaching on the left lane according to an embodiment of the present invention. 15B is an exemplary diagram illustrating abnormal driving by temporarily infiltrating a right lane according to an embodiment of the present invention.

Hereinafter, a process of storing and transmitting a driving image by determining whether a vehicle departs from a lane according to an embodiment of the present invention will be described in detail with reference to FIGS. 10 to 15B .

According to an embodiment, the processor 260 may acquire a driving image of the vehicle ( S1010 ). The process ( S1010 ) may include at least one operation performed in the process ( S310 ) of FIG. 3 .

According to an embodiment, the processor 260 may obtain vehicle-based road coordinate information and lane information (S1012). The process S1012 may include at least one operation performed in the processes S312 and S314 of FIG. 3 .

According to an embodiment, the processor 260 may set at least one section for determining whether to depart a lane in the driving image (S1014). The process ( S1014 ) may include at least one operation performed in the process ( S316 ) of FIG. 3 .

According to an embodiment, the processor 260 may determine whether the vehicle departs from a lane (S1016). The processor 260 may determine whether the vehicle departs from the lane after setting at least one section for determining whether the vehicle departs from the lane in the driving image. For example, based on the at least one section set in the driving image, the processor 260 determines whether the vehicle is driving in a lane, whether it is a normal driving in the lane, whether it is temporarily intrusive driving, whether it is completely intrusion normal driving, or whether it is temporarily intrusive abnormal driving, It can be determined whether the driving is completely intrusive or abnormal.

According to an embodiment, the processor 260 determines that the vehicle is driving normally when the vanishing point is located within a preset margin and the left or right coordinates of the second horizontal coordinates exist in the third section. can do.

Normal driving means that the vehicle travels along the lane. Such normal driving may include normal driving in the lane and normal driving temporarily intrusive. In addition, the temporary invasion normal driving may include a left temporary invasion normal driving and a right temporary invasion normal driving. The left temporary invasion normal driving and the right temporary invasion normal driving may be determined based on a lane occupancy ratio of tires of the vehicle.

Referring to FIG. 11 , when the vanishing point 921 is within the margin 918 and the lanes 1111 and 11112 are both within the normal sections 916 and 917 ( 1110 ), the processor 260 determines that the vehicle is within the lane. can be judged to be running normally. For example, if the left lane 1111 is within the range of the first normal section 916 and the right lane 1112 is within the range of the second normal section 917, the processor 260 determines that the vehicle is within the lane. It can be judged that the vehicle is driving normally.

According to an embodiment, the processor 260 may determine whether the vehicle is normally driving within a lane through Equation 5 below.

In [Equation 5], x _tl is the coordinate of the vanishing point of the right lane, x _tr is the coordinate of the vanishing point of the left lane, and w _vp is the margin. And, x _bl is a coordinate at which the left lane 1111 intersects the lower end of the image 1110 , and x _br is a coordinate at which the right lane 1112 intersects the lower end of the image 1110 . And, w _vehicle represents the width of the vehicle.

Referring to FIG. 12A , when the vanishing point 921 is within the margin 918 , the left lane 1211 is within the temporary intrusion section 914 , and the right lane 1212 is within the normal section 917 ( 1210 ) , the processor 260 may determine that the vehicle travels by temporarily infiltrating the left lane 1211 .

According to an embodiment, the processor 260 may determine whether the vehicle temporarily invades the left lane 1211 and travels through Equation 6 below.

In [Equation 6], x _tl is the coordinate of the vanishing point of the right lane, x _tr is the coordinate of the vanishing point of the left lane, and w _vp is the margin. And, x _bl is a coordinate at which the left lane 1211 intersects the lower end of the image 1210 . And, w _vehicle represents the width of the vehicle, and α is the ratio of determining temporary invasion and complete invasion. For example, α represents the ratio at which the left tire of the vehicle invades the left lane 1211 .

Referring to FIG. 12B , when the vanishing point 921 is within the margin 918 , the right lane 1222 is within the temporary intrusion section 915 , and the left lane 1221 is within the normal section 916 ( 1220 ) , the processor 260 may determine that the vehicle travels by temporarily infiltrating the right lane 1222 .

According to an embodiment, the processor 260 may determine whether the vehicle temporarily invades the right lane 1222 and travels through Equation 7 below.

In [Equation 7], x _tl is the coordinate of the vanishing point of the right lane, x _tr is the coordinate of the vanishing point of the left lane, and w _vp is the margin. And, x _br is a coordinate at which the right lane 1222 intersects the lower end of the image 1220 . And, w _vehicle represents the width of the vehicle, and β is the ratio of determining temporary invasion and complete invasion. For example, β represents the ratio at which the right tire of the vehicle invades the right lane 1222 .

Referring to FIG. 13A , when the vanishing point 921 is within the margin 918 , the left lane 1311 is within the full encroachment section 912 , and the right lane 1312 is within the normal section 917 ( 1310 ) , the processor 260 may determine that the vehicle completely invades the left lane 1211 and drives (eg, completely invades and drives normally).

According to an embodiment, the processor 260 may determine whether the vehicle travels completely in the left lane 1311 through Equation 8 below.

In [Equation 8], x _tl is the coordinate of the vanishing point of the right lane, x _tr is the coordinate of the vanishing point of the left lane, and w _vp is the margin. And, x _bl is a coordinate at which the left lane 1311 intersects the lower end of the image 1310 . And, w _vehicle represents the width of the vehicle, and α is the ratio of determining temporary invasion and complete invasion. For example, α represents the ratio at which the left tire of the vehicle invades the left lane 1311 .

Referring to FIG. 13B , when the vanishing point 921 is within the margin 918 , the right lane 1322 is within the full encroachment section 913 , and the left lane 1321 is within the normal section 916 ( 1320 ) , the processor 260 may determine that the vehicle completely invades the right lane 1322 and drives (eg, completely invades and drives normally).

According to an embodiment, the processor 260 may determine whether the vehicle travels by completely encroaching on the right lane 1322 through Equation 9 below.

In [Equation 9], x _tl is the coordinate of the vanishing point of the right lane, x _tr is the coordinate of the vanishing point of the left lane, and w _vp is the margin. And, x _br is a coordinate at which the right lane 1322 intersects the lower end of the image 1320 . And, w _vehicle represents the width of the vehicle, and β is the ratio of determining temporary invasion and complete invasion. For example, β represents the rate at which the right tire of the vehicle invades the right lane 1322 .

14A , when the vanishing point 921 is outside the margin 918 (eg, the second section 1432) and the left lane 1411 is within the temporary intrusion section 914 (1410), the processor ( 260 may determine that the vehicle temporarily invades the left lane 1411 and drives (eg, temporarily invades and drives abnormally).

According to an embodiment, the processor 260 may determine whether the vehicle temporarily invades the left lane 1411 and travels (eg, temporarily invades and drives abnormally) through Equation 10 below.

In [Equation 10], x _tl is the coordinate of the vanishing point of the right lane, and w _vp is the margin. And, x _bl is a coordinate at which the left lane 1411 intersects the lower end of the image 1410 . And, w _vehicle represents the width of the vehicle, and α is the ratio of determining temporary invasion and complete invasion. For example, α represents the ratio at which the left tire of the vehicle invades the left lane 1411 .

14B, when the vanishing point 921 is outside the margin 918 (eg, the first section 1431), and the right lane 1421 is within the temporary intrusion section 915 (1420), the processor ( 260 may determine that the vehicle temporarily invades the right lane 1421 and drives (eg, temporarily invades and drives abnormally).

According to an embodiment, the processor 260 may determine whether the vehicle temporarily invades the right lane 1421 and travels through Equation 11 below.

In [Equation 11], x _tr is the coordinate of the vanishing point of the left lane, and w _vp is the margin. And, x _br is a coordinate at which the right lane 1421 intersects the lower end of the image 1420 . And, w _vehicle represents the width of the vehicle, and β is the ratio of determining temporary invasion and complete invasion. For example, β represents the rate at which the right tire of the vehicle invades the right lane 1421 .

15A , when the vanishing point 921 is outside the margin 918 (eg, the second section 1432) and the left lane 1511 is within the completely encroached section 912 (1510), the processor ( 260 may determine that the vehicle completely invades the left lane 1511 and drives (eg, completely invades and drives abnormally).

According to an embodiment, the processor 260 may determine whether the vehicle completely invades the left lane 1511 and drives (eg, temporarily invades and drives abnormally) through Equation 12 below.

In [Equation 12], x _tl is the coordinate of the vanishing point of the right lane, and w _vp is the margin. And, x _bl is a coordinate at which the left lane 1511 intersects the lower end of the image 1510 . And, w _vehicle represents the width of the vehicle, and α is the ratio of determining temporary invasion and complete invasion. For example, α represents the ratio at which the left tire of the vehicle invades the left lane 1511 .

15B , when the vanishing point 921 is outside the margin 918 (eg, the first section 1431) and the right lane 1521 is within the completely encroached section 913 (1520), the processor ( 260 may determine that the vehicle completely invades the right lane 1521 and drives (eg, completely invades and drives abnormally).

According to an embodiment, the processor 260 may determine whether the vehicle completely invades the right lane 1521 and travels through Equation 13 below.

In [Equation 13], x _tr is the coordinate of the vanishing point of the left lane, and w _vp is the margin. And, x _br is a coordinate at which the right lane 1421 intersects the lower end of the image 1420 . And, w _vehicle represents the width of the vehicle, and β is the ratio of determining temporary invasion and complete invasion. For example, β represents the rate at which the right tire of the vehicle invades the right lane 1421 .

According to an embodiment, the processor 260 determines whether the vehicle is driving normally in the lane or temporarily intruding (eg, temporarily violating the left side or temporarily violating the right side) normal driving through [Equation 1] to [Equation 13]. , it is possible to determine whether the driving is normal, whether it is completely invaded (eg, completely invaded the left side or completely invaded the right side), whether it is an abnormal driving with temporary invasion, or abnormal driving with complete invasion.

According to an embodiment, the processor 260 may identify whether a lane is impossible. For example, since a lane is generally regarded as a long line in the driving direction of a vehicle, when a line crossing the screen is not recognized as a lane or the lane is not visible, the processor 260 determines that it is impossible to recognize a lane can do.

According to an embodiment, the processor 260 may store the driving image within a predetermined time based on the time when the lane is invaded ( S1018 ). When it is determined that the vehicle invades the lane based on the contents disclosed in FIGS. 11 to 15 , the processor 260 may store the driving image within a predetermined time in the storage unit 240 based on the time at which the lane is violated. there is.

For example, the processor 260 may store (eg, temporarily) the driving image acquired through the camera 210 in the storage unit 240 in real time. Thereafter, when it is determined that the vehicle invades the lane, a driving image corresponding to a predetermined time (eg, 30 seconds) before and after the determined time from among the stored driving images is determined by determining the time at which the vehicle has invaded the lane. can be stored separately. The separately stored driving image may not be deleted even if the storage capacity of the storage unit 240 is insufficient.

According to an embodiment, the processor 260 may identify whether a communication channel is connected (S1020). For example, when the vehicle is located adjacent to a garage (eg, an area where a server collecting vehicle driving information is located), the processor 260 may set up a communication channel with the server through the communication unit 230 . can The processor 260 may establish a communication channel (eg, 3G, 4G, WiFi, Bluetooth, etc.) with the server in order to transmit at least one image stored in the storage unit 240 to the server.

According to an embodiment, the processor 260 may transmit the stored driving image through the connected communication channel (S1022). The processor 260 may transmit at least one image stored in the storage unit 240 to the server through the set communication channel. The processor 260 may maintain the communication channel while at least one image stored in the storage unit 240 is transmitted to the server.

16 is an exemplary diagram illustrating a change in a GPS travel direction when a vehicle changes to a left lane according to an embodiment of the present invention.

Referring to FIG. 16 , when the vehicle drives by changing the lane to the left (eg, changing from the second lane to the first lane), the angle of the driving direction temporarily decreases and then returns to the original value ( 1610 ). When the original angle is equal to or less than the reference value, the processor 260 may determine that the driving direction is maintained. The peak value of the angle of the traveling direction may be different depending on the traveling speed. For example, the peak value may be small during high-speed driving and large during low-speed driving.

17 is an exemplary diagram illustrating a change in a GPS traveling direction when a vehicle makes a U-turn according to an embodiment of the present invention.

Referring to FIG. 17 , when the vehicle makes a U-turn, the rotation angle of the vehicle changes to -180 ^o . In this case, if the rotation angle falls below 0 ^o , roll-over may be performed based on 360 ^o . In consideration of this, when the angle change (eg, -180 ^o ) is less than or equal to the reference value, the processor 260 may determine that the driving direction is reversed (1710).

18 is an exemplary diagram illustrating a format of a GPRMC message and definitions of each field according to an embodiment of the present invention.

In general, a global positioning system (GPS) transmits data in a standard format called NMEA (National Marine Electronics Association)-0183. The NMEA standard is an electrical interface and data protocol defined for communication between marine electronic equipment and equipment. In the GPRMC (referred to as recommended minimum data) message of the NMEA-0183 protocol, the Course over ground fields 1810 and 1820 indicate the progress direction ³⁶⁰ degrees clockwise based on the true north direction.

For example, when the vehicle turns left, the angle decreases, and when the vehicle turns right, the angle increases (360↔ conversion).

According to an embodiment, the processor 260 may identify a driving direction (eg, rotation) of the vehicle using a GPS signal.

19 is an exemplary view for identifying the rotation of a vehicle according to an embodiment of the present invention. 20 is an exemplary diagram illustrating a yaw rate change when the vehicle changes a left lane according to an embodiment of the present invention. 21 is an exemplary view illustrating a yaw rate change when the vehicle makes a U-turn according to an embodiment of the present invention.

19, 20, and 21 , the vehicle 101 may rotate in the longitudinal direction (eg, Roll) based on the driving direction or rotate in the vertical direction 1930 (eg, Yaw). or may be rotated (eg, pitched) in the transverse direction 1910 . The vehicle 101 may be rotated in the longitudinal direction 1920 (eg, Roll), rotated in a vertical direction (eg, Yaw), and rotated in a transverse direction (eg, Pitch) in combination.

The movement in which the vehicle 101 rotates left and right is called yaw, and the amount of change in the angle of rotation in the left and right with time is called yaw rate. This yaw rate is output from the sensor unit 220 (eg, an inertial sensor). Alternatively, the yaw rate may be output from the sensor unit 220 built into the vehicle 101 (or the device 200), or may be output from an angular rotation rate sensor from an acceleration sensor for detecting an impact.

For example, when the vehicle 101 travels in a straight line, yaw is 0, and a case in which the vehicle 101 rotates to the right is indicated by (+) and a case in which the vehicle 101 rotates to the left is indicated by a (-) sign.

The processor 260 may calculate the change in the traveling direction of the vehicle 101 during the time period of interest by accumulating the yaw rate measured at a predetermined time interval Δt.

According to an embodiment, when the vehicle 101 changes a lane to the left while driving and maintains the original driving direction in the left lane (2010), the processor 260 changes the yaw rate to (-) and It is judged to be a (+) value. In addition, the processor 260 may determine that the driving direction is maintained when the accumulated rotation angle in this time section is less than or equal to a reference value.

According to an embodiment, when the vehicle 101 makes a U-turn, the accumulated rotation angle during the U-turn section is close to about -180 ^o (the accumulated rotation angle is +180 ^o ). The processor 260 may determine that the driving direction of the vehicle 101 is opposite when the accumulated rotation angle 2111 in this time period is equal to or less than the reference value ( 2110 ).

22 is a flowchart illustrating a process of identifying a vehicle intrusion into a center lane according to an embodiment of the present invention. 23 is an exemplary diagram illustrating a process of identifying a vehicle intrusion into a center lane according to an embodiment of the present invention.

Hereinafter, with reference to FIG. 22 , a process for identifying an intrusion of a center lane of a vehicle according to an embodiment of the present invention will be described in detail as follows.

According to an embodiment, the processor 260 may identify whether the vehicle is in a normal driving state in a lane (S2210). As shown in (a) of FIG. 23 , in the processor 260 , the vanishing point 2313 is within the margin 2314 , the center lane 2311 is within the normal section 2315 , and the general lane 2312 is When the vehicle is within the normal section 2316 ( 2310 ), it may be determined that the vehicle is normally driving within the lane. The processor 260 may determine whether the vehicle is normally driving within the lane through Equation 5.

According to an embodiment, the processor 260 may identify that the vehicle proceeds to a temporary intrusion driving state (S2212). As shown in (b) of FIG. 23 , in the processor 260 , the vanishing point 2323 is within the margin 2324 , the center lane 2321 is within the temporary intrusion section 2325 , and the general lane 2322 . When the vehicle is within the normal section 2326 ( 2320 ), it may be determined that the vehicle is traveling by temporarily infiltrating the center lane 2321 . The processor 260 may determine whether the vehicle temporarily invades the center lane 2321 and travels through [Equation 6] and [Equation 7].

According to an embodiment, the processor 260 may identify that the vehicle proceeds to a fully intrusive driving state (S2214). As shown in (c) of FIG. 23 , in the processor 260 , the vanishing point 2333 is within the margin 2334 , the center lane 2311 is within the completely encroached section 2335 , and the general lane 2332 . When the vehicle is within the normal section 2336 ( 2330 ), it may be determined that the vehicle completely invades the center lane 2331 and drives (eg, completely invades and drives normally). The processor 260 may determine whether the vehicle completely invades the center lane 2331 and travels through Equation 8 and Equation 9 above.

According to an embodiment, the processor 260 may identify whether the vehicle is driving for a predetermined period of time in a completely intrusive driving state (S2216). The processor 260 may identify that the processes S2210, S2212, and S2214 are sequentially generated. After the processes S2210, S2212, and S2214 are sequentially performed, the processor 260 may identify whether the vehicle is maintained in a completely intrusive driving state for a predetermined time (eg, 15 seconds).

According to an embodiment, the processor 260 may determine that the vehicle has invaded the center lane (S2218). After the processes S2210, S2212, and S2214 are sequentially generated, the processor 260 determines that the vehicle has invaded the center lane when the vehicle is maintained for a predetermined period of time (eg, 15 seconds) in a fully intrusive driving state. can be judged as

For example, when it is determined that the vehicle is continuously maintained for a predetermined period of time (eg, 15 seconds) in a completely intrusive state, the processor 260 may determine that the vehicle has invaded the center lane.

According to an embodiment, the processor 260 may store a driving image corresponding to a predetermined time based on the time when the vehicle invades the lane (S2220). The processor 260 is configured to operate in the center lane at a predetermined time (eg, at a time before crossing the center lane (eg 1:29:45 seconds) based on the time (eg 1:30) when the center lane is violated). The driving image acquired during the time (eg, 1:30:15) after the intrusion of the may be stored in the storage unit 240 . The predetermined time may be variably adjusted.

According to an embodiment, the processor 260 maintains the traveling direction of the vehicle through the sensor unit 220 (eg, GPS, yaw rate sensor) at any time during the processes S2210 to S2214. It can be identified whether

24 is a flowchart illustrating a process of identifying driving in an opposite lane of a vehicle according to an embodiment of the present invention. 25 is an exemplary diagram illustrating a process of identifying driving in an opposite lane of a vehicle according to an embodiment of the present invention.

24 and 25 are examples of identifying driving in the opposite lane when the driver intentionally crosses the center lane.

Hereinafter, with reference to FIGS. 24 and 25 , a process for identifying driving in the opposite lane of a vehicle according to an embodiment of the present invention will be described in detail as follows.

According to an embodiment, the processor 260 may identify that the vehicle is in a normal driving state in a lane in which the left side is the center lane (S2410).

As shown in (a) of FIG. 25 , the vanishing point 2513 is within the margin 2514 , the center lane 2511 is within the normal section 2515 , and the general lane 2512 is within the normal section 2516 . If there is (2510), the processor 260 may determine that the vehicle normally travels within the lane. The processor 260 may determine whether the vehicle is normally driving within the lane through Equation 5.

According to an embodiment, the processor 260 may identify that the vehicle proceeds to a completely intrusive driving state (S2412).

As shown in (b) of FIG. 25 , the vanishing point 2523 is within the margin 2524 , the center lane 2521 is within the full encroachment section 2525 , and the general lane 2522 is the normal section 2526 . In the case of being inside the vehicle ( 2520 ), the processor 260 may determine that the vehicle is driving while completely encroaching on the center lane 2321 . The processor 260 may determine whether the vehicle completely invades the center lane 2521 and travels through Equation 8 and Equation 9 above.

According to an embodiment, the processor 260 may identify that the vehicle is in a normal driving state in a lane in which the right side is the center lane ( S2414 ).

As shown in (c) of FIG. 25 , the vanishing point 2533 is within the margin 2534 , the general lane 2531 is within the normal section 2535 , and the center lane 2532 is within the normal section 2536 . If there is ( 2530 ), the processor 260 may determine that the vehicle is driving while completely infiltrating the center lane 2321 . In this case, the vehicle completely invades the center lane and travels in the reverse direction. The processor 260 may determine whether the vehicle completely invades the center lane 2521 and travels (eg, reverses driving) through [Equation 8] and [Equation 9].

According to an embodiment, the processor 260 may identify whether the vehicle travels in a normal driving state for a predetermined time in a lane whose right side is the center lane ( S2416 ). The processor 260 may identify that the processes S2410, S2412, and S2414 are sequentially generated. The processor 260 may identify whether the vehicle is maintained in a normal driving state for a predetermined period of time (eg, 15 seconds) after the processes S2410 , S2412 , and S2414 are sequentially performed.

According to an embodiment, the processor 260 may determine that the vehicle is traveling in the opposite lane (S2418). The processor 260 determines that after the processes S2410, S2412, and S2414 occur sequentially, when the vehicle is maintained in a normal driving state for a predetermined time (eg, 15 seconds), the vehicle is driving in the opposite lane can be judged as

For example, when it is determined that the vehicle is continuously maintained for a predetermined time (eg, 15 seconds) in a state in which the vehicle has invaded the center lane, it may be determined that the vehicle is traveling in the opposite lane.

According to an embodiment, the processor 260 may store a driving image corresponding to a predetermined time based on a time when the vehicle is driven in the opposite lane ( S2420 ). The processor 260 is configured to operate in the opposite lane at a predetermined time (eg, a time prior to intrusion of the center lane (eg 1:29:45)) based on the time (eg, 1:30) of driving in the opposite lane. The driving image acquired during the time (eg, 1:30:15 seconds) after driving to . The predetermined time may be variably adjusted.

26 is a flowchart illustrating a process of identifying driving in an opposite lane of a vehicle according to another embodiment of the present invention. 27 is an exemplary diagram illustrating a process of identifying driving in an opposite lane of a vehicle according to another embodiment of the present invention.

26 and 27 are examples of identifying driving in the opposite lane when the driver crosses the center lane due to inability to distinguish directions.

Hereinafter, with reference to FIGS. 26 and 27 , a process for identifying driving in an opposite lane of a vehicle according to another embodiment of the present invention will be described in detail as follows.

According to an embodiment, the processor 260 may identify that the vehicle is in a normal driving state in a lane in which the left and right sides are a normal lane ( S2610 ).

As shown in (a) of FIG. 27 , when the vanishing point 2713 is within the margin 2714 and the normal lanes 2711 and 2712 are respectively within the normal sections 2715 and 2716 ( 2710 ), the processor At 260 , it may be determined that the vehicle normally travels within the lane. The processor 260 may determine whether the vehicle is normally driving within the lane through Equation 5.

According to an embodiment, the processor 260 may identify that the vehicle proceeds to the right-side complete intrusion driving state (S2612).

As shown in (b) of FIG. 27, the vanishing point 2723 is within the margin 2724, the left general lane 2721 is within the normal section 2725, and the right general lane 2722 is completely encroached on ( 2726), the processor 260 may determine that the vehicle completely invades the center lane and normally travels within the lane. The processor 260 may determine whether the vehicle is driving normally by completely encroaching on the center lane through [Equation 9].

According to an embodiment, the processor 260 may identify that the vehicle is in a normal driving state in a lane in which the right side is a center lane (S2614).

As shown in (c) of FIG. 27 , the vanishing point 2733 is within the margin 2734 , the left general lane 2731 is within the normal section 2735 , and the right center lane 2732 is within the normal section 2736 . ), the processor 260 may determine that the vehicle is in a normal driving state in a lane in which the right side is a center lane. The processor 260 may determine whether the vehicle is driving normally by completely encroaching on the center lane through [Equation 5].

According to an embodiment, the processor 260 may identify whether the vehicle travels in a normal driving state for a predetermined time in a lane in which the right side is the center lane ( S2616 ). The processor 260 may identify that the processes S2610, S2612, and S2614 are sequentially generated. After the processes S2610, S2612, and S2614 are sequentially generated, the processor 260 identifies whether the vehicle is maintained in a normal driving state for a predetermined time (eg, 15 seconds) in a lane whose right side is the center lane. can do.

According to an embodiment, the processor 260 may determine that the vehicle is traveling in the opposite lane (S2618). The processor 260 determines that after the processes (S2610, S2612, and S2614) occur sequentially, when the vehicle is maintained in a normal driving state for a predetermined time (eg, 15 seconds), the vehicle is driving in the opposite lane can be judged as

For example, if it is determined that the vehicle is continuously traveling in the opposite lane for a predetermined time (eg, 15 seconds), the processor 260 may determine that the vehicle is traveling in the opposite lane.

According to an embodiment, the processor 260 may store a driving image corresponding to a predetermined time based on the time the vehicle travels in the opposite lane (S2620). The processor 260 is configured to operate in the opposite lane at a predetermined time (eg, a time prior to intrusion of the center lane (eg 1:29:45)) based on the time (eg, 1:30) of driving in the opposite lane. The driving image acquired during the time (eg, 1:30:15 seconds) after driving to . The predetermined time may be variably adjusted.

28 is a flowchart illustrating a process of identifying driving in an opposite lane of a vehicle according to another embodiment of the present invention. 29 is an exemplary diagram illustrating a process of identifying driving in an opposite lane of a vehicle according to another embodiment of the present invention.

28 and 29 are examples of identifying that a foreign driver (eg, a Japanese driver) who is not familiar with right-hand traffic enters the road in the opposite direction.

Hereinafter, with reference to FIGS. 28 and 29 , a process for identifying driving in an opposite lane of a vehicle according to another embodiment of the present invention will be described in detail as follows.

According to an embodiment, the processor 260 may identify whether a lane is in an unrecognizable state (S2810). While the vehicle is driving in the opposite lane, the processor 260 may not recognize the lane in the driving image obtained through the camera 210 .

As shown in (a) of FIG. 29 , the processor 260 may not recognize a lane because the lane is not included in the driving image 2910 obtained through the camera 210 .

According to an embodiment, the processor 260 may identify that the vehicle is in a normal driving state in a lane whose right side is a center lane (S2812). The processor 260 identifies a center lane on the right side of the vehicle based on the operation of the vehicle in a state in which it does not recognize a lane for a while (eg, a time before the vehicle travels in reverse and a lane in the opposite lane is identified). can do.

As shown in (a) of FIG. 29 , the vanishing point 2923 is within the margin 2924 , the left general lane 2911 is within the normal section 2925 , and the right center lane 2922 is within the normal section 2926 . ), the processor 260 may determine that the vehicle is driving normally in the lane (eg, driving along the lane in the opposite lane). The processor 260 may determine whether the vehicle is normally driving within the lane through Equation 5.

According to an embodiment, the processor 260 may identify whether the vehicle travels in a normal driving state for a predetermined time in a lane in which the right side is the center lane (S2814). The processor 260 may identify that the processes S2810 and S2812 are sequentially generated. After the processes S2810 and S2812 are sequentially generated, the processor 260 determines that the vehicle is in a normal driving state (eg, driving along a lane in the opposite lane) in a lane whose right side is the center lane for a predetermined time (eg, : 15 seconds) can be identified.

According to an embodiment, the processor 260 may determine that the vehicle is traveling in the opposite lane (S2816). After the processes S2810 and S2812 are sequentially generated, the processor 260 maintains the vehicle in a normal driving state (eg, driving along a lane in the opposite lane) for a predetermined time (eg, 15 seconds). , it may be determined that the vehicle is traveling in the opposite lane.

For example, if it is determined that the vehicle is continuously traveling in the opposite lane for a predetermined time (eg, 15 seconds), the processor 260 may determine that the vehicle is traveling in the opposite lane.

According to an embodiment, the processor 260 may store a driving image corresponding to a predetermined time based on the time the vehicle travels in the opposite lane (S2818). The processor 260 is configured to operate in the opposite lane at a predetermined time (eg, a time prior to intrusion of the center lane (eg 1:29:45)) based on the time (eg, 1:30) of driving in the opposite lane. The driving image acquired during the time (eg, 1:30:15 seconds) after driving to . The predetermined time may be variably adjusted.

30 is a flowchart illustrating a process of identifying that a vehicle makes a U-turn in a prohibited section according to an embodiment of the present invention. 31A and 31B are exemplary views illustrating a process in which a vehicle makes a U-turn in a prohibited section according to an embodiment of the present invention.

30, 31A, and 31B are examples of identifying that a vehicle makes a U-turn in a prohibited section by determining a continuous driving state.

Hereinafter, a process for identifying that a vehicle makes a U-turn in a prohibited section according to an embodiment of the present invention will be described in detail with reference to FIGS. 30, 31A and 31B .

According to an embodiment, the processor 260 may identify that the vehicle is in a normal driving state in a lane in which the left side is the center lane ( S3010 ).

As shown in (a) of FIG. 31A , the vanishing point 3113 is within the margin 3114 , the center lane 3111 is within the normal section 3115 , and the general lane 3112 is within the normal section 3116 . If there is ( 3110 ), the processor 260 may determine that the vehicle normally travels within the lane. The processor 260 may determine whether the vehicle is normally driving within the lane through Equation 5.

According to an embodiment, the processor 260 may identify that the vehicle completely invades the left side and proceeds to an abnormal driving state (S3012).

As shown in (b) of FIG. 31A , when the vanishing point 3123 is not within the margin 3214 and the center lane 3121 is within the full encroachment section 3125 (3120), the processor 260 is It may be determined that the vehicle is driving abnormally by completely encroaching on the center lane. The processor 260 may determine whether the vehicle is driving abnormally by completely encroaching on the center lane through Equation (8).

According to an embodiment, the processor 260 may identify whether lane recognition is impossible (S3014).

While the vehicle completely invades the center lane and drives abnormally, the processor 260 may not recognize the lane in the driving image obtained through the camera 210 .

As illustrated in (c) of FIG. 31A , the processor 260 may not recognize a lane because the lane is not included in the driving image 3130 obtained through the camera 210 .

According to an embodiment, the processor 260 may identify that the vehicle completely invades the right side and proceeds to an abnormal driving state (S3016).

As shown in (d) of FIG. 31B , when the vanishing point 3143 is not within the margin 3244 and the general lane 3141 is within the complete encroachment section 3146 ( 3140 ), the processor 260 is It may be determined that the vehicle is driving abnormally by completely encroaching on the center lane. The processor 260 may determine whether the vehicle is driving abnormally by completely encroaching on the center lane through [Equation 9].

According to an embodiment, the processor 260 may identify whether the vehicle is driving normally in the lane (S3018).

As shown in (e) of FIG. 31B , when the vanishing point 3153 is within the margin 3154 and each of the normal lanes 3151 and 3152 is within the normal sections 3155 and 3156 ( 3150 ), the processor At 260 , it may be determined that the vehicle normally travels within the lane. The processor 260 may determine whether the vehicle is normally driving within the lane through Equation 5.

According to an embodiment, the processor 260 may determine that the vehicle has made a U-turn in the prohibited section (S3020). After the processes (S3010, S3012, S3014, S3016, S3018) are sequentially generated, the processor 260 determines that the vehicle is in a normal driving state (eg, driving along a lane in the opposite lane) for a predetermined time (eg: 15 seconds), it may be determined that the vehicle is driving by making a U-turn in the prohibited section.

For example, if it is determined that the vehicle is continuously maintained for a predetermined time (eg, 15 seconds) in a driving state after making a U-turn in the prohibited section, the processor 260 may determine that the vehicle has made a U-turn in the prohibited section. .

According to an embodiment, the processor 260 may store a driving image corresponding to a predetermined time based on the time when the vehicle makes a U-turn in the prohibited section (S3022). The processor 260 performs a U-turn at a predetermined time (eg, at a time before making a U-turn in the prohibited section (eg, 1:29:45) based on the time (eg, 1:30) of driving in the prohibited section) The driving image acquired during the time (eg, 1:30:15) after making a U-turn in the prohibited section may be stored in the storage unit 240 . The predetermined time may be variably adjusted.

32 is a flowchart illustrating a process of identifying that a vehicle makes a U-turn in a prohibited section according to another embodiment of the present invention. 33 is an exemplary diagram illustrating a process in which a vehicle makes a U-turn in a prohibited section according to another embodiment of the present invention.

32 and 33 are examples of identifying that a vehicle makes a U-turn in a prohibited section through a sensor.

Hereinafter, with reference to FIGS. 32 and 33 , a process for identifying that a vehicle makes a U-turn in a prohibited section according to an embodiment of the present invention will be described in detail as follows.

According to an embodiment, the processor 260 may identify that the vehicle is in a normal driving state in a lane in which the left side is the center lane ( S3210 ).

As shown in (a) of FIG. 33 , the vanishing point 3313 is within the margin 3314 , the center lane 3311 is within the normal section 3315 , and the general lane 3312 is within the normal section 3316 . If there is ( 3310 ), the processor 260 may determine that the vehicle normally travels within the lane. The processor 260 may determine whether the vehicle is normally driving within the lane through Equation 5 above.

According to an embodiment, the processor 260 may identify whether lane recognition is impossible (S3212).

While the vehicle is making a U-turn, the processor 260 may not recognize a lane in the driving image obtained through the camera 210 .

As illustrated in (b) of FIG. 33 , the processor 260 may not recognize a lane because the lane is not included in the driving image 3320 obtained through the camera 210 .

According to an embodiment, the processor 260 may identify whether rotation information of the vehicle is obtained through a sensor (S3214). The processor 260 includes information obtained from the sensor unit 220 (eg, information about a change in the traveling direction of the vehicle obtained from GPS, and information about a rotation angle of the vehicle obtained from a yaw sensor) At least a part of it may be received from the sensor unit 220 .

According to an embodiment, the processor 260 may identify that the vehicle completely invades the general lane of the opposite lane and proceeds to an abnormal driving state (S3216).

As shown in (c) of FIG. 33 , when the vanishing point 333 is within the margin 3334 and each of the normal lanes 3331 and 3332 is within the normal sections 3335 and 3336 ( 3330 ), the processor Reference numeral 260 may determine that the vehicle completely invades the general lane of the opposite lane and drives abnormally. The processor 260 may determine whether the vehicle is driving abnormally by completely encroaching on the general lane of the opposite lane through [Equation 5].

According to an embodiment, the processor 260 may determine that the vehicle has made a U-turn in the prohibited section (S3218). After the processes S3210, S3212, S3214, and S3216 are sequentially generated, the processor 260 returns the vehicle to a normal driving state (eg, driving along a lane in the opposite lane) for a predetermined time (eg, 15 seconds). ), it can be determined that the vehicle is driving by making a U-turn in the prohibited section.

For example, if it is determined that the vehicle is continuously maintained for a predetermined time (eg, 15 seconds) in a driving state after making a U-turn in the prohibited section, the processor 260 may determine that the vehicle has made a U-turn in the prohibited section. .

According to an embodiment, the processor 260 may store a driving image corresponding to a predetermined time based on the time when the vehicle makes a U-turn in the prohibited section (S3220). The processor 260 performs a U-turn at a predetermined time (eg, at a time before making a U-turn in the prohibited section (eg, 1:29:45) based on the time (eg, 1:30) of driving in the prohibited section) The driving image acquired during the time (eg, 1:30:15) after making a U-turn in the prohibited section may be stored in the storage unit 240 . The predetermined time may be variably adjusted.

Each step in each of the above-described flowcharts may be operated regardless of the illustrated order, or may be performed simultaneously. In addition, at least one component of the present invention and at least one operation performed by the at least one component may be implemented in hardware and/or software.

In addition, the values, sizes, weights, etc. presented in the above description are examples, and various values, sizes, weights, etc. may be adjusted by those skilled in the art.

As described above, the present invention has been described with reference to the illustrated drawings, but the present invention is not limited by the embodiments and drawings disclosed in this specification, and various methods can be obtained by those skilled in the art within the scope of the technical spirit of the present invention. It is obvious that variations can be made. In addition, although the effects according to the configuration of the present invention have not been explicitly described and described while describing the embodiments of the present invention, it is natural that the effects predictable by the configuration should also be recognized.

101: vehicle 200: device
210: camera 220: sensor unit
230: communication unit 240: storage unit
250: display unit 260: processor

Claims (17)

In the device for determining the driving state of the vehicle,
camera;
communication department;
storage; and
Obtaining a driving image of the vehicle through the camera, obtaining road coordinate information of the vehicle reference, obtaining information on a lane on which the vehicle travels, and lane departure of the vehicle based on the road coordinate information and the lane information It includes a processor for setting at least one section for determining whether or not the driving image,
The processor is
Identifying a vanishing point where the first inner reference line of the left lane and the second inner reference line of the right lane are in contact with the vehicle,
Setting a margin with a predetermined interval to the left and right of the identified elemental point,
A complete invasion section having a first predetermined interval to the left and right around a reference position that is a midpoint of the width of the vehicle at the lower end of the obtained driving image, a temporary invasion section having a second predetermined interval to the left and right of the complete invasion section , and setting a normal section that deviates from the temporary intrusion section to the left and right,
The at least one section is
It is set based on the reference position, a first horizontal coordinate that is a point that meets the horizon in the acquired driving image, and a second horizontal coordinate that is a point where the lower end of the acquired driving image intersects,
The processor is
If the vanishing point according to the driving of the vehicle is located within the set margin, and the vehicle is traveling for a predetermined time within the completely intrusion section of the lane, it is determined that the vehicle is driving by completely violating the lane, Storing, in an area where overwriting of the storage unit is prohibited, the driving image from a predetermined time before a predetermined time based on the time when the vehicle completely invades the lane to a predetermined time after a predetermined time based on the time when the complete invasion is terminated,
If the vanishing point according to the driving of the vehicle is located within the set margin and the lane is within the temporary intrusion section, it is determined that the vehicle is traveling by temporarily violating the lane,
If the vanishing point according to the driving of the vehicle is located within the set margin and the lane is within the normal section, it is determined that the vehicle is traveling in the lane normally,
The processor is
Identifies whether a communication channel is set up between the communication unit and the server for collecting vehicle driving information,
When the communication channel is set up, from a predetermined time before the time when the vehicle stored in the overwrite-prohibited area of the storage unit completely invades the lane and drives after a predetermined time based on the time when the complete invasion ends A device for transmitting the driving image to the server through the established communication channel.
According to claim 1,
The road coordinate information is
an apparatus comprising at least a part of actual coordinates for the lane in which the vehicle is traveling, coordinates of the lane in an image on which the road is projected by a camera, a focal length of the camera, and a mounting height of the camera in the lane.
According to claim 1,
The lane information is
In the obtained driving image, the first inner reference line of the left lane, the second inner reference line of the right lane, the vanishing point at which the first inner reference line and the second inner reference line are in contact with the vehicle, and the first An apparatus comprising at least a portion of a lane width between an inner reference line and the second inner reference line, a ratio of a vehicle width crossing the lane, a coordinate of the vanishing point, and a left and right margin of the vanishing point.
4. The method of claim 3,
The processor is
setting a midpoint of the width of the vehicle at the lower end of the obtained driving image as the reference position,
a point where each of the first inner reference line and the second inner reference line and a horizon in the obtained driving image meet is set as the first horizontal coordinate,
An apparatus for setting a point where each of the first inner reference line and the second inner reference line and a lower end of the obtained driving image intersect as the second horizontal coordinates.
5. The method of claim 4,
The processor is
An apparatus configured to determine the margin by identifying a weight of an occupant of the vehicle, a curvature state of a lane, and a shake of the vehicle.
delete delete According to claim 1,
The first predetermined interval is,
Device characterized in that it is an interval obtained by subtracting a value obtained by multiplying a ratio of the width of the vehicle and the width of the vehicle that invades the lane at a point corresponding to 1/2 of the width of the vehicle with respect to the reference position.
According to claim 1,
The second predetermined interval is,
Apparatus according to claim 1, wherein the interval is the interval obtained by subtracting the first predetermined interval in the direction of the reference position from a point corresponding to 1/2 of the width of the vehicle.
In the method of determining the driving state of the vehicle,
A process of acquiring a driving image of the vehicle through a camera;
obtaining road coordinate information based on the vehicle;
obtaining information on a lane in which the vehicle travels;
setting at least one section in the driving image; and
and determining whether the vehicle departs from a lane based on the road coordinate information and the lane information,
The at least one section is
A complete invasion section having a first predetermined interval to the left and right around a reference position that is a midpoint of the width of the vehicle at the lower end of the obtained driving image, a temporary invasion section having a second predetermined interval to the left and right of the complete invasion section , and including a normal section that deviates from the temporary intrusion section to the left and right,
The at least one section is
It is set based on the reference position, a first horizontal coordinate that is a point that meets the horizon in the acquired driving image, and a second horizontal coordinate that is a point where the lower end of the acquired driving image intersects,
The process of determining whether the vehicle departs from the lane is
identifying a vanishing point where a first inner reference line of a left lane and a second inner reference line of a right lane come into contact with each other with respect to the vehicle;
setting a margin having a predetermined interval to the left and right of the identified vanishing point;
If the vanishing point according to the driving of the vehicle is located within the set margin, and the vehicle is traveling for a predetermined time within the completely intrusion section of the lane, it is determined that the vehicle is driving by completely violating the lane, storing driving images from a predetermined time before a predetermined time based on the time when the vehicle completely invades the lane to a predetermined time after a predetermined time based on the time when the complete invasion is terminated in an area where overwriting of the storage unit is prohibited;
determining that the vehicle temporarily invades the lane and travels when the vanishing point according to the driving of the vehicle is located within the set margin and the lane is within the temporary intrusion section;
determining that the vehicle normally travels in the lane when the vanishing point according to the driving of the vehicle is located within the set margin and the lane is within the normal section;
A process of identifying whether a communication channel with a server for collecting vehicle driving information is set up; and
When the communication channel is set up, from a predetermined time before the time when the vehicle stored in the overwrite-prohibited area of the storage unit completely invades the lane and drives after a predetermined time based on the time when the complete invasion ends and transmitting the driving image to the server through the established communication channel.
11. The method of claim 10,
The road coordinate information is
at least a portion of real coordinates for the lane in which the vehicle is traveling, coordinates of the lane, a focal length of the camera, and a mounting height of the camera in the lane.
11. The method of claim 10,
The lane information is
In the acquired driving image, a first inner reference line of a left lane, a second inner reference line of a right lane, a vanishing point at which the first inner reference line and the second inner reference line are in contact with the vehicle, and the first inner reference line and and at least a portion of a lane width between the second inner reference lines, a ratio of a vehicle width crossing the lane, a coordinate of the vanishing point, and a left and right margin of the vanishing point.
13. The method of claim 12,
The process of setting the at least one section in the driving image includes:
setting a midpoint of the width of the vehicle at the lower end of the obtained driving image as the reference position;
setting a point where each of the first inner reference line and the second inner reference line and a horizon in the obtained driving image meet as the first horizontal coordinates; and
and setting, as the second horizontal direction coordinates, a point where each of the first inner reference line and the second inner reference line and a lower end of the obtained driving image intersect.
delete delete 11. The method of claim 10,
The first predetermined interval is,
The method, characterized in that it is an interval obtained by subtracting a value obtained by multiplying the width of the vehicle by a ratio of the width of the vehicle that invades the lane at a point corresponding to 1/2 of the width of the vehicle with respect to the reference position.
11. The method of claim 10,
The second predetermined interval is,
Method according to claim 1, wherein the interval is the interval obtained by subtracting the first predetermined interval in the direction of the reference position from a point corresponding to 1/2 of the width of the vehicle.

Images