KR102452636B1 - Apparatus and method for assisting driving of a vehicle - Google Patents

Abstract

An apparatus and method for assisting driving of a vehicle are provided. A wearable device that assists driving of a vehicle recognizes a driving pattern of a driver who drives a vehicle with the device mounted on the basis of the sensing unit and driving data obtained through the sensing unit while the vehicle is driving, and is acquired through the sensing unit A control signal for determining the current driving state of the driver based on the current driving data and the recognized driving pattern of the driver, determining the driving behavior of the vehicle according to the current driving state of the driver, and controlling the driving behavior of the vehicle It may include a processor that generates

Description

Apparatus and method for assisting driving of a vehicle

The present disclosure relates to an apparatus and method for assisting driving of a vehicle.

In addition, the present disclosure relates to an artificial intelligence (AI) system using a machine learning algorithm such as deep learning and its application.

An artificial intelligence (AI) system is a computer system that implements human-level intelligence, and unlike the existing rule-based smart system, the machine learns, judges, and becomes smarter by itself. As artificial intelligence systems are used, the recognition rate improves and users can understand user preferences more accurately, and the existing rule-based smart systems are gradually being replaced by deep learning-based artificial intelligence systems.

Artificial intelligence technology consists of machine learning (deep learning) and elemental technologies using machine learning.

Machine learning is an algorithm technology that classifies/learns characteristics of input data by itself, and element technology is a technology that uses machine learning algorithms such as deep learning, such as language understanding, visual understanding, reasoning/prediction, knowledge expression, motion control, etc. It consists of technical fields of

The various fields where artificial intelligence technology is applied are as follows. Linguistic understanding is a technology for recognizing and applying/processing human language/text, and includes natural language processing, machine translation, dialogue system, question and answer, and speech recognition/synthesis. Visual understanding is a technology for recognizing and processing objects like human vision, and includes object recognition, object tracking, image search, human recognition, scene understanding, spatial understanding, image improvement, and the like. Inferential prediction is a technology for logically reasoning and predicting by judging information, and includes knowledge/probability-based reasoning, optimization prediction, preference-based planning, and recommendation. Knowledge expression is a technology that automatically processes human experience information into knowledge data, and includes knowledge construction (data generation/classification) and knowledge management (data utilization). Motion control is a technology for controlling autonomous driving of a vehicle and movement of a robot, and includes motion control (navigation, collision, driving), manipulation control (action control), and the like.

An object of the present invention is to provide an apparatus and method for assisting driving of a vehicle. Another object of the present invention is to provide a computer-readable recording medium in which a program for executing the method in a computer is recorded. The technical problem to be solved is not limited to the technical problems as described above, and other technical problems may exist.

A wearable device for assisting driving of a vehicle according to an aspect recognizes a driving pattern of a driver who mounts the device and drives the vehicle, based on a sensing unit and driving data obtained through the sensing unit while the vehicle is driving, Based on the current driving data acquired through the sensing unit and the recognized driving pattern of the driver, the current driving state of the driver is determined, and the driving behavior of the vehicle is determined according to the current driving state of the driver, and the driving behavior of the vehicle is controlled. It may include a processor that generates a control signal for

According to another aspect, there is provided an operating method of a wearable device for assisting driving of a vehicle, the method comprising: recognizing a driving pattern of a driver who drives the vehicle by mounting the device, based on driving data obtained through a sensing unit while the vehicle is driving; Determining a current driving state of the driver based on the current driving data acquired through the sensing unit and the recognized driving pattern of the driver, determining a driving operation of the vehicle according to the current driving state of the driver, and driving of the vehicle generating a control signal for controlling the operation.

A computer-readable recording medium according to another aspect includes a recording medium recording a program for executing the above-described method on a computer.

1 is a diagram illustrating an example in which an apparatus for assisting driving of a vehicle operates according to an exemplary embodiment.
2 is a flowchart of a method of operating an electronic device according to an exemplary embodiment.
3 is a view for explaining an example of acquiring driving data while a vehicle is driving, according to an exemplary embodiment.
4 is a diagram for describing an example of a current driving state of a driver, according to an exemplary embodiment.
5 is a diagram for describing an example of determining driving control of a vehicle according to an exemplary embodiment.
6 is a flowchart illustrating an example of outputting information related to driving control of a vehicle according to an exemplary embodiment.
7 is a view for explaining an example of outputting information related to driving control of a vehicle according to an exemplary embodiment.
8 is a flowchart illustrating an example of displaying information about driving control of a vehicle on a display of a vehicle, according to an exemplary embodiment.
9 to 10 are diagrams for explaining an example of outputting information related to driving control of a vehicle according to an exemplary embodiment.
11 to 12 are flowcharts for explaining an example of controlling a vehicle driving operation in conjunction with a vehicle system according to an exemplary embodiment.
13 is a block diagram of an electronic device according to an exemplary embodiment.
14 is a detailed block diagram of an electronic device according to an exemplary embodiment.
15 is a block diagram of a vehicle according to an exemplary embodiment.
16 is a block diagram of a processor according to an embodiment.
17 is a block diagram of a data learning unit according to an exemplary embodiment.
18 is a block diagram of a data recognition unit according to an exemplary embodiment.
19 is a diagram illustrating an example of learning and recognizing data by interworking with an electronic device and a server according to an embodiment.

Terms used in the embodiments are selected as currently widely used general terms as possible while considering functions in the present invention, but may vary according to intentions or precedents of those of ordinary skill in the art, emergence of new technologies, and the like. In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than the name of a simple term.

In the entire specification, when a part “includes” a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated. In addition, the “… wealth", "… The term “module” means a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those of ordinary skill in the art to which the present invention pertains can easily implement them. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

1 is a diagram illustrating an example in which an apparatus for assisting driving of a vehicle operates according to an exemplary embodiment.

In the present specification, the device 100 (FIG. 15) (hereinafter, the electronic device 100, or the wearable device 100) for assisting or controlling the driving of the vehicle 1 (FIG. 15) (hereinafter, the vehicle 1) is, It may be a device that can be worn on the user's body.

Referring to FIG. 1 , an electronic device 100 according to an embodiment may be a wearable robot. For example, the wearable robot may be a robot 100a wearable on a user's leg or a robot 100b wearable on a user's arm, but is not limited thereto.

A wearable robot according to an embodiment is, for example, a device that is worn on the body of a physically handicapped or the elderly to assist the movement of the wearer's body. When the wearer's brain sends motion signals to the muscles, the wearable robot generates a minute electric current on the surface of the skin, and the worn robot detects this and operates the motor to assist the body's movement.

According to an embodiment, when the driver mounts the wearable device 100 and drives the vehicle 1 by riding in the driver's seat of the vehicle 1 , the wearable device 100 provides driving data while the vehicle 1 is driving. may be sensed, and a driver's driving pattern may be recognized and learned based on the sensed driving data.

According to an exemplary embodiment, while the vehicle 1 is driving, if it is determined that the driving control operation of the driver is inappropriate when compared with the previously learned driving pattern of the driver, for example, the wearable device 100 may When a dangerous situation such as a collision is predicted due to a slow reaction speed, the wearable device 100 may directly control the driving operation of the vehicle 1 to stop the vehicle 1 or change the driving direction.

According to an embodiment, when a situation in which safe driving is difficult due to driver's negligence or health abnormality occurs while driving a vehicle, the wearable device 100 controls the driving operation of the vehicle 1 , thereby driving safer and more convenient for the driver environment can be provided. The physically handicapped or the elderly who have difficulty in physical activity may wear the wearable device 100 for smoother physical activity. Driving of the wearable device 100 while driving the vehicle 1 while wearing the wearable device 100 Due to the control assistance, dangerous situations may be prevented in advance and safer driving may be possible.

According to an embodiment, the driving data acquired by the wearable device 100 while the vehicle 1 is driving may include at least one of driving information of the vehicle, surrounding image information of the vehicle, and state information of the driver, However, the present invention is not limited thereto.

The driving information of the vehicle according to an embodiment is information about driving of the vehicle acquired by the sensing unit of the wearable device 100, and includes vehicle speed information, acceleration information, speed maintenance time information, driving direction information, rotation direction information, It may include rotation speed information, vehicle location information, and the like, but is not limited thereto.

In addition, the driving information of the vehicle may include driving record information (eg, steering wheel angle information, RPM information, etc.) generated from on-board diagnostics (OBD) installed in the vehicle.

In addition, the surrounding image information of the vehicle according to an embodiment is information about the external situation of the vehicle acquired by the sensing unit of the wearable device 100 , and includes a lane change (eg, a section in which the lane is narrowed) on a driving road, front and rear It may include, but is not limited to, information on whether there is an obstacle, whether a nearby vehicle is in proximity, and a signal change of a traffic light.

In addition, the surrounding image information may include environment information within a predetermined radius of the vehicle 1 in which the driver wearing the wearable device 100 rides. For example, weather information (e.g., whether it is difficult to secure forward visibility due to snow, rain, fog, etc.), temperature information, humidity information, illuminance information, noise information, sound information, road surface condition (slippery road surface, etc.) , road conditions (eg, a section under construction, a section where the road is narrowed to a single lane, a one-way section, etc.), but is not limited thereto.

In addition, the driver's state information according to an exemplary embodiment is information about a driver wearing the wearable device 100 obtained by the sensing unit of the wearable device 100 , and includes information about the driver's body movement, biometric information of the driver, and the like. may include, but is not limited thereto. The driver's biometric information may include information about the driver's heart rate, electrocardiogram, and skin resistance, but is not limited thereto.

Also, according to an embodiment, the sensing unit 110 ( FIG. 14 ) including the image sensor 228 ( FIG. 14 ) and the like may detect the driver's facial expression, gaze, and behavior. The electronic device 100 may determine that the driver is in a drowsy state based on the driver's facial expression detected through the image sensor 228 . For example, when the driver yawns frequently or the number of eye blinks increases, the electronic device 100 may determine that the driver is in a drowsy state.

Also, according to an embodiment, a sensing unit (not shown) included in the vehicle 1 may acquire driving data while the vehicle 1 is driving. The electronic device 100 may receive driving data from the vehicle 1 .

Meanwhile, according to an embodiment, the electronic device 100 learns a driver's driving pattern based on driving data sensed while driving the vehicle 1 , and controls the driving of the vehicle 1 with an appropriate driving motion. For this purpose, one or more data recognition models may be used.

According to an embodiment, the processor 120 ( FIG. 13 ) may use a data recognition model based on a neural network, such as a deep neural network (DNN) or a recurrent neural network (RNN).

According to an embodiment, the electronic device 100 may be a wearable device such as a watch, glasses, a hair band, and a ring having a communication function and a data processing function. However, the present invention is not limited thereto, and the electronic device 100 may include any type of device capable of sensing driving data of the vehicle and controlling the driving operation of the vehicle.

In addition, the electronic device 100 communicates with the vehicle 1, the server (not shown), and an external device (not shown) through a predetermined network in order to acquire driving data while the vehicle is driving and to recognize the driver's driving pattern. can do. In this case, the network includes a local area network (LAN), a wide area network (WAN), a value added network (VAN), a mobile radio communication network, a satellite communication network, and their mutual It is a data communication network in a comprehensive sense that includes a combination and enables each network constituent entity to communicate smoothly with each other, and may include a wired Internet, a wireless Internet, and a mobile wireless communication network. Wireless communication is, for example, wireless LAN (Wi-Fi), Bluetooth, Bluetooth low energy, Zigbee, WFD (Wi-Fi Direct), UWB (ultra wideband), infrared communication (IrDA, infrared Data Association) ), NFC (Near Field Communication), etc. may be there, but is not limited thereto.

2 is a flowchart of a method of operating an electronic device according to an exemplary embodiment. 3 is a view for explaining an example of acquiring driving data while a vehicle is driving, according to an exemplary embodiment. 4 is a diagram for describing an example of a current driving state of a driver, according to an exemplary embodiment. 5 is a diagram for describing an example of determining driving control of a vehicle according to an exemplary embodiment. The flowchart of FIG. 2 will be described with reference to the drawings of FIGS. 3 to 5 .

In step S201 of FIG. 2 , the electronic device 100 according to an exemplary embodiment performs the driving of a driver who mounts the electronic device 100 and drives the vehicle based on the driving data acquired through the sensing unit while the vehicle is driving. pattern can be recognized.

Referring to FIG. 3 , the wearable device 100b mounted on the driver's arm according to an embodiment may acquire driving data while the vehicle 1 is driving, and using one or more data recognition models, driving data based on the driver's driving pattern.

According to an embodiment, the electronic device 100 is a wearable device that can be worn on a user's body, and may learn a driving pattern of a specific user wearing the device.

According to an embodiment, the electronic device 100 may provide vehicle driving information (eg, driving speed, average braking distance when stopping, etc.), surrounding image information (eg, road congestion, presence of front and rear obstacles, etc.), and driver's state. Based on driving data including information (eg, driver's drowsiness, carelessness, heart rate, etc.), it is possible to recognize which driving pattern the driver shows in which driving environment. For example, as the driver's usual driving pattern, driving patterns such as average driving speed, average braking distance, sudden stop, sudden acceleration, rapid lane change frequency, and excessive speed driving tendency may be recognized.

Meanwhile, according to an embodiment, the electronic device 100 may acquire driving data of the vehicle 1 at predetermined intervals while the vehicle 1 is driving. For example, the electronic device 100 may acquire driving data through the sensing unit 110 ( FIG. 14 ) at intervals of 10 seconds, but is not limited thereto.

According to an embodiment, the electronic device 100 may learn a driver's driving pattern based on driving data using one or more data recognition models.

According to an embodiment, in order to recognize the driving pattern of the driver, which data regarding the current driving state of the vehicle acquired while the vehicle is driving may be used, it may be determined according to learning based on a preset criterion.

For example, supervised learning in which the current driving state of the vehicle is sensed and the driving pattern of the vehicle during a predetermined dangerous driving as input values while driving of a predetermined vehicle, learning the driving pattern of the vehicle without special guidance Unsupervised learning of discovering a driving pattern of a vehicle by self-learning the type of information sensed by the current state of the vehicle during driving of the vehicle required for driving the vehicle may be used for recognizing the driving pattern of the driver.

Also, for example, reinforcement learning using feedback on whether a result of recognition of a driving pattern of a driver according to learning is correct may be used for recognition of a driving pattern of the driver.

In step S202 of FIG. 2 , the electronic device 100 according to an embodiment may determine the current driving state of the driver based on the current driving data acquired through the sensing unit and the recognized driving pattern of the driver.

According to an embodiment, the electronic device 100 may compare a pre-learned driving pattern with the acquired current driving data while acquiring driving data through the sensing unit 110 ( FIG. 14 ) while the vehicle 1 is driving. have. As a result of comparing the pre-learned driving pattern with the currently acquired driving data, the electronic device 100 compares the previously learned driving pattern with the currently acquired driving data. When the current driving data shows a different pattern out of a predetermined safe driving range of the previously learned driving pattern, the driver's current driving state may be determined to be at risk.

For example, when the traffic light on the traveling path changes to red while the vehicle 1 is driving, the electronic device 100 starts brake control in consideration of the distance from the current location of the vehicle 1 to the crosswalk, the average braking distance, etc. It recognizes the driver's driving pattern, which has already been learned. When it is determined that the currently acquired driving data is a different driving pattern out of the safe driving range of the pre-learned driving pattern, the electronic device 100 may determine that the driving control of the current driver is in a dangerous driving state.

4 shows an example of the current driving state of the driver driving the vehicle 1 .

For example, in a situation where it is difficult to secure a forward view due to a driver's drowsy driving (41), distraction (42, 43), heavy rain, etc. (44), a dangerous situation in which the vehicle 1 crosses lanes (45) , a collision risk situation 46 may occur due to the proximity of the mileage with the vehicle in front (2).

According to an embodiment, when a driving pattern according to the acquired driving data shows a different driving pattern outside a predetermined safe driving range (45, 46), the electronic device 100 determines the current driving state of the driver as a dangerous state. can

Also, for example, when recognizing that the vehicle 1 shows a driving pattern that repeats sudden acceleration and sudden stop, the electronic device 100 may determine that the current driving state of the driver is a dangerous state.

Also, for example, if the electronic device 100 recognizes that the driving pattern of overspeed is displayed when compared with the previously learned driving pattern of the driver, the electronic device 100 may determine that the current driving state of the driver is in a dangerous state.

In step S203 of FIG. 2 , the electronic device 100 according to an embodiment may determine a driving operation of the vehicle according to the current driving state of the driver. In operation S204, the electronic device 100 according to an embodiment may generate a control signal for controlling the driving operation of the vehicle.

For example, when a traffic light on a traveling path changes to red while the vehicle 1 is driving, the electronic device 100 determines that the current driver's driving control is in a dangerous driving state, so that the vehicle 1 moves to the vehicle in front or Brake action control can be determined so that it can stop before approaching a crosswalk. The electronic device 100 may generate a control signal for performing brake operation control and control the driving operation of the vehicle based on the control signal.

Referring to FIG. 5 , according to an embodiment, the electronic device 100b mounted on the driver's arm performs an abrupt stopping in consideration of the driving distance between the vehicle 1 in which the driver is riding and the vehicle in front 2 . A control signal for the operation control 51 may be generated for the operation control 51 , and a sudden stop operation control of the vehicle may be performed based on the control signal.

According to an embodiment, the electronic device 100 acquires driving data while driving the vehicle 1 and learns a driving pattern, and when it is determined that the currently acquired driving data is outside the safe driving range of the pre-learned driving pattern, By blocking the driving operation control by the driver and directly controlling the driving operation of the vehicle 1 , it is possible to provide a safe driving environment for the driver.

Also, as the electronic device 100 learns the driver's driving pattern based on the acquired driving data, the electronic device 100 may update the data recognition model. The electronic device 100 may learn a driver's driving pattern by using the updated data recognition model, and may control a driving operation of the vehicle based on the learned driving pattern.

1 to 5 are illustrated for explaining an embodiment, but are not limited thereto.

6 is a flowchart illustrating an example of outputting information related to driving control of a vehicle according to an exemplary embodiment. 7 is a view for explaining an example of outputting information related to driving control of a vehicle according to an exemplary embodiment. The flowchart of FIG. 6 will be described with reference to the drawing of FIG. 7 .

In step S601 of FIG. 6 , the electronic device 100 according to an embodiment may generate a control signal for controlling the driving operation of the vehicle 1 . In step S602 of FIG. 6 , the electronic device 100 according to an embodiment may perform a driving operation of the vehicle 1 based on the control signal.

According to an embodiment, while the driver equipped with the electronic device 100 drives the vehicle 1 , the electronic device 100 determines that the driver's driving is in a dangerous state, and directly controls the driving operation of the vehicle 1 . A control signal for controlling may be generated, and a driving operation of the vehicle may be performed.

In step S603 of FIG. 6 , the electronic device 100 according to an exemplary embodiment may output information related to the driving operation control of the vehicle through the output unit.

According to an embodiment, the electronic device 100 may output information about the driving operation control of the vehicle as a sound through the sound output unit 282 ( FIG. 14 ).

Referring to FIG. 7 , for example, when the electronic device 100b controls the driving of the vehicle 1 to change the lane to the right lane, for example, (“Move to the right lane” (61)) information may be output through the sound output unit 282 ( FIG. 14 ).

6 to 7 illustrate an exemplary embodiment, but are not limited thereto.

8 is a flowchart for explaining an example of displaying information about driving control of a vehicle on a display of a vehicle, according to an exemplary embodiment. 9 to 10 are diagrams for explaining an example of outputting information related to driving control of a vehicle according to an exemplary embodiment. The flowchart of FIG. 8 will be described with reference to the drawings of FIGS. 9 to 10 .

In step S801 of FIG. 8 , the electronic device 100 according to an embodiment may operate in conjunction with a vehicle system.

According to an embodiment, the electronic device 100 may communicate with the vehicle system 1 through a predetermined network in order to control the driving operation of the vehicle and transmit/receive data while the vehicle is driving. In this case, the network may include, but is not limited to, a wired Internet, a wireless Internet, and a mobile wireless communication network. Wireless communication is, for example, wireless LAN (Wi-Fi), Bluetooth, Bluetooth low energy, Zigbee, WFD (Wi-Fi Direct), UWB (ultra wideband), infrared communication (IrDA, infrared Data Association) ), NFC (Near Field Communication), etc. may be there, but is not limited thereto.

In step S802 of FIG. 8 , the electronic device 100 according to an embodiment may perform a driving operation of the vehicle 1 . According to an embodiment, when it is determined that the driver's driving pattern is in a dangerous state, the electronic device 100 may directly control the driving operation of the vehicle.

In step S803 of FIG. 8 , the electronic device 100 according to an embodiment may control the display unit of the vehicle to display information related to driving operation control of the vehicle.

According to an embodiment, the electronic device 100 informs the user about controlling the driving operation of the vehicle 1 by the electronic device 100 on the display unit 202 ( FIG. 15 ) mounted on the vehicle 1 . You can control to display information for

Referring to FIG. 9 , the electronic device 100 displays information indicating interworking with the vehicle system 1 (eg, “connected to the wearable robot 1” 91 ) on the display unit 202 of the vehicle system 1 . can be controlled to be displayed in

Referring to FIG. 10 , the electronic device 100 transmits information indicating the driving control of the vehicle 1 (eg, “Dangerous state detection of the driver! Start driving control of the wearable robot 1” (103)) to the vehicle system. It can be displayed on the display unit 202 of (1).

8 to 10 illustrate an exemplary embodiment, but are not limited thereto.

11 to 12 are flowcharts for explaining an example of controlling a vehicle driving operation in conjunction with a vehicle system according to an exemplary embodiment.

In step S1101 of FIG. 11 , the electronic device 100 according to an embodiment may receive, from the vehicle system, a control signal related to a driving operation of the vehicle according to the current driving state of the driver.

According to an embodiment, the vehicle system 1 may perform an ADAS system (Advanced Driver Assistance System) function that blocks the driver's driving control and directly controls the driving operation of the vehicle. For example, the vehicle system 1 may perform an emergency automatic braking system (AEB) function to measure a distance and a relative speed with respect to a vehicle in front to automatically perform brake operation control when it is determined that there is a risk of a collision. In addition, the vehicle system 1 may perform functions such as a lane departure warning system (LDWS), a lane keeping assistance system (LKAS), and a blind spot detection system (BSD).

According to an embodiment, the electronic device 100 may receive, from the vehicle system 1 , a control signal related to a driving operation of the vehicle when the driver is in a dangerous driving state. In step S1102 of FIG. 11 , the electronic device 100 according to an embodiment may determine a driving operation of the vehicle based on the received control signal. In step S1103 of FIG. 11 , the electronic device 100 according to an embodiment may learn a driving motion of the vehicle according to the current driving state of the driver.

According to an embodiment, the electronic device 100 may learn about an appropriate driving control operation in response to the driver's current driving state based on the control signal received from the vehicle system 1 .

The electronic device 100 may learn about a safe driving control operation in a specific driving state by using one or more data recognition models and update the data recognition model. The electronic device 100 may use the updated data recognition model to perform more appropriate driving motion control even in a situation in which a control signal cannot be received from the vehicle system 1 .

12 is a flowchart illustrating an example in which the electronic device 100 and the vehicle system 1 operate in cooperation with each other.

In step S1201 of FIG. 12 , the electronic device 100 according to an embodiment may learn the driver's driving pattern based on driving data.

According to an embodiment, the electronic device 100 obtains driving data (eg, driving speed, acceleration, driving direction, etc.) of the vehicle obtained by using the sensing unit 110 ( FIG. 14 ), and state data of the driver (eg, the driver). based on the reaction speed of the driver, etc.), it is possible to recognize which driving pattern the driver shows in which situation.

In step S1202 of FIG. 12 , the electronic device 100 according to an embodiment may detect that the current driving state of the driver is a dangerous state.

According to an embodiment, the electronic device 100 may determine whether the current driving state of the driver is a dangerous driving state or a safe driving state by acquiring current driving data and comparing the current driving data with a previously learned driving pattern. have.

In step S1203 of FIG. 12 , the electronic device 100 according to an embodiment may block driving operation control by the driver.

According to an embodiment, if the electronic device 100 determines that the driver's current driving state is a dangerous state, the electronic device 100 may block the driving motion control signal due to the driver's steering wheel manipulation, brake manipulation, etc. from being transmitted to the vehicle system 1 . can

Meanwhile, in step S1204 of FIG. 12 , the vehicle system 1 according to an exemplary embodiment may determine a first driving operation of the vehicle.

According to an embodiment, when the vehicle system 1 has a function to determine the driver's dangerous driving state and to perform driving control of the vehicle 1 , the vehicle system 1 may be in the driver's dangerous driving state. A first driving operation for safe driving may be determined.

In step S1205 of FIG. 12 , the vehicle system 1 according to an embodiment may generate a first control signal for controlling the first driving operation of the vehicle.

For example, when it is determined that the relative distance of the vehicle to the vehicle in front is less than or equal to a predetermined threshold and the driver detects a dangerous driving condition in which the driver does not attempt to brake, the vehicle system 1 considers the braking distance for an abrupt stop. It is possible to determine brake operation control and generate a control signal.

In step S1206 of FIG. 12 , the wearable device 100 according to an embodiment may receive a first control signal of a vehicle.

According to an embodiment, the vehicle system 1 interworking with the wearable device 100 may transmit a first control signal to the wearable device 100 in response to a request signal from the wearable device 100 .

Meanwhile, in step S1207 of FIG. 12 , the electronic device 100 according to an exemplary embodiment performs a second driving operation of the vehicle based on the first control signal received from the vehicle system 1 and the previously learned driving pattern. can be decided

According to an embodiment, the electronic device 100 may determine a driving operation suitable for safe driving based on a current driving state and a previously learned driving pattern. At this time, when the electronic device 100 receives the first control signal for controlling the driving operation from the vehicle system 1 , the electronic device 100 considers the previously learned driving pattern and the first control signal to obtain a more suitable system for safe driving. 2 The driving behavior can be determined.

In step S1208 of FIG. 12 , the electronic device 100 according to an embodiment may generate a second control signal for controlling a second driving operation of the vehicle. In step S1209 of FIG. 12 , the electronic device 100 according to an embodiment may perform a driving operation of the vehicle based on the second control signal.

For example, the electronic device 100 may determine the driving deceleration control in a dangerous state requiring a sudden stop control, and compare the driving deceleration of the first control signal determined by the vehicle system 1 to a more suitable driving speed. It can be determined and carried out by driving motions.

Meanwhile, in step S1210 of FIG. 12 , the electronic device 100 according to an embodiment may transmit a display control signal to the vehicle system 1 . In step S1211 of FIG. 12 , the vehicle system 1 according to an exemplary embodiment may display information related to driving operation control of the vehicle on the display unit.

According to an embodiment, in response to the display request signal of the wearable device 100 , the vehicle system 1 outputs information about the driving operation of the vehicle to the display unit 202 ( FIG. 15 ), thereby providing the driver with driving control. information can be provided.

Meanwhile, in step S1212 of FIG. 12 , the electronic device 100 according to an embodiment may learn a driving motion of the vehicle according to the current driving state of the driver.

According to an embodiment, the electronic device 100 may learn data on which driving operation is suitable to perform in which driving state by using one or more data recognition models. Also, the electronic device 100 may update the data recognition model by learning the driving motion of the vehicle according to the current driving state using one or more data recognition models.

11 to 12 illustrate an exemplary embodiment, but are not limited thereto.

13 is a block diagram of an electronic device according to an exemplary embodiment. According to an embodiment, the electronic device 100 may include a sensing unit 110 and a processor 120 .

Those of ordinary skill in the art related to this embodiment may understand that other general-purpose components may be further included in addition to the components shown in FIG. 13 .

According to an embodiment, the sensing unit 110 may sense driving data of the vehicle 1 while the vehicle 1 is driving.

The sensing unit 110 may include a plurality of sensors for detecting the driving state of the vehicle, the driver's state, the surrounding state of the driving vehicle, and the like. For example, the sensing unit 110 may include a distance sensor such as a LIDAR sensor and a RADAR sensor, and an image sensor such as a camera.

In addition, the sensing unit 110 may include one or more actuators configured to correct the position and/or orientation of the plurality of sensors. can sense

In addition, the sensing unit 110 may sense a shape of an object located in the vicinity, a shape of a road, and the like using an image sensor.

Also, the sensing unit 110 may sense biometric information (eg, a user's heart rate, a user's body temperature, etc.) of a user wearing the electronic device 100 .

According to an embodiment, the processor 120 may include at least one processor.

According to an embodiment, the processor 120 is a driver who mounts the wearable device 100 and drives the vehicle 1 based on the driving data acquired through the sensing unit 110 while the vehicle 1 is driving. of driving patterns can be recognized.

The processor 120 may acquire, through the sensing unit 110 , driving data including at least one of driving information of the vehicle, surrounding image information of the vehicle, and state information of the driver.

Also, the processor 120 may learn the driver's driving pattern based on the driving data using one or more data recognition models.

Also, the processor 120 may determine the current driving state of the driver based on the current driving data acquired through the sensing unit 110 and the recognized driving pattern of the driver.

Also, the processor 120 may determine the driving operation of the vehicle 1 according to the current driving state of the driver.

In addition, the processor 120 may block driving operation control by the driver as it is determined that the driver's current driving state is a dangerous state.

Also, the processor 120 may determine a driving operation of the vehicle including at least one of a driving speed and a driving direction of the vehicle.

Also, the processor 120 may determine a driving motion of the vehicle based on the recognized driving pattern of the driver using one or more data recognition models.

Also, the processor 120 may generate a control signal for controlling the driving operation of the vehicle 1 .

In addition, the processor 120 may receive, from the vehicle system 1 through the communication unit 160 , a control signal related to a driving operation of the vehicle according to the current driving state of the driver. Also, the processor 120 may determine the driving operation of the vehicle based on the received control signal.

Also, the processor 120 may learn the driving motion of the vehicle according to the current driving state of the driver, received from the vehicle system 1 , by using one or more data recognition models.

In addition, the processor 120 may control to display information regarding the driving operation control of the vehicle 1 on the display unit 202 mounted on the vehicle as the driving operation of the vehicle is controlled.

In addition, as the processor 120 controls the driving operation of the vehicle, the processor 120 may output information regarding the driving operation control of the vehicle through the output unit 130 .

14 is a detailed block diagram of an electronic device according to an embodiment.

As shown in FIG. 14 , the electronic device 100 according to an embodiment includes an output unit 130 , a storage unit 140 , an input unit 150 , and a communication unit in addition to the sensing unit 110 and the processor 120 . (160) may be further included. It can be understood by those of ordinary skill in the art related to the present embodiment that other general-purpose components may be further included in addition to the components shown in FIG. 14 .

The sensing unit 110 according to an embodiment may include a plurality of sensors configured to sense information about the surrounding environment in which the vehicle 1 is located, and one configured to correct the position and/or orientation of the sensors. It may include more than one actuator. For example, the sensing unit 110 includes a Global Positioning System (GPS) 224 , an Inertial Measurement Unit (IMU) 225 , a RADAR sensor 226 , a LIDAR sensor 227 , an image sensor 228 , and an Odometery sensor. (230) may be included. In addition, the sensing unit 110 may include at least one of a temperature/humidity sensor 232 , an infrared sensor 233 , a barometric pressure sensor 235 , a proximity sensor 236 , and an RGB sensor (illuminance sensor) 237 . However, the present invention is not limited thereto. Since a function of each sensor can be intuitively inferred from a person skilled in the art from the name, a detailed description thereof will be omitted.

Also, the sensing unit 110 may include a motion sensing unit 238 capable of sensing the motion of the vehicle 1 . The motion sensing unit 238 may include a magnetic sensor 229 , an acceleration sensor 231 , and a gyroscope sensor 234 .

The GPS 224 may be a sensor configured to estimate the geographic location of the vehicle 1 . That is, the GPS 224 may include a transceiver configured to estimate the position of the vehicle 1 with respect to the Earth.

The IMU 225 may be a combination of sensors configured to detect changes in position and orientation of the vehicle 1 based on inertial acceleration. For example, the combination of sensors may include accelerometers and gyroscopes.

The RADAR sensor 226 may be a sensor configured to detect objects in the environment in which the vehicle 1 is located using a wireless signal. Further, the RADAR sensor 226 may be configured to sense the velocity and/or direction of objects.

The LIDAR sensor 227 may be a sensor configured to detect objects in the environment in which the vehicle 1 is located using a laser. More specifically, the LIDAR sensor 227 may include a laser light source and/or a laser scanner configured to emit a laser, and a detector configured to detect the reflection of the laser. The LIDAR sensor 227 may be configured to operate in either a coherent (eg, using hetyrodyne detection) or incoherent detection mode.

The image sensor 228 may be a still camera or a video camera configured to record the environment outside the vehicle 1 . For example, the image sensor 228 may include multiple cameras, which may be disposed at multiple locations on the inside and outside of the vehicle 1 .

The odometery sensor 230 may estimate the position of the vehicle 1 and measure the moving distance. For example, the odometery sensor 230 may measure a position change value of the vehicle 1 using the number of rotations of the wheels of the vehicle 1 .

The storage unit 140 may include a magnetic disk drive, an optical disk drive, and a flash memory. Alternatively, the storage unit 140 may be a portable USB data storage device. The storage unit 140 may store system software for executing examples related to the present application. System software for executing examples related to the present application may be stored in a portable storage medium.

The communication unit 160 may include at least one antenna for wirelessly communicating with other devices. For example, the communication unit 160 may be used to communicate with a cellular network or other wireless protocols and systems wirelessly through Wi-Fi or Bluetooth. The communication unit 160 controlled by the processor 120 may transmit and receive wireless signals. For example, the processor 120 may execute a program included in the storage unit 140 in order for the communication unit 160 to transmit and receive wireless signals to and from the cellular network.

The input unit 150 means a means for inputting data for controlling the vehicle 1 . For example, the input unit 150 includes a key pad, a dome switch, and a touch pad (contact capacitance method, pressure resistance film method, infrared sensing method, surface ultrasonic conduction method, integral tension type). measuring method, piezo effect method, etc.), a jog wheel, a jog switch, etc., but is not limited thereto. Also, the input unit 150 may include a microphone, and the microphone may be configured to receive audio (eg, a voice command) from an occupant of the vehicle 1 .

The output unit 130 may output an audio signal or a video signal, and the output device 280 may include a display 281 and an audio output unit 282 .

The display 281 includes a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, and a three-dimensional display (3D). display) and at least one of an electrophoretic display. Depending on the implementation of the output unit 130 , the output unit 130 may include two or more displays 281 .

The sound output unit 282 outputs audio data received from the communication unit 160 or stored in the storage unit 140 . Also, the sound output unit 282 may include a speaker, a buzzer, and the like.

The input unit 150 and the output unit 130 may include a network interface and may be implemented as a touch screen.

The processor 120 generally controls the sensing unit 110 , the communication unit 160 , the input unit 150 , the storage unit 140 , and the output unit 130 by executing the programs stored in the storage unit 140 . can do.

15 is a block diagram of a vehicle according to an exemplary embodiment.

The electronic device 100 according to an embodiment may include a sensing unit 110 , a processor 120 , and a communication unit 160 . Since the description of the sensing unit 110 , the processor 120 , and the communication unit 160 has been described above with reference to FIGS. 13 to 14 , they will be omitted.

According to an embodiment, the electronic device 100 may transmit/receive data to and from the vehicle 1 through the communication unit 160 .

The vehicle system 1 may include a processor 201 , a display unit 202 , a communication unit 203 , and a driving device 200 . Also, the vehicle system 1 may include a sensing unit (not shown), and may include a function of the sensing unit 110 illustrated in FIG. 14 .

The driving device 200 may include a brake unit 221 , a steering unit 222 , and a throttle 223 .

The steering unit 222 may be a combination of mechanisms configured to steer the vehicle 1 .

The throttle 223 may be a combination of mechanisms configured to control the operating speed of the engine/motor 211 to control the speed of the vehicle 1 . In addition, the throttle 223 may adjust the amount of the mixed gas of fuel air flowing into the engine/motor 211 by adjusting the throttle opening amount, and may control the power and thrust by adjusting the throttle opening amount.

The brake unit 221 may be a combination of mechanisms configured to decelerate the vehicle 1 . For example, the brake unit 221 may use friction to reduce the speed of the wheel/tire 214 .

16 is a block diagram of a processor 120 according to an embodiment.

Referring to FIG. 16 , the processor 120 according to some embodiments may include a data learning unit 1310 and a data recognition unit 1320 .

The data learning unit 1310 may learn a criterion for determining a situation. The data learning unit 1310 may learn a criterion regarding which data to use to determine a predetermined situation and how to determine the situation by using the data. The data learning unit 1310 may acquire data to be used for learning, and apply the acquired data to a data recognition model to be described later, thereby learning a criterion for determining a situation.

The data recognition unit 1320 may determine a situation based on data. The data recognition unit 1320 may recognize a situation from predetermined data by using the learned data recognition model. The data recognition unit 1320 may determine a predetermined situation based on the predetermined data by acquiring predetermined data according to a preset criterion by learning, and using the data recognition model using the acquired data as an input value. . In addition, a result value output by the data recognition model using the obtained data as an input value may be used to update the data recognition model.

At least one of the data learning unit 1310 and the data recognition unit 1320 may be manufactured in the form of at least one hardware chip and mounted in an electronic device. For example, at least one of the data learning unit 1310 and the data recognition unit 1320 may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or a conventional general-purpose processor (eg, CPU). Alternatively, it may be manufactured as a part of an application processor) or a graphics-only processor (eg, GPU) and mounted on the various electronic devices described above.

In this case, the data learning unit 1310 and the data recognition unit 1320 may be mounted on one electronic device or may be mounted on separate electronic devices, respectively. For example, one of the data learning unit 1310 and the data recognition unit 1320 may be included in the electronic device, and the other may be included in the server. In addition, the data learning unit 1310 and the data recognition unit 1320 may provide the model information built by the data learning unit 1310 to the data recognition unit 1320 through wired or wireless communication, and the data recognition unit ( Data input to 1320 may be provided to the data learning unit 1310 as additional learning data.

Meanwhile, at least one of the data learning unit 1310 and the data recognition unit 1320 may be implemented as a software module. When at least one of the data learning unit 1310 and the data recognition unit 1320 is implemented as a software module (or a program module including an instruction), the software module is a computer-readable, non-transitory, non-transitory It may be stored in a readable recording medium (non-transitory computer readable media). Also, in this case, the at least one software module may be provided by an operating system (OS) or may be provided by a predetermined application. Alternatively, a part of the at least one software module may be provided by an operating system (OS), and the other part may be provided by a predetermined application.

17 is a block diagram of a data learning unit according to an exemplary embodiment.

17 is a block diagram of a data learning unit 1310 according to some exemplary embodiments.

Referring to FIG. 17 , the data learning unit 1310 according to some exemplary embodiments includes a data acquiring unit 1310-1, a preprocessing unit 1310-2, a training data selection unit 1310-3, and a model learning unit 1310. -4) and a model evaluation unit 1310-5.

The data acquisition unit 1310-1 may acquire data necessary for situation determination. The data acquisition unit 1310-1 may acquire data necessary for learning for situation determination.

According to an embodiment, the data acquisition unit 1310-1 may acquire driving data by detecting a driving state of a vehicle, a driver's state, a driving state of another vehicle, etc. required for vehicle driving control. Also, the data acquisition unit 1310-1 may receive driving data from the server.

For example, the data acquisition unit 1310-1 may receive an image surrounding the vehicle 1 . The surrounding image may be composed of a plurality of images (or frames). For example, the data acquisition unit 1310-1 may be configured to communicate with a camera of an electronic device including the data learning unit 1310 or an external camera (eg, CCTV or black) capable of communicating with the electronic device including the data learning unit 1310 . box, etc.) to receive video input. Here, the camera may include one or more image sensors (eg, a front sensor or a rear sensor), a lens, an image signal processor (ISP), or a flash (eg, an LED or a xenon lamp, etc.).

Also, for example, the data acquisition unit 1310-1 may acquire the driving state of the vehicle, the driver's state, surrounding image information, and the driving state of the surrounding vehicle. For example, the data acquisition unit 1310-1 may receive data through an input device (eg, a microphone, a camera, or a sensor) of the electronic device. Alternatively, the data acquisition unit 1310-1 may acquire data through an external device communicating with the electronic device.

The preprocessor 1310 - 2 may preprocess the acquired data so that the acquired data can be used for learning for situation determination. The preprocessor 1310-2 may process the acquired data into a preset format so that the model learning unit 1310-4, which will be described later, uses the acquired data for learning for situation determination. For example, the preprocessor 1310 - 2 overlaps at least a portion of the plurality of images based on a common area included in each of a plurality of images (or frames) constituting at least a portion of the input video. Thus, one composite image can be created. In this case, a plurality of composite images may be generated from one moving picture. The common area may be an area including the same or similar common object (eg, object, animal, plant, or person) in each of the plurality of images. Alternatively, the common area may be an area having the same or similar color, shading, RGB value, or CMYK value in each of the plurality of images.

The learning data selection unit 1310 - 3 may select data required for learning from among the preprocessed data. The selected data may be provided to the model learning unit 1310 - 4 . The learning data selection unit 1310 - 3 may select data required for learning from among preprocessed data according to a preset criterion for situation determination. In addition, the training data selection unit 1310 - 3 may select data according to a preset criterion by learning by the model learning unit 1310 - 4 to be described later.

For example, data about the driving state of the vehicle, such as the average speed of the vehicle, the acceleration, the driver's reaction speed, the average braking distance, etc., obtained while the vehicle 1 is driving, may affect the driving path of the vehicle 1 . Data regarding other possible vehicles, obstacles, pedestrians, etc. may be selected. Also, for example, data related to an image in which the driver driving the vehicle 1 performs an action other than driving may be selected.

The model learning unit 1310 - 4 may learn a criterion regarding how to determine a situation based on the learning data. Also, the model learning unit 1310 - 4 may learn a criterion for which learning data to use for situation determination.

According to an embodiment, the model learning unit 1310 - 4 learns a criterion for determining the driving pattern of the driver based on driving data including the driving state of the vehicle, the driver's state, and the external state around the vehicle. can do.

Also, the model learning unit 1310 - 4 may learn a data recognition model used for situation determination by using the training data. In this case, the data recognition model may be a pre-built model. For example, the data recognition model may be a model built in advance by receiving basic training data (eg, a sample image, etc.).

The data recognition model may be constructed in consideration of the field of application of the recognition model, the purpose of learning, or the computer performance of the device. The data recognition model may be, for example, a model based on a neural network. For example, a model such as a deep neural network (DNN), a recurrent neural network (RNN), or a bidirectional recurrent deep neural network (BRDNN) may be used as the data recognition model, but is not limited thereto.

According to various embodiments, the model learning unit 1310 - 4 uses a data recognition model to learn a data recognition model having a high correlation between the input training data and the basic training data when there are a plurality of pre-built data recognition models. can decide In this case, the basic learning data may be pre-classified for each type of data, and the data recognition model may be previously built for each type of data. For example, the basic training data is pre-classified by various criteria such as the region where the training data is generated, the time when the training data is generated, the size of the training data, the genre of the training data, the creator of the training data, the type of object in the training data, etc. may have been

Also, the model learning unit 1310 - 4 may train the data recognition model using, for example, a learning algorithm including error back-propagation or gradient descent. .

Also, the model learning unit 1310 - 4 may train the data recognition model through supervised learning using, for example, learning data as an input value. In addition, the model learning unit 1310-4, for example, through unsupervised learning for discovering a criterion for situation determination by self-learning the type of data required for situation determination without any guidance, A data recognition model can be trained. Also, the model learning unit 1310 - 4 may train the data recognition model through, for example, reinforcement learning using feedback on whether the result of the situation determination according to the learning is correct.

Also, when the data recognition model is learned, the model learning unit 1310 - 4 may store the learned data recognition model. In this case, the model learning unit 1310 - 4 may store the learned data recognition model in the memory of the electronic device including the data recognition unit 1320 . Alternatively, the model learning unit 1310 - 4 may store the learned data recognition model in the memory of the electronic device including the data recognition unit 1320 to be described later. Alternatively, the model learning unit 1310 - 4 may store the learned data recognition model in a memory of a server connected to the electronic device through a wired or wireless network.

In this case, the memory in which the learned data recognition model is stored may also store, for example, commands or data related to at least one other component of the electronic device. The memory may also store software and/or programs. A program may include, for example, a kernel, middleware, an application programming interface (API) and/or an application program (or "application"), and the like.

The model evaluator 1310-5 may input evaluation data into the data recognition model, and when the recognition result output from the evaluation data does not satisfy a predetermined criterion, may cause the model learning unit 1310-4 to learn again. have. In this case, the evaluation data may be preset data for evaluating the data recognition model.

For example, the model evaluator 1310-5 sets a predetermined criterion when the number or ratio of the evaluation data for which the recognition result is not accurate among the recognition results of the learned data recognition model for the evaluation data exceeds a preset threshold. It can be evaluated as unsatisfactory. For example, when the predetermined criterion is defined as a ratio of 2%, when the learned data recognition model outputs an erroneous recognition result for more than 20 evaluation data out of a total of 1000 evaluation data, the model evaluation unit 1310-5 can be evaluated that the trained data recognition model is not suitable.

On the other hand, when there are a plurality of learned data recognition models, the model evaluation unit 1310 - 5 evaluates whether each learned video recognition model satisfies a predetermined criterion, and recognizes a model that satisfies the predetermined criterion as final data can be determined as a model. In this case, when there are a plurality of models satisfying the predetermined criteria, the model evaluator 1310 - 5 may determine any one or a predetermined number of models preset in the order of the highest evaluation score as the final data recognition model.

On the other hand, in the data learning unit 1310, the data acquisition unit 1310-1, the preprocessor 1310-2, the training data selection unit 1310-3, the model learning unit 1310-4, and the model evaluation unit 1310 At least one of -5) may be manufactured in the form of at least one hardware chip and mounted in an electronic device. For example, at least one of the data acquisition unit 1310-1, the preprocessor 1310-2, the training data selection unit 1310-3, the model learning unit 1310-4, and the model evaluation unit 1310-5 One may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or it may be manufactured as part of an existing general-purpose processor (eg, CPU or application processor) or graphics-only processor (eg, GPU) as described above. It may be mounted on various electronic devices.

In addition, the data acquisition unit 1310-1, the preprocessor 1310-2, the training data selection unit 1310-3, the model learning unit 1310-4, and the model evaluation unit 1310-5 are one electronic unit. It may be mounted on the device, or it may be mounted on separate electronic devices, respectively. For example, some of the data acquisition unit 1310-1, the preprocessor 1310-2, the training data selection unit 1310-3, the model learning unit 1310-4, and the model evaluation unit 1310-5 may be included in the electronic device, and the remaining part may be included in the server.

In addition, at least one of the data acquisition unit 1310-1, the preprocessor 1310-2, the training data selection unit 1310-3, the model learning unit 1310-4, and the model evaluation unit 1310-5 is It can be implemented as a software module. At least one of the data acquisition unit 1310-1, the preprocessor 1310-2, the training data selection unit 1310-3, the model learning unit 1310-4, and the model evaluation unit 1310-5 is a software module When implemented as (or, a program module including instructions), the software module may be stored in a computer-readable non-transitory computer readable medium. Also, in this case, the at least one software module may be provided by an operating system (OS) or may be provided by a predetermined application. Alternatively, a part of the at least one software module may be provided by an operating system (OS), and the other part may be provided by a predetermined application.

18 is a block diagram of a data recognition unit according to an exemplary embodiment.

18 is a block diagram of a data recognition unit 1320 according to some exemplary embodiments.

Referring to FIG. 18 , the data recognition unit 1320 according to some embodiments includes a data acquisition unit 1320-1, a pre-processing unit 1320-2, a recognition data selection unit 1320-3, and a recognition result providing unit ( 1320-4) and a model update unit 1320-5.

The data acquisition unit 1320-1 may acquire data necessary for situation determination, and the preprocessor 1320-2 may pre-process the acquired data so that the acquired data may be used for situation determination. The preprocessor 1320-2 may process the acquired data into a preset format so that the recognition result providing unit 1320-4, which will be described later, uses the acquired data for situation determination.

The recognition data selection unit 1320-3 may select data necessary for situation determination from among the pre-processed data. The selected data may be provided to the recognition result providing unit 1320 - 4 . The recognition data selection unit 1320-3 may select some or all of the pre-processed data according to a preset criterion for situation determination. Also, the recognition data selection unit 1320 - 3 may select data according to a preset criterion by learning by the model learning unit 1310 - 4 to be described later.

The recognition result providing unit 1320 - 4 may determine the situation by applying the selected data to the data recognition model. The recognition result providing unit 1320-4 may provide a recognition result according to the purpose of data recognition. The recognition result providing unit 1320 - 4 may apply the selected data to the data recognition model by using the data selected by the recognition data selecting unit 1320 - 3 as an input value. Also, the recognition result may be determined by a data recognition model.

According to an embodiment, the driver's driving pattern or the driver's current driving state may be provided as text, voice, video, image, or a command (eg, an application execution command, a module function execution command, etc.).

The recognition result providing unit 1320-4 may provide, as the driver's current driving state, 'dangerous situation occurrence', 'collision risk', 'center line intrusion risk', etc. in text, voice, video, image, and the like.

The model updater 1320 - 5 may update the data recognition model based on the evaluation of the recognition result provided by the recognition result providing unit 1320 - 4 . For example, the model updating unit 1320-5 provides the recognition result provided by the recognition result providing unit 1320-4 to the model learning unit 1310-4, so that the model learning unit 1310-4 is You can update the data recognition model.

On the other hand, in the data recognition unit 1320, the data acquisition unit 1320-1, the preprocessor 1320-2, the recognition data selection unit 1320-3, the recognition result providing unit 1320-4, and the model update unit ( 1320-5) may be manufactured in the form of at least one hardware chip and mounted in an electronic device. For example, among the data acquisition unit 1320-1, the preprocessor 1320-2, the recognition data selection unit 1320-3, the recognition result providing unit 1320-4, and the model update unit 1320-5 At least one may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or may be manufactured as part of an existing general-purpose processor (eg, CPU or application processor) or graphics-only processor (eg, GPU). It may be mounted on one various electronic devices.

In addition, the data acquisition unit 1320-1, the preprocessor 1320-2, the recognition data selection unit 1320-3, the recognition result providing unit 1320-4, and the model update unit 1320-5 are one It may be mounted on an electronic device, or may be respectively mounted on separate electronic devices. For example, among the data acquisition unit 1320-1, the preprocessor 1320-2, the recognition data selection unit 1320-3, the recognition result providing unit 1320-4, and the model update unit 1320-5 Some may be included in the electronic device, and some may be included in the server.

In addition, at least one of the data acquisition unit 1320-1, the preprocessor 1320-2, the recognition data selection unit 1320-3, the recognition result providing unit 1320-4, and the model update unit 1320-5 may be implemented as a software module. At least one of the data acquisition unit 1320-1, the preprocessor 1320-2, the recognition data selection unit 1320-3, the recognition result providing unit 1320-4, and the model update unit 1320-5 is software. When implemented as a module (or a program module including instructions), the software module may be stored in a computer-readable non-transitory computer readable medium. Also, in this case, the at least one software module may be provided by an operating system (OS) or may be provided by a predetermined application. Alternatively, a part of the at least one software module may be provided by an operating system (OS), and the other part may be provided by a predetermined application.

19 is a diagram illustrating an example of learning and recognizing data by interworking with an electronic device and a server according to an embodiment.

19 is a diagram illustrating an example in which the electronic device 100 and the server 2000 learn and recognize data by interworking with each other, according to some embodiments.

Referring to FIG. 19 , the server 2000 may learn a criterion for determining the situation, and the device 1000 may determine the situation based on the learning result by the server 2000 .

In this case, the model learning unit 2340 of the server 2000 may perform the function of the data learning unit 1310 illustrated in FIG. 17 . The model learning unit 2340 of the server 2000 may learn a criterion regarding which data to use to determine a predetermined situation and how to determine the situation by using the data. The model learning unit 2340 acquires data to be used for learning, and applies the acquired data to a data recognition model to be described later, thereby learning a criterion for determining a situation.

In addition, the recognition result providing unit 1320-4 of the device 1000 may determine the situation by applying the data selected by the recognition data selection unit 1320-3 to the data recognition model generated by the server 2000. have. For example, the recognition result providing unit 1320-4 transmits the data selected by the recognition data selection unit 1320-3 to the server 2000, and the server 2000 performs the recognition data selection unit 1320-3 It can be requested to judge the situation by applying the selected data to the recognition model. Also, the recognition result providing unit 1320 - 4 may receive information about the situation determined by the server 2000 from the server 2000 .

For example, the electronic device 100 transmits driving data including the driving state of the vehicle 1 and the driver's state to the server 2000 , and the server 2000 applies the driving data to the data recognition model to apply the driving data to the data recognition model. You can request to judge your driving pattern. Also, the electronic device 100 may receive the driver's driving pattern determined by the server 2000 from the server 2000 .

Alternatively, the recognition result providing unit 1320 - 4 of the electronic device 100 may receive the recognition model generated by the server 2000 from the server 2000 and determine the situation using the received recognition model. . In this case, the recognition result providing unit 1320-4 of the device 1000 may determine the situation by applying the data selected by the recognition data selection unit 1320-3 to the data recognition model received from the server 2000. have.

For example, the electronic device 100 may learn the driving pattern of the driver by applying driving data including the driving state of the vehicle 1 and the driver's state to the data recognition model received from the server 2000 . have.

The device according to the above-described embodiments includes a processor, a memory for storing and executing program data, a permanent storage such as a disk drive, a communication port for communicating with an external device, a touch panel, a key, a button, etc. It may include the same user interface device and the like. Methods implemented as software modules or algorithms may be stored on a computer-readable recording medium as computer-readable codes or program instructions executable on the processor. Here, the computer-readable recording medium includes a magnetic storage medium (eg, read-only memory (ROM), random-access memory (RAM), floppy disk, hard disk, etc.) and an optically readable medium (eg, CD-ROM). ), DVD (Digital Versatile Disc), and the like. The computer-readable recording medium is distributed among network-connected computer systems, so that the computer-readable code can be stored and executed in a distributed manner. The medium may be readable by a computer, stored in a memory, and executed on a processor.

This embodiment may be represented by functional block configurations and various processing steps. These functional blocks may be implemented in any number of hardware and/or software configurations that perform specific functions. For example, an embodiment may be an integrated circuit configuration, such as memory, processing, logic, look-up table, etc., capable of executing various functions by the control of one or more microprocessors or other control devices. can be hired Similar to how components may be implemented as software programming or software components, this embodiment includes various algorithms implemented in a combination of data structures, processes, routines or other programming constructs, including C, C++, Java ( Java), assembler, etc. may be implemented in a programming or scripting language. Functional aspects may be implemented in an algorithm running on one or more processors. In addition, the present embodiment may employ the prior art for electronic environment setting, signal processing, and/or data processing. Terms such as “mechanism”, “element”, “means” and “configuration” may be used broadly and are not limited to mechanical and physical configurations. The term may include the meaning of a series of routines of software in connection with a processor or the like.

100: electronic device
110: sensing unit
120: processor

Claims (18)

In the wearable device assisting the driving of a vehicle,
sensing unit;
communication department; and
Recognizing a driving pattern of a driver who mounts the device and drives the vehicle based on the driving data acquired through the sensing unit while the vehicle is driving,
determining the current driving state of the driver based on the current driving data acquired through the sensing unit and the recognized driving pattern of the driver,
Receive a control signal related to a driving operation of the vehicle according to the current driving state of the driver from the vehicle through the communication unit,
determining a driving operation of the vehicle based on the current driving state of the driver and the received control signal;
generating a control signal for controlling the driving operation of the vehicle;
and a processor configured to learn a driving operation of the vehicle according to the current driving state of the driver, received from the vehicle, by using one or more data recognition models. The method of claim 1,
The processor is
An apparatus for learning the driving pattern of the driver based on the driving data by using one or more data recognition models.
The method of claim 1,
The processor is
When it is determined that the current driving state of the driver is in a dangerous state, the driving operation control by the driver is blocked,
an apparatus for determining a traveling motion of the vehicle including at least one of a traveling speed and a traveling direction of the vehicle.
The method of claim 1,
The processor is
determining a driving behavior of the vehicle based on the recognized driving pattern of the driver using one or more data recognition models.
The method of claim 1,
The processor is
An apparatus for acquiring the driving data including at least one of driving information of the vehicle, surrounding image information of the vehicle, and state information of the driver through the sensing unit.
delete delete The method of claim 1,
The processor is
An apparatus for controlling to display information related to the driving operation control of the vehicle on a display unit mounted on the vehicle as the driving operation of the vehicle is controlled.
The method of claim 1,
further comprising an output unit,
The processor is
In response to controlling the driving operation of the vehicle, the apparatus outputs information about the driving operation control of the vehicle through the output unit.
In the operating method of a wearable device assisting driving of a vehicle,
recognizing a driving pattern of a driver who mounts the device and drives the vehicle based on driving data acquired through a sensing unit while the vehicle is driving;
determining the current driving state of the driver based on the current driving data acquired through the sensing unit and the recognized driving pattern of the driver;
receiving, from the vehicle, a control signal related to a driving operation of the vehicle according to the current driving state of the driver;
determining a driving operation of the vehicle based on the current driving state of the driver and the received control signal;
generating a control signal for controlling the driving operation of the vehicle; and
and learning a driving motion of the vehicle according to the current driving state of the driver, received from the vehicle, by using one or more data recognition models.
11. The method of claim 10,
Recognizing the driver's driving pattern comprises:
Learning the driving pattern of the driver based on the driving data using one or more data recognition models.
11. The method of claim 10,
In response to determining that the current driving state of the driver is in a dangerous state, further comprising the step of blocking driving operation control by the driver;
The step of determining the driving operation of the vehicle comprises:
determining a driving motion of the vehicle including at least one of a traveling speed and a traveling direction of the vehicle.
◈Claim 13 was abandoned when paying the registration fee.◈ 11. The method of claim 10,
The method further comprising the step of acquiring the driving data including at least one of driving information of the vehicle, surrounding image information of the vehicle, and state information of the driver through the sensing unit.
delete delete ◈Claim 16 has been abandoned at the time of payment of the registration fee.◈ 11. The method of claim 10,
The method further comprising the step of controlling to display information on the driving operation control of the vehicle on a display mounted on the vehicle as the driving operation of the vehicle is controlled.
◈Claim 17 was abandoned when paying the registration fee.◈ 11. The method of claim 10,
In response to controlling the driving operation of the vehicle, the method further comprising the step of outputting information about the driving operation control of the vehicle through an output unit.
◈Claim 18 was abandoned when paying the registration fee.◈ A computer-readable recording medium recording a program for executing the method of claim 10 on a computer.

Images