KR20230169645A - Object recognition-based intelligence glasses system for preventing traffic accidents, and operation method for the same - Google Patents

Abstract

When it is necessary to be aware of dangerous situations in the vehicle driver's blind spot or dangerous driving behavior due to drowsy driving, the object recognition-based intelligent glasses can determine the dangerous driving behavior of the vehicle driver and issue a voice warning to the driver. In addition, it detects dangerous situations ahead using daytime image or nighttime image object recognition mounted on glasses that the vehicle driver can wear, and warns the vehicle driver of this with a voice to prevent traffic that occurs due to driver inattention or lack of visibility at night. Accidents can be prevented, and dangerous driving behavior can be determined by searching for vehicle driver risky driving behavior patterns such as speeding, rapid acceleration, rapid deceleration, sudden lane changes, or sharp turns for each vehicle type such as trucks, buses, passenger cars, and vans. An object recognition-based intelligent glasses system and its operating method for preventing traffic accidents that can immediately deliver warning messages to drivers by voice are provided.

Description

OBJECT RECOGNITION-BASED INTELLIGENCE GLASSES SYSTEM FOR PREVENTING TRAFFIC ACCIDENTS, AND OPERATION METHOD FOR THE SAME}

The present invention relates to an object recognition-based intelligent glasses system. More specifically, the object recognition-based intelligent glasses detect the driving behavior of the vehicle driver, determines dangerous driving behavior, and warns the vehicle driver to prevent traffic accidents. It relates to an object recognition-based intelligent glasses system for traffic accident prevention and its operating method.

According to traffic accident statistics from the National Police Agency, among the causes of traffic accidents, accidents caused by driver inattention were reported to be the highest at approximately 52%. In particular, traffic accidents caused by drowsy driving and lack of visibility at night are highly likely to lead to major accidents with casualties, so there is an urgent need to come up with solutions.

As a conventional technology for preventing traffic accidents, for example, an unexpected situation detector is installed on the road to inform the driver of dangerous situations ahead, such as falling objects, or a radar sensor attached to the vehicle is used to alert the driver of the risk of collision. Information technology is being applied.

However, in the case of emergency detectors installed on roads according to conventional technology, they are installed and operated only in certain sections of the road, and there is a limitation in that it is impossible to collect information on the entire section.

In addition, since the sensor-based collision risk service attached to the vehicle is applied only to some expensive vehicles, there is a limitation that the risk of traffic accidents still exists in the case of general vehicles.

Meanwhile, as a prior art for preventing drowsy driving, Republic of Korea Patent Publication No. 2011-5380 discloses an invention titled “Drowsiness warning system and glasses equipped with a drowsiness warning system.”

This relates to glasses that detect when a user lowers his or her head when drowsy and generate a vibration or voice warning to warn of drowsy driving, etc. This will be described in detail with reference to FIG. 1 .

1 is a configuration diagram showing a drowsiness warning system according to the prior art.

Referring to FIG. 1, the drowsiness warning system according to the prior art includes a control unit 11, an angle sensor unit 12, and a warning unit 13, and also includes an input unit 14 and a mode switching unit 15. and a timer 16.

The angle sensor unit 12 detects the angle at which the user's head is tilted. It is located near the user's head and detects the angle at which the user's head is tilted forward and backward when the user is dozing off.

This angle sensor unit 12 can be attached to a hair band, hair pin, or glasses to apply a drowsiness warning system without a separate device or terminal.

At this time, the angle sensor unit 12 measures the tilt angle from the reference plane, that is, the horizontal state, and generally uses angle measurement using an encoder, angle measurement using an acceleration sensor, and angle measurement using a gyro sensor. You can use it to measure the angle.

The control unit 11 controls the warning unit 13 by receiving a signal about the tilt angle detected by the angle sensor unit 12,

For example, in order to process noise for general head tilt, the warning unit 13 is controlled to output a warning signal only when the angle exceeds a predetermined angle. Therefore, the control unit 11 determines whether the received tilt angle exceeds a preset reference angle and operates the warning unit 13 only when it exceeds the reference angle. If it does not exceed the reference value, the warning signal is not output.

In other words, the drowsiness warning system according to the prior art includes an angle sensor unit, a control unit, and a warning unit. The angle sensor unit detects the tilt angle of the user's head, and the control unit detects the tilt angle detected by the angle sensor unit. It receives a signal and controls the warning unit, and outputs a warning signal from the warning unit only when the angle exceeds a predetermined angle, thereby preventing the user from drowsy driving.

According to the drowsiness warning system according to the prior art, if the user dozes off while driving, studying or working, this can be detected and a warning is issued to alert the user, thereby preventing the risk of traffic accidents.

In addition, it is possible to efficiently generate a drowsiness warning by detecting the head tilt angle step by step and amplifying the output signal in response.

However, in the case of the drowsiness warning system according to the prior art, because the drowsiness warning is generated only in response to the angle at which the user's head is tilted, there is a limitation in that the drowsiness warning can be generated even when the user is not actually dozing off.

Meanwhile, Smart Glasses are glasses-shaped devices equipped with a see-through function and a computer, and are a type of computer that the user can wear.

These smart glasses are composed of hardware such as a camera, an interface device capable of inputting commands, and a device that transmits images viewed through the human eye.

In addition, smart glasses can display augmented reality information on the display using a separate controller, or can use functions such as calls, texts, and alarms by linking with a smartphone.

As prior art related to such smart glasses, Republic of Korea Patent Publication No. 2020-68277 discloses an invention titled “Smart Glasses System for the Visually Impaired for Providing Traffic Information,” which discloses a smart glasses system for the visually impaired for providing traffic information. Regarding the glasses system, it will be described with reference to FIGS. 2A to 2C.

Figure 2a is a configuration diagram of a smart glasses system for the visually impaired for providing traffic information according to conventional technology, and Figure 2b is a diagram showing the configuration of a smart glasses system for the visually impaired shown in Figure 2a to detect objects by operating a camera and an ultrasonic sensor. This is a diagram to explain the process of generating an input signal, and FIG. 2C is a diagram to explain the process of converting traffic light-related information into a voice signal and outputting it in the smart glasses system for the visually impaired shown in FIG. 2A.

Referring to FIG. 2A, a smart glasses system for the visually impaired for providing traffic information according to the prior art may include smart glasses 20 and a server 30.

Smart glasses 20 may include an input interface 21, a camera 22, an ultrasonic sensor 23, a communication interface 24, an output interface 25, and a control unit 26.

The input interface 21 can generate an input signal according to the input operation of the visually impaired user.

For example, when the user presses the input button provided on the handle of the smart glasses 20 for the visually impaired or the user's voice command is input through the microphone provided on the handle of the smart glasses 20 for the visually impaired, The input interface 21 can generate an input signal according to the operation of an input button or a voice command.

In addition, the input interface 21, as shown in FIG. 2B, when the smart glasses 20 are adjacent to the display device 42 that receives traffic information by the traffic information provision system 41 within a certain distance, By recognizing this, an input signal can be generated. To this end, the display device 42 can generate a beacon signal, and when the smart glasses 20 are adjacent to each other within a certain distance from the display device 42, the input interface 21 detects the beacon signal and An input signal can be generated.

The camera 22 can capture an image of an object located in front, and the ultrasonic sensor 23 can measure the distance to the object.

The communication interface 24 can transmit output signals from each of the camera 22 and the ultrasonic sensor 23 to the server 30.

This communication interface 24 can exchange data with an external device or server 30 through a network.

That is, the communication interface 24 can transmit data processed inside the smart glasses 20 to the outside, and conversely, it can also receive data or information from an external server or external device.

Here, the server 30 may include an object detection unit 31, a memory 32, an image processing unit 33, a text-to-voice conversion unit 34, and a control unit 35.

In addition, as shown in FIG. 2C, the communication interface 24 is a traffic light that includes the location and signal status of a traffic light present at the closest location relative to the user, in other words, the closest location relative to the smart glasses 20. Related information can be provided from the traffic information provision system 41.

Accordingly, the output interface 24 can output information related to traffic lights converted into voice signals through the TTS system 43, which provides text-to-speech (TTS) technology so that users can recognize their voices.

The output interface 25 may receive a voice signal corresponding to the extracted text from the server 30 and output it. At this time, the output interface 25 can gradually speed up the generation cycle of the warning signal including a voice signal or a vibration signal as the arrival time of the means of transportation included in the extracted text becomes shorter.

The control unit 26 generally controls the operations of the input interface 21, camera 22, ultrasonic sensor 23, communication interface unit 24, and output interface 25 of the smart glasses 20.

In other words, in the case of a smart glasses system for the visually impaired for providing traffic information according to the conventional technology, an input signal is generated when adjacent to a display device that receives traffic information by a traffic information providing system within a certain distance, or the user's By including an input interface that generates an input signal by input manipulation, a camera and an ultrasonic sensor that captures an image of an object,

Detect objects from images captured based on the camera's output signal, run an object recognition algorithm to detect objects, identify objects related to traffic information, recognize text from the object, and convert it into a voice signal to provide traffic information. can do.

According to the smart glasses system for the visually impaired for providing traffic information according to conventional technology, it not only provides convenience of transportation use to the visually impaired but also provides safety by recognizing text from major objects related to traffic information, converting it into a voice signal and outputting it. Accidents can be prevented.

However, the smart glasses system for the visually impaired for providing traffic information according to conventional technology only provides traffic information to the visually impaired, and has limitations in application to vehicle drivers.

Meanwhile, as another prior art, Republic of Korea Patent Publication No. 2020-134401 discloses an invention titled "Method of operating smart glasses in conjunction with a mobile device," which provides real-time image processing according to the operation mode while linking with a mobile device. This relates to a method of operating smart glasses created and shown, and is explained with reference to FIGS. 3A and 3B.

FIG. 3A is a configuration diagram of smart glasses interoperating with a mobile device according to the prior art, and FIG. 3B is an operation flowchart of a method of operating smart glasses interoperating with a mobile device according to the prior art.

Referring to FIG. 3A, smart glasses that work with a mobile device according to the prior art relate to smart glasses that work with a mobile device 60 that runs a specific application for linking with the smart glasses 50.

The mobile device 60 is capable of interconnecting with various networks, including the Internet, and includes a GPS sensor, acceleration sensor, gyro sensor, gravity sensor, temperature/humidity sensor, altitude sensor, barometric pressure sensor, heart rate sensor, geomagnetic sensor, and image sensor. It refers to a mobile device that is equipped with one or more of the following and has a specific application installed that can collect and process sensing data from sensors in conjunction with smart glasses.

In addition, smart glasses 50 refer to a glasses-type projection HMD that can display image data generated by the mobile device 60 in a form close to the eyeball with multimedia content.

The smart glasses 50 may be equipped with a display interface, a control device connected to the display interface and including a communication unit, a processor unit, and memory, and an inertial measurement unit (IMU) connected to the control device.

Here, the communication unit provides a wireless or wired method for the smart glasses 50 to interact with the mobile device 60.

In addition, referring to FIG. 3B, a method of operating smart glasses according to the prior art is a method of operating smart glasses that links with a mobile device running a specific application for linking with smart glasses,

a) mode setting step (S11), b) IMU data acquisition step (S12), c) IMU data wireless transmission step (S13), d) mode image generation step (S14), e) mode image transmission step (S15), It includes f) a mode image reception step (S16) and g) a mode image projection step (S17).

First, in the mode setting step (S11) of the mobile device 60, a situation suitable for the situation desired by the user wearing smart glasses is set in the application of the mobile device.

At this time, the mode setting may be additionally set according to the operation situation of the smart glasses in the application of the mobile device. In other words, the display type of the mode image generated in the application can be changed by additionally setting the desired situation.

Specifically, this mode setting step (S11) is set directly by the user interface (UI) of the application of the mobile device, or the mode setting signal is transmitted from the smart glasses to the mobile device using voice recognition means or gesture recognition means provided by the smart glasses. It can be sent to the device and set in the application.

To this end, the control device of smart glasses may be equipped with a means to detect hand movements through image processing and control the images shown in the smart glasses based on this.

Additionally, the control device for smart glasses may be provided with a means to identify the user's voice and control the image displayed in the smart glasses based on this.

For example, the control device recognizes the hand in the image frame input for hand motion recognition, detects the image where the hand is located, or recognizes or predicts the direction of hand movement, and places content on the screen or displays the current image accordingly. You can move it, or place the current image overlapping with another image.

Such hand movement prediction includes recognizing hands, fingers, or a combination thereof, and gesture recognition can be performed based on image processing and preset gesture information.

In addition, the control device recognizes the user's input voice and can place content on the screen, move the current image, or arrange the current image to overlap with other images by commands given by the voice.

Voice recognition can be performed based on voice processing and preset voice information.

Next, the IMU data acquisition step (S12) acquires sensing data in real time through the IMU operating inside the smart glasses.

Here, the IMU may include one or more of an acceleration sensor, a gyro sensor, a rotational speed sensor, a geomagnetic sensor, an altitude sensor, and an atmospheric pressure sensor built into the smart glasses.

Next, in the IMU data wireless transmission step (S13), IMU data wirelessly transmits sensing data to a mobile device.

Next, the mode image generation step (S14) receives IMU data from the mobile device and processes it according to the mode preset in the mode setting step (S11) to generate image data.

This mode image creation step (S14) is a mode setting step by collecting and processing sensing information obtained from IMU sensors mounted on smart glasses in the application, information from one or more sensors mounted on the mobile device, and information on network services. This means generating an image of real-time data based on a preset mode in (S11).

Next, in the mode image transmission step (S15), the mode image is compressed and encrypted on the mobile device and transmitted to smart glasses in a streaming manner. Accordingly, it is possible to guarantee a certain level of performance in data transmission by performing encoding encryption, transmission control, and QoS (Quality of Service) on the mode image according to the streaming method.

Next, in the mode image reception step (S16), the smart glasses receive the mode image transmitted from the mobile device, and decrypt and decompress it.

Next, the mode image projection step (S17) displays the decompressed mode image in the smart glasses by HMD projection.

In the case of smart glasses that link with mobile devices according to conventional technology, the application installed on the mobile device uses sensing information obtained from the IMU sensor mounted on the smart glasses and information with sensors mounted on the mobile device as well as network service information. By collecting and imaging real-time data based on the mode set in the mode setting step, a mode image suitable for the situation can be created and displayed by projecting it onto smart glasses.

According to smart glasses that interoperate with mobile devices according to the conventional technology, a mode image according to each set mode is displayed on the smart glasses according to the operation method of the smart glasses that interoperate with a mobile device running a specific application for interoperating with smart glasses. By being projected, users of smart glasses can efficiently visualize information according to each situation.

However, smart glasses that link with mobile devices according to the prior art are intended for users of the smart glasses to visualize information according to each situation, and image data generated by the mobile device 60 provides multimedia content close to the eye. It is a glasses-type projection HMD that can display.

Meanwhile, as another prior art, Republic of Korea Patent No. 10-2287855 discloses an invention titled "Obstacle Detection Glasses for the Visually Impaired", which allows visually impaired people walking to easily recognize the presence or absence of obstacles while walking. Glasses that can prevent safety accidents, such as a visually impaired person falling over an obstacle and getting injured, will be described with reference to FIGS. 4A and 4B.

FIG. 4A is a perspective view schematically showing obstacle detection glasses for the visually impaired according to the prior art, and FIG. 4B is a configuration diagram schematically showing the control state of the control unit shown in FIG. 4A.

Referring to FIG. 4A, obstacle detection glasses 70 for the visually impaired according to the prior art are provided with an obstacle detection unit 71 and a notification unit 72.

First, the obstacle detection unit 71 is provided between the two lens fixing parts 71a of the glasses and is stably positioned in the receiving groove 72a formed on the inside of the connection frame 71b connecting the two lens fixing parts 71a. It can be accommodated and fixed, and at this time, the front side of the receiving groove (72a) can be open.

The obstacle detection unit 71 may be made of various types, such as an ultrasonic sensor that detects an obstacle by outputting ultrasonic waves.

In particular, it can be done with a lidar sensor that can predict the direction by observing the movement of people or animals, and can calculate and predict the location of static and dynamic obstacles at each point.

The notification unit 72 is provided on the temple 71c of the glasses and can inform the visually impaired of the presence of an obstacle according to the detection signal of the obstacle detection unit 71.

This notification unit 72 can be made of various types, and in particular, it can be made of bone conduction earphones such as bone conduction Bluetooth earphones that transmit vibration corresponding to the voice signal to the skull of a visually impaired person to enable the user to recognize the sound. You can.

In the case of obstacle detection glasses for the visually impaired according to the conventional technology, the visually impaired person walking can easily recognize the presence of an obstacle through the obstacle detection unit 71 and the notification unit 72, so that the visually impaired person can easily recognize the presence of an obstacle while walking. It is possible to prevent safety accidents such as falls and injuries due to

Furthermore, it allows visually impaired people to walk safely without discomfort and without being noticeable to others.

Additionally, referring to FIG. 4B, glasses equipped with an obstacle detection unit 71 and a notification unit 72 may be equipped with a motion detection unit 73 that detects movement of the glasses as a visually impaired person walks.

The motion detection unit 73 may be provided on the lens fixing part 71a or the temple 71c of the glasses. This motion detection unit 73,

For example, it consists of a gyro sensor and a 3-axis acceleration sensor that detects the movement of the glasses when a visually impaired person wearing glasses walks, or occurs when the beads built into the lens fixing part 71a or the temple 71c shake. It can be made up of various types, such as a vibration detection sensor that detects vibration.

In addition, a timer 74 that sets the operation time of the notification unit 72 and a control unit 75 that operates the notification unit 72 for the operating time set by the timer 74 according to the detection signal from the motion detection unit 73. may be provided. Additionally, a power supply unit 76 may be provided to supply power to the control unit 75, etc.

The timer 74 may be provided in various ways, such as a push button or dial button, on the control unit 75, and the control unit 75 may be provided on the temple 71c of the glasses.

In the case of obstacle detection glasses for the visually impaired according to the conventional technology, an obstacle detection unit consisting of an ultrasonic sensor and a lidar sensor is provided, and a notification unit is provided to indicate the presence or absence of an obstacle according to the detection signal of the obstacle detection unit, thereby visually indicating the presence or absence of an obstacle. It is designed to provide guidance to people with disabilities so that they can walk safely.

According to the obstacle detection glasses for the visually impaired according to the conventional technology, a notification unit that informs the visually impaired of the presence of an obstacle according to the detection signal of the obstacle detection unit provided in the glasses worn by the visually impaired and detects the obstacle is used to detect the presence of an obstacle while the visually impaired are walking. The presence or absence of this obstacle can be easily recognized. Accordingly, it is possible to prevent the possibility of safety accidents such as a visually impaired person falling over an obstacle and getting injured while walking.

As described above, various types of smart glasses have been disclosed for the visually impaired, but smart glasses that provide traffic information for general vehicle drivers have not been disclosed.

Republic of Korea Patent Publication No. 2020-68277 (publication date: June 15, 2020), title of invention: “Smart glasses system for the visually impaired for providing traffic information” Republic of Korea Patent Publication No. 2011-5380 (publication date: January 18, 2011), title of invention: “Drowsiness warning system and glasses with drowsiness warning system installed” Republic of Korea Patent Publication No. 2020-134401 (publication date: December 2, 2020), title of invention: “Method of operating smart glasses that interoperate with mobile devices” Republic of Korea Patent No. 10-2287855 (registration date: August 3, 2021), title of invention: “Obstacle detection glasses for the visually impaired” Republic of Korea Patent Publication No. 2015-144510 (publication date: December 28, 2015), title of invention: “Glasses system for the visually impaired”

The technical problem that the present invention aims to achieve in order to solve the above-mentioned problems is that when it is necessary to be careful of dangerous driving behavior due to dangerous situations in the vehicle driver's blind spot or drowsy driving, the vehicle driver can use object recognition-based intelligent glasses. The purpose is to provide an object recognition-based intelligent glasses system and its operating method for preventing traffic accidents that can determine dangerous driving behavior and warn the driver by voice.

Another technical problem to be achieved by the present invention is to detect a dangerous situation ahead using daytime or nighttime image object recognition mounted on glasses that can be worn by the vehicle driver, and to warn the vehicle driver of this with a voice, thereby preventing driver inattention or The purpose is to provide an object recognition-based intelligent glasses system and operating method for preventing traffic accidents that occur due to lack of visibility at night.

Another technical task to be achieved by the present invention is to detect dangerous driving behavior patterns of vehicle drivers such as speeding, rapid acceleration, rapid deceleration, sudden lane changes, or sharp turns for each vehicle type such as trucks, buses, passenger cars, and vans. The purpose is to provide an object recognition-based intelligent glasses system and its operating method for preventing traffic accidents that can deliver a warning message to the vehicle driver by voice by making a decision.

As a means of achieving the above-described technical problem, the object recognition-based intelligent glasses system for traffic accident prevention according to the present invention is mounted on intelligent glasses worn by a vehicle driver and is operated by a voice command from the vehicle driver A voice recognition unit that converts commands into text; It is mounted on intelligent glasses and connected to a mobile phone, acquires daytime or nighttime images, performs object recognition, and analyzes data from the driving behavior detection sensor to drive dangerous driving where the vehicle driver is driving dangerously due to drowsiness or distraction. An intelligent glasses control unit that searches for behavior patterns, determines risky driving behavior, and generates a warning message; and a voice conversion unit that converts the warning message generated by the intelligent glasses control unit into voice and generates a voice warning sound to guide the vehicle driver to drive safely.

Here, the intelligent glasses control unit searches for the vehicle driver's risky driving behavior pattern of speeding, rapid acceleration, rapid deceleration, sudden lane change, or sharp turn for each vehicle type such as a truck, bus, passenger car, or van, and determines the risky driving behavior criteria to determine the criteria for the vehicle driver. It is characterized by delivering a warning message by voice.

Here, the mobile phone is characterized by linking weather information provided by a weather information provider and traffic information provided by a traffic information provider.

Here, the intelligent glasses control unit includes a daytime image object recognition module that acquires a daytime image and recognizes an object; A night image object recognition module that acquires night images and recognizes objects; A mobile phone linkage module that provides weather information and traffic information on surrounding roads received by the mobile phone to a vehicle driver wearing intelligent glasses; A dangerous driving behavior determination module that searches for the vehicle driver's risky driving behavior pattern according to the detection signal detected from the driving behavior detection sensor and determines whether the driver is engaging in dangerous driving behavior; And it may include a warning message generation module that generates a warning message according to the determination result of the dangerous driving behavior determination module and transmits the warning message to the voice conversion unit to warn the vehicle driver by voice.

Here, the daytime image object recognition module can recognize an object by acquiring a daytime image based on a vision sensor, and the nighttime image object recognition module can recognize an object by acquiring a nighttime image based on an infrared camera.

Here, the object is an object that requires the attention of the vehicle driver and may be a falling object on the road, a pothole, a pedestrian, a preceding vehicle that suddenly stops, or a bicycle.

The object recognition-based intelligent glasses system for traffic accident prevention according to the present invention may further include an image output unit that outputs the daytime image and the nighttime image and stores them in the mobile phone.

Here, each of the daytime image object recognition module and the nighttime image object recognition module uses the RetinaNet model, an algorithm that recognizes the surrounding environment, to resolve imbalance in the data set, and the RetinaNet model measures the performance of the object recognition algorithm when learning through the algorithm. It is characterized by using the Focal Loss (FL) function for improvement.

Here, the dangerous driving behavior determination module includes: a driving behavior detection sensor attached to the intelligent glasses to detect the driving behavior of the vehicle driver; a data analysis unit that searches for and compares and analyzes driving behavior patterns corresponding to sensor data detected by the driving behavior detection sensor; Dangerous driving behavior pattern DB, which stores the dangerous driving behavior patterns of vehicle drivers; and a dangerous driving behavior determination unit that determines whether the driving behavior pattern compared and analyzed by the data analysis unit is dangerous driving behavior.

Here, the driving behavior detection sensor is a G (gravity) sensor or a gyro sensor that is attached to the intelligent glasses and can detect the driving behavior of the vehicle driver.

Here, the speech recognition unit can recognize speech based on an LSTM-HMM hybrid structure that combines a long short term model (LSTM) structural neural network with a hidden Markov model (HMM).

Meanwhile, as another means of achieving the above-described technical problem, the method of operating the object recognition-based intelligent glasses system for traffic accident prevention according to the present invention includes: a) a voice command through a voice recognition unit mounted on the intelligent glasses; Driving an intelligent glasses system; b) acquiring daytime image or nighttime image data through a daytime image or nighttime image object recognition module mounted on intelligent glasses, and recognizing an object from the daytime image or nighttime image data through an object recognition algorithm; c) detecting the driving behavior of the vehicle driver through a driving behavior detection sensor mounted on the intelligent glasses; d) searching the driving behavior pattern of the vehicle driver through a dangerous driving behavior determination module to determine whether the driving behavior of the vehicle driver is dangerous driving behavior; e) generating a warning message when the driving behavior is determined to be dangerous according to the risky driving behavior determination result; and f) generating a voice warning sound by converting the warning message into voice through a voice conversion unit for safe driving of the vehicle driver.

According to the present invention, when it is necessary to pay attention to dangerous situations in the vehicle driver's blind spot or dangerous driving behavior due to drowsy driving, etc., the dangerous driving behavior of the vehicle driver is determined through object recognition-based intelligent glasses and a voice message is sent to the driver. You can warn.

According to the present invention, a dangerous situation ahead is detected using daytime image or nighttime image object recognition mounted on glasses that can be worn by the vehicle driver, and a warning is given to the vehicle driver by voice to prevent driver inattention or lack of visibility at night. Traffic accidents can be prevented.

According to the present invention, the dangerous driving behavior patterns of vehicle drivers such as speeding, rapid acceleration, rapid deceleration, rapid lane change, or sharp turns are searched for each vehicle type such as trucks, buses, passenger cars, and vans, and warnings are issued to vehicle drivers by determining dangerous driving behavior. Messages can be delivered immediately by voice.

1 is a configuration diagram showing a drowsiness warning system according to the prior art.
Figure 2a is a configuration diagram of a smart glasses system for the visually impaired for providing traffic information according to conventional technology, and Figure 2b is a diagram showing the configuration of a smart glasses system for the visually impaired shown in Figure 2a to detect objects by operating a camera and an ultrasonic sensor. This is a diagram to explain the process of generating an input signal, and FIG. 2C is a diagram to explain the process of converting traffic light-related information into a voice signal and outputting it in the smart glasses system for the visually impaired shown in FIG. 2A.
FIG. 3A is a configuration diagram of smart glasses interoperating with a mobile device according to the prior art, and FIG. 3B is an operation flowchart of a method of operating smart glasses interoperating with a mobile device according to the prior art.
FIG. 4A is a perspective view schematically showing obstacle detection glasses for the visually impaired according to the prior art, and FIG. 4B is a configuration diagram schematically showing the control state of the control unit shown in FIG. 4A.
Figure 5 is a diagram for explaining the concept of an object recognition-based intelligent glasses system for traffic accident prevention according to an embodiment of the present invention.
Figure 6 is a schematic configuration diagram of an object recognition-based intelligent glasses system for traffic accident prevention according to an embodiment of the present invention.
Figure 7 is a detailed configuration diagram of an object recognition-based intelligent glasses system for traffic accident prevention according to an embodiment of the present invention.
Figure 8 is a diagram for specifically explaining object recognition in the object recognition-based intelligent glasses system for traffic accident prevention according to an embodiment of the present invention.
Figure 9 is a diagram specifically explaining the dangerous driving behavior determination module in the object recognition-based intelligent glasses system for traffic accident prevention according to an embodiment of the present invention.
Figures 10A to 10C are diagrams for specifically explaining dangerous driving behavior for each vehicle in the object recognition-based intelligent glasses system for traffic accident prevention according to an embodiment of the present invention.
Figure 11 is an operation flowchart showing a method of operating an object recognition-based intelligent glasses system for traffic accident prevention according to an embodiment of the present invention.

Below, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. In order to clearly explain the present invention in the drawings, parts that are not related to the description are omitted, and similar parts are given similar reference numerals throughout the specification.

Throughout the specification, when a part is said to “include” a certain element, this means that it may further include other elements rather than excluding other elements, unless specifically stated to the contrary. Additionally, terms such as “…unit” used in the specification refer to a unit that processes at least one function or operation, and may be implemented as hardware, software, or a combination of hardware and software.

[Object recognition-based intelligent glasses system for traffic accident prevention (100)]

FIG. 5 is a diagram illustrating the concept of an object recognition-based intelligent glasses system for traffic accident prevention according to an embodiment of the present invention, and FIG. 6 is an object recognition-based view for traffic accident prevention according to an embodiment of the present invention. This is a schematic diagram of the intelligent glasses system.

The object recognition-based intelligent glasses system 100 for traffic accident prevention according to an embodiment of the present invention,

As shown in Figure 5,

It is connected to the vehicle driver's mobile phone (or smartphone) 200 via Bluetooth or Wi-Fi, and provides weather information and traffic information received by the mobile phone 200 to the driver.

At this time, weather information on surrounding roads may be received from a weather information provider 300, for example, the Korea Meteorological Administration, and traffic information on surrounding roads may be received from a traffic information provider 400, for example, a traffic control center. It is not limited to this.

In addition, in the object recognition-based intelligent glasses system 100 for traffic accident prevention according to an embodiment of the present invention, a function of transmitting and storing an image (still cut) of an object recognized by the intelligent glasses to the mobile phone 200 Also performs

Specifically, as shown in FIG. 6, the object recognition-based intelligent glasses system 100 for traffic accident prevention according to an embodiment of the present invention,

It includes intelligent glasses 110, a voice recognition unit 120, an intelligent glasses control unit 130, an image output unit 140, and a voice conversion unit 150, and at this time, the intelligent glasses worn by the vehicle operator ( 110), the voice recognition unit 120, the intelligent glasses control unit 130, the image output unit 140, and the voice conversion unit 150 are mounted.

In the case of the object recognition-based intelligent glasses system 100 for traffic accident prevention according to an embodiment of the present invention,

When it is necessary to pay attention to dangerous situations in the vehicle driver's blind spot or dangerous driving behavior due to drowsy driving, etc., object recognition-based intelligent glasses can determine the dangerous driving behavior of the vehicle driver and warn the driver with a voice.

In addition, it detects dangerous situations ahead using daytime image or nighttime image object recognition mounted on glasses that the vehicle driver can wear, and warns the vehicle driver of this with a voice to prevent traffic that occurs due to driver inattention or lack of visibility at night. Accidents can be prevented.

Meanwhile, Figure 7 is a detailed configuration diagram of an object recognition-based intelligent glasses system for preventing traffic accidents according to an embodiment of the present invention.

Referring to FIG. 7, the object recognition-based intelligent glasses system 100 for traffic accident prevention according to an embodiment of the present invention is,

It includes intelligent glasses 110, voice recognition unit 120, intelligent glasses control unit 130, video output unit 140, and voice conversion unit 150,

The intelligent glasses control unit 130 includes a daytime image object recognition module 131, a nighttime image object recognition module 132, a mobile phone linkage module 133, a dangerous driving behavior determination module 134, and a warning message generation module 135. may include.

The voice recognition unit 120 is mounted on the intelligent glasses 110 worn by the vehicle driver, and is operated by the vehicle driver's voice command to convert the voice command into text.

For example, the object recognition-based intelligent glasses system 100 for traffic accident prevention according to an embodiment of the present invention is operated by the vehicle driver's voice command, and the voice recognition unit 120 is LSTM (Long Short-Term Memory)-HMM (Hidden Markov Model) converts voice commands into text using a voice recognition model based on the algorithm.

Specifically, the speech recognition unit 120 recognizes speech based on an LSTM-HMM hybrid structure that combines a Long Short Term Memory (LSTM) structured neural network with a Hidden Markov Model (HMM).

Here, LSTM is a modified form of RNN (Recurrent Neural Networks), one of the deep learning techniques specialized for time series data prediction, etc.

It is a model that complements the limitation of the existing RNN (Recurrent Neural Network), which shows a decrease in learning ability (vanishing gradient problem) on long-term data with long distances between information.

In addition, HMM (Hidden Markov Model) is the most representative model that recognizes sequential data or context-dependent data and refers to a model that uses probability.

The intelligent glasses control unit 130 is mounted on the intelligent glasses 110 and connected to the mobile phone 200, acquires daytime images or night images, performs object recognition, and analyzes data from the driving behavior detection sensor. It searches for patterns of dangerous driving behavior in which vehicle drivers drive dangerously due to drowsiness or distraction, determines dangerous driving behavior, and generates a warning message.

Specifically, here, the intelligent glasses control unit 130,

A daytime image object recognition module 131 that acquires daytime images and recognizes objects;

A night image object recognition module 132 that acquires night images and recognizes objects;

A mobile phone linking module 133 that provides weather information and traffic information on surrounding roads received by the mobile phone 200 to a vehicle driver wearing intelligent glasses 110;

A dangerous driving behavior determination module 134 that searches for the vehicle driver's risky driving behavior pattern according to the detection signal detected from the driving behavior detection sensor 134a and determines whether the driver is engaging in dangerous driving behavior. and

It includes a warning message generation module 135 that generates a warning message according to the determination result of the dangerous driving behavior determination module 134 and transmits the warning message to the voice conversion unit 150 to warn the vehicle driver by voice. can do.

For example, in the daytime, the daytime image object recognition module 131 based on a vision sensor (EO camera) operates, and in the case at night, the nighttime image object recognition module 132 based on an infrared (IR) camera operates.

At this time, the recognized object types may include objects that require attention information from the driver, such as falling objects on the road, potholes, pedestrians, vehicles that suddenly stop, and bicycles, but are not limited to these.

Accordingly, by using daytime or nighttime image object recognition mounted on glasses that the vehicle driver can wear, dangerous situations ahead are detected and a voice warning is issued to the vehicle driver to prevent damage caused by driver inattention or lack of visibility at night. Traffic accidents can be prevented.

Weather information and traffic information on surrounding roads received by the mobile phone 200 are provided to the vehicle driver wearing the intelligent glasses 110 through the mobile phone linking module 133.

The dangerous driving behavior determination module 134 analyzes G and Gyro sensor data mounted on the intelligent glasses 110 to find patterns of dangerous driving behavior in which the vehicle driver drives dangerously due to drowsiness or distraction.

The criteria for determining this risky driving behavior pattern are as shown in FIGS. 10A to 10C, which will be described later.

The video output unit 140 outputs the daytime video and the nighttime video and stores them in the mobile phone 200.

Additionally, the mobile phone 200 can link with weather information provided by the weather information provider 300 and traffic information provided by the traffic information provider 400.

The information detected in the dangerous driving behavior determination module 134 generates a warning message by the warning message generation module 135, and this warning message is sent to the voice conversion unit 150 to immediately warn the vehicle driver by voice. It is delivered.

The voice conversion unit 150 converts the warning message generated by the intelligent glasses control unit 130 into a voice and generates a voice warning sound to guide the vehicle driver to drive safely.

Therefore, the intelligent glasses control unit 130 searches for the vehicle driver's risky driving behavior pattern of speeding, rapid acceleration, rapid deceleration, sudden lane change, or sharp turn for each vehicle type such as a truck, bus, passenger car, or van, and determines the risky driving behavior standard. Warning messages can be delivered by voice to vehicle drivers.

Meanwhile, Figure 8 is a diagram for specifically explaining object recognition in the object recognition-based intelligent glasses system for traffic accident prevention according to an embodiment of the present invention.

Referring to FIG. 8, in the case of the object recognition-based intelligent glasses system 100 for traffic accident prevention according to an embodiment of the present invention,

The daytime image object recognition module 131 acquires daytime images based on a vision sensor and recognizes objects, and the night image object recognition module 132 acquires nighttime images based on an infrared camera to recognize objects. can be recognized,

The object requires the driver's attention and may be a falling object on the road, a pothole, a pedestrian, a suddenly stopping preceding vehicle, or a bicycle, but is not limited thereto.

The object recognition modules 131 and 132 used in the object recognition-based intelligent glasses system 100 for traffic accident prevention according to an embodiment of the present invention include a data collection unit, a modeling unit, a learning unit, an object recognition DB, and an object, respectively. It can be implemented as a recognition unit,

For example, recognize objects using the RetinaNet model.

That is, each of the daytime image object recognition module 131 and the nighttime image object recognition module 132 resolves imbalance in the data set using the RetinaNet model, an algorithm that recognizes the surrounding environment, and the RetinaNet model learns through an algorithm. The Focal Loss (FL) function can be used to improve the performance of the object recognition algorithm, but is not limited to this.

Specifically, the RetinaNet model, which is an algorithm that recognizes the surrounding environment, uses the Focal Loss (FL) function to improve the performance of the object recognition algorithm when learning through the algorithm. This FL function is based on the Cross Entropy (CE) function as shown in Equation 1 and Equation 2. Equation 1 can be simply expressed as Equation 2 below.

[Equation 1]

[Equation 2]

Here, (p,y) means that the probability of being recognized as a y object is p. In most machine learning algorithms, the imbalance of learning data (for example, when training an algorithm to recognize potholes, there are 10,000 images without potholes, but only 100 images with potholes) reduces the performance of the algorithm. It acts as a cause of deterioration.

To resolve this imbalance in the data set, the RetinaNet model assigns a low weight to images with a large number of samples (e.g., images without potholes) as shown in Equation 3, and to images with a small number of samples (e.g., images with potholes). video) uses a method that assigns high weight. In equation 3: is a weight parameter for resolving data set imbalance.

[Equation 3]

Accordingly, by using daytime or nighttime image object recognition mounted on glasses that the vehicle driver can wear, dangerous situations ahead are detected and a voice warning is issued to the vehicle driver to prevent damage caused by driver inattention or lack of visibility at night. Traffic accidents can be prevented.

In addition, it provides road weather/traffic information in conjunction with a mobile phone, configures an algorithm to recognize objects to be careful of when driving, and trains the algorithm to improve the performance of the object recognition algorithm.

Meanwhile, Figure 9 is a diagram for specifically explaining the dangerous driving behavior determination module in the object recognition-based intelligent glasses system for traffic accident prevention according to an embodiment of the present invention, and Figures 10a to 10c are embodiments of the present invention. This is a drawing to specifically explain the risky driving behavior of each vehicle in the object recognition-based intelligent glasses system for traffic accident prevention.

Referring to FIG. 9, the dangerous driving behavior determination module 134 in the object recognition-based intelligent glasses system for traffic accident prevention according to an embodiment of the present invention is,

A driving behavior detection sensor 134a attached to the intelligent glasses 110 and detecting the driving behavior of the vehicle driver; a data analysis unit (134b) that searches for and compares and analyzes driving behavior patterns corresponding to sensor data detected by the driving behavior detection sensor (134a); Dangerous driving behavior pattern DB (134c), which stores the dangerous driving behavior pattern of the vehicle driver; And it may include a dangerous driving behavior determination unit 134d that determines whether the driving behavior pattern compared and analyzed by the data analysis unit 134b is dangerous driving behavior.

Here, the driving behavior detection sensor 134a is a G (gravity) sensor or a gyro sensor that is attached to the intelligent glasses 110 and can detect the driving behavior of the vehicle driver.

Search for vehicle driver risky driving behavior patterns such as speeding, rapid acceleration, rapid deceleration, rapid lane change, or sharp turns by vehicle type, such as the truck shown in Figure 10a, the bus shown in Figure 10b, and the passenger car or van shown in Figure 10c. By determining the standards for dangerous driving behavior, a warning message can be immediately delivered by voice to the vehicle driver.

In other words, according to an embodiment of the present invention, vehicle driver risky driving behavior patterns such as speeding, rapid acceleration, rapid deceleration, sudden lane change, or sharp turns are searched for each vehicle type such as trucks, buses, passenger cars, and vans, and risky driving behavior criteria are determined. By determining, a warning message can be delivered by voice to the vehicle driver.

According to an embodiment of the present invention, the driver's voice command is converted into text, and objects requiring driver attention information, such as falling objects on the road and potholes, are recognized and images are output through a vision sensor and an infrared camera, and the mobile phone receives the information in conjunction with the mobile phone. It provides weather and road traffic information and determines whether the driver is drowsy or distracted through gravity (G) and gyro sensors, and generates a voice warning message to the driver through the provided information. By training the algorithm to improve the performance of the object recognition algorithm and resolving the learning data set imbalance, information on forward danger situations can be received precisely.

[How an object recognition-based intelligent glasses system works to prevent traffic accidents]

Figure 11 is an operation flowchart showing a method of operating an object recognition-based intelligent glasses system for traffic accident prevention according to an embodiment of the present invention.

Referring to Figure 11, the method of operating the object recognition-based intelligent glasses system for traffic accident prevention according to an embodiment of the present invention is:

First, the vehicle driver links the intelligent glasses 110 with the mobile phone 200 and drives the vehicle while receiving weather information and traffic information on surrounding roads linked to the mobile phone 200 (S110).

Next, the intelligent glasses system 100 is driven by a voice command through the voice recognition unit 120 mounted on the intelligent glasses 110 (S120).

For example, speech can be recognized based on an LSTM-HMM hybrid structure that combines a Long Short Term Memory (LSTM) structured neural network with a Hidden Markov Model (HMM).

Next, daytime image or nighttime image data is acquired through the daytime image or nighttime image object recognition modules 131 and 132 mounted on the intelligent glasses 110 (S130).

Here, the daytime image can be acquired based on a vision sensor, and the nighttime image can be acquired based on an infrared camera, and the object requires the attention of the vehicle driver, such as a falling object on the road, a pothole, a pedestrian, a vehicle ahead of a sudden stop, or a bicycle. It may be, but it is not limited to this.

Next, an object is recognized from the daytime image or nighttime image data through an object recognition algorithm, and the image is output to the mobile phone 200 through the image output unit 140 (S140).

At this time, the imbalance of the data set is resolved using the RetinaNet model, an algorithm that recognizes the surrounding environment, and the RetinaNet model can use the Focal Loss (FL) function to improve the performance of the object recognition algorithm when learning through the algorithm.

Additionally, the daytime video or nighttime video data can be output to the mobile phone 200 through the video output unit 140 and stored. Therefore, by using daytime or nighttime image object recognition mounted on glasses that can be worn by the vehicle driver, dangerous situations ahead are detected and a voice warning is issued to the vehicle driver to prevent traffic accidents caused by driver inattention or lack of visibility at night. can be prevented.

Next, the driving behavior of the vehicle driver is detected through the driving behavior detection sensor 141 mounted on the intelligent glasses 110 (S150).

For example, the driving behavior detection sensor 134a is a G (gravity) sensor or a gyro sensor and can be attached to the intelligent glasses 110 to detect the driving behavior of the vehicle driver.

Next, the driving behavior pattern of the vehicle driver is searched through the dangerous driving behavior determination module 134 to determine whether the driving behavior of the vehicle driver is dangerous driving behavior (S160).

Next, if the driving behavior is determined to be dangerous according to the results of the determination of the dangerous driving behavior, a warning message is generated (S170).

Therefore, when it is necessary to pay attention to dangerous situations in the vehicle driver's blind spot or dangerous driving behavior due to drowsy driving, etc., the vehicle driver's dangerous driving behavior can be determined through object recognition-based intelligent glasses and a voice warning to the driver can be issued. .

For example, by searching for patterns of dangerous driving behavior of vehicle drivers such as speeding, rapid acceleration, rapid deceleration, sudden lane changes, or sharp turns for each vehicle type such as trucks, buses, passenger cars, and vans, it determines dangerous driving behavior and sends a warning message to the vehicle driver. can be conveyed immediately by voice.

Next, for safe driving of the vehicle driver, a voice warning sound is generated by converting the warning message into voice through the voice conversion unit 150 (S180).

Ultimately, according to an embodiment of the present invention, the driver's voice command is converted into text, and objects that require driver attention information, such as falling objects on the road and potholes, are recognized and images are output through a vision sensor and an infrared camera, and linked with a mobile phone. It provides weather and road traffic information received, and determines whether the driver is drowsy or distracted through G and Gyro sensors, generates a voice warning message to the driver through the provided information, and provides object recognition algorithm performance. By learning the algorithm to increase the learning data set imbalance, it is possible to receive information on forward danger situations precisely.

In particular, the existing technology had limitations in preventing traffic accidents because it could only receive information on dangerous situations ahead in some sections or some vehicles. However, according to an embodiment of the present invention, any driver wearing glasses can receive information on dangerous situations ahead from anywhere. By providing information, road traffic risks can be significantly reduced.

Additionally, according to a report from the U.S. Department of Transportation, traffic accidents can be reduced by about 80% if drivers are aware of information about dangerous situations ahead in advance.

According to an embodiment of the present invention, it can be applied to any vehicle driver who can wear glasses while driving.

The description of the present invention described above is for illustrative purposes, and those skilled in the art will understand that the present invention can be easily modified into other specific forms without changing the technical idea or essential features of the present invention. will be. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive. For example, each component described as unitary may be implemented in a distributed manner, and similarly, components described as distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention. do.

100: Intelligent glasses system 200: Mobile phone (smart phone)
300: Weather information provider 400: Traffic information provider
110: Intelligent glasses 120: Voice recognition unit
130: Intelligent glasses control unit 140: Image output unit
150: Voice conversion unit 131: Weekly video object recognition module
132: Night image object recognition module 133: Mobile phone linkage module
134: Dangerous driving behavior determination module 135: Warning message generation module
134a: Driving behavior detection sensor 134b: Data analysis unit
134c: Dangerous driving behavior pattern DB 134d: Dangerous driving behavior determination unit

Claims (20)

In the intelligent glasses system for traffic accident prevention,
A voice recognition unit 120 mounted on the intelligent glasses 110 worn by the vehicle driver and operated by the vehicle driver's voice command to convert the voice command into text;
It is mounted on the intelligent glasses 110 and connected to the mobile phone 200, acquires daytime or nighttime images, performs object recognition, and analyzes data from the driving behavior detection sensor to prevent the vehicle driver from being drowsy or distracted. An intelligent glasses control unit 130 that searches for dangerous driving behavior patterns, determines dangerous driving behavior, and generates a warning message; and
An object for preventing traffic accidents, including a voice conversion unit 150 that converts the warning message generated by the intelligent glasses control unit 130 into a voice and generates a voice warning sound to encourage the vehicle driver to drive safely. Perception-based intelligent glasses system. According to paragraph 1,
The intelligent glasses control unit 130 searches for vehicle driver risky driving behavior patterns such as speeding, rapid acceleration, rapid deceleration, sudden lane change, or sharp turns for each vehicle type such as a truck, bus, passenger car, or van, and determines the risky driving behavior standards to determine the vehicle's risky driving behavior standards. An object recognition-based intelligent glasses system for traffic accident prevention that delivers a warning message to the driver by voice. According to paragraph 1,
The mobile phone 200 is an object recognition-based intelligent glasses system for traffic accident prevention, characterized in that it is linked to weather information provided by the weather information provider 300 and traffic information provided by the traffic information provider 400. The method of claim 1, wherein the intelligent glasses control unit 130,
A daytime image object recognition module 131 that acquires daytime images and recognizes objects;
A night image object recognition module 132 that acquires night images and recognizes objects;
A mobile phone linking module 133 that provides weather information and traffic information on surrounding roads received by the mobile phone 200 to a vehicle driver wearing intelligent glasses 110;
A dangerous driving behavior determination module 134 that searches for the vehicle driver's risky driving behavior pattern according to the detection signal detected from the driving behavior detection sensor 134a and determines whether the driver is engaging in dangerous driving behavior. and
It includes a warning message generation module 135 that generates a warning message according to the determination result of the dangerous driving behavior determination module 134 and transmits the warning message to the voice conversion unit 150 to warn the vehicle driver by voice. Object recognition-based intelligent glasses system for traffic accident prevention. According to paragraph 4,
The daytime image object recognition module 131 recognizes objects by acquiring daytime images based on a vision sensor, and the night image object recognition module 132 recognizes objects by acquiring nighttime images based on an infrared camera. An object recognition-based intelligent glasses system for preventing traffic accidents. According to clause 5,
The object is an object that requires the attention of the vehicle driver and is an object recognition-based intelligent glasses system for preventing traffic accidents, characterized in that it is a falling object on the road, a pothole, a pedestrian, a vehicle ahead of a sudden stop, or a bicycle. According to paragraph 4,
The daytime image object recognition module 131 and the nighttime image object recognition module 132 each resolve imbalance in the data set using the RetinaNet model, an algorithm that recognizes the surrounding environment, and the RetinaNet model recognizes objects when learning through the algorithm. An object recognition-based intelligent glasses system for traffic accident prevention, characterized by using the Focal Loss (FL) function to improve the performance of the recognition algorithm. The method of claim 4, wherein the dangerous driving behavior determination module 134,
A driving behavior detection sensor 134a attached to the intelligent glasses 110 and detecting the driving behavior of the vehicle driver;
a data analysis unit (134b) that searches for and compares and analyzes driving behavior patterns corresponding to sensor data detected by the driving behavior detection sensor (134a);
Dangerous driving behavior pattern DB (134c), which stores the dangerous driving behavior pattern of the vehicle driver; and
An object recognition-based intelligent glasses system for traffic accident prevention, including a dangerous driving behavior determination unit (134d) that determines whether the driving behavior pattern compared and analyzed by the data analysis unit (134b) is dangerous driving behavior. According to clause 8,
The driving behavior detection sensor 134a is a G (gravity) sensor or a gyro sensor attached to the intelligent glasses 110 and detects the driving behavior of the vehicle driver. Glasses system. According to paragraph 1,
The voice recognition unit 120 is a traffic accident characterized in that it recognizes voices based on an LSTM-HMM hybrid structure that combines a Long Short Term Memory (LSTM) structured neural network with a Hidden Markov Model (HMM). Object recognition-based intelligent glasses system for prevention. According to paragraph 1,
An object recognition-based intelligent glasses system for traffic accident prevention, further comprising an image output unit (140) that outputs the daytime image and the nighttime image and stores them in the mobile phone (200). In the method of operating an intelligent glasses system for traffic accident prevention,
a) operating the intelligent glasses system 100 with a voice command through the voice recognition unit 120 mounted on the intelligent glasses 110;
b) Obtain daytime or nighttime image data through the daytime or nighttime image object recognition modules 131 and 132 mounted on the intelligent glasses 110, and identify objects from the daytime or nighttime image data through an object recognition algorithm. Recognizing step;
c) detecting the driving behavior of the vehicle driver through the driving behavior detection sensor 134a mounted on the intelligent glasses 110;
d) searching the driving behavior pattern of the vehicle driver through the dangerous driving behavior determination module 134 to determine whether the driving behavior of the vehicle driver is dangerous driving behavior;
e) generating a warning message when the driving behavior is determined to be dangerous according to the risky driving behavior determination result; and
f) A method of operating an object recognition-based intelligent glasses system for traffic accident prevention, including the step of generating a voice warning sound by converting a warning message into voice through the voice conversion unit 150 for safe driving of the vehicle driver. According to clause 12,
The dangerous driving behavior determination module 134 in step d) searches for the dangerous driving behavior patterns of vehicle drivers such as speeding, rapid acceleration, rapid deceleration, sudden lane changes, or sharp turns for each vehicle type such as a truck, bus, passenger car, or van, and determines the dangerous driving behavior. A method of operating an object recognition-based intelligent glasses system for traffic accident prevention, characterized in that a warning message is delivered by voice to the vehicle driver by determining a standard. According to clause 12,
Before performing step a), the vehicle driver links the intelligent glasses 110 with the mobile phone 200 and drives the vehicle while receiving weather information and traffic information on surrounding roads linked to the mobile phone 200. How an object recognition-based intelligent glasses system works to prevent traffic accidents. According to clause 12,
A method of operating an object recognition-based intelligent glasses system for traffic accident prevention, characterized in that the daytime image or nighttime image data in step b) is output to the mobile phone 200 through the image output unit 140. According to clause 12,
A method of operating an object recognition-based intelligent glasses system for traffic accident prevention, characterized in that the daytime image is acquired based on a vision sensor, and the nighttime image is acquired based on an infrared camera. According to clause 12,
The object is an object that requires the attention of the vehicle driver and is a falling object on the road, a pothole, a pedestrian, a suddenly stopping preceding vehicle, or a bicycle. A method of operating an object recognition-based intelligent glasses system for traffic accident prevention. According to clause 12,
In step b), the imbalance of the data set is resolved using the RetinaNet model, an algorithm that recognizes the surrounding environment, and the RetinaNet model uses the Focal Loss (FL) function to improve the performance of the object recognition algorithm when learning through the algorithm. A method of operating an object recognition-based intelligent glasses system for preventing traffic accidents. According to clause 12,
The driving behavior detection sensor 134a in step c) is a G (gravity) sensor or a gyro sensor and is attached to the intelligent glasses 110 to detect the driving behavior of the vehicle driver. How a perception-based intelligent glasses system works. According to clause 12,
In step a), voice recognition is performed based on a LSTM-HMM hybrid structure that combines a Long Short Term Memory (LSTM) neural network with a Hidden Markov Model (HMM) to prevent traffic accidents. How an object recognition-based intelligent glasses system works.