KR102051136B1 - Artificial intelligence dashboard robot base on cloud server for recognizing states of a user - Google Patents

Abstract

The present invention captures the driver's face information and driving conditions of the vehicle and transmits it to the cloud server, the cloud server monitors the driver's state through image recognition, processing, and analysis, and the dashboard robot sends notification information when the driver is drowsy. The present invention relates to a user state recognition system using an artificial intelligence robot robot based on a cloud server that can be transmitted to the server. The present invention not only recognizes the driver's face, but also various factors such as the vehicle's driving speed, driving direction, and the like, and transmits the data to the artificial intelligence cloud server to determine whether the user's drowsiness is analyzed step by step after being analyzed by the server, and determined to be drowsy driving. In case of drowsy anti-drowsy spray or by providing a function such as oxygen supply, warning sound, artificial intelligence and telephone connection to the set phone number, there is an advantage that can prevent drowsy driving of the driver.

Description

User State Recognition System using AI Dashboard Robot Based on Cloud Server {ARTIFICIAL INTELLIGENCE DASHBOARD ROBOT BASE ON CLOUD SERVER FOR RECOGNIZING STATES OF A USER}

The present invention relates to a system for recognizing a user's state through a robot installed on a dashboard of a vehicle, and more particularly, photographing a driver's face information and a driving situation of a vehicle and transmitting the same to a cloud server, and the cloud server displays an image. The present invention relates to a user state recognition system using an intelligent dashboard robot based on a cloud server that monitors a driver's state through recognition, processing, and analysis, and transmits notification information to a dashboard robot when the driver is drowsy.

Recently, the most common cause of death accidents is drowsiness driving, with the increase in the number of passengers alone and the increase of long-distance and long-distance drivers such as trucks and buses due to the reduction of the number of passengers in the car. Accordingly, there is a demand for measures to prevent drowsy driving accidents. There are many technical approaches for detecting and determining drowsy driving, but many resources such as a high-end CPU and a GPU for graphics processing are required to install and process the drowsy driving. .

In addition, there are many uses of cars and various technical approaches to prevent drowsiness driving.However, a method of detecting drowsiness by detecting eyelids is most commonly used, and a technical approach of judging a state through artificial intelligence by recognizing a face is still in use. This has not been attempted, because a high specification graphic processing apparatus is required for image processing through artificial intelligence.

In particular, the robot's central processing unit and image processing unit for installing the robot robot in the car and performing algorithms for image processing and drowsiness recognition are very large in size and are very expensive to install and operate. It is difficult to access easily because it is difficult to put it on the board and use it, but there is a risk of an accident, and there is a risk of an accident due to the delay of judgment of drowsy driving in a fast car due to the limitation of processing speed.

Korean Patent No. 10-0778059 (hereinafter referred to as “priority document”) is equipped with a GPS and a communication unit inside the driver's drowsy driving prevention device to provide information on the drowsy driving situation and the driving position of the vehicle during drowsy driving. The present invention relates to a drowsiness driving prevention device using face recognition technology and a drowsiness driving prevention system using the same, which can prevent a large accident in advance by notifying a center or a traffic accident prevention management agency to the center.

Stand-alone dashboard robots, such as the prior literature, is a very expensive robotic system requires a high-end hardware platform. Therefore, it is necessary to configure the driver condition monitoring and other services by providing a state-based service using artificial intelligence in the cloud based to enable cheaper and faster processing power and services, and to configure big data collection. .

1. Korea Patent Publication No. 10-0778059 'Drowsiness operation prevention device using face recognition technology and drowsiness operation prevention system using the same' (application date 2007.05.22) 2. Korean Patent Publication No. 10-1023819 'Pan Tilt Device' (Application Date 2009.05.26.)

The present invention has been devised to solve the above problems, as well as the driver's face recognition, data of various factors such as the driving speed, driving direction, etc. of the data and transmits it to the AI cloud server to analyze the server sleepiness after analysis If it is determined step by step, if it is determined that drowsiness operation is to spray drowsiness-preventing fragrance or provide functions such as oxygen supply, warning sound, artificial intelligence and dialogue and telephone connection to the set phone number.

In addition, the present invention is intended to receive a lot of resources required for artificial intelligence through the cloud service based on wireless communication in order to minimize the resources required for state recognition through the driver's face recognition.

A user state recognition system using an intelligent dashboard robot based on a cloud server in accordance with an aspect of the present invention for achieving the above object is a front camera for photographing the driver's face and the steering wheel of the vehicle, to shoot the driving situation of the vehicle Rear camera, sensor unit for measuring the internal environment of the vehicle; The image information collecting unit which stores the front image photographed by the front camera and the driving image photographed by the rear camera, and the internal environment information measured by the front image, the driving image and the sensor unit can be read through a wireless communication network. Receives the front image, the driving image and the internal environment information from the dashboard robot and the dashboard robot including the image environment information transmission unit for transmitting to the outside every set period and generates the first face image of the driver in the front image as template data. The information collecting unit, and comparing and analyzing the template data and the face image in the front image to determine the emotion and state of the driver, the drowsiness of the driver through the steering image, the driving image of the vehicle in the front image It includes a cloud server containing an artificial intelligence processing unit for analyzing the.

The dashboard robot according to the present invention receives the driver's emotion and state information and the driver's drowsy information from the cloud server, and the driver's drowsiness according to the frequency of the drowsy driving of the driver, the level of sensitivity and the state information. A motion control unit equipped with an analysis information collecting unit for analyzing the initial, middle, and drowsy driving stages, and a notification device for recognizing a notification visually, tactilely, and audibly to the driver, and the motion control unit according to the drowsiness stage of the driver. It further includes a main control unit to operate selectively or sequentially.

The cloud server according to the present invention is a user information database for storing the cumulative information of the driver based on the front image, the driving image and the internal environment information transmitted from the dashboard robot, and the accumulated information stored in the user information database Based on the information learning processing unit for increasing the accuracy of the drowsiness determination of the driver.

The present invention not only recognizes the driver's face, but also various factors such as the vehicle's driving speed, driving direction, and the like, and transmits the data to the AI cloud server to determine whether the user's drowsiness is analyzed step by step after being analyzed by the server. In case of drowsy anti-drowsy spray or by providing a function such as oxygen supply, warning sound, artificial intelligence and telephone connection to the set phone number, there is an advantage that can prevent drowsy driving of the driver.

In addition, the present invention provides a more economical drowsy driving and safe driving assistant by receiving a lot of resources required for artificial intelligence through a cloud service based on wireless communication in order to minimize the resources required for state recognition through the face recognition of the driver There is an advantage to implement a dashboard robot of functionality.

1 is a perspective view showing a dashboard robot according to the present invention.
2 is a conceptual diagram of a user state recognition system using an artificial intelligence dashboard robot based on a cloud server according to the present invention.
3 is a configuration diagram of a user state recognition system using an artificial intelligence dashboard robot based on a cloud server according to the present invention.
4 is a diagram illustrating an internal hierarchical structure of the dashboard robot according to the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In describing the embodiments of the present invention, when it is determined that the detailed description of the related known technology may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

1 is a perspective view showing a dashboard robot according to the present invention. The dashboard robot of the present invention operates on an ecloud basis. The dashboard robot is preferably installed on the indoor space of the vehicle, in particular on the dashboard, and corresponds to a device and a black box for determining whether the driver is drowsy. In addition, it is installed in an indoor space of a home or building to check the appearance of the indoor space through a desired image, and two-way conversation is possible.

Referring to FIG. 1, the dashboard robot according to the present invention may include an upper housing 100, a photographing guide 200, a front camera 300 and a lower housing, a rear camera 500, and a sensor unit 210. have.

The upper housing 100 forms a hollow shape and one side is open. The photographing guide 200 is mounted inside the upper housing 100 and rotates up, down, left, and right. At this time, the photographing guide 200 and the upper housing 100 is made of a pan tilt structure, a detailed description of the 'pan tilt device' described in Korea Patent Publication No. 10-2009-0046015 (prior document 2) See. 'Pan tilt device' is a known technique for moving the camera left and right (pan), up and down (tilt), its detailed description will be omitted.

The photographing guide unit 200 includes a front camera 300 that photographs the driver's face and the handle. The shooting guide unit 200 tracks the driver's face so that the driver's face can be continuously photographed using the image information and artificial intelligence photographed through the front camera 300.

Referring to FIG. 1, a rear camera 500 installed in the accommodation space S of the lower housing 400 is provided. The rear camera 500 continuously photographs the driving situation of the vehicle.

Sensor unit 510 is a device for measuring the internal environment of the vehicle. The sensor unit 510 may be provided as a device for sensing air by measuring the carbon dioxide concentration of the indoor space. In addition, the sensor unit 510 may detect temperature, humidity, brightness, and the like in the vehicle.

2 is a conceptual diagram of a user state recognition system using an artificial intelligence dashboard robot based on a cloud server according to the present invention. Referring to FIG. 2, the artificial intelligence is not mounted on the dashboard robot 900 but is mounted on the cloud server 700. The cloud server 700 analyzes and processes various images and environmental information collected from the dashboard robot 900, accumulates them as big data, and uses them as data for providing and responding to various information for safe driving. . In addition, the cloud server 700 is linked with not only the dashboard robot 900 but also the smart phone 800 to analyze and process the image information and environmental information transmitted from the smart phone 800. In addition, it provides an interface to enable a smartphone AI service or a chat service using a variety of apps.

3 is a configuration diagram of a user state recognition system using an artificial intelligence robot robot based on a cloud server according to the present invention. Referring to FIG. 3, the dashboard robot 900 includes a main controller 901, an image information collector 920, an image environment information transmitter 930, an analysis information collector 940, and a motion controller 950. can do.

The main controller 910 is a device for controlling the dashboard robot 900. The main control unit 910 collects image information transmitted from the front camera 300 and the rear camera 500, and collects environmental information measured by the sensor unit 510. In addition, the motion controller 950 is controlled according to the driver's drowsy information transmitted from the cloud server 700.

The image information collecting unit 920 is a device for storing the front image photographed by the front camera 300 and the driving image photographed by the rear camera 500. In addition, the image information collecting unit 920 may be linked with the application of the portable terminal, preferably a smartphone. As a result, the acquired image data or the stored face image data may be transmitted through wireless communication, and may be connected and confirmed from the outside.

The image environment information transmitter 930 transmits the internal environment information measured by the front image, the driving image, and the sensor unit 510 to the cloud server 700 through a wireless communication network. The image environment information transmitting unit 930 transmits the front image, the driving image, and the internal environment information to the cloud server 700 when a predetermined cycle or / and a change of the image occurs.

The analysis information collecting unit 940 receives the driver's emotion and state information and the driver's drowsiness information from the cloud server 700. In addition, the analysis information collection unit 940 analyzes the drowsiness of the driver in the initial, middle and drowsy driving stages according to the frequency of the driver's drowsy operation, the level of the kind and the state information. The driver's emotion and state information analyzed by the cloud server 700 and the driver's drowsiness information will be described later.

The motion controller 950 is a device equipped with a notification device for recognizing a notification visually, tactilely, olfactoryly, or audibly to the driver. The motion controller 950 may include an oxygen spray unit mounted on one side of the upper housing 100 to discharge the oxygen to the outside in order to recognize the driver tactilely. In addition, it may include a fragrance spraying unit for spraying the fragrance to the outside of the lower housing 400 is mounted inside the lower housing 400 in order to recognize the driver olfactory notification. It may include an alarm device for operating the driver so that the driver can detect the sound reproduced by hearing by playing the built-in voice data to recognize the notification to the driver.

The main controller 910 selectively or sequentially operates the motion controller 950 according to the driver's drowsiness stage. For example, if the driver's drowsiness stage is the initial stage, the driver may activate only olfactory notification, but if the driver's drowsiness stage is the drowsy stage, it may generate an audible, tactile, and olfactory notification. .

On the other hand, the motion controller 920 may include a microphone device for obtaining a voice signal of the driver and a speaker device for outputting a voice to communicate with the driver.

The cloud server 700 may include an information collecting unit 710, an artificial intelligence processing unit 720, an analysis information transmitting unit 730, a user information database 740, and an information learning processing unit 750.

The information collecting unit 710 is a device that receives the front image, the driving image and the internal environment information from the dashboard robot 900. In addition, the information collecting unit 710 generates the first face image of the driver in the front image as template data. Template data refers to data obtained by extracting and digitizing all the feature elements constituting the driver's face through data captured in the image. That is, it corresponds to data for analyzing eye blinking, eyelid size, face angle, and face shaking for recognition of drowsy driving, and AI previously learned to determine driver's condition through deep learning. It is used as data to determine whether the driver is drowsy driving.

The artificial intelligence processing unit 720 compares and analyzes the template data and the face image in the front image to determine the driver's emotion and state, and analyzes the driver's drowsiness through the steering image and the driving image of the vehicle in the front image. The artificial intelligence processor 720 determines the driver's emotion and state based on the deep learning based on the driver's facial part image, and determines whether the driver is drowsy more accurately, including lane change and steering wheel movement. The dashboard robot 900 may determine not only drowsy driving through various factors such as the vehicle's driving speed and driving direction movement, as well as the driver's face recognition, and may prevent the driver's drowsy driving through various methods. There is an advantage.

The analysis information transmitter 730 transmits the analyzed driver's drowsy information to the dashboard robot 900. The dashboard robot 900 recognizes the driver's state through the received drowsy information of the driver.

The user information database 740 is a device that stores the accumulated information of the driver based on the front image, the driving image, and the internal environment information transmitted from the dashboard robot.

The information learning processor 750 is an apparatus for increasing the accuracy of the driver's drowsiness determination based on the accumulated information stored in the user information database 740. As the state information of the driver using artificial intelligence accumulates, judgment accuracy becomes higher.

The information learning processing unit 750 accumulates the received accumulated information and uses it as big data, and uses it as data for providing and providing various information for safe driving.

Figure 4 shows the internal hierarchy of the dashboard robot according to the present invention. Referring to FIG. 4, the dashboard robot 900 is configured as a motion module that independently controls behavior or controls behavior based on information. In addition, the sensor module includes a communication module for collecting and transmitting environment and image information, an image sensor for capturing images, an environment sensor for measuring the environment, and a motion interface for detecting a user's movement.

100: upper housing 200: shooting guide
300: front camera 400: lower housing
500: rear camera 700: cloud server
800: smartphone 900: dashboard robot

Claims (3)

A front camera for photographing a driver's face and a steering wheel of the vehicle; A rear camera for photographing driving conditions of the vehicle; Sensor unit for measuring the internal environment of the vehicle; Image information collecting unit for storing the front image photographed by the front camera and the driving image photographed by the rear camera, and the internal environment information measured by the front image, the driving image and the sensor through the wireless communication network Image environment information transmission unit for transmitting to the outside every set period; An analysis information collecting unit for analyzing the driver's drowsiness in the initial, middle and drowsy driving stages according to the frequency of the driver's drowsy driving, the emotion and the state information; A motion control unit including a notification device for recognizing a notification visually, tactilely, olfactoryly, and aurally to the driver; And a main controller for selectively or sequentially operating the motion control unit according to the drowsiness step of the driver.
An information collecting unit which receives the front image, the driving image and the internal environment information from the dashboard robot and generates the first face image of the driver in the front image as template data; And comparing and analyzing the template data and the face image in the front image to determine the emotion and state of the driver, and determine the driving speed and the driving direction of the vehicle through the steering image and the driving image of the vehicle in the front image. An artificial intelligence processor for identifying elements to analyze sleepiness of the driver; And a cloud server including an analysis information transmitting unit for transmitting the analyzed driver's drowsy information to the dashboard robot.
delete The method of claim 1, wherein the cloud server
A user information database for storing cumulative information of a driver based on the front image, the driving image, and the internal environment information transmitted from the dashboard robot; And
A user state recognition system using an intelligent dashboard robot based on a cloud server, characterized in that it comprises an information learning processing unit for increasing the accuracy of the driver's drowsiness determination based on the accumulated information stored in the user information database.

Images