WO2022048139A1

WO2022048139A1 - Vehicle-mounted human-computer interface having face and emotion recognition functions

Info

Publication number: WO2022048139A1
Application number: PCT/CN2021/083409
Authority: WO
Inventors: 沈磊
Original assignee: 上海本安仪表系统有限公司
Priority date: 2020-09-02
Filing date: 2021-03-26
Publication date: 2022-03-10
Also published as: CN112036329A

Abstract

Disclosed is a vehicle-mounted human-computer interface having face and emotion recognition functions. The vehicle-mounted human-computer interface comprises a vehicle-mounted human-computer interface screen having a camera; the vehicle-mounted human-computer interface screen further comprises a CAN network interface, a liquid crystal display, a loudspeaker, a processor, a storage device, and a wireless communication module. The processor of the vehicle-mounted human-computer interface screen receives video data by mean of the camera and decomposes the video data into picture data. According to the present invention, no extra device is added for the whole vehicle, and a vehicle system function safety framework cannot be affected while a face recognition and driver emotion and state monitoring system is provided; moreover, after a remote driver state monitoring platform related to the present invention confirms alarms of a driver, the information can be pushed to mobile phone application software by means of a variety of methods so as to remind related personnel to pay attention to the alarm state.

Description

Vehicle-mounted human-machine interface with face and emotion recognition

technical field

The invention relates to the application field of automotive electronic technology, the technical field of information storage, the field of image recognition, and the field of artificial intelligence, in particular, to a vehicle-mounted human-machine interface with functions of face and emotion recognition.

Background technique

With the vigorous development of the automobile and transportation industry, people have higher and higher requirements for the safe driving of cars and their management. It is necessary to monitor the driver's identity and mental state while driving the car, and to remind the driver that the driver is in the process of driving the car. concentrate. At present, my country's transportation industry has begun to popularize the installation of driver identification and fatigue driving early warning equipment, but there is no report on the installation of equipment with driver's mental emotion recognition and early warning function. There are particularly many reports; on the other hand, as a separate installation device, the related camera is often installed above the instrument panel, which is not aesthetically pleasing, and may become a projectile after the collision when the vehicle moves and a collision occurs. , and not too safe.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a vehicle-mounted human-machine interface with a face and emotion recognition function, which can solve the above problems existing in the prior art.

To achieve the above object, the present invention can be achieved through the following technical solutions:

The invention discloses a vehicle-mounted human-machine interface with a face and emotion recognition function, including a vehicle-mounted human-machine interface screen with a camera, and the vehicle-mounted human-machine interface screen also includes a CAN network interface, a liquid crystal display, a speaker, a processor, and a storage device. and wireless communication module.

The present invention also proposes a method implemented by the above vehicle-mounted human-machine interface, wherein the processor of the vehicle-mounted human-machine interface screen receives video data through a camera, and decomposes the video data into picture data.

Further, the processor of the vehicle-mounted human-machine interface screen performs preliminary face analysis on the picture data, if there is no driver's face information in the picture data, then generates no driver information, if the vehicle is driving at this time. , then the vehicle-mounted human-machine interface screen sends an emergency stop command through the CAN network interface, generates a driverless sound and light alarm signal through the liquid crystal display and the horn, and sends the wireless communication module through the wireless communication module. The alarm signal is sent to the remote driver status monitoring platform.

Further, the processor of the vehicle-mounted human-machine interface screen performs a preliminary face analysis on the picture data, and if the picture data contains the driver's face information, then obtains the picture data containing the driver's face.

Further, the processor of the vehicle-mounted human-machine interface screen performs driver emotion detection on the picture data containing the driver's face. Other ADAS assisted driving alarm signals, then the vehicle man-machine interface screen sends an emergency stop command through the CAN network interface, generates an acousto-optic alarm signal of the driver's extreme negative emotions through the liquid crystal display and the horn, and sends the alarm signal through the wireless The communication module sends the alarm signal to the remote driver status monitoring platform in the form of wireless communication.

Further, the processor of the vehicle-mounted human-machine interface screen performs driver fatigue detection and driving behavior identification detection on the picture data containing the driver's face. The man-machine interface screen generates a corresponding sound and light alarm signal through the liquid crystal display and the speaker, and sends it to the remote driver status monitoring platform through the wireless communication module in the form of wireless communication.

Further, after the processor of the vehicle-mounted human-machine interface screen obtains the picture data containing the driver's face, obtains the driver's face information database for comparison through the storage device, performs face recognition, and compares one by one: when When the picture data containing the driver's face can be consistent with a certain driver's face information in the storage device, the driver's information can be confirmed; otherwise, the picture data containing the driver's face is passed through the wireless communication module. It is sent to the remote driver's face information platform by wireless communication, so that the platform can perform face recognition and comparison. If the picture data containing the driver's face is consistent with the face information of a driver on the platform, the platform will use wireless communication The method sends the confirmed driver information and its face information to the vehicle-mounted human-machine interface screen together through the wireless communication module, and the vehicle-mounted human-machine interface screen processor receives the judgment result and saves the driver's face information to the vehicle-mounted human-machine interface screen. storage device; otherwise, the platform will send the conclusion and alarm signal that the driver cannot be identified through the wireless communication module to the vehicle-mounted human-machine interface screen, and the vehicle-mounted human-machine interface screen will be generated by the liquid crystal display and the speaker The driver cannot recognize the audible and visual warning signals.

Further, the other ADAS assisted driving warning signals are "vehicle deviation lane warning" and "vehicle forward collision warning".

further,

The beneficial effects of the present invention

1. The present invention not only has the functions of driver identification and fatigue driving warning, but also has the functions of driver's mental emotion recognition and warning and emergency stop, reminding the driver to drive safely and preventing major accidents from occurring.

2. For the whole vehicle, the present invention is beautiful, reliable and ergonomic through the integrated structure design of the face camera and the human-machine interface, and will not adversely affect the vehicle functional safety structure. Identity authentication through face recognition can achieve no Key start and drive, increase the function of vehicle driving data management.

3. The present invention can remind relevant personnel to pay attention to the alarm state through various methods after confirming various alarms of the driver through the relevant remote driver state monitoring platform.

Description of drawings

Fig. 1-Fig. 4 are schematic diagrams of the equipment of the present invention, wherein 1 is the vehicle-mounted human-machine interface screen, 2 is the built-in camera on the vehicle-mounted human-machine interface screen, and the built-in camera is arranged on the upper side close to the driver, and Fig. 1-Fig. 4 is four A specific application scenario, in which Figure 1 is a vertical screen of the left driver's seat, Figure 2 is a vertical screen of the right driver's seat, Figure 3 is a horizontal screen of the left driver's seat, and Figure 4 is a horizontal screen of the right driver's seat.

Fig. 5 is the data flow chart of the driver's emotion and state monitoring system of the present invention

Fig. 6 is the flow chart of the face recognition data of the present invention

detailed description

Below in conjunction with accompanying drawing, the present invention is described in further detail:

Referring to Figures 1-4, schematic diagrams of the equipment of the present invention. The device of the present invention includes a vehicle-mounted human-machine interface screen with a camera. The vehicle man-machine interface screen also contains CAN network interface, liquid crystal display, speaker, processor, storage device and wireless communication module.

Referring to Fig. 5, the data flow chart of the driver's mood and state monitoring system of the present invention. The processor of the vehicle-mounted human-machine interface screen receives the video data through the camera, and decomposes the video data into picture data. The processor of the vehicle-mounted human-machine interface screen performs preliminary face analysis on the picture data. If there is no driver's face information in the picture data, no driver information will be generated. If the vehicle is running at this time, the vehicle-mounted human-machine interface screen will Send an emergency stop command through the CAN network interface, and generate a sound and light alarm signal for no driver through the LCD display and horn, and send it to the remote driver status monitoring platform through wireless communication through the wireless communication module; if there is a driver in the picture data face information, then obtain the picture data containing the driver's face, and use the picture data containing the driver's face to perform driver emotion detection, driver fatigue detection, and driving behavior recognition detection. If the driver's emotion is detected to be extremely negative And at the same time, other ADAS assisted driving alarm signals (such as "vehicle deviation lane warning" and "vehicle forward collision warning") are received through the CAN network interface, then the vehicle HMI screen will send an emergency stop command through the CAN network interface , will generate the sound and light alarm signal of the driver's extreme negative emotions through the LCD display and the horn, and send the alarm signal to the remote driver status monitoring platform through the wireless communication module in the form of wireless communication; If the driving behavior is correct, the vehicle-mounted human-machine interface screen will generate the corresponding sound and light alarm signal through the liquid crystal display and the speaker, and send the alarm signal to the remote driver status monitoring platform through wireless communication through the wireless communication module.

Referring to FIG. 6, the flow chart of the face recognition data of the present invention is shown. After the processor of the vehicle-mounted human-machine interface screen obtains the picture data containing the driver's face, it obtains the comparison driver's face information database through the storage device, and performs face recognition and one-to-one comparison. When the picture data containing the driver's face can be consistent with the driver's face information in the storage device, the driver's information can be confirmed; otherwise, the picture data containing the driver's face will be sent by wireless communication through the wireless communication module For the remote driver's face information platform, let the platform perform face recognition and comparison. If the picture data containing the driver's face is consistent with the face information of a driver on the platform, the platform will use wireless communication through the wireless communication module. Send the confirmed driver information and its face information to the vehicle-mounted HMI screen, and the vehicle-mounted HMI screen processor receives the judgment result and saves the driver's face information to the storage device; otherwise, the platform uses wireless communication to pass The wireless communication module sends the conclusion that the driver cannot be identified and the alarm signal to the vehicle-mounted human-machine interface screen, and the vehicle-mounted human-machine interface screen will generate a sound and light alarm signal that the driver cannot identify the driver through the liquid crystal display and the horn.

However, the above-mentioned specific embodiments are only exemplary, in order to better enable those skilled in the art to understand the present patent, and should not be construed as limiting the scope of the present patent; Equivalent changes or modifications all fall within the scope of this patent.

Claims

The vehicle-mounted human-machine interface with face and emotion recognition function is characterized in that it includes a vehicle-mounted human-machine interface screen with a camera, and the vehicle-mounted human-machine interface screen also includes a CAN network interface, a liquid crystal display, a speaker, a processor, and a storage device. and wireless communication module.
The method for implementing a vehicle-mounted human-machine interface according to claim 1, wherein the processor of the vehicle-mounted human-machine interface screen receives video data through the camera, and decomposes the video data into picture data.
The method for realizing the vehicle-mounted human-machine interface according to claim 2, wherein the processor of the vehicle-mounted human-machine interface screen performs a preliminary face analysis on the picture data, if there is no driver in the picture data If the vehicle is running at this time, the vehicle man-machine interface screen sends an emergency stop command through the CAN network interface, and generates sound and light without a driver through the liquid crystal display and the horn. alarm signal, and send the alarm signal to the remote driver status monitoring platform in the form of wireless communication through the wireless communication module.
The method for realizing the vehicle-mounted human-machine interface according to claim 2, wherein the processor of the vehicle-mounted human-machine interface screen performs a preliminary face analysis on the picture data, and if there is a driver in the picture data face information, then obtain the picture data containing the driver's face.
The method for implementing a vehicle-mounted human-machine interface according to claim 4, wherein the processor of the vehicle-mounted human-machine interface screen performs the driver's emotion detection on the picture data containing the driver's face. When people are in extreme negative emotions and receive other ADAS assisted driving alarm signals through the CAN network interface at the same time, the vehicle man-machine interface screen sends an emergency stop command through the CAN network interface, which is generated by the liquid crystal display and the horn. The sound and light alarm signal of the driver's extremely negative emotions is sent to the remote driver status monitoring platform through the wireless communication module in the form of wireless communication.
The method for realizing the vehicle-mounted human-machine interface according to claim 4, wherein the processor of the vehicle-mounted human-machine interface screen performs driver fatigue detection and driving behavior identification detection on the picture data containing the driver's face, if It is detected that the driver is fatigued or has improper driving behavior, then the vehicle-mounted human-machine interface screen generates a corresponding sound and light alarm signal through the liquid crystal display and the speaker, and sends it to the remote through the wireless communication module by wireless communication. Driver status monitoring platform.
The method for realizing the vehicle-mounted human-machine interface according to claim 4, wherein after the processor of the vehicle-mounted human-machine interface screen obtains the picture data containing the driver's face, it obtains the comparison driver through the storage device. Face information database, face recognition, one-to-one comparison: when the picture data containing the driver's face can be consistent with a certain driver's face information in the storage device, the driver information can be confirmed; otherwise, all the driver information can be confirmed. The picture data containing the driver's face is sent to the remote driver's face information platform by means of wireless communication through the wireless communication module, so that the platform can perform face recognition and comparison, if the picture data containing the driver's face and the platform When the face information of a certain driver is consistent, the platform sends the confirmed driver information and its face information together to the vehicle-mounted human-machine interface screen by means of wireless communication through the wireless communication module, and the vehicle-mounted human-machine interface screen The processor receives the judgment result, and saves the driver's face information to the storage device; otherwise, the platform sends the conclusion that the driver cannot be identified and the alarm signal to the vehicle-mounted human-machine interface screen through the wireless communication module through wireless communication, The vehicle-mounted human-machine interface screen will generate an audible and visual alarm signal that the driver cannot recognize through the liquid crystal display and the horn.
The method according to claim 5, wherein the other ADAS assisted driving warning signals are "vehicle deviation lane warning" and "vehicle forward collision warning".