WO2022048139A1 - Vehicle-mounted human-computer interface having face and emotion recognition functions - Google Patents

Vehicle-mounted human-computer interface having face and emotion recognition functions Download PDF

Info

Publication number
WO2022048139A1
WO2022048139A1 PCT/CN2021/083409 CN2021083409W WO2022048139A1 WO 2022048139 A1 WO2022048139 A1 WO 2022048139A1 CN 2021083409 W CN2021083409 W CN 2021083409W WO 2022048139 A1 WO2022048139 A1 WO 2022048139A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
driver
machine interface
mounted human
face
Prior art date
Application number
PCT/CN2021/083409
Other languages
French (fr)
Chinese (zh)
Inventor
沈磊
Original Assignee
上海本安仪表系统有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海本安仪表系统有限公司 filed Critical 上海本安仪表系统有限公司
Publication of WO2022048139A1 publication Critical patent/WO2022048139A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174Facial expression recognition
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08Interaction between the driver and the control system
    • B60W50/14Means for informing the driver, warning the driver or prompting a driver intervention
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/59Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/597Recognising the driver's state or behaviour, e.g. attention or drowsiness
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08Interaction between the driver and the control system
    • B60W50/14Means for informing the driver, warning the driver or prompting a driver intervention
    • B60W2050/143Alarm means
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08Interaction between the driver and the control system
    • B60W50/14Means for informing the driver, warning the driver or prompting a driver intervention
    • B60W2050/146Display means

Definitions

  • the invention relates to the application field of automotive electronic technology, the technical field of information storage, the field of image recognition, and the field of artificial intelligence, in particular, to a vehicle-mounted human-machine interface with functions of face and emotion recognition.
  • the purpose of the present invention is to provide a vehicle-mounted human-machine interface with a face and emotion recognition function, which can solve the above problems existing in the prior art.
  • the invention discloses a vehicle-mounted human-machine interface with a face and emotion recognition function, including a vehicle-mounted human-machine interface screen with a camera, and the vehicle-mounted human-machine interface screen also includes a CAN network interface, a liquid crystal display, a speaker, a processor, and a storage device. and wireless communication module.
  • the present invention also proposes a method implemented by the above vehicle-mounted human-machine interface, wherein the processor of the vehicle-mounted human-machine interface screen receives video data through a camera, and decomposes the video data into picture data.
  • the processor of the vehicle-mounted human-machine interface screen performs preliminary face analysis on the picture data, if there is no driver's face information in the picture data, then generates no driver information, if the vehicle is driving at this time. , then the vehicle-mounted human-machine interface screen sends an emergency stop command through the CAN network interface, generates a driverless sound and light alarm signal through the liquid crystal display and the horn, and sends the wireless communication module through the wireless communication module. The alarm signal is sent to the remote driver status monitoring platform.
  • the processor of the vehicle-mounted human-machine interface screen performs a preliminary face analysis on the picture data, and if the picture data contains the driver's face information, then obtains the picture data containing the driver's face.
  • the processor of the vehicle-mounted human-machine interface screen performs driver emotion detection on the picture data containing the driver's face. Other ADAS assisted driving alarm signals, then the vehicle man-machine interface screen sends an emergency stop command through the CAN network interface, generates an acousto-optic alarm signal of the driver's extreme negative emotions through the liquid crystal display and the horn, and sends the alarm signal through the wireless
  • the communication module sends the alarm signal to the remote driver status monitoring platform in the form of wireless communication.
  • the processor of the vehicle-mounted human-machine interface screen performs driver fatigue detection and driving behavior identification detection on the picture data containing the driver's face.
  • the man-machine interface screen generates a corresponding sound and light alarm signal through the liquid crystal display and the speaker, and sends it to the remote driver status monitoring platform through the wireless communication module in the form of wireless communication.
  • the processor of the vehicle-mounted human-machine interface screen obtains the picture data containing the driver's face, obtains the driver's face information database for comparison through the storage device, performs face recognition, and compares one by one: when When the picture data containing the driver's face can be consistent with a certain driver's face information in the storage device, the driver's information can be confirmed; otherwise, the picture data containing the driver's face is passed through the wireless communication module. It is sent to the remote driver's face information platform by wireless communication, so that the platform can perform face recognition and comparison.
  • the platform will use wireless communication
  • the method sends the confirmed driver information and its face information to the vehicle-mounted human-machine interface screen together through the wireless communication module, and the vehicle-mounted human-machine interface screen processor receives the judgment result and saves the driver's face information to the vehicle-mounted human-machine interface screen.
  • storage device otherwise, the platform will send the conclusion and alarm signal that the driver cannot be identified through the wireless communication module to the vehicle-mounted human-machine interface screen, and the vehicle-mounted human-machine interface screen will be generated by the liquid crystal display and the speaker The driver cannot recognize the audible and visual warning signals.
  • ADAS assisted driving warning signals are "vehicle deviation lane warning” and “vehicle forward collision warning”.
  • the present invention not only has the functions of driver identification and fatigue driving warning, but also has the functions of driver's mental emotion recognition and warning and emergency stop, reminding the driver to drive safely and preventing major accidents from occurring.
  • the present invention is beautiful, reliable and ergonomic through the integrated structure design of the face camera and the human-machine interface, and will not adversely affect the vehicle functional safety structure.
  • Identity authentication through face recognition can achieve no Key start and drive, increase the function of vehicle driving data management.
  • the present invention can remind relevant personnel to pay attention to the alarm state through various methods after confirming various alarms of the driver through the relevant remote driver state monitoring platform.
  • Fig. 1-Fig. 4 are schematic diagrams of the equipment of the present invention, wherein 1 is the vehicle-mounted human-machine interface screen, 2 is the built-in camera on the vehicle-mounted human-machine interface screen, and the built-in camera is arranged on the upper side close to the driver, and Fig. 1-Fig. 4 is four A specific application scenario, in which Figure 1 is a vertical screen of the left driver's seat, Figure 2 is a vertical screen of the right driver's seat, Figure 3 is a horizontal screen of the left driver's seat, and Figure 4 is a horizontal screen of the right driver's seat.
  • Fig. 5 is the data flow chart of the driver's emotion and state monitoring system of the present invention
  • Fig. 6 is the flow chart of the face recognition data of the present invention
  • the device of the present invention includes a vehicle-mounted human-machine interface screen with a camera.
  • the vehicle man-machine interface screen also contains CAN network interface, liquid crystal display, speaker, processor, storage device and wireless communication module.
  • the processor of the vehicle-mounted human-machine interface screen receives the video data through the camera, and decomposes the video data into picture data.
  • the processor of the vehicle-mounted human-machine interface screen performs preliminary face analysis on the picture data. If there is no driver's face information in the picture data, no driver information will be generated.
  • the vehicle-mounted human-machine interface screen will Send an emergency stop command through the CAN network interface, and generate a sound and light alarm signal for no driver through the LCD display and horn, and send it to the remote driver status monitoring platform through wireless communication through the wireless communication module; if there is a driver in the picture data face information, then obtain the picture data containing the driver's face, and use the picture data containing the driver's face to perform driver emotion detection, driver fatigue detection, and driving behavior recognition detection.
  • the vehicle HMI screen will send an emergency stop command through the CAN network interface , will generate the sound and light alarm signal of the driver's extreme negative emotions through the LCD display and the horn, and send the alarm signal to the remote driver status monitoring platform through the wireless communication module in the form of wireless communication; If the driving behavior is correct, the vehicle-mounted human-machine interface screen will generate the corresponding sound and light alarm signal through the liquid crystal display and the speaker, and send the alarm signal to the remote driver status monitoring platform through wireless communication through the wireless communication module.
  • ADAS assisted driving alarm signals such as "vehicle deviation lane warning” and “vehicle forward collision warning”
  • the processor of the vehicle-mounted human-machine interface screen obtains the picture data containing the driver's face, it obtains the comparison driver's face information database through the storage device, and performs face recognition and one-to-one comparison.
  • the processor of the vehicle-mounted human-machine interface screen obtains the picture data containing the driver's face, it obtains the comparison driver's face information database through the storage device, and performs face recognition and one-to-one comparison.
  • the picture data containing the driver's face can be consistent with the driver's face information in the storage device, the driver's information can be confirmed; otherwise, the picture data containing the driver's face will be sent by wireless communication through the wireless communication module For the remote driver's face information platform, let the platform perform face recognition and comparison. If the picture data containing the driver's face is consistent with the face information of a driver on the platform, the platform will use wireless communication through the wireless communication module.
  • the vehicle-mounted HMI screen processor sends the judgment result and saves the driver's face information to the storage device; otherwise, the platform uses wireless communication to pass
  • the wireless communication module sends the conclusion that the driver cannot be identified and the alarm signal to the vehicle-mounted human-machine interface screen, and the vehicle-mounted human-machine interface screen will generate a sound and light alarm signal that the driver cannot identify the driver through the liquid crystal display and the horn.

Landscapes

  • Engineering & Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Traffic Control Systems (AREA)
  • Image Analysis (AREA)

Abstract

Disclosed is a vehicle-mounted human-computer interface having face and emotion recognition functions. The vehicle-mounted human-computer interface comprises a vehicle-mounted human-computer interface screen having a camera; the vehicle-mounted human-computer interface screen further comprises a CAN network interface, a liquid crystal display, a loudspeaker, a processor, a storage device, and a wireless communication module. The processor of the vehicle-mounted human-computer interface screen receives video data by mean of the camera and decomposes the video data into picture data. According to the present invention, no extra device is added for the whole vehicle, and a vehicle system function safety framework cannot be affected while a face recognition and driver emotion and state monitoring system is provided; moreover, after a remote driver state monitoring platform related to the present invention confirms alarms of a driver, the information can be pushed to mobile phone application software by means of a variety of methods so as to remind related personnel to pay attention to the alarm state.

Description

具有人脸及情绪识别功能的车载人机界面Vehicle-mounted human-machine interface with face and emotion recognition 技术领域technical field
本发明涉及汽车电子技术应用领域、信息存储技术领域、图像识别领域、人工智能领域,具体地说,是具有人脸及情绪识别功能的车载人机界面。The invention relates to the application field of automotive electronic technology, the technical field of information storage, the field of image recognition, and the field of artificial intelligence, in particular, to a vehicle-mounted human-machine interface with functions of face and emotion recognition.
背景技术Background technique
随着汽车和交通行业的蓬勃发展,人们对于汽车安全行驶及其管理的要求越来越高,需要对驾驶人在驾驶汽车过程中的身份和精神状态进行监控和提醒驾驶人在驾驶汽车过程中集中注意力。当前我国交通运输行业开始普及安装驾驶员身份识别和疲劳驾驶预警设备,但是市场上没有看到安装具有驾驶员精神情绪识别预警功能的设备报道,而由于公交车驾驶员精神情绪失常造成的事故的报道则特别多;另外一方面,作为一个单独的加装设备,相关的摄像头往往安装在仪表台上方,这样的设备既不美观,而且在车辆运动和产生碰撞事故时可能成为碰撞后的弹射物,也不太安全。With the vigorous development of the automobile and transportation industry, people have higher and higher requirements for the safe driving of cars and their management. It is necessary to monitor the driver's identity and mental state while driving the car, and to remind the driver that the driver is in the process of driving the car. concentrate. At present, my country's transportation industry has begun to popularize the installation of driver identification and fatigue driving early warning equipment, but there is no report on the installation of equipment with driver's mental emotion recognition and early warning function. There are particularly many reports; on the other hand, as a separate installation device, the related camera is often installed above the instrument panel, which is not aesthetically pleasing, and may become a projectile after the collision when the vehicle moves and a collision occurs. , and not too safe.
发明内容SUMMARY OF THE INVENTION
本发明的目的就是提供具有人脸及情绪识别功能的车载人机界面,其可解决现有技术存在的上述问题。The purpose of the present invention is to provide a vehicle-mounted human-machine interface with a face and emotion recognition function, which can solve the above problems existing in the prior art.
为实现上述目的,本发明可通过以下技术方案予以实现:To achieve the above object, the present invention can be achieved through the following technical solutions:
本发明公开具有人脸及情绪识别功能的车载人机界面,包括带有摄像头的车载人机界面屏,所述车载人机界面屏还含有CAN网络接口、液晶显示器、喇叭、处理器、存储设备和无线通讯模块。The invention discloses a vehicle-mounted human-machine interface with a face and emotion recognition function, including a vehicle-mounted human-machine interface screen with a camera, and the vehicle-mounted human-machine interface screen also includes a CAN network interface, a liquid crystal display, a speaker, a processor, and a storage device. and wireless communication module.
本发明还提出通过以上车载人机界面实现的方法,所述车载人机界面屏的处理器通过摄像头接收视频数据,将视频数据分解为图片数据。The present invention also proposes a method implemented by the above vehicle-mounted human-machine interface, wherein the processor of the vehicle-mounted human-machine interface screen receives video data through a camera, and decomposes the video data into picture data.
进一步地,所述车载人机界面屏的处理器对所述图片数据进行初步人脸分析,如果所述图片数据中没有驾驶人人脸信息,那么产生无驾驶员信息,如果这时车辆在行驶中,那么所述车载人机界面屏通过所述CAN网络接口发送紧急停车指令,通过所述液晶显示器和喇叭产生无驾驶人声光报警信号,并通过所述无线通讯模块以无线通讯的方式将所述报警信号发送给远程驾驶人状态监控平台。Further, the processor of the vehicle-mounted human-machine interface screen performs preliminary face analysis on the picture data, if there is no driver's face information in the picture data, then generates no driver information, if the vehicle is driving at this time. , then the vehicle-mounted human-machine interface screen sends an emergency stop command through the CAN network interface, generates a driverless sound and light alarm signal through the liquid crystal display and the horn, and sends the wireless communication module through the wireless communication module. The alarm signal is sent to the remote driver status monitoring platform.
进一步地,所述车载人机界面屏的处理器对所述图片数据进行初步人脸分析,如果所述图片数据中有驾驶人人脸信息,那么获得含有驾驶人人脸的图片数据。Further, the processor of the vehicle-mounted human-machine interface screen performs a preliminary face analysis on the picture data, and if the picture data contains the driver's face information, then obtains the picture data containing the driver's face.
进一步地,所述车载人机界面屏的处理器将所述含有驾驶人人脸的图片数据进行驾驶人情绪检测,如果检测出驾驶人情绪处于极端负面情绪并且同时通过所述CAN网络接口接收到其他ADAS辅助驾驶报警信号,那么所述车载人机界面屏通过所述CAN网络接口发送紧急停车指令,通过所述液晶显示器和喇叭产生驾驶人极端负面情绪的声光报警信号,并通过所述无线通讯模块以无线通讯的方式将所述报警信号发送给远程驾驶人状态监控平台。Further, the processor of the vehicle-mounted human-machine interface screen performs driver emotion detection on the picture data containing the driver's face. Other ADAS assisted driving alarm signals, then the vehicle man-machine interface screen sends an emergency stop command through the CAN network interface, generates an acousto-optic alarm signal of the driver's extreme negative emotions through the liquid crystal display and the horn, and sends the alarm signal through the wireless The communication module sends the alarm signal to the remote driver status monitoring platform in the form of wireless communication.
进一步地,所述车载人机界面屏的处理器将含有驾驶人人脸的图片数据进行驾驶人疲劳检测和驾驶行为识别检测,如果检测出驾驶人疲劳或者有不正当驾驶行为,那么所述车载人机界面屏通过所述液晶显示器和喇叭产生对应的声光报警信号,并通过所述无线通讯模块以无线通讯的方式发送给远程驾驶人状态监控平台。Further, the processor of the vehicle-mounted human-machine interface screen performs driver fatigue detection and driving behavior identification detection on the picture data containing the driver's face. The man-machine interface screen generates a corresponding sound and light alarm signal through the liquid crystal display and the speaker, and sends it to the remote driver status monitoring platform through the wireless communication module in the form of wireless communication.
进一步地,所述车载人机界面屏的处理器获得含有驾驶人人脸的图片数据后,通过所述存储设备获得对照的驾驶人人脸信息库,进行人脸识别,一一比对:当所述含有驾驶人人脸的图片数据能与存储设备的某个驾驶人人脸信息一致时,可确认驾驶人信息;否则将所述含有驾驶人人脸的图片数据通过所述无线通讯模块以无线通讯的方式发送给远程驾驶人人脸信息平台,让平台进行人脸识别比对,如果含有驾驶人人脸的图片数据与平台上某个驾驶人人脸信息一致时,平台以无线通讯的方式通过无线通讯模块将确认的驾驶人信息及其人脸信息一起发送到所述车载人机界面屏,所述车载人机界面屏处理器接收判断结果,并保存驾驶人人脸信息到所述存储设备;否则平台以无线通讯的方式通过无线通讯模块将无法识别驾驶人的结论和报警信号发送到所述车载人机界面屏,所述车载人机界面屏将通过所述液晶显示器和喇叭产生驾驶人无法识别声光报警信号。Further, after the processor of the vehicle-mounted human-machine interface screen obtains the picture data containing the driver's face, obtains the driver's face information database for comparison through the storage device, performs face recognition, and compares one by one: when When the picture data containing the driver's face can be consistent with a certain driver's face information in the storage device, the driver's information can be confirmed; otherwise, the picture data containing the driver's face is passed through the wireless communication module. It is sent to the remote driver's face information platform by wireless communication, so that the platform can perform face recognition and comparison. If the picture data containing the driver's face is consistent with the face information of a driver on the platform, the platform will use wireless communication The method sends the confirmed driver information and its face information to the vehicle-mounted human-machine interface screen together through the wireless communication module, and the vehicle-mounted human-machine interface screen processor receives the judgment result and saves the driver's face information to the vehicle-mounted human-machine interface screen. storage device; otherwise, the platform will send the conclusion and alarm signal that the driver cannot be identified through the wireless communication module to the vehicle-mounted human-machine interface screen, and the vehicle-mounted human-machine interface screen will be generated by the liquid crystal display and the speaker The driver cannot recognize the audible and visual warning signals.
进一步地,所述其他ADAS辅助驾驶报警信号为“车辆偏移车道报警”和“车辆前向碰撞预警”。Further, the other ADAS assisted driving warning signals are "vehicle deviation lane warning" and "vehicle forward collision warning".
进一步地,further,
本发明的有益效果The beneficial effects of the present invention
1.本发明不仅具有驾驶员身份识别和疲劳驾驶预警功能,还具有驾驶员精神情绪识别预警和紧急停车功能,提醒驾驶员安全驾驶的同时防止重大事故发生。1. The present invention not only has the functions of driver identification and fatigue driving warning, but also has the functions of driver's mental emotion recognition and warning and emergency stop, reminding the driver to drive safely and preventing major accidents from occurring.
2.本发明对于整车而言,通过人脸摄像头和人机界面一体化结构设计,美观可靠符合人体工学,不会对车辆功能安全构架产生不利影响,通过人脸识别进行身份认证可以实现无钥匙启动和驾驶,增加车辆驾驶数据管理的功能。2. For the whole vehicle, the present invention is beautiful, reliable and ergonomic through the integrated structure design of the face camera and the human-machine interface, and will not adversely affect the vehicle functional safety structure. Identity authentication through face recognition can achieve no Key start and drive, increase the function of vehicle driving data management.
3.本发明可以通过相关的远程驾驶人状态监控平台在确认驾驶人各种报警后,可通过各种方法提醒相关人员关注报警状态。3. The present invention can remind relevant personnel to pay attention to the alarm state through various methods after confirming various alarms of the driver through the relevant remote driver state monitoring platform.
附图说明Description of drawings
图1-图4为本发明的设备示意图,其中1为车载人机界面屏,2为车载人机界面屏上内置摄像头,内置摄像头布置在靠近驾驶员的上侧,图1-图4是四种具体的应用场景,其中图1为左驾驶位竖屏,图2为右驾驶位竖屏,图3为左驾驶位横屏,图4为右驾驶位横屏。Fig. 1-Fig. 4 are schematic diagrams of the equipment of the present invention, wherein 1 is the vehicle-mounted human-machine interface screen, 2 is the built-in camera on the vehicle-mounted human-machine interface screen, and the built-in camera is arranged on the upper side close to the driver, and Fig. 1-Fig. 4 is four A specific application scenario, in which Figure 1 is a vertical screen of the left driver's seat, Figure 2 is a vertical screen of the right driver's seat, Figure 3 is a horizontal screen of the left driver's seat, and Figure 4 is a horizontal screen of the right driver's seat.
图5为本发明的驾驶人情绪和状态监控系统数据流程图Fig. 5 is the data flow chart of the driver's emotion and state monitoring system of the present invention
图6为本发明的人脸识别数据流程图Fig. 6 is the flow chart of the face recognition data of the present invention
具体实施方式detailed description
下面结合附图对本发明作进一步详细描述:Below in conjunction with accompanying drawing, the present invention is described in further detail:
参见图1-图4,本发明的设备示意图。本发明的设备包括带有摄像头的车载人机界面屏。车载人机界面屏还含有CAN网络接口、液晶显示器、喇叭、处理器、存储设备和无线通讯模块。Referring to Figures 1-4, schematic diagrams of the equipment of the present invention. The device of the present invention includes a vehicle-mounted human-machine interface screen with a camera. The vehicle man-machine interface screen also contains CAN network interface, liquid crystal display, speaker, processor, storage device and wireless communication module.
参见图5,本发明的驾驶人情绪和状态监控系统数据流程图。车载人机界面屏的处理器通过摄像头接收视频数据,将视频数据分解为图片数据。车载人机界面屏的处理器对图片数据进行初步人脸分析,如果图片数据中没有驾驶人人脸信息,那么产生无驾驶员信息,如果这时车辆在行驶中,那么车载人机界面屏将通过CAN网络接口发送紧急停车指令,将通过液晶显示器和喇叭产生无驾驶人声光报警信号,并通过无线通讯模块以无线通讯的方式发送给远程驾驶人状态监控平台;如果图片数据中有驾驶人人脸信息,那么获得含有驾驶人人脸的图片数据,将含有驾驶人人脸的图片数据进行驾驶人情绪检测、驾驶人疲劳检测和驾驶行为识别检测,如果检测出驾驶人情绪处于极端负面情绪而且同时通过所述CAN网络接口接收到其他ADAS辅助驾驶报警信号(如“车辆偏移车道报警”和“车辆前向碰撞预警”),那么车载人机界面屏将通过CAN网络接口发送紧急停车指令,将通过液晶显示器和喇叭产生驾驶人极端负面情绪的声光报警信号,并通过无线通讯模块以无线通讯的方式将报警信号发送给远程驾驶人状态监控平台;如果检测出驾驶人疲劳或者有不正当驾驶行为,那么车载人机界面屏将通过液晶显示器和喇叭产生对应的声光报警信号,并通过无线通讯模块以无线通讯的方式将报警信号发送给远程驾驶人状态监控平台。Referring to Fig. 5, the data flow chart of the driver's mood and state monitoring system of the present invention. The processor of the vehicle-mounted human-machine interface screen receives the video data through the camera, and decomposes the video data into picture data. The processor of the vehicle-mounted human-machine interface screen performs preliminary face analysis on the picture data. If there is no driver's face information in the picture data, no driver information will be generated. If the vehicle is running at this time, the vehicle-mounted human-machine interface screen will Send an emergency stop command through the CAN network interface, and generate a sound and light alarm signal for no driver through the LCD display and horn, and send it to the remote driver status monitoring platform through wireless communication through the wireless communication module; if there is a driver in the picture data face information, then obtain the picture data containing the driver's face, and use the picture data containing the driver's face to perform driver emotion detection, driver fatigue detection, and driving behavior recognition detection. If the driver's emotion is detected to be extremely negative And at the same time, other ADAS assisted driving alarm signals (such as "vehicle deviation lane warning" and "vehicle forward collision warning") are received through the CAN network interface, then the vehicle HMI screen will send an emergency stop command through the CAN network interface , will generate the sound and light alarm signal of the driver's extreme negative emotions through the LCD display and the horn, and send the alarm signal to the remote driver status monitoring platform through the wireless communication module in the form of wireless communication; If the driving behavior is correct, the vehicle-mounted human-machine interface screen will generate the corresponding sound and light alarm signal through the liquid crystal display and the speaker, and send the alarm signal to the remote driver status monitoring platform through wireless communication through the wireless communication module.
参见图6,本发明的人脸识别数据流程图。车载人机界面屏的处理器获得含有驾驶人人脸的图片数据后,通过存储设备获得对照的驾驶人人脸信息库,进行人脸识别,一一比对。当含有驾驶人人脸的图片数据能与存储设备的某个驾驶人人脸信息一致时,可确认驾驶人信息;否则将含有驾驶人人脸的图片数据通过无线通讯模块以无线通讯的方式发送给远程驾驶人人脸信息平台,让平台进行人脸识别比对,如果含有驾驶人人脸的图片数据与平台上某个 驾驶人人脸信息一致时,平台以无线通讯的方式通过无线通讯模块将确认的驾驶人信息及其人脸信息一起发送到车载人机界面屏,车载人机界面屏处理器接收判断结果,并保存驾驶人人脸信息到存储设备;否则平台以无线通讯的方式通过无线通讯模块将无法识别驾驶人的结论和报警信号发送到车载人机界面屏,车载人机界面屏将通过液晶显示器和喇叭产生驾驶人无法识别声光报警信号。Referring to FIG. 6, the flow chart of the face recognition data of the present invention is shown. After the processor of the vehicle-mounted human-machine interface screen obtains the picture data containing the driver's face, it obtains the comparison driver's face information database through the storage device, and performs face recognition and one-to-one comparison. When the picture data containing the driver's face can be consistent with the driver's face information in the storage device, the driver's information can be confirmed; otherwise, the picture data containing the driver's face will be sent by wireless communication through the wireless communication module For the remote driver's face information platform, let the platform perform face recognition and comparison. If the picture data containing the driver's face is consistent with the face information of a driver on the platform, the platform will use wireless communication through the wireless communication module. Send the confirmed driver information and its face information to the vehicle-mounted HMI screen, and the vehicle-mounted HMI screen processor receives the judgment result and saves the driver's face information to the storage device; otherwise, the platform uses wireless communication to pass The wireless communication module sends the conclusion that the driver cannot be identified and the alarm signal to the vehicle-mounted human-machine interface screen, and the vehicle-mounted human-machine interface screen will generate a sound and light alarm signal that the driver cannot identify the driver through the liquid crystal display and the horn.
但是,上述的具体实施方式只是示例性的,是为了更好的使本领域技术人员能够理解本专利,不能理解为是对本专利包括范围的限制;只要是根据本专利所揭示精神的所作的任何等同变更或修饰,均落入本专利包括的范围。However, the above-mentioned specific embodiments are only exemplary, in order to better enable those skilled in the art to understand the present patent, and should not be construed as limiting the scope of the present patent; Equivalent changes or modifications all fall within the scope of this patent.

Claims (8)

  1. 具有人脸及情绪识别功能的车载人机界面,其特征在于,包括带有摄像头的车载人机界面屏,该车载人机界面屏还含有CAN网络接口、液晶显示器、喇叭、处理器、存储设备和无线通讯模块。The vehicle-mounted human-machine interface with face and emotion recognition function is characterized in that it includes a vehicle-mounted human-machine interface screen with a camera, and the vehicle-mounted human-machine interface screen also includes a CAN network interface, a liquid crystal display, a speaker, a processor, and a storage device. and wireless communication module.
  2. 根据权利要求1所述的车载人机界面实现的方法,其特征在于,所述车载人机界面屏的处理器通过所述摄像头接收视频数据,将所述视频数据分解为图片数据。The method for implementing a vehicle-mounted human-machine interface according to claim 1, wherein the processor of the vehicle-mounted human-machine interface screen receives video data through the camera, and decomposes the video data into picture data.
  3. 根据权利要求2所述的车载人机界面实现的方法,其特征在于,所述车载人机界面屏的处理器对所述图片数据进行初步人脸分析,如果所述图片数据中没有驾驶人人脸信息,那么产生无驾驶员信息,如果这时车辆在行驶中,那么所述车载人机界面屏通过所述CAN网络接口发送紧急停车指令,通过所述液晶显示器和喇叭产生无驾驶人声光报警信号,并通过所述无线通讯模块以无线通讯的方式将所述报警信号发送给远程驾驶人状态监控平台。The method for realizing the vehicle-mounted human-machine interface according to claim 2, wherein the processor of the vehicle-mounted human-machine interface screen performs a preliminary face analysis on the picture data, if there is no driver in the picture data If the vehicle is running at this time, the vehicle man-machine interface screen sends an emergency stop command through the CAN network interface, and generates sound and light without a driver through the liquid crystal display and the horn. alarm signal, and send the alarm signal to the remote driver status monitoring platform in the form of wireless communication through the wireless communication module.
  4. 根据权利要求2所述的车载人机界面实现的方法,其特征在于,所述车载人机界面屏的处理器对所述图片数据进行初步人脸分析,如果所述图片数据中有驾驶人人脸信息,那么获得含有驾驶人人脸的图片数据。The method for realizing the vehicle-mounted human-machine interface according to claim 2, wherein the processor of the vehicle-mounted human-machine interface screen performs a preliminary face analysis on the picture data, and if there is a driver in the picture data face information, then obtain the picture data containing the driver's face.
  5. 根据权利要求4所述的车载人机界面实现的方法,其特征在于,所述车载人机界面屏的处理器将所述含有驾驶人人脸的图片数据进行驾驶人情绪检测,如果检测出驾驶人情绪处于极端负面情绪并且同时通过所述CAN网络接口接收到其他ADAS辅助驾驶报警信号,那么所述车载人机界面屏通过所述CAN网络接口发送紧急停车指令,通过所述液晶显示器和喇叭产生驾驶人极端负面情绪的声光报警信号,并通过所述无线通讯模块以无线通讯的方式将所述报警信号发送给远程驾驶人状态监控平台。The method for implementing a vehicle-mounted human-machine interface according to claim 4, wherein the processor of the vehicle-mounted human-machine interface screen performs the driver's emotion detection on the picture data containing the driver's face. When people are in extreme negative emotions and receive other ADAS assisted driving alarm signals through the CAN network interface at the same time, the vehicle man-machine interface screen sends an emergency stop command through the CAN network interface, which is generated by the liquid crystal display and the horn. The sound and light alarm signal of the driver's extremely negative emotions is sent to the remote driver status monitoring platform through the wireless communication module in the form of wireless communication.
  6. 根据权利要求4所述的车载人机界面实现的方法,其特征在于,所述车载人机界面屏的处理器将含有驾驶人人脸的图片数据进行驾驶人疲劳检测和驾驶行为识别检测,如果检测出驾驶人疲劳或者有不正当驾驶行为,那么所述车载人机界面屏通过所述液晶显示器和喇叭产生对应的声光报警信号,并通过所述无线通讯模块以无线通讯的方式发送给远程驾驶人状态监控平台。The method for realizing the vehicle-mounted human-machine interface according to claim 4, wherein the processor of the vehicle-mounted human-machine interface screen performs driver fatigue detection and driving behavior identification detection on the picture data containing the driver's face, if It is detected that the driver is fatigued or has improper driving behavior, then the vehicle-mounted human-machine interface screen generates a corresponding sound and light alarm signal through the liquid crystal display and the speaker, and sends it to the remote through the wireless communication module by wireless communication. Driver status monitoring platform.
  7. 根据权利要求4所述的车载人机界面实现的方法,其特征在于,所述车载人机界面屏的处理器获得含有驾驶人人脸的图片数据后,通过所述存储设备获得对照的驾驶人人脸信息库,进行人脸识别,一一比对:当所述含有驾驶人人脸的图片数据能与存储设备的某个驾驶人人脸信息一致时,可确认驾驶人信息;否则将所述含有驾驶人人脸的图片数据通过所述无线通讯模块以无线通讯的方式发送给远程驾驶人人脸信息平台,让平台进行 人脸识别比对,如果含有驾驶人人脸的图片数据与平台上某个驾驶人人脸信息一致时,平台以无线通讯的方式通过无线通讯模块将确认的驾驶人信息及其人脸信息一起发送到所述车载人机界面屏,所述车载人机界面屏处理器接收判断结果,并保存驾驶人人脸信息到所述存储设备;否则平台以无线通讯的方式通过无线通讯模块将无法识别驾驶人的结论和报警信号发送到所述车载人机界面屏,所述车载人机界面屏将通过所述液晶显示器和喇叭产生驾驶人无法识别声光报警信号。The method for realizing the vehicle-mounted human-machine interface according to claim 4, wherein after the processor of the vehicle-mounted human-machine interface screen obtains the picture data containing the driver's face, it obtains the comparison driver through the storage device. Face information database, face recognition, one-to-one comparison: when the picture data containing the driver's face can be consistent with a certain driver's face information in the storage device, the driver information can be confirmed; otherwise, all the driver information can be confirmed. The picture data containing the driver's face is sent to the remote driver's face information platform by means of wireless communication through the wireless communication module, so that the platform can perform face recognition and comparison, if the picture data containing the driver's face and the platform When the face information of a certain driver is consistent, the platform sends the confirmed driver information and its face information together to the vehicle-mounted human-machine interface screen by means of wireless communication through the wireless communication module, and the vehicle-mounted human-machine interface screen The processor receives the judgment result, and saves the driver's face information to the storage device; otherwise, the platform sends the conclusion that the driver cannot be identified and the alarm signal to the vehicle-mounted human-machine interface screen through the wireless communication module through wireless communication, The vehicle-mounted human-machine interface screen will generate an audible and visual alarm signal that the driver cannot recognize through the liquid crystal display and the horn.
  8. 根据权利要求5所述的方法,其特征在于,所述其他ADAS辅助驾驶报警信号为“车辆偏移车道报警”和“车辆前向碰撞预警”。The method according to claim 5, wherein the other ADAS assisted driving warning signals are "vehicle deviation lane warning" and "vehicle forward collision warning".
PCT/CN2021/083409 2020-09-02 2021-03-26 Vehicle-mounted human-computer interface having face and emotion recognition functions WO2022048139A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010911625.9A CN112036329A (en) 2020-09-02 2020-09-02 Vehicle-mounted human-computer interface with face and emotion recognition function
CN202010911625.9 2020-09-02

Publications (1)

Publication Number Publication Date
WO2022048139A1 true WO2022048139A1 (en) 2022-03-10

Family

ID=73591276

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/083409 WO2022048139A1 (en) 2020-09-02 2021-03-26 Vehicle-mounted human-computer interface having face and emotion recognition functions

Country Status (2)

Country Link
CN (1) CN112036329A (en)
WO (1) WO2022048139A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112036329A (en) * 2020-09-02 2020-12-04 上海本安仪表系统有限公司 Vehicle-mounted human-computer interface with face and emotion recognition function

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2528084A (en) * 2014-07-08 2016-01-13 Jaguar Land Rover Ltd Notification system and method
CN106627589A (en) * 2016-12-27 2017-05-10 科世达(上海)管理有限公司 Vehicle driving safety auxiliary method and system and vehicle
CN107776579A (en) * 2017-09-14 2018-03-09 中国第汽车股份有限公司 A kind of direct feeling driver status alarm set
CN112036329A (en) * 2020-09-02 2020-12-04 上海本安仪表系统有限公司 Vehicle-mounted human-computer interface with face and emotion recognition function

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102800024A (en) * 2012-07-11 2012-11-28 深圳市飞瑞斯科技有限公司 Whole-journey identity verification method and system for driving examinee based on face identification
CN103313033A (en) * 2013-05-14 2013-09-18 浙江大学 Remote and local video monitoring vehicle-mounted device for construction machine
CN104966359B (en) * 2015-07-20 2018-01-30 京东方科技集团股份有限公司 anti-theft alarm system and method
CN208867942U (en) * 2018-09-06 2019-05-17 烟台市安特洛普网络科技有限公司 Intelligent vehicle-carried interactive system
CN109606386A (en) * 2018-12-12 2019-04-12 北京车联天下信息技术有限公司 Cockpit in intelligent vehicle
CN110154757A (en) * 2019-05-30 2019-08-23 电子科技大学 The multi-faceted safe driving support method of bus

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2528084A (en) * 2014-07-08 2016-01-13 Jaguar Land Rover Ltd Notification system and method
CN106627589A (en) * 2016-12-27 2017-05-10 科世达(上海)管理有限公司 Vehicle driving safety auxiliary method and system and vehicle
CN107776579A (en) * 2017-09-14 2018-03-09 中国第汽车股份有限公司 A kind of direct feeling driver status alarm set
CN112036329A (en) * 2020-09-02 2020-12-04 上海本安仪表系统有限公司 Vehicle-mounted human-computer interface with face and emotion recognition function

Also Published As

Publication number Publication date
CN112036329A (en) 2020-12-04

Similar Documents

Publication Publication Date Title
EP3675121B1 (en) Computer-implemented interaction with a user
US10054948B2 (en) Vehicle control apparatus
CN107901828B (en) Method and device for reminding overtaking through intelligent equipment cooperation
US9916762B2 (en) Parallel parking system
WO2021017057A1 (en) Passenger-carrying operating vehicle supervision system and device, and medium
JP2010128920A (en) Safety device for vehicle
CN109448409A (en) Method, apparatus, equipment and the computer storage medium of traffic information interaction
US9522632B2 (en) Method and device for providing information about failure in virtual engine sound system unit for vehicle
CN112071309B (en) Network appointment vehicle safety monitoring device and system
CN112885031B (en) Intelligent alarm system and method based on Internet of vehicles
WO2022048139A1 (en) Vehicle-mounted human-computer interface having face and emotion recognition functions
KR20180052077A (en) Interaction method between user and autonomous driving system in fully autonomous driving
Chen et al. A wearable-glasses-based drowsiness-fatigue-detection system for improving road safety
CN107016997A (en) A kind of intelligent people's car mutual system
JP2013156893A (en) Vehicle accident countermeasure device
CN107472131A (en) Automobile door opening source of early warning, automobile door opening early warning system, automobile door opening method for early warning
CN106004439A (en) Remote control intelligent on-board host computer for bus
WO2023065998A1 (en) Vehicle control method and apparatus, and vehicle and storage medium
CN210793101U (en) Intelligent vehicle-mounted terminal, intelligent bus and intelligent bus management system
McKeown et al. Auditory displays as occasion setters
CN106627425A (en) Safety driving control system based on internet of things
KR20160099947A (en) Method of preventing driver from falling asleep at the wheel
TWM597476U (en) Vehicle Assistant System
CN103413397A (en) Parking prevention system
CN117325788A (en) Vehicle-mounted information prompting and concealing alarm system, method and medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21863210

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21863210

Country of ref document: EP

Kind code of ref document: A1