WO2020042678A1

WO2020042678A1 - Oculomotorius information-based alarming method and apparatus, device and storage medium

Info

Publication number: WO2020042678A1
Application number: PCT/CN2019/087756
Authority: WO
Inventors: 赵海杰; 路伟成; 黄通兵
Original assignee: 北京七鑫易维信息技术有限公司
Priority date: 2018-08-28
Filing date: 2019-05-21
Publication date: 2020-03-05
Also published as: CN109118714A; CN109118714B

Abstract

An oculomotorius information-based alarming method and apparatus, a device and a storage medium. The method comprises: acquiring oculomotorius information of a user (S110); determining whether the oculomotorius information satisfies a set condition (S120); and if yes, generating alarming information according to the oculomotorius information, and sending the alarming information to a set platform (S130). According to the oculomotorius information-based alarming method, the oculomotorius information of a user is recognized, and if the oculomotorius information satisfies the set condition, the alarming information is generated, so that a user who has an accident gets timely help, and the timeliness of alarms is improved, thereby reducing the risk of life safety.

Description

Alarm method, device, equipment and storage medium based on eye movement information

Technical field

The embodiments of the present application relate to the field of vehicle-mounted technology, and in particular, to an alarm method, device, device, and storage medium based on eye movement information.

Background technique

With the rapid development of the automobile industry, automobiles have become an indispensable means of travel for people to go out. During the driving process, due to various reasons, users may have traffic accidents, vehicle failures, etc., especially when traffic accidents occur and people in the car are injured, it is particularly important to get timely assistance.

In the prior art, after a traffic accident or being hijacked by a gangster, the person concerned usually calls the police for assistance, but if the person is unable to act and there are no other people around, the person may be in danger of life because he has not received timely assistance. Obviously, the current on-board vehicles urgently need an alarm method, which will promptly give an alarm after a vehicle failure.

Summary of the Invention

The embodiments of the present application provide an alarm method, device, device, and storage medium based on eye movement information, which can improve the timeliness of the alarm and reduce the loss of life safety.

In a first aspect, an embodiment of the present application provides an alarm method based on eye movement information, including:

Get user's eye movement information;

Determining whether the eye movement information satisfies a set condition;

If it is satisfied, an alarm message is generated according to the eye movement information, and the alarm message is sent to a setting platform.

Optionally, the obtaining the user's eye movement information includes:

Collect at least two frames of eye images of the user;

Perform feature analysis on the at least two frames of eye images to obtain user eye movement information.

Optionally, the determining whether the eye movement information satisfies a set condition includes:

Judging whether the length of time the user looks at the set position exceeds a set threshold;

Correspondingly, if satisfied, generating alarm information according to the eye movement information includes:

If it exceeds, an alarm message is generated.

Determining whether the vital signs of the user are in a set dangerous range according to the eye movement information;

If so, an alarm message is generated.

Optionally, the generating alarm information according to the eye movement information includes:

Locate the user's current location to obtain location information;

Generate alarm information based on the position information and the eye movement information.

Optionally, after sending the alarm information to the setting platform, the method further includes:

Send the alarm information to a mobile device associated with the vehicle-mounted device.

Optionally, the eye movement information includes at least one of the following: gaze point information, pupil radius change information, and upper and lower eyelid information.

In a second aspect, an embodiment of the present application further provides an alarm device based on eye movement information, and the device includes:

Eye movement information acquisition module, configured to obtain user's eye movement information;

A setting condition determining module, configured to determine whether the eye movement information meets a setting condition;

The alarm information generating module is configured to, when satisfied, generate alarm information according to the eye movement information, and send the alarm information to a setting platform.

According to a third aspect, an embodiment of the present application further provides a mobile device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the implementation is implemented as in the present application. The alarm method based on eye movement information according to the embodiment.

In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the alarm method based on the eye movement information described in the embodiment of the present application is implemented.

In the embodiment of the present application, the user's eye movement information is first obtained, and then it is determined whether the eye movement information meets the set conditions. If it is satisfied, the alarm information is generated according to the eye movement information, and the alarm information is sent to the setting platform. The alarm method based on the eye movement information provided by this embodiment recognizes the user's eye movement information, and generates alarm messages when the eye movement information meets the set conditions, so that users who have accidents can get timely assistance, which can improve the timeliness of the alarm. And reduce the loss of life safety.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flowchart of an alarm method based on eye movement information in Embodiment 1 of the present application;

2 is a flowchart of another alarm method based on eye movement information in Embodiment 1 of the present application;

3 is a schematic structural diagram of an alarm device based on eye movement information in Embodiment 2 of the present application;

FIG. 4 is a schematic structural diagram of a mobile device in Embodiment 3 of the present application.

detailed description

The following describes the present application in detail with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present application, rather than limiting the present application. In addition, it should be noted that, for convenience of description, only some parts related to the present application are shown in the drawings instead of the entire structure.

Example one

Eye tracking can also be called gaze tracking, which is a technique that estimates eye sight and / or gaze point by measuring eye movements. Among them, the line of sight can be understood as a three-dimensional vector, and the fixation point can be understood as the two-dimensional coordinates of the three-dimensional vector projected on a certain plane.

In this embodiment, the tracking of the eyeball can be implemented by an optical recording method. The principle of the optical recording method is to use a camera or a video camera to record the eye movement of the test subject, that is, to obtain an eye image that can reflect the eye movement, and extract eye characteristics from the obtained eye image for establishing sight / fixation points. Estimation model. The eye features may include: pupil position, pupil shape, iris position, iris shape, eyelid position, eye corner position, light spot position (or Purchin spot), and the like.

Optical recording methods include pupil-corneal reflection method. The principle of the pupil-corneal reflection method is that a light source shines on the eye, and the image is captured by an image acquisition device. At the same time, the reflection point of the light source on the cornea, that is, a light spot, is obtained, thereby obtaining an eye image with a light spot.

With the rotation of the eyeball, the relative positional relationship between the pupil center and the spot changes. Multiple collected eye images with light spots can reflect the position change relationship, and the sight / fixation point is estimated based on the position change relationship.

FIG. 1 is a flowchart of an alarm method based on eye movement information provided in Embodiment 1 of this application. This embodiment can be applied to an automatic alarm situation after a vehicle fault occurs. The method can be implemented by an alarm device based on eye movement information. Execution, the device may be integrated in a terminal having an alarm function based on eye movement information, such as a wearable device, a mobile terminal, and a vehicle-mounted device. As shown in FIG. 1, the method specifically includes the following steps:

Step 110: Obtain user's eye movement information.

The eye movement information may include gaze point information, pupil radius change information, and upper and lower eyelid information. In this embodiment, a device integrated with an eye tracking module may be used to obtain the user's eye movement information, and the device may be a vehicle-mounted device, a mobile terminal, or a wearable device (such as glasses, a bracelet, and a watch). In this embodiment, the device can establish a wired or wireless connection with the control system of the vehicle, and the control system can detect the performance status of the vehicle in real time. When the vehicle is driving, due to a traffic accident or other accident, the vehicle's control system sends vehicle fault information to the device with the integrated eye tracking module. When the vehicle fault information is received, the device starts to obtain the user's eyes. Action information. Or when the user has an accident (such as being hijacked by a gangster), the device is activated by voice or gestures, so that the device can obtain the user's eye movement information in Kashgar. The user can be the driver or any passenger in the car. For example, if the device is an in-vehicle device, the driver's eye movement information is obtained. If the device is a mobile device or a wearable device, the user who carries or wears the device is obtained. User eye movement information.

Optionally, acquiring the user's eye movement information may be implemented by collecting at least two frames of the eye image of the user; performing feature analysis on the at least two frames of the eye image to obtain the user's eye movement information.

Specifically, after receiving the vehicle failure information, multiple frames of eye images of the user can be acquired within a set time, feature analysis is performed on each frame of eye images, eye characteristics in each frame of images are obtained, and then integrated Analyze the eye features in all eye images to obtain the user's eye movement information. The eye features may include: a pupil center position, a pupil radius, an iris position, an iris shape, an eyelid position, a spot center position, and the like. Gaze point information can be obtained according to the relative positional relationship between the pupil center and the spot center; the pupil radius change information can be obtained according to the pupil radius; the upper and lower eyelid information can be obtained through the eyelid position; the eyelid stagnation information and closed eye information can be obtained according to the eyelid information. Exemplarily, by analyzing multiple frames of eye images within a set time, the pupil radius is gradually enlarged, or the user keeps looking at the set position, or the user's eyes are closed.

Step 120: Determine whether the eye movement information satisfies a set condition.

The setting conditions may include a length of time when the user looks at the set position or a vital sign parameter of the user.

Optionally, judging whether the eye movement information satisfies a setting condition may be implemented in the following manner: judging whether the length of time when the user looks at the set position exceeds a set threshold. If it exceeds, an alarm message is generated.

The setting threshold can be set to any value between 3-10 seconds, for example, 5 seconds. The setting position can be any position in the car, such as the center position of the steering wheel or the position of the frame under the rearview mirror. In this embodiment, when the device integrated with the eye tracking module is a vehicle-mounted device, the position of the user's gaze can be obtained through the gaze point information, and the duration of the user's gaze at the set position can be obtained by analyzing the gaze point information of the multi-frame eye image. Whether it exceeds the set threshold. When the device integrated with the eye tracking module is a mobile terminal or a wearable device, the user's gaze position can be obtained through the gaze point information and the current three-dimensional coordinate information of the device, that is, comprehensively analyzing the gaze point information and the device's three-dimensional coordinate information, and then Map to the corresponding position in the car, and analyze whether the length of time that the user looks at the set position exceeds the set threshold by analyzing multiple frames of eye images. Specifically, when the length of time that the user looks at the set position exceeds the set threshold, it can be determined that the user currently needs assistance, and an alarm message is generated.

Optionally, judging whether the eye movement information satisfies a set condition may be implemented in the following manner: judging whether the vital signs of the user are in a set dangerous range according to the eye movement information. If so, an alarm message is generated.

In this embodiment, whether the user has low consciousness or is in a coma can be determined based on pupil radius change information and eyelid information. Exemplarily, by analyzing multiple frames of eye images and obtaining that the user's eyes are always closed, it can be preliminarily determined that the user is already in a comatose state, or the user's eyelids have been in a stagnation state, then it can be preliminarily determined that the user's consciousness is low. When the user's vital signs are within the set dangerous range, timely rescue is required. At this time, an alarm message is generated.

Step 130: If it is satisfied, generate alarm information according to the eye movement information, and send the alarm information to the setting platform.

Among them, the alarm information may include the license plate number, the owner information, and the user's accident. The setting platform can be a medical monitoring platform or a 110 alarm platform.

Optionally, generating the alarm information according to the eye movement information may be implemented by: locating the user's current position to obtain the position information; and generating the alarm information according to the position information and the eye movement information.

In this embodiment, a global positioning system (Global Positioning System, GPS) can be installed in a device integrated with an eye tracking module, and when a vehicle malfunctions, the user's current geographic location is located by GPS to obtain the user Location information. Combining location information with eye movement information to generate alarm information, the advantage of this is that not only can ambulance personnel obtain the user's current location in time, so as to find the user in time, but also obtain the user based on the user's eye movement information Current vital signs status to prepare effective rescue measures.

Specifically, after the alarm information is generated, the alarm information is sent to the setting platform, so that the user can get timely assistance.

Optionally, after sending the alarm information to the setting platform, the method further includes the following steps: sending the alarm information to a mobile device associated with the vehicle-mounted device.

The mobile terminal associated with the in-vehicle device may be a mobile device of a user who has a kinship relationship with the user. The advantage of this is that the relatives of the user can know the current situation of the user in a timely manner, and then take corresponding measures.

The technical solution of this embodiment first obtains the user's eye movement information, and then determines whether the eye movement information satisfies a set condition. If it is satisfied, an alarm message is generated based on the eye movement information, and the alarm message is sent to a setting platform. The alarm method based on the eye movement information provided by this embodiment recognizes the user's eye movement information, and generates alarm messages when the eye movement information meets the set conditions, so that users who have accidents can get timely assistance, which can improve the timeliness of the alarm. And reduce the loss of life safety.

FIG. 2 is a flowchart of another alarm method based on eye movement information provided in Embodiment 1 of the present application. As a further explanation of the foregoing embodiment, as shown in FIG. 2, the method includes the following steps:

Step 210: Collect at least two frames of eye images of the user.

Step 220: Perform feature analysis on at least two frames of eye images to obtain user's fixation point information, pupil radius change information, and upper and lower eyelid information.

Step 230: When the length of time the user looks at the set position exceeds a set threshold or if the vital signs of the user are in a set dangerous range, position information of the user is acquired.

Step 240: Generate alarm information according to the position information and the user's eye movement information, and send the alarm information to the setting platform.

Example two

FIG. 3 is a schematic structural diagram of an alarm device based on eye movement information provided in Embodiment 2 of the present application. As shown in FIG. 3, the device includes an eye movement information acquisition module 310, a setting condition determination module 320, and an alarm information generation module 330.

The eye movement information acquisition module 310 is configured to obtain the user's eye movement information when the vehicle failure information is received;

The setting condition determining module 320 is configured to determine whether the eye movement information meets a setting condition;

The alarm information generating module 330 is configured to, when satisfied, generate alarm information according to the eye movement information, and send the alarm information to a setting platform.

Optionally, the eye movement information acquisition module 310 is further configured to:

Collect at least two frames of eye images of the user;

Perform feature analysis on at least two frames of eye images to obtain user's eye movement information.

Optionally, the setting condition determining module 320 is further configured to:

The alarm information generating module 330 is further set to

If it exceeds, an alarm message is generated.

Judging whether the user's vital signs are in a set dangerous range according to the eye movement information;

The alarm information generating module 330 is further configured to:

If so, an alarm message is generated.

The alarm information generating module 330 is further configured as:

Locate the user's current location to obtain location information;

Generate alarm information based on position information and eye movement information.

Optionally, the method further includes: sending alarm information to a mobile device associated with the vehicle-mounted device.

The above device can execute the methods provided by all the foregoing embodiments of the present application, and has corresponding functional modules and beneficial effects for executing the above methods. For technical details not described in detail in this embodiment, reference may be made to the methods provided in all the foregoing embodiments of the present application.

Example three

FIG. 4 is a schematic structural diagram of a mobile device provided in Embodiment 3 of the present application. As shown in FIG. 4, a mobile device provided in this embodiment includes a processor 41 and a memory 42. The processor in the mobile device may be one or more. In FIG. 4, a processor 41 is used as an example. The processor 41 and the memory 42 in the mobile device may be connected through a bus or other methods. Take the bus connection as an example.

The processor 41 of the mobile device in this embodiment integrates the alarm device based on the eye movement information provided in the foregoing embodiment. In addition, the memory 42 in the mobile device is used as a computer-readable storage medium, and may be configured to store one or more programs. The programs may be software programs, computer-executable programs, and modules. Program instructions / modules for alarm methods of eye movement information. The processor 41 runs software programs, instructions, and modules stored in the memory 42 to execute various functional applications and data processing of the device, that is, to implement the alarm method based on the eye movement information in the foregoing method embodiment.

The memory 42 may include a storage program area and a storage data area, where the storage program area may store an operating system and application programs required for at least one function; the storage data area may store data created according to the use of the device, and the like. In addition, the memory 42 may include a high-speed random access memory, and may further include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 42 may further include a memory remotely disposed with respect to the processor 41, and these remote memories may be connected to the device through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

The processor 41 executes various functional applications and data processing by running a program stored in the memory 42 to implement the alarm method based on the eye movement information provided in the embodiment of the present application.

Embodiment 4

The computer storage medium of the embodiment of the present application stores a computer program thereon. When the program is executed by the data backup device, the alarm method based on the eye movement information provided by the embodiment of the present application is implemented.

The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), Erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing. In this document, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in combination with an instruction execution system, apparatus, or device.

The computer-readable signal medium may include a data signal in baseband or propagated as part of a carrier wave, which carries a computer-readable program code. Such a propagated data signal may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, and the computer-readable medium may send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device .

Program code embodied on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for performing the operations of this application may be written in one or more programming languages, or a combination thereof, including programming languages such as Java, Smalltalk, C ++, and also conventional Procedural programming language—such as "C" or similar programming language. The program code can be executed entirely on the user's computer, partly on the user's computer, as an independent software package, partly on the user's computer, partly on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as through an Internet service provider) Internet connection).

Note that the above are only the preferred embodiments of this application and the technical principles applied. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and that those skilled in the art can make various obvious changes, readjustments, and substitutions without departing from the scope of protection of this application. Therefore, although the present application has been described in detail through the above embodiments, the present application is not limited to the above embodiments, and without departing from the concept of the present application, it may also include more other equivalent embodiments. The scope is determined by the scope of the appended claims.

Claims

An alarm method based on eye movement information includes:

Get user's eye movement information;

Determining whether the eye movement information satisfies a set condition;

If it is satisfied, generate alarm information according to the eye movement information, and send the alarm information to a setting platform.
The method according to claim 1, wherein the obtaining the user's eye movement information comprises:

Collect at least two frames of eye images of the user;

Perform feature analysis on the at least two frames of eye images to obtain user eye movement information.
The method according to claim 2, wherein the determining whether the eye movement information satisfies a set condition comprises:

Judging whether the length of time the user looks at the set position exceeds a set threshold;

Correspondingly, if satisfied, generating alarm information according to the eye movement information includes:

If it exceeds, an alarm message is generated.
The method according to claim 2, wherein the determining whether the eye movement information satisfies a set condition comprises:

Determining whether the vital signs of the user are in a set dangerous range according to the eye movement information;

Correspondingly, if satisfied, generating alarm information according to the eye movement information includes:

If so, an alarm message is generated.
The method according to claim 2, wherein the generating alarm information based on the eye movement information comprises:

Locate the user's current location to obtain location information;

Generate alarm information based on the position information and the eye movement information.
The method according to claim 1, wherein after sending the alarm information to a setting platform, further comprising:

Send the alarm information to a mobile device associated with the vehicle-mounted device.
The method according to any one of claims 1-6, wherein the eye movement information comprises at least one of the following: gaze point information, pupil radius change information, and upper and lower eyelid information.
An alarm device based on eye movement information includes:

The eye movement information acquisition module is configured to obtain the user's eye movement information when the vehicle failure information is received;

A setting condition determining module, configured to determine whether the eye movement information meets a setting condition;

The alarm information generating module is configured to, when satisfied, generate alarm information according to the eye movement information, and send the alarm information to a setting platform.
A mobile device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method according to any one of claims 1-7 is implemented.
A computer-readable storage medium having stored thereon a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.