CN115168163A

CN115168163A - User action detection method and device, electronic equipment and storage medium

Info

Publication number: CN115168163A
Application number: CN202110374220.0A
Authority: CN
Inventors: 李国成
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2021-04-07
Filing date: 2021-04-07
Publication date: 2022-10-11

Abstract

The present disclosure relates to the technical field of electronic devices, and in particular, to a method and an apparatus for detecting user actions, an electronic device, and a storage medium, where the method includes: receiving first data, wherein the first data signal is sent by the earphone according to the head state of a user; determining whether the user is in a head-down state according to the first data; when a user is in a head-down state, acquiring second data, wherein the second data comprises the inclination angle of the terminal equipment; and determining the current user action according to the second data, wherein the current user action comprises the underhead used terminal equipment and the non-underhead used terminal equipment. The detection of the user lowering the head to use the terminal equipment is realized.

Description

User action detection method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the technical field of electronic devices, and in particular, to a user action detection method and apparatus, an electronic device, and a storage medium.

Background

With the development and progress of the technology, the application of mobile communication devices such as mobile phones and the like is more and more extensive, and the functions of the mobile phones are more and more abundant. The user may use the mobile phone by lowering his head for a long time, and the user may be injured on the cervical vertebra and the like when using the mobile phone for a long time. Therefore, a method for detecting that a user uses an electronic device such as a mobile phone with his/her head down is needed.

It is noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present disclosure and therefore may include information that does not constitute prior art that is already known to a person of ordinary skill in the art.

Disclosure of Invention

The present disclosure is directed to a user action detection method and apparatus, an electronic device, and a storage medium, which further enable detection of a user using a terminal device with his head down.

According to a first aspect of the present disclosure, there is provided a user action detection method, the method comprising:

receiving first data, wherein the first data signal is sent by the earphone according to the head state of a user;

determining whether the user is in a head-down state according to the first data;

when a user is in a head-down state, acquiring second data, wherein the second data comprises the inclination angle of the terminal equipment;

and determining the current user action according to the second data, wherein the current user action comprises the head lowering use terminal equipment and the non-head lowering use terminal equipment.

According to a second aspect of the present disclosure, there is provided a user motion detection apparatus comprising:

the receiving module is used for receiving first data, and the first data signal is sent by the earphone according to the head state of a user;

the first determining module is used for determining whether the user is in a head lowering state or not according to the first data;

the acquisition module is used for acquiring second data when a user is in a head-lowering state, wherein the second data comprises the inclination angle of the terminal equipment;

and the second determining module is used for determining the current user action according to the second data, wherein the current user action comprises the underhead used terminal equipment and the non-underhead used terminal equipment.

According to a third aspect of the present disclosure, there is provided an electronic device comprising

A processor; and

a memory having computer readable instructions stored thereon which, when executed by the processor, implement a method according to any of the above.

According to a fourth aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method according to any one of the above.

According to the user action detection method provided by the embodiment of the disclosure, whether the user is in a head-down state is determined through the first data, the second data is obtained when the user is in the head-down state, and whether the current user is in a state of using the terminal device in a head-down state is determined through the second data, so that the detection of using the terminal device in a head-down state is realized. And then can remind according to the state that the user used terminal equipment to hang down, avoid long-time hanging down to use terminal equipment to cause the injury to user's glasses to and avoid hanging down to walk and use terminal equipment, stop the potential safety hazard.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The above and other features and advantages of the present disclosure will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.

Fig. 1 is a flowchart of a first user action detection method provided in an exemplary embodiment of the present disclosure;

fig. 2 is a flowchart of a second user action detection method provided in an exemplary embodiment of the present disclosure;

FIG. 3 is a schematic view of a relationship between air pressure and height provided by an exemplary embodiment of the present disclosure;

FIG. 4 is a schematic diagram of an air pressure sensor provided in an exemplary embodiment of the present disclosure;

fig. 5 is a flowchart of a third user action detection method according to an exemplary embodiment of the present disclosure;

fig. 6 is a flowchart of a fourth method for detecting user actions according to an exemplary embodiment of the disclosure;

fig. 7 is a flowchart of a fifth user action detection method provided in an exemplary embodiment of the present disclosure;

fig. 8 is a flowchart of a sixth user action detection method according to an exemplary embodiment of the disclosure;

fig. 9 is a flowchart of a seventh user action detection method according to an exemplary embodiment of the disclosure;

fig. 10 is a block diagram of a user action detection apparatus provided in an exemplary embodiment of the present disclosure;

fig. 11 is a schematic diagram of an electronic device provided in an exemplary embodiment of the present disclosure;

fig. 12 is a schematic diagram of a computer-readable storage medium according to an exemplary embodiment of the disclosure.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and a repetitive description thereof will be omitted.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the embodiments of the disclosure can be practiced without one or more of the specific details, or with other methods, components, materials, devices, steps, and so forth. In other instances, well-known structures, methods, devices, implementations, materials, or operations are not shown or described in detail to avoid obscuring aspects of the disclosure.

The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, or in one or more software-hardened modules, or in different networks and/or processor devices and/or microcontroller devices.

First, an exemplary embodiment of the present disclosure provides a user action detection method, as shown in fig. 1, the user action detection method may include the following steps:

step S110, receiving first data, wherein a first data signal is sent by the earphone according to the head state of a user;

step S120, determining whether the user is in a head-down state according to the first data;

step S130, when the user is in a head-down state, second data is obtained, and the second data comprises the inclination angle of the terminal equipment;

step S140, determining the current user action according to the second data, wherein the current user action comprises the underhead used terminal equipment and the non-underhead used terminal equipment.

According to the user action detection method provided by the embodiment of the disclosure, whether the user is in the head lowering state is determined through the first data, the second data is obtained when the user is in the head lowering state, and whether the current user is in the state of using the terminal device by lowering the head is determined through the second data, so that the detection of using the terminal device by lowering the head of the user is realized. And then can remind according to the state that the user stoop down and used terminal equipment, avoid long-time stoop down and use terminal equipment and cause the injury to user's glasses to and avoid the walking of stoop down and use terminal equipment, stop the potential safety hazard.

In the embodiment of the disclosure, when the headset is in a wearing state, the headset is connected with the terminal device. The headset and the terminal device may be connected wirelessly, e.g. the headset and the terminal device may be connected via bluetooth. Or the terminal device and the headset may be connected by a wire, for example, the headset and the terminal device may be connected by a connecting wire.

In the embodiment of the present disclosure, the terminal device may be a mobile phone or a tablet computer, and the terminal device may be connected to an earphone and a wearable device (watch). The earphone and the terminal equipment can perform data interaction, a sensing device is arranged in the earphone and can be used for sensing and generating first data, and the first data can be used for determining whether a user is in a head-down state or not. The terminal equipment can be further connected with wearable equipment (watch), and the wearable equipment (watch) can perform data interaction with the terminal equipment and the earphone. The wearable device (watch) can be provided with a collecting device for collecting physical sign data such as the heart rate of a user. The user action detection method provided in the embodiments of the present disclosure may be executed in a terminal device or may also be executed in a wearable device (watch).

The following will describe in detail the steps of the user action detection method provided by the embodiment of the present disclosure:

in step S110, first data may be received, and the first data signal is transmitted by the headset according to the head state of the user.

In a possible embodiment of the present disclosure, as shown in fig. 2, the receiving the first data in step S110 may be implemented by:

and step S111, acquiring height data of the earphone, wherein the height data is the distance between the earphone and the first reference surface.

The first data may comprise height data of the headset, the height data of the headset being a distance of the headset from the first reference surface. The earphone can send the height data once at intervals of preset time when the earphone is in a wearing state, and the corresponding terminal equipment or the watch receives the height data once at intervals of preset time. For example, the headset may transmit the altitude data every 0.1 second, 0.2 second, 0.3 second, 0.4 second, etc., and the terminal device or the watch may receive the altitude data every 0.1 second, 0.2 second, 0.3 second, 0.4 second, etc.

Wherein the height refers to a distance of the headset with respect to a reference plane perpendicular to the height direction. For example, the height may be the height of the headset relative to sea level, or the height may be the height of the headset from the plane of the user's feet.

When the earphone is a wireless earphone, the earphone can transmit the first data to the terminal device or the watch through wireless transmission modes such as Bluetooth and the like. When the earphone is a wired earphone, the earphone can transmit the first data to the terminal device or the watch in a wired transmission mode.

The height difference of the earphone at different time represents the height difference of the head of the user at different time, so that whether the user performs head lowering action or not can be judged according to the height difference of the earphone at different time.

In the embodiment of the present disclosure, the height of the earphone may be determined by detecting the air pressure of the earphone. Thus, obtaining the height data of the earphone can be realized by the following steps: and acquiring air pressure data detected by the earphone, and determining height data according to the air pressure data.

An air pressure sensor can be arranged on the earphone, the air pressure sensor collects the current atmospheric pressure value once every preset time interval, and the atmospheric pressure value is sent to the mobile phone or the watch. The mobile phone or the watch can obtain the current height value of the earphone according to the current atmospheric pressure value. In practical application, the height of the earphone can be calculated through the atmospheric pressure value, and the height value of the earphone is calculated and then sent to the mobile phone or the watch.

The air pressure sensor may collect the atmospheric pressure value once every preset time, for example, the air pressure sensor may collect the atmospheric pressure once every 0.1 second, 0.2 second, 0.3 second, 0.4 second. Or the atmospheric pressure sensor can acquire the atmospheric pressure value in real time, and the earphone sends the atmospheric pressure value once every preset time. That is, the conversion of the atmospheric pressure value into the altitude data may be performed in the headset or in the terminal device, and the embodiments of the present disclosure are not limited thereto.

The relationship between atmospheric pressure and altitude is shown in fig. 3, that is, the atmospheric pressure and altitude are inversely related, and the altitude can be determined by the corresponding relationship between atmospheric pressure and altitude.

The gas pressure sensor is an instrument for measuring absolute pressure of gas, and as shown in fig. 4, the gas pressure sensor may include: a sensing element 410, a conversion unit 420, a microprocessing unit 430 and a memory unit 440. The sensing element 410 is a thin film sensitive to the pressure and a pin-on control, which is electrically connected to a flexible resistor. When the pressure of the gas to be detected is reduced or increased, the film is deformed to drive the thimble, and the resistance value of the resistor is changed. The resistance of the resistor changes, and a signal voltage of 0-5V is obtained from the sensing element. The conversion unit 420 is connected to the sensor element 410, and the electric signal detected by the sensor element 410 is received by the microprocessor unit 430 through a/D conversion. The micro-processing unit 430 is connected to the storage unit 440, and the micro-processing unit can store data in a memory, or transmit the air pressure data to other modules, such as a processor in the earphone or a terminal device, through the single chip 450.

It will be appreciated that the acquisition of the altitude data of the headset may also be achieved in other ways, such as by a satellite positioning sensor detecting the altitude of the headset. Wherein, the height of the headset can be detected by arranging a GPS sensor on the headset. The GPS sensor may collect the altitude data of the headset once every preset time interval, and send the altitude data to the terminal device or the watch once every preset time interval.

In a possible embodiment of the present disclosure, as shown in fig. 5, the receiving the first data in step S110 may be implemented by:

and step S112, acquiring inclination angle data of the earphone, wherein the inclination angle data of the earphone is the inclination angle of the earphone and the second reference surface.

The first data may comprise tilt angle data of the headset, and the tilt angle of the headset may be a tilt angle of the headset and the second reference plane. The gyroscope can be arranged in the earphone, and the inclination angle of the earphone is detected through the gyroscope. The inclination angle of the earphone is sent to the terminal equipment or the watch in a wired or wireless mode, and the terminal equipment or the watch receives the inclination angle data sent by the earphone.

The headset can transmit the inclination angle of the headset to the terminal device or the watch at preset intervals when the headset is in a wearing state. Accordingly, the terminal device or the watch can receive the inclination angle data of the earphone once at preset time intervals. For example, the headset may transmit the tilt angle data every 0.1 second, 0.2 second, 0.3 second, 0.4 second, and the like, and the terminal device or the watch may receive the tilt angle data every 0.1 second, 0.2 second, 0.3 second, 0.4 second, and the like.

The second reference surface may be a vertical surface or a horizontal surface, or the second reference surface may be a plane where a preset axis in the headset is located when the user is in the head-up state. The predetermined axis in the earpiece may be a reference line virtually present in the earpiece. The reference line is fixed in its relative position in the headset, for example the reference line may be a line passing through the centre of gravity of the headset. The tilt angle of the headset may represent a tilt angle of the head of the user, and when the change in the angle of the head of the user is greater than a preset angle threshold, the user is considered to be in a heads-down state.

In a possible embodiment of the present disclosure, the determination of whether the user is in a heads-down state may be performed in a headset. As shown in fig. 6, the receiving of the first data in step S110 may be implemented by:

step S113, a head lowering signal is obtained, where the head lowering signal is a signal sent when the headset detects that the user lowers his head.

The earphone can detect the height of the earphone in a wearing state, the earphone determines whether the user is in a head lowering state according to the height difference of different time, when the earphone determines that the user is in the head lowering state, the earphone sends first data (head lowering signals) to the terminal equipment or the watch, and the terminal equipment or the watch receives the head lowering signals.

The headset may collect the height data once at preset intervals, for example, the headset may receive the height data every 0.1 second, 0.2 second, 0.3 second, 0.4 second, etc. After the height data of the earphones are collected, whether the user is in a head lowering state or not can be determined according to the height data of the earphones.

For example, when the difference between the first height and the second height is greater than a preset height value and the duration of the second height is greater than a preset threshold, the headset determines that the user is in a head-down state, the first height is a height value of the headset at a first time, the second height is a height value of the headset at a second time, and the second time and the first time are separated by a preset time.

Wherein the first height refers to a distance of the earphone relative to a reference plane perpendicular to the height direction at a first time, and the second height refers to a distance of the earphone relative to the reference plane perpendicular to the height direction at a second time. For example, the first altitude may be an altitude of the headset relative to sea level at a first time and the second altitude may be an altitude of the headset relative to sea level at a second time. Or the first height may be a height of the headset from a plane in which the user's feet are located at a first time, the second height may be a height of the headset from the plane in which the user's feet are located at a second time, and so on.

When the height data of the headset is determined by the air pressure data of the headset, determining whether the user is in a heads-down state according to the first data may be accomplished by: and determining the height difference of the earphone according to the first air pressure value and the second air pressure value detected by the earphone, and determining that the user is in a head-lowering state when the height difference of the earphone is greater than a preset height threshold value and the duration time of the earphone at the second height is greater than the preset threshold value.

Wherein the first altitude may be determined from the first air pressure value and the second altitude may be determined from the second air pressure value. The first air pressure value is the air pressure value of the earphone at the first moment, and the second air pressure value is the air pressure value of the earphone at the second moment.

In step S120, it may be determined whether the user is in a heads-down state according to the first data.

In a possible embodiment of the present disclosure, the first data comprises height data of the headset. As shown in fig. 2, the determination of whether the user is in the heads-down state according to the first data in step S120 may be implemented by:

step S121, when the difference value between the first height and the second height is larger than a preset height value and the duration of the second height is larger than a preset threshold value, it is determined that the user is in a head-down state, the first height is the height value of the earphone at the first moment, the second height is the height value of the earphone at the second moment, and the second moment and the first moment are separated by a preset time.

That is, when the height difference of the earphones is greater than the preset height value at the preset time interval and the duration time of the second height of the earphones is greater than the preset threshold value, it is determined that the user is in a head lowering state. Wherein the preset height threshold may be 30 mm, 35 mm, 40 mm, etc., and the preset threshold for the duration of the second height may be 3 seconds, 5 seconds, 6 seconds, etc.

When the earphone is in a wearing state, the terminal equipment or the watch can detect the height difference of the earphone at the adjacent time points in real time. That is, the first time and the second time are separated by a preset time, and the first time is changed along with the advance of the time, and meanwhile, the second time is also changed along with the advance of the time.

When the difference value between the first height and the second height is larger than the preset height threshold value, the terminal device or the watch determines that the user is possibly in a head-down state. Further, if the earphone detects that the duration time of the earphone at the second height is greater than the preset threshold time, the terminal device or the watch determines that the user is in a head-down state.

The earphone is continuously arranged at the second height for the preset threshold time, namely, the earphone is always arranged at the second height in the preset threshold time from the second moment. The height of the headset at the second height may be the same as the height of the headset or a difference between the height of the headset and the second height is within an allowable range (e.g., 5mm, 8mm, 10mm, etc.).

In a possible embodiment of the present disclosure, when the first data may include tilt angle data of the headset, as shown in fig. 5, the step S120 of determining whether the user is in a head-down state according to the first data may be implemented by:

step S122, when the difference between the first inclination angle and the second inclination angle is greater than the preset angle value and the duration is greater than the preset threshold, it is determined that the user is in a head-down state, where the first inclination angle is an inclination angle of the headset at a first moment, the second inclination angle is an inclination angle of the headset at a second moment, and the interval between the second moment and the first moment is preset time.

The tilt angle of the headset can represent the tilt angle of the user's head relative to the second reference plane, i.e. the tilt angle of the user's head can be determined by the tilt angle of the headset. When the head of the user is switched from the head-up state to the head-down state, the inclination angle of the earphone can be changed, so that whether the user is in the head-down state or not can be detected by detecting the inclination angle of the earphone.

For example, the second reference surface may be a vertical surface, the tilt angle of the headset and the second reference surface is 0 degree when the user is in a head-up state, and the tilt angle of the headset and the second reference surface may be greater than 30 degrees when the user is in a head-down state. However, when the body of the user is in a tilted state, such as when the user is in an uphill or downhill state, the angle between the headset and the second reference plane is also nonzero when the user is in a head-up state. However, when the user switches from the head-up state to the head-down state, the angle of rotation of the user's head is not changed under different circumstances. It is thus possible to determine whether the user is in a heads-down state by the difference in the inclination angles of the earphones at preset time intervals.

When the earphone is in a wearing state, the terminal equipment or the watch can detect the inclination angle difference of the earphone at the adjacent time points in real time. That is, the first time and the second time are separated by a preset time, and the first time is changed along with the advance of the time, and meanwhile, the second time is also changed along with the advance of the time.

When the difference between the first inclination angle and the second inclination angle is greater than a preset angle threshold (for example, 30 degrees), the terminal device or the watch determines that the user may be in a head-down state. Further, if the earphone detects that the duration time of the second inclination angle of the earphone is greater than the preset threshold time, the terminal device or the watch determines that the user is in a head-down state. Wherein, the preset angle threshold may be 30 degrees, 35 degrees, 40 degrees, etc., and the preset threshold of the duration of the second inclination angle may be 3 seconds, 5 seconds, 6 seconds, etc.

The earphone continuously keeps the preset threshold time at the second inclination angle means that the earphone is always at the second inclination angle in the preset threshold time from the second moment. The headset at the second tilt angle may be that the tilt angle of the headset is the same as the second tilt angle or that the difference between the tilt angle of the headset and the second tilt angle is within an allowable range (e.g., 5 degrees, 8 degrees, 10 degrees, etc.).

In a possible implementation manner of the embodiment of the present disclosure, the first data includes a low head signal, and the low head signal is a signal transmitted when the user is in a low head state. On this basis, as shown in fig. 6, the determination of whether the user is in the head-down state according to the first data in step S120 can be realized by the following steps:

step S123, when the terminal device or the watch receives the head lowering signal, the terminal device or the watch determines that the user is in the head lowering state.

The head-down signal may be a trigger signal, and when the terminal device or the watch receives the head-down signal, it may be determined that the user is in a head-down state. And when the terminal equipment or the watch does not receive the head lowering signal, determining that the user is in a head raising state.

In step S130, when the user is in a heads-down state, second data is obtained, where the second data includes the tilt angle of the terminal device.

When the user is in a head-down state, it is necessary to further detect whether the user is using the terminal device. Whether the terminal device is in a use state can be judged by detecting the inclination angle of the terminal device. The inclination angle of the terminal device can be obtained through a gyroscope arranged inside the terminal device.

A gyroscope, also called an angular velocity sensor, is different from an accelerometer (G-sensor), and measures a physical quantity, i.e., a rotational angular velocity in yaw and pitch. The gyroscope is used for sensing and maintaining the direction and is designed based on the theory of conservation of angular momentum. The gyroscope is mainly composed of a rotatable rotor which is positioned at an axis. When the gyroscope is rotating, the gyroscope tends to resist changes in direction due to the angular momentum of the rotor.

The gyroscope may include: the gyroscope rotor generally adopts a dragging method such as a synchronous motor, a hysteresis motor, a three-phase alternating current motor and the like to enable the gyroscope rotor to rotate around a rotating shaft at a high speed, and the rotating speed of the gyroscope rotor is approximate to a constant value; the frame can comprise an inner frame (inner ring) and an outer frame (outer ring), and the inner ring and the outer ring are hinged to enable the spinning top to obtain a required angular rotation degree of freedom; the accessories may include devices such as torque motors and signal sensors for detecting rotational angular velocity during yaw and pitch.

The way the gyroscope calculates the offset angle is as follows:

the quaternion of the DMP output is in q30 format, i.e., the floating point number is raised by a factor of 30 to the power of 2. Therefore, before converting into euler angles, the euler angles must be converted into floating point numbers, namely, the floating point numbers are divided by 2 to the power of 30, and then calculation is performed, wherein the calculation formula is as follows:

q0= quat [0]/q30; // q30 format to floating point number

q1＝quat[1]/q30；

q2＝quat[2]/q30；

q3= quat [3]/q30; calculating to obtain pitch angle/roll angle/course angle

pitch = asin (-2 × q1 × q3+2 × q0 × q 2) × 57.3; v/pitch angle

rol = atan2 (2 × q3+2 × q0 × q1, -2 × q1-2 × q2+ q 1) +57.3; v/roll angle

yaw = atan2 (q 1 × q2+ q0 × q 3), q0 × q0+ q1 × q1-q2 × q2-q3 × q 3) 57.3; // heading angle

Where quat [0] to quat [3] are the quaternions of the motion processing component MPU after DMP resolution, which are in q30 format and therefore divided by a power of 30 of 2, where q30 is a constant: 1073741824, i.e. 30 th power of 2, is then substituted into the formula to calculate the euler angle. 57.3 of the above calculation is the conversion of radians to angles, i.e. 180/pi, so that the result is obtained in degrees (°). Thus, the angles of the heading angle (yaw), roll angle (roll) and pitch angle (pitch) can be obtained, and whether the preset angle threshold value is exceeded or not can be judged through the angles.

The orientation of the screen of the terminal equipment can be detected by detecting the inclination angle of the terminal equipment through the gyroscope, and when the screen of the terminal equipment is detected by the gyroscope to face the eyes of a user, the user is determined to be in a state of using the terminal equipment with the user in a head-down state.

Since it is possible that the user does not use the terminal device even when the screen of the terminal device is directed toward the eyes of the user, whether the user is using the terminal device can be further judged by detecting whether the screen is lit. Detecting whether the screen is lit may be accomplished by detecting whether the screen is powered on.

Of course, in practical applications, the second data may also include image data, the image data may be obtained by a front camera of the terminal device, and when the image data includes an eye image of the user, the user may be considered to be in a state of using the terminal device with a head down.

When the user state detection method provided by the embodiment of the disclosure is executed in wearable devices such as a watch, the watch acquires the inclination angle data of the terminal device from the terminal device. The communication mode can be realized through Bluetooth, infrared rays or WiFi and the like.

Furthermore, when a user lowers the head to use the terminal equipment and walks, the potential safety hazard is high, and various accidents are easily caused. It is therefore necessary to detect whether the user is moving (walking) while using the terminal device with his head down. At this time, the number of steps of the user can be obtained from the terminal device or a step counting device in the watch, and the heart rate of the user can be obtained from a heart rate monitoring device in the terminal device. The user may be determined to be in a state of motion when the number of steps of the user increases. Or may be considered in motion when the heart rate of the user increases.

The second data further includes the number of steps of the user and/or the heart rate of the user, as shown in fig. 7, the step S130 of acquiring the second data when the user is in a head-down state may be implemented by:

step S131, when the user is in a head-down state, the inclination angle of the terminal device is obtained, and the step number of the user and/or the heart rate of the user are obtained.

The step number of the user can be detected by a step counting device in the terminal device or the watch, for example, a GPS sensor arranged in the terminal device or the watch detects the step number of the user. The heart rate of the user may be detected by a heart rate sensor disposed in the watch.

The heart rate sensor collects heart rate signals of a user through a photoelectric volume method. The basic principle of the photoplethysmography is to measure the pulse by using the difference of the light transmittance of human tissues caused by the pulsation of blood vessels. The sensor used by the device consists of a light source and a photoelectric converter, and is fixed on the finger or the earlobe of a patient through a bandage or a clip. The light source is typically a light emitting diode of a certain wavelength (500 nm to 700 nm) selective for oxygen and hemoglobin in arterial blood. When light beam penetrates through peripheral blood vessel of human body, the light transmittance of the light beam is changed due to blood congestion volume change of artery pulsation, and at the moment, the light reflected by human body tissue is received by the photoelectric transducer, converted into electric signal, amplified and output. Since the pulse is a signal that changes periodically with the pulsation of the heart and the arterial blood vessel volume also changes periodically, the change period of the electrical signal of the photoelectric transducer is the pulse rate.

The wave with wavelength of about 560nm can reflect the information of the skin superficial arterioles, and is suitable for extracting pulse signals. The heart rate sensor employs a green LED with a peak wavelength of 515nm and an ambient light sensor. The ambient light receptor has the sensing peak wavelength of 565nm, the green light LED is close to the ambient light receptor in peak wavelength, and the sensitivity is higher. In addition, because the frequency band of the pulse signal is generally between 0.05 Hz and 200Hz, the signal amplitude is very small, generally at the level of millivolt, and the pulse signal is easily interfered by various signals. A low-pass filter and an amplifier are used behind the sensor to amplify the signal by 330 times, and meanwhile, a divider resistor is adopted to set the direct-current bias voltage to be 1/2 of the power voltage, so that the amplified signal can be well collected by an AD (analog-to-digital) circuit of a single chip in the watch.

For example, the heart rate sensor may include a green LED, a photoreceptor, a filter circuit and an amplifying circuit, the green LED emits green light, the photoreceptor receives light reflected by the human body, the filter circuit is connected to the photoreceptor, the filter circuit is used for filtering signals collected by the photoreceptor, and the amplifying circuit is connected to the filter circuit and is used for amplifying signals output by the filter circuit.

In step S140, a current user action may be determined based on the second data, the current user action including a underhead-used terminal device and a non-underhead-used terminal device.

And when the second data comprises the inclination angle of the terminal equipment, and when the inclination angle of the terminal equipment is in a preset angle range, determining that the user lowers the head to use the terminal equipment. When the tilt angle of the terminal device is outside the preset angle range, it may be determined that the user is in a state of using the terminal device without lowering his head.

When the user leans down to use the electronic equipment for a long time, cervical spondylosis and other problems can be caused, so that the user can be reminded after the user leans down to use the electronic equipment for too long. As shown in fig. 8, the user action detection method provided by the embodiment of the present disclosure may further include the following steps:

step S150, when the time that the user lowers the head to use the terminal equipment is detected to be larger than a first time threshold value, outputting a first reminding signal, wherein the first reminding signal is used for reminding the user that the user is in the state of lowering the head to use the terminal equipment currently.

The first reminding signal can be realized by carrying out popup on the terminal device, and when the time for the user to use the electronic device by lowering the head is greater than a first time threshold value, the popup is carried out on a display screen of the terminal device for reminding. Or the first reminding signal can also be a voice reminding signal or a vibration reminding signal and the like. Of course, in practical application, the user can be reminded simultaneously by combining various modes such as pop-up window, voice and vibration.

Further, the user's lowering head using the terminal device may include a lowering head moving using terminal device and a user's lowering head still using terminal device state. The second data may also include a number of steps of the user and/or a heart rate of the user. Determining the current user action according to the second data, which may further include:

step S160, when the user uses the terminal device while lowering his head and the number of steps of the user and/or the heart rate of the user increases, it is determined that the user uses the terminal device while lowering his head. And when the user lowers head to use the terminal equipment and the step number of the user and the heart rate of the user are not increased, determining that the user is at the head-lowering state and uses the terminal equipment.

When the user uses the electronic device by long-term head lowering movement, traffic accidents may occur or the road surface is not good, and the user is easy to fall down. Therefore, the user can be reminded when the user uses the electronic equipment for long-term head lowering movement. As shown in fig. 9, the user action detection method provided by the embodiment of the present disclosure may further include the following steps:

step S170, when the time for the user to use the terminal equipment in the head lowering motion is greater than a second time threshold, outputting a second reminding signal, wherein the second reminding signal is used for reminding the user that the user is in the state of using the terminal equipment in the head lowering motion.

The second reminding signal can be implemented by performing popup on the terminal device, and when the time for the user to use the electronic device by lowering the head is greater than a second time threshold, popup reminding is performed on a display screen of the terminal device. Or the first reminding signal can also be a voice reminding signal or a vibration reminding signal and the like. Of course, in practical application, the user can be reminded simultaneously by combining various modes such as pop-up window, voice and vibration.

The embodiment of the disclosure provides a user action detection method, which determines whether a user is in a head-down state or not through first data, acquires second data when the user is in the head-down state, and determines whether the current user is in a state of using terminal equipment in a head-down state or not through the second data, thereby realizing detection of using the terminal equipment in a head-down state. And then can remind according to the state that the user stoop down and used terminal equipment, avoid long-time stoop down and use terminal equipment and cause the injury to user's glasses to and avoid the walking of stoop down and use terminal equipment, stop the potential safety hazard.

It should be noted that although the various steps of the methods of the present disclosure are depicted in the drawings in a particular order, this does not require or imply that these steps must be performed in this particular order, or that all of the depicted steps must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions, etc.

The exemplary embodiment of the present disclosure also provides a user motion detection apparatus 100, as shown in fig. 10, the user motion detection apparatus includes:

a receiving module 101, configured to receive first data, where the first data signal is sent by an earphone according to a head state of a user;

a first determining module 102, configured to determine whether the user is in a heads-down state according to the first data;

the obtaining module 103 is configured to obtain second data when the user is in a head-down state, where the second data includes an inclination angle of the terminal device;

a second determining module 104, configured to determine a current user action according to the second data, where the current user action includes a underhead-used terminal device and a non-underhead-used terminal device.

The embodiment of the disclosure provides a user action detection device, which determines whether a user is in a head-down state or not through first data, acquires second data when the user is in the head-down state, and determines whether the current user is in a state of using terminal equipment in a head-down state or not through the second data, thereby realizing detection of using the terminal equipment in a head-down state. And then can remind according to the state that the user used terminal equipment to hang down, avoid long-time hanging down to use terminal equipment to cause the injury to user's glasses to and avoid hanging down to walk and use terminal equipment, stop the potential safety hazard.

According to an embodiment of the present disclosure, the receiving module may include:

the first acquisition unit is used for acquiring the height data of the earphone, and the height data is the distance between the earphone and the first reference surface.

According to an embodiment of the present disclosure, the first determining module may include:

the first determining unit is used for determining that the user is in a head-down state when the difference value of the first height and the second height is larger than a preset height value and the duration of the second height is larger than a preset threshold value, wherein the first height is the height value of the earphone at a first moment, the second height is the height value of the earphone at a second moment, and the second moment and the first moment are separated by a preset time.

According to an embodiment of the present disclosure, the first obtaining unit may include:

and the first acquisition subunit is used for acquiring the air pressure data detected by the earphone and determining the height data according to the air pressure data.

and the second acquisition unit is used for acquiring inclination angle data of the earphone, and the cutting angle data of the earphone is the inclination angle of the earphone and the second reference surface.

According to an embodiment of the present disclosure, the first determining module includes:

and the second determining unit is used for determining that the user is in a head-lowering state when the difference value between the first inclination angle and the second inclination angle is larger than a preset angle value and the duration time is larger than a preset threshold value, wherein the first inclination angle is the inclination angle of the earphone at the first moment, the second inclination angle is the inclination angle of the earphone at the second moment, and the second moment and the first moment are separated by preset time.

and the third acquisition unit is used for acquiring a head lowering signal, wherein the head lowering signal is a signal sent when the earphone detects that the user lowers the head.

and the third determining unit is used for determining that the user is in a head-down state when the head-down signal is received.

According to an embodiment of the present disclosure, the second data further includes a step number of the user and/or a heart rate of the user, and the acquiring module may include:

and the fourth acquisition unit is used for acquiring the inclination angle of the terminal equipment when the user heads down, and acquiring the step number of the user and/or the heart rate of the user.

According to an embodiment of the present disclosure, the second determining module may include:

and the fourth determining unit is used for determining that the user lowers the head to use the terminal equipment when the inclination angle of the terminal equipment is in the preset angle range.

According to an embodiment of the present disclosure, the user motion detection apparatus may further include:

and the first reminding module is used for outputting a first reminding signal, and the first reminding signal is used for reminding a user that the user is in a head-down use terminal equipment state at present.

According to an embodiment of the present disclosure, the second determining module may further include:

a fifth determining unit for determining that the user is using the terminal device with the user's head down motion when the user is using the terminal device with the head down and the number of steps of the user and/or the heart rate of the user is increased.

the time acquisition module is used for acquiring the time of using the terminal equipment when a user moves down;

and the second reminding module is used for outputting a second reminding signal when the time for the user to use the terminal equipment in the head lowering movement is greater than a preset time threshold value, and the second reminding signal is used for reminding the user of being in the state of using the terminal equipment in the head lowering movement.

The embodiment of the disclosure provides a user action detection device, which determines whether a user is in a head-down state or not through first data, acquires second data when the user is in the head-down state, and determines whether the current user is in a state of using terminal equipment in a head-down state or not through the second data, thereby realizing detection of using the terminal equipment in a head-down state. And then can remind according to the state that the user stoop down and used terminal equipment, avoid long-time stoop down and use terminal equipment and cause the injury to user's glasses to and avoid the walking of stoop down and use terminal equipment, stop the potential safety hazard.

The specific details of the user motion detection apparatus have been described in detail in the corresponding user motion detection method, and therefore are not described herein again.

It should be noted that although in the above detailed description several modules or units of the user action detection means are mentioned, this division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.

In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided. The electronic device can be a mobile phone, a tablet computer and other terminal devices, or can also be a watch and other wearable electronic devices. The method can be executed in the terminal device and the wearable device.

As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Accordingly, various aspects of the present invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.

An electronic device 1100 according to such an embodiment of the invention is described below with reference to fig. 11. The electronic device 1100 shown in fig. 11 is only an example and should not bring any limitations to the function and the scope of use of the embodiments of the present invention.

As shown in fig. 11, the electronic device 1100 is in the form of a general purpose computing device. The components of the electronic device 1100 may include, but are not limited to: the at least one processing unit 1110, the at least one memory unit 1120, a bus 1130 connecting different system components (including the memory unit 1120 and the processing unit 1110), and a display unit 1140.

Wherein the storage unit stores program code that is executable by the processing unit 1110 to cause the processing unit 1110 to perform steps according to various exemplary embodiments of the present invention as described in the above section "exemplary methods" of the present specification.

The storage unit 1120 may include a readable medium in the form of a volatile memory unit, such as a random access memory unit (RAM) 11201 and/or a cache memory unit 11202, and may further include a read only memory unit (ROM) 11203.

The memory unit 1120 may also include a program/utility 11204 having a set (at least one) of program modules 11205, such program modules 11205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.

Bus 1130 may be representative of one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

The electronic device 1100 may also communicate with one or more external devices 1170 (e.g., keyboard, pointing device, bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 1100, and/or any devices (e.g., router, modem, etc.) that enable the electronic device 1100 to communicate with one or more other computing devices. Such communication may occur via an input/output (I/O) interface 1150. Also, the electronic device 1100 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the internet) via the network adapter 1160. As shown, the network adapter 1140 communicates with the other modules of the electronic device 1100 via the bus 1130. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 1100, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

According to the electronic equipment provided by the embodiment of the disclosure, whether the user is in the head lowering state is determined through the first data, the second data is obtained when the user is in the head lowering state, and whether the current user is in the state of using the terminal equipment by lowering the head is determined through the second data, so that the detection of using the terminal equipment by lowering the head of the user is realized. And then can remind according to the state that the user used terminal equipment to hang down, avoid long-time hanging down to use terminal equipment to cause the injury to user's glasses to and avoid hanging down to walk and use terminal equipment, stop the potential safety hazard.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

In an exemplary embodiment of the present disclosure, there is also provided a computer readable storage medium having stored thereon a program product capable of implementing the above-described method of the present specification. In some possible embodiments, aspects of the invention may also be implemented in the form of a program product comprising program code means for causing a terminal device to carry out the steps according to various exemplary embodiments of the invention described in the above section "exemplary method" of this description, when said program product is run on the terminal device.

Referring to fig. 12, a program product 1200 for implementing the above method according to an embodiment of the present invention is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited in this respect, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).

Furthermore, the above-described drawings are only schematic illustrations of processes involved in methods according to exemplary embodiments of the invention, and are not intended to be limiting. It will be readily appreciated that the processes illustrated in the above figures are not intended to indicate or limit the temporal order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.

The computer-readable storage medium provided by the embodiment of the disclosure determines whether a user is in a head-down state through the first data, acquires the second data when the user is in the head-down state, and determines whether the current user is in a state of using the terminal device in a head-down state through the second data, thereby realizing detection of using the terminal device in a head-down state. And then can remind according to the state that the user used terminal equipment to hang down, avoid long-time hanging down to use terminal equipment to cause the injury to user's glasses to and avoid hanging down to walk and use terminal equipment, stop the potential safety hazard.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice in the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is to be limited only by the terms of the appended claims.

Claims

1. A method for detecting user actions, the method comprising:

when a user is in a head-down state, acquiring second data, wherein the second data comprises an inclination angle of the terminal equipment;

and determining the current user action according to the second data, wherein the current user action comprises the underhead used terminal equipment and the non-underhead used terminal equipment.

2. The user action detection method of claim 1, wherein the receiving first data comprises:

and acquiring height data of the earphone, wherein the height data is the distance between the earphone and the first reference surface.

3. The user action detection method of claim 2, wherein determining whether the user is in a heads-down state based on the first data comprises:

when the difference value of the first height and the second height is larger than a preset height value and the duration time of the second height is larger than a preset threshold value, it is determined that the user is in a head-down state, the first height is the height value of the earphone at a first moment, the second height is the height value of the earphone at a second moment, and the second moment and the first moment are separated by a preset time.

4. The user action detection method of claim 2, wherein obtaining height data for a headset comprises:

and acquiring air pressure data detected by the earphone, and determining the height data according to the air pressure data.

5. The user action detection method of claim 1, wherein the receiving first data comprises:

and acquiring inclination angle data of the earphone, wherein the inclination angle data of the earphone is the inclination angle of the earphone and the second reference surface.

6. The method for user action detection of claim 5 wherein determining whether the user is in a heads-down state based on the first data comprises:

when the difference between the first inclination angle and the second inclination angle is greater than a preset angle value and the duration is greater than a preset threshold value, it is determined that the user is in a head-lowering state, the first inclination angle is the inclination angle of the earphone at the first moment, the second inclination angle is the inclination angle of the earphone at the second moment, and the second moment and the first moment interval are preset for time.

7. The user action detection method of claim 1, wherein the receiving first data comprises:

and acquiring a head lowering signal, wherein the head lowering signal is a signal sent when the earphone detects that the user lowers the head.

8. The user action detection method of claim 7, wherein determining whether the user is in a heads-down state based on the first data comprises:

and when the head lowering signal is acquired, determining that the user is in a head lowering state.

9. The user action detection method of claim 1, wherein the second data further comprises a user's step count and/or a user's heart rate, the obtaining second data while the user is in a heads-down state comprising:

when the user is in a head-down state, the inclination angle of the terminal device is obtained, and the step number of the user and/or the heart rate of the user are/is obtained.

10. The method for detecting user actions according to claim 9, wherein the determining a current user action based on the second data comprises:

and when the inclination angle of the terminal equipment is in a preset angle range, determining that the user lowers the head to use the terminal equipment.

11. The user action detection method of claim 10, wherein the method further comprises:

when the fact that the time that the user lowers head to use the terminal equipment is larger than a first time threshold value is detected, a first reminding signal is output and used for reminding the user that the user is in the state that the user lowers head to use the terminal equipment at present.

12. The method for detecting user actions according to claim 10, wherein said user lowering the head of the user using the terminal device comprises lowering the head of the user using the terminal device, and said determining the current user action based on said second data further comprises:

when the user lowers his head to use the terminal device and the number of steps of the user and/or the heart rate of the user increases, it is determined that the user lowers his head to use the terminal device.

13. The user action detection method of claim 12, wherein the method further comprises:

and when the time for using the terminal equipment by the user in the head lowering motion is greater than a second time threshold value, outputting a second reminding signal, wherein the second reminding signal is used for reminding the user of using the terminal equipment in the head lowering motion.

14. A user motion detection apparatus, characterized in that the user motion detection apparatus comprises:

the first determining module is used for determining whether the user is in a head-down state according to the first data;

15. An electronic device, comprising

A processor; and

a memory having computer readable instructions stored thereon which, when executed by the processor, implement the method of any of claims 1 to 13.

16. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 13.