WO2015158258A1 - User posture detection method, device and system - Google Patents

User posture detection method, device and system Download PDF

Info

Publication number
WO2015158258A1
WO2015158258A1 PCT/CN2015/076622 CN2015076622W WO2015158258A1 WO 2015158258 A1 WO2015158258 A1 WO 2015158258A1 CN 2015076622 W CN2015076622 W CN 2015076622W WO 2015158258 A1 WO2015158258 A1 WO 2015158258A1
Authority
WO
WIPO (PCT)
Prior art keywords
attitude
posture
user
module
data
Prior art date
Application number
PCT/CN2015/076622
Other languages
French (fr)
Chinese (zh)
Inventor
王立宁
Original Assignee
苏州尚德智产通信技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN201410153791.1A external-priority patent/CN103927012B/en
Priority claimed from CN201410153807.9A external-priority patent/CN103955272B/en
Priority claimed from CN201410153783.7A external-priority patent/CN103927011B/en
Priority claimed from CN201410153795.XA external-priority patent/CN103927250B/en
Priority claimed from CN201410153857.7A external-priority patent/CN103955273B/en
Priority claimed from CN201410153793.0A external-priority patent/CN103955271B/en
Priority claimed from CN201410153781.8A external-priority patent/CN103927010B/en
Application filed by 苏州尚德智产通信技术有限公司 filed Critical 苏州尚德智产通信技术有限公司
Publication of WO2015158258A1 publication Critical patent/WO2015158258A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer

Definitions

  • the application is submitted to the Chinese Patent Office on April 16, 2014.
  • the application numbers are 201410153795.X, 201410153807.9, 201410153783.7, 201410153857.7, 201410153793.0, 201410153791.1, 201410153781.8, respectively.
  • the invention names are respectively “a terminal device user attitude detection method”.
  • a terminal device user attitude detection system "operating system running device and application module for realizing user posture detection”
  • a mobile terminal and method for realizing user posture detection through an operating system” "one through an operating system Chinese patent application for realizing user posture detection and method, "a mobile terminal and method for realizing user attitude detection by operating system”
  • a mobile terminal and method for realizing user attitude detection by monitoring program The entire contents are hereby incorporated by reference.
  • the present invention relates to the field of terminal equipment technologies, and in particular, to a user attitude detection method, apparatus and system.
  • Mobile devices such as mobile phones, tablets, e-books, and notebook computers have become very popular in modern life. When using these devices, users generally hold these devices with one hand or both hands, so that the device's display screen is aimed at learning, reading, and entertainment. With the advent of a large number of applications, users are using these devices for longer and longer. For example, some users use the mobile phone to read novels on the subway, and the continuous use time is usually several tens of minutes; watching the video on a tablet or mobile phone is often a few hours. With the opening of 4G services, the mobile Internet will provide more colorful content, which will allow us to use these terminal devices for a longer period of time.
  • the embodiment of the invention provides a video processing method and device, which are used to solve the problem that the user in the prior art faces the incorrect posture caused by the user facing the display device.
  • the method for detecting the absolute attitude parameter of the terminal device is: the attitude sensor collects the spatial state of the display screen of the terminal to obtain the device attitude sensing data, and calculates the absolute attitude parameter of the terminal device;
  • the method for detecting the relative posture parameter of the user is: collecting a user image or a video by using a camera, identifying a feature point of the user's face from the image signal, and calculating a relative posture parameter of the user;
  • the user absolute posture parameter is a combination of the user relative posture parameter and the device absolute posture parameter
  • a prompt signal and/or a correction suggestion signal is issued when at least one of the attitude parameters exceeds a threshold.
  • the method for detecting the absolute attitude parameter of the terminal device is: the attitude sensor collects the spatial state of the display screen of the terminal to obtain the device attitude sensing data, and calculates the absolute attitude parameter of the terminal device;
  • the method for detecting the relative posture parameter of the user is: collecting a user image or a video by using a camera, identifying a feature point of the user's face from the image signal, and calculating a relative posture parameter of the user;
  • the user absolute posture parameter is a combination of the user relative posture parameter and the device absolute posture parameter
  • the above steps are performed cyclically, and when at least one of the attitude parameters repeatedly exceeds a threshold value for a plurality of times, a prompt signal and/or a correction suggestion signal is issued.
  • One or more user-facing cameras that obtain user facial images or videos
  • One or more attitude sensors obtain an absolute attitude parameter of the terminal device, and the absolute attitude parameter of the terminal device reflects a horizontal inclination of a plane where the screen is located;
  • An information processing module includes an attitude sensing data interface, an image data interface, and a gesture reminding interface
  • a gesture reminding module comprising an information output device
  • the information processing module obtains the absolute posture parameter of the terminal device by using the attitude sensing data interface
  • the information processing module obtains data of the user's facial image or video through the image data interface
  • the information processing module calculates the user posture parameter by using the terminal device absolute posture parameter and the data of the user facial image or video, and determines the posture, and sends a signal through the posture reminding interface when the posture is incorrect, triggering the Gesture reminder module;
  • the gesture reminder module sends a sensory identifiable signal to the user, including a prompt signal and/or a correction suggestion signal.
  • the operating system running device for implementing user posture detection includes: a sensing module driving unit, an input module driving unit, an output module driving unit, a posture service module, an application database, and an application programming interface;
  • the sensing module driving unit outputs a sensing control signal and an input sensing signal
  • the input module driving unit outputs a configuration control signal and receives a user instruction
  • the output module driving unit outputs an alarm control signal and an output signal including alarm data and/or attitude correction data
  • the attitude service module includes an attitude detecting unit, a posture comparing unit, and a posture reminding unit;
  • the posture detecting unit activates the sensing module driving unit, reads posture data output by the sensing module driving unit, recognizes a position of the user's face, and generates a posture parameter; the posture comparing unit sets the posture parameter and the corresponding door The limit value is compared; the posture reminding unit generates alarm data and/or attitude correction data when the user posture is incorrect, and starts the output module driving unit;
  • the application database stores work data and default settings data.
  • the operating system running device for implementing user posture detection comprises: a sensing module driving unit, an input module driving unit, an output module driving unit, a posture service module, an application database, and an application programming interface;
  • the sensing module driving unit outputs a sensing control signal and an input sensing signal
  • the input module driving unit outputs a configuration control signal and receives a user instruction
  • the output module driving unit outputs an alarm control signal and an output signal including alarm data and/or attitude correction data
  • the attitude service module includes an attitude detection unit and a posture comparison unit; the attitude detection unit activates the sensing module driving unit, reads posture data output by the sensing module driving unit, and identifies a position of the user's face. Positioning, generating a posture parameter; the posture comparison unit compares the attitude parameter with a corresponding threshold value; and when the posture parameter exceeds the threshold value, sends an attitude overrun notification through the application programming interface;
  • the application database stores work data and default settings data.
  • the operating system running device for implementing user posture detection comprises: a sensing module driving unit, an input module driving unit, an output module driving unit, a posture service module, an application database, and an application programming interface;
  • the sensing module driving unit outputs a sensing control signal and an input sensing signal
  • the input module driving unit outputs a configuration control signal and receives a user instruction
  • the output module driving unit outputs an alarm control signal and an output signal including alarm data and/or attitude correction data
  • the posture service module includes an attitude detecting unit, the posture detecting unit starts the sensing module driving unit, reads posture data output by the sensing module driving unit, recognizes a position of a user's face, and generates a posture parameter, and The pose parameter is saved in an application database;
  • the application database stores work data and default settings data.
  • An application program module is provided for the second embodiment of the operating system running device for detecting user posture, comprising: a posture reminding unit and an input and output processing unit;
  • the gesture reminding unit generates alarm data and/or attitude correction data
  • the input/output processing unit reads or sets working data in an application database through an application programming interface
  • the input/output processing unit receives an attitude overrun notification from an operating system running device through an application programming interface
  • the input/output processing unit activates an input module driving unit through an application programming interface to receive a user instruction
  • the input/output processing unit activates the output module drive unit through an application programming interface to issue a signal containing alarm data and/or attitude correction data.
  • An application program module is provided for the third operating system running device for realizing user posture detection, comprising: a posture comparison unit, a posture reminding unit, and an input/output processing unit;
  • the input/output processing unit sets or reads working data in an application database through the application programming interface
  • the attitude comparison unit compares the attitude parameter with a corresponding threshold value
  • the gesture reminding unit generates alarm data and/or attitude correction data
  • the input/output processing unit activates an input module driving unit through an application programming interface to receive a user instruction
  • the input/output processing unit activates the output module drive unit through an application programming interface to issue a signal containing alarm data and/or attitude correction data.
  • a mobile terminal for implementing user posture detection by using an operating system includes an attitude sensing module, a user configuration input module, an alarm signal output module, an operating system running device, a display screen, and one or more application modules. ;
  • the attitude sensing module generates an attitude sensing signal
  • the user configuration input module identifies a user operation and transmits the user instruction to an operating system running device
  • the alarm signal output module sends a signal recognizable by the user's senses
  • the operating system running device includes a sensing module driving unit, an input module driving unit, and an output module driving unit; the sensing module driving unit controls the operation of the attitude sensing module, and outputs the attitude sensing module
  • the attitude sensing signal is converted into attitude data;
  • the input module driving unit controls the user configuration input module to operate, and receives a user instruction;
  • the output module driving unit outputs a signal including alarm data and/or attitude correction data, and controls the The alarm signal output module works;
  • the operating system running device further includes a posture service module, wherein the posture service module includes an attitude detecting unit, a posture comparing unit, and a posture reminding unit; the posture detecting unit starts the sensing module driving unit, and reads the The attitude data output by the sensing module driving unit identifies the position of the user's face to generate a posture parameter; the attitude comparison unit compares the attitude parameter with a corresponding threshold value; the gesture reminding unit generates alarm data and / or posture correction data;
  • the operating system running device further includes an application database, and the application database stores working data and default setting data.
  • the attitude detecting unit in the operating system running device starts the sensing module driving unit to collect the sensing signal and converts into the attitude data;
  • the attitude detecting unit in the operating system running device reads the attitude data output by the sensing module driving unit;
  • the attitude detecting unit in the operating system running device analyzes the attitude data to obtain a relative attitude parameter and/or an absolute posture parameter;
  • the attitude comparison unit in the operating system running device compares the posture parameter and the user posture threshold value
  • the gesture reminding unit in the operating system running device generates the alarm data when the attitude parameter exceeds the threshold And/or posture correction data.
  • a mobile terminal for implementing user posture detection by using an operating system includes an attitude sensing module, a user configuration input module, an alarm signal output module, an operating system running device, a display screen, and one or more application modules. ;
  • the attitude sensing module generates an attitude sensing signal
  • the user configuration input module identifies a user operation, and transmits the user instruction to the operating system running device and the at least one application module;
  • the alarm signal output module sends a signal recognizable by the user's senses
  • the operating system running device includes a sensing module driving unit, an input module driving unit, and an output module driving unit; the sensing module driving unit controls the operation of the attitude sensing module, and outputs the attitude sensing module
  • the attitude sensing signal is converted into attitude data;
  • the input module driving unit controls the user configuration input module to operate, and receives a user instruction;
  • the output module driving unit outputs a signal including alarm data and/or attitude correction data, and controls the The alarm signal output module works;
  • the operating system running device further includes a posture service module, wherein the attitude service module includes an attitude detecting unit and a posture comparing unit; the posture detecting unit starts the sensing module driving unit, and reads the sensing module driving The attitude data output by the unit, the position of the user's face is recognized, and the attitude parameter is generated; the attitude comparison unit compares the attitude parameter with a corresponding threshold value; and when the attitude parameter exceeds the threshold value, at least one of the applications The module issues a gesture of over-limitation;
  • the operating system running device includes an application programming interface
  • the operating system running device further includes an application database, where the application database stores working data and default setting data;
  • At least one of the application modules includes a gesture reminding unit, an input and output processing unit; the gesture reminding unit generates alarm data and/or attitude correction data; the input/output processing unit sets an application through the application programming interface The working data in the database, receiving the attitude overrun notification from the operating system running device, starting the input module driving unit, and starting the output module driving unit.
  • the attitude detecting unit in the operating system running device starts the sensing module driving unit to collect the sensing signal and converts into the attitude data;
  • the attitude detecting unit in the operating system running device reads the attitude data output by the sensing module driving unit;
  • the attitude detecting unit in the operating system running device analyzes the attitude data to obtain a relative attitude parameter and/or an absolute posture parameter;
  • the attitude comparison unit in the operating system running device compares the attitude parameter and the threshold value; when the posture parameter exceeds the threshold value, sends an attitude overrun notification to the at least one application module through the application programming interface;
  • the input/output processing unit in the application module occupying the display screen receives the gesture overrun notification through the application programming interface;
  • the gesture reminding unit in the application module occupying the display generates alarm data and/or attitude correction data when the attitude parameter exceeds a threshold.
  • a mobile terminal for implementing user posture detection by using an operating system includes an attitude sensing module, a user configuration input module, an alarm signal output module, an operating system running device, a display screen, and one or more application modules. ;
  • the attitude sensing module generates an attitude sensing signal
  • the user configuration input module identifies a user operation, and transmits user instructions to the operating system running device and at least one of the application modules;
  • the alarm signal output module sends a signal recognizable by the user's senses
  • the operating system running device includes a sensing module driving unit, an input module driving unit, and an output module driving unit; the sensing module driving unit controls the operation of the attitude sensing module, and outputs the attitude sensing module
  • the attitude sensing signal is converted into attitude data;
  • the input module driving unit controls the user configuration input module to operate, and receives a user instruction;
  • the output module driving unit outputs a signal including alarm data and/or attitude correction data, and controls the The alarm signal output module works;
  • the operating system running device further includes a posture service module, wherein the attitude service module includes an attitude detecting unit; the posture detecting unit starts the sensing module driving unit, and reads a posture of the sensing unit driving unit Data, identifying the location of the user's face, and generating gesture parameters;
  • the operating system running device includes an application programming interface
  • At least one of the application modules includes a posture comparison unit, a posture reminding unit, and an input/output processing unit; the posture comparison unit compares the posture parameter with a corresponding threshold value; and the posture reminding unit generates an alarm Data and/or posture correction data; the input/output processing unit reads or sets working data in the application database, starts the input module driving unit, and starts the output module driving unit through the application programming interface;
  • the operating system running device further includes an application database, and the application database stores data generated in the work and default setting data.
  • the attitude detecting unit in the operating system running device starts the sensing module driving unit to collect the sensing signal and converts into the attitude data;
  • the attitude detecting unit in the operating system running device reads the attitude data output by the sensing module driving unit;
  • the attitude detecting unit in the operating system running device analyzes the attitude data to obtain a relative attitude parameter and/or an absolute posture parameter;
  • the attitude detecting unit in the operating system running device saves the posture parameter in the application database
  • the input and output processing unit in the application module reads the attitude parameter and the threshold from the application database through the application programming interface;
  • An attitude comparison unit in the application module compares the attitude parameter with a threshold value
  • the gesture reminding unit in the application module generates alarm data and/or attitude correction data when the attitude parameter exceeds a threshold.
  • a mobile terminal for implementing user posture detection by using a monitoring program includes an attitude sensing module, a user configuration input module, an alarm signal output module, one or more application modules, a state database, and a monitoring program module.
  • the attitude sensing module generates attitude data
  • the user configuration input module identifies a user operation and generates a user instruction
  • the alarm signal output module sends a signal recognizable by the user's senses
  • the monitoring program module includes an input and output processing unit, an attitude processing unit, and a status scanning unit;
  • the input/output processing unit is connected to the attitude sensing module, the user configuration input module, and the alarm signal output module, and receives the posture data, receives the user instruction, and generates output data;
  • the posture processing unit identifies a position of the user's face according to the posture data, generates a posture parameter, compares the posture parameter with a corresponding threshold value, and generates alarm data and/or posture correction data when the user posture is incorrect;
  • the state scanning unit periodically scans the state database, and starts the gesture processing unit according to the state data in the state database.
  • a mobile terminal for realizing user posture detection by using a monitoring program comprising an attitude sensing module, a user configuration input module, an alarm signal output module, one or more application modules, a state database, a processing unit library, Monitoring program module;
  • the attitude sensing module generates attitude data
  • the user configuration input module identifies a user operation and generates a user instruction
  • the alarm signal output module sends a signal recognizable by the user's senses
  • the processing unit library includes an attitude processing unit and an application programming interface
  • the posture processing unit identifies a position of the user's face according to the posture data, generates a posture parameter, compares the posture parameter with a corresponding threshold value, and generates alarm data and/or posture correction data when the user posture is incorrect;
  • the application programming interface enables the gesture processing unit to receive the enable signal and data and output the result
  • the monitoring program module includes an input and output processing unit, a gesture processing calling unit, and a state scanning unit;
  • the input/output processing unit is connected to the attitude sensing module, the user configuration input module, and the alarm signal output module, and receives the posture data, receives the user instruction, and generates output data;
  • the gesture processing calling unit is capable of querying a position of the gesture processing unit in the processing unit library, issuing a start signal, and transmitting attitude data and a threshold value required for posture detection and comparison;
  • the status scanning module periodically scans the status database and initiates an attitude processing unit based on the status data in the status database.
  • a state scanning unit in the monitoring program module periodically scanning the state database, and starting the gesture processing unit according to the state information in the state database;
  • the input/output processing unit in the monitoring program module controls the attitude sensing module to collect the user posture and receive the attitude data; the attitude processing unit analyzes the attitude data to obtain a relative attitude parameter and/or an absolute posture parameter;
  • the attitude processing unit in the monitor module compares the user attitude parameter with the corresponding threshold value
  • the attitude processing unit in the monitoring program module generates alarm data and/or attitude correction data when the user posture is incorrect; the input and output processing unit output signal triggers the alarm signal output module to send a signal to the customer that the user feels identifiable, including the alarm information and / or correction suggestions.
  • a monitoring program module provided by an embodiment of the present invention includes an input/output processing unit, a posture processing unit, and a state scanning unit;
  • the input/output processing unit is connected to the attitude sensing module, the user configuration input module, and the alarm signal output module in the mobile terminal, and receives the posture data generated by the attitude sensing module, and receives the user generated by the user configuration input module. Command and generate output data;
  • the posture processing unit identifies a position of the user's face according to the posture data, generates a posture parameter, compares the posture parameter with a corresponding threshold value, and generates alarm data and/or posture correction data when the user posture is incorrect;
  • the state scanning unit periodically scans the state database in the mobile terminal, and starts the gesture processing unit according to the state data in the state database.
  • a monitoring program module provided by an embodiment of the present invention includes an input/output processing unit, a gesture processing calling unit, and a state scanning unit;
  • the input/output processing unit is connected to the attitude sensing module, the user configuration input module, and the alarm signal output module in the mobile terminal, and receives the posture data generated by the attitude sensing module, and receives the user generated by the user configuration input module. Command and generate output data;
  • the gesture processing calling unit is capable of querying a position of the gesture processing unit in the processing unit library of the mobile terminal, issuing a start signal, and transmitting posture data and a threshold value required for posture detection and comparison;
  • the state scanning module periodically scans the state database in the mobile terminal, and starts the gesture processing unit according to the state data in the state database, so that the gesture processing unit identifies the location of the user's face according to the posture data, and generates a posture parameter, which will The attitude parameter is compared with a corresponding threshold value, and alarm data and/or attitude correction data is generated when the user posture is incorrect.
  • the embodiment of the invention provides a method, a device and a system for detecting a user's posture, which can enable the user to maintain a relatively correct posture by effectively reminding the user. This can be beneficial to the user because the user does not harm the body by using these electronic devices. On the other hand, maintaining the correct posture, users avoid fatigue, can use these devices for a long time, and improve the user's adhesion to equipment producers and applications.
  • Figure 1 is a terminal device user attitude detection system
  • FIG. 2 is a flowchart of an embodiment of a method for detecting a posture of a terminal device user
  • FIG. 3 is a flowchart of another embodiment of a terminal device user attitude detecting method
  • FIG. 4 is a schematic diagram of the position and function of the camera on the panel of the terminal device
  • Figure 5 is a schematic diagram of the position and function of the attitude sensor on the terminal device
  • Figure 6 is a schematic diagram of the acquisition of the face position of the monocular machine vision method
  • FIG. 7 is a schematic diagram of acquisition of a face position of a feature value extraction method
  • Figure 8 is a flow diagram of an embodiment using distance and angle parameters
  • Figure 9 is a flow chart of an embodiment using area and angle parameters
  • Figure 10 is an embodiment of a terminal device user attitude detecting system
  • 11 is an embodiment of a terminal device attitude detecting system for wired connection telematics
  • 12 is an embodiment of a terminal device attitude detection system for wirelessly connecting telematics
  • 13 is an embodiment of a terminal device attitude detection system for data transmission network connection telematics
  • 14 is a mobile terminal that implements user gesture detection by an operating system
  • Figure 15 is a view of the attitude sensing module using a camera
  • Figure 16 is a gesture sensing module using a camera and tilt sensor scheme
  • 17 is a working process of an operating system running device and an application module for realizing user posture detection
  • 18 is an embodiment of an operating system running device implementing attitude detection, attitude comparison, and attitude reminding
  • 20 is an embodiment of an operating system operating device implementing attitude detection and attitude comparison
  • 21 is a flowchart of a scheme based on the automatic operation posture detection, posture comparison, and posture reminding of the operating system running device 4 of FIG. 18;
  • FIG. 22 is a flow chart of a scheme for operating an attitude detection, a posture comparison, and a gesture reminder according to the needs of an application module of the operating system operating device of FIG. 18;
  • FIG. 23 is a flowchart of a scenario in which an application module of FIG. 18 uses a flag bit to start and shut down an attitude service module in an operating system operating device;
  • Figure 24 is an embodiment based on Figure 19;
  • Figure 25 is an embodiment based on Figure 19;
  • 26 is a flowchart of a scheme for starting an attitude detection according to an application state table by the operating system operating device of FIG. 19;
  • FIG. 27 is a flowchart of a method for issuing a gesture overrun notification completion reminder to an application module based on the operating system running device of FIG. 20;
  • 29 is a flowchart of a scheme based on the operating system running device of FIG. 20 initiating attitude detection according to an application registry and issuing a gesture overrun notification;
  • 30 is a mobile terminal that implements user gesture detection by a monitoring program module
  • Figure 31 is a view of the attitude sensing module using a camera
  • 32 is a gesture sensing module using a camera and a tilt sensor scheme
  • 33 is a schematic diagram of a scheme of a monitoring program module calling gesture processing unit
  • Figure 34 is a flow chart of a method for implementing user gesture detection by the monitoring program module 4';
  • Figure 35 is a flow chart showing the monitoring program module 4' scanning state database 8 obtaining application instructions and performing gesture processing;
  • Fig. 36 is a flow chart showing the state in which the monitoring program module 4' scans the state database 8 to obtain the application state and performs the posture processing.
  • the terminal device user attitude detection system includes: one or more user-oriented cameras 100, one or more posture sensors 200, an information processing module 300, and a gesture reminding module 400.
  • a user's face image or video is obtained by the camera 100.
  • the terminal device absolute attitude parameter (reflecting the horizontal tilt of the plane in which the display is located) is obtained by one or more attitude sensors on the device (eg using a gravity sensor).
  • the information processing module 300 includes an attitude sensing data interface a, an image data interface b, and a gesture reminding interface c.
  • the data transmitted by the attitude sensing data interface a includes terminal device attitude parameters.
  • the information processing module 300 calculates a user posture parameter, determines the posture, and triggers the posture reminding module 400 when the posture is incorrect.
  • the data passed by the image data interface b contains image or video data.
  • the data transmitted by the gesture reminder interface c includes alarm data and attitude correction data.
  • the alarm data is generated when the user's attitude parameter exceeds the threshold, including the mode of the alarm, the strength of the alarm, and the length of the alarm. These data are determined according to the degree and time when the user deviates from the correct posture.
  • the attitude correction data is based on the difference between the user's posture parameters and the threshold value, and how to change the user's posture to achieve the correct posture data, including distance, angle, and direction of adjustment.
  • the gesture reminder module 400 includes an information output device (one or more of a cue light, a display screen, a vibration device, and an audio device).
  • the signal output by the gesture reminder module 400 is a signal recognizable by the user's senses.
  • the gesture reminding module 400 sends a prompt signal and/or a correction suggestion signal to the user by one or more of the following forms: a prompt light switch, a color change, a flashing; a text, a graphic, an image or a video displayed on the display; the audio device emits a sound ; vibration from the vibration device.
  • the “attitude parameters” described above include “relative attitude parameters” and "absolute attitude parameters”.
  • the relative attitude parameter that is, the “user relative attitude parameter” refers to the position and angle parameters of the user's face relative to the display screen; the so-called absolute attitude parameter refers to the parameter that reflects the object's attitude in space, independent of the relative position, and the absolute attitude parameter.
  • Including “terminal device absolute attitude parameters”, such as the angle t1 between the display screen and the horizontal plane and also includes “user absolute attitude parameters”, such as the angle t4 between the plane of the feature points of the user's face and the horizontal plane.
  • FIG. 2 is a flowchart of an embodiment of a method for detecting a posture of a user of a terminal device, which is specifically described as follows:
  • Step 101 Display attitude detection.
  • the space state of the display screen of the terminal is collected to obtain the device attitude data, and the horizontal inclination of the plane where the display screen is located is calculated as the absolute attitude parameter of the terminal device.
  • Step 102 User posture detection. Collecting a user image, identifying a feature point of the user's face from the image signal, and calculating a relative posture parameter of the user;
  • Step 103 Posture comparison. Comparing at least one of the attitude parameters with a threshold of the parameter to determine whether at least one of the attitude parameters exceeds a threshold.
  • Step 104 Signal output. When at least one of the attitude parameters exceeds a threshold, a prompt signal is issued and / Or correct the suggestion signal.
  • step 103 does not have the attitude parameter beyond the threshold value, or after the completion of step 104, return to step 101.
  • FIG. 3 is a flowchart of another embodiment of a method for detecting a posture of a user of a terminal device, which is specifically described as follows:
  • Step 201 User posture detection: collecting a user image, identifying a feature point of the user's face from the image signal, and calculating a relative posture parameter of the user;
  • Step 202 Display attitude detection: collect the space state of the display screen of the terminal to obtain the device attitude data, and calculate the horizontal inclination of the plane where the display screen is located, as the absolute attitude parameter of the terminal device;
  • Step 203 Pose comparison: comparing at least one of the posture parameters with a threshold value of the parameter to determine whether at least one of the posture parameters exceeds a threshold value for a plurality of consecutive cycles.
  • Step 204 Signal output: when at least one of the attitude parameters exceeds a threshold for a plurality of consecutive cycles, a prompt signal and/or a correction suggestion signal is issued.
  • step 203 does not have the attitude parameter beyond the threshold value, or after step 204 is completed, return to step 201.
  • a prompt signal and/or a correction suggestion signal is issued when at least one of the attitude parameters repeatedly exceeds a threshold value for a plurality of consecutive times.
  • the embodiment further includes step 205: delay.
  • the direct regression is a case where the delay is zero.
  • the user relative posture parameter and the device absolute posture parameter may also be combined to calculate the user absolute posture parameter, which is used as the posture comparison (steps 103 and 203).
  • Gesture parameters may also be combined to calculate the user absolute posture parameter, which is used as the posture comparison (steps 103 and 203).
  • FIG. 4 is a schematic diagram of the position and function of the camera on the panel of the terminal device.
  • the terminal device 500 When the terminal device 500 is used, it is substantially flat and has one or more camera 100 facing the user. At least one user-oriented display screen 600 should be included.
  • the plane of the lens of the camera is parallel to the plane of the display screen, or has other stable positional relationships, so that the user faces the camera and is in the camera while reading or working on the device. In the image acquisition space.
  • FIG. 5 is a schematic diagram of the position and function of the attitude sensor on the terminal device.
  • the hardware of the terminal device 500 includes at least a display screen 600 facing the user and an attitude sensor 200 capable of detecting the plane inclination of the display screen.
  • the sensor is mounted on the terminal device and has a fixed positional relationship with the display. Through the data output by the sensor, the inclination angle of the plane of the display screen 600 of the terminal device (such as t1 in FIG. 5), that is, the angle between the plane where the display screen of the terminal device is located and the horizontal plane can be obtained as the absolute attitude parameter of the terminal device.
  • a parameter describing the posture of the user is: a horizontal distance L1 between the user's face reference point and the terminal display reference point, a vertical distance L2, a relative inclination angle t0 between the user's facial middle line and the display screen, and a user's facial middle line and The angle t4 between the horizontal planes.
  • L1, L2, and t0 reflect the relative positional relationship between the user's face and the display of the terminal, so it is called “user relative attitude parameter”, where t4 reflects the position parameter of the user's face and the terminal display, which is called “user”.
  • Absolute attitude parameter That is, the user relative attitude parameter is at least one of [L1, L2, t0].
  • the user absolute attitude parameter t4 t1+t0.
  • a parameter describing the posture of the user is: a range Af occupied by the user's face in the image data of the camera, an average distance of the middle line Cf of the face from the intermediate line Cc of the camera image is d, and an angle t0 between the middle line of the user's face and the horizontal plane Etc.
  • a relative pose parameter it is also possible to construct a relative pose parameter, and the user relative pose parameter is at least one of [Af, d, t0].
  • the user absolute attitude parameter t4 t1+t0.
  • the threshold value of the attitude parameter refers to a threshold value defined by the relative attitude parameter and the absolute attitude parameter for calibrating the correct posture, for example, the maximum value or the minimum value of L1, L2, Af, t1, t0, and t4.
  • the threshold value of the relative attitude parameter T1 [Min(L1), Max(L1), Min(L2), Max(L2), Min(t0), Max(t0) ]
  • T1 [Min(Af), Max(Af), Max(d), Min(t0), Max(t0)]
  • threshold value of absolute attitude parameter T2 [Min(t1), Max(t1 ), Min(t0), Max(t0)].
  • Incorrect user posture means that the user's attitude parameter exceeds the threshold.
  • FIG. 6 is a schematic diagram of the acquisition of the face position of the monocular machine vision method. Based on the principle of monocular machine vision, a method for acquiring the position of the face is exemplified herein. With the monocular detection method, there is a camera that takes two easily recognizable feature points A and B in the middle of the human face, and the center of the camera lens. Point C, the length AC, BC is obtained by the monocular machine vision method; the length AB is a typical value; and each attitude parameter [L1, L2, t0] is calculated. The details are as follows:
  • a and B are two easily distinguishable feature points of the human face, such as the midpoint of the two eyes and the midpoint of the mouth.
  • Point C is the center point of the camera lens on the front panel of the device.
  • the plane where DE is located is the plane where the camera sensor is located, and the two points D and E are the images of A and B on the sensor. Since the geometric relationship between the sensor and the lens is fixed, the angle t2, that is, the angle between BD and DE, can be calculated from the position of D and E on the sensor and the distance between the lens and the sensor.
  • the length of AC and BC can be obtained by a method of monocular machine vision, such as geometric similarity method and geometrical optical method.
  • the length of AB can be measured in advance, or some typical values can be used directly.
  • the cosine theorem can be used to calculate the angle t3, and the horizontal distance L1 and the vertical distance L2 of the C from A.
  • t4 t1+t3-t2 that is, the horizontal inclination of the head.
  • the interface displays a corresponding prompt message.
  • T1 80 degrees
  • T4 55 degrees
  • Lt1 23 Cm
  • Lt2 8 cm.
  • the length AB is typically obtained by a calibration process.
  • a calibration process can be added.
  • the user When calibrating, the user must use the correct posture to maintain a specific known attitude parameter value, such as the head straightness, and the mobile phone display screen is almost the same height as the eyes. Facing the eyes, the display is about 30 cm away from the eyes.
  • the camera analyzes the facial information at this time, and can obtain the characteristic information of the user, such as the geometric relationship of the points that are relatively easy to recognize, such as two eyes, a nose, and a lower jaw.
  • the typical value of the AB length used in the identification process described above can be obtained by the monocular machine vision method.
  • the calibration process of the head attitude measurement can be omitted.
  • Binocular or multi-mesh machine vision method for obtaining facial position The discussion of the above embodiment is based on a method of a camera. If there are two or more cameras, the cameras have parallax for images taken in the same scene, and by analyzing the parallax between the feature points, points A and B can be obtained. The distance between the two points L1 and L2 and the angle t3 can be calculated from the distance between the center point C of the camera (which can be defined as the center point of a certain camera lens) in space, and the angle t4 can be calculated. With the binocular or multi-eye detection method, there are two or more cameras, and the method of binocular and multi-eye machine vision does not require a calibration process.
  • the virtual multi-camera acquires the face position.
  • the optical imaging system can be combined with a single camera to form two or more virtual cameras that form a single-camera binocular or multi-view vision system for virtual stereo vision. This can solve the consistency requirements for multiple cameras in a general binocular or multi-view system.
  • the acquisition of the face position is performed using a 3D camera.
  • the 3D camera that has appeared now can directly output the stereoscopic image, and as long as the required feature points are extracted therefrom, as in points A and B in the previous embodiment, the distance information of L1 and L2 can be calculated and Angles t2 and t3, and angles t0 and t4 are calculated.
  • FIG. 7 is a schematic diagram of the acquisition of the face position of the feature value extraction method.
  • the range occupied by the user's face in the camera image data is Af, and the two feature points A and B which are easily recognized by the human facial line are obtained, and the middle line Cf of the face is obtained, and the middle line of the face is obtained.
  • the average distance of Cf from the camera image intermediate line Cc is d.
  • 7(a) shows a case where the range Af occupied by the user's face is excessively large in the image data of the camera;
  • FIG. 7(b) shows a case where the range Af occupied by the user's face is too small in the image data of the camera;
  • FIG. 7(c) shows the face.
  • the intermediate line Cf deviates from the middle of the camera image intermediate line Cc by a distance d;
  • FIG. 7(d) shows a case where the user's posture is normal.
  • Figure 8 is a flow chart of an embodiment using distance and angle parameters
  • Step 301 Display attitude detection.
  • the attitude sensor collects the spatial state of the terminal display and outputs the device attitude data.
  • the attitude calculation module calculates the device absolute attitude parameter t1 using the device attitude data output by the attitude sensor.
  • Step 302 User posture detection.
  • the camera captures the user image, extracts feature points of the user's face and face from the image, such as eyes, nose, mouth, etc., and analyzes the relative posture parameters and absolute posture parameters of the user's face.
  • the distance between the midpoint of the facial organs and the camera is measured, and the geometric relationship is further calculated, including the horizontal distance L1 of the eye and the display screen, the vertical distance L2, and the facial midline.
  • the relative inclination of the plane of the display of the terminal device (t2, t3, and t0).
  • the attitude calculation module calculates the horizontal inclination angle t4 of the line of the user's face based on the above angle values.
  • Step 303 User posture analysis.
  • the posture judging module compares at least one of the above posture parameters (L1, L2, t0, t1, t2, t3, t4) with a threshold value of the parameter to determine whether the user posture is correct.
  • the user posture is incorrect, which means that at least one of the posture parameters exceeds a range represented by the parameter threshold.
  • Step 304 Signal output.
  • the trigger gesture reminding module issues a prompt message.
  • step 301 is returned.
  • step 305 delay.
  • a delay of 0 indicates a direct regression.
  • the embodiment shown in FIG. 8 is further optimized.
  • at least one of the attitude parameters exceeds a threshold value during a loop formed by steps 301 to 304, and the parameters are exceeded for a plurality of cycles.
  • the limit value, the step 304 triggers the gesture reminding module to issue a prompt signal and/or a correction suggestion signal.
  • the user posture is incorrect, that is, at least one of the posture parameters exceeds a threshold value, and the average parameter exceeds the threshold value for a plurality of cycles.
  • Figure 9 is a flow chart of an embodiment using area and angle parameters
  • Step 401 Display attitude detection.
  • the attitude sensor collects the spatial state of the terminal display and outputs the device attitude data.
  • the attitude calculation module calculates the device absolute attitude parameter t1 using the device attitude data output by the attitude sensor.
  • Step 402 User posture detection.
  • the camera collects the user image, extracts the pixels of the user's head and face from the image, determines the user's face and head image A by feature analysis, further ranges Af occupied by the camera image data user's face standard outline As, and The degree d of the middle line of the face deviating from the middle line of the image, and the relative inclination angle t0 of the middle line of the face and the display screen.
  • Step 403 User posture analysis.
  • the attitude judgment module will at least have the above attitude parameters (Af, d, t1, t0) A parameter is compared with the threshold of the parameter to determine whether the user gesture is correct. If the user posture is incorrect, it means that at least one of the posture parameters exceeds a range represented by the parameter threshold.
  • Step 404 Signal output.
  • the trigger gesture reminding module issues a prompt message.
  • the embodiment further includes step 405: delay.
  • a delay of 0 indicates a direct regression.
  • the embodiment shown in FIG. 9 is further optimized.
  • at least one of the attitude parameters exceeds a threshold value during a loop constituted by steps 401 to 404, and the parameters are exceeded for a plurality of cycles.
  • the limit value the step 404 triggers the gesture reminding module to issue a prompt signal and/or a correction suggestion signal.
  • the user posture is incorrect, that is, at least one of the posture parameters exceeds a threshold value, and the average parameter exceeds the threshold value for a plurality of cycles.
  • the camera can only take a shot every tens of seconds, and other units make correspondings according to the result of the shooting. response.
  • An example of the method implemented is to make the delay time in steps 205, 305, and 405 of the above embodiment tens of seconds, and the present method is also applicable to other embodiments described in the present specification.
  • the system and method of the present invention is only meaningful in the operational state of the display screen.
  • the invention first detects whether the display screen is in a display state prior to initiating attitude detection. As shown normally, the steps described in the previous embodiments are performed. If there is no display, the detection process is not turned on to save power.
  • the operating system of the smart terminal When applied to a mobile intelligent terminal (such as a 3G mobile phone or an LTE mobile phone), the operating system of the smart terminal (such as Android, iOS, BlackBerry) can provide a system service based on the method of the present invention, as a background running service, the service Check the user's posture when the display is occupied by the application and display the content, and promptly when the gesture is incorrect.
  • the operating system of the smart terminal can provide an API (Application Programming Interface), which provides a gesture of the user to the application by the technical solution of the present invention, and the gesture is reminded by the application.
  • the present invention can also be a general application or a general monitoring application running on the smart terminal, and the application determines whether to issue a gesture reminder by the method of the present invention.
  • the method can also include the step of setting the function.
  • an application Can be set up so that the feature is only enabled in certain applications, or it can be set to make it not enabled in some applications. If the foreground application is answering, making a call, listening to radio or other audio content, since the user does not pay attention to the display for a long time, it is not necessary to turn on this function.
  • This setting can be set manually by the user, or it can be set by the operating system or application.
  • FIG. 10 is an embodiment of a terminal device user attitude detection system.
  • the information processing module 300 further includes a machine vision processing module 31, a posture calculation module 32, and a posture determination module 33.
  • the machine vision processing module 31 processes the image or video obtained by the camera, and extracts the posture parameters of the user relative to the device, including the distance and the relative angle.
  • the attitude calculation module 32 calculates the absolute attitude parameter of the user according to the absolute attitude parameter of the terminal device and the user relative device geometric relationship data.
  • the posture determination module 33 determines the relative posture parameter and the absolute posture parameter of the user. When it is found that one or more posture parameters are incorrect, the attitude reminding interface c sends data to the posture reminding module 400, and triggers the posture reminding module 400. A prompt signal and/or a correction suggestion signal is issued.
  • the information processing module 300 takes local information processing
  • the calculation can be completed locally in the smart terminal, that is, the camera 100, the posture sensor 200, the information processing module 300, and the gesture reminding module 400 are all integrated in one terminal device.
  • the acquisition of the entire head pose and associated calibration procedures can be performed by local computing resources such as software, firmware or hardware.
  • the attitude sensing data interface a, the image data interface b, and the gesture reminding interface c all use local communication technology.
  • the terminal device 500 includes a camera 100, an attitude sensor 200, and a gesture reminder module 400.
  • the information processing module 300 is external to the terminal device 500.
  • the local camera 100 transmits the obtained picture or video to the application service provider's server through the communication device on the smart terminal for analysis and calculation, and finally the server sends an indication of whether to issue a reminder to the smart terminal. Since the computing power of a general server is much stronger than that of a smart terminal, this can achieve better performance.
  • the attitude sensing data interface a, the image data interface b, and the gesture reminding interface c are implemented using a wired or wireless communication device capable of supporting transmission over a long distance. These communication devices include the wired connection device 7, the wireless connection devices 81, 82, and the data transfer network 900, and the like in Figs.
  • FIG. 11 is an embodiment of a terminal device attitude detection system for wired connection telematics, further comprising a wired connection device 7 in the terminal device user attitude detection system; the attitude sensing data interface a, the image data interface b, and the gesture reminding interface c This is achieved by the wired connection device 7; the wired connection device 7 is a cable or fiber optic cable.
  • the terminal device user attitude detection system further includes wireless connection devices 81 and 82; the attitude sensing data interface a, the image data interface b, and the gesture reminding interface.
  • c is implemented by the wireless connection means 81, 82; the wireless connection means comprises antennas 812, 822 and modems 811, 821.
  • FIG. 13 is an embodiment of a terminal device attitude detection system for data transmission network connection telematics processing, and the terminal device user attitude detection system further includes a data transmission network 900.
  • the attitude sensing data interface a, the image data interface b, and the gesture reminding interface c are implemented by the data transmission network 900.
  • the data transfer network 900 is a local area network or the Internet; the data transfer network includes a computer network, or a switch network, or a router network.
  • the apparatus of the present invention is used for implementing a mobile terminal 5 for user gesture detection by an operating system, where the mobile terminal includes a gesture sensing module 1 and a user.
  • the input module 2, the alarm signal output module 3, the operating system running device 4, and the application modules 71, 72, ..., 7N are configured.
  • the attitude sensing module 1 generates an attitude sensing signal S.
  • the user configuration input module 2 identifies the user operation and passes the user command I to the application modules 71, 72, ..., 7N and/or the operating system operating device 4.
  • the alarm signal output module 3 emits a signal recognizable by the user.
  • the operating system running device of the present invention is a device for loading and running a mobile terminal operating system, such as loading and running Android, iOS, BlackBerry operating system and its replacement program.
  • the application modules 71, 72, ..., 7N of the present invention are devices for loading and running applications, particularly loading and running applications that may be used continuously by users for a long time, such as video players, web browsers, and reading. Software, etc.
  • the components of the application module, the operating system running module and the like of the present invention can be implemented by a combination of a CPU system, a programmable logic circuit and a memory.
  • FIG. 14 further depicts the operating system operating device 4 of the present invention, including a sensing module driving unit 41, an input module driving unit 42, and an output module driving unit 43.
  • the sensing module driving unit 41 controls the operation of the attitude sensing module 1 and converts the attitude sensing signal S output from the attitude sensing module 1 into the attitude data D.
  • the input module driving unit 42 controls the user to configure the input module to work, recognizes the user command I, and generates the user command data; the output module driving unit 43 outputs the signal O, including the alarm data A and/or the attitude correction data R, and controls the alarm signal output module 3 to work. .
  • the operating system running device 4 further includes a posture service module 40.
  • the posture service module 40 further includes an attitude detecting unit 46 that reads the posture data D output by the sensing module driving unit, recognizes the position of the user's face, and generates the posture parameter G.
  • the gesture service module 40 (or at least one of the application modules 71, 72, ..., 7N) further includes a posture comparison unit 47 that compares the attitude parameter G with a corresponding threshold value; the attitude service module 40 (or at least one of the application modules 71, 72, ..., 7N) further includes a gesture reminding unit 48 that generates alarm data A and/or attitude correction data R.
  • the gesture service module 40 (or at least one of the application modules 71, 72, ..., 7N) generates alarm data A and/or attitude correction data R to activate the input module drive unit.
  • the operating system running device further includes an application database 44, which stores the working data and the default Confirm the setting data.
  • the work data stored in the application database 44 refers to data changed in the operation of the apparatus of the present invention, including "application status table”, “attitude detection flag”, “application registry”, and posture parameters, etc. .
  • the application status table contains information about the working state of the application module in the terminal, such as whether the application module is running, whether it occupies the display screen, and the like.
  • the application registry contains an identifier that detects the necessary application module name or system allocation.
  • the application modules 71, 72, ..., 7N can add their own names or identifiers to the application database 44 via the application programming interface 45.
  • the attitude detection flag bit is a flag indicating whether the attitude service module 40 in the operating system running device 4 is operating.
  • the default setting data stored by the application database 44 includes one or more of the following: an attitude detection flag bit, an application registry, and a threshold value. These data can also be set manually, where the gesture detection flag and the application registry may also be reset at work.
  • An application programming interface 45 is also included in the operating system operating device 4.
  • the sensing signal S further includes an image signal S1 and an angle sensing signal S2.
  • the posture data D further includes user posture data D1 and local posture data D2.
  • the attitude parameter G further includes a relative attitude parameter G1 and an absolute attitude parameter G2.
  • the threshold values corresponding to G1 and G2 are T1 and T2, respectively.
  • the attitude sensing module 1 uses a camera solution, including one or more front cameras 100, to generate an image signal S1.
  • a camera driving unit 411 is included in the sensing module driving unit 41.
  • the posture detecting unit 46 reads the user posture data D1 output from the camera driving unit 411, detects the relative position of the user's face and the display screen 6, and generates a relative posture parameter G1.
  • the attitude service module 40 further includes a posture comparison unit 47 that compares the attitude parameter G1 with a corresponding threshold value T1.
  • the operating system operating device 4 or the application module (for example, 71) further includes a gesture reminding unit that generates alarm data A and/or attitude correction data R.
  • the attitude sensing module 1 uses a combination of a camera and a local attitude sensor.
  • the attitude sensing module 1 includes one or more front cameras 100 to generate an image signal S1.
  • the attitude sensing module also includes the ability to detect the flatness of the display screen.
  • the face angle sensor 12 (such as a gravity sensor, also known as a tilt sensor) produces an angle sensing signal S2.
  • the sensor module driving unit 41 includes a camera driving unit 411 and a driving unit 412 of a local attitude sensor (gravity sensor, tilt sensor).
  • the posture detecting unit 46 reads the user posture data D1 output from the camera driving unit, detects the relative position of the user's face and the display screen 6, and generates a relative posture parameter G1.
  • the posture detecting unit 46 reads the local posture sensor driving unit 412 to output the local posture data D2, and determines the inclination angle t1 of the plane in which the display screen 6 of the unit is located.
  • the attitude detecting unit 46 generates an absolute attitude parameter G2 based on the inclination angle t1 of the relative posture parameter G1 and the plane in which the local display 6 is located.
  • the operating system running device or the application module further includes a posture comparing unit, and comparing the posture parameters G1 and G2 with corresponding threshold values T1 and T2; the operating system running device or the application module further includes a posture reminding unit, Alarm data A and/or attitude correction data R are generated.
  • FIG. 17 is a working process of the operating system running device and the application module implementing user posture detection, comprising the following steps:
  • Step 501 configuration input: application modules 71, 72, ..., 7N and / or operating system operating device 4 accepts input data from the user configuration input module instruction I;
  • Step 502 attitude detection: the sensing module driving unit 41 controls the attitude sensing module to acquire the sensing signals S1 and S2, and converts into the attitude data D1 and D2; the posture service module 40 reads the posture of the sensing module driving unit 41. Data D1, D2, the attitude detecting unit analyzes the attitude data to obtain a relative attitude parameter G1 and/or an absolute attitude parameter G2;
  • Step 503 posture comparison: the posture comparison unit compares the user posture parameters G1, G2 and the threshold values T1, T2, and when the posture is normal, the process proceeds to step 502, otherwise the step 504;
  • Step 504 posture reminding: when the user posture parameter exceeds the threshold value, the posture reminding unit generates alarm data A and/or posture correction data R; the alarm signal output module sends a signal to the customer that the user feels identifiable, including the alarm After the information and / or correction suggestions, after the alarm is output, go to step 502.
  • the step of posture detection is completed in the operating system running device 4, and the steps of posture comparison and posture reminding may be completed by the posture service module 40 in the operating system running device, or may be performed by the application module 71, 72, ... , 7N completed.
  • a delay step (step 505) is passed.
  • 18 is an embodiment of an operating system running device and an application module in a mobile terminal, the operating system running device implementing posture detection, posture comparison, and gesture reminding.
  • the operating system running device 4 includes a sensing module driving unit 41, an input module driving unit 42, and an output module driving unit 43.
  • the sensing module driving unit outputs a sensing control signal and an input sensing signal S;
  • the input module drives
  • the unit outputs a configuration control signal, receives a user command I;
  • the output module drive unit outputs an alarm control signal, and an output signal O including the alarm data A and/or the attitude correction data R.
  • the operating system operating device 4 further includes a posture service module 40.
  • the posture service module 40 includes an attitude detecting unit 46, a posture comparing unit 47, and a posture reminding unit 48.
  • the posture detecting unit 46 activates the sensing module driving unit 41, reads the posture data D output from the sensing module driving unit 41, recognizes the position of the user's face, and generates the posture parameter G.
  • the posture comparing unit 47 compares the posture parameter G with a corresponding threshold value T; the posture reminding unit 48 generates alarm data A and/or posture correction data R when the user posture is incorrect, and starts The output module drives the unit.
  • the operating system running device 4 further includes an application database 44 that stores work data and default setting data.
  • the operating system operating device shown in FIG. 18 further includes the following parts:
  • the gesture service module 40 further includes a data scanning unit 49 that queries the work data in the application database, and triggers the attitude detecting unit 46 when the gesture detection process needs to be initiated according to the indication of the working data.
  • the operating system running device 4 further includes an application programming interface (API) 45, and the application modules 71, 72, ..., 7N call the application database 44 through the application programming interface 45 to set data therein. Or clear.
  • API application programming interface
  • 19 is an embodiment of an operating system running device and an application module in a mobile terminal, the operating system running device performing posture detection.
  • the operating system running device 4 includes a sensing module driving unit 41, an input module driving unit 42, and an output module driving unit 43, the sensing module driving unit outputs a sensing control signal, an input sensing signal S; and an input module driving unit output configuration control
  • the signal receives the user command I; the output module drive unit outputs an alarm control signal and an output signal O including the alarm data A and/or the attitude correction data R.
  • the operating system running device 4 further includes a posture service module 40.
  • the posture service module 40 includes an attitude detecting unit 46.
  • the posture detecting unit 46 activates the sensing module driving unit 41 to read the posture data output by the sensing module driving unit 41. D, identifying the position of the user's face, generating a pose parameter G, and saving the pose parameter in the application database.
  • the operating system running device 4 further includes an application database 44 that stores work data and default setting data.
  • An application programming interface (API) 45 is also included in the operating system operating device 4.
  • the operating system operating device 4 shown in FIG. 19 further includes the following Minute:
  • the gesture service module 40 further includes a data scanning unit 49 that queries the work data in the application database 44 to trigger the gesture detection unit 46 when the gesture detection process needs to be initiated according to the indication of the work data.
  • 20 is an embodiment of an operating system running device and an application module in a mobile terminal, the operating system operating device performing posture detection and posture comparison.
  • the operating system running device 4 includes a sensing module driving unit 41, an input module driving unit 42, and an output module driving unit 43, the sensing module driving unit outputs a sensing control signal, an input sensing signal S; and an input module driving unit output configuration control
  • the signal receives the user command I; the output module drive unit outputs an alarm control signal and an output signal O including the alarm data A and/or the attitude correction data R.
  • the operating system running device 4 further includes a posture service module 40.
  • the posture service module 40 includes an attitude detecting unit 46 and a posture comparing unit 47, and activates the sensing module driving unit 41 to read the attitude data D output by the sensing module driving unit 41. Identifying the position of the user's face, generating a posture parameter G; the posture comparison unit 47 compares the attitude parameter G with a corresponding threshold value T, and at least through the application programming interface 45 when the attitude parameter exceeds the threshold value
  • One of the application modules 71, 72, ..., 7N issues an attitude overrun notification.
  • the operating system running device 4 further includes an application database 44 that stores work data and default setting data.
  • At least one of the application modules includes a gesture reminding unit 48 and an input and output processing unit 70.
  • the gesture reminding unit 48 generates alarm data A and/or attitude correction data R.
  • the input/output processing unit 70 sets the work data in the application database 44 through the application programming interface 45, receives the attitude overrun notification from the operating system operating device 4, activates the input module driving unit 42, and activates the output module driving unit 43.
  • the operating system operating device 4 shown in FIG. 20 further includes the following parts:
  • the gesture service module 40 further includes a data scanning unit 49 that queries the work data in the application database 44 to trigger the gesture detection unit 46 when the gesture detection process needs to be initiated according to the indication of the work data.
  • the application module (for example, 71) of the present invention includes a posture comparing unit 47, a posture reminding unit 48, and an input/output processing unit 70.
  • the attitude comparison unit 47 compares the attitude parameter G with a corresponding threshold value T; the gesture reminding unit 48 generates alarm data A and/or attitude correction data R.
  • the input/output processing unit 70 sets or reads the working data in the application database 44, activates the input module driving unit 42, and starts the output module driving unit 43 through the application programming interface 45.
  • the application module and the data scanning unit process the working data as follows:
  • the "Application Status Table" in the application database 44 is modified by the application programming interface 45 to write the application.
  • the name of the module (or some identifier assigned by the system).
  • the user sets the "application column registry” through the user configuration input module 2, and some application modules can also modify the "application registry” by themselves and add themselves to this list.
  • the data scanning unit 49 in the gesture service module 40 queries the work data of the application database, where the work data includes an application state table and an application registry; according to the indication of the work data, the data scanning unit is from the application.
  • the status table obtains the application module name or identifier that is using the display, checks if the application module is in the application registry, and if so, initiates the gesture detection process.
  • the application programming interface 45 provided by the operating system running device 4 is used to set or clear the "attitude detection flag bit".
  • the application programming interface is called to set the flag bit to activate the attitude detection function.
  • the data scanning unit 49 in the gesture service module 40 queries the work data of the application database, and the work data includes a posture detection flag bit. According to the indication of the work data, the data scanning unit queries whether the posture detection flag bit is queried. Is set, if so, you need to start the attitude detection process. This flag is cleared when an application module exits or other conditions that do not require monitoring of the pose.
  • the operating system operating device 4 and at least one application module accept command data I from the user configuration input module.
  • the input module driving unit 42 Under the control of the input module driving unit 42, the user command input by the user configuration input module 2 is recognized.
  • the input module drive unit can be activated either by the operating system operating device 4 or by the input/output processing unit 70 of the application module (e.g., 71) via the application programming interface 45.
  • the instruction data includes at least one of the following instructions: 1.
  • the background service performs gesture detection only when the application is running in the foreground. Some applications can also actively join this list on their own. By setting this feature is not enabled in some applications.
  • the foreground application is answering, making a call, listening to radio or other audio content, since the user does not pay attention to the display for a long time, it is not necessary to turn on this function.
  • the gesture detection function is activated or disabled directly by setting this flag bit.
  • Threshold command for setting the attitude parameter threshold By setting thresholds to meet the needs of different users. In the absence of a user command, the above command data has default setting data in the application database 44 of the operating system running device 4.
  • the alarm output scheme is specifically described.
  • Output module in operating system running device 4 Under the control of the drive unit 43, the alarm signal output module 3 processes the alarm data A and the attitude correction data R.
  • the alarm data is generated when the user's attitude parameter exceeds the threshold, including the mode of the alarm, the strength of the alarm, and the length of the alarm. These data are determined according to the degree and time when the user deviates from the correct posture.
  • the attitude correction data is based on the difference between the attitude parameter and the threshold value, and obtains information on how to change the user posture to achieve the correct posture, including distance, angle, and direction of adjustment.
  • the signal output by the alarm output module is a sensory identifiable signal of the user, and includes one or more of the following types: a prompt sound emitted by the audio module, a vibration generated by the vibration device, a graphic or text prompt given through the display screen. .
  • the mobile terminal 5 that implements user posture detection by an operating system includes an attitude sensing module 1, a user configuration input module 2, an alarm signal output module 3, an operating system running device 4, and a display screen (not shown in FIG. 18). Out, one or more application modules 71, 72, ..., 7N.
  • the attitude sensing module generates a sensing signal S.
  • the user configuration input module 2 recognizes the user operation and passes the user command I to the operating system operating device 4.
  • the alarm signal output module 3 emits a signal recognizable by the user.
  • the posture service module 40 in the operating system operating device 4 includes an attitude detecting unit 46, a posture comparing unit 47, and a posture reminding unit 48.
  • the gesture service module 40 works in parallel with the application modules 71, 72, ..., 7N to alert when a user gesture is found to be incorrect. The work steps are as follows:
  • Step 602 posture detection: the attitude detecting unit 46 activates the sensing module driving unit 41, controls the attitude sensing module 1, collects the sensing signals S1, S2, and converts into attitude data D1, D2; the attitude service module 40 reads and transmits The attitude data D1, D2 output by the sensor module driving unit 41; the attitude detecting unit 46 in the attitude service module 40 analyzes the attitude data to obtain a relative attitude parameter G1 and/or an absolute attitude parameter G2;
  • Step 603 posture comparison: the posture comparison unit 47 in the attitude service module 40 compares the posture parameters G1, G2 and the user posture threshold values T1, T2, and when the posture is normal, the process proceeds to step 605, otherwise the step 604 is changed. ;
  • Step 604 posture reminder: the posture reminder 48 unit in the attitude service module 40 generates alarm data A and/or attitude correction data R when the attitude parameter exceeds the threshold value; the alarm signal output module 48 sends a user feeling to the customer.
  • the identifiable signal contains alarm information and/or correction suggestions. After the alarm is output, go to step 605.
  • Step 605 wait, delay and then go to step 602.
  • gesture detection service program there is no data exchange between the gesture detection service program and the application module.
  • the application module For example, in Android or iOS, you can include a gesture service module.
  • posture measurement as a system background service, it calls other system services, obtains the output of the front camera, and measures the posture. When the situation is not correct, the background service switches to the foreground and prompts the user.
  • the posture service module in the operating system running device 40 includes an attitude detecting unit 46, a posture comparing unit 47, a posture reminding unit 48, and a data scanning unit 49.
  • the gesture service module 40 works in parallel with the application modules 71, 72, ..., 7N.
  • the data scanning unit 49 in the gesture service module 40 detects the currently running application module at intervals, and when it is necessary to monitor the user gesture, the gesture is initiated.
  • the detection unit 46 operates.
  • the "application status table" in the application database 44 is modified by the application programming interface 45, and the name of the application module (or some identifier assigned by the system) is written. symbol).
  • the user can configure the "application registry" through the user configuration input module 2, which performs posture detection and reminding when occupying the display screen.
  • Some application modules can also modify the "application registry" by themselves and add themselves to this list.
  • Step 702a the data scanning unit 49 obtains the application module name (or some identifier assigned by the system) that is using the display screen from the "application status table" in the application database 44;
  • Step 702b the data scanning unit 49 checks whether the application module is in the "application registry", and if so, proceeds to step 703 to trigger the gesture detecting unit 46; otherwise, the step 702c is performed;
  • Step 702c delay waiting and then proceed to step 702a;
  • Step 703 attitude detection the attitude detecting unit 46 activates the sensing module driving unit 41, controls the attitude sensing module 1 to acquire the sensing signals S1, S2, and converts into the attitude data D1, D2; the attitude detecting unit 46 reads the sensing The attitude data D1, D2 output by the module driving unit; the attitude detecting unit 46 analyzes the attitude data to obtain a relative attitude parameter G1 and/or an absolute attitude parameter G2;
  • Step 704 attitude comparison: the attitude comparison unit 47 in the attitude service module 40 compares the posture parameters G1 and G2 with the user posture threshold values T1 and T2, and if the posture is normal, the operation proceeds to step 706. Otherwise, the operation proceeds to step 705. ;
  • Step 705 posture reminder: the posture reminding unit 48 in the attitude service module 40 generates the alarm data A and/or the attitude correction data R when the posture parameter exceeds the threshold value; the alarm signal output module 43 sends the user feeling to the customer. An identifiable signal containing alarm information and/or correction suggestions. After the alarm is output,
  • Step 706, wait, delay and then go to step 702a.
  • the attitude service module 40 in the operating system running device 4 includes an attitude detecting unit 46.
  • the gesture service module 40 operates in parallel with the application modules 71, 72, ..., 7N.
  • the application programming interface 45 provided by the operating system operating device 4 is used to set or clear the "attitude detection flag".
  • the application programming interface is called to set the flag bit, thereby activating the attitude detection function. This flag is cleared when an application module exits or other conditions that do not require monitoring of the pose.
  • each application module can set the flag bit. If any of the application modules that have set the flag bit does not clear the flag, then the gesture detection and reminder function It is activated.
  • FIG. 23 (a) shows the working process of the application modules 71, 72, ..., 7N, the steps are as follows:
  • Step 800 the application module sets or clears the "attitude detection flag bit" in the application database 44 through the application programming interface
  • step 801 the application module runs the main function of the application, and then proceeds to step 800 after completing;
  • FIG. 23 (b) shows the working process of the attitude service module 40, the steps are as follows:
  • Step 802 the data scanning unit 49 queries whether the "attitude detection flag bit" is set, and if so, then proceeds to step 803 to trigger the attitude detecting unit 46; otherwise, the process proceeds to step 806;
  • Step 803 posture detection: the attitude detecting unit 46 activates the sensing module driving unit 41, controls the attitude sensing module 1, collects the sensing signals S1, S2, and converts them into attitude data D1, D2; the posture detecting unit 46 reads and transmits The attitude data D1, D2 output by the module driving unit; the attitude detecting unit 46 analyzes the attitude data to obtain a relative attitude parameter G1 and/or an absolute attitude parameter G2;
  • Step 804 posture comparison: the posture comparison unit 47 in the attitude service module 40 compares the posture parameters G1 and G2 with the user posture threshold values T1 and T2, and if the posture is normal, the operation proceeds to step 806, otherwise the operation proceeds to step 805. ;
  • Step 805 posture reminder: the posture reminding unit 48 in the attitude service module 40 generates alarm data A and/or posture correction data R when the posture parameter exceeds the threshold value; the alarm signal output module 43 sends a user feeling to the customer. An identifiable signal containing alarm information and/or correction suggestions. After the alarm is output, go to step 806;
  • Step 806, wait, delay and then proceed to step 802.
  • the mobile terminal 5 that implements user gesture detection by an operating system includes an attitude sensing module 1, a user configuration input module 2, an alarm signal output module 3, an operating system running device 4, and a display screen (not shown in FIG. 19).
  • an attitude sensing module 1 a user configuration input module 2
  • an alarm signal output module 3 an operating system running device 4
  • a display screen not shown in FIG. 19
  • application modules 71, 72, ..., 7N one or more application modules 71, 72, ..., 7N.
  • the attitude sensing module generates a sensing signal S.
  • the user configuration input module 2 identifies the user operation and passes the user command I to the operating system operating device 4 and at least one of the application modules (e.g., 71).
  • Alarm signal output mode Block 3 signals the user's sensory identifiable.
  • At least one of the application modules includes a gesture comparison unit 47, a gesture reminder unit 48, and an input and output processing unit 70.
  • the attitude comparison unit 47 compares the attitude parameter G with a corresponding threshold value T; the gesture reminding unit 48 generates alarm data A and/or attitude correction data R.
  • the input/output processing unit 70 reads the data in the application database 44 through the application programming interface 45, activates the input module driving unit 42, and activates the output module driving unit 43.
  • the posture service module 40 in the operating system running device 4 includes only the posture detecting unit 46; the application program 71 includes a posture comparing unit 47, a posture reminding unit 48, and an input/output processing unit 70.
  • the attitude service module 40 in the operating system running device 4 automatically runs and saves the posture parameters in the application database 44 to form a posture parameter library.
  • the input/output processing unit 70 in the application module 71 periodically reads these posture parameters through the application programming interface 45 provided by the operating system running device 4.
  • Figure 24 (a) shows the working steps in the operating system operating device:
  • Step 901 posture detection: the attitude detecting unit 46 activates the sensing module driving unit 41, controls the attitude sensing module 1 to acquire the sensing signals S1, S2, and converts into the attitude data D1, D2; the attitude detecting unit 46 reads the sensing The attitude data D1, D2 output by the module driving unit; the attitude detecting unit 46 analyzes the attitude data to obtain a relative attitude parameter G1 and/or an absolute attitude parameter G2;
  • Step 902 Data Storage: The pose parameters generated by the attitude detecting unit 46 are stored in the application database 44.
  • Step 903 wait, delay and then go to step 901.
  • Figure 24(b) shows the working steps in the application module:
  • Step 904 the application module 71 performs the function of the main program
  • Step 905 data reading: the input and output processing unit 70 in the application module 71 reads the attitude parameter and the threshold data in the application database 44 through the application programming interface 45 provided by the operating system running device 4;
  • Step 906 posture comparison: the posture comparison unit 47 in the application module 71 compares the posture parameters G1, G2 and the threshold values T1, T2, and when the posture is normal, the process proceeds to step 904, otherwise the process proceeds to step 907;
  • Step 907 posture reminder: the posture reminding unit 48 in the application module 71 generates alarm data A and/or posture correction data R when the posture parameter exceeds the threshold value; the input/output processing unit 70 runs the device through the operating system.
  • the application programming interface 45 is provided to activate the output module driving unit 43 to control the alarm signal output module 3 to send a signal to the customer that the user feels identifiable, including alarm information and/or correction suggestions. After the alarm is output, the process proceeds to step 904.
  • FIG 25 is an embodiment based on Figure 19, in Figure 25, the application module 71 uses the attitude detection flag to The posture detecting unit 46 of the operating system running device 4 is turned on and off, and the result stored in the application database 44 is read.
  • the attitude service module 40 in the operating system operating device 4 includes an attitude detecting unit 46 and a data scanning unit 49.
  • the gesture comparison unit 47 and the gesture reminder unit 48 are included in the application module 71.
  • the input/output processing unit 70 in the application module 71 sets the "attitude detection flag bit" through the application programming interface 45 (API) provided by the operating system running device 4.
  • the data scanning unit 49 periodically queries the flag bit, and when the flag bit is set, runs the attitude detecting unit 46 and saves the posture parameter in the application database 44 to form a posture parameter library.
  • the application module 71 reads these gesture parameters through the application programming interface 45 provided by the operating system running device, and determines whether to make a reminder according to the posture parameters.
  • Figure 25 (a) is the working steps in the operating system operating device:
  • Step 1001 the data scanning unit 49 queries whether the "attitude detection flag bit" is set, and if so, proceeds to step 1002, triggers the attitude detecting unit 46; otherwise, proceeds to step 1004;
  • Step 1002 posture detection: the attitude detecting unit 46 activates the sensing module driving unit 41, controls the attitude sensing module 1 to acquire the sensing signals S1, S2, and converts into the attitude data D1, D2; the attitude detecting unit 46 reads the sensing The attitude data D1, D2 output by the module driving unit; the attitude detecting unit 46 analyzes the attitude data to obtain a relative attitude parameter G1 and/or an absolute attitude parameter G2;
  • Step 1003 data storage: the posture parameters generated by the attitude detecting unit 46 are saved in the application database 44;
  • Step 1004 wait, delay and then go to step 601.
  • FIG. 25(b) shows the working steps in the application module:
  • Step 1005 the application module 71 input and output processing unit 70 sets or clears the "attitude detection flag bit" in the application database 44 through the application programming interface 45;
  • Step 1006 wait for a delay, or perform other functions
  • Step 1007 data reading: the input/output processing unit 70 in the application module 71 reads the attitude parameter and the threshold data in the application database 44 through the application programming interface 45 provided by the operating system running device 4;
  • Step 1008 posture comparison: the posture comparison unit 47 in the application module 71 compares the posture parameters G1, G2 and the threshold values T1, T2, and when the posture is normal, the process proceeds to step 1005, otherwise, the process proceeds to step 1009;
  • Step 1009 posture reminder: the posture reminding unit 48 in the application module 71 generates alarm data A and/or posture correction data R when the posture parameter exceeds the threshold value; the input/output processing unit 70 runs the device through the operating system.
  • the application programming interface 45 is provided to activate the output module driving unit 43, and the control alarm signal output module sends a signal to the customer that the user feels identifiable, including the alarm information and/or the correction suggestion. After the alarm is output, the process proceeds to step 1005.
  • the attitude service module in the operating system operating device 40 includes an attitude detecting unit 46 and a data scanning unit 49.
  • the gesture service module 40 works in parallel with the application modules 71, 72, ..., 7N.
  • the data scanning unit 49 in the gesture service module 40 detects the currently running application module at intervals, and when it is necessary to monitor the user gesture, the gesture is initiated.
  • the detection unit 46 operates.
  • the "application status table" in the application database 44 is modified by the application programming interface 45, and the name of the application module is written (or some type assigned by the system) Identifier).
  • the user can configure the "application registry" through the user configuration input module 2, which performs posture detection and reminding when occupying the display screen.
  • Some application modules can also modify the "application registry" by themselves and add themselves to this list.
  • Figure 26 (a) is the working steps in the operating system operating device:
  • Step 1101a judging whether the application module (for example, 71) occupies the display screen: the data scanning unit 49 obtains the application module name (or system allocation) that is using the display screen from the "application status table" in the application database 44. Kind of identifier), if yes, step 1101b, otherwise go to step 1104;
  • Step 1101b the data scanning unit 49 checks whether the application module 71 is in the "application registry", and if so, proceeds to step 1102, triggering the gesture detecting unit 46; otherwise, proceeds to step 1104;
  • Step 1102 posture detection: the attitude detecting unit 46 activates the sensing module driving unit 41, controls the attitude sensing module 1 to acquire the sensing signals S1, S2, and converts into the attitude data D1, D2; the posture detecting unit 46 reads the sensing The attitude data D1, D2 output by the module driving unit; the attitude detecting unit 46 analyzes the attitude data to obtain a relative attitude parameter G1 and/or an absolute attitude parameter G2;
  • Step 1103, Data Storage The pose parameters generated by the attitude detecting unit 46 are stored in the application database 44.
  • Step 1104 wait, delay and then go to step 1101a.
  • Figure 26(b) is the working steps in the application module:
  • Step 1105 when the application module 71 occupies the display screen, the input/output processing unit 70 modifies the "application status table" in the application database 44 through the application programming interface 45, and writes the name or identifier of the application module;
  • Step 1106 delay waiting, or performing other functions of the application module
  • Step 1107 data reading: the input/output processing unit 70 in the application module 71 reads the attitude parameter and the threshold data in the application database 44 through the application programming interface 45 provided by the operating system running device 4;
  • Step 1108 posture comparison: the posture comparison unit 47 in the application module 71 compares the posture parameters G1, G2 with the threshold values T1, T2, and when the posture is normal, the first step is step 105, otherwise the step 1109 is turned;
  • Step 1109 posture reminder: the posture reminding unit 48 in the application module 71 generates alarm data A and/or posture correction data R when the posture parameter exceeds the threshold value; the input/output processing unit 70 runs the device through the operating system.
  • the application programming interface 45 is provided to activate the output module driving unit 43.
  • the control alarm signal output module sends a signal to the customer that the user feels identifiable, including alarm information and/or correction suggestions. After the alarm is output, the process proceeds to step 1105.
  • the mobile terminal 5 that implements user posture detection by an operating system includes an attitude sensing module 1, a user configuration input module 2, an alarm signal output module 3, an operating system operating device 4, a display screen 6, one or more Application modules 71, 72, ..., 7N.
  • the attitude sensing module generates a sensing signal S.
  • the user configuration input module 2 identifies the user operation and passes the user command I to the operating system operating device 4 and at least one of the application modules (e.g., 71).
  • the alarm signal output module 3 emits a signal recognizable by the user.
  • FIG. 27 is a flow chart of a method for issuing an attitude overrun notification completion reminder to an application module based on the operating system running device of FIG.
  • the posture detecting unit in the operating system running device determines the user posture, and the posture comparing unit completes the posture judgment.
  • the notification is sent to the application module, and the posture is executed by the application module.
  • Reminder feature is a method for issuing an attitude overrun notification completion reminder to an application module based on the operating system running device of FIG.
  • Figure 27 (a) is the working steps in the operating system operating device:
  • Step 1201 attitude detection: the attitude detecting unit 46 activates the sensing module driving unit 41, controls the attitude sensing module 1 to acquire the sensing signals S1, S2, and converts into attitude data D1, D2; the attitude detecting unit 46 reads the sensing The attitude data D1, D2 output by the module driving unit analyzes the attitude data to obtain a relative attitude parameter G1 and/or an absolute attitude parameter G2;
  • Step 1202 attitude comparison: The attitude comparison unit 47 compares the attitude parameters G1, G2 with the threshold values T1, T2, and when the posture is normal, the process proceeds to step 1204, and when the attitude parameter exceeds the threshold value range, the process proceeds to step 1203. ,
  • the gesture comparison unit 47 issues a gesture overrun notification to at least one of the application modules 71, 72, ..., 7N (e.g., the application module 71) via the application programming interface 45.
  • the manner in which the operating system running device 4 issues a notification includes: 1. Broadcasting: issuing a notification to all application modules; 2. Selecting unicast: issuing a notification to the application module occupying the display screen (the application of the application database 44 can be queried) The status table "gets the application module name or identifier occupying the display screen"; 3.
  • Register unicast issues a notification to the application module that occupies the display screen and registers the gesture detection service (the application status of the application database 44 can be queried) Table" and "applications The order registry "gets the application module name or identifier that occupies the display and registers the gesture detection service).
  • Step 1204 Wait: After the delay, go to step 1201.
  • Figure 27 (b) is the working steps in the application module 71:
  • Step 1205 message monitoring: If the application module 71 is occupying the display screen, and the input/output processing unit 70 included therein receives the gesture overrun notification from the operating system running device 4, then the step 1206 is triggered to trigger the gesture reminding unit 48. ;
  • Step 1206 gesture reminder: the gesture reminding unit 48 generates the alarm data A and/or the attitude correction data R; the input/output processing unit 70 activates the output module driving unit 43 through the application programming interface 45 provided by the operating system running device 4, and controls The alarm signal output module sends a signal to the customer that the user feels identifiable, including alarm information and/or correction suggestions.
  • the service broadcasts all the application modules, and the broadcast information may further include information describing the posture parameter G, and then the application module determines how to output the alarm information.
  • the application module uses the attitude detection flag.
  • the gesture service module 40 of the operating system running device 4 is activated and deactivated.
  • the attitude service module 40 in the operating system operating device 4 includes an attitude detecting unit 46, a posture comparing unit 47, and a data scanning unit 49.
  • the gesture reminder unit 48 is included in the application module 71.
  • the input/output processing unit 70 in the application module 71 sets the "attitude detection flag bit" through the application programming interface 45 (API) provided by the operating system running device 4.
  • API application programming interface 45
  • the data scanning unit 49 periodically queries the flag bit, and when the flag bit is set, the running posture detecting unit 46 and the posture comparing unit 47 emit a gesture to the application modules 71, 72, ..., 7N when the attitude parameter exceeds the threshold value. Overdue notification.
  • Figure 28 (a) is the working steps in the operating system operating device:
  • Step 1301 the data scanning unit 49 queries whether the "attitude detection flag bit" is set, if yes, then proceeds to step 1302, triggers the attitude detecting unit 46; otherwise, steps 1305;
  • Step 1302 posture detection: the attitude detecting unit 46 activates the sensing module driving unit 41, controls the attitude sensing module 1 to acquire the sensing signals S1, S2, and converts into the attitude data D1, D2; the attitude detecting unit 46 reads the sensing The attitude data D1, D2 output by the module driving unit analyzes the attitude data to obtain a relative attitude parameter G1 and/or an absolute attitude parameter G2;
  • Step 1303 attitude comparison: The attitude comparison unit 47 compares the attitude parameters G1 and G2 with the threshold values T1 and T2, and when the attitude parameter exceeds the threshold value range, the process proceeds to step 1304, and when the posture is normal, the process proceeds to step 1305. ;
  • step 1304 the posture comparing unit 47 issues an attitude overrun notification (broadcast) to the application modules 71, 72, ..., 7N via the application programming interface 45.
  • Step 1305, wait, delay and then go to step 1301.
  • Figure 28(b) is the working steps in the application module:
  • Step 1307 wait for a delay, or perform other functions
  • Step 1309 posture reminder: the gesture reminding unit 48 generates the alarm data A and/or the attitude correction data R; the input/output processing unit 70 activates the output module driving unit 43 through the application programming interface 45 provided by the operating system running device 4, and controls The alarm signal output module sends a signal to the customer that the user feels identifiable, including alarm information and/or correction suggestions. After the alarm is output, go to step 1306.
  • the posture service module in the operating system operating device 40 includes an attitude detecting unit 46, a posture comparing unit 47, and a data scanning unit 49.
  • the gesture service module 40 works in parallel with the application modules 71, 72, ..., 7N, and the data scanning unit 49 in the gesture service module 40 periodically detects the currently running application module from the application database, such as the application module (eg When the name or identifier of 71) is in the application registry, the gesture detecting unit 46 is activated to operate.
  • the application module (eg, 71) occupies the display screen, the "application status table" in the application database 44 is modified by the application programming interface 45, and the name of the application module (or some identifier assigned by the system) is written. ).
  • the user can configure the "application registry" through the user configuration input module 2, which performs posture detection and reminding when occupying the display screen.
  • Some application modules can also modify the "application registry” by themselves and add themselves to this list.
  • the name or identifier of the application module 71 is in the "application registry", indicating that the application module registers the service provided by the gesture service module in the operating system running device.
  • Figure 29 (a) is the working steps in the operating system operating device:
  • Step 1401a determining whether the application module (eg, 71) occupies the display screen: the data scanning unit 49 queries the "application status table" in the application database 44 for the application module name (or identifier) that is using the display screen, If yes, then step 1401b, otherwise go to step 1405;
  • the application module eg, 71
  • the data scanning unit 49 queries the "application status table" in the application database 44 for the application module name (or identifier) that is using the display screen, If yes, then step 1401b, otherwise go to step 1405;
  • Step 1401b checking whether the application module 71 is in the "application registry”: the data scanning unit 49 queries the application module name (or identifier) registered with the gesture detection service from the "application registry" in the application database 44. ⁇ ), if yes, then go to step 1402, trigger attitude detection unit 46; otherwise, go to step 1405;
  • Step 1402 attitude detection: the attitude detecting unit 46 activates the sensing module driving unit 41, controls the attitude sensing module 1 to acquire the sensing signals S1, S2, and converts into the attitude data D1, D2; the attitude detecting unit 46 reads the sensing The attitude data D1, D2 output by the module driving unit; the attitude detecting unit 46 analyzes the attitude data to obtain a relative attitude parameter G1 and/or an absolute attitude parameter G2;
  • Step 1403 posture comparison: the posture comparison unit 47 in the application module 71 compares the posture parameters G1 and G2 with the threshold values T1 and T2, and when the posture is normal, the first step is step 405; otherwise, the first step is step 404;
  • step 1404 the issued gesture comparing unit 47 issues a gesture overrun notification (register unicast) to the application module 71 via the application programming interface 45.
  • Step 1405 wait, delay and then go to step 1401a.
  • Figure 29 (b) is the work steps in the application module:
  • Step 1406 when the application module 71 occupies the display screen, the input/output processing unit 70 modifies the "application status table" in the application database 44 through the application programming interface 45, and writes the name or identifier of the application module;
  • Step 1407 delay waiting, or performing other functions of the application module
  • Step 1409 posture reminder: the posture reminding unit 48 in the application module 71 generates alarm data A and/or posture correction data R when the posture parameter exceeds the threshold value; the input/output processing unit 70 runs the device through the operating system.
  • the application programming interface 45 is provided to activate the output module driving unit 43, and the control alarm signal output module sends a signal to the customer that the user feels identifiable, including the alarm information and/or the correction suggestion. After the alarm is output, the process proceeds to step 1406.
  • the application module can obtain the judgment of whether the current user's posture is correct from the operating system running device through the monitoring interface (Interface) that implements gesture detection.
  • the application module can also be shipped from the operating system by implementing a delegate of the gesture measurement class (Delegate).
  • the line device obtains a judgment as to whether the current user's posture is correct.
  • the mobile terminal for realizing user posture detection by using the application program is shown in FIG. 30, and includes an attitude sensing module 1, a user configuration input module 2, an alarm signal output module 3, application program modules 71, 72, ..., 7N, a state database. 8. Monitor program module 4'.
  • the attitude sensing module 1 generates the attitude data D; the user configuration input module 2 recognizes the user operation, and generates the user command I; the alarm signal output module 3 issues a signal recognizable by the user.
  • the application modules 71, 72, ..., 7N of the present invention are devices for loading and running applications, particularly loading and running applications that may be used continuously by users for a long time, such as video players, web browsers, and reading. Software, etc.
  • the status database 8 stores the status data of the work.
  • the status data 8 includes an "application registry”, an "application status table”, and a "pose processing flag bit”.
  • the application status table contains information about the working state of the application module in the terminal, such as whether the application module is running, whether it occupies the display screen, and the like.
  • the application registry contains an identifier that detects the necessary application module name or system allocation.
  • the application modules 71, 72, ..., 7N can add their own names or identifiers to the state database 8, indicating that the gesture processing function is registered.
  • the gesture processing flag bit is a flag indicating whether or not the gesture processing module 40' is operating.
  • the monitoring program module 4' includes an input/output processing unit 41', and is connected to the attitude sensing module 1, the user configuration input module 2, and the alarm signal output module 3, and receives the attitude data D and the user instruction I. Generate output data O.
  • the monitoring program module 4' includes an attitude processing unit 40', identifies the position of the user's face according to the attitude data D, generates a posture parameter G, compares the posture parameter G with a corresponding threshold value T, and does not Alarm data A and/or attitude correction data R are generated when correct.
  • the monitoring program module 4' includes a state scanning unit 42' that periodically scans the state database 8 to know the working state of the application modules 71, 72, ..., 7N, in the at least one application module (for example 71) When the display screen is occupied, the attitude processing unit 40' is activated.
  • the attitude sensing module 1 includes a camera and a driving device thereof, and an image of a user's face is generated by one or more front cameras.
  • the signal generates user posture data D1.
  • the input/output processing unit 41' in the monitor program module 4' controls the attitude sensing module 1 to receive the user posture data D1, and the posture processing unit 40' detects the relative position of the user's face and the display screen.
  • the relative attitude parameter G1 is generated, the attitude parameter G1 is compared with the corresponding threshold value T1, and the alarm data A and/or the attitude correction data R are generated when the user posture is incorrect.
  • a mobile terminal that implements user attitude detection by a monitoring program includes a camera and a driving device 11 thereof, and image signals are generated by one or more front cameras. , user posture data D1 is generated.
  • the attitude sensing module 1 further includes a local attitude sensor (such as a gravity sensor, also referred to as a tilt sensor) capable of detecting the plane inclination of the display screen and its driving device 12, and generates an angle sensing signal to generate the local attitude data D2.
  • the input/output processing unit 41' in the monitor program module 4' controls the attitude sensing module 1 to receive the attitude data D1, D2.
  • the attitude processing unit 40' detects the relative position of the user's face and the display screen based on the user's posture data D1, and generates a relative posture parameter G1; the attitude processing unit 40' determines the inclination of the plane of the local display screen based on the local posture data D2.
  • the attitude processing unit 40' generates an absolute attitude parameter G2 based on the inclination of the relative attitude parameter G1 and the plane in which the local display screen is located.
  • the attitude processing unit 40' compares the attitude parameters G1, G2 with corresponding threshold values T1, T2; when the user posture is incorrect, alarm data A and/or attitude correction data R are generated.
  • Fig. 33 shows another embodiment of the mobile terminal which realizes attitude detection by the monitoring program, and is a scheme in which the monitoring program module 4' calls the posture processing unit 40'.
  • the processing unit library 9 includes an attitude processing unit 40' and other processing units (e.g., 91).
  • the posture processing unit 40' detects the posture of the user according to the posture data D, generates the posture parameter G, compares the posture parameter G with the corresponding threshold value T, and generates alarm data A and/or when the user posture is incorrect.
  • Posture correction data R The processing unit library 9 also includes an application programming interface (API) 90 that enables the gesture processing unit 40' and other processing units (e.g., 91) to receive the enable signals and data and output the results.
  • API application programming interface
  • the monitor program module 4' includes an input/output processing unit 41', a status scan unit 42', and a gesture processing call unit 43'.
  • the input/output processing unit 41' is connected to the attitude sensing module 1, the user configuration input module 2, and the alarm signal output module 3, and receives the attitude data D and the user command I to generate output data O.
  • the status scanning unit 42' periodically scans the status database 8 to know the working status of the application modules 71, 72, ..., 7N when at least one of the application modules (e.g., 71) occupies the display screen.
  • the gesture processing calling unit 43' is started.
  • the gesture processing invoking unit 43' is capable of querying the position of the gesture processing unit 40' in the processing unit library 9, issuing an activation signal, and transmitting the data required for the attitude detection and comparison (including the attitude data D1, D2, and the threshold value). T1, T2), and return alarm data A and attitude correction data R.
  • a scheme of adding a library function in the Android operating system is adopted.
  • the Android system running device In the Android system running device, the attitude processing list of the attitude detection, the posture comparison, and the gesture reminding function will be realized.
  • meta 40 exists as a library function running device and provides an API in the Application Framework, which can provide other application modules with the current user's gesture. The correct judgment.
  • the monitor program module 4' performs the functions implemented by the present invention by calling the library function running means.
  • the gesture processing unit 40' is implemented in the manner of a library function running device, including posture detection, posture comparison, gesture reminding function, as a part of the media layer or the core service layer, and provides an API in the CocoaTouch.
  • the API can provide the monitor module 4' with a determination as to whether the current user's gesture is correct.
  • the monitor program module 4' performs the functions implemented by the present invention by calling the library function running means.
  • the gesture processing unit 40' is implemented in the manner of the library function running device, and includes a posture detection, a posture comparison, and a gesture reminding function, so that the device developed by the user can directly call the API related to the function library running device.
  • the monitoring program module 4' of the present invention performs the functions implemented by the present invention by calling a library function running device.
  • the method for realizing user posture detection by the monitoring program module 4' is as shown in FIG. 34, and includes the following steps:
  • Step 1501 configuration input: the input/output processing unit 41' in the monitor program module 4' accepts the user command I from the user configuration input module 2;
  • Step 1502 Application Scan: The status scan unit 42' in the monitor program module 4' periodically scans the status database 8, and activates the pose processing unit 40' based on the information of the status database 8.
  • Step 1503 attitude detection: the input/output processing unit 41' in the monitoring program module 4' controls the attitude sensing module 1 to collect the user posture, and receives the attitude data D1, D2; the posture processing unit 40' analyzes the posture data D1, D2 Obtaining a user relative posture parameter G1 and/or an absolute posture parameter G2;
  • Step 1504 attitude comparison: the attitude processing unit 40' in the monitoring program module 4' compares the attitude parameters G1, G2 with the corresponding threshold values T1, T2, and if the posture is correct, the step 1502 is performed, otherwise the step 1505;
  • Step 1505 posture reminder: the attitude processing unit 40' in the monitoring program module 4' generates alarm data A and/or posture correction data R when the user posture is incorrect; the input/output processing unit 41 in the monitoring program module 4' The output signal (O) triggers the alarm signal output module 3 to send a signal to the customer that the user feels identifiable, including the alarm information and/or correction suggestions. After the alarm is output, the process proceeds to step 1502.
  • the loop execution method may include a step of delay from the step 1504, the step 1505 to the step 1502.
  • Fig. 35 is a flow chart showing the monitoring program module 4' scanning state database 8 obtaining an application command and performing posture processing.
  • the posture processing unit 40' realizes the functions of posture detection, posture comparison, and posture reminding.
  • the monitoring program module 4' runs in parallel with the application modules 71, 72, ..., 7N, and any one of the application modules 71, 72, ..., 7N is occupying the display screen and it is necessary to perform posture detection on the user (for example, long-term presence
  • the "attitude processing flag bit" is set in the state database 8 for starting the gesture processing unit 40', and the flag is cleared when the application module exits the display screen or other situations where the gesture is not required to be detected.
  • Figure 35 (a) shows a paragraph of the working process of the application modules 71, 72, ..., 7N, the steps are as follows:
  • Step 1601 any one of the application modules (for example, 71) sets or clears the "attitude processing flag bit" in the application database 8;
  • step 1602 the application module (eg, 71) runs the application main function
  • FIG 35 (b) shows the working process of the monitoring program module, the steps are as follows:
  • step 1603 the status scanning unit 42' queries the status database 8 to confirm whether the gesture processing flag is set. If yes, proceed to step 1604 to start the gesture processing unit 40', otherwise go to step 1607;
  • Step 1604 posture detection: the input/output processing unit 41' in the monitoring program module 4' controls the attitude sensing module 1 to collect the user posture, and receives the attitude data D1, D2; the posture processing unit 40' analyzes the posture data D1, D2 Obtaining a relative attitude parameter G1 and/or an absolute attitude parameter G2;
  • Step 1605 attitude comparison: the attitude processing unit 40' compares the posture parameters G1, G2 with the threshold values T1, T2, and if the posture is correct, the first step is step 607; otherwise, the step 1606 is entered;
  • Step 1606 posture reminder: the attitude processing unit 40 generates alarm data A and/or attitude correction data R when the attitude parameters G1, G2 exceed the threshold value T1, T2; the input and output processing unit 41' outputs a signal O trigger Alarm information output module 3;
  • Step 1607 wait, delay and then go to step 1603.
  • Fig. 36 is a flow chart showing the state in which the monitoring program module 4' scans the state database 8 to obtain the application state and performs the posture processing.
  • the attitude processing unit 40' implements attitude detection, attitude comparison, and gesture reminding functions.
  • the application modules 71, 72, ..., 7N when occupying the display screen, write the name or identifier of the application module to the state database 8, forming an application state table.
  • the status scanning unit 42' in the monitoring program module 4' obtains an "application registry" in operation, and determines an application module that registers the monitoring function; if the application module occupying the display is registered in the application with a name or identifier In the table, the gesture processing unit is triggered.
  • Figure 36 (a) shows a paragraph of the working process of the application modules 71, 72, ..., 7N, the steps are as follows:
  • Step 1701 the application module (for example, 71) sets or clears the "application status table" in the application database through the application programming interface when occupying the display screen;
  • step 1702 the application module (eg, 71) runs the application main function
  • FIG. 36 (b) shows the working process of the monitoring program module, the steps are as follows:
  • step 1703 the status scanning unit 42' queries the status database to confirm whether the application module name or identifier of the display being used is in the "application registry"? If yes, proceed to step 1704, start the attitude processing unit 40, otherwise go to step 1707;
  • Step 1704 posture detection: the input/output processing unit 41' in the monitoring program module 4' controls the attitude sensing module 1 to collect the user posture, and receives the attitude data D1, D2; the posture processing unit 40' analyzes the posture data D1, D2 Obtaining a relative attitude parameter G1 and/or an absolute attitude parameter G2;
  • Step 1705 posture comparison: the attitude processing unit 40' compares the posture parameters G1, G2 with the threshold values T1, T2, and if the posture is correct, the first step is step 7078; otherwise, the step 1706 is entered;
  • Step 1706 posture reminder: the attitude processing unit 40' generates alarm data A and/or attitude correction data R when the posture parameters G1, G2 exceed the threshold value T1, T2; the input and output processing unit 41' outputs a signal O Triggering an alarm information output module 3;
  • Step 1707 wait, delay and then go to step 1703.
  • the input/output processing unit 41' in the monitor module 4' receives the user command from the user configuration input module 2.
  • the instruction data includes at least one of the following instructions: 1.
  • the application selects the instruction, and the user sets the "application registry".
  • the background service performs gesture detection only when the application is running in the foreground. Some application modules can also actively join this list. By setting this feature is not enabled in some applications. If the foreground application is answering, making a call, listening to radio or other audio content, since the user does not pay attention to the display for a long time, it is not necessary to turn on this function. 2.
  • Threshold command the user sets the attitude parameter threshold. By setting thresholds to meet the needs of different users. In the absence of a user command, the above setting may also be that there is a default setting in the gesture processing unit 40' or the gesture processing invoking unit 43', which is manually set or modified by the user configuration input module 2.
  • the alarm signal output module 3 processes the alarm data A and the attitude correction data R. These data are contained in the output signal O of the input/output processing unit 41' of the monitor program module 4'.
  • the alarm data is generated when the attitude parameter exceeds the threshold, and includes the manner of the alarm, the strength of the alarm, and the length of the alarm. These data are determined according to the degree and time when the user deviates from the correct posture.
  • the attitude correction data is based on the difference between the attitude parameter and the threshold value, and obtains information on how to change the user posture to achieve the correct posture, including distance, angle, and direction of adjustment.
  • the signal output by the alarm output module is a sensory identifiable signal of the user, and includes one or more of the following types: a prompt sound emitted by the audio module, a vibration generated by the vibration device, a graphic or text prompt given through the display screen. .
  • embodiments of the present application can be provided as a method, system, or computer program product.
  • the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment in combination of software and hardware.
  • the application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.
  • the computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device.
  • the apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device.
  • the instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

Abstract

A user posture detection method, device and system, for solving the problem in the prior art that a user in a wrong posture when using a terminal device facing the display screen causes bodily harm. The method can effectively alert a user to maintain a correct posture, preventing the user from being injured when using the electronic device, thus being beneficial to the user. On the other hand, a user maintaining a correct posture will not easily get tired and can use the devices for a long time, improving loyalty of the user to equipment manufacturers and applications.

Description

一种用户姿态检测方法、装置及系统User attitude detection method, device and system
本申请要求在2014年4月16日提交中国专利局，申请号分别为201410153795.X、201410153807.9、201410153783.7、201410153857.7、201410153793.0、201410153791.1、201410153781.8，发明名称分别为“一种终端设备用户姿态检测方法”、“一种终端设备用户姿态检测系统”、“实现用户姿态检测的操作系统运行装置和应用程序模块”、“一种通过操作系统实现用户姿态检测的移动终端和方法”、“一种通过操作系统实现用户姿态检测的移动终端和方法”、“一种通过操作系统实现用户姿态检测的移动终端和方法”、“一种通过监控程序实现用户姿态检测的移动终端和方法”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application is submitted to the Chinese Patent Office on April 16, 2014. The application numbers are 201410153795.X, 201410153807.9, 201410153783.7, 201410153857.7, 201410153793.0, 201410153791.1, 201410153781.8, respectively. The invention names are respectively “a terminal device user attitude detection method”. "A terminal device user attitude detection system", "operating system running device and application module for realizing user posture detection", "a mobile terminal and method for realizing user posture detection through an operating system", "one through an operating system Chinese patent application for realizing user posture detection and method, "a mobile terminal and method for realizing user attitude detection by operating system", and "a mobile terminal and method for realizing user attitude detection by monitoring program" The entire contents are hereby incorporated by reference.
技术领域Technical field
本发明涉及终端设备技术领域，尤指一种用户姿态检测方法、装置及系统。The present invention relates to the field of terminal equipment technologies, and in particular, to a user attitude detection method, apparatus and system.
背景技术Background technique
手机、平板电脑、电子书以及笔记本电脑等移动终端设备已经在现代人的生活中非常普及。用户在使用这些设备时一般都以单手或双手手持这些设备，让设备的显示屏对着自己进行学习、阅读、娱乐等活动。随着大量的应用程序的出现，用户每天使用这些设备的时间越来越长。例如有的用户在地铁上用手机看小说，一般连续使用时间为几十分钟；在平板电脑或手机上看视屏，经常一看就是几个小时。随着4G业务的开通，移动互联网将提供更加丰富多彩的内容，这会让我们使用这些终端设备的时间更长。Mobile devices such as mobile phones, tablets, e-books, and notebook computers have become very popular in modern life. When using these devices, users generally hold these devices with one hand or both hands, so that the device's display screen is aimed at learning, reading, and entertainment. With the advent of a large number of applications, users are using these devices for longer and longer. For example, some users use the mobile phone to read novels on the subway, and the continuous use time is usually several tens of minutes; watching the video on a tablet or mobile phone is often a few hours. With the opening of 4G services, the mobile Internet will provide more colorful content, which will allow us to use these terminal devices for a longer period of time.
虽然使用这些设备让我们很便捷地获得了各种信息、随时可以娱乐，但是，对这些设备的不正确使用方法也给用户带来了很多烦恼。比较典型的一个情形是，用户长时间手持这些设备后，手臂疲惫，设备越放越低，自然而然地头也越来越低。如果颈椎长期处于极度前屈的异常稳定状态，就会对颈椎造成伤害，容易造成颈肩部肌肉僵硬、痉挛，时间久了，就可能会导致颈椎曲度变直、颈椎间盘突出等病症。还有的人喜欢在地铁等拥挤的场所看手机、平板电脑或电子书，由于空间有限，这些设备与眼睛的距离很近。还有一些人习惯在休息的时候卧床阅读。这些不正确的姿态对用户的眼睛和颈椎造成了很大的伤害，而且这些伤害很多是不可逆的，需要服用药物、物理疗法乃至手术才能得到有效的治疗。Although the use of these devices makes it easy for us to get all kinds of information and to be entertained at any time, the incorrect use of these devices also brings a lot of troubles to users. A typical situation is that after the user holds the device for a long time, the arm is exhausted, the device is lowered, and the head is naturally lower and lower. If the cervical vertebra is in an abnormal state of extreme flexion for a long time, it will cause damage to the cervical vertebrae, which may cause stiffness and spasm of the neck and shoulder muscles. For a long time, it may lead to cervical curvature and cervical disc herniation. Others like to watch mobile phones, tablets or e-books in crowded places such as subways. Due to limited space, these devices are close to the eyes. Still others are used to reading in bed while resting. These incorrect postures cause great damage to the user's eyes and cervical spine, and many of these injuries are irreversible and require medication, physical therapy, and surgery to get effective treatment.
由于手机、平板电脑、电子书以及笔记本电脑等移动终端设备在日常生活中使用越来越多，不正确的使用方式对用户身体造成的伤害越来越明显。但是目前终端设备的功能没有避免该问题。开发有效的技术手段减少甚至消除这些不正确情形，可以有效解决 设备使用过程中给用户健康造成的损失。As mobile terminal devices such as mobile phones, tablet computers, e-books, and notebook computers are used more and more in daily life, the damage caused by incorrect use methods to the user's body is becoming more and more obvious. However, the function of the terminal device does not avoid this problem. Develop effective technical means to reduce or even eliminate these incorrect situations, which can effectively solve The loss to the user's health during the use of the device.
发明内容Summary of the invention
本发明实施例提供一种视频处理方法及装置，用以解决现有技术中用户面对显示屏使用终端设备时姿态不正确造成身体伤害的问题。The embodiment of the invention provides a video processing method and device, which are used to solve the problem that the user in the prior art faces the incorrect posture caused by the user facing the display device.
本发明实施例提供的一种用户姿态检测方法，包括：A user gesture detection method provided by an embodiment of the present invention includes:
检测终端设备绝对姿态参数、用户相对姿态参数、用户绝对姿态参数中的至少一个参数；Detecting at least one of an absolute posture parameter, a user relative posture parameter, and a user absolute posture parameter of the terminal device;
所述终端设备绝对姿态参数的检测方法是：姿态传感器采集终端显示屏所处空间状态获得设备姿态传感数据，计算出终端设备绝对姿态参数；The method for detecting the absolute attitude parameter of the terminal device is: the attitude sensor collects the spatial state of the display screen of the terminal to obtain the device attitude sensing data, and calculates the absolute attitude parameter of the terminal device;
所述用户相对姿态参数的检测方法是：用摄像头采集用户图像或视频，从图像信号中识别用户面部的特征点，计算出用户相对姿态参数；The method for detecting the relative posture parameter of the user is: collecting a user image or a video by using a camera, identifying a feature point of the user's face from the image signal, and calculating a relative posture parameter of the user;
所述用户绝对姿态参数，是将所述用户相对姿态参数和所述设备绝对姿态参数相结合；The user absolute posture parameter is a combination of the user relative posture parameter and the device absolute posture parameter;
将至少一个所述姿态参数和该参数的门限值进行比较；Comparing at least one of the attitude parameters with a threshold of the parameter;
当至少一个所述姿态参数超过门限值时，发出提示信号和/或矫正建议信号。A prompt signal and/or a correction suggestion signal is issued when at least one of the attitude parameters exceeds a threshold.
本发明实施例提供的一种用户姿态检测方法，包括：A user gesture detection method provided by an embodiment of the present invention includes:
检测终端设备绝对姿态参数、用户相对姿态参数、用户绝对姿态参数中的至少一个参数；Detecting at least one of an absolute posture parameter, a user relative posture parameter, and a user absolute posture parameter of the terminal device;
所述终端设备绝对姿态参数的检测方法是：姿态传感器采集终端显示屏所处空间状态获得设备姿态传感数据，计算出终端设备绝对姿态参数；The method for detecting the absolute attitude parameter of the terminal device is: the attitude sensor collects the spatial state of the display screen of the terminal to obtain the device attitude sensing data, and calculates the absolute attitude parameter of the terminal device;
所述用户相对姿态参数的检测方法是：用摄像头采集用户图像或视频，从图像信号中识别用户面部的特征点，计算出用户相对姿态参数；The method for detecting the relative posture parameter of the user is: collecting a user image or a video by using a camera, identifying a feature point of the user's face from the image signal, and calculating a relative posture parameter of the user;
所述用户绝对姿态参数，是将所述用户相对姿态参数和所述设备绝对姿态参数相结合；The user absolute posture parameter is a combination of the user relative posture parameter and the device absolute posture parameter;
将至少一个所述姿态参数和该参数的门限值进行比较；Comparing at least one of the attitude parameters with a threshold of the parameter;
以上步骤循环进行，当至少一个所述姿态参数连续多次循环超过门限值时，发出提示信号和/或矫正建议信号。The above steps are performed cyclically, and when at least one of the attitude parameters repeatedly exceeds a threshold value for a plurality of times, a prompt signal and/or a correction suggestion signal is issued.
本发明实施例提供的一种用户姿态检测系统，包括：A user attitude detection system provided by an embodiment of the present invention includes:
一个或多个面向用户的摄像头，获得用户面部图像或视频；One or more user-facing cameras that obtain user facial images or videos;
一个或多个姿态传感器，获得终端设备的绝对姿态参数，所述终端设备的绝对姿态参数反映屏幕所在平面的水平倾角； One or more attitude sensors obtain an absolute attitude parameter of the terminal device, and the absolute attitude parameter of the terminal device reflects a horizontal inclination of a plane where the screen is located;
一个信息处理模块，包含姿态传感数据接口、图像数据接口、姿态提醒接口；An information processing module includes an attitude sensing data interface, an image data interface, and a gesture reminding interface;
一个姿态提醒模块，包含信息输出装置；a gesture reminding module, comprising an information output device;
所述信息处理模块通过所述姿态传感数据接口获得所述终端设备绝对姿态参数；The information processing module obtains the absolute posture parameter of the terminal device by using the attitude sensing data interface;
所述信息处理模块通过所述图像数据接口获得所述用户面部图像或视频的数据；The information processing module obtains data of the user's facial image or video through the image data interface;
所述信息处理模块运用所述终端设备绝对姿态参数和所述用户面部图像或视频的数据计算用户姿态参数，对姿态进行判断，当姿态不正确时通过所述姿态提醒接口发出信号，触发所述姿态提醒模块；The information processing module calculates the user posture parameter by using the terminal device absolute posture parameter and the data of the user facial image or video, and determines the posture, and sends a signal through the posture reminding interface when the posture is incorrect, triggering the Gesture reminder module;
所述姿态提醒模块向用户发出感官可识别的信号，包括提示信号和/或矫正建议信号。The gesture reminder module sends a sensory identifiable signal to the user, including a prompt signal and/or a correction suggestion signal.
本发明实施例提供的第一种实现用户姿态检测的操作系统运行装置，包括：传感模块驱动单元、输入模块驱动单元、输出模块驱动单元、姿态服务模块、应用数据库、应用程序编程接口；The operating system running device for implementing user posture detection according to the embodiment of the present invention includes: a sensing module driving unit, an input module driving unit, an output module driving unit, a posture service module, an application database, and an application programming interface;
所述传感模块驱动单元输出传感控制信号、输入传感信号；The sensing module driving unit outputs a sensing control signal and an input sensing signal;
所述输入模块驱动单元输出配置控制信号、接收用户指令；The input module driving unit outputs a configuration control signal and receives a user instruction;
所述输出模块驱动单元输出告警控制信号、以及包含报警数据和/或姿态矫正数据的输出信号；The output module driving unit outputs an alarm control signal and an output signal including alarm data and/or attitude correction data;
所述姿态服务模块中包含姿态检测单元、姿态比较单元、姿态提醒单元；The attitude service module includes an attitude detecting unit, a posture comparing unit, and a posture reminding unit;
所述姿态检测单元启动传感模块驱动单元，读取所述传感模块驱动单元输出的姿态数据，识别用户面部的位置，产生姿态参数；所述姿态比较单元将所述姿态参数与对应的门限值进行比较；所述姿态提醒单元在用户姿态不正确时产生报警数据和/或姿态矫正数据，并启动所述输出模块驱动单元；The posture detecting unit activates the sensing module driving unit, reads posture data output by the sensing module driving unit, recognizes a position of the user's face, and generates a posture parameter; the posture comparing unit sets the posture parameter and the corresponding door The limit value is compared; the posture reminding unit generates alarm data and/or attitude correction data when the user posture is incorrect, and starts the output module driving unit;
所述应用数据库存储工作数据和默认设置数据。The application database stores work data and default settings data.
本发明实施例提供的第二种实现用户姿态检测的操作系统运行装置，包括：传感模块驱动单元、输入模块驱动单元、输出模块驱动单元、姿态服务模块、应用数据库、应用程序编程接口；The operating system running device for implementing user posture detection according to the embodiment of the present invention comprises: a sensing module driving unit, an input module driving unit, an output module driving unit, a posture service module, an application database, and an application programming interface;
所述传感模块驱动单元输出传感控制信号、输入传感信号；The sensing module driving unit outputs a sensing control signal and an input sensing signal;
所述输入模块驱动单元输出配置控制信号、接收用户指令；The input module driving unit outputs a configuration control signal and receives a user instruction;
所述输出模块驱动单元输出告警控制信号、以及包含报警数据和/或姿态矫正数据的输出信号；The output module driving unit outputs an alarm control signal and an output signal including alarm data and/or attitude correction data;
所述姿态服务模块中包含姿态检测单元和姿态比较单元；所述姿态检测单元启动所述传感模块驱动单元，读取所述传感模块驱动单元输出的姿态数据，识别用户面部的位 置，产生姿态参数；所述姿态比较单元将所述姿态参数与对应的门限值进行比较；在姿态参数超出门限值时通过所述应用程序编程接口发出姿态超限通知；The attitude service module includes an attitude detection unit and a posture comparison unit; the attitude detection unit activates the sensing module driving unit, reads posture data output by the sensing module driving unit, and identifies a position of the user's face. Positioning, generating a posture parameter; the posture comparison unit compares the attitude parameter with a corresponding threshold value; and when the posture parameter exceeds the threshold value, sends an attitude overrun notification through the application programming interface;
所述应用数据库存储工作数据和默认设置数据。The application database stores work data and default settings data.
本发明实施例提供的第三种实现用户姿态检测的操作系统运行装置，包括：传感模块驱动单元、输入模块驱动单元、输出模块驱动单元、姿态服务模块、应用数据库、应用程序编程接口；The operating system running device for implementing user posture detection according to the embodiment of the present invention comprises: a sensing module driving unit, an input module driving unit, an output module driving unit, a posture service module, an application database, and an application programming interface;
所述传感模块驱动单元输出传感控制信号、输入传感信号；The sensing module driving unit outputs a sensing control signal and an input sensing signal;
所述输入模块驱动单元输出配置控制信号、接收用户指令；The input module driving unit outputs a configuration control signal and receives a user instruction;
所述输出模块驱动单元输出告警控制信号、以及包含报警数据和/或姿态矫正数据的输出信号；The output module driving unit outputs an alarm control signal and an output signal including alarm data and/or attitude correction data;
所述姿态服务模块中包含姿态检测单元，所述姿态检测单元启动所述传感模块驱动单元，读取所述传感模块驱动单元输出的姿态数据，识别用户面部的位置，产生姿态参数，将所述姿态参数保存在应用数据库中；The posture service module includes an attitude detecting unit, the posture detecting unit starts the sensing module driving unit, reads posture data output by the sensing module driving unit, recognizes a position of a user's face, and generates a posture parameter, and The pose parameter is saved in an application database;
所述应用数据库存储工作数据和默认设置数据。The application database stores work data and default settings data.
本发明实施例提供的一种应用程序模块，用于第二种实现用户姿态检测的操作系统运行装置，包括：包含姿态提醒单元和输入输出处理单元；An application program module is provided for the second embodiment of the operating system running device for detecting user posture, comprising: a posture reminding unit and an input and output processing unit;
所述姿态提醒单元产生报警数据和/或姿态矫正数据；The gesture reminding unit generates alarm data and/or attitude correction data;
所述输入输出处理单元通过应用程序编程接口读取或设置应用数据库中的工作数据；The input/output processing unit reads or sets working data in an application database through an application programming interface;
所述输入输出处理单元通过应用程序编程接口接收来自操作系统运行装置的姿态超限通知；The input/output processing unit receives an attitude overrun notification from an operating system running device through an application programming interface;
所述输入输出处理单元通过应用程序编程接口启动输入模块驱动单元，接收用户指令；The input/output processing unit activates an input module driving unit through an application programming interface to receive a user instruction;
所述输入输出处理单元通过应用程序编程接口启动输出模块驱动单元，发出包含报警数据和/或姿态矫正数据的信号。The input/output processing unit activates the output module drive unit through an application programming interface to issue a signal containing alarm data and/or attitude correction data.
本发明实施例提供的一种应用程序模块，用于第三种实现用户姿态检测的操作系统运行装置，包括：姿态比较单元、姿态提醒单元和输入输出处理单元；An application program module is provided for the third operating system running device for realizing user posture detection, comprising: a posture comparison unit, a posture reminding unit, and an input/output processing unit;
所述输入输出处理单元，通过所述应用程序编程接口设置或读取应用数据库中的工作数据；The input/output processing unit sets or reads working data in an application database through the application programming interface;
所述姿态比较单元将所述姿态参数与对应的门限值进行比较；The attitude comparison unit compares the attitude parameter with a corresponding threshold value;
所述姿态提醒单元产生报警数据和/或姿态矫正数据； The gesture reminding unit generates alarm data and/or attitude correction data;
所述输入输出处理单元通过应用程序编程接口启动输入模块驱动单元，接收用户指令；The input/output processing unit activates an input module driving unit through an application programming interface to receive a user instruction;
所述输入输出处理单元通过应用程序编程接口启动输出模块驱动单元，发出包含报警数据和/或姿态矫正数据的信号。The input/output processing unit activates the output module drive unit through an application programming interface to issue a signal containing alarm data and/or attitude correction data.
本发明实施例提供的一种通过操作系统实现用户姿态检测的移动终端，包含姿态传感模块、用户配置输入模块、告警信号输出模块、操作系统运行装置、显示屏、一个或多个应用程序模块；A mobile terminal for implementing user posture detection by using an operating system includes an attitude sensing module, a user configuration input module, an alarm signal output module, an operating system running device, a display screen, and one or more application modules. ;
所述姿态传感模块产生姿态传感信号；The attitude sensing module generates an attitude sensing signal;
所述用户配置输入模块识别用户操作，将用户指令传递至操作系统运行装置；The user configuration input module identifies a user operation and transmits the user instruction to an operating system running device;
所述告警信号输出模块发出用户感官可识别的信号；The alarm signal output module sends a signal recognizable by the user's senses;
所述操作系统运行装置中包含传感模块驱动单元、输入模块驱动单元、输出模块驱动单元；所述传感模块驱动单元控制所述姿态传感模块工作、并将所述姿态传感模块输出的姿态传感信号转换为姿态数据；所述输入模块驱动单元控制所述用户配置输入模块工作，接收用户指令；所述输出模块驱动单元输出包含报警数据和/或姿态矫正数据的信号，控制所述告警信号输出模块工作；The operating system running device includes a sensing module driving unit, an input module driving unit, and an output module driving unit; the sensing module driving unit controls the operation of the attitude sensing module, and outputs the attitude sensing module The attitude sensing signal is converted into attitude data; the input module driving unit controls the user configuration input module to operate, and receives a user instruction; the output module driving unit outputs a signal including alarm data and/or attitude correction data, and controls the The alarm signal output module works;
所述操作系统运行装置中还包含姿态服务模块，所述姿态服务模块中包含姿态检测单元、姿态比较单元和姿态提醒单元；所述姿态检测单元启动所述传感模块驱动单元，读取所述传感模块驱动单元输出的姿态数据，识别用户面部的位置，产生姿态参数；所述姿态比较单元，将所述姿态参数与对应的门限值进行比较；所述姿态提醒单元，产生报警数据和/或姿态矫正数据；The operating system running device further includes a posture service module, wherein the posture service module includes an attitude detecting unit, a posture comparing unit, and a posture reminding unit; the posture detecting unit starts the sensing module driving unit, and reads the The attitude data output by the sensing module driving unit identifies the position of the user's face to generate a posture parameter; the attitude comparison unit compares the attitude parameter with a corresponding threshold value; the gesture reminding unit generates alarm data and / or posture correction data;
所述操作系统运行装置中还包含应用数据库，所述应用数据库存储工作数据和默认设置数据。The operating system running device further includes an application database, and the application database stores working data and default setting data.
本发明实施例提供的一种通过操作系统实现用户姿态检测的方法，包括：A method for implementing user gesture detection by using an operating system according to an embodiment of the present invention includes:
操作系统运行装置中的姿态检测单元启动传感模块驱动单元采集传感信号，并转换为姿态数据；The attitude detecting unit in the operating system running device starts the sensing module driving unit to collect the sensing signal and converts into the attitude data;
操作系统运行装置中的姿态检测单元读取传感模块驱动单元输出的姿态数据；The attitude detecting unit in the operating system running device reads the attitude data output by the sensing module driving unit;
操作系统运行装置中的姿态检测单元对姿态数据进行分析，得出相对姿态参数和/或绝对姿态参数；The attitude detecting unit in the operating system running device analyzes the attitude data to obtain a relative attitude parameter and/or an absolute posture parameter;
操作系统运行装置中的姿态比较单元将所述姿态参数和用户姿态门限值进行姿态比较；The attitude comparison unit in the operating system running device compares the posture parameter and the user posture threshold value;
操作系统运行装置中的姿态提醒单元在所述姿态参数超过门限值时，生成报警数据 和/或姿态矫正数据。The gesture reminding unit in the operating system running device generates the alarm data when the attitude parameter exceeds the threshold And/or posture correction data.
本发明实施例提供的一种通过操作系统实现用户姿态检测的移动终端，包含姿态传感模块、用户配置输入模块、告警信号输出模块、操作系统运行装置、显示屏、一个或多个应用程序模块；A mobile terminal for implementing user posture detection by using an operating system includes an attitude sensing module, a user configuration input module, an alarm signal output module, an operating system running device, a display screen, and one or more application modules. ;
所述姿态传感模块产生姿态传感信号；The attitude sensing module generates an attitude sensing signal;
所述用户配置输入模块，识别用户操作，将用户指令传递至操作系统运行装置和至少一个所述应用程序模块；The user configuration input module identifies a user operation, and transmits the user instruction to the operating system running device and the at least one application module;
所述告警信号输出模块发出用户感官可识别的信号；The alarm signal output module sends a signal recognizable by the user's senses;
所述操作系统运行装置中包含传感模块驱动单元、输入模块驱动单元、输出模块驱动单元；所述传感模块驱动单元控制所述姿态传感模块工作、并将所述姿态传感模块输出的姿态传感信号转换为姿态数据；所述输入模块驱动单元控制所述用户配置输入模块工作，接收用户指令；所述输出模块驱动单元输出包含报警数据和/或姿态矫正数据的信号，控制所述告警信号输出模块工作；The operating system running device includes a sensing module driving unit, an input module driving unit, and an output module driving unit; the sensing module driving unit controls the operation of the attitude sensing module, and outputs the attitude sensing module The attitude sensing signal is converted into attitude data; the input module driving unit controls the user configuration input module to operate, and receives a user instruction; the output module driving unit outputs a signal including alarm data and/or attitude correction data, and controls the The alarm signal output module works;
所述操作系统运行装置中还包含姿态服务模块，所述姿态服务模块中包含姿态检测单元和姿态比较单元；所述姿态检测单元启动所述传感模块驱动单元，读取所述传感模块驱动单元输出的姿态数据，识别用户面部的位置，产生姿态参数；所述姿态比较单元将所述姿态参数与对应的门限值进行比较；在姿态参数超出门限值时向至少一个所述应用程序模块发出姿态超限通知；The operating system running device further includes a posture service module, wherein the attitude service module includes an attitude detecting unit and a posture comparing unit; the posture detecting unit starts the sensing module driving unit, and reads the sensing module driving The attitude data output by the unit, the position of the user's face is recognized, and the attitude parameter is generated; the attitude comparison unit compares the attitude parameter with a corresponding threshold value; and when the attitude parameter exceeds the threshold value, at least one of the applications The module issues a gesture of over-limitation;
所述操作系统运行装置包含应用程序编程接口；The operating system running device includes an application programming interface;
所述操作系统运行装置中还包含应用数据库，所述应用数据库存储工作数据和默认设置数据；The operating system running device further includes an application database, where the application database stores working data and default setting data;
至少一个所述应用程序模块中包含姿态提醒单元、输入输出处理单元；所述姿态提醒单元，产生报警数据和/或姿态矫正数据；所述输入输出处理单元，通过所述应用程序编程接口设置应用数据库中的工作数据、接收来自操作系统运行装置的姿态超限通知、启动输入模块驱动单元、启动输出模块驱动单元。At least one of the application modules includes a gesture reminding unit, an input and output processing unit; the gesture reminding unit generates alarm data and/or attitude correction data; the input/output processing unit sets an application through the application programming interface The working data in the database, receiving the attitude overrun notification from the operating system running device, starting the input module driving unit, and starting the output module driving unit.
本发明实施例提供的一种通过操作系统实现用户姿态检测的方法，包括：A method for implementing user gesture detection by using an operating system according to an embodiment of the present invention includes:
操作系统运行装置中的姿态检测单元启动传感模块驱动单元采集传感信号，并转换为姿态数据；The attitude detecting unit in the operating system running device starts the sensing module driving unit to collect the sensing signal and converts into the attitude data;
操作系统运行装置中的姿态检测单元读取传感模块驱动单元输出的姿态数据；The attitude detecting unit in the operating system running device reads the attitude data output by the sensing module driving unit;
操作系统运行装置中的姿态检测单元对姿态数据进行分析，得出相对姿态参数和/或绝对姿态参数； The attitude detecting unit in the operating system running device analyzes the attitude data to obtain a relative attitude parameter and/or an absolute posture parameter;
操作系统运行装置中的姿态比较单元对姿态参数和门限值进行比较；在所述姿态参数超过门限值时，通过应用程序编程接口向至少一个应用程序模块发出姿态超限通知；The attitude comparison unit in the operating system running device compares the attitude parameter and the threshold value; when the posture parameter exceeds the threshold value, sends an attitude overrun notification to the at least one application module through the application programming interface;
占用显示屏的应用程序模块中的输入输出处理单元通过应用程序编程接口接收姿态超限通知；The input/output processing unit in the application module occupying the display screen receives the gesture overrun notification through the application programming interface;
占用显示屏的应用程序模块中的姿态提醒单元在所述姿态参数超过门限值时，生成报警数据和/或姿态矫正数据。The gesture reminding unit in the application module occupying the display generates alarm data and/or attitude correction data when the attitude parameter exceeds a threshold.
本发明实施例提供的一种通过操作系统实现用户姿态检测的移动终端，包含姿态传感模块、用户配置输入模块、告警信号输出模块、操作系统运行装置、显示屏、一个或多个应用程序模块；A mobile terminal for implementing user posture detection by using an operating system includes an attitude sensing module, a user configuration input module, an alarm signal output module, an operating system running device, a display screen, and one or more application modules. ;
所述姿态传感模块产生姿态传感信号；The attitude sensing module generates an attitude sensing signal;
所述用户配置输入模块，识别用户操作，将用户指令传递至所述操作系统运行装置和至少一个所述应用程序模块；The user configuration input module identifies a user operation, and transmits user instructions to the operating system running device and at least one of the application modules;
所述告警信号输出模块发出用户感官可识别的信号；The alarm signal output module sends a signal recognizable by the user's senses;
所述操作系统运行装置中包含传感模块驱动单元、输入模块驱动单元、输出模块驱动单元；所述传感模块驱动单元控制所述姿态传感模块工作、并将所述姿态传感模块输出的姿态传感信号转换为姿态数据；所述输入模块驱动单元控制所述用户配置输入模块工作，接收用户指令；所述输出模块驱动单元输出包含报警数据和/或姿态矫正数据的信号，控制所述告警信号输出模块工作；The operating system running device includes a sensing module driving unit, an input module driving unit, and an output module driving unit; the sensing module driving unit controls the operation of the attitude sensing module, and outputs the attitude sensing module The attitude sensing signal is converted into attitude data; the input module driving unit controls the user configuration input module to operate, and receives a user instruction; the output module driving unit outputs a signal including alarm data and/or attitude correction data, and controls the The alarm signal output module works;
所述操作系统运行装置中还包含姿态服务模块，所述姿态服务模块中包含姿态检测单元；所述姿态检测单元启动所述传感模块驱动单元，读取所述传感模块驱动单元输出的姿态数据，识别用户面部的位置，产生姿态参数；The operating system running device further includes a posture service module, wherein the attitude service module includes an attitude detecting unit; the posture detecting unit starts the sensing module driving unit, and reads a posture of the sensing unit driving unit Data, identifying the location of the user's face, and generating gesture parameters;
所述操作系统运行装置包含应用程序编程接口；The operating system running device includes an application programming interface;
至少一个所述应用程序模块中包含姿态比较单元、姿态提醒单元、输入输出处理单元；所述姿态比较单元，将所述姿态参数与对应的门限值进行比较；所述姿态提醒单元，产生报警数据和/或姿态矫正数据；所述输入输出处理单元，通过所述应用程序编程接口读取或设置应用数据库中工作数据、启动输入模块驱动单元、启动输出模块驱动单元；At least one of the application modules includes a posture comparison unit, a posture reminding unit, and an input/output processing unit; the posture comparison unit compares the posture parameter with a corresponding threshold value; and the posture reminding unit generates an alarm Data and/or posture correction data; the input/output processing unit reads or sets working data in the application database, starts the input module driving unit, and starts the output module driving unit through the application programming interface;
所述操作系统运行装置中还包含应用数据库，所述应用数据库存储工作中产生的数据和默认设置数据。The operating system running device further includes an application database, and the application database stores data generated in the work and default setting data.
本发明实施例提供的一种通过操作系统实现用户姿态检测的方法，包括：A method for implementing user gesture detection by using an operating system according to an embodiment of the present invention includes:
操作系统运行装置中的姿态检测单元启动传感模块驱动单元采集传感信号，并转换为姿态数据； The attitude detecting unit in the operating system running device starts the sensing module driving unit to collect the sensing signal and converts into the attitude data;
操作系统运行装置中的姿态检测单元读取传感模块驱动单元输出的姿态数据；The attitude detecting unit in the operating system running device reads the attitude data output by the sensing module driving unit;
操作系统运行装置中的姿态检测单元对姿态数据进行分析，得出相对姿态参数和/或绝对姿态参数；The attitude detecting unit in the operating system running device analyzes the attitude data to obtain a relative attitude parameter and/or an absolute posture parameter;
操作系统运行装置中的姿态检测单元将姿态参数保存在应用数据库中；The attitude detecting unit in the operating system running device saves the posture parameter in the application database;
应用程序模块中的输入输出处理单元通过应用程序编程接口从应用数据库中读取姿态参数和门限值；The input and output processing unit in the application module reads the attitude parameter and the threshold from the application database through the application programming interface;
应用程序模块中的姿态比较单元将所述姿态参数和门限值进行姿态比较；An attitude comparison unit in the application module compares the attitude parameter with a threshold value;
应用程序模块中的姿态提醒单元在所述姿态参数超过门限值时，生成报警数据和/或姿态矫正数据。The gesture reminding unit in the application module generates alarm data and/or attitude correction data when the attitude parameter exceeds a threshold.
本发明实施例提供的一种通过监控程序实现用户姿态检测的移动终端，包含姿态传感模块、用户配置输入模块、告警信号输出模块、一个或多个应用程序模块、状态数据库、监控程序模块；A mobile terminal for implementing user posture detection by using a monitoring program includes an attitude sensing module, a user configuration input module, an alarm signal output module, one or more application modules, a state database, and a monitoring program module.
所述姿态传感模块产生姿态数据；The attitude sensing module generates attitude data;
所述用户配置输入模块识别用户操作，产生用户指令；The user configuration input module identifies a user operation and generates a user instruction;
所述告警信号输出模块发出用户感官可识别的信号；The alarm signal output module sends a signal recognizable by the user's senses;
所述状态数据库中存储所述应用程序模块的状态数据；Storing state data of the application module in the state database;
所述监控程序模块中包含输入输出处理单元、姿态处理单元、状态扫描单元；The monitoring program module includes an input and output processing unit, an attitude processing unit, and a status scanning unit;
所述输入输出处理单元与所述姿态传感模块、用户配置输入模块、告警信号输出模块相连接，接收所述姿态数据、接收所述用户指令，产生输出数据；The input/output processing unit is connected to the attitude sensing module, the user configuration input module, and the alarm signal output module, and receives the posture data, receives the user instruction, and generates output data;
所述姿态处理单元根据姿态数据识别用户面部的位置，产生姿态参数，将所述姿态参数与对应的门限值进行比较，在用户姿态不正确时产生报警数据和/或姿态矫正数据；The posture processing unit identifies a position of the user's face according to the posture data, generates a posture parameter, compares the posture parameter with a corresponding threshold value, and generates alarm data and/or posture correction data when the user posture is incorrect;
所述状态扫描单元周期性地扫描所述状态数据库，根据状所述态数据库中的状态数据启动姿态处理单元。The state scanning unit periodically scans the state database, and starts the gesture processing unit according to the state data in the state database.
本发明实施例提供的一种通过监控程序实现用户姿态检测的移动终端，包含姿态传感模块、用户配置输入模块、告警信号输出模块、一个或多个应用程序模块、状态数据库、处理单元库、监控程序模块；A mobile terminal for realizing user posture detection by using a monitoring program, comprising an attitude sensing module, a user configuration input module, an alarm signal output module, one or more application modules, a state database, a processing unit library, Monitoring program module;
所述姿态传感模块产生姿态数据；The attitude sensing module generates attitude data;
所述用户配置输入模块识别用户操作，产生用户指令；The user configuration input module identifies a user operation and generates a user instruction;
所述告警信号输出模块发出用户感官可识别的信号；The alarm signal output module sends a signal recognizable by the user's senses;
所述状态数据库中存储所述应用程序模块的状态数据；Storing state data of the application module in the state database;
所述处理单元库包含姿态处理单元、应用程序编程接口； The processing unit library includes an attitude processing unit and an application programming interface;
所述姿态处理单元根据姿态数据识别用户面部的位置，产生姿态参数，将所述姿态参数与对应的门限值进行比较，在用户姿态不正确时产生报警数据和/或姿态矫正数据；The posture processing unit identifies a position of the user's face according to the posture data, generates a posture parameter, compares the posture parameter with a corresponding threshold value, and generates alarm data and/or posture correction data when the user posture is incorrect;
所述应用程序编程接口使姿态处理单元能够接收启动信号和数据、并输出结果；The application programming interface enables the gesture processing unit to receive the enable signal and data and output the result;
所述监控程序模块中包含输入输出处理单元、姿态处理调用单元、状态扫描单元；The monitoring program module includes an input and output processing unit, a gesture processing calling unit, and a state scanning unit;
所述输入输出处理单元与所述姿态传感模块、用户配置输入模块、告警信号输出模块相连接，接收所述姿态数据、接收所述用户指令，产生输出数据；The input/output processing unit is connected to the attitude sensing module, the user configuration input module, and the alarm signal output module, and receives the posture data, receives the user instruction, and generates output data;
所述姿态处理调用单元能够查询姿态处理单元在处理单元库中的位置、发出启动信号、并传递姿态检测和比较所需要的姿态数据和门限值；The gesture processing calling unit is capable of querying a position of the gesture processing unit in the processing unit library, issuing a start signal, and transmitting attitude data and a threshold value required for posture detection and comparison;
所述状态扫描模块周期性地扫描状态数据库，根据状态数据库中的状态数据启动姿态处理单元。The status scanning module periodically scans the status database and initiates an attitude processing unit based on the status data in the status database.
本发明实施例提供的一种通过监控程序实现用户姿态检测的方法，包括：A method for implementing user gesture detection by using a monitoring program according to an embodiment of the present invention includes:
监控程序模块中的状态扫描单元，周期性地扫描状态数据库，根据状态数据库中的状态信息启动姿态处理单元；a state scanning unit in the monitoring program module, periodically scanning the state database, and starting the gesture processing unit according to the state information in the state database;
监控程序模块中的输入输出处理单元控制姿态传感模块采集用户姿态，接收姿态数据；姿态处理单元对姿态数据进行分析，得出相对姿态参数和/或绝对姿态参数；The input/output processing unit in the monitoring program module controls the attitude sensing module to collect the user posture and receive the attitude data; the attitude processing unit analyzes the attitude data to obtain a relative attitude parameter and/or an absolute posture parameter;
监控程序模块中的姿态处理单元将用户姿态参数和对应的门限值进行比较；The attitude processing unit in the monitor module compares the user attitude parameter with the corresponding threshold value;
监控程序模块中的姿态处理单元在用户姿态不正确时，生成报警数据和/或姿态矫正数据；输入输出处理单元输出信号触发告警信号输出模块向客户发出用户感觉可识别的信号，包含报警信息和/或矫正建议。The attitude processing unit in the monitoring program module generates alarm data and/or attitude correction data when the user posture is incorrect; the input and output processing unit output signal triggers the alarm signal output module to send a signal to the customer that the user feels identifiable, including the alarm information and / or correction suggestions.
本发明实施例提供的一种监控程序模块，包含输入输出处理单元、姿态处理单元、状态扫描单元；A monitoring program module provided by an embodiment of the present invention includes an input/output processing unit, a posture processing unit, and a state scanning unit;
所述输入输出处理单元与移动终端中的姿态传感模块、用户配置输入模块、告警信号输出模块相连接，接收所述姿态传感模块产生的姿态数据，接收所述用户配置输入模块产生的用户指令，并产生输出数据；The input/output processing unit is connected to the attitude sensing module, the user configuration input module, and the alarm signal output module in the mobile terminal, and receives the posture data generated by the attitude sensing module, and receives the user generated by the user configuration input module. Command and generate output data;
所述姿态处理单元根据所述姿态数据识别用户面部的位置，产生姿态参数，将所述姿态参数与对应的门限值进行比较，在用户姿态不正确时产生报警数据和/或姿态矫正数据；The posture processing unit identifies a position of the user's face according to the posture data, generates a posture parameter, compares the posture parameter with a corresponding threshold value, and generates alarm data and/or posture correction data when the user posture is incorrect;
所述状态扫描单元周期性地扫描所述移动终端中的状态数据库，根据状所述态数据库中的状态数据启动姿态处理单元。The state scanning unit periodically scans the state database in the mobile terminal, and starts the gesture processing unit according to the state data in the state database.
本发明实施例提供的一种监控程序模块，包含输入输出处理单元、姿态处理调用单元、状态扫描单元； A monitoring program module provided by an embodiment of the present invention includes an input/output processing unit, a gesture processing calling unit, and a state scanning unit;
所述输入输出处理单元与移动终端中的姿态传感模块、用户配置输入模块、告警信号输出模块相连接，接收所述姿态传感模块产生的姿态数据，接收所述用户配置输入模块产生的用户指令，并产生输出数据；The input/output processing unit is connected to the attitude sensing module, the user configuration input module, and the alarm signal output module in the mobile terminal, and receives the posture data generated by the attitude sensing module, and receives the user generated by the user configuration input module. Command and generate output data;
所述姿态处理调用单元能够查询姿态处理单元在所述移动终端的处理单元库中的位置、发出启动信号、并传递姿态检测和比较所需要的姿态数据和门限值；The gesture processing calling unit is capable of querying a position of the gesture processing unit in the processing unit library of the mobile terminal, issuing a start signal, and transmitting posture data and a threshold value required for posture detection and comparison;
所述状态扫描模块周期性地扫描所述移动终端中的状态数据库，根据状态数据库中的状态数据启动姿态处理单元，使所述姿态处理单元根据姿态数据识别用户面部的位置，产生姿态参数，将所述姿态参数与对应的门限值进行比较，在用户姿态不正确时产生报警数据和/或姿态矫正数据。The state scanning module periodically scans the state database in the mobile terminal, and starts the gesture processing unit according to the state data in the state database, so that the gesture processing unit identifies the location of the user's face according to the posture data, and generates a posture parameter, which will The attitude parameter is compared with a corresponding threshold value, and alarm data and/or attitude correction data is generated when the user posture is incorrect.
本发明实施例提供一种用户姿态检测方法、装置及系统，该方法通过对用户进行有效的提醒，可使用户一直维持一个比较正确的姿态。这可以让用户不会因为使用这些电子设备对身体造成伤害，因此有益于用户。另一方面，保持正确的姿态，用户避免疲惫，可以较长时间使用这些设备，提高用户对设备生产者和应用程序的黏着度。The embodiment of the invention provides a method, a device and a system for detecting a user's posture, which can enable the user to maintain a relatively correct posture by effectively reminding the user. This can be beneficial to the user because the user does not harm the body by using these electronic devices. On the other hand, maintaining the correct posture, users avoid fatigue, can use these devices for a long time, and improve the user's adhesion to equipment producers and applications.
附图说明DRAWINGS
图1是终端设备用户姿态检测系统；Figure 1 is a terminal device user attitude detection system;
图2是终端设备用户姿态检测方法实施例的流程图；2 is a flowchart of an embodiment of a method for detecting a posture of a terminal device user;
图3是终端设备用户姿态检测方法另一实施例的流程图；3 is a flowchart of another embodiment of a terminal device user attitude detecting method;
图4是摄像头在终端设备面板上的位置和功能示意图；4 is a schematic diagram of the position and function of the camera on the panel of the terminal device;
图5是姿态传感器在终端设备上的位置和功能示意图；Figure 5 is a schematic diagram of the position and function of the attitude sensor on the terminal device;
图6是单目机器视觉方法面部位置的获取原理图；Figure 6 is a schematic diagram of the acquisition of the face position of the monocular machine vision method;
图7是特征值提取法面部位置的获取原理图；7 is a schematic diagram of acquisition of a face position of a feature value extraction method;
图8是使用距离和角度参数的实施例流程图；Figure 8 is a flow diagram of an embodiment using distance and angle parameters;
图9是使用面积和角度参数的实施例流程图；Figure 9 is a flow chart of an embodiment using area and angle parameters;
图10是终端设备用户姿态检测系统的一个实施例；Figure 10 is an embodiment of a terminal device user attitude detecting system;
图11是有线连接远程信息处理的终端设备姿态检测系统实施例；11 is an embodiment of a terminal device attitude detecting system for wired connection telematics;
图12是无线连接远程信息处理的终端设备姿态检测系统实施例；12 is an embodiment of a terminal device attitude detection system for wirelessly connecting telematics;
图13是数据传送网络连接远程信息处理的终端设备姿态检测系统实施例；13 is an embodiment of a terminal device attitude detection system for data transmission network connection telematics;
图14是通过操作系统实现用户姿态检测的移动终端；14 is a mobile terminal that implements user gesture detection by an operating system;
图15是姿态传感模块使用摄像头方案；Figure 15 is a view of the attitude sensing module using a camera;
图16是姿态传感模块使用摄像头和倾角传感器方案；Figure 16 is a gesture sensing module using a camera and tilt sensor scheme;
图17是实现用户姿态检测的操作系统运行装置和应用程序模块的工作过程； 17 is a working process of an operating system running device and an application module for realizing user posture detection;
图18是操作系统运行装置实现姿态检测、姿态比较和姿态提醒的实施例；18 is an embodiment of an operating system running device implementing attitude detection, attitude comparison, and attitude reminding;
图19是操作系统运行装置实现姿态检测的实施例；19 is an embodiment of an operating system operating device implementing attitude detection;
图20是操作系统运行装置实现姿态检测和姿态比较的实施例；20 is an embodiment of an operating system operating device implementing attitude detection and attitude comparison;
图21是基于图18的操作系统运行装置4自动运行姿态检测、姿态比较和姿态提醒的方案流程图；21 is a flowchart of a scheme based on the automatic operation posture detection, posture comparison, and posture reminding of the operating system running device 4 of FIG. 18;
图22是基于图18的操作系统运行装置按应用程序模块的需要运行姿态检测、姿态比较和姿态提醒的方案流程图；22 is a flow chart of a scheme for operating an attitude detection, a posture comparison, and a gesture reminder according to the needs of an application module of the operating system operating device of FIG. 18;
图23是基于图18的应用程序模块使用标志位启动和关闭操作系统运行装置中姿态服务模块的方案流程图；23 is a flowchart of a scenario in which an application module of FIG. 18 uses a flag bit to start and shut down an attitude service module in an operating system operating device;
图24是基于图19的实施例；Figure 24 is an embodiment based on Figure 19;
图25是基于图19的实施例；Figure 25 is an embodiment based on Figure 19;
图26是基于图19的操作系统运行装置按照应用程序状态表启动姿态检测的方案流程图；26 is a flowchart of a scheme for starting an attitude detection according to an application state table by the operating system operating device of FIG. 19;
图27是基于图20的操作系统运行装置向应用程序模块发出姿态超限通知完成提醒的方法流程图；27 is a flowchart of a method for issuing a gesture overrun notification completion reminder to an application module based on the operating system running device of FIG. 20;
图28是基于图20的应用程序模块使用姿态检测标志位启动和关闭姿态检测并监听姿态超限通知的方案流程图；28 is a flow chart of a scheme based on the application module of FIG. 20 using the attitude detection flag to initiate and disable attitude detection and to monitor the attitude overrun notification;
图29是基于图20的操作系统运行装置按照应用程序注册表启动姿态检测并发出姿态超限通知的方案流程图；29 is a flowchart of a scheme based on the operating system running device of FIG. 20 initiating attitude detection according to an application registry and issuing a gesture overrun notification;
图30是通过监控程序模块实现用户姿态检测的移动终端；30 is a mobile terminal that implements user gesture detection by a monitoring program module;
图31是姿态传感模块使用摄像头方案；Figure 31 is a view of the attitude sensing module using a camera;
图32是姿态传感模块使用摄像头和倾角传感器方案；32 is a gesture sensing module using a camera and a tilt sensor scheme;
图33是监控程序模块调用姿态处理单元方案示意图；33 is a schematic diagram of a scheme of a monitoring program module calling gesture processing unit;
图34是通过监控程序模块4’实现用户姿态检测的方法流程图；Figure 34 is a flow chart of a method for implementing user gesture detection by the monitoring program module 4';
图35是监控程序模块4’扫描状态数据库8获得应用程序指令、进行姿态处理的流程图；Figure 35 is a flow chart showing the monitoring program module 4' scanning state database 8 obtaining application instructions and performing gesture processing;
图36是监控程序模块4’扫描状态数据库8获得应用程序状态、进行姿态处理的流程图。Fig. 36 is a flow chart showing the state in which the monitoring program module 4' scans the state database 8 to obtain the application state and performs the posture processing.
具体实施方式Detailed ways
下面结合说明书附图，对本发明实施例进行详细描述。The embodiments of the present invention are described in detail below with reference to the accompanying drawings.
实施例一。 Embodiment 1.
图1是终端设备用户姿态检测系统。终端设备用户姿态检测系统包括：一个或多个面向用户的摄像头100，一个或多个姿态传感器200，信息处理模块300，姿态提醒模块400。1 is a terminal device user attitude detecting system. The terminal device user attitude detection system includes: one or more user-oriented cameras 100, one or more posture sensors 200, an information processing module 300, and a gesture reminding module 400.
通过摄像头100获得用户面部图像或视频。通过设备上的一个或多个姿态传感器(例如采用重力传感器)获得终端设备绝对姿态参数(反映了显示屏所在平面的水平倾角)。A user's face image or video is obtained by the camera 100. The terminal device absolute attitude parameter (reflecting the horizontal tilt of the plane in which the display is located) is obtained by one or more attitude sensors on the device (eg using a gravity sensor).
信息处理模块300，包含姿态传感数据接口a、图像数据接口b、姿态提醒接口c。姿态传感数据接口a传递的数据包含终端设备姿态参数。信息处理模块300计算用户姿态参数，对姿态进行判断，当姿态不正确时触发姿态提醒模块400。图像数据接口b传递的数据包含图像或视频数据。姿态提醒接口c传递的数据包含报警数据和姿态矫正数据。报警数据是指用户姿态参数超出门限值时生成的，包含报警的方式、报警的强度、报警的时间长短等特征，这些数据是根据用户偏离正确姿态的程度和时间来确定的。姿态矫正数据是根据用户姿态参数和门限值之间的差，得到如何改变用户姿态以达到正确姿态的数据，包括距离、角度、及调整的方向等信息。The information processing module 300 includes an attitude sensing data interface a, an image data interface b, and a gesture reminding interface c. The data transmitted by the attitude sensing data interface a includes terminal device attitude parameters. The information processing module 300 calculates a user posture parameter, determines the posture, and triggers the posture reminding module 400 when the posture is incorrect. The data passed by the image data interface b contains image or video data. The data transmitted by the gesture reminder interface c includes alarm data and attitude correction data. The alarm data is generated when the user's attitude parameter exceeds the threshold, including the mode of the alarm, the strength of the alarm, and the length of the alarm. These data are determined according to the degree and time when the user deviates from the correct posture. The attitude correction data is based on the difference between the user's posture parameters and the threshold value, and how to change the user's posture to achieve the correct posture data, including distance, angle, and direction of adjustment.
姿态提醒模块400包含信息输出装置(提示灯、显示屏、振动装置、音频装置中一个或多个)。姿态提醒模块400输出的信号是用户感官可识别的信号。姿态提醒模块400通过以下一种或多种形式向用户发出提示信号和/或矫正建议信号：提示灯开关、颜色变化、闪烁；显示屏上显示的文本、图形、图像或视频；音频装置发出声音；振动装置发出的振动。The gesture reminder module 400 includes an information output device (one or more of a cue light, a display screen, a vibration device, and an audio device). The signal output by the gesture reminder module 400 is a signal recognizable by the user's senses. The gesture reminding module 400 sends a prompt signal and/or a correction suggestion signal to the user by one or more of the following forms: a prompt light switch, a color change, a flashing; a text, a graphic, an image or a video displayed on the display; the audio device emits a sound ; vibration from the vibration device.
上文所述“姿态参数”包括“相对姿态参数”和“绝对姿态参数”。相对姿态参数即“用户相对姿态参数”，是指用户面部相对于显示屏的位置和角度参数；所谓绝对姿态参数，是指反映物体在空间中的姿态、与相对位置无关的参数，绝对姿态参数包括“终端设备绝对姿态参数”，例如显示屏和水平面之间的夹角t1，还包括“用户绝对姿态参数”，例如用户面部各特征点所在平面与水平面之间的夹角t4。The "attitude parameters" described above include "relative attitude parameters" and "absolute attitude parameters". The relative attitude parameter, that is, the “user relative attitude parameter” refers to the position and angle parameters of the user's face relative to the display screen; the so-called absolute attitude parameter refers to the parameter that reflects the object's attitude in space, independent of the relative position, and the absolute attitude parameter. Including "terminal device absolute attitude parameters", such as the angle t1 between the display screen and the horizontal plane, and also includes "user absolute attitude parameters", such as the angle t4 between the plane of the feature points of the user's face and the horizontal plane.
图2是终端设备用户姿态检测方法实施例的流程图，具体说明如下：2 is a flowchart of an embodiment of a method for detecting a posture of a user of a terminal device, which is specifically described as follows:
第101步：显示屏姿态检测。采集终端显示屏所处空间状态获得设备姿态数据，计算得出显示屏所在平面的水平倾角，作为终端设备绝对姿态参数。Step 101: Display attitude detection. The space state of the display screen of the terminal is collected to obtain the device attitude data, and the horizontal inclination of the plane where the display screen is located is calculated as the absolute attitude parameter of the terminal device.
第102步：用户姿态检测。采集用户图像，从图像信号中识别用户面部的特征点，计算出用户相对姿态参数；Step 102: User posture detection. Collecting a user image, identifying a feature point of the user's face from the image signal, and calculating a relative posture parameter of the user;
第103步：姿态比较。将至少一个所述姿态参数和该参数的门限值进行比较，判断是否至少一个所述姿态参数超过门限值。Step 103: Posture comparison. Comparing at least one of the attitude parameters with a threshold of the parameter to determine whether at least one of the attitude parameters exceeds a threshold.
第104步：信号输出。当至少一个所述姿态参数超过门限值时，发出提示信号和/ 或矫正建议信号。Step 104: Signal output. When at least one of the attitude parameters exceeds a threshold, a prompt signal is issued and / Or correct the suggestion signal.
以上步骤循环进行。当第103步未有姿态参数超出门限值的范围时、或第104步完成后，回归至第101步。The above steps are repeated. When step 103 does not have the attitude parameter beyond the threshold value, or after the completion of step 104, return to step 101.
图3是终端设备用户姿态检测方法另一实施例的流程图，具体说明如下：FIG. 3 is a flowchart of another embodiment of a method for detecting a posture of a user of a terminal device, which is specifically described as follows:
第201步：用户姿态检测：采集用户图像，从图像信号中识别用户面部的特征点，计算出用户相对姿态参数；Step 201: User posture detection: collecting a user image, identifying a feature point of the user's face from the image signal, and calculating a relative posture parameter of the user;
第202步：显示屏姿态检测：采集终端显示屏所处空间状态获得设备姿态数据，计算得出显示屏所在平面的水平倾角，作为终端设备绝对姿态参数；Step 202: Display attitude detection: collect the space state of the display screen of the terminal to obtain the device attitude data, and calculate the horizontal inclination of the plane where the display screen is located, as the absolute attitude parameter of the terminal device;
第203步：姿态比较：将至少一个所述姿态参数和该参数的门限值进行比较，判断是否有至少一个所述姿态参数连续多次循环均超过门限值。Step 203: Pose comparison: comparing at least one of the posture parameters with a threshold value of the parameter to determine whether at least one of the posture parameters exceeds a threshold value for a plurality of consecutive cycles.
第204步：信号输出：当至少一个所述姿态参数连续多次循环均超过门限值时，发出提示信号和/或矫正建议信号。Step 204: Signal output: when at least one of the attitude parameters exceeds a threshold for a plurality of consecutive cycles, a prompt signal and/or a correction suggestion signal is issued.
以上步骤循环进行。当第203步未有姿态参数超出门限值的范围时、或第204步完成后，回归至第201步。作为与图2所示方案相比进一步优化的实施方式，为了减少信号发生频率，当至少一个所述姿态参数连续多次循环超过门限值时，发出提示信号和/或矫正建议信号。为了减少检测频率，本实施例还包含第205步：延迟。所述直接回归即延迟为0的情形。The above steps are repeated. When step 203 does not have the attitude parameter beyond the threshold value, or after step 204 is completed, return to step 201. As a further optimized embodiment compared to the solution shown in FIG. 2, in order to reduce the signal occurrence frequency, a prompt signal and/or a correction suggestion signal is issued when at least one of the attitude parameters repeatedly exceeds a threshold value for a plurality of consecutive times. In order to reduce the detection frequency, the embodiment further includes step 205: delay. The direct regression is a case where the delay is zero.
在图2和图3所示实施例中，还可以将用户相对姿态参数和设备绝对姿态参数相结合，计算得出用户绝对姿态参数，作为所述姿态比较(第103、203步)所使用的姿态参数。In the embodiment shown in FIG. 2 and FIG. 3, the user relative posture parameter and the device absolute posture parameter may also be combined to calculate the user absolute posture parameter, which is used as the posture comparison (steps 103 and 203). Gesture parameters.
图4是摄像头在终端设备面板上的位置和功能示意图，一个终端设备500使用时大致呈平板状，有一个或多个面向用户的摄像头100。还要至少要包含一个面向用户的显示屏600，摄像头的镜头所在平面和显示屏所在平面相平行、或具有其它稳定的位置关系，使用户在面向设备阅读或工作时，面向摄像头、处于摄像头的图像采集空间中。4 is a schematic diagram of the position and function of the camera on the panel of the terminal device. When the terminal device 500 is used, it is substantially flat and has one or more camera 100 facing the user. At least one user-oriented display screen 600 should be included. The plane of the lens of the camera is parallel to the plane of the display screen, or has other stable positional relationships, so that the user faces the camera and is in the camera while reading or working on the device. In the image acquisition space.
图5是姿态传感器在终端设备上的位置和功能示意图。该终端设备500的硬件至少要包含一个面向用户的显示屏600和一个能检测出显示屏所在平面倾角的姿态传感器200。该传感器安装在终端设备上，与显示屏具有固定的位置关系。通过该传感器输出的数据，可以得到终端设备的显示屏600所在平面的倾角(如图5中t1)，即终端设备的显示屏所在平面和水平面之间的夹角，作为终端设备绝对姿态参数。Figure 5 is a schematic diagram of the position and function of the attitude sensor on the terminal device. The hardware of the terminal device 500 includes at least a display screen 600 facing the user and an attitude sensor 200 capable of detecting the plane inclination of the display screen. The sensor is mounted on the terminal device and has a fixed positional relationship with the display. Through the data output by the sensor, the inclination angle of the plane of the display screen 600 of the terminal device (such as t1 in FIG. 5), that is, the angle between the plane where the display screen of the terminal device is located and the horizontal plane can be obtained as the absolute attitude parameter of the terminal device.
在上述各方案中所提到的“姿态参数”，可以用多种方法定义，以下举例说明定义和分析的方法。 The "attitude parameters" mentioned in the above various schemes can be defined by various methods, and the methods of definition and analysis are exemplified below.
例如一种描述用户姿态的参数是：用户面部参考点与终端显示屏参考点之间的水平距离L1、垂直距离L2，用户面部中间线和显示屏之间的相对倾角t0，用户面部中间线和水平面之间的夹角t4。其中L1、L2、t0反映了用户面部与终端显示屏之间的相对位置关系，因此称为“用户相对姿态参数”，其中t4反映了用户面部与终端显示屏无关的位置参数，称为“用户绝对姿态参数”。即用户相对姿态参数为[L1，L2，t0]中至少一个参数。用户绝对姿态参数t4＝t1+t0。For example, a parameter describing the posture of the user is: a horizontal distance L1 between the user's face reference point and the terminal display reference point, a vertical distance L2, a relative inclination angle t0 between the user's facial middle line and the display screen, and a user's facial middle line and The angle t4 between the horizontal planes. L1, L2, and t0 reflect the relative positional relationship between the user's face and the display of the terminal, so it is called “user relative attitude parameter”, where t4 reflects the position parameter of the user's face and the terminal display, which is called “user”. Absolute attitude parameter". That is, the user relative attitude parameter is at least one of [L1, L2, t0]. The user absolute attitude parameter t4=t1+t0.
再例如一种描述用户姿态的参数是：摄像头图像数据中用户面部占据的范围Af、面部中间线Cf偏离摄像头图像中间线Cc的平均距离为d，用户面部中间线和水平面之间的夹角t0等，也可以构建相对姿态参数，用户相对姿态参数为[Af，d，t0]中至少一个参数。用户绝对姿态参数t4＝t1+t0。For example, a parameter describing the posture of the user is: a range Af occupied by the user's face in the image data of the camera, an average distance of the middle line Cf of the face from the intermediate line Cc of the camera image is d, and an angle t0 between the middle line of the user's face and the horizontal plane Etc., it is also possible to construct a relative pose parameter, and the user relative pose parameter is at least one of [Af, d, t0]. The user absolute attitude parameter t4=t1+t0.
姿态参数的门限值是指相对姿态参数和绝对姿态参数为标定正确姿态定义的门限值，例如L1、L2、Af、t1、t0、t4的最大值或最小值。按照前述描述用户姿态的参数的实施例，相对姿态参数的门限值T1＝[Min(L1)，Max(L1)，Min(L2)，Max(L2)，Min(t0)，Max(t0)]，或者T1＝[Min(Af)，Max(Af)，Max(d)，Min(t0)，Max(t0)]；绝对姿态参数的门限值T2＝[Min(t1)，Max(t1)，Min(t0)，Max(t0)]。用户姿态不正确，是指用户的姿态参数超出门限值的范围。The threshold value of the attitude parameter refers to a threshold value defined by the relative attitude parameter and the absolute attitude parameter for calibrating the correct posture, for example, the maximum value or the minimum value of L1, L2, Af, t1, t0, and t4. According to the foregoing embodiment of the parameter describing the posture of the user, the threshold value of the relative attitude parameter T1=[Min(L1), Max(L1), Min(L2), Max(L2), Min(t0), Max(t0) ], or T1=[Min(Af), Max(Af), Max(d), Min(t0), Max(t0)]; threshold value of absolute attitude parameter T2=[Min(t1), Max(t1 ), Min(t0), Max(t0)]. Incorrect user posture means that the user's attitude parameter exceeds the threshold.
图6是单目机器视觉方法面部位置的获取原理图。在单目机器视觉原理基础上，这里举例说明一种面部位置的获取方法，采用所述单目检测方法，存在一个摄像头，取人面部中线两个容易识别的特征点A、B，摄像头镜头中心点C，用单目机器视觉的方法得到长度AC、BC；长度AB使用典型值；再计算出各个姿态参数[L1、L2、t0]。详细说明如下：Figure 6 is a schematic diagram of the acquisition of the face position of the monocular machine vision method. Based on the principle of monocular machine vision, a method for acquiring the position of the face is exemplified herein. With the monocular detection method, there is a camera that takes two easily recognizable feature points A and B in the middle of the human face, and the center of the camera lens. Point C, the length AC, BC is obtained by the monocular machine vision method; the length AB is a typical value; and each attitude parameter [L1, L2, t0] is calculated. The details are as follows:
通过设备的前置摄像头获得的图像，采用计算机图形学的方法从这些图像中提取出用户面部的特征点，如眼睛、鼻子、嘴等。图中A、B分别为人面部两个容易识别的特征点，如两个眼睛中点位置、嘴巴中点位置。C点为设备前面板上的摄像头镜头中心点位置。DE所在平面为摄像头传感器所在的平面，D、E两点分别是A、B在传感器上的像。由于传感器与镜头的几何关系固定，因此可以由D、E在传感器上的位置及镜头与传感器的距离计算出角度t2，即BD与DE的夹角。通过单目机器视觉的方法，如几何相似法、几何光学法，可以得到AC、BC的长度。而AB的长度可以事先通过测量得到，或直接使用一些典型值。这样又可以用余弦定理计算出角度t3、以及C距A的水平距离L1和垂直距离L2。通过简单的平面几何推导可以得到t0＝t3-t2，即上文所述头部和终端显示屏之间的相对倾角；t4＝t1+t3-t2，即头部的水平倾角。 Through the images obtained by the front camera of the device, computer graphics are used to extract feature points of the user's face, such as eyes, nose, mouth, and the like. In the figure, A and B are two easily distinguishable feature points of the human face, such as the midpoint of the two eyes and the midpoint of the mouth. Point C is the center point of the camera lens on the front panel of the device. The plane where DE is located is the plane where the camera sensor is located, and the two points D and E are the images of A and B on the sensor. Since the geometric relationship between the sensor and the lens is fixed, the angle t2, that is, the angle between BD and DE, can be calculated from the position of D and E on the sensor and the distance between the lens and the sensor. The length of AC and BC can be obtained by a method of monocular machine vision, such as geometric similarity method and geometrical optical method. The length of AB can be measured in advance, or some typical values can be used directly. In this way, the cosine theorem can be used to calculate the angle t3, and the horizontal distance L1 and the vertical distance L2 of the C from A. By simple planar geometric derivation, it is possible to obtain t0=t3-t2, which is the relative inclination angle between the head and the terminal display screen described above; t4=t1+t3-t2, that is, the horizontal inclination of the head.
当t1超出其门限T1，或t4超出其门限T4，或L1超出其门限Lt1，或L2超出其门限Lt2时，界面显示相应的提示信息。例如将正确姿态定义为：t1＝t4≈90度，L1>30厘米，L2≈0，t1、t4、L1和L2门限值可分别定为T1＝80度，T4＝55度，Lt1＝23厘米，Lt2＝8厘米。以上门限值仅为举例，并不排除用其它最大或最小值方式确定门限值。When t1 exceeds its threshold T1, or t4 exceeds its threshold T4, or L1 exceeds its threshold Lt1, or L2 exceeds its threshold Lt2, the interface displays a corresponding prompt message. For example, the correct posture is defined as: t1=t4≈90 degrees, L1>30 cm, L2≈0, t1, t4, L1 and L2 thresholds can be determined as T1=80 degrees, T4=55 degrees, Lt1=23 Cm, Lt2 = 8 cm. The above thresholds are only examples and do not preclude the determination of thresholds by other maximum or minimum methods.
长度AB通过校准过程得到典型值。测量面部相对于设备的姿态是否正确，可以加一个校准过程，校准时，要求用户必须用正确的姿态，保持特定已知的姿态参数值，例如头部正直，手机显示屏与眼睛几乎同高且正对眼睛，显示屏距离眼睛30厘米左右。摄像头将此时的面部信息进行分析，可以得到用户的特征信息，如两只眼睛、鼻子、下颌等比较容易识别的点的几何关系。这样前文所述识别过程中用到的AB长度的典型值就可以用单目机器视觉的方法得到了。The length AB is typically obtained by a calibration process. To measure the correct posture of the face relative to the device, a calibration process can be added. When calibrating, the user must use the correct posture to maintain a specific known attitude parameter value, such as the head straightness, and the mobile phone display screen is almost the same height as the eyes. Facing the eyes, the display is about 30 cm away from the eyes. The camera analyzes the facial information at this time, and can obtain the characteristic information of the user, such as the geometric relationship of the points that are relatively easy to recognize, such as two eyes, a nose, and a lower jaw. Thus, the typical value of the AB length used in the identification process described above can be obtained by the monocular machine vision method.
如果直接使用经验值来确定面部器官中点间距的典型值，可省略头部姿态测量的校准过程。If the empirical value is used directly to determine the typical value of the midpoint spacing in the facial organs, the calibration process of the head attitude measurement can be omitted.
双目或多目机器视觉方法面部位置的获取。上述实施例的讨论是基于一个摄像头的方法，如果有两个或多个摄像头，这些摄像头对同一场景摄得的图像具有视差，通过对特征点之间视差的分析，可以得到A点和B点在空间中相对于摄像头中心点C(此时可定义为某一个摄像头镜头中心点)的距离，从而可以算出L1和L2两个距离参数以及角度t3，进而算出角度t4。采用所述双目或多目检测方法，存在两个或多个摄像头，采用双目及多目机器视觉的方法不需要校准过程。Binocular or multi-mesh machine vision method for obtaining facial position. The discussion of the above embodiment is based on a method of a camera. If there are two or more cameras, the cameras have parallax for images taken in the same scene, and by analyzing the parallax between the feature points, points A and B can be obtained. The distance between the two points L1 and L2 and the angle t3 can be calculated from the distance between the center point C of the camera (which can be defined as the center point of a certain camera lens) in space, and the angle t4 can be calculated. With the binocular or multi-eye detection method, there are two or more cameras, and the method of binocular and multi-eye machine vision does not require a calibration process.
虚拟多摄像机进行面部位置的获取。可以将光学成像系统和单个摄像头结合，形成两个或多个虚拟摄像机，组成单摄像机的双目或多目视觉系统，获得虚拟的立体视觉。这样可以解决一般双目或多目系统中对多个摄像头的一致性要求。The virtual multi-camera acquires the face position. The optical imaging system can be combined with a single camera to form two or more virtual cameras that form a single-camera binocular or multi-view vision system for virtual stereo vision. This can solve the consistency requirements for multiple cameras in a general binocular or multi-view system.
使用3D相机进行面部位置的获取。随着技术的进步，现在已经出现的3D相机可以直接输出立体图形，只要从中提取出所需要的特征点，如前面实施例中的A点和B点，就可以算出L1和L2两个距离信息以及角度t2、t3，进而算出角度t0、t4。The acquisition of the face position is performed using a 3D camera. With the advancement of technology, the 3D camera that has appeared now can directly output the stereoscopic image, and as long as the required feature points are extracted therefrom, as in points A and B in the previous embodiment, the distance information of L1 and L2 can be calculated and Angles t2 and t3, and angles t0 and t4 are calculated.
图7是特征值提取法面部位置的获取原理图，摄像头图像数据中用户面部占据的范围为Af，取人面部中线两个容易识别的特征点A、B，获得面部中间线Cf，面部中间线Cf偏离摄像头图像中间线Cc的平均距离为d。其中，图7(a)表示摄像头图像数据中用户面部占据的范围Af过大的情形；图7(b)摄像头图像数据中用户面部占据的范围Af过小的情形；图7(c)表示面部中间线Cf偏离摄像头图像中间线Cc的平均距离为d较远的情形；图7(d)表示用户姿态正常的情形。7 is a schematic diagram of the acquisition of the face position of the feature value extraction method. The range occupied by the user's face in the camera image data is Af, and the two feature points A and B which are easily recognized by the human facial line are obtained, and the middle line Cf of the face is obtained, and the middle line of the face is obtained. The average distance of Cf from the camera image intermediate line Cc is d. 7(a) shows a case where the range Af occupied by the user's face is excessively large in the image data of the camera; FIG. 7(b) shows a case where the range Af occupied by the user's face is too small in the image data of the camera; FIG. 7(c) shows the face. The intermediate line Cf deviates from the middle of the camera image intermediate line Cc by a distance d; FIG. 7(d) shows a case where the user's posture is normal.
图8是使用距离和角度参数的实施例流程图 Figure 8 is a flow chart of an embodiment using distance and angle parameters
第301步：显示屏姿态检测。姿态传感器采集终端显示屏所处空间状态，输出设备姿态数据。姿态计算模块使用姿态传感器输出的设备姿态数据计算设备绝对姿态参数t1。Step 301: Display attitude detection. The attitude sensor collects the spatial state of the terminal display and outputs the device attitude data. The attitude calculation module calculates the device absolute attitude parameter t1 using the device attitude data output by the attitude sensor.
第302步：用户姿态检测。摄像头采集用户图像，从图像中提取出用户的面部及面部的特征点，如眼睛、鼻子、嘴等，分析计算用户面部的相对姿态参数和绝对姿态参数。采用单目、双目或多目视觉的方法，测量出面部各器官中点与摄像头之间的距离，进一步计算出的几何关系，包括眼睛与显示屏的水平距离L1、垂直距离L2，面部中线(面部各器官中点连线)的与终端设备显示屏所在平面的相对倾角(如图6所示t2，t3，以及计算得出t0)。姿态计算模块根据以上角度值，计算用户面部中线的水平倾角t4。Step 302: User posture detection. The camera captures the user image, extracts feature points of the user's face and face from the image, such as eyes, nose, mouth, etc., and analyzes the relative posture parameters and absolute posture parameters of the user's face. Using the monocular, binocular or multi-eye vision method, the distance between the midpoint of the facial organs and the camera is measured, and the geometric relationship is further calculated, including the horizontal distance L1 of the eye and the display screen, the vertical distance L2, and the facial midline. The relative inclination of the plane of the display of the terminal device (t2, t3, and t0). The attitude calculation module calculates the horizontal inclination angle t4 of the line of the user's face based on the above angle values.
第303步：用户姿态分析。姿态判断模块将以上姿态参数(L1、L2、t0、t1、t2、t3、t4)中至少一个参数和该参数的门限值进行比较，判断用户姿态是否正确。此处用户姿态不正确，是指至少一个所述姿态参数超出该参数门限值表示的范围。Step 303: User posture analysis. The posture judging module compares at least one of the above posture parameters (L1, L2, t0, t1, t2, t3, t4) with a threshold value of the parameter to determine whether the user posture is correct. Here, the user posture is incorrect, which means that at least one of the posture parameters exceeds a range represented by the parameter threshold.
第304步：信号输出。当第303步用户姿态分析的结果中发现用户姿态不正确时，触发姿态提醒模块发出提示信息。Step 304: Signal output. When the user posture is found to be incorrect in the result of the user posture analysis in step 303, the trigger gesture reminding module issues a prompt message.
以上步骤循环进行。当第303步用户姿态分析结果中发现姿态参数均不超过门限值、或第304步完成后，回归第301步。为了减少检测的频率，本实施例中还包含第305步：延迟。延迟为0表示直接回归。The above steps are repeated. When the posture parameter of the user posture analysis in step 303 is found that the posture parameter does not exceed the threshold value, or after the completion of the step 304, the step 301 is returned. In order to reduce the frequency of detection, the embodiment further includes step 305: delay. A delay of 0 indicates a direct regression.
对图8所示实施例进一步优化，为了减少信号发生频率，在第301～304步构成的循环过程中，至少一个所述姿态参数超过门限值，且持续多次循环该均参数均超出门限值，所述第304步才触发姿态提醒模块发出提示信号和/或矫正建议信号。此处用户姿态不正确，是指至少一个所述姿态参数超过门限值，且持续多次循环该均参数均超出门限值。The embodiment shown in FIG. 8 is further optimized. In order to reduce the frequency of signal generation, at least one of the attitude parameters exceeds a threshold value during a loop formed by steps 301 to 304, and the parameters are exceeded for a plurality of cycles. The limit value, the step 304 triggers the gesture reminding module to issue a prompt signal and/or a correction suggestion signal. Here, the user posture is incorrect, that is, at least one of the posture parameters exceeds a threshold value, and the average parameter exceeds the threshold value for a plurality of cycles.
图9是使用面积和角度参数的实施例流程图Figure 9 is a flow chart of an embodiment using area and angle parameters
第401步：显示屏姿态检测。姿态传感器采集终端显示屏所处空间状态，输出设备姿态数据。姿态计算模块使用姿态传感器输出的设备姿态数据计算设备绝对姿态参数t1。Step 401: Display attitude detection. The attitude sensor collects the spatial state of the terminal display and outputs the device attitude data. The attitude calculation module calculates the device absolute attitude parameter t1 using the device attitude data output by the attitude sensor.
第402步：用户姿态检测。摄像头采集用户图像，从图像中提取出用户的头部和面部的像素，用特征分析的方法确定用户面部和头部图像A，进一步在摄像头图像数据用户面部标准轮廓As中占据的范围Af、及面部中间线偏离图像中间线的程度d、以及面部中间线和显示屏的相对倾角t0。Step 402: User posture detection. The camera collects the user image, extracts the pixels of the user's head and face from the image, determines the user's face and head image A by feature analysis, further ranges Af occupied by the camera image data user's face standard outline As, and The degree d of the middle line of the face deviating from the middle line of the image, and the relative inclination angle t0 of the middle line of the face and the display screen.
第403步：用户姿态分析。姿态判断模块将以上姿态参数(Af、d、t1、t0)中至少 一个参数和该参数的门限值进行比较，判断用户姿态是否正确。用户姿态不正确，是指所述姿态参数中至少一个参数超出该参数门限值表示的范围。Step 403: User posture analysis. The attitude judgment module will at least have the above attitude parameters (Af, d, t1, t0) A parameter is compared with the threshold of the parameter to determine whether the user gesture is correct. If the user posture is incorrect, it means that at least one of the posture parameters exceeds a range represented by the parameter threshold.
第404步：信号输出。当第403步用户姿态分析的结果中发现用户姿态不正确时，触发姿态提醒模块发出提示信息。Step 404: Signal output. When the user posture is found to be incorrect in the result of the user posture analysis in step 403, the trigger gesture reminding module issues a prompt message.
以上步骤循环进行。当第403步用户姿态分析结果中发现姿态参数均不超过门限值、或第404步完成后，回归第401步。为了减少检测的频率，本实施例中还包含第405步：延迟。延迟为0表示直接回归。The above steps are repeated. When the posture parameter of the step 403 is found to not exceed the threshold value, or after the completion of the 404th step, the step 401 is returned. In order to reduce the frequency of detection, the embodiment further includes step 405: delay. A delay of 0 indicates a direct regression.
对图9所示实施例进一步优化，为了减少信号发生频率，在第401～404步构成的循环过程中，至少一个所述姿态参数超过门限值，且持续多次循环该均参数均超出门限值，所述第404步才触发姿态提醒模块发出提示信号和/或矫正建议信号。此处用户姿态不正确，是指至少一个所述姿态参数超过门限值，且持续多次循环该均参数均超出门限值。The embodiment shown in FIG. 9 is further optimized. In order to reduce the frequency of signal generation, at least one of the attitude parameters exceeds a threshold value during a loop constituted by steps 401 to 404, and the parameters are exceeded for a plurality of cycles. The limit value, the step 404 triggers the gesture reminding module to issue a prompt signal and/or a correction suggestion signal. Here, the user posture is incorrect, that is, at least one of the posture parameters exceeds a threshold value, and the average parameter exceeds the threshold value for a plurality of cycles.
关于检测周期。由于人体从正确姿态变为不正确姿态的过程比较缓慢，作为本发明方法的优化的实施例，摄像头可以只是每隔数十秒才进行一次拍摄，其它单元根据这次拍摄的结果做出相应的响应。实现的方法的例子是使上述实施例中第205、305、405步中的延迟时间为数十秒，本方法也适用于本说明书所述其他实施例。About the detection cycle. Since the process of changing the human body from a correct posture to an incorrect posture is relatively slow, as an optimized embodiment of the method of the present invention, the camera can only take a shot every tens of seconds, and other units make correspondings according to the result of the shooting. response. An example of the method implemented is to make the delay time in steps 205, 305, and 405 of the above embodiment tens of seconds, and the present method is also applicable to other embodiments described in the present specification.
关于显示模式。本发明的系统和方法在显示屏工作的状态下才有意义，作为本发明方法的优化的实施例，该发明在开始姿态检测之前先检测显示屏是否处于显示状态。如正常显示，则执行前述实施例所述的步骤。如无显示，则不开启检测过程以节省电能。About the display mode. The system and method of the present invention is only meaningful in the operational state of the display screen. As an optimized embodiment of the method of the present invention, the invention first detects whether the display screen is in a display state prior to initiating attitude detection. As shown normally, the steps described in the previous embodiments are performed. If there is no display, the detection process is not turned on to save power.
当终端所运行的应用软件占据显示屏时，用户面向终端显示屏。将用户姿势检测及提醒功能开发成函数库，集成在操作系统或开发工具中的函数库中供一般应用程序开发者调用，这样可以降低一般应用程序开发者的开发难度，在更多的应用程序中为用户提供健康保证。When the application running on the terminal occupies the display, the user faces the display of the terminal. Develop user gesture detection and reminder functions into function libraries, integrated in the function library of the operating system or development tools for general application developers to call, which can reduce the development difficulty of general application developers, in more applications Provide users with health guarantees.
应用在移动智能终端(例如3G手机或LTE手机)时，智能终端的操作系统(如Android、iOS、黑莓)可以基于本发明的方法提供一种系统服务，作为一种后台运行的服务，该服务在显示屏被应用程序占用并显示内容时检查用户的姿态，在姿态不正确时直接发出提示。例如，智能终端的操作系统可以提供一个API(应用程序编程接口)，该API通过本发明的技术方案向应用程序提供用户的姿态，由应用程序发出姿态提醒。再例如，本发明也可以是智能终端上运行的一个普通应用程序或通用监控应用程序，该应用程序通过本发明的方法来决定是否发出姿态提醒。When applied to a mobile intelligent terminal (such as a 3G mobile phone or an LTE mobile phone), the operating system of the smart terminal (such as Android, iOS, BlackBerry) can provide a system service based on the method of the present invention, as a background running service, the service Check the user's posture when the display is occupied by the application and display the content, and promptly when the gesture is incorrect. For example, the operating system of the smart terminal can provide an API (Application Programming Interface), which provides a gesture of the user to the application by the technical solution of the present invention, and the gesture is reminded by the application. For another example, the present invention can also be a general application or a general monitoring application running on the smart terminal, and the application determines whether to issue a gesture reminder by the method of the present invention.
本方法还可以包含对功能进行设置的步骤。在操作系统、应用程序中对本发明的功 能进行设置，使得该功能只在某些应用中启用，也可以通过设置让该功能在某些应用中不被启用。如前台应用程序是接听、拨打电话、听收音机或其它音频内容时，由于此时用户不会长时间关注显示屏，此时不必要开启本功能。这种设置可以是用户手动设置，也可以是操作系统、应用程序进行默认设置。The method can also include the step of setting the function. The work of the present invention in an operating system, an application Can be set up so that the feature is only enabled in certain applications, or it can be set to make it not enabled in some applications. If the foreground application is answering, making a call, listening to radio or other audio content, since the user does not pay attention to the display for a long time, it is not necessary to turn on this function. This setting can be set manually by the user, or it can be set by the operating system or application.
图10是终端设备用户姿态检测系统的一种实施例。信息处理模块300还包括机器视觉处理模块31、姿态计算模块32、姿态判断模块33。机器视觉处理模块31处理摄像头得到的图像或视频，从中提取出用户相对设备的姿态参数，包括距离和相对角度。姿态计算模块32根据终端设备的绝对姿态参数与用户相对设备几何关系数据计算用户的绝对姿态参数。姿态判断模块33对用户的相对姿态参数和绝对姿态参数进行判断，当发现有一个或多个姿态参数不正确时，通过姿态提醒接口c向姿态提醒模块400发出数据，触发所述姿态提醒模块400发出提示信号和/或矫正建议信号。Figure 10 is an embodiment of a terminal device user attitude detection system. The information processing module 300 further includes a machine vision processing module 31, a posture calculation module 32, and a posture determination module 33. The machine vision processing module 31 processes the image or video obtained by the camera, and extracts the posture parameters of the user relative to the device, including the distance and the relative angle. The attitude calculation module 32 calculates the absolute attitude parameter of the user according to the absolute attitude parameter of the terminal device and the user relative device geometric relationship data. The posture determination module 33 determines the relative posture parameter and the absolute posture parameter of the user. When it is found that one or more posture parameters are incorrect, the attitude reminding interface c sends data to the posture reminding module 400, and triggers the posture reminding module 400. A prompt signal and/or a correction suggestion signal is issued.
信息处理模块300采取本地信息处理时，计算可以在智能终端本地完成，即所述摄像头100、姿态传感器200、信息处理模块300、姿态提醒模块400都集成在一个终端设备内。整个头部姿态的获取及相关的校准过程都可以由本地的计算资源，如软件、固件或硬件来完成。此时，姿态传感数据接口a、图像数据接口b、姿态提醒接口c都使用本地通信技术。When the information processing module 300 takes local information processing, the calculation can be completed locally in the smart terminal, that is, the camera 100, the posture sensor 200, the information processing module 300, and the gesture reminding module 400 are all integrated in one terminal device. The acquisition of the entire head pose and associated calibration procedures can be performed by local computing resources such as software, firmware or hardware. At this time, the attitude sensing data interface a, the image data interface b, and the gesture reminding interface c all use local communication technology.
图11～13是远程信息处理的终端设备姿态检测系统实施例，终端设备500包含摄像头100、姿态传感器200、姿态提醒模块400，所述信息处理模块300在终端设备500的外部。例如，本地摄像头100将获得的图片或视频通过智能终端上的通信装置传输到应用服务提供者的服务器上进行分析与计算，最终服务器将是否发出提醒的指示发给智能终端。由于一般服务器的计算能力比智能终端本地的计算能力要强的多，因此这样能获得更好的性能。此时，姿态传感数据接口a、图像数据接口b、姿态提醒接口c使用能支持较远距离传输的有线或无线通信装置来实现。这些通信装置包含图11～13中的有线连接装置7，无线连接装置81、82，以及数据传送网络900等。11 to 13 are embodiments of a terminal device attitude detection system for telematics. The terminal device 500 includes a camera 100, an attitude sensor 200, and a gesture reminder module 400. The information processing module 300 is external to the terminal device 500. For example, the local camera 100 transmits the obtained picture or video to the application service provider's server through the communication device on the smart terminal for analysis and calculation, and finally the server sends an indication of whether to issue a reminder to the smart terminal. Since the computing power of a general server is much stronger than that of a smart terminal, this can achieve better performance. At this time, the attitude sensing data interface a, the image data interface b, and the gesture reminding interface c are implemented using a wired or wireless communication device capable of supporting transmission over a long distance. These communication devices include the wired connection device 7, the wireless connection devices 81, 82, and the data transfer network 900, and the like in Figs.
图11是有线连接远程信息处理的终端设备姿态检测系统实施例，在终端设备用户姿态检测系统中还包括有线连接装置7；所述姿态传感数据接口a、图像数据接口b、姿态提醒接口c通过所述有线连接装置7实现；所述有线连接装置7是电缆或光缆。11 is an embodiment of a terminal device attitude detection system for wired connection telematics, further comprising a wired connection device 7 in the terminal device user attitude detection system; the attitude sensing data interface a, the image data interface b, and the gesture reminding interface c This is achieved by the wired connection device 7; the wired connection device 7 is a cable or fiber optic cable.
图12是无线连接远程信息处理的终端设备姿态检测系统实施例，终端设备用户姿态检测系统中还包括无线连接装置81、82；所述姿态传感数据接口a、图像数据接口b、姿态提醒接口c通过所述无线连接装置81、82实现；所述无线连接装置包括天线812、822和调制解调器811、821。 12 is an embodiment of a terminal device attitude detection system for wirelessly connecting telematics. The terminal device user attitude detection system further includes wireless connection devices 81 and 82; the attitude sensing data interface a, the image data interface b, and the gesture reminding interface. c is implemented by the wireless connection means 81, 82; the wireless connection means comprises antennas 812, 822 and modems 811, 821.
图13是数据传送网络连接远程信息处理的终端设备姿态检测系统实施例，终端设备用户姿态检测系统中还包括数据传送网络900。所述姿态传感数据接口a、图像数据接口b、姿态提醒接口c通过所述数据传送网络900实现。所述数据传送网络900是局域网或互联网；所述数据传送网络中包含计算机网络、或交换机网络、或路由器网络。FIG. 13 is an embodiment of a terminal device attitude detection system for data transmission network connection telematics processing, and the terminal device user attitude detection system further includes a data transmission network 900. The attitude sensing data interface a, the image data interface b, and the gesture reminding interface c are implemented by the data transmission network 900. The data transfer network 900 is a local area network or the Internet; the data transfer network includes a computer network, or a switch network, or a router network.
实施例二。Example 2.
图14是通过操作系统实现用户姿态检测的移动终端，如图14所示，本发明的装置用于通过操作系统实现用户姿态检测的移动终端5，该移动终端中包含姿态传感模块1、用户配置输入模块2、告警信号输出模块3、操作系统运行装置4、应用程序模块71，72，…，7N。姿态传感模块1产生姿态传感信号S。用户配置输入模块2识别用户操作，将用户指令I传递至应用程序模块71，72，…，7N和/或操作系统运行装置4。告警信号输出模块3发出用户感官可识别的信号。14 is a mobile terminal that implements user gesture detection by an operating system. As shown in FIG. 14, the apparatus of the present invention is used for implementing a mobile terminal 5 for user gesture detection by an operating system, where the mobile terminal includes a gesture sensing module 1 and a user. The input module 2, the alarm signal output module 3, the operating system running device 4, and the application modules 71, 72, ..., 7N are configured. The attitude sensing module 1 generates an attitude sensing signal S. The user configuration input module 2 identifies the user operation and passes the user command I to the application modules 71, 72, ..., 7N and/or the operating system operating device 4. The alarm signal output module 3 emits a signal recognizable by the user.
本发明所述操作系统运行装置，是加载和运行移动终端操作系统的装置，例如加载和运行Android、iOS、黑莓操作系统及其替代程序。本发明所述应用程序模块71，72，…，7N，是加载和运行应用程序的装置，特别是加载和运行一些用户可能长时间连续使用的应用程序，如视频播放器、网页浏览器、阅读软件等。本发明所述应用程序模块、操作系统运行模块等各组成部分能用CPU系统、可编程逻辑电路和存储器的组合来实现。The operating system running device of the present invention is a device for loading and running a mobile terminal operating system, such as loading and running Android, iOS, BlackBerry operating system and its replacement program. The application modules 71, 72, ..., 7N of the present invention are devices for loading and running applications, particularly loading and running applications that may be used continuously by users for a long time, such as video players, web browsers, and reading. Software, etc. The components of the application module, the operating system running module and the like of the present invention can be implemented by a combination of a CPU system, a programmable logic circuit and a memory.
图14进一步描述了本发明提出的操作系统运行装置4，包含传感模块驱动单元41、输入模块驱动单元42、输出模块驱动单元43。传感模块驱动单元41控制姿态传感模块1工作、并将姿态传感模块1输出的姿态传感信号S转换为姿态数据D。输入模块驱动单元42控制用户配置输入模块工作，识别用户指令I，产生用户指令数据；输出模块驱动单元43输出信号O，包含报警数据A和/或姿态矫正数据R，控制告警信号输出模块3工作。FIG. 14 further depicts the operating system operating device 4 of the present invention, including a sensing module driving unit 41, an input module driving unit 42, and an output module driving unit 43. The sensing module driving unit 41 controls the operation of the attitude sensing module 1 and converts the attitude sensing signal S output from the attitude sensing module 1 into the attitude data D. The input module driving unit 42 controls the user to configure the input module to work, recognizes the user command I, and generates the user command data; the output module driving unit 43 outputs the signal O, including the alarm data A and/or the attitude correction data R, and controls the alarm signal output module 3 to work. .
操作系统运行装置4中还包含姿态服务模块40，姿态服务模块40中还包含姿态检测单元46，读取传感模块驱动单元输出的姿态数据D、识别用户面部的位置、产生姿态参数G。The operating system running device 4 further includes a posture service module 40. The posture service module 40 further includes an attitude detecting unit 46 that reads the posture data D output by the sensing module driving unit, recognizes the position of the user's face, and generates the posture parameter G.
所述姿态服务模块40(或应用程序模块71，72，…，7N中至少一个)还包含姿态比较单元47，将所述姿态参数G与对应的门限值进行比较；所述姿态服务模块40(或应用程序模块71，72，…，7N中至少一个)还包含姿态提醒单元48，产生报警数据A和/或姿态矫正数据R。所述姿态服务模块40(或者应用程序模块71，72，…，7N中至少一个)产生报警数据A和/或姿态矫正数据R，启动输入模块驱动单元。The gesture service module 40 (or at least one of the application modules 71, 72, ..., 7N) further includes a posture comparison unit 47 that compares the attitude parameter G with a corresponding threshold value; the attitude service module 40 (or at least one of the application modules 71, 72, ..., 7N) further includes a gesture reminding unit 48 that generates alarm data A and/or attitude correction data R. The gesture service module 40 (or at least one of the application modules 71, 72, ..., 7N) generates alarm data A and/or attitude correction data R to activate the input module drive unit.
操作系统运行装置中还包含应用数据库44，所述应用数据库44存储工作数据和默 认设置数据。The operating system running device further includes an application database 44, which stores the working data and the default Confirm the setting data.
所述应用数据库44存储的工作数据，是指在本发明的装置工作中改变的数据，包括“应用程序状态表”、“姿态检测标志位”、“应用程序注册表”及所述姿态参数等。The work data stored in the application database 44 refers to data changed in the operation of the apparatus of the present invention, including "application status table", "attitude detection flag", "application registry", and posture parameters, etc. .
应用程序状态表，包含终端中应用程序模块工作状态的信息，例如该应用程序模块是否运行、是否占据显示屏等。The application status table contains information about the working state of the application module in the terminal, such as whether the application module is running, whether it occupies the display screen, and the like.
应用程序注册表，包含有检测必要的应用程序模块名称或系统分配的某种标识符。应用程序模块71，72，…，7N可以通过应用程序编程接口45把自己的名称或标识符添加到应用数据库44中。The application registry contains an identifier that detects the necessary application module name or system allocation. The application modules 71, 72, ..., 7N can add their own names or identifiers to the application database 44 via the application programming interface 45.
姿态检测标志位，是指操作系统运行装置4中姿态服务模块40是否运行的标志。The attitude detection flag bit is a flag indicating whether the attitude service module 40 in the operating system running device 4 is operating.
所述应用数据库44存储的默认设置数据，包含以下一种或多种：姿态检测标志位、应用程序注册表、门限值。这些数据也可以通过手动设置，其中姿态检测标志位和应用程序注册表还可能在工作中被重新设置。The default setting data stored by the application database 44 includes one or more of the following: an attitude detection flag bit, an application registry, and a threshold value. These data can also be set manually, where the gesture detection flag and the application registry may also be reset at work.
操作系统运行装置4中还包含应用程序编程接口45(API)。所述应用程序模块有一个或多个，所述应用程序模块71，72，…，7N装载并运行应用程序，通过所述应用程序编程接口45调用所述传感模块驱动单元41、输入模块驱动单元42、输出模块驱动单元43、姿态服务模块40、应用数据库44等组件。An application programming interface 45 (API) is also included in the operating system operating device 4. There are one or more application modules, and the application modules 71, 72, ..., 7N load and run an application, and the sensing module driving unit 41 and the input module are called by the application programming interface 45. The unit 42, the output module driving unit 43, the attitude service module 40, the application database 44, and the like.
作为本发明进一步优化的实施例，图15～16所示方案中，传感信号S进一步包含图像信号S1和角度传感信号S2。姿态数据D进一步包含用户姿态数据D1和本机姿态数据D2。姿态参数G进一步包含相对姿态参数G1和绝对姿态参数G2。G1、G2对应的门限值分别为T1、T2。As a further optimized embodiment of the present invention, in the solutions shown in FIGS. 15-16, the sensing signal S further includes an image signal S1 and an angle sensing signal S2. The posture data D further includes user posture data D1 and local posture data D2. The attitude parameter G further includes a relative attitude parameter G1 and an absolute attitude parameter G2. The threshold values corresponding to G1 and G2 are T1 and T2, respectively.
如图15所示，是本发明操作系统运行装置和应用程序模块的一个实施例，姿态传感模块1使用摄像头方案，包含一只或多只前置摄像头100，产生图像信号S1。传感模块驱动单元41中包含摄像头驱动单元411。姿态检测单元46读取摄像头驱动单元411输出的用户姿态数据D1，检测用户面部与显示屏6的相对位置，产生相对姿态参数G1。姿态服务模块40中还包含姿态比较单元47，将所述姿态参数G1与对应的门限值T1进行比较。As shown in FIG. 15, which is an embodiment of the operating system operating device and application module of the present invention, the attitude sensing module 1 uses a camera solution, including one or more front cameras 100, to generate an image signal S1. A camera driving unit 411 is included in the sensing module driving unit 41. The posture detecting unit 46 reads the user posture data D1 output from the camera driving unit 411, detects the relative position of the user's face and the display screen 6, and generates a relative posture parameter G1. The attitude service module 40 further includes a posture comparison unit 47 that compares the attitude parameter G1 with a corresponding threshold value T1.
操作系统运行装置4或应用程序模块(例如71)中还包含姿态提醒单元，产生报警数据A和/或姿态矫正数据R。The operating system operating device 4 or the application module (for example, 71) further includes a gesture reminding unit that generates alarm data A and/or attitude correction data R.
如图16所示，是本发明操作系统运行装置和应用程序模块的一个实施例，姿态传感模块1使用摄像头和本机姿态传感器相结合的方案。其中，姿态传感模块1包含一只或多只前置摄像头100，产生图像信号S1。姿态传感模块中还包含能检测显示屏所在平 面倾角的本机姿态传感器12(如重力传感器，又称为倾角传感器)，产生角度传感信号S2。传感模块驱动单元41中包含摄像头驱动单元411和本机姿态传感器(重力传感器、倾角传感器)的驱动单元412。姿态检测单元46读取摄像头驱动单元输出的用户姿态数据D1，检测用户面部与显示屏6的相对位置，产生相对姿态参数G1。姿态检测单元46读取本机姿态传感器驱动单元412输出本机姿态数据D2，确定本机显示屏6所在平面的倾角t1。姿态检测单元46根据相对姿态参数G1和本机显示屏6所在平面的倾角t1，产生绝对姿态参数G2。As shown in FIG. 16, which is an embodiment of the operating system operating device and the application module of the present invention, the attitude sensing module 1 uses a combination of a camera and a local attitude sensor. The attitude sensing module 1 includes one or more front cameras 100 to generate an image signal S1. The attitude sensing module also includes the ability to detect the flatness of the display screen. The face angle sensor 12 (such as a gravity sensor, also known as a tilt sensor) produces an angle sensing signal S2. The sensor module driving unit 41 includes a camera driving unit 411 and a driving unit 412 of a local attitude sensor (gravity sensor, tilt sensor). The posture detecting unit 46 reads the user posture data D1 output from the camera driving unit, detects the relative position of the user's face and the display screen 6, and generates a relative posture parameter G1. The posture detecting unit 46 reads the local posture sensor driving unit 412 to output the local posture data D2, and determines the inclination angle t1 of the plane in which the display screen 6 of the unit is located. The attitude detecting unit 46 generates an absolute attitude parameter G2 based on the inclination angle t1 of the relative posture parameter G1 and the plane in which the local display 6 is located.
操作系统运行装置或应用程序模块中还包含姿态比较单元，将所述姿态参数G1、G2与对应的门限值T1、T2进行比较；操作系统运行装置或应用程序模块中还包含姿态提醒单元，产生报警数据A和/或姿态矫正数据R。The operating system running device or the application module further includes a posture comparing unit, and comparing the posture parameters G1 and G2 with corresponding threshold values T1 and T2; the operating system running device or the application module further includes a posture reminding unit, Alarm data A and/or attitude correction data R are generated.
图17是操作系统运行装置和应用程序模块实现用户姿态检测的工作过程，包含如下步骤：FIG. 17 is a working process of the operating system running device and the application module implementing user posture detection, comprising the following steps:
第501步、配置输入：应用程序模块71，72，…，7N和/或操作系统运行装置4接受来自用户配置输入模块指令数据I； Step 501, configuration input: application modules 71, 72, ..., 7N and / or operating system operating device 4 accepts input data from the user configuration input module instruction I;
第502步、姿态检测：传感模块驱动单元41控制姿态传感模块采集传感信号S1、S2，并转换为姿态数据D1、D2；姿态服务模块40读取传感模块驱动单元41输出的姿态数据D1、D2，姿态检测单元对姿态数据进行分析，得出相对姿态参数G1和/或绝对姿态参数G2；Step 502: attitude detection: the sensing module driving unit 41 controls the attitude sensing module to acquire the sensing signals S1 and S2, and converts into the attitude data D1 and D2; the posture service module 40 reads the posture of the sensing module driving unit 41. Data D1, D2, the attitude detecting unit analyzes the attitude data to obtain a relative attitude parameter G1 and/or an absolute attitude parameter G2;
第503步、姿态比较：姿态比较单元将所述用户姿态参数G1、G2和门限值T1、T2进行比较，姿态正常时转第502步，否则第504步； Step 503, posture comparison: the posture comparison unit compares the user posture parameters G1, G2 and the threshold values T1, T2, and when the posture is normal, the process proceeds to step 502, otherwise the step 504;
第504步、姿态提醒：在所述用户姿态参数超过门限值时，姿态提醒单元生成报警数据A和/或姿态矫正数据R；告警信号输出模块向客户发出用户感觉可识别的信号，包含报警信息和/或矫正建议，告警输出后，转第502步。 Step 504, posture reminding: when the user posture parameter exceeds the threshold value, the posture reminding unit generates alarm data A and/or posture correction data R; the alarm signal output module sends a signal to the customer that the user feels identifiable, including the alarm After the information and / or correction suggestions, after the alarm is output, go to step 502.
在上述方法中，姿态检测的步骤在操作系统运行装置4中完成，姿态比较、姿态提醒的步骤可以由操作系统运行装置中的姿态服务模块40完成，也可以由应用程序模块71，72，…，7N完成。在上述方法中，为了降低检测的频率，从第503步、第504步回归到第502步时，经过一个延时步骤(第505步)。In the above method, the step of posture detection is completed in the operating system running device 4, and the steps of posture comparison and posture reminding may be completed by the posture service module 40 in the operating system running device, or may be performed by the application module 71, 72, ... , 7N completed. In the above method, in order to reduce the frequency of detection, from step 503, step 504 to step 502, a delay step (step 505) is passed.
图18是移动终端中操作系统运行装置和应用程序模块的实施例，操作系统运行装置实现姿态检测、姿态比较和姿态提醒。18 is an embodiment of an operating system running device and an application module in a mobile terminal, the operating system running device implementing posture detection, posture comparison, and gesture reminding.
操作系统运行装置4中包含传感模块驱动单元41、输入模块驱动单元42、输出模块驱动单元43，传感模块驱动单元输出传感控制信号、输入传感信号S；输入模块驱动 单元输出配置控制信号、接收用户指令I；输出模块驱动单元输出告警控制信号、以及包含报警数据A和/或姿态矫正数据R的输出信号O。The operating system running device 4 includes a sensing module driving unit 41, an input module driving unit 42, and an output module driving unit 43. The sensing module driving unit outputs a sensing control signal and an input sensing signal S; the input module drives The unit outputs a configuration control signal, receives a user command I; the output module drive unit outputs an alarm control signal, and an output signal O including the alarm data A and/or the attitude correction data R.
操作系统运行装置4中还包含姿态服务模块40，姿态服务模块40中包含姿态检测单元46、姿态比较单元47、姿态提醒单元48。所述姿态检测单元46启动传感模块驱动单元41，读取传感模块驱动单元41输出的姿态数据D，识别用户面部的位置，产生姿态参数G。所述姿态比较单元47，将所述姿态参数G与对应的门限值T进行比较；所述姿态提醒单元48，在用户姿态不正确时产生报警数据A和/或姿态矫正数据R，并启动所述输出模块驱动单元。The operating system operating device 4 further includes a posture service module 40. The posture service module 40 includes an attitude detecting unit 46, a posture comparing unit 47, and a posture reminding unit 48. The posture detecting unit 46 activates the sensing module driving unit 41, reads the posture data D output from the sensing module driving unit 41, recognizes the position of the user's face, and generates the posture parameter G. The posture comparing unit 47 compares the posture parameter G with a corresponding threshold value T; the posture reminding unit 48 generates alarm data A and/or posture correction data R when the user posture is incorrect, and starts The output module drives the unit.
操作系统运行装置4中还包含应用数据库44，所述应用数据库44存储工作数据和默认设置数据。The operating system running device 4 further includes an application database 44 that stores work data and default setting data.
作为本发明进一步优化的实施例，图18所示操作系统运行装置还包含以下部分：As a further optimized embodiment of the present invention, the operating system operating device shown in FIG. 18 further includes the following parts:
姿态服务模块40中还包含数据扫描单元49，所述数据扫描单元49查询所述应用数据库中的工作数据，按照所述工作数据的指示，在需要启动姿态检测过程时，触发姿态检测单元46。The gesture service module 40 further includes a data scanning unit 49 that queries the work data in the application database, and triggers the attitude detecting unit 46 when the gesture detection process needs to be initiated according to the indication of the working data.
操作系统运行装置4中还包含应用程序编程接口(API)45，所述应用程序模块71，72，…，7N通过所述应用程序编程接口45调用所述应用数据库44，对其中的数据进行设置或清除。The operating system running device 4 further includes an application programming interface (API) 45, and the application modules 71, 72, ..., 7N call the application database 44 through the application programming interface 45 to set data therein. Or clear.
图19是移动终端中操作系统运行装置和应用程序模块的实施例，操作系统运行装置进行姿态检测。19 is an embodiment of an operating system running device and an application module in a mobile terminal, the operating system running device performing posture detection.
操作系统运行装置4中包含传感模块驱动单元41、输入模块驱动单元42、输出模块驱动单元43，传感模块驱动单元输出传感控制信号、输入传感信号S；输入模块驱动单元输出配置控制信号、接收用户指令I；输出模块驱动单元输出告警控制信号、以及包含报警数据A和/或姿态矫正数据R的输出信号O。The operating system running device 4 includes a sensing module driving unit 41, an input module driving unit 42, and an output module driving unit 43, the sensing module driving unit outputs a sensing control signal, an input sensing signal S; and an input module driving unit output configuration control The signal receives the user command I; the output module drive unit outputs an alarm control signal and an output signal O including the alarm data A and/or the attitude correction data R.
操作系统运行装置4中还包含姿态服务模块40，姿态服务模块40中包含姿态检测单元46，所述姿态检测单元46启动传感模块驱动单元41，读取传感模块驱动单元41输出的姿态数据D，识别用户面部的位置，产生姿态参数G，将所述姿态参数保存在应用数据库中。The operating system running device 4 further includes a posture service module 40. The posture service module 40 includes an attitude detecting unit 46. The posture detecting unit 46 activates the sensing module driving unit 41 to read the posture data output by the sensing module driving unit 41. D, identifying the position of the user's face, generating a pose parameter G, and saving the pose parameter in the application database.
操作系统运行装置4中还包含应用数据库44，所述应用数据库44存储工作数据和默认设置数据。The operating system running device 4 further includes an application database 44 that stores work data and default setting data.
操作系统运行装置4中还包含应用程序编程接口(API)45。An application programming interface (API) 45 is also included in the operating system operating device 4.
作为本发明进一步优化的实施例，图19中所示操作系统运行装置4，还包含以下部 分：As a further optimized embodiment of the present invention, the operating system operating device 4 shown in FIG. 19 further includes the following Minute:
姿态服务模块40中还包含数据扫描单元49，所述数据扫描单元查询所述应用数据库44中的工作数据，按照所述工作数据的指示，在需要启动姿态检测过程时，触发姿态检测单元46。The gesture service module 40 further includes a data scanning unit 49 that queries the work data in the application database 44 to trigger the gesture detection unit 46 when the gesture detection process needs to be initiated according to the indication of the work data.
图20是移动终端中操作系统运行装置和应用程序模块的实施例，操作系统运行装置进行姿态检测和姿态比较。20 is an embodiment of an operating system running device and an application module in a mobile terminal, the operating system operating device performing posture detection and posture comparison.
操作系统运行装置4中包含传感模块驱动单元41、输入模块驱动单元42、输出模块驱动单元43，传感模块驱动单元输出传感控制信号、输入传感信号S；输入模块驱动单元输出配置控制信号、接收用户指令I；输出模块驱动单元输出告警控制信号、以及包含报警数据A和/或姿态矫正数据R的输出信号O。The operating system running device 4 includes a sensing module driving unit 41, an input module driving unit 42, and an output module driving unit 43, the sensing module driving unit outputs a sensing control signal, an input sensing signal S; and an input module driving unit output configuration control The signal receives the user command I; the output module drive unit outputs an alarm control signal and an output signal O including the alarm data A and/or the attitude correction data R.
操作系统运行装置4中还包含姿态服务模块40，姿态服务模块40中包含姿态检测单元46、姿态比较单元47，启动传感模块驱动单元41，读取传感模块驱动单元41输出的姿态数据D，识别用户面部的位置，产生姿态参数G；所述姿态比较单元47将所述姿态参数G与对应的门限值T进行比较，在姿态参数超出门限值时通过应用程序编程接口45向至少一个所述应用程序模块71，72，…，7N发出姿态超限通知。The operating system running device 4 further includes a posture service module 40. The posture service module 40 includes an attitude detecting unit 46 and a posture comparing unit 47, and activates the sensing module driving unit 41 to read the attitude data D output by the sensing module driving unit 41. Identifying the position of the user's face, generating a posture parameter G; the posture comparison unit 47 compares the attitude parameter G with a corresponding threshold value T, and at least through the application programming interface 45 when the attitude parameter exceeds the threshold value One of the application modules 71, 72, ..., 7N issues an attitude overrun notification.
操作系统运行装置4中还包含应用数据库44，所述应用数据库44存储工作数据和默认设置数据。The operating system running device 4 further includes an application database 44 that stores work data and default setting data.
至少一个所述应用程序模块(例如71)中包含姿态提醒单元48、输入输出处理单元70。所述姿态提醒单元48产生报警数据A和/或姿态矫正数据R。所述输入输出处理单元70，通过应用程序编程接口45设置应用数据库44中的工作数据、接收来自操作系统运行装置4的姿态超限通知、启动输入模块驱动单元42、启动输出模块驱动单元43。At least one of the application modules (eg, 71) includes a gesture reminding unit 48 and an input and output processing unit 70. The gesture reminding unit 48 generates alarm data A and/or attitude correction data R. The input/output processing unit 70 sets the work data in the application database 44 through the application programming interface 45, receives the attitude overrun notification from the operating system operating device 4, activates the input module driving unit 42, and activates the output module driving unit 43.
作为本发明进一步优化的实施例，图20中所示操作系统运行装置4，还包含以下部分：As a further optimized embodiment of the present invention, the operating system operating device 4 shown in FIG. 20 further includes the following parts:
姿态服务模块40中还包含数据扫描单元49，所述数据扫描单元查询所述应用数据库44中的工作数据，按照所述工作数据的指示，在需要启动姿态检测过程时，触发姿态检测单元46。The gesture service module 40 further includes a data scanning unit 49 that queries the work data in the application database 44 to trigger the gesture detection unit 46 when the gesture detection process needs to be initiated according to the indication of the work data.
本发明所述应用程序模块(例如71)中包含姿态比较单元47、姿态提醒单元48、输入输出处理单元70。姿态比较单元47将所述姿态参数G与对应的门限值T进行比较；所述姿态提醒单元48产生报警数据A和/或姿态矫正数据R。所述输入输出处理单元70，通过所述应用程序编程接口45设置或读取应用数据库44中的工作数据、启动输入模块驱动单元42、启动输出模块驱动单元43。 The application module (for example, 71) of the present invention includes a posture comparing unit 47, a posture reminding unit 48, and an input/output processing unit 70. The attitude comparison unit 47 compares the attitude parameter G with a corresponding threshold value T; the gesture reminding unit 48 generates alarm data A and/or attitude correction data R. The input/output processing unit 70 sets or reads the working data in the application database 44, activates the input module driving unit 42, and starts the output module driving unit 43 through the application programming interface 45.
在图18～20所示实施例中，当姿态服务模块中包含数据扫描单元49时，应用程序模块和数据扫描单元对工作数据处理过程举例如下：In the embodiment shown in FIG. 18 to FIG. 20, when the data scanning unit 49 is included in the attitude service module, the application module and the data scanning unit process the working data as follows:
过程一：应用程序模块71，72，…，7N中的至少一个(例如71)占据显示屏时，通过应用程序编程接口45修改应用数据库44中的“应用程序状态表”，写入该应用程序模块的名称(或系统分配的某种标识符)。用户通过用户配置输入模块2设置“应用程序列注册表”，某些应用程序模块也可以自己主动修改“应用程序注册表”，将自己加入到这个列表中。姿态服务模块40中的数据扫描单元49查询应用数据库的工作数据，所述工作数据包含应用程序状态表和应用程序注册表；按照所述工作数据的指示，是指所述数据扫描单元从应用程序状态表中获得正在使用显示屏的应用程序模块名称或标识符、检查所述应用程序模块是否在应用程序注册表中，如是，则需要启动姿态检测过程。Process 1: When at least one of the application modules 71, 72, ..., 7N (e.g., 71) occupies the display screen, the "Application Status Table" in the application database 44 is modified by the application programming interface 45 to write the application. The name of the module (or some identifier assigned by the system). The user sets the "application column registry" through the user configuration input module 2, and some application modules can also modify the "application registry" by themselves and add themselves to this list. The data scanning unit 49 in the gesture service module 40 queries the work data of the application database, where the work data includes an application state table and an application registry; according to the indication of the work data, the data scanning unit is from the application. The status table obtains the application module name or identifier that is using the display, checks if the application module is in the application registry, and if so, initiates the gesture detection process.
过程二：操作系统运行装置4提供的应用程序编程接口45用于设置或清除“姿态检测标志位”。应用程序模块在占用显示屏且有必要对用户进行姿态检测时，调用该应用程序编程接口，设置该标志位，从而激活姿态检测功能。姿态服务模块40中的数据扫描单元49查询应用数据库的工作数据，所述工作数据包含姿态检测标志位；按照所述工作数据的指示，是指所述数据扫描单元查询所述姿态检测标志位是否被设置，如是，则需要启动姿态检测过程。应用程序模块退出或其它不需要对姿态进行监测的情况发生时清除该标志位。Process 2: The application programming interface 45 provided by the operating system running device 4 is used to set or clear the "attitude detection flag bit". When the application module occupies the display screen and it is necessary to perform posture detection on the user, the application programming interface is called to set the flag bit to activate the attitude detection function. The data scanning unit 49 in the gesture service module 40 queries the work data of the application database, and the work data includes a posture detection flag bit. According to the indication of the work data, the data scanning unit queries whether the posture detection flag bit is queried. Is set, if so, you need to start the attitude detection process. This flag is cleared when an application module exits or other conditions that do not require monitoring of the pose.
针对以上各个实施例，具体说明用户功能设置方案。操作系统运行装置4和至少一个应用程序模块(例如71)接受来自用户配置输入模块指令数据I。在输入模块驱动单元42的控制下，识别用户配置输入模块2输入的用户指令。输入模块驱动单元既可以由操作系统运行装置4启动，也可以由应用程序模块(例如71)中输入输出处理单元70通过应用程序编程接口45启动。指令数据至少包括以下指令中一种：1.应用程序选择指令，用于设置“应用程序注册表”。只有前台运行这些应用程序时该后台服务才进行姿态检测。某些应用程序也可以自己主动加入到这个列表中。通过设置让该功能在某些应用中不被启用。如前台应用程序是接听、拨打电话、听收音机或其它音频内容时，由于此时用户不会长时间关注显示屏，此时不必要开启本功能。2.启动指令，用于设置“姿态检测标志位”。直接通过设置该标志位启动或禁止姿态检测功能。3.门限指令，用于设置姿态参数门限值。通过设置门限值适应不同用户的需求。在没有用户指令时，上述指令数据在操作系统运行装置4的应用数据库44中存在默认设置数据。For the above various embodiments, the user function setting scheme is specifically described. The operating system operating device 4 and at least one application module (e.g., 71) accept command data I from the user configuration input module. Under the control of the input module driving unit 42, the user command input by the user configuration input module 2 is recognized. The input module drive unit can be activated either by the operating system operating device 4 or by the input/output processing unit 70 of the application module (e.g., 71) via the application programming interface 45. The instruction data includes at least one of the following instructions: 1. An application selection instruction for setting an "application registry". The background service performs gesture detection only when the application is running in the foreground. Some applications can also actively join this list on their own. By setting this feature is not enabled in some applications. If the foreground application is answering, making a call, listening to radio or other audio content, since the user does not pay attention to the display for a long time, it is not necessary to turn on this function. 2. Start command to set the "attitude detection flag". The gesture detection function is activated or disabled directly by setting this flag bit. 3. Threshold command for setting the attitude parameter threshold. By setting thresholds to meet the needs of different users. In the absence of a user command, the above command data has default setting data in the application database 44 of the operating system running device 4.
针对以上各个实施例，具体说明告警输出方案。在操作系统运行装置4中输出模块 驱动单元43的控制下，告警信号输出模块3处理报警数据A和姿态矫正数据R。报警数据是指用户姿态参数超出门限值时生成的，包含报警的方式、报警的强度、报警的时间长短等特征，这些数据是根据用户偏离正确姿态的程度和时间来确定的。姿态矫正数据是根据姿态参数和门限值之间的差，得到如何改变用户姿态以达到正确姿态的数据，包括距离、角度、及调整的方向等信息。报警输出模块输出的信号是用户感官可识别的信号，包含以下一种或多种类型：通过音频模块发出来的提示音、通过振动设备发出来的振动，通过显示屏给出的图形或文字提示。For the above embodiments, the alarm output scheme is specifically described. Output module in operating system running device 4 Under the control of the drive unit 43, the alarm signal output module 3 processes the alarm data A and the attitude correction data R. The alarm data is generated when the user's attitude parameter exceeds the threshold, including the mode of the alarm, the strength of the alarm, and the length of the alarm. These data are determined according to the degree and time when the user deviates from the correct posture. The attitude correction data is based on the difference between the attitude parameter and the threshold value, and obtains information on how to change the user posture to achieve the correct posture, including distance, angle, and direction of adjustment. The signal output by the alarm output module is a sensory identifiable signal of the user, and includes one or more of the following types: a prompt sound emitted by the audio module, a vibration generated by the vibration device, a graphic or text prompt given through the display screen. .
下面分别说明图18～20的三种移动终端。The three types of mobile terminals of Figs. 18 to 20 will be separately described below.
第一种。The first.
在图18中，通过操作系统实现用户姿态检测的移动终端5，包含姿态传感模块1、用户配置输入模块2、告警信号输出模块3、操作系统运行装置4、显示屏(图18中未示出)、一个或多个应用程序模块71，72，…，7N。In FIG. 18, the mobile terminal 5 that implements user posture detection by an operating system includes an attitude sensing module 1, a user configuration input module 2, an alarm signal output module 3, an operating system running device 4, and a display screen (not shown in FIG. 18). Out, one or more application modules 71, 72, ..., 7N.
姿态传感模块产生传感信号S。用户配置输入模块2识别用户操作，将用户指令I传递至操作系统运行装置4。告警信号输出模块3发出用户感官可识别的信号。The attitude sensing module generates a sensing signal S. The user configuration input module 2 recognizes the user operation and passes the user command I to the operating system operating device 4. The alarm signal output module 3 emits a signal recognizable by the user.
图21是基于图18的操作系统运行装置4自动运行姿态检测、姿态比较和姿态提醒的方案流程图。操作系统运行装置4中的姿态服务模块40包含姿态检测单元46、姿态比较单元47、姿态提醒单元48。姿态服务模块40与应用程序模块71，72，…，7N并行工作，当发现用户姿态不正确时发出提醒。工作步骤如下：21 is a flow chart of a scheme based on the automatic operation posture detection, posture comparison, and posture reminding of the operating system running device 4 of FIG. The posture service module 40 in the operating system operating device 4 includes an attitude detecting unit 46, a posture comparing unit 47, and a posture reminding unit 48. The gesture service module 40 works in parallel with the application modules 71, 72, ..., 7N to alert when a user gesture is found to be incorrect. The work steps are as follows:
第602步、姿态检测：姿态检测单元46启动传感模块驱动单元41，控制姿态传感模块1，采集传感信号S1、S2，并转换为姿态数据D1、D2；姿态服务模块40读取传感模块驱动单元41输出的姿态数据D1、D2；姿态服务模块40中的姿态检测单元46对姿态数据进行分析，得出相对姿态参数G1和/或绝对姿态参数G2； Step 602, posture detection: the attitude detecting unit 46 activates the sensing module driving unit 41, controls the attitude sensing module 1, collects the sensing signals S1, S2, and converts into attitude data D1, D2; the attitude service module 40 reads and transmits The attitude data D1, D2 output by the sensor module driving unit 41; the attitude detecting unit 46 in the attitude service module 40 analyzes the attitude data to obtain a relative attitude parameter G1 and/or an absolute attitude parameter G2;
第603步、姿态比较：姿态服务模块40中的姿态比较单元47将所述姿态参数G1、G2和用户姿态门限值T1、T2进行比较，姿态正常时转第605步，否则转第604步； Step 603, posture comparison: the posture comparison unit 47 in the attitude service module 40 compares the posture parameters G1, G2 and the user posture threshold values T1, T2, and when the posture is normal, the process proceeds to step 605, otherwise the step 604 is changed. ;
第604步、姿态提醒：姿态服务模块40中的姿态提醒48单元在所述姿态参数超过门限值时，生成报警数据A和/或姿态矫正数据R；告警信号输出模块48向客户发出用户感觉可识别的信号，包含报警信息和/或矫正建议，告警输出后，转第605步。 Step 604, posture reminder: the posture reminder 48 unit in the attitude service module 40 generates alarm data A and/or attitude correction data R when the attitude parameter exceeds the threshold value; the alarm signal output module 48 sends a user feeling to the customer. The identifiable signal contains alarm information and/or correction suggestions. After the alarm is output, go to step 605.
第605步、等待，延时后转第602步。 Step 605, wait, delay and then go to step 602.
图21所示方法实施例中，姿态检测服务程序和应用程序模块之间不存在数据交换。例如，在Android或iOS系统中，可以包含一个姿态服务模块。将姿势测量作为一个系统后台服务，它调用其它的系统服务，获得前置摄像头的输出，对姿势进行测量，当姿 势不正确时，该后台服务切换至前台，对用户进行提示。In the method embodiment shown in Figure 21, there is no data exchange between the gesture detection service program and the application module. For example, in Android or iOS, you can include a gesture service module. Using posture measurement as a system background service, it calls other system services, obtains the output of the front camera, and measures the posture. When the situation is not correct, the background service switches to the foreground and prompts the user.
图22是基于图18的操作系统运行装置按应用程序模块的需要运行姿态检测、姿态比较和姿态提醒的方案流程图。操作系统运行装置40中的姿态服务模块包含姿态检测单元46、姿态比较单元47、姿态提醒单元48、数据扫描单元49。姿态服务模块40与应用程序模块71，72，…，7N并行工作，姿态服务模块40中的数据扫描单元49每隔一段时间检测当前运行的应用程序模块，当有必要监测用户姿态时，启动姿态检测单元46工作。22 is a flow chart of a scheme based on the operation of the operating system of FIG. 18 to perform attitude detection, attitude comparison, and gesture reminding according to the needs of the application module. The posture service module in the operating system running device 40 includes an attitude detecting unit 46, a posture comparing unit 47, a posture reminding unit 48, and a data scanning unit 49. The gesture service module 40 works in parallel with the application modules 71, 72, ..., 7N. The data scanning unit 49 in the gesture service module 40 detects the currently running application module at intervals, and when it is necessary to monitor the user gesture, the gesture is initiated. The detection unit 46 operates.
应用程序模块71，72，…，7N占据显示屏时，通过应用程序编程接口45修改应用数据库44中的“应用程序状态表”，写入该应用程序模块的名称(或系统分配的某种标识符)。When the application modules 71, 72, ..., 7N occupy the display screen, the "application status table" in the application database 44 is modified by the application programming interface 45, and the name of the application module (or some identifier assigned by the system) is written. symbol).
用户可以通过用户配置输入模块2设置“应用程序注册表”，这些应用程序占据显示屏时进行姿态检测和提醒。某些应用程序模块也可以自己主动修改“应用程序注册表”，将自己加入到这个列表中。The user can configure the "application registry" through the user configuration input module 2, which performs posture detection and reminding when occupying the display screen. Some application modules can also modify the "application registry" by themselves and add themselves to this list.
数据扫描、姿态检测、姿态比较和姿态提醒的步骤如下：The steps for data scanning, attitude detection, attitude comparison, and gesture reminder are as follows:
第702a步、数据扫描单元49从应用数据库44中的“应用程序状态表”中获得正在使用显示屏的应用程序模块名称(或系统分配的某种标识符)； Step 702a, the data scanning unit 49 obtains the application module name (or some identifier assigned by the system) that is using the display screen from the "application status table" in the application database 44;
第702b步、数据扫描单元49检查该应用程序模块是否在“应用程序注册表”中，如是，则转第703步，触发姿态检测单元46；否则转第702c步； Step 702b, the data scanning unit 49 checks whether the application module is in the "application registry", and if so, proceeds to step 703 to trigger the gesture detecting unit 46; otherwise, the step 702c is performed;
第702c步、延时等待后转入第702a步； Step 702c, delay waiting and then proceed to step 702a;
第703步、姿态检测：姿态检测单元46启动传感模块驱动单元41，控制姿态传感模块1采集传感信号S1、S2，并转换为姿态数据D1、D2；姿态检测单元46读取传感模块驱动单元输出的姿态数据D1、D2；姿态检测单元46对姿态数据进行分析，得出相对姿态参数G1和/或绝对姿态参数G2； Step 703, attitude detection: the attitude detecting unit 46 activates the sensing module driving unit 41, controls the attitude sensing module 1 to acquire the sensing signals S1, S2, and converts into the attitude data D1, D2; the attitude detecting unit 46 reads the sensing The attitude data D1, D2 output by the module driving unit; the attitude detecting unit 46 analyzes the attitude data to obtain a relative attitude parameter G1 and/or an absolute attitude parameter G2;
第704步、姿态比较：姿态服务模块40中的姿态比较单元47将所述姿态参数G1、G2和用户姿态门限值T1、T2进行比较，姿态正常时转第706步，否则转第705步；Step 704: attitude comparison: the attitude comparison unit 47 in the attitude service module 40 compares the posture parameters G1 and G2 with the user posture threshold values T1 and T2, and if the posture is normal, the operation proceeds to step 706. Otherwise, the operation proceeds to step 705. ;
第705步、姿态提醒：姿态服务模块40中的姿态提醒单元48在所述姿态参数超过门限值时，生成报警数据A和/或姿态矫正数据R；告警信号输出模块43向客户发出用户感觉可识别的信号，包含报警信息和/或矫正建议，告警输出后， Step 705, posture reminder: the posture reminding unit 48 in the attitude service module 40 generates the alarm data A and/or the attitude correction data R when the posture parameter exceeds the threshold value; the alarm signal output module 43 sends the user feeling to the customer. An identifiable signal containing alarm information and/or correction suggestions. After the alarm is output,
第706步、等待，延时后转第702a步。 Step 706, wait, delay and then go to step 702a.
图23是基于图18的应用程序模块使用标志位启动和关闭操作系统运行装置中姿态服务模块的方案流程图。操作系统运行装置4中的姿态服务模块40包含姿态检测单元 46、姿态比较单元47、姿态提醒单元48、数据扫描单元49。姿态服务模块40与应用程序模块71，72，…，7N并行工作。操作系统运行装置4提供的应用程序编程接口45用于设置或清除“姿态检测标志位”。应用程序模块在占用显示屏且有必要对用户进行姿态检测时，调用该应用程序编程接口，设置该标志位，从而激活了姿态检测功能。应用程序模块退出或其它不需要对姿态进行监测的情况发生时清除该标志位。当有多个应用程序模块运行时，每个应用程序模块都可以对该标志位进行设置，所有设置过该标志位的应用程序模块中只要有一个没有清除该标志位，那么姿态检测与提醒功能就是被激活的。23 is a flow chart of a scheme for starting and shutting down a gesture service module in an operating system running device based on the application module of FIG. The attitude service module 40 in the operating system running device 4 includes an attitude detecting unit 46. The posture comparison unit 47, the posture reminding unit 48, and the data scanning unit 49. The gesture service module 40 operates in parallel with the application modules 71, 72, ..., 7N. The application programming interface 45 provided by the operating system operating device 4 is used to set or clear the "attitude detection flag". When the application module occupies the display screen and it is necessary to perform posture detection on the user, the application programming interface is called to set the flag bit, thereby activating the attitude detection function. This flag is cleared when an application module exits or other conditions that do not require monitoring of the pose. When there are multiple application modules running, each application module can set the flag bit. If any of the application modules that have set the flag bit does not clear the flag, then the gesture detection and reminder function It is activated.
图23(a)表示应用程序模块71，72，…，7N的工作过程，步骤如下：Figure 23 (a) shows the working process of the application modules 71, 72, ..., 7N, the steps are as follows:
第800步、应用程序模块通过应用程序编程接口设置或清除应用数据库44中的“姿态检测标志位”； Step 800, the application module sets or clears the "attitude detection flag bit" in the application database 44 through the application programming interface;
第801步、应用程序模块运行应用程序主功能，完毕后再转入第800步；In step 801, the application module runs the main function of the application, and then proceeds to step 800 after completing;
图23(b)表示姿态服务模块40的工作过程，步骤如下：Figure 23 (b) shows the working process of the attitude service module 40, the steps are as follows:
第802步、数据扫描单元49查询所述“姿态检测标志位”是否被设置，如是，则转第803步，触发姿态检测单元46；否则转第806步； Step 802, the data scanning unit 49 queries whether the "attitude detection flag bit" is set, and if so, then proceeds to step 803 to trigger the attitude detecting unit 46; otherwise, the process proceeds to step 806;
第803步、姿态检测：姿态检测单元46启动传感模块驱动单元41，控制姿态传感模块1，采集传感信号S1、S2，并转换为姿态数据D1、D2；姿态检测单元46读取传感模块驱动单元输出的姿态数据D1、D2；姿态检测单元46对姿态数据进行分析，得出相对姿态参数G1和/或绝对姿态参数G2； Step 803, posture detection: the attitude detecting unit 46 activates the sensing module driving unit 41, controls the attitude sensing module 1, collects the sensing signals S1, S2, and converts them into attitude data D1, D2; the posture detecting unit 46 reads and transmits The attitude data D1, D2 output by the module driving unit; the attitude detecting unit 46 analyzes the attitude data to obtain a relative attitude parameter G1 and/or an absolute attitude parameter G2;
第804步、姿态比较：姿态服务模块40中的姿态比较单元47将所述姿态参数G1、G2和用户姿态门限值T1、T2进行比较，姿态正常时转第806步，否则转第805步； Step 804, posture comparison: the posture comparison unit 47 in the attitude service module 40 compares the posture parameters G1 and G2 with the user posture threshold values T1 and T2, and if the posture is normal, the operation proceeds to step 806, otherwise the operation proceeds to step 805. ;
第805步、姿态提醒：姿态服务模块40中的姿态提醒单元48在所述姿态参数超过门限值时，生成报警数据A和/或姿态矫正数据R；告警信号输出模块43向客户发出用户感觉可识别的信号，包含报警信息和/或矫正建议，告警输出后，转第806步； Step 805, posture reminder: the posture reminding unit 48 in the attitude service module 40 generates alarm data A and/or posture correction data R when the posture parameter exceeds the threshold value; the alarm signal output module 43 sends a user feeling to the customer. An identifiable signal containing alarm information and/or correction suggestions. After the alarm is output, go to step 806;
第806步、等待，延时后转入第802步。 Step 806, wait, delay and then proceed to step 802.
第二种。Second.
在图19中，通过操作系统实现用户姿态检测的移动终端5，包含姿态传感模块1、用户配置输入模块2、告警信号输出模块3、操作系统运行装置4、显示屏(图19中未示出)、一个或多个应用程序模块71，72，…，7N。In FIG. 19, the mobile terminal 5 that implements user gesture detection by an operating system includes an attitude sensing module 1, a user configuration input module 2, an alarm signal output module 3, an operating system running device 4, and a display screen (not shown in FIG. 19). Out, one or more application modules 71, 72, ..., 7N.
姿态传感模块产生传感信号S。用户配置输入模块2识别用户操作，将用户指令I传递至操作系统运行装置4和至少一个所述应用程序模块(例如71)。告警信号输出模 块3发出用户感官可识别的信号。The attitude sensing module generates a sensing signal S. The user configuration input module 2 identifies the user operation and passes the user command I to the operating system operating device 4 and at least one of the application modules (e.g., 71). Alarm signal output mode Block 3 signals the user's sensory identifiable.
至少一个所述应用程序模块(例如71)中包含姿态比较单元47、姿态提醒单元48、输入输出处理单元70。姿态比较单元47将所述姿态参数G与对应的门限值T进行比较；所述姿态提醒单元48产生报警数据A和/或姿态矫正数据R。所述输入输出处理单元70，通过应用程序编程接口45读取应用数据库44中的数据、启动输入模块驱动单元42、启动输出模块驱动单元43。At least one of the application modules (eg, 71) includes a gesture comparison unit 47, a gesture reminder unit 48, and an input and output processing unit 70. The attitude comparison unit 47 compares the attitude parameter G with a corresponding threshold value T; the gesture reminding unit 48 generates alarm data A and/or attitude correction data R. The input/output processing unit 70 reads the data in the application database 44 through the application programming interface 45, activates the input module driving unit 42, and activates the output module driving unit 43.
图24是基于图19的实施例，在图24中，应用程序模块71直接读取应用数据库44中的姿态参数。操作系统运行装置4中的姿态服务模块40仅包含姿态检测单元46；应用程序模块71中包含姿态比较单元47、姿态提醒单元48、输入输出处理单元70。操作系统运行装置4中姿态服务模块40自动运行并将姿态参数保存在应用数据库44，形成姿态参数库。应用程序模块71中输入输出处理单元70通过操作系统运行装置4提供的应用程序编程接口45周期性地读取这些姿态参数。24 is an embodiment based on FIG. 19 in which the application module 71 directly reads the pose parameters in the application database 44. The posture service module 40 in the operating system running device 4 includes only the posture detecting unit 46; the application program 71 includes a posture comparing unit 47, a posture reminding unit 48, and an input/output processing unit 70. The attitude service module 40 in the operating system running device 4 automatically runs and saves the posture parameters in the application database 44 to form a posture parameter library. The input/output processing unit 70 in the application module 71 periodically reads these posture parameters through the application programming interface 45 provided by the operating system running device 4.
图24(a)表示操作系统运行装置中的工作步骤：Figure 24 (a) shows the working steps in the operating system operating device:
第901步、姿态检测：姿态检测单元46启动传感模块驱动单元41，控制姿态传感模块1采集传感信号S1、S2，并转换为姿态数据D1、D2；姿态检测单元46读取传感模块驱动单元输出的姿态数据D1、D2；姿态检测单元46对姿态数据进行分析，得出相对姿态参数G1和/或绝对姿态参数G2； Step 901, posture detection: the attitude detecting unit 46 activates the sensing module driving unit 41, controls the attitude sensing module 1 to acquire the sensing signals S1, S2, and converts into the attitude data D1, D2; the attitude detecting unit 46 reads the sensing The attitude data D1, D2 output by the module driving unit; the attitude detecting unit 46 analyzes the attitude data to obtain a relative attitude parameter G1 and/or an absolute attitude parameter G2;
第902步、数据存储：姿态检测单元46产生的姿态参数被保存在应用数据库44中。 Step 902, Data Storage: The pose parameters generated by the attitude detecting unit 46 are stored in the application database 44.
第903步、等待，延时后转第901步。 Step 903, wait, delay and then go to step 901.
图24(b)表示应用程序模块中的工作步骤：Figure 24(b) shows the working steps in the application module:
第904步、应用程序模块71执行主程序的功能 Step 904, the application module 71 performs the function of the main program
第905步、数据读取：应用程序模块71中输入输出处理单元70通过操作系统运行装置4提供的应用程序编程接口45读取应用数据库44中的姿态参数和门限值数据； Step 905, data reading: the input and output processing unit 70 in the application module 71 reads the attitude parameter and the threshold data in the application database 44 through the application programming interface 45 provided by the operating system running device 4;
第906步、姿态比较：应用程序模块71中的姿态比较单元47将所述姿态参数G1、G2和门限值T1、T2进行比较，姿态正常时转第904步，否则转第907步； Step 906, posture comparison: the posture comparison unit 47 in the application module 71 compares the posture parameters G1, G2 and the threshold values T1, T2, and when the posture is normal, the process proceeds to step 904, otherwise the process proceeds to step 907;
第907步、姿态提醒：应用程序模块71中的姿态提醒单元48在所述姿态参数超过门限值时，生成报警数据A和/或姿态矫正数据R；输入输出处理单元70通过操作系统运行装置4提供的应用程序编程接口45，启动输出模块驱动单元43，控制告警信号输出模块3向客户发出用户感觉可识别的信号，包含报警信息和/或矫正建议，告警输出后，转第904步。 Step 907, posture reminder: the posture reminding unit 48 in the application module 71 generates alarm data A and/or posture correction data R when the posture parameter exceeds the threshold value; the input/output processing unit 70 runs the device through the operating system. The application programming interface 45 is provided to activate the output module driving unit 43 to control the alarm signal output module 3 to send a signal to the customer that the user feels identifiable, including alarm information and/or correction suggestions. After the alarm is output, the process proceeds to step 904.
图25是基于图19的实施例，在图25中，应用程序模块71使用姿态检测标志位来 启动和关闭操作系统运行装置4的姿态检测单元46、并读取应用数据库44中保存的结果。操作系统运行装置4中的姿态服务模块40包含姿态检测单元46、数据扫描单元49。姿态比较单元47和姿态提醒单元48则包含在应用程序模块71中。应用程序模块71中的输入输出处理单元70通过操作系统运行装置4提供的应用程序编程接口45(API)来设置“姿态检测标志位”。数据扫描单元49周期性查询该标志位，当该标志位被设置时运行姿态检测单元46并将姿态参数保存在应用数据库44，形成姿态参数库。应用程序模块71通过操作系统运行装置提供的应用程序编程接口45来读取这些姿态参数，并根据姿态参数来决定是否进行提醒。Figure 25 is an embodiment based on Figure 19, in Figure 25, the application module 71 uses the attitude detection flag to The posture detecting unit 46 of the operating system running device 4 is turned on and off, and the result stored in the application database 44 is read. The attitude service module 40 in the operating system operating device 4 includes an attitude detecting unit 46 and a data scanning unit 49. The gesture comparison unit 47 and the gesture reminder unit 48 are included in the application module 71. The input/output processing unit 70 in the application module 71 sets the "attitude detection flag bit" through the application programming interface 45 (API) provided by the operating system running device 4. The data scanning unit 49 periodically queries the flag bit, and when the flag bit is set, runs the attitude detecting unit 46 and saves the posture parameter in the application database 44 to form a posture parameter library. The application module 71 reads these gesture parameters through the application programming interface 45 provided by the operating system running device, and determines whether to make a reminder according to the posture parameters.
图25(a)是操作系统运行装置中的工作步骤：Figure 25 (a) is the working steps in the operating system operating device:
第1001步、数据扫描单元49查询所述“姿态检测标志位”是否被设置，如是，则转第1002步，触发姿态检测单元46；否则转第1004步； Step 1001, the data scanning unit 49 queries whether the "attitude detection flag bit" is set, and if so, proceeds to step 1002, triggers the attitude detecting unit 46; otherwise, proceeds to step 1004;
第1002步、姿态检测：姿态检测单元46启动传感模块驱动单元41，控制姿态传感模块1采集传感信号S1、S2，并转换为姿态数据D1、D2；姿态检测单元46读取传感模块驱动单元输出的姿态数据D1、D2；姿态检测单元46对姿态数据进行分析，得出相对姿态参数G1和/或绝对姿态参数G2； Step 1002, posture detection: the attitude detecting unit 46 activates the sensing module driving unit 41, controls the attitude sensing module 1 to acquire the sensing signals S1, S2, and converts into the attitude data D1, D2; the attitude detecting unit 46 reads the sensing The attitude data D1, D2 output by the module driving unit; the attitude detecting unit 46 analyzes the attitude data to obtain a relative attitude parameter G1 and/or an absolute attitude parameter G2;
第1003步、数据存储：姿态检测单元46产生的姿态参数被保存在应用数据库44中； Step 1003, data storage: the posture parameters generated by the attitude detecting unit 46 are saved in the application database 44;
第1004步、等待，延时后转第601步。 Step 1004, wait, delay and then go to step 601.
图25(b)是应用程序模块中的工作步骤：Figure 25(b) shows the working steps in the application module:
第1005步、应用程序模块71输入输出处理单元70通过应用程序编程接口45设置或清除应用数据库44中的“姿态检测标志位”； Step 1005, the application module 71 input and output processing unit 70 sets or clears the "attitude detection flag bit" in the application database 44 through the application programming interface 45;
第1006步、延时等待，或执行其它功能； Step 1006, wait for a delay, or perform other functions;
第1007步、数据读取：应用程序模块71中输入输出处理单元70通过操作系统运行装置4提供的应用程序编程接口45读取应用数据库44中的姿态参数和门限值数据； Step 1007, data reading: the input/output processing unit 70 in the application module 71 reads the attitude parameter and the threshold data in the application database 44 through the application programming interface 45 provided by the operating system running device 4;
第1008步、姿态比较：应用程序模块71中的姿态比较单元47将所述姿态参数G1、G2和门限值T1、T2进行比较，姿态正常时转第1005步，否则转第1009步； Step 1008, posture comparison: the posture comparison unit 47 in the application module 71 compares the posture parameters G1, G2 and the threshold values T1, T2, and when the posture is normal, the process proceeds to step 1005, otherwise, the process proceeds to step 1009;
第1009步、姿态提醒：应用程序模块71中的姿态提醒单元48在所述姿态参数超过门限值时，生成报警数据A和/或姿态矫正数据R；输入输出处理单元70通过操作系统运行装置4提供的应用程序编程接口45，启动输出模块驱动单元43，控制告警信号输出模块向客户发出用户感觉可识别的信号，包含报警信息和/或矫正建议，告警输出后，转第1005步。 Step 1009, posture reminder: the posture reminding unit 48 in the application module 71 generates alarm data A and/or posture correction data R when the posture parameter exceeds the threshold value; the input/output processing unit 70 runs the device through the operating system. The application programming interface 45 is provided to activate the output module driving unit 43, and the control alarm signal output module sends a signal to the customer that the user feels identifiable, including the alarm information and/or the correction suggestion. After the alarm is output, the process proceeds to step 1005.
图26是基于图19的操作系统运行装置按照应用程序状态表启动姿态检测的方案流程图。操作系统运行装置40中的姿态服务模块包含姿态检测单元46、数据扫描单元49。姿态服务模块40与应用程序模块71，72，…，7N并行工作，姿态服务模块40中的数据扫描单元49每隔一段时间检测当前运行的应用程序模块，当有必要监测用户姿态时，启动姿态检测单元46工作。26 is a flow chart of a scheme for starting an attitude detection according to an application state table by the operating system operating device of FIG. The attitude service module in the operating system operating device 40 includes an attitude detecting unit 46 and a data scanning unit 49. The gesture service module 40 works in parallel with the application modules 71, 72, ..., 7N. The data scanning unit 49 in the gesture service module 40 detects the currently running application module at intervals, and when it is necessary to monitor the user gesture, the gesture is initiated. The detection unit 46 operates.
至少一个所述应用程序模块(例如71)占据显示屏时，通过应用程序编程接口45修改应用数据库44中的“应用程序状态表”，写入该应用程序模块的名称(或系统分配的某种标识符)。When at least one of the application modules (eg, 71) occupies the display screen, the "application status table" in the application database 44 is modified by the application programming interface 45, and the name of the application module is written (or some type assigned by the system) Identifier).
用户可以通过用户配置输入模块2设置“应用程序注册表”，这些应用程序占据显示屏时进行姿态检测和提醒。某些应用程序模块也可以自己主动修改“应用程序注册表”，将自己加入到这个列表中。The user can configure the "application registry" through the user configuration input module 2, which performs posture detection and reminding when occupying the display screen. Some application modules can also modify the "application registry" by themselves and add themselves to this list.
图26(a)是操作系统运行装置中的工作步骤：Figure 26 (a) is the working steps in the operating system operating device:
第1101a步、判断应用程序模块(例如71)是否占用显示屏：数据扫描单元49从应用数据库44中的“应用程序状态表”中获得正在使用显示屏的应用程序模块名称(或系统分配的某种标识符)，如是，则1101b步，否则转第1104步； Step 1101a, judging whether the application module (for example, 71) occupies the display screen: the data scanning unit 49 obtains the application module name (or system allocation) that is using the display screen from the "application status table" in the application database 44. Kind of identifier), if yes, step 1101b, otherwise go to step 1104;
第1101b步、数据扫描单元49检查应用程序模块71是否在“应用程序注册表”中，如是，则转第1102步，触发姿态检测单元46；否则转第1104步； Step 1101b, the data scanning unit 49 checks whether the application module 71 is in the "application registry", and if so, proceeds to step 1102, triggering the gesture detecting unit 46; otherwise, proceeds to step 1104;
第1102步、姿态检测：姿态检测单元46启动传感模块驱动单元41，控制姿态传感模块1采集传感信号S1、S2，并转换为姿态数据D1、D2；姿态检测单元46读取传感模块驱动单元输出的姿态数据D1、D2；姿态检测单元46对姿态数据进行分析，得出相对姿态参数G1和/或绝对姿态参数G2； Step 1102, posture detection: the attitude detecting unit 46 activates the sensing module driving unit 41, controls the attitude sensing module 1 to acquire the sensing signals S1, S2, and converts into the attitude data D1, D2; the posture detecting unit 46 reads the sensing The attitude data D1, D2 output by the module driving unit; the attitude detecting unit 46 analyzes the attitude data to obtain a relative attitude parameter G1 and/or an absolute attitude parameter G2;
第1103步、数据存储：姿态检测单元46产生的姿态参数被保存在应用数据库44中。 Step 1103, Data Storage: The pose parameters generated by the attitude detecting unit 46 are stored in the application database 44.
第1104步、等待，延时后转第1101a步。 Step 1104, wait, delay and then go to step 1101a.
图26(b)是应用程序模块中的工作步骤：Figure 26(b) is the working steps in the application module:
第1105步、应用程序模块71在占据显示屏时，输入输出处理单元70通过应用程序编程接口45修改应用数据库44中的“应用程序状态表”，写入该应用程序模块的名称或标识符； Step 1105, when the application module 71 occupies the display screen, the input/output processing unit 70 modifies the "application status table" in the application database 44 through the application programming interface 45, and writes the name or identifier of the application module;
第1106步、延时等待，或执行应用程序模块的其它功能； Step 1106, delay waiting, or performing other functions of the application module;
第1107步、数据读取：应用程序模块71中输入输出处理单元70通过操作系统运行装置4提供的应用程序编程接口45读取应用数据库44中的姿态参数和门限值数据； Step 1107, data reading: the input/output processing unit 70 in the application module 71 reads the attitude parameter and the threshold data in the application database 44 through the application programming interface 45 provided by the operating system running device 4;
第1108步、姿态比较：应用程序模块71中的姿态比较单元47将所述姿态参数G1、G2和门限值T1、T2进行比较，姿态正常时转第1105步，否则转第1109步； Step 1108, posture comparison: the posture comparison unit 47 in the application module 71 compares the posture parameters G1, G2 with the threshold values T1, T2, and when the posture is normal, the first step is step 105, otherwise the step 1109 is turned;
第1109步、姿态提醒：应用程序模块71中的姿态提醒单元48在所述姿态参数超过门限值时，生成报警数据A和/或姿态矫正数据R；输入输出处理单元70通过操作系统运行装置4提供的应用程序编程接口45，启动输出模块驱动单元43，控制告警信号输出模块向客户发出用户感觉可识别的信号，包含报警信息和/或矫正建议，告警输出后，转第1105步。 Step 1109, posture reminder: the posture reminding unit 48 in the application module 71 generates alarm data A and/or posture correction data R when the posture parameter exceeds the threshold value; the input/output processing unit 70 runs the device through the operating system. The application programming interface 45 is provided to activate the output module driving unit 43. The control alarm signal output module sends a signal to the customer that the user feels identifiable, including alarm information and/or correction suggestions. After the alarm is output, the process proceeds to step 1105.
第三种。The third.
在图20中，通过操作系统实现用户姿态检测的移动终端5，包含姿态传感模块1、用户配置输入模块2、告警信号输出模块3、操作系统运行装置4、显示屏6、一个或多个应用程序模块71，72，…，7N。In FIG. 20, the mobile terminal 5 that implements user posture detection by an operating system includes an attitude sensing module 1, a user configuration input module 2, an alarm signal output module 3, an operating system operating device 4, a display screen 6, one or more Application modules 71, 72, ..., 7N.
姿态传感模块产生传感信号S。用户配置输入模块2识别用户操作，将用户指令I传递至操作系统运行装置4和至少一个所述应用程序模块(例如71)。告警信号输出模块3发出用户感官可识别的信号。The attitude sensing module generates a sensing signal S. The user configuration input module 2 identifies the user operation and passes the user command I to the operating system operating device 4 and at least one of the application modules (e.g., 71). The alarm signal output module 3 emits a signal recognizable by the user.
图27是基于图20的操作系统运行装置向应用程序模块发出姿态超限通知完成提醒的方法流程图。在图27所示的实施例中，操作系统运行装置中的姿态检测单元确定用户姿态，由姿态比较单元完成姿态判断，发现姿态不正确时，向应用程序模块发出通知，由应用程序模块执行姿态提醒功能。27 is a flow chart of a method for issuing an attitude overrun notification completion reminder to an application module based on the operating system running device of FIG. In the embodiment shown in FIG. 27, the posture detecting unit in the operating system running device determines the user posture, and the posture comparing unit completes the posture judgment. When the posture is found to be incorrect, the notification is sent to the application module, and the posture is executed by the application module. Reminder feature.
图27(a)是操作系统运行装置中的工作步骤：Figure 27 (a) is the working steps in the operating system operating device:
第1201步、姿态检测：姿态检测单元46启动传感模块驱动单元41，控制姿态传感模块1采集传感信号S1、S2，并转换为姿态数据D1、D2；姿态检测单元46读取传感模块驱动单元输出的姿态数据D1、D2，对姿态数据进行分析，得出相对姿态参数G1和/或绝对姿态参数G2； Step 1201, attitude detection: the attitude detecting unit 46 activates the sensing module driving unit 41, controls the attitude sensing module 1 to acquire the sensing signals S1, S2, and converts into attitude data D1, D2; the attitude detecting unit 46 reads the sensing The attitude data D1, D2 output by the module driving unit analyzes the attitude data to obtain a relative attitude parameter G1 and/or an absolute attitude parameter G2;
第1202步、姿态比较：姿态比较单元47将所述姿态参数G1、G2和门限值T1、T2进行比较，在姿态正常时转第1204步，姿态参数超过门限值范围时转第1203步， Step 1202, attitude comparison: The attitude comparison unit 47 compares the attitude parameters G1, G2 with the threshold values T1, T2, and when the posture is normal, the process proceeds to step 1204, and when the attitude parameter exceeds the threshold value range, the process proceeds to step 1203. ,
第1203步、姿态比较单元47通过应用程序编程接口45向应用程序模块71，72，…，7N中至少一个(例如应用程序模块71)发出姿态超限通知。操作系统运行装置4发出通知的方式包括：1、广播：向所有应用程序模块发出通知；2、选定单播：向占用显示屏的应用程序模块发出通知(可查询应用数据库44的“应用程序状态表”得到占用显示屏的应用程序模块名称或标识符)；3、注册单播：向占用显示屏且注册了姿态检测服务的应用程序模块发出通知(可查询应用数据库44的“应用程序状态表”和“应用程 序注册表”得到占用显示屏且注册了姿态检测服务的应用程序模块名称或标识符)。In step 1203, the gesture comparison unit 47 issues a gesture overrun notification to at least one of the application modules 71, 72, ..., 7N (e.g., the application module 71) via the application programming interface 45. The manner in which the operating system running device 4 issues a notification includes: 1. Broadcasting: issuing a notification to all application modules; 2. Selecting unicast: issuing a notification to the application module occupying the display screen (the application of the application database 44 can be queried) The status table "gets the application module name or identifier occupying the display screen"; 3. Register unicast: issues a notification to the application module that occupies the display screen and registers the gesture detection service (the application status of the application database 44 can be queried) Table" and "applications The order registry "gets the application module name or identifier that occupies the display and registers the gesture detection service).
第1204步、等待：延时后转第1201步 Step 1204, Wait: After the delay, go to step 1201.
图27(b)是应用程序模块71中的工作步骤：Figure 27 (b) is the working steps in the application module 71:
第1205步、消息监听：如果应用程序模块71正在占用显示屏，且其包含的输入输出处理单元70接收到来自操作系统运行装置4的姿态超限通知，则转第1206步触发姿态提醒单元48； Step 1205, message monitoring: If the application module 71 is occupying the display screen, and the input/output processing unit 70 included therein receives the gesture overrun notification from the operating system running device 4, then the step 1206 is triggered to trigger the gesture reminding unit 48. ;
第1206步、姿态提醒：姿态提醒单元48生成报警数据A和/或姿态矫正数据R；输入输出处理单元70通过操作系统运行装置4提供的应用程序编程接口45，启动输出模块驱动单元43，控制告警信号输出模块向客户发出用户感觉可识别的信号，包含报警信息和/或矫正建议。 Step 1206, gesture reminder: the gesture reminding unit 48 generates the alarm data A and/or the attitude correction data R; the input/output processing unit 70 activates the output module driving unit 43 through the application programming interface 45 provided by the operating system running device 4, and controls The alarm signal output module sends a signal to the customer that the user feels identifiable, including alarm information and/or correction suggestions.
例如在Android或iOS操作系统中当姿态不正确时，该服务对所有应用程序模块进行广播，广播的信息中还可以包含描述姿态参数G等信息，再由应用程序模块确定如何输出告警信息。For example, when the gesture is not correct in the Android or iOS operating system, the service broadcasts all the application modules, and the broadcast information may further include information describing the posture parameter G, and then the application module determines how to output the alarm information.
图28是基于图20的应用程序模块使用姿态检测标志位启动和关闭姿态检测并监听姿态超限通知的方案流程图，所示实施例中，应用程序模块(例如71)使用姿态检测标志位来启动和关闭操作系统运行装置4的姿态服务模块40。操作系统运行装置4中的姿态服务模块40包含姿态检测单元46、姿态比较单元47、数据扫描单元49。姿态提醒单元48则包含在应用程序模块71中。应用程序模块71中的输入输出处理单元70通过操作系统运行装置4提供的应用程序编程接口45(API)来设置“姿态检测标志位”。数据扫描单元49周期性查询该标志位，当该标志位被设置时运行姿态检测单元46和姿态比较单元47，在姿态参数超过门限值时向应用程序模块71，72，…，7N发出姿态超限通知。28 is a flow chart of a scheme based on the application module of FIG. 20 using the attitude detection flag to initiate and disable attitude detection and to monitor the attitude overrun notification. In the illustrated embodiment, the application module (eg, 71) uses the attitude detection flag. The gesture service module 40 of the operating system running device 4 is activated and deactivated. The attitude service module 40 in the operating system operating device 4 includes an attitude detecting unit 46, a posture comparing unit 47, and a data scanning unit 49. The gesture reminder unit 48 is included in the application module 71. The input/output processing unit 70 in the application module 71 sets the "attitude detection flag bit" through the application programming interface 45 (API) provided by the operating system running device 4. The data scanning unit 49 periodically queries the flag bit, and when the flag bit is set, the running posture detecting unit 46 and the posture comparing unit 47 emit a gesture to the application modules 71, 72, ..., 7N when the attitude parameter exceeds the threshold value. Overdue notification.
图28(a)是操作系统运行装置中的工作步骤：Figure 28 (a) is the working steps in the operating system operating device:
第1301步、数据扫描单元49查询所述“姿态检测标志位”是否被设置，如是，则转第1302步，触发姿态检测单元46；否则转第1305步； Step 1301, the data scanning unit 49 queries whether the "attitude detection flag bit" is set, if yes, then proceeds to step 1302, triggers the attitude detecting unit 46; otherwise, steps 1305;
第1302步、姿态检测：姿态检测单元46启动传感模块驱动单元41，控制姿态传感模块1采集传感信号S1、S2，并转换为姿态数据D1、D2；姿态检测单元46读取传感模块驱动单元输出的姿态数据D1、D2，对姿态数据进行分析，得出相对姿态参数G1和/或绝对姿态参数G2； Step 1302, posture detection: the attitude detecting unit 46 activates the sensing module driving unit 41, controls the attitude sensing module 1 to acquire the sensing signals S1, S2, and converts into the attitude data D1, D2; the attitude detecting unit 46 reads the sensing The attitude data D1, D2 output by the module driving unit analyzes the attitude data to obtain a relative attitude parameter G1 and/or an absolute attitude parameter G2;
第1303步、姿态比较：姿态比较单元47将所述姿态参数G1、G2和门限值T1、T2进行比较，在姿态参数超过门限值范围时转第1304步，姿态正常时转第1305步； Step 1303, attitude comparison: The attitude comparison unit 47 compares the attitude parameters G1 and G2 with the threshold values T1 and T2, and when the attitude parameter exceeds the threshold value range, the process proceeds to step 1304, and when the posture is normal, the process proceeds to step 1305. ;
第1304步、姿态比较单元47通过应用程序编程接口45向应用程序模块71，72，…，7N发出姿态超限通知(广播)。In step 1304, the posture comparing unit 47 issues an attitude overrun notification (broadcast) to the application modules 71, 72, ..., 7N via the application programming interface 45.
第1305步、等待，延时后转第1301步。 Step 1305, wait, delay and then go to step 1301.
图28(b)是应用程序模块中的工作步骤：Figure 28(b) is the working steps in the application module:
第1306步、任何一个应用程序模块(不失一般性，例如应用程序模块71)所包含的输入输出处理单元70通过应用程序编程接口45设置或清除应用数据库44中的“姿态检测标志位”； Step 1306, the input/output processing unit 70 included in any one of the application modules (for example, the application module 71) is configured to set or clear the "attitude detection flag" in the application database 44 through the application programming interface 45;
第1307步、延时等待，或执行其它功能； Step 1307, wait for a delay, or perform other functions;
第1308步、消息监听：任何一个占用显示屏的应用程序模块71，72，…，7N(不失一般性，例如应用程序模块71)所包含的输入输出处理单元70接收到来自操作系统运行装置4的姿态超限通知，则转第1309步触发姿态提醒单元48，否则转第1307步继续等待； Step 1308, message monitoring: any of the application modules 71, 72, ..., 7N (not lost in general, such as the application module 71) that occupy the display screen receives the input and output processing unit 70 from the operating system If the gesture of 4 is over-limit, then the step 1309 is triggered to trigger the gesture reminding unit 48, otherwise the step 1307 is continued;
第1309步、姿态提醒：姿态提醒单元48生成报警数据A和/或姿态矫正数据R；输入输出处理单元70通过操作系统运行装置4提供的应用程序编程接口45，启动输出模块驱动单元43，控制告警信号输出模块向客户发出用户感觉可识别的信号，包含报警信息和/或矫正建议，告警输出后，转第1306步。 Step 1309, posture reminder: the gesture reminding unit 48 generates the alarm data A and/or the attitude correction data R; the input/output processing unit 70 activates the output module driving unit 43 through the application programming interface 45 provided by the operating system running device 4, and controls The alarm signal output module sends a signal to the customer that the user feels identifiable, including alarm information and/or correction suggestions. After the alarm is output, go to step 1306.
图29是基于图20的操作系统运行装置按照应用程序注册表启动姿态检测并发出姿态超限通知的方案流程图。操作系统运行装置40中的姿态服务模块包含姿态检测单元46、姿态比较单元47、数据扫描单元49。姿态服务模块40与应用程序模块71，72，…，7N并行工作，姿态服务模块40中的数据扫描单元49周期性地从应用数据库中检测当前运行的应用程序模块，当该应用程序模块(例如71)的名称或标识符在应用程序注册表中时，启动姿态检测单元46工作。29 is a flow chart of a scheme based on the operating system running device of FIG. 20 initiating attitude detection according to an application registry and issuing a gesture overrun notification. The posture service module in the operating system operating device 40 includes an attitude detecting unit 46, a posture comparing unit 47, and a data scanning unit 49. The gesture service module 40 works in parallel with the application modules 71, 72, ..., 7N, and the data scanning unit 49 in the gesture service module 40 periodically detects the currently running application module from the application database, such as the application module (eg When the name or identifier of 71) is in the application registry, the gesture detecting unit 46 is activated to operate.
所述应用程序模块(例如71)占据显示屏时，通过应用程序编程接口45修改应用数据库44中的“应用程序状态表”，写入该应用程序模块的名称(或系统分配的某种标识符)。When the application module (eg, 71) occupies the display screen, the "application status table" in the application database 44 is modified by the application programming interface 45, and the name of the application module (or some identifier assigned by the system) is written. ).
用户可以通过用户配置输入模块2设置“应用程序注册表”，这些应用程序占据显示屏时进行姿态检测和提醒。某些应用程序模块也可以自己主动修改“应用程序注册表”，将自己加入到这个列表中。在本实施例中，假设应用程序模块71的名称或标识符在“应用程序注册表”中，表明该应用程序模块注册了操作系统运行装置中姿态服务模块提供的服务。The user can configure the "application registry" through the user configuration input module 2, which performs posture detection and reminding when occupying the display screen. Some application modules can also modify the "application registry" by themselves and add themselves to this list. In the present embodiment, it is assumed that the name or identifier of the application module 71 is in the "application registry", indicating that the application module registers the service provided by the gesture service module in the operating system running device.
图29(a)是操作系统运行装置中的工作步骤： Figure 29 (a) is the working steps in the operating system operating device:
第1401a步、判断应用程序模块(例如71)是否占用显示屏：数据扫描单元49从应用数据库44中的“应用程序状态表”中查询正在使用显示屏的应用程序模块名称(或标识符)，如是，则1401b步，否则转第1405步； Step 1401a, determining whether the application module (eg, 71) occupies the display screen: the data scanning unit 49 queries the "application status table" in the application database 44 for the application module name (or identifier) that is using the display screen, If yes, then step 1401b, otherwise go to step 1405;
第1401b步、检查应用程序模块71是否在“应用程序注册表”中：数据扫描单元49从应用数据库44中的“应用程序注册表”中查询注册了姿态检测服务的应用程序模块名称(或标识符)，如是，则转第1402步，触发姿态检测单元46；否则转第1405步； Step 1401b, checking whether the application module 71 is in the "application registry": the data scanning unit 49 queries the application module name (or identifier) registered with the gesture detection service from the "application registry" in the application database 44.符), if yes, then go to step 1402, trigger attitude detection unit 46; otherwise, go to step 1405;
第1402步、姿态检测：姿态检测单元46启动传感模块驱动单元41，控制姿态传感模块1采集传感信号S1、S2，并转换为姿态数据D1、D2；姿态检测单元46读取传感模块驱动单元输出的姿态数据D1、D2；姿态检测单元46对姿态数据进行分析，得出相对姿态参数G1和/或绝对姿态参数G2； Step 1402, attitude detection: the attitude detecting unit 46 activates the sensing module driving unit 41, controls the attitude sensing module 1 to acquire the sensing signals S1, S2, and converts into the attitude data D1, D2; the attitude detecting unit 46 reads the sensing The attitude data D1, D2 output by the module driving unit; the attitude detecting unit 46 analyzes the attitude data to obtain a relative attitude parameter G1 and/or an absolute attitude parameter G2;
第1403、姿态比较：应用程序模块71中的姿态比较单元47将所述姿态参数G1、G2和门限值T1、T2进行比较，姿态正常时转第1405步，否则转第1404步；Step 1403: posture comparison: the posture comparison unit 47 in the application module 71 compares the posture parameters G1 and G2 with the threshold values T1 and T2, and when the posture is normal, the first step is step 405; otherwise, the first step is step 404;
第1404步、发出姿态比较单元47通过应用程序编程接口45向应用程序模块71发出姿态超限通知(注册单播)。In step 1404, the issued gesture comparing unit 47 issues a gesture overrun notification (register unicast) to the application module 71 via the application programming interface 45.
第1405步、等待，延时后转第1401a步。 Step 1405, wait, delay and then go to step 1401a.
图29(b)是应用程序模块中的工作步骤：Figure 29 (b) is the work steps in the application module:
第1406步、应用程序模块71在占据显示屏时，输入输出处理单元70通过应用程序编程接口45修改应用数据库44中的“应用程序状态表”，写入该应用程序模块的名称或标识符； Step 1406, when the application module 71 occupies the display screen, the input/output processing unit 70 modifies the "application status table" in the application database 44 through the application programming interface 45, and writes the name or identifier of the application module;
第1407步、延时等待，或执行应用程序模块的其它功能； Step 1407, delay waiting, or performing other functions of the application module;
第1408步、消息监听：如果应用程序模块71所包含的输入输出处理单元70接收到从操作系统运行装置4发出的姿态超限通知，则转第1409步，触发姿态提醒单元48，否则转第1407步； Step 1408, message monitoring: if the input/output processing unit 70 included in the application module 71 receives the gesture overrun notification sent from the operating system running device 4, then step 1409 is performed to trigger the gesture reminding unit 48, otherwise Step 1407;
第1409步、姿态提醒：应用程序模块71中的姿态提醒单元48在所述姿态参数超过门限值时，生成报警数据A和/或姿态矫正数据R；输入输出处理单元70通过操作系统运行装置4提供的应用程序编程接口45，启动输出模块驱动单元43，控制告警信号输出模块向客户发出用户感觉可识别的信号，包含报警信息和/或矫正建议，告警输出后，转第1406步。 Step 1409, posture reminder: the posture reminding unit 48 in the application module 71 generates alarm data A and/or posture correction data R when the posture parameter exceeds the threshold value; the input/output processing unit 70 runs the device through the operating system. The application programming interface 45 is provided to activate the output module driving unit 43, and the control alarm signal output module sends a signal to the customer that the user feels identifiable, including the alarm information and/or the correction suggestion. After the alarm is output, the process proceeds to step 1406.
例如，在Android操作系统中，其他应用程序模块可以通过实现姿态检测的监听接口(Interface)，从操作系统运行装置中获取当前用户的姿态是否正确的判断。在iOS操作系统中，应用程序模块还可以通过实现姿势测量类的委托(Delegate)，从操作系统运 行装置中获取当前使用者的姿势是否正确的判断。For example, in the Android operating system, other application modules can obtain the judgment of whether the current user's posture is correct from the operating system running device through the monitoring interface (Interface) that implements gesture detection. In the iOS operating system, the application module can also be shipped from the operating system by implementing a delegate of the gesture measurement class (Delegate). The line device obtains a judgment as to whether the current user's posture is correct.
实施例三。Example 3.
本发明通过应用程序实现用户姿态检测的移动终端如图30所示，包含姿态传感模块1、用户配置输入模块2、告警信号输出模块3、应用程序模块71，72，…，7N、状态数据库8、监控程序模块4’。The mobile terminal for realizing user posture detection by using the application program is shown in FIG. 30, and includes an attitude sensing module 1, a user configuration input module 2, an alarm signal output module 3, application program modules 71, 72, ..., 7N, a state database. 8. Monitor program module 4'.
姿态传感模块1产生姿态数据D；用户配置输入模块2识别用户操作，产生用户指令I；告警信号输出模块3发出用户感官可识别的信号。The attitude sensing module 1 generates the attitude data D; the user configuration input module 2 recognizes the user operation, and generates the user command I; the alarm signal output module 3 issues a signal recognizable by the user.
本发明所述应用程序模块71，72，…，7N，是加载和运行应用程序的装置，特别是加载和运行一些用户可能长时间连续使用的应用程序，如视频播放器、网页浏览器、阅读软件等。The application modules 71, 72, ..., 7N of the present invention are devices for loading and running applications, particularly loading and running applications that may be used continuously by users for a long time, such as video players, web browsers, and reading. Software, etc.
所述状态数据库8中存储工作的状态数据。所述状态数据8包含“应用程序注册表”、“应用程序状态表”、“姿态处理标志位”。The status database 8 stores the status data of the work. The status data 8 includes an "application registry", an "application status table", and a "pose processing flag bit".
应用程序状态表，包含终端中应用程序模块工作状态的信息，例如该应用程序模块是否运行、是否占据显示屏等。The application status table contains information about the working state of the application module in the terminal, such as whether the application module is running, whether it occupies the display screen, and the like.
应用程序注册表，包含有检测必要的应用程序模块名称或系统分配的某种标识符。应用程序模块71，72，…，7N可以把自己的名称或标识符添加到状态数据库8中，表示注册了姿态处理功能。The application registry contains an identifier that detects the necessary application module name or system allocation. The application modules 71, 72, ..., 7N can add their own names or identifiers to the state database 8, indicating that the gesture processing function is registered.
姿态处理标志位，是指姿态处理模块40’是否运行的标志。The gesture processing flag bit is a flag indicating whether or not the gesture processing module 40' is operating.
所述监控程序模块4’中包含输入输出处理单元41’，与所述姿态传感模块1、用户配置输入模块2、告警信号输出模块3相连接，接收所述姿态数据D、用户指令I，产生输出数据O。The monitoring program module 4' includes an input/output processing unit 41', and is connected to the attitude sensing module 1, the user configuration input module 2, and the alarm signal output module 3, and receives the attitude data D and the user instruction I. Generate output data O.
所述监控程序模块4’中包含姿态处理单元40’，根据姿态数据D识别用户面部的位置，产生姿态参数G，将所述姿态参数G与对应的门限值T进行比较，在用户姿态不正确时产生报警数据A和/或姿态矫正数据R。The monitoring program module 4' includes an attitude processing unit 40', identifies the position of the user's face according to the attitude data D, generates a posture parameter G, compares the posture parameter G with a corresponding threshold value T, and does not Alarm data A and/or attitude correction data R are generated when correct.
所述监控程序模块4’中包含状态扫描单元42’，周期性地扫描状态数据库8，了解所述应用程序模块71，72，…，7N的工作状态，在所述至少一个应用程序模块(例如71)占用显示屏时，启动姿态处理单元40’。The monitoring program module 4' includes a state scanning unit 42' that periodically scans the state database 8 to know the working state of the application modules 71, 72, ..., 7N, in the at least one application module (for example 71) When the display screen is occupied, the attitude processing unit 40' is activated.
如图31所示，是本发明通过监控程序实现用户姿态检测的移动终端的一个实施例，姿态传感模块1包含摄像头及其驱动装置，由一只或多只前置摄像头产生用户面部的图像信号，生成用户姿态数据D1。监控程序模块4’中的输入输出处理单元41’控制姿态传感模块1，接收用户姿态数据D1，姿态处理单元40’检测用户面部与显示屏的相对位 置，产生相对姿态参数G1，将所述姿态参数G1与对应的门限值T1进行比较，用户姿态不正确时产生报警数据A和/或姿态矫正数据R。As shown in FIG. 31, it is an embodiment of a mobile terminal for realizing user posture detection by a monitoring program. The attitude sensing module 1 includes a camera and a driving device thereof, and an image of a user's face is generated by one or more front cameras. The signal generates user posture data D1. The input/output processing unit 41' in the monitor program module 4' controls the attitude sensing module 1 to receive the user posture data D1, and the posture processing unit 40' detects the relative position of the user's face and the display screen. The relative attitude parameter G1 is generated, the attitude parameter G1 is compared with the corresponding threshold value T1, and the alarm data A and/or the attitude correction data R are generated when the user posture is incorrect.
如图32所示，是本发明通过监控程序实现用户姿态检测的移动终端的另一个实施例，姿态传感模块1包含摄像头及其驱动装置11，由一只或多只前置摄像头产生图像信号，生成用户姿态数据D1。姿态传感模块1中还包含能检测显示屏所在平面倾角的本机姿态传感器(如重力传感器，又称为倾角传感器)及其驱动装置12，产生角度传感信号，生成本机姿态数据D2。监控程序模块4’中的输入输出处理单元41’控制姿态传感模块1，接收姿态数据D1、D2。姿态处理单元40’根据用户姿态数据D1检测用户面部与显示屏的相对位置，产生相对姿态参数G1；姿态处理单元40’根据本机姿态数据D2确定本机显示屏所在平面的倾角。姿态处理单元40’根据相对姿态参数G1和本机显示屏所在平面的倾角，产生绝对姿态参数G2。As shown in FIG. 32, in another embodiment of the present invention, a mobile terminal that implements user attitude detection by a monitoring program includes a camera and a driving device 11 thereof, and image signals are generated by one or more front cameras. , user posture data D1 is generated. The attitude sensing module 1 further includes a local attitude sensor (such as a gravity sensor, also referred to as a tilt sensor) capable of detecting the plane inclination of the display screen and its driving device 12, and generates an angle sensing signal to generate the local attitude data D2. The input/output processing unit 41' in the monitor program module 4' controls the attitude sensing module 1 to receive the attitude data D1, D2. The attitude processing unit 40' detects the relative position of the user's face and the display screen based on the user's posture data D1, and generates a relative posture parameter G1; the attitude processing unit 40' determines the inclination of the plane of the local display screen based on the local posture data D2. The attitude processing unit 40' generates an absolute attitude parameter G2 based on the inclination of the relative attitude parameter G1 and the plane in which the local display screen is located.
姿态处理单元40’将所述姿态参数G1、G2与对应的门限值T1、T2进行比较；用户姿态不正确时产生报警数据A和/或姿态矫正数据R。The attitude processing unit 40' compares the attitude parameters G1, G2 with corresponding threshold values T1, T2; when the user posture is incorrect, alarm data A and/or attitude correction data R are generated.
图33作为通过监控程序实现姿态检测的移动终端的另一个实施例，是监控程序模块4’调用姿态处理单元40’的方案。Fig. 33 shows another embodiment of the mobile terminal which realizes attitude detection by the monitoring program, and is a scheme in which the monitoring program module 4' calls the posture processing unit 40'.
此时，移动终端5中存在一个处理单元库9，所述处理单元库9中包含姿态处理单元40’、及其他处理单元(例如91)。所述姿态处理单元40’根据姿态数据D检测用户的姿态，产生姿态参数G，将所述姿态参数G与对应的门限值T进行比较，在用户姿态不正确时产生报警数据A和/或姿态矫正数据R。处理单元库9中还包含一个应用程序编程接口(API)90，使所述姿态处理单元40’和其他处理单元(例如91)能够接收启动信号和数据，并输出结果。At this time, there is one processing unit library 9 in the mobile terminal 5, and the processing unit library 9 includes an attitude processing unit 40' and other processing units (e.g., 91). The posture processing unit 40' detects the posture of the user according to the posture data D, generates the posture parameter G, compares the posture parameter G with the corresponding threshold value T, and generates alarm data A and/or when the user posture is incorrect. Posture correction data R. The processing unit library 9 also includes an application programming interface (API) 90 that enables the gesture processing unit 40' and other processing units (e.g., 91) to receive the enable signals and data and output the results.
监控程序模块4’中包含输入输出处理单元41’、状态扫描单元42’、姿态处理调用单元43’。所述输入输出处理单元41’与所述姿态传感模块1、用户配置输入模块2、告警信号输出模块3相连接，接收所述姿态数据D、用户指令I，产生输出数据O。所述状态扫描单元42’，周期性地扫描状态数据库8，了解所述应用程序模块71，72，…，7N的工作状态，在所述应用程序模块中至少一个(例如71)占用显示屏时，启动姿态处理调用单元43’。所述姿态处理调用单元43’，能够查询姿态处理单元40’在处理单元库9中的位置、发出启动信号、并传递姿态检测和比较所需要的数据(包括姿态数据D1、D2和门限值T1、T2)，并返回报警数据A和姿态矫正数据R。The monitor program module 4' includes an input/output processing unit 41', a status scan unit 42', and a gesture processing call unit 43'. The input/output processing unit 41' is connected to the attitude sensing module 1, the user configuration input module 2, and the alarm signal output module 3, and receives the attitude data D and the user command I to generate output data O. The status scanning unit 42' periodically scans the status database 8 to know the working status of the application modules 71, 72, ..., 7N when at least one of the application modules (e.g., 71) occupies the display screen. The gesture processing calling unit 43' is started. The gesture processing invoking unit 43' is capable of querying the position of the gesture processing unit 40' in the processing unit library 9, issuing an activation signal, and transmitting the data required for the attitude detection and comparison (including the attitude data D1, D2, and the threshold value). T1, T2), and return alarm data A and attitude correction data R.
与图33所示实施例相对应，例如采用在Android操作系统中增设库函数的方案。在Android系统运行装置中，将实现姿态检测、姿态比较、姿态提醒功能的姿态处理单 元40作为一个本地库(Library)运行装置的一部分，以库函数运行装置的方式存在，并在应用框架(Application Framework)中提供一个API，该API可以向其它应用程序模块提供当前用户的姿态是否正确的判断。监控程序模块4’通过调用库函数运行装置完成本发明实现的功能。Corresponding to the embodiment shown in FIG. 33, for example, a scheme of adding a library function in the Android operating system is adopted. In the Android system running device, the attitude processing list of the attitude detection, the posture comparison, and the gesture reminding function will be realized. As part of a local library running device, meta 40 exists as a library function running device and provides an API in the Application Framework, which can provide other application modules with the current user's gesture. The correct judgment. The monitor program module 4' performs the functions implemented by the present invention by calling the library function running means.
再例如在iOS操作系统中增设库函数。在iOS系统运行装置中，以库函数运行装置的方式实现姿态处理单元40’，包含姿态检测、姿态比较、姿态提醒功能，作为媒体层或核心服务层的一部分，并在CocoaTouch中提供一个API，该API可以向监控程序模块4’提供当前用户的姿态是否正确的判断。监控程序模块4’通过调用库函数运行装置完成本发明实现的功能。For example, add a library function in the iOS operating system. In the iOS system running device, the gesture processing unit 40' is implemented in the manner of a library function running device, including posture detection, posture comparison, gesture reminding function, as a part of the media layer or the core service layer, and provides an API in the CocoaTouch. The API can provide the monitor module 4' with a determination as to whether the current user's gesture is correct. The monitor program module 4' performs the functions implemented by the present invention by calling the library function running means.
还有在第三方开发工具中库函数的方案。以库函数运行装置的方式实现姿态处理单元40’，包含姿态检测、姿态比较、姿态提醒功能，这样用户开发的装置可以直接调用与该函数库运行装置有关的API。本发明所述监控程序模块4’通过调用库函数运行装置完成本发明实现的功能。There are also programs for library functions in third-party development tools. The gesture processing unit 40' is implemented in the manner of the library function running device, and includes a posture detection, a posture comparison, and a gesture reminding function, so that the device developed by the user can directly call the API related to the function library running device. The monitoring program module 4' of the present invention performs the functions implemented by the present invention by calling a library function running device.
通过监控程序模块4’实现用户姿态检测的方法如图34所示，包含如下步骤：The method for realizing user posture detection by the monitoring program module 4' is as shown in FIG. 34, and includes the following steps:
第1501步、配置输入：监控程序模块4’中输入输出处理单元41’接受来自用户配置输入模块2的用户指令I； Step 1501, configuration input: the input/output processing unit 41' in the monitor program module 4' accepts the user command I from the user configuration input module 2;
第1502步、应用程序扫描：监控程序模块4’中的状态扫描单元42’，周期性地扫描状态数据库8，根据状态数据库8的信息启动姿态处理单元40’。 Step 1502, Application Scan: The status scan unit 42' in the monitor program module 4' periodically scans the status database 8, and activates the pose processing unit 40' based on the information of the status database 8.
第1503步、姿态检测：监控程序模块4’中的输入输出处理单元41’控制姿态传感模块1采集用户姿态，接收姿态数据D1、D2；姿态处理单元40’对姿态数据D1、D2进行分析，得出用户相对姿态参数G1和/或绝对姿态参数G2； Step 1503, attitude detection: the input/output processing unit 41' in the monitoring program module 4' controls the attitude sensing module 1 to collect the user posture, and receives the attitude data D1, D2; the posture processing unit 40' analyzes the posture data D1, D2 Obtaining a user relative posture parameter G1 and/or an absolute posture parameter G2;
第1504步、姿态比较：监控程序模块4’中的姿态处理单元40’将姿态参数G1、G2和对应的门限值T1、T2进行比较，姿态正确时转第1502步，否则第1505步； Step 1504, attitude comparison: the attitude processing unit 40' in the monitoring program module 4' compares the attitude parameters G1, G2 with the corresponding threshold values T1, T2, and if the posture is correct, the step 1502 is performed, otherwise the step 1505;
第1505步、姿态提醒：监控程序模块4’中的姿态处理单元40’在用户姿态不正确的时候，生成报警数据A和/或姿态矫正数据R；监控程序模块4’中输入输出处理单元41’输出信号(O)触发告警信号输出模块3向客户发出用户感觉可识别的信号，包含报警信息和/或矫正建议，告警输出后，转第1502步。 Step 1505, posture reminder: the attitude processing unit 40' in the monitoring program module 4' generates alarm data A and/or posture correction data R when the user posture is incorrect; the input/output processing unit 41 in the monitoring program module 4' The output signal (O) triggers the alarm signal output module 3 to send a signal to the customer that the user feels identifiable, including the alarm information and/or correction suggestions. After the alarm is output, the process proceeds to step 1502.
在上述方法中，循环执行的方式从第1504步、第1505步回归到第1502步时，可包含一个时延的步骤。In the above method, the loop execution method may include a step of delay from the step 1504, the step 1505 to the step 1502.
以下各个实施例是在上述工作方法的基础上，对第1502～1504步在移动终端中的具体实现方式给出多种方案。 The following various embodiments are based on the above working methods, and various solutions are given for the specific implementation manners of the steps 1502 to 1504 in the mobile terminal.
图35是监控程序模块4’扫描状态数据库8获得应用程序指令、进行姿态处理的流程图。姿态处理单元40’实现姿态检测、姿态比较、姿态提醒的功能。监控程序模块4’与应用程序模块71，72，…，7N并列运行，应用程序模块71，72，…，7N中任一个在占用显示屏、且有必要对用户进行姿态检测时(例如长期处于浏览状态时候)，在状态数据库8中设置“姿态处理标志位”，用于启动姿态处理单元40’，应用程序模块退出显示屏或其它不需要对姿态进行检测的情况发生时清除该标志位。Fig. 35 is a flow chart showing the monitoring program module 4' scanning state database 8 obtaining an application command and performing posture processing. The posture processing unit 40' realizes the functions of posture detection, posture comparison, and posture reminding. The monitoring program module 4' runs in parallel with the application modules 71, 72, ..., 7N, and any one of the application modules 71, 72, ..., 7N is occupying the display screen and it is necessary to perform posture detection on the user (for example, long-term presence When browsing the state, the "attitude processing flag bit" is set in the state database 8 for starting the gesture processing unit 40', and the flag is cleared when the application module exits the display screen or other situations where the gesture is not required to be detected.
图35(a)表示应用程序模块71，72，…，7N的工作过程的一个段落，步骤如下：Figure 35 (a) shows a paragraph of the working process of the application modules 71, 72, ..., 7N, the steps are as follows:
第1601步、任意一个应用程序模块(例如71)设置或清除应用数据库8中的“姿态处理标志位”； Step 1601, any one of the application modules (for example, 71) sets or clears the "attitude processing flag bit" in the application database 8;
第1602步、应用程序模块(例如71)运行应用程序主功能；In step 1602, the application module (eg, 71) runs the application main function;
图35(b)表示监控程序模块的工作过程，步骤如下：Figure 35 (b) shows the working process of the monitoring program module, the steps are as follows:
第1603步、状态扫描单元42’查询状态数据库8，确认姿态处理标志位是否被设置？如果是则进入第1604步，启动姿态处理单元40’，否则转第1607步；In step 1603, the status scanning unit 42' queries the status database 8 to confirm whether the gesture processing flag is set. If yes, proceed to step 1604 to start the gesture processing unit 40', otherwise go to step 1607;
第1604步、姿态检测：监控程序模块4’中的输入输出处理单元41’控制姿态传感模块1采集用户姿态，接收姿态数据D1、D2；姿态处理单元40’对姿态数据D1、D2进行分析，得出相对姿态参数G1和/或绝对姿态参数G2； Step 1604, posture detection: the input/output processing unit 41' in the monitoring program module 4' controls the attitude sensing module 1 to collect the user posture, and receives the attitude data D1, D2; the posture processing unit 40' analyzes the posture data D1, D2 Obtaining a relative attitude parameter G1 and/or an absolute attitude parameter G2;
第1605步、姿态比较：姿态处理单元40’将所述姿态参数G1、G2和门限值T1、T2进行比较，姿态正确时转第1607步，否则进入第1606步； Step 1605, attitude comparison: the attitude processing unit 40' compares the posture parameters G1, G2 with the threshold values T1, T2, and if the posture is correct, the first step is step 607; otherwise, the step 1606 is entered;
第1606步、姿态提醒：姿态处理单元40在所述姿态参数G1、G2超过门限值时T1、T2，生成报警数据A和/或姿态矫正数据R；输入输出处理单元41’输出信号O触发告警信息输出模块3； Step 1606, posture reminder: the attitude processing unit 40 generates alarm data A and/or attitude correction data R when the attitude parameters G1, G2 exceed the threshold value T1, T2; the input and output processing unit 41' outputs a signal O trigger Alarm information output module 3;
第1607步、等待，延时后转第1603步。 Step 1607, wait, delay and then go to step 1603.
图36是监控程序模块4’扫描状态数据库8获得应用程序状态、进行姿态处理的流程图。姿态处理单元40’实现姿态检测、姿态比较、姿态提醒功能。应用程序模块71，72，…，7N在占据显示屏时，将应用程序模块的名称或标识符写入状态数据库8，形成应用程序状态表。监控程序模块4’中的状态扫描单元42’在运行中取得“应用程序注册表”，确定注册监控功能的应用程序模块；如果该占用显示屏的应用程序模块在名称或标识符在应用程序注册表中，则触发姿态处理单元。Fig. 36 is a flow chart showing the state in which the monitoring program module 4' scans the state database 8 to obtain the application state and performs the posture processing. The attitude processing unit 40' implements attitude detection, attitude comparison, and gesture reminding functions. The application modules 71, 72, ..., 7N, when occupying the display screen, write the name or identifier of the application module to the state database 8, forming an application state table. The status scanning unit 42' in the monitoring program module 4' obtains an "application registry" in operation, and determines an application module that registers the monitoring function; if the application module occupying the display is registered in the application with a name or identifier In the table, the gesture processing unit is triggered.
图36(a)表示应用程序模块71，72，…，7N的工作过程的一个段落，步骤如下：Figure 36 (a) shows a paragraph of the working process of the application modules 71, 72, ..., 7N, the steps are as follows:
第1701步、应用程序模块(例如71)在占用显示屏的时候，通过应用程序编程接口设置或清除应用数据库中的“应用程序状态表”； Step 1701, the application module (for example, 71) sets or clears the "application status table" in the application database through the application programming interface when occupying the display screen;
第1702步、应用程序模块(例如71)运行应用程序主功能；In step 1702, the application module (eg, 71) runs the application main function;
图36(b)表示监控程序模块的工作过程，步骤如下：Figure 36 (b) shows the working process of the monitoring program module, the steps are as follows:
第1703步、状态扫描单元42’查询状态数据库，确认正在使用显示屏的应用程序模块名称或标识符是否在“应用程序注册表”中？如果是则进入第1704步，启动姿态处理单元40，否则转第1707步；In step 1703, the status scanning unit 42' queries the status database to confirm whether the application module name or identifier of the display being used is in the "application registry"? If yes, proceed to step 1704, start the attitude processing unit 40, otherwise go to step 1707;
第1704步、姿态检测：监控程序模块4’中的输入输出处理单元41’控制姿态传感模块1采集用户姿态，接收姿态数据D1、D2；姿态处理单元40’对姿态数据D1、D2进行分析，得出相对姿态参数G1和/或绝对姿态参数G2； Step 1704, posture detection: the input/output processing unit 41' in the monitoring program module 4' controls the attitude sensing module 1 to collect the user posture, and receives the attitude data D1, D2; the posture processing unit 40' analyzes the posture data D1, D2 Obtaining a relative attitude parameter G1 and/or an absolute attitude parameter G2;
第1705步、姿态比较：姿态处理单元40’将所述姿态参数G1、G2和门限值T1、T2进行比较，姿态正确时转第1707步，否则进入第1706步； Step 1705, posture comparison: the attitude processing unit 40' compares the posture parameters G1, G2 with the threshold values T1, T2, and if the posture is correct, the first step is step 7078; otherwise, the step 1706 is entered;
第1706步、姿态提醒：姿态处理单元40’在所述姿态参数G1、G2超过门限值时T1、T2，生成报警数据A和/或姿态矫正数据R；输入输出处理单元41’输出信号O触发告警信息输出模块3； Step 1706, posture reminder: the attitude processing unit 40' generates alarm data A and/or attitude correction data R when the posture parameters G1, G2 exceed the threshold value T1, T2; the input and output processing unit 41' outputs a signal O Triggering an alarm information output module 3;
第1707步、等待，延时后转第1703步。 Step 1707, wait, delay and then go to step 1703.
针对上述各个实施例，具体说明用户功能设置方案。监控程序模块4’中的输入输出处理单元41’接收来自用户配置输入模块2的用户指令。指令数据至少包括以下指令中一种：1.应用程序选择指令，用户设置“应用程序注册表”。只有前台运行这些应用程序时该后台服务才进行姿态检测。某些应用程序模块也可以自己主动加入到这个列表中。通过设置让该功能在某些应用中不被启用。如前台应用程序是接听、拨打电话、听收音机或其它音频内容时，由于此时用户不会长时间关注显示屏，此时不必要开启本功能。2.门限指令，用户设置姿态参数门限值。通过设置门限值适应不同用户的需求。在没有用户指令时，上述设置也可以是在姿态处理单元40’或姿态处理调用单元43’中存在默认设置，通过用户配置输入模块2进行手动设置或修改。For the above embodiments, the user function setting scheme is specifically described. The input/output processing unit 41' in the monitor module 4' receives the user command from the user configuration input module 2. The instruction data includes at least one of the following instructions: 1. The application selects the instruction, and the user sets the "application registry". The background service performs gesture detection only when the application is running in the foreground. Some application modules can also actively join this list. By setting this feature is not enabled in some applications. If the foreground application is answering, making a call, listening to radio or other audio content, since the user does not pay attention to the display for a long time, it is not necessary to turn on this function. 2. Threshold command, the user sets the attitude parameter threshold. By setting thresholds to meet the needs of different users. In the absence of a user command, the above setting may also be that there is a default setting in the gesture processing unit 40' or the gesture processing invoking unit 43', which is manually set or modified by the user configuration input module 2.
针对上述各个实施例，具体说明告警输出方案。告警信号输出模块3，处理报警数据A和姿态矫正数据R。这些数据包含在监控程序模块4’的输入输出处理单元41’输出信号O中。报警数据是指姿态参数超出门限值时生成的，包含报警的方式、报警的强度、报警的时间长短等特征，这些数据是根据用户偏离正确姿态的程度和时间来确定的。姿态矫正数据是根据姿态参数和门限值之间的差，得到如何改变用户姿态以达到正确姿态的数据，包括距离、角度、及调整的方向等信息。报警输出模块输出的信号是用户感官可识别的信号，包含以下一种或多种类型：通过音频模块发出来的提示音、通过振动设备发出来的振动，通过显示屏给出的图形或文字提示。 The alarm output scheme is specifically described for each of the above embodiments. The alarm signal output module 3 processes the alarm data A and the attitude correction data R. These data are contained in the output signal O of the input/output processing unit 41' of the monitor program module 4'. The alarm data is generated when the attitude parameter exceeds the threshold, and includes the manner of the alarm, the strength of the alarm, and the length of the alarm. These data are determined according to the degree and time when the user deviates from the correct posture. The attitude correction data is based on the difference between the attitude parameter and the threshold value, and obtains information on how to change the user posture to achieve the correct posture, including distance, angle, and direction of adjustment. The signal output by the alarm output module is a sensory identifiable signal of the user, and includes one or more of the following types: a prompt sound emitted by the audio module, a vibration generated by the vibration device, a graphic or text prompt given through the display screen. .
本领域内的技术人员应明白，本申请的实施例可提供为方法、系统、或计算机程序产品。因此，本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。Those skilled in the art will appreciate that embodiments of the present application can be provided as a method, system, or computer program product. Thus, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment in combination of software and hardware. Moreover, the application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.
本申请是参照根据本申请实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (system), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine for the execution of instructions for execution by a processor of a computer or other programmable data processing device. Means for implementing the functions specified in one or more of the flow or in a block or blocks of the flow chart.
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。The computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device. The apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device. The instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.
尽管已描述了本申请的优选实施例，但本领域内的技术人员一旦得知了基本创造性概念，则可对这些实施例作出另外的变更和修改。所以，所附权利要求意欲解释为包括优选实施例以及落入本申请范围的所有变更和修改。While the preferred embodiment of the present application has been described, it will be apparent that those skilled in the art can make further changes and modifications to the embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and the modifications and
显然，本领域的技术人员可以对本申请实施例进行各种改动和变型而不脱离本申请实施例的精神和范围。这样，倘若本申请实施例的这些修改和变型属于本申请权利要求及其等同技术的范围之内，则本申请也意图包含这些改动和变型在内。 It is apparent that those skilled in the art can make various changes and modifications to the embodiments of the present application without departing from the spirit and scope of the embodiments of the present application. Thus, it is intended that the present invention cover the modifications and variations of the embodiments of the present invention.

Claims (52)

  1. 一种用户姿态检测方法，其特征在于，包括：A user attitude detecting method, comprising:
    检测终端设备绝对姿态参数、用户相对姿态参数、用户绝对姿态参数中的至少一个参数；Detecting at least one of an absolute posture parameter, a user relative posture parameter, and a user absolute posture parameter of the terminal device;
    所述终端设备绝对姿态参数的检测方法是：姿态传感器采集终端显示屏所处空间状态获得设备姿态传感数据，计算出终端设备绝对姿态参数；The method for detecting the absolute attitude parameter of the terminal device is: the attitude sensor collects the spatial state of the display screen of the terminal to obtain the device attitude sensing data, and calculates the absolute attitude parameter of the terminal device;
    所述用户相对姿态参数的检测方法是：用摄像头采集用户图像或视频，从图像信号中识别用户面部的特征点，计算出用户相对姿态参数；The method for detecting the relative posture parameter of the user is: collecting a user image or a video by using a camera, identifying a feature point of the user's face from the image signal, and calculating a relative posture parameter of the user;
    所述用户绝对姿态参数，是将所述用户相对姿态参数和所述设备绝对姿态参数相结合；The user absolute posture parameter is a combination of the user relative posture parameter and the device absolute posture parameter;
    将至少一个所述姿态参数和该参数的门限值进行比较；Comparing at least one of the attitude parameters with a threshold of the parameter;
    当至少一个所述姿态参数超过门限值时，发出提示信号和/或矫正建议信号。A prompt signal and/or a correction suggestion signal is issued when at least one of the attitude parameters exceeds a threshold.
  2. 一种用户姿态检测方法，其特征在于，包括：A user attitude detecting method, comprising:
    检测终端设备绝对姿态参数、用户相对姿态参数、用户绝对姿态参数中的至少一个参数；Detecting at least one of an absolute posture parameter, a user relative posture parameter, and a user absolute posture parameter of the terminal device;
    所述终端设备绝对姿态参数的检测方法是：姿态传感器采集终端显示屏所处空间状态获得设备姿态传感数据，计算出终端设备绝对姿态参数；The method for detecting the absolute attitude parameter of the terminal device is: the attitude sensor collects the spatial state of the display screen of the terminal to obtain the device attitude sensing data, and calculates the absolute attitude parameter of the terminal device;
    所述用户相对姿态参数的检测方法是：用摄像头采集用户图像或视频，从图像信号中识别用户面部的特征点，计算出用户相对姿态参数；The method for detecting the relative posture parameter of the user is: collecting a user image or a video by using a camera, identifying a feature point of the user's face from the image signal, and calculating a relative posture parameter of the user;
    所述用户绝对姿态参数，是将所述用户相对姿态参数和所述设备绝对姿态参数相结合；The user absolute posture parameter is a combination of the user relative posture parameter and the device absolute posture parameter;
    将至少一个所述姿态参数和该参数的门限值进行比较；Comparing at least one of the attitude parameters with a threshold of the parameter;
    以上步骤循环进行，当至少一个所述姿态参数连续多次循环超过门限值时，发出提示信号和/或矫正建议信号。The above steps are performed cyclically, and when at least one of the attitude parameters repeatedly exceeds a threshold value for a plurality of times, a prompt signal and/or a correction suggestion signal is issued.
  3. 如权利要求1或2所述的方法，其特征在于，所述终端设备的绝对姿态参数，是显示屏所在平面的水平倾角。The method according to claim 1 or 2, wherein the absolute attitude parameter of the terminal device is a horizontal inclination of a plane on which the display screen is located.
  4. 如权利要求1或2所述的方法，其特征在于，所述用户相对姿态参数包括以下参数中一个或多个：用户面部参考点与终端显示屏参考点之间的水平距离、用户面部参考点与终端显示屏参考点之间的垂直距离、用户面部中间线和显示屏之间的相对倾角。 The method according to claim 1 or 2, wherein the user relative posture parameter comprises one or more of the following parameters: a horizontal distance between the user's face reference point and the terminal display reference point, and a user's face reference point The vertical distance from the terminal display reference point, the relative inclination between the user's facial middle line and the display.
  5. 如权利要求1或2所述的方法，其特征在于，所述用户相对姿态参数包括以下参数中一个或多个：摄像头图像数据中用户面部占据的范围、面部中间线偏离摄像头图像中间线的平均距离、用户面部中间线和显示屏之间的相对倾角。The method according to claim 1 or 2, wherein the user relative posture parameter comprises one or more of the following parameters: a range occupied by the user's face in the camera image data, and an average of the middle line of the face deviating from the middle line of the camera image The relative inclination between the distance, the middle line of the user's face, and the display.
  6. 如权利要求1或2所述的方法，其特征在于，还包含：检测显示屏是否处于显示状态，如正常显示，执行权利要求1或2所述各步骤。The method of claim 1 or 2, further comprising: detecting whether the display screen is in a display state, such as a normal display, performing the steps of claim 1 or 2.
  7. 如权利要求1或2所述的方法，其特征在于，还包含延迟的步骤。The method of claim 1 or 2 further comprising the step of delaying.
  8. 如权利要求4所述的方法，其特征在于，所述计算用户相对姿态参数的方法是使用单目机器视觉的方法、或者双目机器视觉的方法、或者多目机器视觉的方法，得到人面部中线特征点A、B与摄像头镜头中心点C的距离，再计算出所述姿态参数。The method according to claim 4, wherein said method of calculating a user's relative posture parameter is a method of using a monocular machine vision, or a binocular machine vision method, or a multi-machine machine vision method to obtain a human face The distance between the center line feature points A, B and the camera lens center point C is calculated.
  9. 如权利要求4所述的方法，其特征在于，所述计算用户相对姿态参数的方法是采用3D摄像机，直接输出立体图形，得到人面部中线特征点A、B与摄像头镜头中心点C的距离，再计算出所述姿态参数。The method according to claim 4, wherein the method for calculating a relative posture parameter of the user is to directly output a stereoscopic image by using a 3D camera, and obtain a distance between a line feature point A, B of the human face and a center point C of the camera lens. The pose parameter is then calculated.
  10. 如权利要求8所述的方法，在使用单目机器视觉的方法时，其特征在于，还包含校准的步骤：用户保持特定已知的姿态参数值，用单目机器视觉的方法得到AB长度的典型值。The method according to claim 8, wherein when the method of monocular machine vision is used, the method further comprises the step of calibrating: the user maintains a specific known attitude parameter value, and obtains the AB length by a monocular machine vision method. Typical value.
  11. 一种用户姿态检测系统，其特征在于，包括：A user attitude detection system, comprising:
    一个或多个面向用户的摄像头，获得用户面部图像或视频；One or more user-facing cameras that obtain user facial images or videos;
    一个或多个姿态传感器，获得终端设备的绝对姿态参数，所述终端设备的绝对姿态参数反映屏幕所在平面的水平倾角；One or more attitude sensors obtain an absolute attitude parameter of the terminal device, and the absolute attitude parameter of the terminal device reflects a horizontal inclination of a plane where the screen is located;
    一个信息处理模块，包含姿态传感数据接口、图像数据接口、姿态提醒接口；An information processing module includes an attitude sensing data interface, an image data interface, and a gesture reminding interface;
    一个姿态提醒模块，包含信息输出装置；a gesture reminding module, comprising an information output device;
    所述信息处理模块通过所述姿态传感数据接口获得所述终端设备绝对姿态参数；The information processing module obtains the absolute posture parameter of the terminal device by using the attitude sensing data interface;
    所述信息处理模块通过所述图像数据接口获得所述用户面部图像或视频的数据；The information processing module obtains data of the user's facial image or video through the image data interface;
    所述信息处理模块运用所述终端设备绝对姿态参数和所述用户面部图像或视频的数据计算用户姿态参数，对姿态进行判断，当姿态不正确时通过所述姿态提醒接口发出信号，触发所述姿态提醒模块；The information processing module calculates the user posture parameter by using the terminal device absolute posture parameter and the data of the user facial image or video, and determines the posture, and sends a signal through the posture reminding interface when the posture is incorrect, triggering the Gesture reminder module;
    所述姿态提醒模块向用户发出感官可识别的信号，包括提示信号和/或矫正建议信号。The gesture reminder module sends a sensory identifiable signal to the user, including a prompt signal and/or a correction suggestion signal.
  12. 如权利要求11所述的系统，其特征在于，所述信息处理模块包括机器视觉处理模块、姿态计算模块、姿态判断模块； The system according to claim 11, wherein the information processing module comprises a machine vision processing module, an attitude calculation module, and an attitude determination module;
    所述机器视觉处理模块处理摄像头得到的图像或视频，从中提取出用户相对姿态参数；The machine vision processing module processes an image or video obtained by the camera, and extracts a user relative posture parameter therefrom;
    所述姿态计算模块根据终端设备的绝对姿态参数与用户相对姿态参数据计算用户的绝对姿态参数；The posture calculation module calculates an absolute posture parameter of the user according to the absolute posture parameter of the terminal device and the relative posture parameter data of the user;
    所述姿态判断模块对用户的相对姿态参数和绝对姿态参数进行判断，当发现有一个或多个姿态参数不正确时，通过所述姿态提醒接口发出数据，触发所述姿态提醒模块向用户发出提示信号和/或矫正建议信号。The posture judging module judges the relative posture parameter and the absolute posture parameter of the user, and when it is found that one or more posture parameters are incorrect, the data is sent through the posture reminding interface, and the posture reminding module is triggered to issue a prompt to the user. Signal and / or correction suggestion signal.
  13. 如权利要求11或12所述的系统，其特征在于，所述信息输出装置为指示灯、显示屏、振动装置、音频装置中的一种或多种；所述感官可识别的信号包括以下一种或多种：提示灯开关、颜色变化、闪烁；显示屏上显示的文本、图形、图像或视频；音频装置发出声音；振动装置发出的振动。A system according to claim 11 or 12, wherein said information output means is one or more of an indicator light, a display screen, a vibration device, and an audio device; said sensory identifiable signal comprises the following Kind or more: cue light switch, color change, flicker; text, graphics, image or video displayed on the display; sound from the audio device; vibration from the vibrating device.
  14. 如权利要求11或12所述的系统，其特征在于，所述摄像头为3D相机。A system according to claim 11 or 12, wherein the camera is a 3D camera.
  15. 如权利要求11或12所述的系统，其特征在于，至少一个所述摄像头，形成两个或多个虚拟摄像机。A system according to claim 11 or 12, wherein at least one of said cameras forms two or more virtual cameras.
  16. 如权利要求11或12所述的系统，其特征在于，所述终端设备中包括摄像头、姿态传感器、信息处理模块、姿态提醒模块。The system according to claim 11 or 12, wherein the terminal device comprises a camera, an attitude sensor, an information processing module, and a gesture reminding module.
  17. 如权利要求11或12所述的系统，其特征在于，所述终端设备中包括摄像头、姿态传感器、姿态提醒模块；所述信息处理模块在终端设备之外。The system according to claim 11 or 12, wherein the terminal device comprises a camera, an attitude sensor, and a gesture reminding module; and the information processing module is outside the terminal device.
  18. 如权利要求17所述的系统，其特征在于，还包括有线连接装置；所述姿态传感数据接口、图像数据接口、姿态提醒接口通过所述有线连接装置实现；所述有线连接装置是电缆或光缆。The system according to claim 17, further comprising a wired connection device; said attitude sensing data interface, image data interface, and gesture reminding interface are implemented by said wired connection device; said wired connection device is a cable or Cable.
  19. 如权利要求17所述的系统，其特征在于，还包括无线连接装置；所述姿态传感数据接口、图像数据接口、姿态提醒接口通过所述无线连接装置实现；所述无线连接装置包括天线和调制解调器。The system of claim 17 further comprising a wireless connection device; said gesture sensing data interface, image data interface, gesture alert interface being implemented by said wireless connection device; said wireless connection device comprising an antenna and modem.
  20. 如权利要求17所述的系统，其特征在于，还包括数据传送网络；所述姿态传感数据接口、图像数据接口、姿态提醒接口通过所述数据传送网络实现；所述数据传送网络是局域网或互联网；所述数据传送网络中包含计算机网络、或交换机网络、或路由器网络。The system of claim 17 further comprising a data transfer network; said gesture sensing data interface, image data interface, gesture alert interface being implemented by said data transfer network; said data transfer network being a local area network or Internet; the data transmission network includes a computer network, or a switch network, or a router network.
  21. 一种实现用户姿态检测的操作系统运行装置，包含传感模块驱动单元、输入模块驱动单元、输出模块驱动单元、姿态服务模块、应用数据库、应用程序编程接口，其 特征在于：An operating system running device for realizing user posture detection, comprising a sensing module driving unit, an input module driving unit, an output module driving unit, a posture service module, an application database, and an application programming interface, Features are:
    所述传感模块驱动单元输出传感控制信号、输入传感信号；The sensing module driving unit outputs a sensing control signal and an input sensing signal;
    所述输入模块驱动单元输出配置控制信号、接收用户指令；The input module driving unit outputs a configuration control signal and receives a user instruction;
    所述输出模块驱动单元输出告警控制信号、以及包含报警数据和/或姿态矫正数据的输出信号；The output module driving unit outputs an alarm control signal and an output signal including alarm data and/or attitude correction data;
    所述姿态服务模块中包含姿态检测单元、姿态比较单元、姿态提醒单元；The attitude service module includes an attitude detecting unit, a posture comparing unit, and a posture reminding unit;
    所述姿态检测单元启动传感模块驱动单元，读取所述传感模块驱动单元输出的姿态数据，识别用户面部的位置，产生姿态参数；所述姿态比较单元将所述姿态参数与对应的门限值进行比较；所述姿态提醒单元在用户姿态不正确时产生报警数据和/或姿态矫正数据，并启动所述输出模块驱动单元；The posture detecting unit activates the sensing module driving unit, reads posture data output by the sensing module driving unit, recognizes a position of the user's face, and generates a posture parameter; the posture comparing unit sets the posture parameter and the corresponding door The limit value is compared; the posture reminding unit generates alarm data and/or attitude correction data when the user posture is incorrect, and starts the output module driving unit;
    所述应用数据库存储工作数据和默认设置数据。The application database stores work data and default settings data.
  22. 一种实现用户姿态检测的操作系统运行装置，包含传感模块驱动单元、输入模块驱动单元、输出模块驱动单元、姿态服务模块、应用数据库、应用程序编程接口，其特征在于：An operating system running device for realizing user posture detection, comprising a sensing module driving unit, an input module driving unit, an output module driving unit, a posture service module, an application database, and an application programming interface, wherein:
    所述传感模块驱动单元输出传感控制信号、输入传感信号；The sensing module driving unit outputs a sensing control signal and an input sensing signal;
    所述输入模块驱动单元输出配置控制信号、接收用户指令；The input module driving unit outputs a configuration control signal and receives a user instruction;
    所述输出模块驱动单元输出告警控制信号、以及包含报警数据和/或姿态矫正数据的输出信号；The output module driving unit outputs an alarm control signal and an output signal including alarm data and/or attitude correction data;
    所述姿态服务模块中包含姿态检测单元和姿态比较单元；所述姿态检测单元启动所述传感模块驱动单元，读取所述传感模块驱动单元输出的姿态数据，识别用户面部的位置，产生姿态参数；所述姿态比较单元将所述姿态参数与对应的门限值进行比较；在姿态参数超出门限值时通过所述应用程序编程接口发出姿态超限通知；The attitude service module includes an attitude detection unit and a posture comparison unit; the attitude detection unit activates the sensing module driving unit, reads posture data output by the sensing module driving unit, and recognizes a position of the user's face, and generates a posture parameter; the attitude comparison unit compares the attitude parameter with a corresponding threshold value; and sends an attitude overrun notification through the application programming interface when the attitude parameter exceeds a threshold value;
    所述应用数据库存储工作数据和默认设置数据。The application database stores work data and default settings data.
  23. 一种实现用户姿态检测的操作系统运行装置，包含传感模块驱动单元、输入模块驱动单元、输出模块驱动单元、姿态服务模块、应用数据库、应用程序编程接口，其特征在于：An operating system running device for realizing user posture detection, comprising a sensing module driving unit, an input module driving unit, an output module driving unit, a posture service module, an application database, and an application programming interface, wherein:
    所述传感模块驱动单元输出传感控制信号、输入传感信号；The sensing module driving unit outputs a sensing control signal and an input sensing signal;
    所述输入模块驱动单元输出配置控制信号、接收用户指令；The input module driving unit outputs a configuration control signal and receives a user instruction;
    所述输出模块驱动单元输出告警控制信号、以及包含报警数据和/或姿态矫正数据的输出信号； The output module driving unit outputs an alarm control signal and an output signal including alarm data and/or attitude correction data;
    所述姿态服务模块中包含姿态检测单元，所述姿态检测单元启动所述传感模块驱动单元，读取所述传感模块驱动单元输出的姿态数据，识别用户面部的位置，产生姿态参数，将所述姿态参数保存在应用数据库中；The posture service module includes an attitude detecting unit, the posture detecting unit starts the sensing module driving unit, reads posture data output by the sensing module driving unit, recognizes a position of a user's face, and generates a posture parameter, and The pose parameter is saved in an application database;
    所述应用数据库存储工作数据和默认设置数据。The application database stores work data and default settings data.
  24. 如权利要求21～23任一项所述的装置，其特征在于，所述工作数据包含以下一种或多种：应用程序状态表、姿态检测标志位、应用程序注册表、姿态参数；The device according to any one of claims 21 to 23, wherein the work data comprises one or more of the following: an application state table, an attitude detection flag bit, an application registry, and a posture parameter;
    所述默认设置数据包含以下一种或多种：姿态检测标志位、应用程序注册表、门限值。The default setting data includes one or more of the following: an attitude detection flag bit, an application registry, and a threshold value.
  25. 如权利要求21～23任一项所述的装置，其特征在于，姿态服务模块中还包含数据扫描单元，所述数据扫描单元查询所述应用数据库中的工作数据，按照所述工作数据的指示，在需要启动姿态检测过程时，触发姿态检测单元；The apparatus according to any one of claims 21 to 23, wherein the gesture service module further comprises a data scanning unit, wherein the data scanning unit queries the working data in the application database according to the indication of the working data. When the gesture detection process needs to be started, the gesture detection unit is triggered;
    所述工作数据包含应用程序状态表和应用程序注册表；The work data includes an application state table and an application registry;
    按照所述工作数据的指示，是指所述数据扫描单元从应用程序状态表中获得正在使用显示屏的应用程序模块名称或标识符、检查所述应用程序模块是否在应用程序注册表中，如是，则需要启动姿态检测过程。According to the indication of the working data, the data scanning unit obtains an application module name or identifier that is using the display screen from the application state table, and checks whether the application module is in the application registry, if , you need to start the attitude detection process.
  26. 如权利要求21～23任一项所述的装置，其特征在于，姿态服务模块中还包含数据扫描单元，所述数据扫描单元查询所述应用数据库中的工作数据，按照所述工作数据的指示，在需要启动姿态检测过程时，触发姿态检测单元；The apparatus according to any one of claims 21 to 23, wherein the gesture service module further comprises a data scanning unit, wherein the data scanning unit queries the working data in the application database according to the indication of the working data. When the gesture detection process needs to be started, the gesture detection unit is triggered;
    所述工作数据包含姿态检测标志位；The working data includes a posture detection flag bit;
    按照所述工作数据的指示，是指所述数据扫描单元查询所述姿态检测标志位是否被设置，如是，则需要启动姿态检测过程。According to the indication of the working data, the data scanning unit queries whether the posture detection flag is set, and if so, the gesture detection process needs to be started.
  27. 如权利要求21～23任一项所述的装置，其特征在于，所述姿态参数，包含终端绝对姿态参数、相对姿态参数、用户绝对姿态参数中的一种或多种。The apparatus according to any one of claims 21 to 23, wherein the attitude parameter comprises one or more of a terminal absolute attitude parameter, a relative attitude parameter, and a user absolute posture parameter.
  28. 一种应用程序模块，用于如权利要求22所述的装置，其特征在于，包含姿态提醒单元和输入输出处理单元；An application module, comprising the apparatus according to claim 22, comprising: a gesture reminding unit and an input and output processing unit;
    所述姿态提醒单元产生报警数据和/或姿态矫正数据；The gesture reminding unit generates alarm data and/or attitude correction data;
    所述输入输出处理单元通过应用程序编程接口读取或设置应用数据库中的工作数据；The input/output processing unit reads or sets working data in an application database through an application programming interface;
    所述输入输出处理单元通过应用程序编程接口接收来自操作系统运行装置的姿态超限通知； The input/output processing unit receives an attitude overrun notification from an operating system running device through an application programming interface;
    所述输入输出处理单元通过应用程序编程接口启动输入模块驱动单元，接收用户指令；The input/output processing unit activates an input module driving unit through an application programming interface to receive a user instruction;
    所述输入输出处理单元通过应用程序编程接口启动输出模块驱动单元，发出包含报警数据和/或姿态矫正数据的信号。The input/output processing unit activates the output module drive unit through an application programming interface to issue a signal containing alarm data and/or attitude correction data.
  29. 一种应用程序模块，用于如权利要求23所述的装置，其特征在于，包含姿态比较单元、姿态提醒单元和输入输出处理单元；An application module, comprising the apparatus according to claim 23, comprising: a posture comparing unit, a posture reminding unit, and an input/output processing unit;
    所述输入输出处理单元，通过所述应用程序编程接口设置或读取应用数据库中的工作数据；The input/output processing unit sets or reads working data in an application database through the application programming interface;
    所述姿态比较单元将所述姿态参数与对应的门限值进行比较；The attitude comparison unit compares the attitude parameter with a corresponding threshold value;
    所述姿态提醒单元产生报警数据和/或姿态矫正数据；The gesture reminding unit generates alarm data and/or attitude correction data;
    所述输入输出处理单元通过应用程序编程接口启动输入模块驱动单元，接收用户指令；The input/output processing unit activates an input module driving unit through an application programming interface to receive a user instruction;
    所述输入输出处理单元通过应用程序编程接口启动输出模块驱动单元，发出包含报警数据和/或姿态矫正数据的信号。The input/output processing unit activates the output module drive unit through an application programming interface to issue a signal containing alarm data and/or attitude correction data.
  30. 一种应用程序模块，用于如权利要求25所述的装置，其特征在于：An application module for use in the apparatus of claim 25, wherein:
    应用程序模块通过应用程序编程接口修改应用数据库中的应用程序注册表，写入该应用程序模块的名称或标识符；The application module modifies the application registry in the application database through the application programming interface, and writes the name or identifier of the application module;
    应用程序模块占据显示屏时，通过应用程序编程接口修改应用数据库中的应用程序状态表，写入该应用程序模块的名称或标识符。When the application module occupies the display, the application state table in the application database is modified through the application programming interface, and the name or identifier of the application module is written.
  31. 一种应用程序模块，用于如权利要求26所述的装置，其特征在于，应用程序模块通过应用程序编程接口设置或清除应用数据库中的姿态检测标志位。An application module for use in the apparatus of claim 26 wherein the application module sets or clears gesture detection flag bits in the application database via an application programming interface.
  32. 如权利要求28～30任一项所述的应用程序模块，其特征在于，所述姿态参数，包含终端绝对姿态参数、相对姿态参数、用户绝对姿态参数中的一种或多种。The application module according to any one of claims 28 to 30, wherein the posture parameter comprises one or more of a terminal absolute posture parameter, a relative posture parameter, and a user absolute posture parameter.
  33. 一种通过操作系统实现用户姿态检测的移动终端，包含姿态传感模块、用户配置输入模块、告警信号输出模块、操作系统运行装置、显示屏、一个或多个应用程序模块，其特征在于：A mobile terminal for realizing user posture detection by an operating system, comprising an attitude sensing module, a user configuration input module, an alarm signal output module, an operating system running device, a display screen, and one or more application modules, wherein:
    所述姿态传感模块产生姿态传感信号；The attitude sensing module generates an attitude sensing signal;
    所述用户配置输入模块识别用户操作，将用户指令传递至操作系统运行装置；The user configuration input module identifies a user operation and transmits the user instruction to an operating system running device;
    所述告警信号输出模块发出用户感官可识别的信号；The alarm signal output module sends a signal recognizable by the user's senses;
    所述操作系统运行装置中包含传感模块驱动单元、输入模块驱动单元、输出模块驱 动单元；所述传感模块驱动单元控制所述姿态传感模块工作、并将所述姿态传感模块输出的姿态传感信号转换为姿态数据；所述输入模块驱动单元控制所述用户配置输入模块工作，接收用户指令；所述输出模块驱动单元输出包含报警数据和/或姿态矫正数据的信号，控制所述告警信号输出模块工作；The operating system running device includes a sensing module driving unit, an input module driving unit, and an output module driving The sensing unit driving unit controls the operation of the attitude sensing module, and converts the attitude sensing signal output by the attitude sensing module into attitude data; the input module driving unit controls the user configuration input The module works to receive a user instruction; the output module driving unit outputs a signal including alarm data and/or attitude correction data, and controls the alarm signal output module to operate;
    所述操作系统运行装置中还包含姿态服务模块，所述姿态服务模块中包含姿态检测单元、姿态比较单元和姿态提醒单元；所述姿态检测单元启动所述传感模块驱动单元，读取所述传感模块驱动单元输出的姿态数据，识别用户面部的位置，产生姿态参数；所述姿态比较单元，将所述姿态参数与对应的门限值进行比较；所述姿态提醒单元，产生报警数据和/或姿态矫正数据；The operating system running device further includes a posture service module, wherein the posture service module includes an attitude detecting unit, a posture comparing unit, and a posture reminding unit; the posture detecting unit starts the sensing module driving unit, and reads the The attitude data output by the sensing module driving unit identifies the position of the user's face to generate a posture parameter; the attitude comparison unit compares the attitude parameter with a corresponding threshold value; the gesture reminding unit generates alarm data and / or posture correction data;
    所述操作系统运行装置中还包含应用数据库，所述应用数据库存储工作数据和默认设置数据。The operating system running device further includes an application database, and the application database stores working data and default setting data.
  34. 一种通过操作系统实现用户姿态检测的方法，用于如权利要求31所述的移动终端，其特征在于，包括以下步骤：A method for implementing user gesture detection by an operating system, the mobile terminal of claim 31, comprising the steps of:
    操作系统运行装置中的姿态检测单元启动传感模块驱动单元采集传感信号，并转换为姿态数据；The attitude detecting unit in the operating system running device starts the sensing module driving unit to collect the sensing signal and converts into the attitude data;
    操作系统运行装置中的姿态检测单元读取传感模块驱动单元输出的姿态数据；The attitude detecting unit in the operating system running device reads the attitude data output by the sensing module driving unit;
    操作系统运行装置中的姿态检测单元对姿态数据进行分析，得出相对姿态参数和/或绝对姿态参数；The attitude detecting unit in the operating system running device analyzes the attitude data to obtain a relative attitude parameter and/or an absolute posture parameter;
    操作系统运行装置中的姿态比较单元将所述姿态参数和用户姿态门限值进行姿态比较；The attitude comparison unit in the operating system running device compares the posture parameter and the user posture threshold value;
    操作系统运行装置中的姿态提醒单元在所述姿态参数超过门限值时，生成报警数据和/或姿态矫正数据。The gesture reminding unit in the operating system operating device generates alarm data and/or attitude correction data when the attitude parameter exceeds a threshold.
  35. 一种通过操作系统实现用户姿态检测的移动终端，包含姿态传感模块、用户配置输入模块、告警信号输出模块、操作系统运行装置、显示屏、一个或多个应用程序模块，其特征在于：A mobile terminal for realizing user posture detection by an operating system, comprising an attitude sensing module, a user configuration input module, an alarm signal output module, an operating system running device, a display screen, and one or more application modules, wherein:
    所述姿态传感模块产生姿态传感信号；The attitude sensing module generates an attitude sensing signal;
    所述用户配置输入模块，识别用户操作，将用户指令传递至操作系统运行装置和至少一个所述应用程序模块；The user configuration input module identifies a user operation, and transmits the user instruction to the operating system running device and the at least one application module;
    所述告警信号输出模块发出用户感官可识别的信号；The alarm signal output module sends a signal recognizable by the user's senses;
    所述操作系统运行装置中包含传感模块驱动单元、输入模块驱动单元、输出模块驱 动单元；所述传感模块驱动单元控制所述姿态传感模块工作、并将所述姿态传感模块输出的姿态传感信号转换为姿态数据；所述输入模块驱动单元控制所述用户配置输入模块工作，接收用户指令；所述输出模块驱动单元输出包含报警数据和/或姿态矫正数据的信号，控制所述告警信号输出模块工作；The operating system running device includes a sensing module driving unit, an input module driving unit, and an output module driving The sensing unit driving unit controls the operation of the attitude sensing module, and converts the attitude sensing signal output by the attitude sensing module into attitude data; the input module driving unit controls the user configuration input The module works to receive a user instruction; the output module driving unit outputs a signal including alarm data and/or attitude correction data, and controls the alarm signal output module to operate;
    所述操作系统运行装置中还包含姿态服务模块，所述姿态服务模块中包含姿态检测单元和姿态比较单元；所述姿态检测单元启动所述传感模块驱动单元，读取所述传感模块驱动单元输出的姿态数据，识别用户面部的位置，产生姿态参数；所述姿态比较单元将所述姿态参数与对应的门限值进行比较；在姿态参数超出门限值时向至少一个所述应用程序模块发出姿态超限通知；The operating system running device further includes a posture service module, wherein the attitude service module includes an attitude detecting unit and a posture comparing unit; the posture detecting unit starts the sensing module driving unit, and reads the sensing module driving The attitude data output by the unit, the position of the user's face is recognized, and the attitude parameter is generated; the attitude comparison unit compares the attitude parameter with a corresponding threshold value; and when the attitude parameter exceeds the threshold value, at least one of the applications The module issues a gesture of over-limitation;
    所述操作系统运行装置包含应用程序编程接口；The operating system running device includes an application programming interface;
    所述操作系统运行装置中还包含应用数据库，所述应用数据库存储工作数据和默认设置数据；The operating system running device further includes an application database, where the application database stores working data and default setting data;
    至少一个所述应用程序模块中包含姿态提醒单元、输入输出处理单元；所述姿态提醒单元，产生报警数据和/或姿态矫正数据；所述输入输出处理单元，通过所述应用程序编程接口设置应用数据库中的工作数据、接收来自操作系统运行装置的姿态超限通知、启动输入模块驱动单元、启动输出模块驱动单元。At least one of the application modules includes a gesture reminding unit, an input and output processing unit; the gesture reminding unit generates alarm data and/or attitude correction data; the input/output processing unit sets an application through the application programming interface The working data in the database, receiving the attitude overrun notification from the operating system running device, starting the input module driving unit, and starting the output module driving unit.
  36. 一种通过操作系统实现用户姿态检测的方法，用于如权利要求33所述的移动终端，其特征在于，包括以下步骤：A method for implementing user gesture detection by an operating system, the mobile terminal of claim 33, comprising the steps of:
    操作系统运行装置中的姿态检测单元启动传感模块驱动单元采集传感信号，并转换为姿态数据；The attitude detecting unit in the operating system running device starts the sensing module driving unit to collect the sensing signal and converts into the attitude data;
    操作系统运行装置中的姿态检测单元读取传感模块驱动单元输出的姿态数据；The attitude detecting unit in the operating system running device reads the attitude data output by the sensing module driving unit;
    操作系统运行装置中的姿态检测单元对姿态数据进行分析，得出相对姿态参数和/或绝对姿态参数；The attitude detecting unit in the operating system running device analyzes the attitude data to obtain a relative attitude parameter and/or an absolute posture parameter;
    操作系统运行装置中的姿态比较单元对姿态参数和门限值进行比较；在所述姿态参数超过门限值时，通过应用程序编程接口向至少一个应用程序模块发出姿态超限通知；The attitude comparison unit in the operating system running device compares the attitude parameter and the threshold value; when the posture parameter exceeds the threshold value, sends an attitude overrun notification to the at least one application module through the application programming interface;
    占用显示屏的应用程序模块中的输入输出处理单元通过应用程序编程接口接收姿态超限通知；The input/output processing unit in the application module occupying the display screen receives the gesture overrun notification through the application programming interface;
    占用显示屏的应用程序模块中的姿态提醒单元在所述姿态参数超过门限值时，生成报警数据和/或姿态矫正数据。The gesture reminding unit in the application module occupying the display generates alarm data and/or attitude correction data when the attitude parameter exceeds a threshold.
  37. 一种通过操作系统实现用户姿态检测的移动终端，包含姿态传感模块、用户配 置输入模块、告警信号输出模块、操作系统运行装置、显示屏、一个或多个应用程序模块，其特征在于：A mobile terminal for realizing user posture detection through an operating system, comprising an attitude sensing module and a user An input module, an alarm signal output module, an operating system running device, a display screen, and one or more application modules are characterized by:
    所述姿态传感模块产生姿态传感信号；The attitude sensing module generates an attitude sensing signal;
    所述用户配置输入模块，识别用户操作，将用户指令传递至所述操作系统运行装置和至少一个所述应用程序模块；The user configuration input module identifies a user operation, and transmits user instructions to the operating system running device and at least one of the application modules;
    所述告警信号输出模块发出用户感官可识别的信号；The alarm signal output module sends a signal recognizable by the user's senses;
    所述操作系统运行装置中包含传感模块驱动单元、输入模块驱动单元、输出模块驱动单元；所述传感模块驱动单元控制所述姿态传感模块工作、并将所述姿态传感模块输出的姿态传感信号转换为姿态数据；所述输入模块驱动单元控制所述用户配置输入模块工作，接收用户指令；所述输出模块驱动单元输出包含报警数据和/或姿态矫正数据的信号，控制所述告警信号输出模块工作；The operating system running device includes a sensing module driving unit, an input module driving unit, and an output module driving unit; the sensing module driving unit controls the operation of the attitude sensing module, and outputs the attitude sensing module The attitude sensing signal is converted into attitude data; the input module driving unit controls the user configuration input module to operate, and receives a user instruction; the output module driving unit outputs a signal including alarm data and/or attitude correction data, and controls the The alarm signal output module works;
    所述操作系统运行装置中还包含姿态服务模块，所述姿态服务模块中包含姿态检测单元；所述姿态检测单元启动所述传感模块驱动单元，读取所述传感模块驱动单元输出的姿态数据，识别用户面部的位置，产生姿态参数；The operating system running device further includes a posture service module, wherein the attitude service module includes an attitude detecting unit; the posture detecting unit starts the sensing module driving unit, and reads a posture of the sensing unit driving unit Data, identifying the location of the user's face, and generating gesture parameters;
    所述操作系统运行装置包含应用程序编程接口；The operating system running device includes an application programming interface;
    至少一个所述应用程序模块中包含姿态比较单元、姿态提醒单元、输入输出处理单元；所述姿态比较单元，将所述姿态参数与对应的门限值进行比较；所述姿态提醒单元，产生报警数据和/或姿态矫正数据；所述输入输出处理单元，通过所述应用程序编程接口读取或设置应用数据库中工作数据、启动输入模块驱动单元、启动输出模块驱动单元；At least one of the application modules includes a posture comparison unit, a posture reminding unit, and an input/output processing unit; the posture comparison unit compares the posture parameter with a corresponding threshold value; and the posture reminding unit generates an alarm Data and/or posture correction data; the input/output processing unit reads or sets working data in the application database, starts the input module driving unit, and starts the output module driving unit through the application programming interface;
    所述操作系统运行装置中还包含应用数据库，所述应用数据库存储工作中产生的数据和默认设置数据。The operating system running device further includes an application database, and the application database stores data generated in the work and default setting data.
  38. 一种通过操作系统实现用户姿态检测的方法，用于如权利要求35所述通过操作系统实现用户姿态检测的移动终端，其特征在于，包括以下步骤：A method for implementing user gesture detection by an operating system, the mobile terminal for implementing user gesture detection by an operating system according to claim 35, comprising the steps of:
    操作系统运行装置中的姿态检测单元启动传感模块驱动单元采集传感信号，并转换为姿态数据；The attitude detecting unit in the operating system running device starts the sensing module driving unit to collect the sensing signal and converts into the attitude data;
    操作系统运行装置中的姿态检测单元读取传感模块驱动单元输出的姿态数据；The attitude detecting unit in the operating system running device reads the attitude data output by the sensing module driving unit;
    操作系统运行装置中的姿态检测单元对姿态数据进行分析，得出相对姿态参数和/或绝对姿态参数；The attitude detecting unit in the operating system running device analyzes the attitude data to obtain a relative attitude parameter and/or an absolute posture parameter;
    操作系统运行装置中的姿态检测单元将姿态参数保存在应用数据库中；The attitude detecting unit in the operating system running device saves the posture parameter in the application database;
    应用程序模块中的输入输出处理单元通过应用程序编程接口从应用数据库中读取 姿态参数和门限值；The input and output processing unit in the application module reads from the application database through the application programming interface Attitude parameters and threshold values;
    应用程序模块中的姿态比较单元将所述姿态参数和门限值进行姿态比较；An attitude comparison unit in the application module compares the attitude parameter with a threshold value;
    应用程序模块中的姿态提醒单元在所述姿态参数超过门限值时，生成报警数据和/或姿态矫正数据。The gesture reminding unit in the application module generates alarm data and/or attitude correction data when the attitude parameter exceeds a threshold.
  39. 如权利要求33、35、37任一所述的移动终端，其特征在于，所述姿态传感模块包含一只或多只前置摄像头；The mobile terminal according to any one of claims 33, 35, 37, wherein the attitude sensing module comprises one or more front cameras;
    所述传感模块驱动单元包含摄像头驱动单元；The sensing module driving unit includes a camera driving unit;
    所述姿态检测单元读取所述摄像头驱动单元输出的用户姿态数据，检测用户面部与显示屏的相对位置，产生相对姿态参数。The posture detecting unit reads the user posture data output by the camera driving unit, detects the relative position of the user's face and the display screen, and generates a relative posture parameter.
  40. 如权利要求33、35、37任一所述的移动终端，其特征在于，所述姿态传感模块包含一只或多只前置摄像头；The mobile terminal according to any one of claims 33, 35, 37, wherein the attitude sensing module comprises one or more front cameras;
    所述传感模块驱动单元包含摄像头驱动单元；The sensing module driving unit includes a camera driving unit;
    所述姿态传感模块中还包含能检测显示屏所在平面倾角的本机姿态传感器；The attitude sensing module further includes a local attitude sensor capable of detecting a plane inclination of the display screen;
    所述传感模块驱动单元中还包含本机姿态传感器驱动单元；The sensing module driving unit further includes a local attitude sensor driving unit;
    所述传感信号中包含用户图像信号和本机姿态信号；The sensing signal includes a user image signal and a local attitude signal;
    所述姿态检测单元读取摄像头驱动单元输出的用户姿态数据，检测用户面部与显示屏的相对位置，产生相对姿态参数；The posture detecting unit reads the user posture data output by the camera driving unit, detects the relative position of the user's face and the display screen, and generates a relative posture parameter;
    所述姿态检测单元读取所述本机姿态传感器驱动单元输出本机姿态数据，确定本机显示屏所在平面的倾角；The attitude detecting unit reads the local attitude sensor driving unit to output the local attitude data, and determines the inclination of the plane where the display screen of the local machine is located;
    所述姿态检测单元根据相对姿态参数和本机显示屏所在平面的倾角，产生绝对姿态参数。The attitude detecting unit generates an absolute attitude parameter according to the relative attitude parameter and the inclination of the plane of the local display screen.
  41. 如权利要求33、35、37任一所述的移动终端，其特征在于，所述姿态参数，包含终端绝对姿态参数、相对姿态参数、用户绝对姿态参数中的一种或多种。The mobile terminal according to any one of claims 33, 35, 37, wherein the attitude parameter comprises one or more of a terminal absolute attitude parameter, a relative attitude parameter, and a user absolute posture parameter.
  42. 如权利要求34、36、38任一所述的方法，其特征在于，所述姿态参数，包含终端绝对姿态参数、相对姿态参数、用户绝对姿态参数中的一种或多种。The method according to any one of claims 34, 36, 38, wherein the attitude parameter comprises one or more of a terminal absolute attitude parameter, a relative attitude parameter, and a user absolute posture parameter.
  43. 一种通过监控程序实现用户姿态检测的移动终端，包含姿态传感模块、用户配置输入模块、告警信号输出模块、一个或多个应用程序模块、状态数据库、监控程序模块，其特征在于：A mobile terminal for realizing user posture detection by a monitoring program, comprising an attitude sensing module, a user configuration input module, an alarm signal output module, one or more application modules, a state database, and a monitoring program module, wherein:
    所述姿态传感模块产生姿态数据；The attitude sensing module generates attitude data;
    所述用户配置输入模块识别用户操作，产生用户指令； The user configuration input module identifies a user operation and generates a user instruction;
    所述告警信号输出模块发出用户感官可识别的信号；The alarm signal output module sends a signal recognizable by the user's senses;
    所述状态数据库中存储所述应用程序模块的状态数据；Storing state data of the application module in the state database;
    所述监控程序模块中包含输入输出处理单元、姿态处理单元、状态扫描单元；The monitoring program module includes an input and output processing unit, an attitude processing unit, and a status scanning unit;
    所述输入输出处理单元与所述姿态传感模块、用户配置输入模块、告警信号输出模块相连接，接收所述姿态数据、接收所述用户指令，产生输出数据；The input/output processing unit is connected to the attitude sensing module, the user configuration input module, and the alarm signal output module, and receives the posture data, receives the user instruction, and generates output data;
    所述姿态处理单元根据姿态数据识别用户面部的位置，产生姿态参数，将所述姿态参数与对应的门限值进行比较，在用户姿态不正确时产生报警数据和/或姿态矫正数据；The posture processing unit identifies a position of the user's face according to the posture data, generates a posture parameter, compares the posture parameter with a corresponding threshold value, and generates alarm data and/or posture correction data when the user posture is incorrect;
    所述状态扫描单元周期性地扫描所述状态数据库，根据状所述态数据库中的状态数据启动姿态处理单元。The state scanning unit periodically scans the state database, and starts the gesture processing unit according to the state data in the state database.
  44. 一种通过监控程序实现用户姿态检测的移动终端，包含姿态传感模块、用户配置输入模块、告警信号输出模块、一个或多个应用程序模块、状态数据库、处理单元库、监控程序模块，其特征在于：A mobile terminal for realizing user posture detection by a monitoring program, comprising an attitude sensing module, a user configuration input module, an alarm signal output module, one or more application modules, a state database, a processing unit library, a monitoring program module, and features thereof Lie in:
    所述姿态传感模块产生姿态数据；The attitude sensing module generates attitude data;
    所述用户配置输入模块识别用户操作，产生用户指令；The user configuration input module identifies a user operation and generates a user instruction;
    所述告警信号输出模块发出用户感官可识别的信号；The alarm signal output module sends a signal recognizable by the user's senses;
    所述状态数据库中存储所述应用程序模块的状态数据；Storing state data of the application module in the state database;
    所述处理单元库包含姿态处理单元、应用程序编程接口；The processing unit library includes an attitude processing unit and an application programming interface;
    所述姿态处理单元根据姿态数据识别用户面部的位置，产生姿态参数，将所述姿态参数与对应的门限值进行比较，在用户姿态不正确时产生报警数据和/或姿态矫正数据；The posture processing unit identifies a position of the user's face according to the posture data, generates a posture parameter, compares the posture parameter with a corresponding threshold value, and generates alarm data and/or posture correction data when the user posture is incorrect;
    所述应用程序编程接口使姿态处理单元能够接收启动信号和数据、并输出结果；The application programming interface enables the gesture processing unit to receive the enable signal and data and output the result;
    所述监控程序模块中包含输入输出处理单元、姿态处理调用单元、状态扫描单元；The monitoring program module includes an input and output processing unit, a gesture processing calling unit, and a state scanning unit;
    所述输入输出处理单元与所述姿态传感模块、用户配置输入模块、告警信号输出模块相连接，接收所述姿态数据、接收所述用户指令，产生输出数据；The input/output processing unit is connected to the attitude sensing module, the user configuration input module, and the alarm signal output module, and receives the posture data, receives the user instruction, and generates output data;
    所述姿态处理调用单元能够查询姿态处理单元在处理单元库中的位置、发出启动信号、并传递姿态检测和比较所需要的姿态数据和门限值；The gesture processing calling unit is capable of querying a position of the gesture processing unit in the processing unit library, issuing a start signal, and transmitting attitude data and a threshold value required for posture detection and comparison;
    所述状态扫描模块周期性地扫描状态数据库，根据状态数据库中的状态数据启动姿态处理单元。The status scanning module periodically scans the status database and initiates an attitude processing unit based on the status data in the status database.
  45. 如权利要求43或44所述的移动终端，其特征在于，所述姿态传感模块包含摄像头及其驱动程序，产生用户姿态数据，所述摄像头为一只或多只前置摄像头；The mobile terminal according to claim 43 or 44, wherein the attitude sensing module comprises a camera and a driver thereof, and generates user posture data, wherein the camera is one or more front cameras;
    所述姿态处理单元检测用户面部与显示屏的相对位置，产生相对姿态参数。 The attitude processing unit detects a relative position of the user's face and the display screen to generate a relative attitude parameter.
  46. 如权利要求43或44所述的移动终端，其特征在于，所述姿态传感模块包含摄像头及其驱动程序，产生用户姿态数据，所述摄像头为一只或多只前置摄像头；The mobile terminal according to claim 43 or 44, wherein the attitude sensing module comprises a camera and a driver thereof, and generates user posture data, wherein the camera is one or more front cameras;
    所述姿态传感模块中还包含能检测主显示屏所在平面倾角的本机姿态传感器及其驱动程序，产生本机姿态数据；The attitude sensing module further includes a local attitude sensor capable of detecting a plane inclination of the main display screen and a driver thereof, and generating local attitude data;
    所述姿态处理单元根据用户姿态数据检测用户面部与显示屏的相对位置，产生相对姿态参数；The posture processing unit detects a relative position of the user's face and the display screen according to the user posture data, and generates a relative posture parameter;
    所述姿态处理单元根据本机姿态数据确定本机显示屏所在平面的倾角；The attitude processing unit determines an inclination of a plane where the display screen of the local machine is located according to the local posture data;
    所述姿态处理模块根据相对姿态参数和本机显示屏所在平面的倾角，产生绝对姿态参数。The attitude processing module generates an absolute attitude parameter according to the relative attitude parameter and the inclination of the plane of the local display screen.
  47. 如权利要求43或44所述的移动终端，其特征在于，所述姿态参数，包含终端绝对姿态参数、相对姿态参数、用户绝对姿态参数中的一种或多种。The mobile terminal according to claim 43 or 44, wherein the posture parameter comprises one or more of a terminal absolute posture parameter, a relative posture parameter, and a user absolute posture parameter.
  48. 一种通过监控程序实现用户姿态检测的方法，其特征在于，包括以下步骤：A method for realizing user posture detection by a monitoring program, comprising the steps of:
    监控程序模块中的状态扫描单元，周期性地扫描状态数据库，根据状态数据库中的状态信息启动姿态处理单元；a state scanning unit in the monitoring program module, periodically scanning the state database, and starting the gesture processing unit according to the state information in the state database;
    监控程序模块中的输入输出处理单元控制姿态传感模块采集用户姿态，接收姿态数据；姿态处理单元对姿态数据进行分析，得出相对姿态参数和/或绝对姿态参数；The input/output processing unit in the monitoring program module controls the attitude sensing module to collect the user posture and receive the attitude data; the attitude processing unit analyzes the attitude data to obtain a relative attitude parameter and/or an absolute posture parameter;
    监控程序模块中的姿态处理单元将用户姿态参数和对应的门限值进行比较；The attitude processing unit in the monitor module compares the user attitude parameter with the corresponding threshold value;
    监控程序模块中的姿态处理单元在用户姿态不正确时，生成报警数据和/或姿态矫正数据；输入输出处理单元输出信号触发告警信号输出模块向客户发出用户感觉可识别的信号，包含报警信息和/或矫正建议。The attitude processing unit in the monitoring program module generates alarm data and/or attitude correction data when the user posture is incorrect; the input and output processing unit output signal triggers the alarm signal output module to send a signal to the customer that the user feels identifiable, including the alarm information and / or correction suggestions.
  49. 如权利要求48所述的方法，其特征在于，所述姿态参数，包含终端绝对姿态参数、相对姿态参数、用户绝对姿态参数中的一种或多种。The method according to claim 48, wherein the attitude parameter comprises one or more of a terminal absolute attitude parameter, a relative attitude parameter, and a user absolute posture parameter.
  50. 一种监控程序模块，其特征在于，包含输入输出处理单元、姿态处理单元、状态扫描单元；A monitoring program module, comprising: an input and output processing unit, an attitude processing unit, and a state scanning unit;
    所述输入输出处理单元与移动终端中的姿态传感模块、用户配置输入模块、告警信号输出模块相连接，接收所述姿态传感模块产生的姿态数据，接收所述用户配置输入模块产生的用户指令，并产生输出数据；The input/output processing unit is connected to the attitude sensing module, the user configuration input module, and the alarm signal output module in the mobile terminal, and receives the posture data generated by the attitude sensing module, and receives the user generated by the user configuration input module. Command and generate output data;
    所述姿态处理单元根据所述姿态数据识别用户面部的位置，产生姿态参数，将所述姿态参数与对应的门限值进行比较，在用户姿态不正确时产生报警数据和/或姿态矫正数据； The posture processing unit identifies a position of the user's face according to the posture data, generates a posture parameter, compares the posture parameter with a corresponding threshold value, and generates alarm data and/or posture correction data when the user posture is incorrect;
    所述状态扫描单元周期性地扫描所述移动终端中的状态数据库，根据状所述态数据库中的状态数据启动姿态处理单元。The state scanning unit periodically scans the state database in the mobile terminal, and starts the gesture processing unit according to the state data in the state database.
  51. 一种监控程序模块，其特征在于，包含输入输出处理单元、姿态处理调用单元、状态扫描单元；A monitoring program module, comprising: an input and output processing unit, a gesture processing calling unit, and a state scanning unit;
    所述输入输出处理单元与移动终端中的姿态传感模块、用户配置输入模块、告警信号输出模块相连接，接收所述姿态传感模块产生的姿态数据，接收所述用户配置输入模块产生的用户指令，并产生输出数据；The input/output processing unit is connected to the attitude sensing module, the user configuration input module, and the alarm signal output module in the mobile terminal, and receives the posture data generated by the attitude sensing module, and receives the user generated by the user configuration input module. Command and generate output data;
    所述姿态处理调用单元能够查询姿态处理单元在所述移动终端的处理单元库中的位置、发出启动信号、并传递姿态检测和比较所需要的姿态数据和门限值；The gesture processing calling unit is capable of querying a position of the gesture processing unit in the processing unit library of the mobile terminal, issuing a start signal, and transmitting posture data and a threshold value required for posture detection and comparison;
    所述状态扫描模块周期性地扫描所述移动终端中的状态数据库，根据状态数据库中的状态数据启动姿态处理单元，使所述姿态处理单元根据姿态数据识别用户面部的位置，产生姿态参数，将所述姿态参数与对应的门限值进行比较，在用户姿态不正确时产生报警数据和/或姿态矫正数据。The state scanning module periodically scans the state database in the mobile terminal, and starts the gesture processing unit according to the state data in the state database, so that the gesture processing unit identifies the location of the user's face according to the posture data, and generates a posture parameter, which will The attitude parameter is compared with a corresponding threshold value, and alarm data and/or attitude correction data is generated when the user posture is incorrect.
  52. 如权利要求50或51所述的监控程序模块，其特征在于，所述姿态参数，包含终端绝对姿态参数、相对姿态参数、用户绝对姿态参数中的一种或多种。 The monitoring program module according to claim 50 or 51, wherein the attitude parameter comprises one or more of a terminal absolute attitude parameter, a relative attitude parameter, and a user absolute posture parameter.
PCT/CN2015/076622 2014-04-16 2015-04-15 User posture detection method, device and system WO2015158258A1 (en)

Applications Claiming Priority (14)

Application Number Priority Date Filing Date Title
CN201410153791.1A CN103927012B (en) 2014-04-16 2014-04-16 A kind of mobile terminal and method that user's attitude detection is realized by operating system
CN201410153807.9A CN103955272B (en) 2014-04-16 2014-04-16 A kind of terminal user's attitude detection system
CN201410153783.7A CN103927011B (en) 2014-04-16 2014-04-16 Realize the operating system device and application program module of user's attitude detection
CN201410153807.9 2014-04-16
CN201410153795.XA CN103927250B (en) 2014-04-16 2014-04-16 A kind of terminal user's attitude detecting method
CN201410153783.7 2014-04-16
CN201410153781.8 2014-04-16
CN201410153857.7A CN103955273B (en) 2014-04-16 2014-04-16 A kind of mobile terminal and method that user's attitude detection is realized by operating system
CN201410153857.7 2014-04-16
CN201410153793.0 2014-04-16
CN201410153795.X 2014-04-16
CN201410153793.0A CN103955271B (en) 2014-04-16 2014-04-16 A kind of mobile terminal and method that user's attitude detection is realized by operating system
CN201410153781.8A CN103927010B (en) 2014-04-16 2014-04-16 A kind of mobile terminal and method that user's attitude detection is realized by monitoring programme
CN201410153791.1 2014-04-16

Publications (1)

Publication Number Publication Date
WO2015158258A1 true WO2015158258A1 (en) 2015-10-22

Family

ID=54323486

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/076622 WO2015158258A1 (en) 2014-04-16 2015-04-15 User posture detection method, device and system

Country Status (1)

Country Link
WO (1) WO2015158258A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111930228A (en) * 2020-07-14 2020-11-13 北京字节跳动网络技术有限公司 Method, device, equipment and storage medium for detecting user gesture
CN112732497A (en) * 2020-12-29 2021-04-30 深圳微步信息股份有限公司 Terminal device and detection method based on terminal device

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090324024A1 (en) * 2008-06-25 2009-12-31 Postureminder Ltd System and method for improving posture
CN202801617U (en) * 2012-08-09 2013-03-20 联想(北京)有限公司 Position detecting device and electric equipment
CN103021140A (en) * 2011-09-20 2013-04-03 三星电子株式会社 Apparatus and method for assisting user to maintain correct posture
CN103927010A (en) * 2014-04-16 2014-07-16 北京尚德智产投资管理有限公司 Mobile terminal achieving user posture detection through monitoring program and method
CN103927012A (en) * 2014-04-16 2014-07-16 北京尚德智产投资管理有限公司 Mobile terminal achieving user posture detection through operating system and method
CN103927250A (en) * 2014-04-16 2014-07-16 北京尚德智产投资管理有限公司 User posture detecting method achieved through terminal device
CN103927011A (en) * 2014-04-16 2014-07-16 北京尚德智产投资管理有限公司 Operating system running device and application program module for detecting postures of users
CN103955273A (en) * 2014-04-16 2014-07-30 北京尚德智产投资管理有限公司 Mobile terminal and method for realizing user posture detection through operation system
CN103955272A (en) * 2014-04-16 2014-07-30 北京尚德智产投资管理有限公司 Terminal equipment user posture detecting system
CN103955271A (en) * 2014-04-16 2014-07-30 北京尚德智产投资管理有限公司 Mobile terminal and method for realizing user posture detection through operation system

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090324024A1 (en) * 2008-06-25 2009-12-31 Postureminder Ltd System and method for improving posture
CN103021140A (en) * 2011-09-20 2013-04-03 三星电子株式会社 Apparatus and method for assisting user to maintain correct posture
CN202801617U (en) * 2012-08-09 2013-03-20 联想(北京)有限公司 Position detecting device and electric equipment
CN103927010A (en) * 2014-04-16 2014-07-16 北京尚德智产投资管理有限公司 Mobile terminal achieving user posture detection through monitoring program and method
CN103927012A (en) * 2014-04-16 2014-07-16 北京尚德智产投资管理有限公司 Mobile terminal achieving user posture detection through operating system and method
CN103927250A (en) * 2014-04-16 2014-07-16 北京尚德智产投资管理有限公司 User posture detecting method achieved through terminal device
CN103927011A (en) * 2014-04-16 2014-07-16 北京尚德智产投资管理有限公司 Operating system running device and application program module for detecting postures of users
CN103955273A (en) * 2014-04-16 2014-07-30 北京尚德智产投资管理有限公司 Mobile terminal and method for realizing user posture detection through operation system
CN103955272A (en) * 2014-04-16 2014-07-30 北京尚德智产投资管理有限公司 Terminal equipment user posture detecting system
CN103955271A (en) * 2014-04-16 2014-07-30 北京尚德智产投资管理有限公司 Mobile terminal and method for realizing user posture detection through operation system

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111930228A (en) * 2020-07-14 2020-11-13 北京字节跳动网络技术有限公司 Method, device, equipment and storage medium for detecting user gesture
CN112732497A (en) * 2020-12-29 2021-04-30 深圳微步信息股份有限公司 Terminal device and detection method based on terminal device

Similar Documents

Publication Publication Date Title
CN108508969B (en) Electronic device including curved display and method of displaying image of electronic device
KR102593824B1 (en) Method for controlling a camera and electronic device thereof
CN103955272B (en) A kind of terminal user's attitude detection system
KR102534724B1 (en) Electronic apparatus and operating method thereof
EP3264744B1 (en) Electronic device and image capturing method thereof
US11497406B2 (en) Apparatus and method for enhancing accuracy of a contactless body temperature measurement
KR102296396B1 (en) Apparatus and method for improving accuracy of contactless thermometer module
KR102561587B1 (en) Electronic apparatus and operating method thereof
KR102439245B1 (en) Electronic device and controlling method thereof
CN108268095B (en) Electronic device
KR102318806B1 (en) Method for charging pen and an electronic device thereof
CN103927250B (en) A kind of terminal user's attitude detecting method
EP3293723A1 (en) Method, storage medium, and electronic device for displaying images
KR102499139B1 (en) Electronic device for displaying image and method for controlling thereof
KR102317021B1 (en) Display apparatus and image correction method thereof
KR102154802B1 (en) Photogrpaphing method of electronic apparatus and electronic apparatus thereof
EP3189412B1 (en) Electronic device and display method thereof
EP3023971A2 (en) Method for displaying image and electronic device thereof
KR102525126B1 (en) Electronic device comprising iris camera
KR102481486B1 (en) Method and apparatus for providing audio
KR20180059191A (en) Electronic device, external electronic device and method for connecting between electronic device and external electronic device
CN109379539A (en) A kind of screen light compensation method and terminal
KR102488580B1 (en) Apparatus and method for providing adaptive user interface
WO2015158258A1 (en) User posture detection method, device and system
EP3504584B1 (en) Electronic device and method for displaying image

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15780057

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15780057

Country of ref document: EP

Kind code of ref document: A1