CN113311984A - Touch screen track data processing method and device, mobile terminal and electronic equipment - Google Patents
Touch screen track data processing method and device, mobile terminal and electronic equipment Download PDFInfo
- Publication number
- CN113311984A CN113311984A CN202010125722.5A CN202010125722A CN113311984A CN 113311984 A CN113311984 A CN 113311984A CN 202010125722 A CN202010125722 A CN 202010125722A CN 113311984 A CN113311984 A CN 113311984A
- Authority
- CN
- China
- Prior art keywords
- track
- contact
- information
- touch screen
- trajectory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 20
- 238000012545 processing Methods 0.000 claims abstract description 51
- 238000003062 neural network model Methods 0.000 claims abstract description 44
- 238000000034 method Methods 0.000 claims description 32
- 238000009877 rendering Methods 0.000 claims description 2
- 230000004044 response Effects 0.000 abstract description 7
- 238000004891 communication Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 8
- 238000012549 training Methods 0.000 description 7
- 238000013528 artificial neural network Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000005236 sound signal Effects 0.000 description 4
- 238000007726 management method Methods 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002650 habitual effect Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The present disclosure relates to a data processing method of a touch screen trajectory, a data processing apparatus of a touch screen trajectory, a mobile terminal, an electronic device, and a computer-readable storage medium. The data processing method of the touch screen track is applied to a mobile terminal, the mobile terminal comprises a touch screen, and the data processing method of the touch screen track comprises the following steps: acquiring contact information of a contact through which a track passes currently; predicting the track through a neural network model based on contact information of contacts through which the track passes at present to obtain predicted information of the track; based on the prediction information of the trajectory, the display image is drawn in advance. Through the neural network model, the prediction information after the current track can be predicted based on the current track, and the drawing and displaying of the image is prompted according to the prediction information, so that the system response time can be reduced, the delay is avoided, and the user experience is improved.
Description
Technical Field
The present disclosure relates to the field of mobile terminal control, and in particular, to a data processing method of a touch screen trajectory, a data processing apparatus of a touch screen trajectory, a mobile terminal, an electronic device, and a computer-readable storage medium.
Background
At present, mobile terminals such as mobile phones and tablet computers basically adopt touch screens as operation and input modes of users, users can click and swipe on the touch screens, and the mobile terminals continue corresponding processing according to contact positions or swiping tracks. In some scenarios, corresponding display needs to be performed according to a touch point or a track of a user on the touch screen, that is, a drawing display image corresponding to a current touch position, such as a touch point cursor, a dragged icon or page, a sliding window, and the like.
Since the mobile terminal is divided into a touch (touch) module for receiving touch and a display (screen) module for displaying, after a user touches the screen with a finger, the system draws (draw) a display image on the screen when the next vertical synchronization (Vsync) is performed according to the touch position. Therefore, there is always a time difference, i.e., delay (latency), from the touch of the user to the final touch point image composition displayed on the screen, the frequency of the vertical synchronization is generally 60Hz, and if the touch position is changed during the waiting period of the vertical synchronization, the display image may deviate from the current position touched by the user due to the delay, which causes the problems of unsmooth display and display delay, and affects the user experience.
Disclosure of Invention
To overcome the problems in the related art, the present disclosure provides a data processing method of a touch screen trajectory, a data processing apparatus of a touch screen trajectory, a mobile terminal, an electronic device, and a computer-readable storage medium.
According to a first aspect of the embodiments of the present disclosure, there is provided a data processing method for a touch screen track, which is applied to a mobile terminal, where the mobile terminal includes a touch screen, and the data processing method for the touch screen track includes: acquiring contact information of a contact through which a track passes currently; predicting the track through a neural network model based on contact information of contacts through which the track passes at present to obtain predicted information of the track; based on the prediction information of the trajectory, the display image is drawn in advance.
In one embodiment, the prediction information includes: predicted contact coordinates of each predicted contact through which the predicted trajectory of the predicted trajectory passes, and predicted contact time of each predicted contact.
In one embodiment, based on the prediction information of the trajectory, the drawing display image in advance comprises: and drawing a display image in sequence at the corresponding position of each predicted touch point coordinate of the touch screen based on each predicted touch point time.
In one embodiment, the contact information includes any one or more of: contact point coordinates of the contact points, contact point time of the contact points, and pressure values of the contact points.
In an embodiment, the method further comprises: after the track is finished, acquiring contact information of all contacts through which the track passes; parameters of the neural network model are updated based on contact information of all contacts of the trajectory and prediction information of the trajectory.
In an embodiment, the method further comprises: judging whether the number of contacts passed by the current track reaches a set number; if the number of the contacts passed by the current track reaches the set number, executing the step of predicting the track through a neural network model based on the contact information of the contacts passed by the current track to obtain the predicted information of the track; and if the number of the contacts passed by the current track does not reach the set number, continuing to execute the step of acquiring the contact information of the contacts passed by the current track.
In one embodiment, the set number is: the neural network model requires the minimum number of contacts when the confidence of obtaining the prediction information reaches a set threshold value based on the contact information of the contacts.
In an embodiment, the method further comprises: after the track is finished, acquiring contact information of all contacts through which the track passes; the set number is updated based on contact point information of all contact points of the track and prediction information of the track.
According to a second aspect of the embodiments of the present disclosure, there is provided a data processing apparatus for a touch screen trajectory, which is applied to a mobile terminal including a touch screen, the data processing apparatus for the touch screen trajectory including: the acquisition unit is used for acquiring contact information of a contact through which a track passes currently; the prediction unit is used for predicting the track through a neural network model based on the contact information of the contact through which the track passes at present to obtain the prediction information of the track; and a display unit for drawing the display image in advance based on the prediction information of the trajectory.
In one embodiment, the prediction information includes: predicted contact coordinates of each predicted contact through which the predicted trajectory of the predicted trajectory passes, and predicted contact time of each predicted contact.
In one embodiment, the display unit is further configured to sequentially draw the display image at a corresponding position of each predicted touch point coordinate on the touch screen based on each predicted touch point time.
In one embodiment, the contact information includes any one or more of: contact point coordinates of the contact points, contact point time of the contact points, and pressure values of the contact points.
In an embodiment, the obtaining unit is further configured to: after the track is finished, acquiring contact information of all contacts through which the track passes; the device still includes: and the parameter updating unit is used for updating the parameters of the neural network model based on the contact information of all the contacts of the track and the prediction information of the track.
In an embodiment, the apparatus further comprises: the judging unit is used for judging whether the number of the contacts passed by the current track reaches the set number or not; when the number of the contacts passed by the current track reaches a set number, executing contact information based on the contacts passed by the current track through a prediction unit, and predicting the track through a neural network model to obtain prediction information of the track; when the number of the contacts which the current track passes does not reach the set number, the acquisition unit continues to acquire the contact information of the contacts which the track passes currently.
In one embodiment, the set number is: the neural network model requires the minimum number of contacts when the confidence of obtaining the prediction information reaches a set threshold value based on the contact information of the contacts.
In an embodiment, the obtaining unit is further configured to: after the track is finished, acquiring contact information of all contacts through which the track passes; the device still includes: and a number adjusting unit for updating the set number based on the contact point information of all the contact points of the track and the predicted information of the track.
According to a third aspect of the embodiments of the present disclosure, a mobile terminal is provided, where the mobile terminal includes a touch screen, and when the touch screen of the mobile terminal receives a touch pressure of a user, the touch screen performs processing according to the data processing method of the touch screen trajectory of the first aspect.
According to a fourth aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including: a memory to store instructions; and the processor is used for calling the instructions stored in the memory to execute the data processing method of the touch screen track in the first aspect.
According to a fifth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium storing instructions which, when executed by a processor, perform the data processing method of the touch screen trajectory of the first aspect.
The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects: through the neural network model, the prediction information after the current track can be predicted based on the current track, and the drawing and displaying of the image is prompted according to the prediction information, so that the system response time can be reduced, the delay is avoided, and the user experience is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
Fig. 1 is a flowchart illustrating a data processing method of a touch screen trace according to an exemplary embodiment.
FIG. 2 is a flowchart illustrating another method of data processing of touch screen traces in accordance with an exemplary embodiment.
FIG. 3 is a flowchart illustrating another method of data processing of touch screen traces in accordance with an exemplary embodiment.
FIG. 4 is a schematic block diagram illustrating a data processing apparatus for touch screen tracking in accordance with an exemplary embodiment.
FIG. 5 is a schematic block diagram illustrating another data processing apparatus for touch screen tracking in accordance with an exemplary embodiment.
FIG. 6 is a schematic block diagram illustrating another data processing apparatus for touch screen tracking in accordance with an exemplary embodiment.
FIG. 7 is a schematic block diagram illustrating another data processing apparatus for touch screen tracking in accordance with an exemplary embodiment.
FIG. 8 is a schematic block diagram illustrating an apparatus in accordance with an exemplary embodiment.
FIG. 9 is a schematic block diagram illustrating an electronic device in accordance with an exemplary embodiment.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
In some related art, upon a user touching a touch screen of a mobile terminal, drawing of a display image based on a current touch position on the screen is forced without waiting for the arrival of the next vertical synchronization. The method only ensures the synchronism of the first contact, and can be used for drawing a display image according to contact information acquired in the process only when the display image corresponding to the first contact is required to be subjected to next vertical synchronization because no event exists at present because the display image corresponding to the first contact is forcibly displayed if a user swipes on the touch screen after touching the touch screen, so that the delay is larger and the display image is obviously jumped.
In order to solve the above problem, an embodiment of the present disclosure provides a data processing method 10 for a touch screen track, which may be used for a mobile terminal such as a mobile phone and a tablet computer, where the mobile terminal includes a touch screen, a user may perform a corresponding operation on the mobile terminal by touching the touch screen to form a contact or a track formed by continuous contacts, and the touch screen may also be used to draw a display image according to a touch position of the user. As shown in fig. 1, the data processing method 10 for touch screen trace may include steps S11-S13:
in step S11, contact point information of the contact point through which the trajectory has currently passed is acquired.
The track of the touch screen is formed by the user continuously touching the touch screen and comprises all the contact points in the continuous time, and if the touch position of the user is not changed, the track comprises the contact points at each time point on the touch position. The contact information acquired in this embodiment may be information of each contact on the track, and the most information can be acquired in the same track length by acquiring the information of each contact, so that a more accurate prediction can be made, and a prediction can also be performed more quickly; the contact points are obtained in such a way, the calculation amount can be reduced, and when the number of the obtained contact points is consistent, the track in such a way passes through a longer distance, so that accurate prediction can be made.
In one embodiment, the contact information includes any one or more of: contact point coordinates of the contact points, contact point time of the contact points, and pressure values of the contact points. After a user touches the touch screen, the contact point information of the contact point through which the track passes includes contact point coordinates of the contact point, namely position information of the contact point on the touch screen, and also includes contact point time of the contact point, namely a time point at which the corresponding contact point is touched. The information is obtained based on the existing trajectory as a basis for prediction. In addition, when the user touches the touch screen to make a stroke, the magnitude and the variation information of the pressure are also related to the track, for example, the pressure is relatively high at the initial position, the pressure may be relatively low at the end of the track, and at the break point of the track, the pressure may also be increased. Therefore, the pressure information of the contact is obtained, the information input into the neural network can be more comprehensive, and a more accurate prediction result can be obtained.
And step S12, predicting the track through the neural network model based on the contact information of the contact point which the track passes through at present, and obtaining the prediction information of the track. Inputting the contact information of the current track into the neural network model, predicting the track through the trained neural network model, predicting the path of the predicted track after the current track, and based on the predicted track, the time required by the temporary response of the system can be reduced through preprocessing, so that the delay is avoided.
In step S13, the display image is drawn in advance based on the prediction information of the trajectory.
According to the predicted prediction information, the display image to be displayed is drawn in advance, so that display without delay can be achieved at the moment when the user actually touches the corresponding position, for example, a dragged window can correspond to a contact of the user at any moment, a contact cursor displayed on a screen can be kept at the position of a fingertip of the user at any moment, and user experience is improved.
In one embodiment, the prediction information includes: predicted contact coordinates of each predicted contact through which the predicted trajectory of the predicted trajectory passes, and predicted contact time of each predicted contact. In this embodiment, the complete trajectory after the current trajectory is predicted by the neural network model, that is, the position and time of each contact after the prediction are predicted, and the end point information is also included. Therefore, the system draws the display image corresponding to each contact on the predicted track in advance according to the predicted track, and the delay of drawing after receiving the contact is avoided through the advance response.
In one embodiment, step S13 may further include sequentially displaying the contact image at a corresponding position of each of the predicted contact coordinates on the touch screen based on each of the predicted contact times. In this embodiment, the display images are sequentially drawn at the corresponding positions at the corresponding times according to the predicted trajectory, for example, in the process of dragging the icon by the user, the icon image is drawn according to the predicted coordinate position of each contact based on the predicted trajectory information, and the corresponding image is displayed on the screen at the predicted contact time. Because the subsequent contact is predicted, the touch position is not required to be acquired in real time and displayed on a screen through system response, but the system responds in advance according to the predicted information, so that the displayed image and the touch position of the user can be synchronized, the following chirality is improved, the delay is avoided, and the user experience is improved.
In the embodiment of the present disclosure, the neural network model is a pre-trained neural network model, and may be locally disposed in the mobile terminal in order to save response time. The neural network is trained in advance through a training set, the training set can be a certain number of collected touch screen sliding tracks, contact information of a plurality of initial contacts in the sliding tracks is used as input, the input number can be determined by referring to the number of contacts contained in the sliding tracks on average and then through a ratio, the minimum sliding distance can be determined through a ratio, and a numerical value can also be fixed, such as twenty initial contacts. After the contact information is input into the neural network, the neural network predicts and outputs a predicted end point or track, then the actual end point or track of the track is used for supervision, and the parameters of the neural network are adjusted through feedback until the confidence coefficient of the predicted end point or track of the neural network exceeds a training threshold.
In some embodiments, a trained general neural network model may be preset in the mobile terminal, and the user may be predicted through the general neural network model. The mode of this embodiment can conveniently set up, need not the later stage training, nevertheless lacks user's characteristic, can't learn some individualized habitual operations of user.
In other embodiments, an untrained neural network model may be set in the mobile terminal or the cloud server, and in the previous period of using the mobile terminal by the user, the trajectory information of the user is collected, and the model training is continued in the local or uploading server, so as to obtain a completely personalized trained neural network model. According to the embodiment, the personalized data of the user can be acquired to the maximum extent, and the trained neural network model can make more accurate prediction according to the personal habits of the user. However, this method requires a lot of operations of the user to train the model, so the mobile terminal can only perform collection work in the early stage, and cannot perform prediction, and the collection process is long because training requires a lot of data.
In still other embodiments, combining the advantages of the two previous embodiments, as shown in fig. 2, the method 10 for processing data of a touch screen trace may further include: step S14, after the track is finished, acquiring contact information of all contacts through which the track passes; in step S15, parameters of the neural network model are updated based on the contact point information of all contact points of the trajectory and the prediction information of the trajectory. In this embodiment, a trained general neural network model is set in the mobile terminal, so that a user can reduce response time and avoid delay through the data processing method 10 for touch screen trajectories when using the mobile terminal, and meanwhile, personal operations of the user are also collected, parameters of the neural network model are adjusted according to the collected trajectories in the actual operations of the user, the adjustment principle can be similar to a training process, and the parameters can be adjusted after collecting a certain amount of user personal trajectories, or after obtaining the user personal trajectories, and the adjustment range is determined according to the data amount, so that excessive fluctuation of the parameters is avoided. Through the method, the experience of the user when the user starts to use the mobile terminal can be guaranteed, and the neural network model can be continuously adjusted to enable prediction to be more accurate.
In an embodiment, as shown in fig. 3, the method 10 for processing data of a touch screen trace may further include: step S16, judging whether the number of the contacts passed by the current track reaches the set number; if the number of the contacts passed by the current track reaches the set number, executing step S12; if the number of the contacts that the current trace has passed does not reach the set number, the step S11 is continuously executed. The neural network model needs to be predicted based on the input information, the amount of the input information also affects the accuracy of prediction, and the more information the neural network model obtains, the more accurate prediction result can be made, so that the set number is set in the embodiment, and the prediction is performed after the set number of contact information is obtained, so that the reliability of prediction is ensured.
The set quantity can be manually set, and the user can also adjust according to the actual use effect. In one embodiment, the set number may be: the neural network model requires the minimum number of contacts when the confidence of obtaining the prediction information reaches a set threshold value based on the contact information of the contacts. In this embodiment, the preset number is the number of the minimum number of contacts required by the neural network model under the condition that the confidence of the prediction information can be guaranteed to reach the set threshold, and the prediction is performed earlier under the condition that the prediction reliability can be guaranteed, so that the delay condition is avoided earlier.
In an embodiment, the method 10 for processing data of a touch screen trajectory may further include: after the track is finished, acquiring contact information of all contacts through which the track passes; the set number is updated based on contact point information of all contact points of the track and prediction information of the track. Similar to the aforementioned embodiment of adjusting parameters of the neural network model, the present embodiment verifies the accuracy of the actual prediction by collecting contact information of the actual trajectory, and if the accuracy of the actual prediction is lower than a first threshold, a set number of values needs to be increased to enable the neural network model to obtain contact information of a greater number of contacts for prediction, so as to ensure the accuracy of the prediction.
Based on the same inventive concept, fig. 4 shows a data processing apparatus 100 for a touch screen track, which is applied to a mobile terminal, where the mobile terminal includes a touch screen, and the data processing apparatus 100 for the touch screen track includes: an obtaining unit 110, configured to obtain contact information of a contact through which a track has currently passed; the prediction unit 120 is configured to predict a trajectory through a neural network model based on contact information of a contact through which the trajectory has currently passed, so as to obtain prediction information of the trajectory; and a display unit 130 for rendering the display image in advance based on the prediction information of the trajectory.
In one embodiment, the prediction information includes: predicted contact coordinates of each predicted contact through which the predicted trajectory of the predicted trajectory passes, and predicted contact time of each predicted contact.
In one embodiment, the display unit 130 is further configured to sequentially draw a display image at a corresponding position of each predicted touch point coordinate on the touch screen based on each predicted touch point time.
In one embodiment, the contact information includes any one or more of: contact point coordinates of the contact points, contact point time of the contact points, and pressure values of the contact points.
In an embodiment, the obtaining unit 110 is further configured to: after the track is finished, acquiring contact information of all contacts through which the track passes; as shown in fig. 5, the data processing apparatus 100 for touch screen trace further includes: and a parameter updating unit 140 for updating parameters of the neural network model based on the contact point information of all contact points of the trajectory and the prediction information of the trajectory.
In one embodiment, as shown in fig. 6, the data processing apparatus 100 for touch screen trace further includes: a judging unit 150, configured to judge whether the number of contacts through which the current trajectory passes reaches a set number; when the number of the contacts passed by the current track reaches the set number, the contact information of the contacts passed by the current track is executed through the prediction unit 120, and the track is predicted through the neural network model to obtain the prediction information of the track; when the number of the contacts that the current track has passed does not reach the set number, the acquisition of the contact information of the contacts that the track has passed is continuously performed by the acquisition unit 110.
In one embodiment, the set number is: the neural network model requires the minimum number of contacts when the confidence of obtaining the prediction information reaches a set threshold value based on the contact information of the contacts.
In an embodiment, the obtaining unit 110 is further configured to: after the track is finished, acquiring contact information of all contacts through which the track passes; as shown in fig. 7, the data processing apparatus 100 for touch screen trace further includes: the number adjusting unit 160 updates the set number based on the contact point information of all the contact points of the trajectory and the predicted information of the trajectory.
With respect to the data processing apparatus 100 of the touch screen trace in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
Based on the same inventive concept, the present disclosure further provides a mobile terminal, which includes a touch screen, and when the touch screen of the mobile terminal receives a touch pressure of a user, the processing is performed by the data processing method 10 of the touch screen trajectory according to any of the foregoing embodiments.
With regard to the mobile terminal in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
Fig. 8 is a schematic block diagram illustrating an apparatus of any of the previous embodiments in accordance with an exemplary embodiment. For example, the apparatus 300 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.
Referring to fig. 8, the apparatus 300 may include one or more of the following components: processing component 302, memory 304, power component 306, multimedia component 308, audio component 310, input/output (I/O) interface 312, sensor component 314, and communication component 316.
The processing component 302 generally controls overall operation of the device 300, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 302 may include one or more processors 320 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 302 can include one or more modules that facilitate interaction between the processing component 302 and other components. For example, the processing component 302 may include a multimedia module to facilitate interaction between the multimedia component 308 and the processing component 302.
The memory 304 is configured to store various types of data to support operations at the apparatus 300. Examples of such data include instructions for any application or method operating on device 300, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 304 may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.
The power supply component 306 provides power to the various components of the device 300. The power components 306 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power supplies for the apparatus 300.
The multimedia component 308 includes a screen that provides an output interface between the device 300 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 308 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the device 300 is in an operating mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
The audio component 310 is configured to output and/or input audio signals. For example, audio component 310 includes a Microphone (MIC) configured to receive external audio signals when apparatus 300 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 304 or transmitted via the communication component 316. In some embodiments, audio component 310 also includes a speaker for outputting audio signals.
The I/O interface 312 provides an interface between the processing component 302 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.
The sensor assembly 314 includes one or more sensors for providing various aspects of status assessment for the device 300. For example, sensor assembly 314 may detect an open/closed state of device 300, the relative positioning of components, such as a display and keypad of device 300, the change in position of device 300 or a component of device 300, the presence or absence of user contact with device 300, the orientation or acceleration/deceleration of device 300, and the change in temperature of device 300. Sensor assembly 314 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 314 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 314 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
The communication component 316 is configured to facilitate wired or wireless communication between the apparatus 300 and other devices. The device 300 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 316 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 316 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
In an exemplary embodiment, the apparatus 300 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.
In an exemplary embodiment, a computer-readable storage medium comprising instructions, such as the memory 304 comprising instructions, executable by the processor 320 of the apparatus 300 to perform the above-described method is also provided. For example, the computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
Fig. 9 is a block diagram illustrating an electronic device 400 according to an example embodiment. For example, the apparatus 400 may be provided as a server. Referring to fig. 9, apparatus 400 includes a processing component 422 that further includes one or more processors and memory resources, represented by memory 442, for storing instructions, such as application programs, that are executable by processing component 422. The application programs stored in memory 442 may include one or more modules that each correspond to a set of instructions. Further, the processing component 422 is configured to execute instructions to perform the above-described methods.
The apparatus 400 may also include a power component 426 configured to perform power management of the apparatus 300, a wired or wireless network interface 450 configured to connect the apparatus 400 to a network, and an input output (I/O) interface 458. The apparatus 400 may operate based on an operating system stored in the memory 442, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, or the like.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.
Claims (19)
1. A data processing method of a touch screen track is applied to a mobile terminal, the mobile terminal comprises a touch screen, and the data processing method of the touch screen track comprises the following steps:
acquiring contact information of a contact through which a track passes currently;
predicting the track through a neural network model based on the contact information of the contact through which the track passes at present to obtain the predicted information of the track;
based on the prediction information of the trajectory, a display image is drawn in advance.
2. The method of claim 1, wherein the predictive information comprises: and predicting the coordinates of the predicted contact point of each predicted contact point through which the predicted track of the track passes and the predicted contact time of each predicted contact point.
3. The method of claim 2, wherein the rendering a display image in advance based on the prediction information of the trajectory comprises:
and sequentially drawing a display image at the corresponding position of each predicted touch point coordinate of the touch screen based on each predicted touch point time.
4. The method of claim 1, wherein the touch point information comprises any one or more of: the contact point coordinates of the contact points, the contact point time of the contact points and the pressure values of the contact points.
5. The method for processing data of a touchscreen trajectory according to any of claims 1 to 4, further comprising:
after the track is finished, acquiring the contact information of all the contacts through which the track passes;
updating parameters of the neural network model based on the contact information for all of the contacts of the trajectory and the predicted information for the trajectory.
6. The method for processing data of a touchscreen trajectory according to any of claims 1 to 4, further comprising:
judging whether the number of the contacts passed by the current track reaches a set number;
if the number of the contacts passed by the current track reaches the set number, executing the contact information based on the contacts passed by the current track, and predicting the track through a neural network model to obtain predicted information of the track;
and if the number of the contacts passed by the current track does not reach the set number, continuing to execute the step of acquiring the contact information of the contacts passed by the current track.
7. The method for processing the data of the touch screen track according to claim 6, wherein the set number is: and the neural network model obtains the minimum number of the contact points when the confidence coefficient of the prediction information reaches a set threshold value based on the contact point information of the contact points.
8. The method of data processing of a touchscreen trajectory according to claim 7, further comprising:
after the track is finished, acquiring the contact information of all the contacts through which the track passes;
updating the set number based on the contact point information of all the contact points of the trajectory and the prediction information of the trajectory.
9. The data processing device of the touch screen track is applied to a mobile terminal, the mobile terminal comprises a touch screen, and the data processing device of the touch screen track comprises:
the acquisition unit is used for acquiring contact information of a contact through which a track passes currently;
the prediction unit is used for predicting the track through a neural network model based on the contact information of the contact through which the track passes at present to obtain the prediction information of the track;
a display unit for drawing a display image in advance based on the prediction information of the trajectory.
10. The data processing apparatus of a touchscreen trajectory according to claim 9, wherein the prediction information comprises: and predicting the coordinates of the predicted contact point of each predicted contact point through which the predicted track of the track passes and the predicted contact time of each predicted contact point.
11. The device for processing the data of the touch screen trajectory according to claim 10, wherein the display unit is further configured to sequentially draw a display image at a corresponding position of each of the predicted touch point coordinates of the touch screen based on each of the predicted touch point times.
12. The touch screen trajectory data processing device of claim 9, wherein the contact information includes any one or more of: the contact point coordinates of the contact points, the contact point time of the contact points and the pressure values of the contact points.
13. The data processing device of touch screen trajectory according to any of the claims 9 to 12,
the acquisition unit is further configured to: after the track is finished, acquiring the contact information of all the contacts through which the track passes;
the device further comprises: a parameter updating unit for updating parameters of the neural network model based on the contact point information of all the contact points of the trajectory and the prediction information of the trajectory.
14. The device for processing data of a touchscreen trajectory according to any of claims 9 to 12, further comprising:
the judging unit is used for judging whether the number of the contacts passed by the current track reaches a set number;
when the number of the contact points which are passed by the track at present reaches the set number, the contact point information based on the contact points which are passed by the track at present is executed through the prediction unit, and the track is predicted through a neural network model to obtain the prediction information of the track;
when the number of the contacts which the track has passed does not reach the set number, the acquisition unit continues to acquire the contact information of the contacts which the track has passed.
15. The device for processing data of a touch screen trajectory according to claim 14, wherein the set number is: and the neural network model obtains the minimum number of the contact points when the confidence coefficient of the prediction information reaches a set threshold value based on the contact point information of the contact points.
16. The data processing device of touch screen trajectory of claim 15,
the acquisition unit is further configured to: after the track is finished, acquiring the contact information of all the contacts through which the track passes;
the device further comprises: a number adjusting unit configured to update the set number based on the contact point information of all the contact points of the trajectory and the prediction information of the trajectory.
17. A mobile terminal, characterized in that the mobile terminal comprises a touch screen, and when the touch screen of the mobile terminal receives the touch pressure of a user, the data processing method of the touch screen track according to any one of claims 1-8 is used for processing.
18. An electronic device, comprising:
a memory to store instructions; and
a processor for invoking the memory-stored instructions to perform a method of data processing of a touchscreen trajectory according to any of claims 1-8.
19. A computer-readable storage medium storing instructions which, when executed by a processor, perform a method of data processing of a touchscreen trajectory according to any of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010125722.5A CN113311984A (en) | 2020-02-27 | 2020-02-27 | Touch screen track data processing method and device, mobile terminal and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010125722.5A CN113311984A (en) | 2020-02-27 | 2020-02-27 | Touch screen track data processing method and device, mobile terminal and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113311984A true CN113311984A (en) | 2021-08-27 |
Family
ID=77370483
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010125722.5A Pending CN113311984A (en) | 2020-02-27 | 2020-02-27 | Touch screen track data processing method and device, mobile terminal and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113311984A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114466006A (en) * | 2021-12-22 | 2022-05-10 | 天翼云科技有限公司 | Touch screen information sending and responding method and device |
| WO2024065252A1 (en) * | 2022-09-28 | 2024-04-04 | Boe Technology Group Co., Ltd. | Display apparatus, display method, and computer-program product |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102890576A (en) * | 2011-07-22 | 2013-01-23 | 宸鸿科技(厦门)有限公司 | Touch locus detection method and touch locus detection device of touch screen |
| CN102929433A (en) * | 2012-11-06 | 2013-02-13 | 山东大学 | Method for reducing dragging delay on embedded device through contact prediction |
| CN103076874A (en) * | 2011-10-26 | 2013-05-01 | 中国科学院声学研究所 | Method and system for improving high delay of computer-vision-based somatosensory input equipment |
| CN105353906A (en) * | 2015-10-21 | 2016-02-24 | 业成光电（深圳）有限公司 | Trajectory prediction system and trajectory prediction method |
| CN107562364A (en) * | 2017-09-19 | 2018-01-09 | 青岛海信电器股份有限公司 | The display methods and device of a kind of touch track |
| CN109891491A (en) * | 2016-10-28 | 2019-06-14 | 雷马克伯有限公司 | Interactive display |
| CN110751683A (en) * | 2019-10-28 | 2020-02-04 | 北京地平线机器人技术研发有限公司 | Trajectory prediction method and device, readable storage medium and electronic equipment |
-
2020
- 2020-02-27 CN CN202010125722.5A patent/CN113311984A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102890576A (en) * | 2011-07-22 | 2013-01-23 | 宸鸿科技(厦门)有限公司 | Touch locus detection method and touch locus detection device of touch screen |
| CN103076874A (en) * | 2011-10-26 | 2013-05-01 | 中国科学院声学研究所 | Method and system for improving high delay of computer-vision-based somatosensory input equipment |
| CN102929433A (en) * | 2012-11-06 | 2013-02-13 | 山东大学 | Method for reducing dragging delay on embedded device through contact prediction |
| CN105353906A (en) * | 2015-10-21 | 2016-02-24 | 业成光电（深圳）有限公司 | Trajectory prediction system and trajectory prediction method |
| CN109891491A (en) * | 2016-10-28 | 2019-06-14 | 雷马克伯有限公司 | Interactive display |
| CN107562364A (en) * | 2017-09-19 | 2018-01-09 | 青岛海信电器股份有限公司 | The display methods and device of a kind of touch track |
| CN110751683A (en) * | 2019-10-28 | 2020-02-04 | 北京地平线机器人技术研发有限公司 | Trajectory prediction method and device, readable storage medium and electronic equipment |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114466006A (en) * | 2021-12-22 | 2022-05-10 | 天翼云科技有限公司 | Touch screen information sending and responding method and device |
| CN114466006B (en) * | 2021-12-22 | 2024-01-02 | 天翼云科技有限公司 | Touch screen information sending and responding method and device |
| WO2024065252A1 (en) * | 2022-09-28 | 2024-04-04 | Boe Technology Group Co., Ltd. | Display apparatus, display method, and computer-program product |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107908351B (en) | Application interface display method and device and storage medium | |
| US20170344192A1 (en) | Method and device for playing live videos | |
| EP3182716A1 (en) | Method and device for video display | |
| CN107124508B (en) | Position adjusting method and device of suspension control, terminal and readable storage medium | |
| CN107656666B (en) | Mobile terminal and scrolling speed determination method | |
| CN107102772B (en) | Touch control method and device | |
| CN111880757A (en) | Screen projection method, screen projection device and storage medium | |
| CN107992257B (en) | Screen splitting method and device | |
| EP3239827B1 (en) | Method and apparatus for adjusting playing progress of media file | |
| US20180240069A1 (en) | Method and apparatus for updating information, and storage medium | |
| CN107656694B (en) | Display control method and device of user interface | |
| CN108829475B (en) | UI drawing method, device and storage medium | |
| CN106095300B (en) | Method and device for adjusting playing progress | |
| CN111552425A (en) | Method and device for controlling small window display and storage medium | |
| CN106775210B (en) | Wallpaper changing method and device | |
| CN111694631A (en) | Pendant display method and device | |
| CN111522498A (en) | Touch response method and device and storage medium | |
| CN111880681A (en) | Touch screen sampling rate adjusting method and device and computer storage medium | |
| CN113311984A (en) | Touch screen track data processing method and device, mobile terminal and electronic equipment | |
| CN106325712B (en) | Terminal display control method and device and terminal | |
| CN107734303B (en) | Video identification method and device | |
| CN108874450B (en) | Method and device for waking up voice assistant | |
| CN112650437B (en) | Cursor control method and device, electronic equipment and storage medium | |
| CN108804009B (en) | Gesture recognition method and device | |
| CN109407942B (en) | Model processing method and device, control client and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |