CN112596664A

CN112596664A - Data communication processing method and device, electronic equipment and touch display equipment

Info

Publication number: CN112596664A
Application number: CN202011636555.7A
Authority: CN
Inventors: 任健
Original assignee: Advanced New Technologies Co Ltd
Current assignee: Alibaba Group Holding Ltd; Advanced New Technologies Co Ltd
Priority date: 2016-08-19
Filing date: 2016-08-19
Publication date: 2021-04-02
Also published as: CN106909297B; CN106909297A

Abstract

The application provides a data communication processing method and device, electronic equipment and touch display equipment. The method comprises the following steps: monitoring a control instruction generated by a control action acting on the touch screen, and identifying a user control track; judging whether the user control track meets a preset trigger condition that the first application points to the second application; sending a communication establishment request of the first application when satisfied; after the second application receives the communication establishment request, judging whether the state of the second application is a multi-window mode in which an interface is displayed on a front desk of a touch screen; and when the judgment result is yes, establishing a data communication link between the first application and the second application. By utilizing the embodiments in the application, the user can establish a communication link for at least two applications displayed on the screen at the same time through simple gestures, so that the application interaction between different applications in the terminal equipment is fast and convenient, the user operation is reduced, and the user operation and use experience are improved.

Description

Data communication processing method and device, electronic equipment and touch display equipment

The present application is a divisional application of an invention patent application having an application number of 201610694486.2, an application date of 2016, 08 and 19 months, and an invention name of "a data communication processing method, an apparatus, an electronic device, and a touch display device".

Technical Field

The present application belongs to the technical field of data communication, and in particular, to a data communication processing method and apparatus, an electronic device, and a touch display device.

Background

With the rapid development of mobile communication technology, the application of various terminal devices such as mobile phones, tablets, self-service terminals, etc. brings great convenience to the daily life of people. For example, a user can chat with friends and colleagues in real time through IM software such as QQ, WeChat, nailing and the like, a browser can be used for inquiring data and reading news, commodities can be purchased through mobile phone terminal application, and the like, so that the daily life of people is greatly facilitated.

With the explosive growth of various APP applications, more and more applications are installed in terminal devices, especially in mobile terminals. Now, users often need to operate and interact among different applications. For example, many applications currently have a content sharing function, such as sharing NEWS into the communication application APP _ CHAT when a user reads the NEWS using the NEWS application APP _ NEWS, or sharing photos in a local gallery into a friend circle/life circle of social software, and the like. At present, the existing cross-application content sharing generally requires a user to execute a plurality of complex operations, and is not intuitive and quick. For example, when the user shares NEWS in the NEWS application APP _ NEWS into the communication application APP _ CHAT, the user generally needs to open a NEWS page of the NEWS application APP _ NEWS, find a sharing button in a page menu, select the sharing button (at this time, another prompt box is popped up to allow the user to select which application to share), and select the communication application APP _ CHAT. And then jumping to a window interface of the communication application APP _ CHAT, and selecting a target object (contact or group) shared in the communication application APP _ CHAT. The sharing function can be completed only by a plurality of steps operated by a user, sharing processes of different applications are often different, and even more sharing operations or sharing operations with an unclear indication are included, so that the user operation is complicated, and the user experience is reduced.

In the prior art, the function operation of the user for performing cross-application content sharing by using the terminal device is complicated, the operation is not intuitive enough, and the content cannot be rapidly sent to other applications. Moreover, the content sharing is unidirectional, and the user needs to repeat the complicated sharing operation process again after sharing the content of the application a with the application B, so that the operation and use experience is reduced.

Disclosure of Invention

The application aims to provide a data communication processing method and device, an electronic device and a touch display device, and the data communication processing method and device can enable at least two applications displayed on a screen to establish a communication link based on simple gesture operation of a user by utilizing the characteristic that the screen can simultaneously display a plurality of application windows in a multi-window mode, so that application interaction between different applications in a terminal device is fast and convenient, user operation is reduced, and user operation and use experience are improved.

In one aspect, an embodiment of the present specification provides a data communication processing method, where the method includes:

monitoring a control instruction generated by a control action acting on the touch screen, and identifying a user control track;

judging whether the user control track meets a preset trigger condition that a first application points to a second application; sending a communication establishment request of a first application when the preset trigger condition is met;

after the second application receives the communication establishment request, judging whether the state of the second application is a multi-window mode in which an interface is displayed on the front desk of the touch screen;

and when the judgment result is yes, establishing a data communication link between the first application and the second application.

In another aspect, an embodiment of the present specification provides a data communication processing method, where the method includes:

judging whether the user control track meets a preset trigger condition that a first service item points to a second service item; and when the judgment result is yes, processing the service processing request of the first service item by adopting the response mode of the second service item.

In another aspect, an embodiment of the present specification provides a data communication processing apparatus, including:

the monitoring module is used for monitoring a control instruction generated by a control action acting on the touch screen and identifying a user control track;

the track processing module is used for judging whether the user control track meets a preset trigger condition that the first application points to the second application; sending a communication establishment request of a first application when the preset trigger condition is met;

the multi-window judging module is used for judging whether the state of the second application is a multi-window mode in which an interface is displayed on the front desk of the touch screen or not after the second application receives the communication establishing request;

and the communication establishing module is used for establishing a data communication link between the first application and the second application when the result of the judgment of the multi-window judging module is positive.

the control command acquisition module is used for monitoring a control command generated by a control behavior on the touch screen and identifying a user control track;

the service processing module is used for judging whether the user control track meets a preset trigger condition that the first service item points to the second service item; and the server is further configured to process the service processing request of the first service item in a response manner of the second service item when the determination result is yes.

In another aspect, an embodiment of the present specification provides an electronic device, including:

the event monitoring unit is used for monitoring a control instruction generated by a control action acting on the touch screen and identifying a user control track; the user control track is also used for judging whether the user control track meets a preset trigger condition that the first application points to the second application;

the window state judging module is used for judging whether the state of the second application is a multi-window mode in which an interface is displayed on the foreground of the touch screen or not after the second application receives a communication establishment request of the first application;

the communication link establishing module is used for sending a communication establishing request of the first application when the user control track is judged to meet a preset trigger condition that the first application points to the second application; and the data communication link between the first application and the second application is established when the state of the second application is judged to be a multi-window mode in which an interface is displayed on the front desk of the touch screen.

In yet another aspect, embodiments of the present specification provide a touch display device, including at least a touch screen and a processor,

the touch screen is used for displaying interface information of an application window and sensing a control behavior acted on the touch screen, and sending a corresponding control instruction generated by the control behavior to the processor;

the processor is configured to identify a user control track according to the control instruction, and send a communication establishment request of a first application when the user control track is determined to meet a preset trigger condition that the first application points to a second application; the second application is further used for judging whether the state of the second application is a multi-window mode in which an interface is displayed on the front desk of the touch screen or not after receiving the communication establishment request; and the data communication link between the first application and the second application is established when the judgment result is yes.

In yet another aspect, embodiments of the present specification provide a touch display device, comprising at least a touch screen and a processor,

the processor is configured to determine whether the user manipulation trajectory satisfies a preset trigger condition for sending a first service item to a second service item; and the server is further configured to process the service processing request of the first service item in a response manner of the second service item when the determination result is yes.

According to the data communication processing method and device, the electronic equipment and the touch display equipment, the characteristic that a plurality of application windows can be displayed on a screen simultaneously in a multi-window mode of the terminal equipment can be utilized, and the control behavior of a user on the touch screen can be judged. When the fact that the user needs to perform information interaction between different applications is determined, communication between the applications is established, and the fact that the user can enable at least two applications displayed on a screen to establish a communication link through a simple and visual sliding gesture is achieved visually. The application can use the communication link established by the technical scheme provided by the application to perform business-level interaction with another application, and specific interaction modes/response contents can be designed by business developers according to different scenes, such as one-click sharing, one-click payment, one-click song listening and the like. Obviously, according to the embodiment of the application, the interactive operation between different applications such as user sharing and searching can be greatly simplified in more and more terminal devices supporting multiple applications such as PC terminals and Android/iOS system terminals, the user requirements are met, the user terminal control experience can be greatly improved through the interactive mode between the applications, and the use perception of user products is improved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without any creative effort.

Fig. 1 is a flowchart of a method according to an embodiment of a data communication processing method provided in the present application;

FIG. 2 is a schematic view of a scene for displaying two application window interfaces simultaneously in a same-screen multi-window mode according to the present application;

FIG. 3 is an application scenario of an embodiment of a window interface in which a user manipulation trajectory covers multiple applications according to the present application;

FIG. 4 is a schematic view of an application scenario of an embodiment of a window interface with a user manipulation trajectory covering multiple applications according to the present application;

fig. 5 is a schematic diagram of a predicted user operation behavior corresponding to a preset trigger condition set according to the present application;

FIG. 6 is a schematic view of an embodiment of a multi-window mode in which an interface according to the present application is displayed in a front stage of a touch screen;

FIG. 7 is a schematic view of another exemplary scenario in which an interface according to the present application is displayed in a front stage of a touch screen;

FIG. 8 is a schematic view of another exemplary scenario in which an interface according to the present application is displayed in a front stage of a touch screen;

FIG. 9 is a schematic diagram illustrating a scenario in which a user establishes cross-application communication through gesture sliding according to the present application;

FIG. 10 is a flow chart of a method of another embodiment of a method of data communication processing provided herein;

FIG. 11 is a schematic view of a scenario in which a user implements cross-application communication in a multi-window mode by using a clicked manipulation behavior according to the present application;

FIG. 12-1 is a schematic view of another scenario in which a user implements cross-application communication in a multi-window mode using a clicked manipulation behavior according to the present application;

FIG. 12-2 is a schematic diagram illustrating a scenario of implementing cross-application communication in a multi-window mode by user sliding according to the present application;

fig. 13 is a schematic flow chart illustrating a scenario of an embodiment of a data communication processing method according to the present application;

FIG. 14 is a method flow diagram of another embodiment of a data communication processing method provided herein;

fig. 15 is a schematic diagram of an application scenario of another embodiment of a data communication processing method provided in the present application;

fig. 16 is a schematic block diagram of an embodiment of a data communication processing apparatus provided in the present application;

fig. 17 is a schematic block diagram of another embodiment of a data communication processing apparatus provided in the present application;

FIG. 18 is a schematic diagram illustrating the structure of one embodiment of a touch display device according to the present application;

FIG. 19 is a schematic diagram of an embodiment of a touch display device according to the present application.

Detailed Description

In order to make those skilled in the art better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Fig. 1 is a flowchart of a method of an embodiment of a data communication processing method according to the present application. Although the present application provides method operational steps or apparatus configurations as illustrated in the following examples or figures, more or fewer operational steps or modular units may be included in the methods or apparatus based on conventional or non-inventive efforts. In the case of steps or structures which do not logically have the necessary cause and effect relationship, the execution sequence of the steps or the module structure of the apparatus is not limited to the execution sequence or the module structure shown in the embodiment or the drawings of the present application. When the described method or module structure is applied to a practical device or an end product, the method or module structure according to the embodiment or the figures may be executed sequentially or executed in parallel (for example, in the environment of parallel processors or multi-thread processing, or even in the environment of distributed processing).

A specific application scenario is shown in fig. 1, and an embodiment of a data communication processing method provided by the present application may include:

s1: and monitoring a control command generated by a control action acting on the touch screen, and identifying a user control track.

In the application scenario of this embodiment, the terminal device may support a multi-window mode. For example, a screen in a multi-window mode of an Android version may simultaneously display two application window interfaces, such as a search application and a news application, as shown in fig. 2, and fig. 2 is a scene diagram illustrating that two application window interfaces are simultaneously displayed in a same-screen multi-window mode according to the present application. Of course, other embodiments of the present application do not exclude an implementation in which more than two application window interfaces are simultaneously displayed on the same screen, for example, some tablet computers may currently support a multi-window mode in which four application window interfaces are simultaneously displayed on the same screen. Generally, the terminal device may include a mobile communication terminal, a tablet computer, a PC, a handheld device, a PDA, a self-service terminal, a server, a business system, and the like. The terminal device may generally include a touch screen (also referred to as a touch screen or a touch panel), and the touch screen in the embodiment of the present application includes a device capable of receiving an input signal and displaying information, such as an inductive liquid crystal display device capable of receiving an input signal, such as a stylus, a touch of a human body, and light intensity induction. Generally, the touch screen is provided with a set of transparent absolute positioning system, so that touch, sliding and the like acting on the touch screen can be detected, and positioning can be realized according to coordinates.

The user can perform a series of control actions on the application on the touch screen of the terminal device, such as clicking, sliding, long-time pressing, or multi-point touch. The control behaviors acted on the touch screen can generate corresponding control instructions, so that the processing unit can recognize the control behaviors of the user and then respond correspondingly. In the embodiment of the application, a control instruction generated by a control behavior of a user acting on the touch screen can be monitored, for example, a title click, a page slide and the like performed when the user uses a news application in a terminal device, and the terminal device can monitor the control instruction generated by the control of the user and then recognize a corresponding user control track. Generally, when a user clicks or slides on the touch screen, a window interface of an application, such as a news reading application, an instant messaging application, a picture viewing application, and a payment application, is displayed in front of the touch screen. In an embodiment, it may also be understood that a business window interface of an application operated by a user at the current time is displayed in the foreground of the touch screen, and in another embodiment, a thumbnail window interface or a zoom window interface of the application is also displayed in the foreground of the touch screen.

In an application scenario of the embodiment shown in fig. 2, the manipulation behavior acting on the touch screen may include a manipulation behavior of directly sliding on the touch screen through a limb of a user as shown in fig. 2. In other embodiments of the present application, the touch behavior may also include a click, a slide, and the like performed on the touch screen by a user using a touch device, such as a stylus, a laser light, or other auxiliary tools. Even in other embodiments, the application does not exclude that the user does not directly contact the touch screen, but utilizes photoelectric induction, magnetic induction and the like to identify the control behavior of the user acting on the touch screen indirectly in the space. Therefore, in an embodiment of the data communication processing method described in this application, the manipulation behavior acting on the touch screen includes at least one of the following:

s101: a manipulation behavior of touching the touch screen through a user limb;

s102: a control action of touching the touch screen through a touch device;

s103: and the control action of the separated space on the touch screen is sensed and recognized through limbs.

In a specific implementation process, the user can touch the touch screen through fingers, even other limbs such as a nose, toes and ears to perform operations such as clicking, sliding, long pressing and the like. Alternatively, the user may use a touch pen (e.g., a capacitive pen), a laser pen (e.g., emitting light to the touch screen for sliding), a magnetic or optical pen, or other devices to contact the touch screen to operate the application in the terminal device. Of course, the manipulation behavior described herein that acts on the touch screen may also include, in some embodiments, manipulation behaviors that are applied by the user on the screen across the space, such as recognized limb sensing actions of the user. Specifically, for example, a user makes a sliding gesture on the touch screen at a certain distance, although a finger of the user does not directly touch the touch screen at this time, the terminal device in the application scenario of the data communication method provided by the embodiment of the present application can still recognize that the user performs a control action of sliding for a certain distance, and translates the control action into a user control track.

It should be noted that the spaced connection described in the above embodiment may include that an indirect medium between an executor of the manipulation behavior (such as a finger or a palm) and the touch screen is air, but the spaced connection described in this embodiment mainly refers to a manipulation action performed on an application of the terminal device by the executor of the manipulation behavior without directly contacting the touch screen. Therefore, the space described in this embodiment is not a special medium in the case of air, and other media, such as water, oil, and the like, or even vacuum may be between the performer of the manipulation behavior and the touch screen in some other implementation scenarios.

In addition, the manipulation behavior acting on the touch screen includes any one or more combination of the above-mentioned manners, and of course, other manners of manipulation behaviors may be included according to the development of information technology. Other embodiments do not exclude embodiments that use one or more operation behaviors, such as a user holding a position area of an application on the touch screen with one hand and sliding the other hand to one side of the touch screen with a distance in a limb sensing manner, and similar implementation scenarios still belong to the operation behaviors applied to the touch screen.

In an embodiment of the data communication processing method, an operation instruction generated by a manipulation behavior directly or indirectly acting on the touch screen may be monitored, and then a corresponding user manipulation track may be identified. The control track can be continuous, discontinuous track within a certain time period, or discontinuous track with multiple points, multiple sections and combination.

S2: judging whether the user control track meets a preset trigger condition that a first application points to a second application; and sending a communication establishment request of the first application when the preset trigger condition is met.

As described in the foregoing, in an embodiment of the present application, the terminal device may monitor a manipulation behavior directly or indirectly acted on a touch of the terminal device by a user, and may obtain a corresponding manipulation instruction. The terminal device may support a same-screen multi-window mode, and specifically, an application interface of two or more applications may be simultaneously displayed on a mobile phone or a tablet computer, as shown in fig. 2. In the embodiment of the application, a preset trigger condition that the first application points to the second application may be preset, and specifically, the preset trigger condition may include various control gestures that a user acts on the touch screen, for example, a certain page element in the application is pressed and then the user slides for a distance to another application. At this time, whether a manipulation event that the first application points to the second application is triggered or not may be determined according to a manipulation behavior of the user on the touch screen. The preset trigger condition that the first application points to the second application can be preset according to a scene or design requirement, so that the terminal device or the application can identify whether the control behavior of the user meets the set control event.

In one embodiment, an application that is currently operated (or currently activated front end window) by a user at the current time may be defined as a first application, and an application that needs to perform information interaction with the first application may be defined as a second application, for example, an application that needs to share or search a manipulation behavior executed by the user at the first application. Specific examples include that a news application displayed at the front end of the mobile phone screen at the current time T1 for the user to read is defined as a first application, or an application corresponding to an application interface operated by the user at the current time T1, such as clicking, sliding, long pressing, and the like, is defined as the first application. In this scenario, the user needs to send the content in the information application a to the search application B displayed on the same screen at this time, and the corresponding search application B may be the second application defined in this embodiment. If the user operates the search application B at another time T2, it is necessary to send a certain search result R of the search application B to the news application a displayed on the same screen at this time, so as to view the search result R of the news application a in the search application B, where the search application B is the first application defined in this embodiment and the news application a is the corresponding second application.

Of course, in some other embodiments of the present application, the second application corresponding to the first application may not be limited to one application. As shown in fig. 3, a user T1 starts from a certain starting point of a window interface in application a, slides continuously through application C, and stops at the window interface of application B at time T2, so as to obtain a sliding trajectory S1 ═ a-C-B, where fig. 3 is an application scenario of an embodiment in which a user manipulation trajectory covers window interfaces of multiple applications. The sliding track of the user can be irregular semicircle, arc or irregular line segment, etc. Then application a at time T1 may be defined as the first application, and both application B and application C traversed by the S1 trace may be the corresponding second application when application a is the first application. Therefore, in another embodiment of the data communication processing method described in the present application,

s201: the first application comprises an application corresponding to a window interface where a starting point of a user control track is located in the user control track formed by continuous sliding of the touch screen;

correspondingly, the second application comprises an associated application except the first application, and the associated application comprises an application of a window interface covered by the user manipulation track displayed in the foreground of the touch screen.

Generally, the touch screen is provided with a corresponding coordinate system, usually an absolute coordinate system, so that each pixel point on the touch screen has corresponding coordinate information. Various application interfaces, windows and other elements displayed on the touch screen also correspond to corresponding coordinate information, and the user control track data formed by continuous sliding can also contain the coordinate information of each pixel point in the track. Therefore, in an embodiment of the application, the application currently displayed on the foreground of the touch screen covered by the continuous sliding track can be determined according to the continuous sliding track of the user and the coordinate information of the application window interface. It should be noted that, the associated application may not generally include the first application corresponding to the sliding track starting point, such as a scene where the sliding track starting point and the sliding track ending point are not in the same application window interface. However, in some application scenarios, the user controls such that the start point and the end point of the sliding track are in an application interface of the same application, for example, the sliding track S2 is a-B-C-a, as shown in fig. 4, where fig. 4 is another application scenario of the application where the user controls the track to cover a window interface of multiple applications. Then it may be determined that the application where the sliding track terminal is located does not include the defined second application, and the case where the second application is the same as the first application is excluded, that is, the second application includes an implementation of an associated application other than the first application. In the application scenario shown in fig. 4, application a may be used as the first application, and application B and application C may be used as the second application in this embodiment. Of course, this application does not exclude the case where the first application and the second application are the same application in some embodiments, and some corresponding measures may be set according to the design for processing.

In an embodiment of the present application, the data communication processing method may preset a series of control events that the first application points to the second application. And meanwhile, preset trigger conditions corresponding to the control events can be preset. When the operation track of the user meets the preset trigger condition, a control event that the first application points to the second application can be triggered. Such as by sliding a finger in one application a's interface to another application a's interface. Or clicking three times within 500 milliseconds of the interface of the application a and then clicking once (or multiple times, such as three times) on the interface of the other application B determines to trigger the manipulation event of the first application a pointing to the second application B. Of course, the preset trigger condition may also include other conditions set by the predicted user operation behavior, for example, fig. 5 is a schematic diagram of a predicted user operation behavior corresponding to the preset trigger condition set by the present application, where a user draws a circle on a touch screen by a single finger and "throws" an element of the application a (e.g., a News title of News _ 2) into an opposite application B displayed in parallel on the same screen, that is, it may indicate that communication between the first application a and the second application B needs to be established, so that the second application B processes the element of the first application that is "thrown" into the opposite application B. One or more specific preset trigger conditions for the first application to point to the second application can be defined according to user experience or design requirements, and whether a user control track of a user meets preset trigger conditions for the first application to point to the second application or not can be identified and determined when a user control instruction of the user is monitored. If the user operation can be monitored by registering and activating the monitor in the Android system, the processing unit judges whether the sliding track of the user meets a preset trigger condition that the first application points to the second application.

In an embodiment of the application, if it is determined that a manipulation event that the first application points to the second application is triggered (for example, a user slides two fingers across applications) according to the user manipulation trajectory, the processor may determine that the user performs a sliding behavior that the first application points to the second application on the touch screen, and may indicate that the user needs to send the content/element of the first application to the second application for processing, such as sharing, querying, data copying/migrating, or the like. At this time, the communication establishment request of the first application may be sent, for example, in a manner of broadcast message, or directly sent to the second application in a point-to-point manner.

S3: and after the second application receives the communication establishment request, judging whether the state of the second application is a multi-window mode in which an interface is displayed on the front desk of the touch screen.

After the second application receives the communication establishment request sent by the first application, whether the second application window interface is in a multi-window mode or not can be judged, and the second application window interface is displayed on the foreground window of the touch screen. The main body of the specific execution judgment logic may be that the second application judges its own current window state, or may be processed by the same processing system (like a processing unit in a mobile phone terminal) where the first application and the second application are located together, or may be executed by a third party judgment logic.

Generally, the window mode may include, for example, that the window interfaces of the first application and the second application are in the same touch screen in a multi-window mode, for example, a current Android version supports that the window interfaces of two applications are simultaneously displayed in a display screen of a mobile phone terminal, or a certain tablet computer supports that the window interfaces of at most four applications are simultaneously displayed on the same touch screen. Fig. 6 is a scene diagram of an embodiment of a multi-window mode in which an interface is displayed in a front stage of a touch screen according to the present application, and both the applications A, B1 and B2 in fig. 6 are applications in the same processing server 1. However, the multi-window mode described in this application does not exclude an implementation in which multiple application windows are foreground-displayed on different touch screens, for example, the first application and the second application may belong to the same processing system and be displayed on different touch screen terminals, as in a multi-screen display of a PC host. The same processing system may be a processing unit, a module, a device including one or more logic units, or a processing system including multiple servers, or may be the same distributed system. In this case, the window interfaces of the first application and the second application may be on the same touch screen, or may be displayed on different touch screens separately or partially in a multi-screen display manner. Of course, the first application and the second application of the multi-window multi-screen display may belong to different processing systems. Fig. 7 is a schematic view of another exemplary scenario in which an interface according to the present application is displayed in a foreground of a touch screen, and as shown in fig. 7, a first application a is an application in a processing server 1, and second applications B1 and B2 are applications in a processing server 2. Or in another implementation scenario, the first application A, B1 is an application in processing server 1 and the second application B2 is an application in processing server 2. Therefore, in another embodiment of the data communication processing method according to the present application, the determining whether the state of the second application is in a multi-window mode in which an interface is displayed in a foreground of the touch screen may include:

s301: and judging whether the window interface of the second application and the window interface of the first application are simultaneously foreground-displayed in the same touch screen.

If the window interfaces of the first application and the second application are simultaneously displayed in the foreground of the touch screen at the current moment, whether the state of the second application is in a multi-window mode in which the interface is displayed in the foreground of the touch screen can be determined. Generally, when a first application is controlled, a window interface of the first application is usually displayed in the foreground of the touch screen, so that it can be determined whether a second window is also displayed in the foreground of the touch screen, and if so, the current multi-window mode in which the window interfaces of the first application and the second application are simultaneously displayed in the foreground of the touch screen can be represented.

Of course, according to the foregoing, in another embodiment of the method described in the present application, the state of the second application is determined to be a multi-window mode in which the interface is displayed in the foreground of the touch screen when at least one of the following conditions is satisfied:

s302: the window interface of the second application and the window interface of the first application are simultaneously and respectively displayed in different touch screens in a foreground mode;

s303: in the first application and the second application which belong to the same processing system, part of the window interface of at least one application is displayed in different touch screens in the foreground at the current moment.

The specific judgment implementation mode can select a corresponding processing mode according to different data communication application environments/systems. For example, in some terminal devices using the Android system version, whether a current Application is in a multi-window mode or not and whether the current Application is displayed in a foreground on a screen may be determined through an Application Programming Interface (API) provided by the Android system. In one embodiment, the first application and the second application may belong to the same processing system, and in other embodiments, the first application and the second application may belong to different processing systems, as shown in fig. 6 and 7. Therefore, the first application and the second application in the above embodiments adopt any one of the following implementation manners:

the first application and the second application belong to the same processing system;

at least one of the first application and the second application belongs to different processing systems.

The multi-window mode generally means that window interfaces of at least two applications in the same processing system are foreground-displayed on a touch screen, for example, a search application and a news application are simultaneously displayed side by side on a display screen of a mobile phone. Of course, in an implementation manner, the multiple windows described in the embodiment of the present application may also be a service window interface of an application operated by a user at the current time, and in another embodiment, the multiple windows may also include a thumbnail window interface or a zoom window interface of the application. A specific example is shown in fig. 8, where a first application a currently activated by a user is a larger window interface, and at the same time, an application B, an application C, and an application D that are displayed in a multi-window foreground are on one side of the window interface of the first application a, and since window interfaces of other applications are still in an inactivated state when the first application a is manipulated, the application B, the application C, and the application D may be displayed on the rightmost side of the touch screen with a smaller zoom image.

S4: and when the judgment result is yes, establishing a data communication link between the first application and the second application.

In this embodiment of the application, after receiving the communication establishment request, the second application may determine whether a state of the second application is a multi-window mode in which an interface is displayed on the front stage of the touch screen. And when the judgment result is yes, the window interface of the second application and the window interface of the first application can be represented as a multi-window mode scene displayed in front of the touch screen at the same time. At this time, the user manipulation trajectory satisfies a preset trigger condition that the first application points to the second application, and may indicate that the user desires to implement a certain operation requirement between the first application and the second application, for example, in a multi-window mode in which the window interfaces of the first application and the second application are displayed side by side on the touch screen, the user needs to share a certain news in the first application to a friend circle of the second application, or send a certain picture element in the first application to a search box of the second application for commodity search, or copy/transfer data selected in the first application to the second application, and so on. Therefore, in an embodiment of the present application, when it is determined that a manipulation behavior (e.g., gesture sliding) of a user on a touch screen triggers a manipulation event that a first application points to a second application in a multi-window mode displayed in at least two application foreground, a data communication link between the first application and the second application is established, and based on the data communication link, the first application may implement service-level data communication with the second application.

In this embodiment, the established communication link is at least a communication connection from the first application to the second application, and the first application may at least transmit data to the second application. In a specific implementation of the embodiment of the present application, the communication link may be a bidirectional communication link, that is, in an implementation, the second application may also transmit data to the first application through the established communication link.

According to the method and the device for establishing the data communication link, the characteristic that the screen can simultaneously display two or more application window interfaces in the multi-window mode is utilized, and the two or more applications simultaneously displayed on the screen can establish the data communication link by identifying the control track of the user on the touch screen, for example, two fingers slide to a distance from the first application to the second application. Once the data communication link between the applications is established, the interaction on the service level between different applications, such as one-touch/fast cross-application sharing, payment, inquiry, data transfer, etc., can be realized. Therefore, the user can intuitively realize business interaction between different applications such as rapid sharing, query and the like through one gesture. Obviously, in more and more terminal devices supporting multiple application windows, such as a PC terminal, an Android/iOS system terminal, and the like, operations such as sharing, searching, and the like performed by a user between different applications can be greatly simplified. And the interaction mode among the applications can also improve the operation and use experience of the user and improve the use perception of the user product.

The user manipulation event described in the above embodiment may include a judgment logic that the first application points to the second application in a plurality of preset multi-window modes. Generally, in most application scenarios, when a user uses a terminal device provided with a touch screen, such as a currently mainstream smart phone, a tablet computer, a self-service terminal, and the like, the user usually uses a finger, a palm, and other user limbs to slide on the touch screen to implement operation on an application, and certainly, the operation may also include using a stylus or other auxiliary devices. Therefore, in an embodiment of the data communication processing method described in the present application, a preset trigger condition based on some expected sliding behaviors of the user on the touch screen may be preset, and therefore in an embodiment of the present application, the preset trigger condition that the first application points to the second application includes:

s111: the identified user manipulation trajectory is a sliding row that slides from the window interface of the first application to the window interface of the second application.

The sliding action may include a sliding action performed by a user's limb, and may also include a sliding action performed using an auxiliary input device like a stylus. The user's limb may comprise a finger, although in other embodiments the user's limb may comprise a human limb or organ such as a head, eye, neck, elbow, arm, body, crotch, foot, etc. In this embodiment, the sliding behavior of the user's limb touching the touch screen may include a continuous sliding operation behavior of the user on the touch screen, and the user may slide to one side of the touch screen for a certain distance after touching the touch screen with a finger, or slide to the other side of the touch screen after circling on the touch screen, or slide a certain curve, and the like.

In an embodiment of the data communication processing method, whether a manipulation behavior of a user points to another application or not may be identified according to information in a manipulation instruction generated by an acquired user manipulation. Specifically, in an embodiment of the data communication processing method, when a user performs a sliding operation on the touch screen, the determining whether the user manipulation trajectory meets a preset trigger condition for triggering the first application to point to the second application may include:

s211: acquiring sliding record information in the control instruction, wherein the sliding record information comprises recorded data information generated when a user performs a sliding action on a touch screen through at least one limb of the user;

s212: calculating at least one behavior index data at least comprising sliding direction, sliding distance, sliding time and sliding speed according to the recorded information;

s213: and comparing the behavior index data with a corresponding preset trigger threshold, and determining whether the control track meets the preset trigger condition according to the comparison result.

Generally, in some application scenarios, if a user needs to share content of a first application into a second application, the user may first touch a window interface of the first application with a finger, and then slide on the touch screen toward a location where the second application is located. When the user slides on the touch screen, the processing system can record information generated by the sliding, such as coordinate information, time information and the like, and some behavior index data of the user can be calculated based on the record information. These behavior index data may be used as a basis for determining whether to trigger the first application to point to the second application. According to the method and the device for identifying the second application, the second application which needs to be communicated with the first application and is displayed on the same-screen multi-window foreground can be identified according to the judgment of the behavior index data including the sliding direction and the like. A specific example application scenario is shown in fig. 9, and fig. 9 is a schematic view of a scenario in which a user establishes cross-application communication through gesture sliding according to the present application. The window interfaces of two applications currently serving as terminal equipment are displayed in the touch screen transversely side by side at the same time, and when a user touches the touch screen with two fingers, the touch time T1 and the coordinates C1 of the two-finger touch are recorded. Then the two fingers of the user slide on the touch screen for a certain distance, and after the sliding of the user is completed, the touch time T2 when the sliding is completed and the coordinate C2 of the two-finger touch are recorded. And then calculating whether the time interval of the user sliding is less than a threshold value 3000ms according to T1 and T2, and if so, the sliding behavior is effective. In addition, the sliding direction C1 → C2 of the user is calculated as the westernness direction according to the coordinates C1 and C2, and the sliding distance is 300 pixels and exceeds the minimum effective distance L, which is 20 pixels, and it can be determined that the sliding behavior of the user is directed to another application displayed in the touch screen side by side in the foreground. Therefore, it can be determined that the manipulation behavior of the user triggers a manipulation event preset to point the first application to the second application according to the sliding time, the sliding distance and the sliding direction of the user sliding operation. Of course, in other embodiments, the control instruction may further include time information when the touch screen is touched, and the behavior index data may also include sliding time, sliding distance, sliding speed, and the like, so that the trigger condition may also set a plurality of logic conditions, for example, it is determined through the sliding trajectory that the sliding direction is the sliding direction from the first application to the second application, and the sliding speed is greater than a certain pixel value. The setting can be specifically carried out according to the user experience and the preset control behavior which is preset to judge that the user needs to establish communication among different applications. The time interval, the minimum effective distance and the like can be set in advance according to design needs or expected user operation behaviors.

According to the implementation manner of the embodiment, a user can intuitively establish a communication link between two or more applications simultaneously displayed on a screen through one gesture, and a first application can quickly realize business-level interaction with another application by using the communication link established by the technical scheme provided by the embodiment. Fig. 10 is a flowchart of a method of another embodiment of a data communication processing method provided in the present application, and as shown in fig. 10, in a further embodiment of the data communication processing method of the present application, after the data communication link between the first application and the second application is established, the method may further include:

s5: and the first application and the second application perform service-level message interaction processing based on the established data communication link.

The communication link is at least a communication connection from the first application to the second application, and the communication link may be a bidirectional communication link when the embodiment of the present application is specifically implemented. After the communication link is established, the first application may send a business-level processing message, such as shared content or one-touch payment, to the second application based on the data communication link. The information interaction of multi-window modes such as cross-application sharing and query is quickly realized based on the user sliding gesture, the operation is more visual, the user can use the information conveniently, and the user experience is effectively improved. The second application responding to the service level processing message in this embodiment may include processing or rejecting processing of the service level processing message, or receiving/processing a processing result returned to the first application response after the service level processing message is processed, and the like. The response may include feedback or no feedback to the first application response message.

Of course, in other embodiments, the user may also use a click manipulation behavior to establish communication between the cross-applications in the multi-window mode. Or by a combination of clicking, sliding, long pressing, etc. In another embodiment of the data communication processing method, the control habit of some users during cross-application operation can be met and facilitated, and the specific preset trigger condition for the first application to point to the second application may include at least one of the following:

s221: the identified user control track is that in a preset effective period Ts, the number of clicks on a window interface of a first application meets a first requirement M, the number of clicks on a window interface of a second application meets a second requirement, M, N is the preset number of clicks, Ts is more than 0 second, M is more than or equal to 1, and N is more than or equal to 1;

s222: the identified user control track is that a preset pattern is slid in a window interface of the first application, and the recognized user control track slides at least a first threshold distance from the position of the preset pattern to the position of a second application window interface;

s223: the identified user operation track is that Ts in a preset effective period, the number of times of clicking on a window interface of a first application meets a first requirement K, the window interface of the first application slides to the position direction of a window interface of a second application for at least a second threshold distance, K is the preset number of times of clicking, Ts is the effective time of the user operation track, Ts is more than 0 second, and K is more than or equal to 1;

s224: the identified user control track is that within a preset effective period Ts, the time is longer than T milliseconds in a window interface of the first application, and at least a third threshold distance slides from the window interface of the first application to the position direction of the window interface of the second application, wherein Ts is larger than 0, and T is larger than 0 second.

Fig. 11 is a scene diagram illustrating a scenario in which a user implements cross-application communication in a multi-window mode by using a click manipulation behavior according to the present application. As shown in fig. 11, if the pre-active period Ts is 5 seconds, M is 3, and N is 1. If the user determines that the user clicks one picture element P1 three times in 1000 milliseconds in the window interface of the news application a, and then clicks another shopping application B1500 milliseconds later. The user's click action on the touch screen within 2500 ms this time meets the preset trigger condition that the first application points to the second application, so that the user-selected picture element P1 in the first application can be input into the second shopping application B at this time. Picture element P1 then appears in the search bar of shopping application B. In the embodiment of S222, one or more preset patterns may be preset, and may include a regular pattern or an irregular pattern, for example, the preset pattern may be a "circle", "triangle", "Z" shape, and the like. If it is recognized that the user draws a circle in the first application according to the user manipulation trajectory, and then slides to the second application by a distance of 500 pixels, which is greater than the first threshold distance of 300 pixels, the preset trigger condition described in S222 may be satisfied. Of course, the preset trigger condition may be set in a manner of combining the click or long press (which may be customized to set the long press time) with the slide described in S223 or S224, and other variations and combinations substantially similar to the above embodiments.

The preset trigger condition in the embodiment provides a plurality of control modes for users to conveniently, visually and quickly realize information interaction among different applications, so that user operation can be simplified, and user control experience is improved. In addition, the sliding from the window interface of the first application to the position direction of the window interface of the second application may include sliding directly acting on the touch screen through a user limb, sliding acting on the touch screen through a touch device, or sliding executed in a space by a user gesture. The first threshold, the second threshold, the third threshold, and the like may be set according to different user terminals, application interface scenes, and the like.

As described above, in an embodiment of the present application, a specific manner of sending the communication establishment request of the first application may include, for example, sending a broadcast message, or sending the communication establishment request directly to the second application in a peer-to-peer manner, where a communication link is established between the first application and the second application. For example, when the application a sends a request for establishing communication through a broadcast message, the application a starts to monitor the port, and the application B receiving the request can directly establish a connection with the application a. In another embodiment, the first application may communicate the setup request in a broadcast message manner, so that all applications receiving the broadcast message may return a response in response to the broadcast message of the first application. At this time, the first application a may autonomously select which application needs to be connected, and the judgment basis of the specific connection object may be set by a developer according to different service scenarios. Therefore, in the data communication processing method provided by the application,

s231: the sending of the communication establishment request of the first application comprises sending the communication establishment request of the first application in a message broadcasting mode;

accordingly, the establishing the data communication link between the first application and the second application may include:

after the second application receives the communication establishment request, establishing a communication link of the second application to the first application, and returning a response message to the first application;

and establishing a point-to-point communication link from the first application to the second application after the first application receives the response message of the second application.

The communication establishment request may include the first application identification, and the second application may return a response message to the first application based on the first application identification. Correspondingly, the response message may include a second application identifier of the second application, and the first application may establish a communication connection with the second application based on the second application identifier. The method for establishing communication between different applications displayed on the foreground can adopt the first application to broadcast and send a communication establishment message, one or more applications receiving the broadcast message can feed back a response message to the first application, and then the first application can actively select which application needs to establish a communication link according to the received response messages of a plurality of responders. The method for actively selecting the communication link to establish the object by the first application is particularly suitable for at least three multi-window mode application scenes with foreground display in a touch screen.

Of course, the way that all applications receiving the first application broadcast message establish communication links with the first application may also be applicable to a multi-window mode application scenario in which at least three foreground displays are in the touch screen. As shown in fig. 12-1, fig. 12-1 is a schematic view of a scenario of implementing cross-application communication in a multi-window mode by user sliding according to the present application, where a user slides a search result to the right side for a distance by a gesture in a first application, and then sends a broadcast message of a communication establishment request. And after receiving the broadcast message, the instant messaging applications IM _1, IM _2 and IM _3 displayed in the foreground can directly establish two-way communication with the search application B. Then, the search application B may send corresponding service-level processing data (a certain search result RE _1) to the instant messaging applications IM _1, IM _2, and IM _3, and the instant messaging applications IM _1, IM _2, and IM _3 may respectively perform corresponding response processing. When the first application selects which responder to establish the communication link, the instant communication applications IM _1, IM _2 and IM _3 all return response messages to the search application after receiving the broadcast message, and the search application B can directly face the IM _2 according to the sliding direction and further select to establish the communication link with the IM _ 2. The search application B may send corresponding service-level processing data (a certain search result RE _1) to the instant messaging application IM _2, and the instant messaging application IM _2 performs corresponding response processing, as shown in fig. 12-2, where fig. 12-2 is a scene diagram illustrating that cross-application communication is implemented in a multi-window mode through user sliding according to the present application.

The following describes a process implemented by applying the data communication processing method to an application scenario of a specific example. Fig. 13 is a schematic implementation flow diagram of a scenario of an embodiment of a data communication processing method according to the present application. In a specific example application scenario, as shown in fig. 13, in a mobile phone terminal of an Android system, an application a may be agreed in advance as an initiator for establishing communication, and an application B may be agreed as a receiver.

The broadcast receiver in fig. 13 is broadcastreever, which is a one-to-many information dissemination mechanism between applications provided by the Android system. Here, the actions (executors or execution processes) of two broadcast requests and responses may be defined as com. Application a, application B need to register both broadcast receivers with the system. In the Android system, it may be configured that the application a and the application B register broadcastreever (Broadcast receiver) after starting, so that the Android system calls back the pre-registered broadcastreever to notify the application when receiving the Broadcast. For example, one common registration method is static registration, which generally refers to defining a receiver within the application of android manifest.

When the user touches the touch screen on the mobile phone with two fingers, the time T1 and the coordinate C1 of the two-finger touch are recorded at the moment, and the time and coordinate data can be recorded all the time during the sliding process. After the sliding is finished, judging whether the sliding time interval T of the user is less than a threshold value 1500ms according to the recorded time T2 when the sliding is finished; and averaging the coordinates of the two fingers at the beginning and the end, and calculating whether the sliding straight-line distance L exceeds a threshold sliding distance which is L300 pixels through the beginning and the end coordinates. And then, judging that the current application A is in a multi-window mode through an API (application programming interface) provided by an Android system, and if the judgment results meet the triggering requirements of the sliding time interval and the sliding distance, determining that the sliding action of the user on the touch screen triggers a control event pointing to another application.

At this time, in the Android system of the mobile phone, the application a may send a request broadcast message carrying its own packet name. The package name is a package and is a unique identifier of an application in the Android system. Each application in the Android system must have a unique package, and the package can be used to distinguish different applications in product development design, for example, the package of application a may be com.

When application B receives the request broadcast, it can first decide whether to respond to the request from application a. If the system does not process the application of the emergency event or the exclusive process or is in other busy states, etc., the request of the application A can be approved, and a one-way communication link from B to A can be established. Then, whether the current application B is in a multi-window mode or not and whether the current application B is displayed on a screen foreground or not can be judged through an API provided by the Android.

After the judgment is passed, the application B randomly monitors an available network Socket port, such as 8080, sends a response broadcast according to the received request of the application, and carries the own packet name and the monitored port of the application B. Application B listens to network Socket port 8080 waiting for the port to be connected.

After receiving the response broadcast of the application B, if the application a agrees to connect with the application B, the application a actively connects to the Socket port 8080 monitored by the application B. After the connection is successful, a point-to-point communication link between the application A and the application B is established. Application A, B may utilize the communication link of the Socket port for high-speed bidirectional point-to-point communication. And the service parties can perform various service interactions, such as sharing and payment, according to the link.

The data communication processing method can be used for various terminal devices such as mobile phones, tablets, PCs, servers and PDAs, and a user can directly establish a two-way communication link for two applications displayed on a screen at the same time through one gesture. For example, a user slides two fingers to another application in any application, a two-way communication channel is established in the process of the two applications, diversified responses can be made to the action of the user according to different service scenes, for example, the content in the application A is shared to the application B, and the interactive use experience of the terminal device can be effectively improved.

The application also provides a data communication processing method which can be used for different service items in the same application, and can realize cross-service interactive processing based on sliding operation and control of a user between different service items on an application window interface. Fig. 14 is a flowchart of a method of another embodiment of a data communication processing method provided by the present application, and specifically, as shown in fig. 14, in an embodiment of a data communication processing method provided by the present application, the method may include:

s1': monitoring a control instruction generated by a control behavior on an application window interface, and identifying a user control track;

s2': judging whether the user control track meets a preset trigger condition for triggering a first service item to point to a second service item; and when the judgment result is yes, processing the service processing request of the first service item by adopting the response mode of the second service item.

In a preferred embodiment, the preset trigger condition that the first service item points to the second service item may include:

s201': the identified user control track slides from the window interface of the first service item to the window interface of the second service item in the application window interfaces displayed on the foreground of the touch screen.

The first service item and the second service item which are processed interactively may belong to different service items in the same application, or may be service items in different applications. Therefore, in another embodiment of the data communication processing method for enabling an existing user control track to achieve information interaction between different service items, the first service item and the second service item may adopt any one of the following implementation manners:

the first service item and the second service item belong to a service item in the same application;

at least one of the first service item and the second service item belongs to different applications.

Fig. 15 is a schematic diagram of an application scenario of another embodiment of a data communication processing method provided in the present application. As shown in fig. 15, a secondary window interface in a certain payment application PAY includes several payment service items, such as "telephone charge recharge", "water and electricity charge", "credit card payment", and the like. Meanwhile, three payment mode options of 'balance payment', 'China bank payment' and 'recruit credit card payment' which can be used for payment by a user are provided below the application window interface. If the certain payment option is pressed and then slid to a certain payment service item after being preset to press the certain payment option, the payment service item to which the payment is slid is selected by using the pressed payment. For example, as shown in fig. 15, if the user touches the "balance payment" option with a finger first and then slides the finger to "telephone fee charging", the payment application PAY may directly enable the "balance payment" service payment mode to charge the telephone fee to the user. Of course, if the user presses the window interface of the "chinese bank payment" service item and then slides to "telephone fee recharge", the payment application PAY may use the chinese bank deposit card bound by the user to perform telephone fee recharge, or click the "chinese bank payment" and then slide to the "red envelope" service item, and may default to use the funds of the chinese bank deposit card bound by the user to perform red envelope. Therefore, interaction processing among different business items can be quickly executed through gesture sliding control and the like on the touch screen among the different business items, and user operation experience is improved.

Based on the data communication processing method, the application also provides a data communication processing device. The data communication processing device can be used in various terminal devices such as a mobile communication terminal, a tablet, a PDA, a PC and the like, and can establish data communication among different applications based on a certain preset control action (such as a certain sliding gesture of a user on a touch screen) on a touch screen of the user in a multi-window mode, thereby realizing service interaction among different applications and improving the control experience of the user. Fig. 16 is a schematic block diagram of an embodiment of a data communication processing apparatus provided in the present application, and specifically, as shown in fig. 16, the apparatus may include:

the monitoring module 101 may be configured to monitor a control instruction generated by a control action acting on the touch screen, and identify a user control track;

the trajectory processing module 102 may be configured to determine whether the user manipulation trajectory meets a preset trigger condition that the first application points to the second application; and sending a communication establishment request of the first application when the judgment result is yes;

the multi-window determining module 103 may determine, after the second application receives the communication establishment request, whether a state of the second application is a multi-window mode in which an interface is displayed on a foreground of the touch screen;

a communication establishing module 104, configured to establish a data communication link between the first application and the second application when the multi-window determining module 103 determines that the result is yes.

In another embodiment of the data communication processing apparatus, the preset trigger condition that the first application points to the second application may include:

the identified user manipulation trajectory is a sliding behavior of a window interface of the first application sliding to a window interface of the second application.

In another embodiment of the data communication processing apparatus according to the present application, the sending a communication establishment request of a first application includes sending the communication establishment request in a message broadcast manner;

correspondingly, the establishing of the data communication link between the first application and the second application comprises:

The preset trigger condition that the first application points to the second application in the device can also be set according to user clicking, sliding, long pressing, combination of the user clicking, sliding, long pressing and the like. For the specific implementation of the foregoing embodiment and the manner of establishing the communication link between the first application and the second application, reference may be made to the description and the extended implementable manner of the related embodiments of the foregoing method, which are not described herein again.

The data communication processing device can be used for various terminal devices based on an Android system platform and supporting a multi-window mode, such as a commonly-used mobile communication terminal. Therefore, when the user uses the terminal device, two or more applications displayed on the touch screen can be quickly and intuitively used for establishing a communication link through simple gesture operation on the touch screen. By utilizing the established communication link, the service level interaction between the cross-applications, such as one-key sharing, sliding payment and the like, is quickly realized, and the user operation and use experience is greatly improved. Of course, the manipulation behavior acting on the touch screen may include a sliding behavior, a click behavior, a long press, and a combination thereof, which are touched on the touch screen by the user's limb.

The application further provides another embodiment of a data communication processing device, which can realize cross-service interaction processing based on sliding operation of a user between different service items of an application window interface. Fig. 17 is a schematic block structure diagram of another embodiment of a data communication processing apparatus provided in the present application, and specifically, as shown in fig. 17, in an embodiment of a data communication processing apparatus provided in the present application, the apparatus may include:

the control instruction acquisition module 201 may be configured to monitor a control instruction generated by a control behavior on the touch screen, and identify a user control track;

the service processing module 202 may be configured to determine whether the user manipulation trajectory meets a preset trigger condition that the first service item points to the second service item; and the method and the device can also be used for processing the service processing request of the first service item in a response mode of the second service item when the judgment result is yes.

In another embodiment of the data communication processing apparatus provided by the present application, the preset trigger condition that the first service item points to the second service item includes:

the identified user control track slides from the window interface of the first service item to the window interface of the second service item in the application window interfaces displayed on the foreground of the touch screen.

Of course, as mentioned above, the first service item and the second service item that are processed interactively may belong to different service items in the same application, or may be service items in different applications. Therefore, in another embodiment of the data communication processing apparatus provided in this application, the first service item and the second service item adopt any one of the following implementation manners:

The specific preset trigger condition that the first service item points to the second service item, the user control track formed by sliding, and the like in the foregoing embodiments may refer to the related description of the foregoing other embodiments, and are not described herein again.

The data communication processing device provided by the embodiment of the application can establish a data communication link between two or more applications simultaneously displayed on the screen by identifying the control track of the user on the touch screen, for example, sliding two fingers from a first application to a second application for a certain distance and the like, by utilizing the characteristic that the screen can simultaneously display two or more application window interfaces in a multi-window mode. Once the data communication link between the applications is established, the interaction on the service level between different applications, such as one-touch/fast cross-application sharing, payment, inquiry, data transfer, etc., can be realized. Therefore, the user can intuitively realize business interaction between different applications such as rapid sharing, query and the like through one gesture. Obviously, in more and more terminal devices supporting multiple application windows, such as a PC terminal, an Android/iOS system terminal, and the like, operations such as sharing, searching, and the like performed by a user between different applications can be greatly simplified. And the interaction mode among the applications can also improve the operation and use experience of the user and improve the use perception of the user product.

The data communication processing method and device provided by the application can be used in various terminal devices, such as mobile communication terminals (mobile phones and the like), tablets, PDAs, PCs, servers, service systems and the like, and can be used for establishing data communication among different applications based on a certain preset control action (such as a certain sliding gesture of a user on a touch screen) on a touch screen of the user in a multi-window mode, so that service interaction among different applications is realized, and the user control experience is improved. Therefore, the present application further provides an electronic device, which may include the above terminal device including a touch screen device, such as a mobile phone and a tablet. The electronic device may also be a part of these terminal devices in some embodiments, such as a data processing electronic device of the terminal device, and the touch screen may be connected with the electronic device separately, and combined to form a terminal used by a user. Specifically, in an embodiment of an electronic device provided in the present application, the electronic device may include:

the event monitoring unit can be used for monitoring a control instruction generated by a control action acting on the touch screen and identifying a user control track; the method can also be used for judging whether the user control track meets a preset trigger condition that the first application points to the second application;

the window state judging module may be configured to judge whether the state of the second application is a multi-window mode in which an interface is displayed on the foreground of the touch screen after the second application receives the communication establishment request of the first application;

the communication link establishing module may be configured to send a communication establishing request of the first application when it is determined that the user manipulation trajectory satisfies a preset trigger condition that the first application points to the second application; and the data communication link between the first application and the second application is established when the state of the second application is judged to be a multi-window mode in which an interface is displayed on the front desk of the touch screen.

In one embodiment, the communication link establishing module 303 of the electronic device establishing the data communication link between the first application and the second application may include establishing a unidirectional data communication link from the first application to the second application, the first application may send communication data to the second application, and the second application receives communication data. In other embodiments, the method may also include establishing a bidirectional data communication link between the first application and the second application, that is, in one embodiment, the first application may send communication data to the second application through the data communication link, and the second application may also send data communication data to the first application through the data communication link. Some embodiments of the electronic device, the data communication processing method and the data communication processing apparatus described above may set a processing mode according to a specific application environment. The application provides a preferred embodiment, after the first application adopts message broadcasting, the first application establishes point-to-point communication with the second application according to the received response message. Specifically, in another embodiment of the electronic device according to the present application, the sending, by the communication link establishment module, the communication establishment request by the first application includes sending, by the first application, the communication establishment request by a message broadcast;

The manipulation behavior acting on the touch screen may include a manipulation behavior of sliding directly on the touch screen through a limb of a user. In other embodiments of the present application, the manipulation behavior may also include a click or a slide operation performed on the touch screen by a user using a touch device, such as a stylus, a laser light, or other auxiliary tools. Even in other embodiments, the application does not exclude that the user does not directly contact the touch screen, but utilizes photoelectric induction, magnetic induction and the like to identify the control behavior of the user acting on the touch screen indirectly in the space. If the recognized limb sensing action of the user is specifically, for example, a sliding gesture made by the user at a certain distance on the touch screen, although the finger of the user does not directly touch the touch screen at this time, the electronic device application scenario provided by the embodiment of the application can still recognize that the user executes a control action of sliding for a certain distance, and translates the control action into a user control track. Therefore, in another embodiment of the electronic device described in the present application, the manipulation behavior acting on the touch screen may include at least one of the following:

a manipulation behavior of touching the touch screen through a user limb;

a control action of touching the touch screen through a touch device;

and the control action of the separated space on the touch screen is sensed and recognized through limbs.

Generally, the window mode may include, for example, that the window interfaces of the first application and the second application are in the same touch screen in a multi-window mode, for example, a current Android version supports that the window interfaces of two applications are simultaneously displayed in a display screen of a mobile phone terminal, or a certain tablet computer supports that the window interfaces of at most four applications are simultaneously displayed on the same touch screen. However, the multi-window mode described in this application does not exclude an implementation in which multiple application windows are foreground-displayed on different touch screens, for example, the first application and the second application may belong to the same processing system and be displayed on different touch screen terminals, as in a multi-screen display of a PC host. The same processing system may be a processing unit, a module, a device including one or more logic units, or a processing system including multiple servers, or may be the same distributed system. In this case, the window interfaces of the first application and the second application may be on the same touch screen, or may be displayed on different touch screens separately or partially in a multi-screen display manner. Of course, the first application and the second application of the multi-window multi-screen display may belong to different processing systems. Therefore, in another embodiment of the electronic device according to the present application, the window state determination module determines that the state of the second application is a multi-window mode in which an interface is displayed in a foreground of the touch screen when at least one of the following conditions is satisfied:

the window interface of the second application and the window interface of the first application are simultaneously displayed in the same touch screen in a foreground mode;

the window interface of the second application and the window interface of the first application are simultaneously and respectively displayed in different touch screens in a foreground mode;

and at least one of the first application and the second application has a part of window interface foreground displayed in different touch screens.

In an embodiment of the electronic device of the present application, the first application and the second application may belong to the same processing system, and in other embodiments, the first application and the second application may belong to different processing systems, as shown in fig. 6 and fig. 7. Therefore, in another embodiment of the electronic device described in the present application, the first application and the second application may be implemented as any one of the following:

Of course, in an implementation manner, the multiple windows described in the embodiment of the present application may also be a service window interface of an application operated by a user at the current time, and in another embodiment, the multiple windows may also include a thumbnail window interface or a zoom window interface of the application.

In other embodiments, the user may also establish communication between the cross-applications in the multi-window mode by using a click manipulation behavior. Or by a combination of clicking, sliding, long pressing, etc. In another embodiment of the electronic equipment, the control habit of some users in cross-application operation can be met and facilitated. Specifically, in another embodiment of the electronic device according to the present application, the preset trigger condition that the first application points to the second application includes at least one of the following:

the identified user control track is Ts in a preset effective period, the number of clicks on a window interface of a first application meets a first requirement M, the number of clicks on a window interface of a second application meets a second requirement, M, N is a preset number of clicks, Ts is effective time of the user control track, Ts is more than or equal to 0 second, M is more than or equal to 1, and N is more than or equal to 1;

the identified user control track is that a first application is selected by sliding a preset pattern in a window interface of the first application, and the first application slides at least a first threshold distance from the position of the preset pattern to the position of a second application window interface;

the identified user operation track is that Ts in a preset effective period, the number of times of clicking on a window interface of a first application meets a first requirement K, the window interface of the first application slides to the position direction of a window interface of a second application for at least a second threshold distance, K is the preset number of times of clicking, Ts is effective time of the user operation track, Ts is more than or equal to 0 second, and K is more than or equal to 1.

The identified user control track is that Ts in a preset effective period, the time is longer than T milliseconds in a window interface of a first application, and at least a third threshold distance is slid from the window interface of the first application to the position direction of a window interface of a second application, wherein T is larger than or equal to 0.

In another embodiment of the electronic device of the present application, the preset trigger condition that the first application points to the second application may include:

the identified user manipulation track is a sliding behavior of sliding from the window interface of the first application to the window interface of the second application

If the user slides a certain search result to the right side for a distance through gestures in the first application, namely the search application B, the user sends out a broadcast message of a communication establishment request. And after receiving the broadcast message, the instant messaging applications IM _1, IM _2 and IM _3 displayed in the foreground can directly establish two-way communication with the search application B. Then, the search application B may send corresponding service-level processing data (a certain search result RE _1) to the instant messaging applications IM _1, IM _2, and IM _3, and the instant messaging applications IM _1, IM _2, and IM _3 may respectively perform corresponding response processing.

According to the implementation manner of the embodiment, a user can intuitively establish a communication link between two or more applications simultaneously displayed on a screen through one gesture, and a first application can quickly realize business-level interaction with another application by using the communication link established by the technical scheme provided by the embodiment. Therefore, in another embodiment of an electronic device described in this application, the electronic device may further include:

and the service processing unit can be used for realizing service level message interactive processing between the first application and the second application by utilizing the established data communication link.

The communication link is at least a communication connection from the first application to the second application, and the communication link may be a bidirectional communication link when the embodiment of the present application is specifically implemented. After the communication link is established, the first application may send a business-level processing message, such as shared content or one-touch payment, to the second application based on the data communication link. The information interaction of multi-window modes such as cross-application sharing and query is quickly realized based on the user sliding gesture, the operation is more visual, the user can use the information conveniently, and the user experience is effectively improved.

The data communication processing method or device or the electronic device provided by the application can be used in various terminal devices with touch display devices, such as mobile communication terminals (mobile phones and the like), tablets, PDAs, PCs, servers, service processing systems and the like, and can be used for establishing data communication among different applications in the touch display devices based on a certain preset control action (such as a certain sliding gesture of a user on a touch screen) on the touch screen of the user in a multi-window mode, realizing service interaction among different applications and improving the control experience of the user touching the display devices. Therefore, the present application also provides a touch display device, fig. 18 is a schematic structural diagram of an embodiment of a touch display device described in the present application, as shown in fig. 18, the touch display device may include at least one touch screen and one processor,

the touch screen can be used for displaying interface information of an application window and sensing a control behavior acted on the touch screen, and sending a corresponding control instruction generated by the control behavior to the processor;

the processor is configured to identify a user manipulation track according to the manipulation instruction, and send a communication establishment request of a first application when it is determined that the user manipulation track meets a preset trigger condition that the first application points to a second application; the method can also be used for judging whether the state of the second application is a multi-window mode in which an interface is displayed on the front desk of the touch screen after the second application receives the communication establishment request; the method may further be configured to establish a data communication link between the first application and the second application when a result of the determination is yes.

In some terminal devices using the Android system version, whether a current Application is in a multi-window mode or not and whether the current Application is displayed in a foreground screen or not may be determined through an Application Programming Interface (API) provided by the Android system. In one embodiment, the first application and the second application may belong to the same processing system, and in other embodiments, the first application and the second application may belong to different processing systems.

In an embodiment of the touch display device, the user may establish a communication link between different applications by a sliding gesture on the touch screen. Therefore, in another embodiment of the touch display device provided by the present application, the preset trigger condition that the first application points to the second application may include:

In another embodiment of a touch display device provided by the present application, the processor sends a communication establishment request of a first application in a message broadcast manner;

establishing a point-to-point communication link from the first application to the second application after the first application receives the reply message of the second application.

In the Android system, it may be configured that the application a and the application B register broadcastreever (Broadcast receiver) after starting, so that the Android system calls back the pre-registered broadcastreever to notify the application when receiving the Broadcast. For example, one common registration method is static registration, which generally refers to defining a receiver within the application of android manifest.

In a specific example, for example, in an Android system of a mobile phone, the application a may send a request broadcast message carrying a packet name of the application a. When application B receives the request broadcast, it can first decide whether to respond to the request from application a. If the system does not process the application of the emergency event or the exclusive process or is in other busy states, etc., the request of the application A can be approved, and a one-way communication link from B to A can be established. Then, whether the current application B is in a multi-window mode or not and whether the current application B is displayed on a screen foreground or not can be judged through an API provided by the Android. After the judgment is passed, the application B randomly monitors an available network Socket port, such as 8080, sends a response broadcast according to the received request of the application, and carries the own packet name and the monitored port of the application B. Application B listens to network Socket port 8080 waiting for the port to be connected. After receiving the response broadcast of the application B, if the application a agrees to connect with the application B, the application a actively connects to the Socket port 8080 monitored by the application B. After the connection is successful, a point-to-point communication link between the application A and the application B is established.

According to the touch display device provided by the embodiment of the application, when the fact that the user needs to perform information interaction between different applications is determined, communication between the applications can be established quickly, and the user can intuitively establish a communication link between at least two applications displayed on a screen at the same time through a simple and intuitive sliding gesture. The touch display device can perform business-level interaction with another application by using a communication link established by the technical scheme provided by the application, and specific interaction modes/response contents can be designed by business developers according to different scenes, such as one-key sharing, one-key payment, one-key song listening and the like. Obviously, according to the embodiment of the application, the interactive operation between different applications such as user sharing and searching can be greatly simplified in more and more terminal devices supporting multiple applications such as PC terminals and Android/iOS system terminals, the user requirements are met, the user terminal control experience can be greatly improved through the interactive mode between the applications, and the use perception of user products is improved.

The data communication processing method or device or the electronic device provided by the application can be used in another terminal device with a touch display device, such as a mobile communication terminal (mobile phone, etc.), a tablet, a PDA, a PC, a server, a service processing system, etc., and can establish information interaction between different service items in the touch display device based on a certain preset control action (such as a certain sliding gesture of a user on the touch screen) on the touch screen of the user in a multi-window mode, so as to quickly implement operation between service items of the same application or different applications, and improve the control experience of the user touching the display device. Therefore, the present application also provides another touch display device, fig. 19 is a schematic structural diagram of an embodiment of a touch display device described in the present application, as shown in fig. 19, the touch display device may include at least one touch screen and one processor,

the processor is configured to determine whether the user manipulation trajectory satisfies a preset trigger condition for sending a first service item to a second service item; and the method and the device can also be used for processing the service processing request of the first service item in a response mode of the second service item when the judgment result is yes.

In another embodiment of the touch display device provided by the present application, the preset trigger condition that the first service item points to the second service item includes:

The first service item and the second service item which are processed interactively may belong to different service items in the same application, or may be service items in different applications. In another embodiment of the touch display device provided by the present application, the first service item and the second service item adopt any one of the following implementation manners:

The other touch display device provided by the embodiment of the application can realize sliding operation among different business items on an application window interface to realize cross-business interactive processing. The interaction processing among different business items can be quickly executed through gesture sliding control and the like on the touch screen among the different business items, and the operation experience of a user is improved.

The application aims to provide a data communication processing method and device, an electronic device and a touch display device, and by utilizing the characteristic that a plurality of application windows can be simultaneously displayed on a screen in a multi-window mode, a communication link can be established between at least two applications simultaneously displayed on the screen through simple and visual control gestures of a user, application interaction between different applications in a terminal device is realized quickly and conveniently, user operation is reduced, and user operation and use experience are improved. According to the application implementation scheme, the interactive operation between the applications such as user sharing and searching can be greatly simplified in more and more terminal devices supporting multiple application windows such as a PC terminal and an Android/iOS system terminal, the user requirements are met, the user terminal control experience can be greatly improved through the interactive mode between the applications, and the use perception of user products is improved.

Although the present application refers to the description of the message interaction mode of message broadcasting or point-to-point communication, the manipulation mode of double-finger touch or limb-induced touch in space, the user manipulation trajectory of clicking or long-pressing or sliding or circling, the continuous or discontinuous user manipulation trajectory definition, the information interaction mode of multi-application in the same screen or in a multi-window display mode of split screen, the user gesture recognition, the multi-window foreground display real-time scene, etc., the present application is not limited to the case of meeting the industry communication standard, the touch screen input information recognition, the sliding trajectory definition or the embodiment description. Certain industry standards, or implementations modified slightly from those described using custom modes or examples, may also achieve the same, equivalent, or similar, or other, contemplated implementations of the above-described examples. The embodiments using the modified or transformed data acquisition, storage, judgment, processing and the like may still fall within the scope of the alternative embodiments of the present application.

Although the present application provides method steps as described in an embodiment or flowchart, more or fewer steps may be included based on conventional or non-inventive means. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of orders and does not represent the only order of execution. When an actual apparatus or end product executes, it may execute sequentially or in parallel (e.g., parallel processors or multi-threaded environments, or even distributed data processing environments) according to the method shown in the embodiment or the figures. The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, the presence of additional identical or equivalent elements in a process, method, article, or apparatus that comprises the recited elements is not excluded.

The units, devices, modules, etc. set forth in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. For convenience of description, the above devices are described as being divided into various modules by functions, and are described separately. Of course, in implementing the present application, the functions of each module may be implemented in one or more software and/or hardware, or a module implementing the same function may be implemented by a combination of a plurality of sub-modules or sub-units, and the like. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

Those skilled in the art will also appreciate that, in addition to implementing the controller as pure computer readable program code, the same functionality can be implemented by logically programming method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Such a controller may therefore be considered as a hardware component, and the means included therein for performing the various functions may also be considered as a structure within the hardware component. Or even means for performing the functions may be regarded as being both a software module for performing the method and a structure within a hardware component.

The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

From the above description of the embodiments, it is clear to those skilled in the art that the present application can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, or the like, and includes several instructions for enabling a computer device (which may be a personal computer, a mobile terminal, a server, or a network device) to execute the method according to the embodiments or some parts of the embodiments of the present application.

The embodiments in the present specification are described in a progressive manner, and the same or similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. The application is operational with numerous general purpose or special purpose computing system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.

While the present application has been described with examples, those of ordinary skill in the art will appreciate that there are numerous variations and permutations of the present application without departing from the spirit of the application, and it is intended that the appended claims encompass such variations and permutations without departing from the spirit of the application.

Claims

1. A method of data communication processing, the method comprising:

judging whether the user control track meets a preset trigger condition that a first application points to a second application, and if the user control track meets the preset trigger condition, sending a communication establishment request of the first application to the second application;

and after the second application receives the communication establishment request, establishing a bidirectional data communication link between the first application and the second application.

2. A data communication processing method according to claim 1, wherein said sending a communication establishment request of the first application to the second application comprises: sending a communication establishment request of a first application in a message broadcasting mode;

said establishing a bidirectional data communication link between said first application and said second application comprises:

after the second application receives the communication establishment request, response information of the second application is sent to the first application in a message broadcasting mode, and a point-to-point bidirectional communication link between the first application and the second application is established based on the communication establishment request and the response information.

3. A data communication processing method according to claim 1, wherein the preset trigger condition that the first application points to the second application comprises:

the user control track is a control behavior of sliding from a window interface of a first application to a window interface of a second application, and the first application and the second application are in a multi-window mode.

4. The data communication processing method according to claim 3, wherein the determining whether the user manipulation trajectory satisfies a preset trigger condition for triggering the first application to point to the second application includes:

after the first application and the second application are determined to be in a multi-window mode, obtaining sliding record information in the control instruction, wherein the sliding record information comprises recorded data information generated when a sliding action is executed on a touch screen through at least one limb of a user;

calculating at least one behavior index data at least comprising sliding direction, sliding distance, sliding time and sliding speed according to the recorded information;

and comparing the behavior index data with a corresponding preset trigger threshold, and determining whether the control track meets the preset trigger condition according to the comparison result.

5. A data communications processing method according to claim 3, the method of determining that the first application and the second application are in a multi-window mode comprising:

the window interface of the second application and the window interface of the first application are simultaneously and respectively displayed in different touch screens or the same touch screen in a foreground mode;

and at least one of the first application and the second application has a part of window interface foreground displayed in the touch screen.

6. A data communications processing method according to claim 3, the second application comprising an associated application other than the first application, the associated application comprising an application in which a window interface to which the user manipulation trajectory is overlaid is displayed in front of the touch screen.

7. The data communication processing method according to claim 1, wherein the determining whether the user manipulation trajectory satisfies a preset trigger condition for triggering the first application to point to the second application includes at least one of:

the user operation track meets the condition that in a preset effective period, the number of clicks on the window interface of the first application meets a first requirement, and the number of clicks on the window interface of the second application meets a second requirement;

the user operation track meets the condition that a preset pattern slides in the window interface of the first application, and slides at least a first threshold distance from the position of the preset pattern to the position of the second application window interface;

the user operation track meets the condition that in a preset effective period, the number of times of clicking on a window interface of a first application meets a first requirement, and the window interface of the first application slides at least a second threshold distance in the direction of the position where a window interface of a second application is located;

and the user operation track meets the condition that in a preset effective period, the time is longer than the preset time in the window interface of the first application, and the user operation track slides at least a third threshold distance from the window interface of the first application to the position direction of the window interface of the second application.

8. A data communication processing method according to claim 1, wherein the manipulation behavior acting on the touch screen includes at least one of the following:

a manipulation behavior of touching the touch screen through a user limb;

a control action of touching the touch screen through a touch device;

9. A data communications processing method according to any one of claims 1 to 8, the method further comprising:

and the first application and the second application perform service-level message interaction processing based on the established data communication link.

10. A data communications processing method according to claim 1, wherein the first application and the second application belong to the same processing system or different processing systems.

11. A method of data communication processing, the method comprising:

if the user control track slides from the window interface of a first service item to the window interface of a second service item in the application window interfaces displayed on the foreground of the touch screen, determining whether the user control track meets a preset trigger condition that the first service item points to the second service item, and processing the service processing request of the first service item by adopting a response mode of the second service item.

12. A data communication processing method according to claim 11, wherein the first service item and the second service item belong to a service item in the same application or belong to a service item in different applications.

13. A data communication processing apparatus, the apparatus comprising:

the track processing module is used for judging whether the user control track meets a preset trigger condition that a first application points to a second application, and if the user control track meets the preset trigger condition, sending a communication establishment request of the first application to the second application;

and the communication establishing module is used for establishing a bidirectional data communication link between the first application and the second application after the second application receives the communication establishing request.

14. A touch display device comprising at least a touch screen and a processor,

the processor is configured to identify a user manipulation track according to the manipulation instruction, send a communication establishment request of the first application to the second application when it is determined that the user manipulation track meets a preset trigger condition that the first application points to the second application, and establish a bidirectional data communication link between the first application and the second application after the second application receives the communication establishment request.

15. A touch display device comprising at least a touch screen and a processor,

the processor is configured to identify a user control track according to the control instruction, and judge whether the user control track meets a preset trigger condition for sending a first service item to a second service item; and the server is further configured to process the service processing request of the first service item in a response manner of the second service item when the determination result is yes.