WO2022032966A1 - 触摸数据处理方法、装置、设备及存储介质 - Google Patents
触摸数据处理方法、装置、设备及存储介质 Download PDFInfo
- Publication number
- WO2022032966A1 WO2022032966A1 PCT/CN2020/139610 CN2020139610W WO2022032966A1 WO 2022032966 A1 WO2022032966 A1 WO 2022032966A1 CN 2020139610 W CN2020139610 W CN 2020139610W WO 2022032966 A1 WO2022032966 A1 WO 2022032966A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- touch
- operating system
- display area
- target display
- screen
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/542—Event management; Broadcasting; Multicasting; Notifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/16—Constructional details or arrangements
- G06F1/1613—Constructional details or arrangements for portable computers
- G06F1/1633—Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
- G06F1/1637—Details related to the display arrangement, including those related to the mounting of the display in the housing
- G06F1/1643—Details related to the display arrangement, including those related to the mounting of the display in the housing the display being associated to a digitizer, e.g. laptops that can be used as penpads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/16—Constructional details or arrangements
- G06F1/1613—Constructional details or arrangements for portable computers
- G06F1/1633—Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
- G06F1/1675—Miscellaneous details related to the relative movement between the different enclosures or enclosure parts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/16—Constructional details or arrangements
- G06F1/1613—Constructional details or arrangements for portable computers
- G06F1/1633—Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
- G06F1/1684—Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
- G06F1/1694—Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2200/00—Indexing scheme relating to G06F1/04 - G06F1/32
- G06F2200/16—Indexing scheme relating to G06F1/16 - G06F1/18
- G06F2200/161—Indexing scheme relating to constructional details of the monitor
- G06F2200/1614—Image rotation following screen orientation, e.g. switching from landscape to portrait mode
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1423—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display
- G06F3/1431—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display using a single graphics controller
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
Definitions
- the present application relates to the technical field of smart devices, and in particular, to a touch data processing method, apparatus, device, and storage medium.
- All kinds of intelligent display devices are used more and more frequently in life or work.
- Some smart display devices have multiple operating systems built into them, or multiple operating systems built into different devices.
- the screen content of the sender such as Windows, Mac or Chrome notebook
- the receiver such as a large-sized smart interactive tablet
- the first operating system may be referred to as the main operating system
- the second operating system may be referred to as the external channel operating system.
- the content of the second operating system (eg Windows system) in the intelligent interactive tablet can be displayed on the display interface of the first operating system (eg Android system).
- the second operating system is controlled to respond.
- the present application provides a touch data processing method, device, device and storage medium to solve the problem in the related art that after rotating the touch screen of the device where the first operating system is located, it is impossible to control the second operating system to respond to the touch operation by touching the touch screen .
- a method for processing touch data wherein the first operating system and the second operating system are operating systems in the same device or in different devices, and the method includes:
- a touch data processing method wherein the first operating system and the second operating system are operating systems in the same device or in different devices, and the method includes:
- the touch position information is acquired, and the touch position information is based on mapping the position information of the target display area in the system coordinate system of the first operating system to Obtained in the touch frame coordinate system of the first operating system;
- a touch data processing method wherein the first operating system and the second operating system are operating systems in the same device or in different devices, and the method includes:
- the device where the first operating system is located displays the display content from the second operating system on the display interface of the first operating system, and the display content is displayed in the target display area of the display interface of the first operating system;
- the device where the second operating system is located responds to the touch operation according to the received touch data.
- a touch data processing apparatus wherein the first operating system and the second operating system are operating systems in the same device or in different devices, and the apparatus includes:
- a data display module configured to display the display content from the second operating system on the display interface of the first operating system, the display content being displayed in the target display area of the display interface of the first operating system;
- An operation determination module configured to determine whether the touch operation is displayed on the target display area if a touch operation on the touch screen is detected after the touch screen on the device where the first operating system is located is rotated by a preset angle manipulation of the content;
- a data processing module configured to send the touch data corresponding to the touch operation to the second operating system if the touch operation is an operation on the displayed content in the target display area, so that the second operating system can respond to the touch Action responds.
- a touch data processing apparatus wherein the first operating system and the second operating system are operating systems in the same device or in different devices, and the apparatus includes:
- a content display module configured to display the display content from the second operating system on the display interface of the first operating system, the display content being displayed in the target display area of the display interface of the first operating system;
- a position obtaining module configured to obtain touch position information, the touch position information is based on mapping the position information of the target display area under the system coordinate system of the first operating system to the touch frame coordinate system of the first operating system get;
- an operation determination module configured to determine whether the touch operation is an effect on the content displayed in the target display area according to the relationship between the touch operation and the touch position information when a touch operation on the display interface of the first operating system is detected operate;
- a data processing module configured to send the touch data corresponding to the touch operation to the second operating system if the touch operation is an operation on the displayed content in the target display area, so that the second operating system can respond to the touch Action responds.
- a computer storage medium stores a plurality of instructions, and the instructions are adapted to be loaded by a processor and execute the steps of any of the above touch data processing methods.
- an electronic device including a memory and a processor; wherein, the memory stores a computer program, and the computer program is adapted to be loaded by the processor and execute any of the above The touch data processing method steps.
- display content from the second operating system is displayed on the display interface of the first operating system, and after the touch screen on the device where the first operating system is located is rotated by a preset angle, if a touch operation on the touch screen is detected When , determine whether the touch operation is an operation on the content displayed in the target display area. In this way, if the touch operation is an operation on the displayed content in the target display area, the touch data corresponding to the touch operation can be sent to the second operating system, so that the second operating system responds to the touch operation. Even if the touch screen on the device where the first operating system is located is rotated, the second operating system can be controlled to respond to the touch operation by touching the touch screen.
- FIG. 1 is a schematic diagram of screen projection display in the related art.
- FIG. 2A is a schematic diagram of a device to which the touch data processing method shown in the present application is applied according to an exemplary embodiment.
- FIG. 2B is a schematic diagram of another device to which the touch data processing method according to an exemplary embodiment of the present application is applied.
- FIG. 3 is a flowchart of a touch data processing method according to an exemplary embodiment of the present application.
- FIG. 4 is a flowchart of another touch data processing method according to an exemplary embodiment of the present application.
- FIG. 5A is a flowchart of another touch data processing method according to an exemplary embodiment of the present application.
- FIG. 5B is a schematic diagram of a touch frame coordinate system according to an exemplary embodiment of the present application.
- FIG. 5C is a schematic diagram of a screen coordinate system according to an exemplary embodiment of the present application.
- FIG. 5D is a flowchart of determining touch position information according to an exemplary embodiment of the present application.
- FIG. 5E is a schematic diagram of system location information according to an exemplary embodiment of the present application.
- FIG. 5F is a schematic diagram of a system coordinate system after the device is rotated by 0 degrees, 90 degrees, 180 degrees, and 270 degrees according to an exemplary embodiment of the present application.
- FIG. 5G is a schematic diagram showing the comparison of the target display area in different coordinate systems after the device is rotated 90 degrees clockwise according to an exemplary embodiment of the present application.
- FIG. 5H is a schematic diagram showing a comparison between system location information and screen location information after a device is rotated by 90 degrees according to an exemplary embodiment of the present application.
- FIG. 5I is a schematic diagram showing a comparison between system location information and screen location information after a device is rotated 180 degrees according to an exemplary embodiment of the present application.
- FIG. 5J is a schematic diagram showing a comparison between system location information and screen location information after a device is rotated by 270 degrees according to an exemplary embodiment of the present application.
- FIG. 5K is a schematic diagram of a window provided with a floating window according to an exemplary embodiment of the present application.
- FIG. 5L shows a schematic diagram of a touch trajectory according to an exemplary embodiment of the present application.
- FIG. 6 is a data flow diagram according to an exemplary embodiment of the present application.
- FIG. 7 is a block diagram of a touch data processing apparatus according to an exemplary embodiment of the present application.
- FIG. 8 is a block diagram of another touch data processing apparatus according to an exemplary embodiment of the present application.
- FIG. 9 is a hardware structure diagram of an electronic device where a touch data processing apparatus is located according to an exemplary embodiment of the present application.
- first, second, third, etc. may be used in this application to describe various information, such information should not be limited by these terms. These terms are only used to distinguish the same type of information from each other.
- first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information without departing from the scope of the present application.
- word "if” as used herein can be interpreted as "at the time of” or "when” or "in response to determining.”
- Different operating systems can share the same display screen.
- the different operating systems may refer to the same or different types of operating systems in the same device, or may refer to the same or different types of operating systems located in different devices.
- Screen sharing is a situation where different devices share the same display screen. In scenarios such as work, study, and entertainment, screen sharing operations are widely used to synchronize multimedia data from the source device to the target device, and the target device plays the content.
- the applicant found that when the content of the second operating system is displayed on the display interface of the first operating system, the touch screen of the device where the first operating system is located must be in a designated horizontal and vertical state, so as to ensure that when touching the touch screen of the first operating system, the The second operating system can respond accordingly.
- FIG. 1 it is a schematic diagram of screen projection display in the related art.
- the terminal device serving as the source device can display the multimedia data on its screen on the screen of the intelligent interactive device (target device) in a full-screen manner.
- the terminal device In the process of displaying the multimedia data of the terminal device by the intelligent interactive device, if the touch screen of the intelligent interactive device is rotated, the terminal device cannot be controlled to respond to the touch operation by touching the touch screen of the intelligent interactive device.
- the present application provides a touch data processing method, which can control the second operating system to respond to the touch operation by touching the touch screen even if the touch screen on the device where the first operating system is located is rotated.
- the touch data processing method provided in this embodiment may be implemented by software, a combination of software and hardware, or implemented by hardware.
- the involved hardware may be composed of two or more physical entities, or may be composed of one physical entity constitute.
- the method of this embodiment can be applied to an electronic device with display and touch functions.
- the electronic device may be an intelligent display device such as an intelligent interactive tablet and a digital tablet, or may be a tablet computer with display and touch functions, a notebook computer, a desktop computer, a PDA (Personal Digital Assistant, personal digital assistant) and other devices.
- the execution body of the touch data processing method may be a processing module in an electronic device, or may be a first operating system.
- the first operating system and the second operating system may be two identical or different operating systems in the same device, or may be the same or different types of operating systems located in different devices.
- the processing module may be a functional module different from the first operating system.
- the first operating system corresponds to a set of modules (for example, including a CPU, a GPU, a processor, a memory, etc.), and the processing module here may be a different MCU (Microcontroller Unit, micro control unit) from the set of modules.
- the electronic device may further include a touch data collection module and a display screen.
- the touch operation surface of the touch data collection module may cover the display screen for detecting touch operations and generating corresponding touch data. The following takes an example of a touch data processing method executed by a processing module for illustration.
- FIG. 2A it is a schematic diagram of a device to which the touch data processing method shown in the present application is applied according to an exemplary embodiment.
- the first operating system and the second operating system can be the same or different operating systems in the same device.
- the first operating system can be the Android system
- the second operating system can be is a Windows system.
- the touch data acquisition module can transmit the collected touch data to the processing module, and the processing module decides whether to transmit the touch data to the first operating system for response or to the second operating system for response.
- FIG. 2B it is a schematic diagram of another device to which the touch data processing method shown in the present application according to an exemplary embodiment is applied.
- the first operating system and the second operating system may be the same or different types of operating systems located in different devices.
- the first device may be a display device with a touch display function
- the second device may be a smartphone, a tablet computer, a notebook computer, a desktop computer, and other devices that have screen projection requirements.
- the touch data acquisition module can transmit the collected touch data of the first device to the processing module, and the processing module decides whether to transmit the touch data to the first operating system for response or to the second operating system for response.
- an intelligent interactive tablet is used as an example for illustration.
- the execution body of the embodiment of the present application may be an intelligent interactive tablet.
- the intelligent interactive tablet also known as the interactive smart tablet, can be an integrated device that controls the content displayed on the display panel and realizes human-computer interaction through touch technology. It integrates a projector, an electronic whiteboard, a screen, One or more functions such as audio, TV and video conference terminal.
- the first operating system and the second operating system perform wired screen sharing. For example, there is no need to pre-install screen transfer software in the first operating system and the second operating system, the first operating system transmits touch data to the second operating system, and the second operating system can transmit the content to the first operating system through standard HDMI system, so as to realize wired screen transmission.
- the first operating system and the second operating system are operating systems in the same device or different devices, and the method include:
- step 302 the display content from the second operating system is displayed on the display interface of the first operating system, and the display content is displayed in the target display area of the display interface of the first operating system;
- step 304 after the touch screen on the device where the first operating system is located is rotated by a preset angle, if a touch operation on the touch screen is detected, it is determined whether the touch operation is for displaying content in the target display area operation;
- step 306 if yes, the touch data corresponding to the touch operation is sent to the second operating system, so that the second operating system responds to the touch operation.
- display content from the second operating system is displayed on the display interface of the first operating system, and after the touch screen on the device where the first operating system is located is rotated by a preset angle, if a touch operation on the touch screen is detected When , determine whether the touch operation is an operation on the content displayed in the target display area. In this way, if yes, the touch data corresponding to the touch operation can be sent to the second operating system, so that the second operating system can respond to the touch operation. Even if the touch screen on the device where the first operating system is located is rotated, the second operating system can be controlled to respond to the touch operation by touching the touch screen.
- the method in this embodiment of the present application may be executed by the device where the first operating system is located.
- it may be executed by the first operating system, or may be executed by other processing modules different from the first operating system.
- the target display area may be a full screen area of the display interface of the first operating system.
- the display interface of the first operating system displays the display content of the second operating system in full screen, even if the touch screen on the device where the first operating system is located is rotated, the second operating system can be controlled by touching the touch screen to perform touch operations. response.
- the target display area is defined as a partial area in the display interface of the first operating system.
- a local area can also be called a partial area, or a non-full-screen area.
- the display interface of the first operating system not only displays the interface content of the second operating system, but also because the interface content only occupies a partial area in the display interface of the first operating system, that is, a non-full-screen area, the first operation
- Other areas of the display interface of the system may display the display content of the first operating system.
- the display content of the first operating system when the display content of the first operating system is displayed in a non-full-screen area, if a touch operation on the touch screen of the first operating system is detected, it is also possible to judge whether the touch operation is within the target display area.
- the operation of displaying the content determines whether to send the touch data corresponding to the touch operation to the second operating system, thereby enabling the second operating system to respond to the touch operation.
- the display content of the second operating system is displayed on the display interface of the first operating system in a window display manner.
- the target display area is the area where the window is located.
- the target display area is the full-screen area, and when the window is reduced, the target display area is the non-full-screen area. It can be seen that the display content of the second operating system can be displayed in full screen or non-full screen in the display interface of the first operating system by scaling the window.
- the target display area is not a specific or preset display area in the display interface of the first operating system, but it can be understood that the display interface of the first operating system displays data from the second operation The display area of the system's display content.
- the touch operation when the display content of the second operating system is displayed in the window of the display interface of the first operating system, if a touch operation for the touch screen of the first operating system is detected, it is determined whether the touch operation is correct or not. The operation of displaying content in the target display area; if yes, send the touch data corresponding to the touch operation to the second operating system, so that the second operating system responds to the touch operation. Realize that even if the content of the second operating system is displayed in the display interface of the first operating system in a windowed form, the touch data can be returned to the second operating system, so that the second operating system can respond to the touch operation .
- the touch screen on the device where the first operating system is located is a designated horizontal and vertical screen, and it can be considered that the touch screen is in a 0-degree state.
- the user may rotate the touch screen to different preset angles.
- the preset angle may be an angle in a specified angle set, for example, the specified angle set includes: 90 degrees, 180 degrees, 270 degrees, 360 degrees, and so on. If supported by the device, the preset angle can also be any angle.
- a sensor with a motion detection function such as a gyroscope or an angle sensor may be configured inside the device where the first operating system is located, and the first operating system can determine whether the device rotates and whether it rotates by acquiring the detection data of the sensor. angle, and then it can be determined whether the detected rotation angle of the device is a preset angle, and which preset angle in the specified angle set.
- the touch screen After the touch screen is rotated, whether the display content of the second operating system is displayed in a partial area (non-full-screen area) or a window, if a touch operation on the touch screen is detected, it can be determined whether the touch operation is an effect on the content displayed in the target display area. operation; if yes, the touch data corresponding to the touch operation can be sent to the second operating system, so that the second operating system responds to the touch operation.
- the touch data corresponding to the touch operation can be sent to the second operating system, so that the second operating system responds to the touch operation.
- the updated display content is displayed in the target display area.
- the updated display content may be obtained by the second operating system updating the display content based on the touch operation. After the second operating system responds to the touch operation, the response result can be transmitted to the first operating system, and the updated display content can be displayed in the target display area of the first operating system.
- the touch operation is an operation on the displayed content in the target display area
- the second operating system in order to quickly display the updated display content, not only the touch data corresponding to the touch operation is sent to the second operating system, so that the second operation
- the system responds to the touch operation, and may also send touch data corresponding to the touch operation to the first operating system, and the first operating system directly responds to the touch operation.
- the updated display content may be obtained by the first operating system updating the display content based on the touch operation, so that the update speed of the display content can be improved.
- a floating window and/or controls for controlling the display state of the target display area may be displayed on the target display area.
- the floating window may be a button for controlling the target display area, or a button for controlling the display interface of the first operating system, or the like.
- the target display area display state may be an area size state, an area position state, and the like.
- the controls may be maximized controls, minimized controls, close controls, etc., which are fixed on the target display area. Touch operations acting on the floating window and controls are responded to by the first operating system. In this way, the area where the floating window and the controls are located can be used as a shielding area, and touch operations on the shielding area are not responded to by the second operating system.
- the display position of the target display area may also be adjusted, based on this, in one embodiment, after the display position of the target display area is adjusted, if the touch operation is an operation on the display content in the adjusted target display area, Then, the touch data of the touch operation is sent to the second operating system, so that the second operating system responds to the touch operation. Therefore, even if the position of the target display area is changed, the second operating system can also respond to the touch operation.
- the touch data processing method is also introduced from the perspective of interaction.
- FIG. 4 is a flowchart of another touch data processing method shown in the present application according to an exemplary embodiment
- the first operating system and the second operating system are operating systems in the same device or in different devices, and the Methods include:
- step 402 the device where the first operating system is located displays the display content from the second operating system on the display interface of the first operating system, and the display content is displayed in the target display area of the display interface of the first operating system;
- step 404 after the touch screen of the device on which the first operating system is located is rotated by a preset angle, if a touch operation on the touch screen is detected, it is determined whether the touch operation is on the touch screen. The operation of displaying content in the target display area; if yes, sending the touch data corresponding to the touch operation to the second operating system;
- step 406 the device where the second operating system is located responds to the touch operation according to the received touch data.
- FIG. 4 is the same as the related technology in FIG. 3 , and in order to save space, details are not repeated here.
- This embodiment realizes that even if the touch screen on the device where the first operating system is located is rotated, the device where the second operating system is located can still respond to a touch operation on the device where the first operating system is located.
- FIG. 5A is a flowchart of another touch data processing method shown in the present application according to an exemplary embodiment
- the first operating system and the second operating system are operating systems in the same device or different devices, and the Methods include:
- step 502 the display content from the second operating system is displayed on the display interface of the first operating system, and the display content is displayed in the target display area of the display interface of the first operating system;
- step 504 if the touch screen on the device where the first operating system is located is rotated by a preset angle, touch position information is obtained, and the touch position information is based on placing the target display area in the system coordinate system of the first operating system The location information is mapped to the touch frame coordinate system of the first operating system and obtained;
- step 506 when a touch operation on the display interface of the first operating system is detected, it is determined whether the touch operation is an operation on the displayed content in the target display area according to the relationship between the touch operation and the touch position information ;
- step 508 if yes, the touch data corresponding to the touch operation is sent to the second operating system, so that the second operating system responds to the touch operation.
- the second operating system in order to display the display content of the second operating system in the target display area, the second operating system can also respond to the touch operation on the touch screen of the first operating system, the first operating system may be different from the first operating system.
- Other processing modules of an operating system can obtain touch position information, and when detecting a touch operation on the display interface of the first operating system, determine whether the touch operation is displayed to the target according to the relationship between the touch operation and the touch position information The operation of displaying content in the area, and then determine whether to send the touch data corresponding to the touch operation to the second operating system, so that the second operating system can also respond to the touch operation on the touch screen of the first operating system.
- the method of this embodiment may be executed by the device where the first operating system is located. For example, it may be executed by the first operating system, or may be executed by other processing modules different from the first operating system. For example, the method of this embodiment can be applied to the processing module in FIG. 2A or FIG. 2B .
- the full-screen content of the second operating system can be displayed in the target display area of the display interface of the first operating system, so as to achieve Display as much of the content of the second operating system as possible.
- part of the screen content of the second operating system may be displayed in the target display area of the display interface of the first operating system, so as to cope with some special scenarios, the screen projection content is more flexible.
- the display interface of the first operating system displays the first information (such as non-private information) in the second operating system, but it is not expected that the display interface of the first operating system displays the first information in the second operating system.
- the second information (such as private information).
- the screen of the second operating system can be divided into at least two areas, one area displays the first information, and the other area displays the second information.
- the first information projection screen is displayed in the target display area of the display interface of the first operating system.
- the screen of the second operating system can be divided into two left and right, and only the content displayed in the left area is displayed on the first operating system, and the content displayed in the right area is not displayed on the display interface of the first operating system.
- the specific display of the content of the local area in the second operating system can be flexibly configured, and is not limited to the left and right sides, which is not limited here.
- the target display area may be a full-screen area of the display interface of the first operating system.
- the second operating system can also be controlled to respond to the touch operation by touching the touch screen.
- the target display area is a partial area in the display interface of the first operating system.
- a partial area can be called a partial area or a non-full-screen area.
- the display interface of the first operating system not only displays the interface content of the second operating system, but also because the interface content only occupies a partial area in the display interface of the first operating system, the display interface of the first operating system is displayed in the display interface of the first operating system.
- Other areas may display the display content of the first operating system.
- the display content of the second operating system may be displayed on the display interface of the first operating system in a window display manner.
- the target display area is the area where the window is located.
- the target display area is the full-screen area, and when the window is reduced, the target display area is the non-full-screen area. It can be seen that the display content of the second operating system can be displayed in full screen or non-full screen in the display interface of the first operating system by scaling the window.
- the display content of the second operating system in the display interface of the first operating system in a windowed form for example, it may be to encode the picture signal of the second operating system into a video data stream, through HDMI/Type/VGA, etc. It is transmitted to the first operating system. After the first operating system obtains the data, the data stream is decoded, and then zoomed and displayed on the window. It should be understood that other manners are also possible, which will not be repeated here.
- the touch position information is the position information of the target display area in the touch frame coordinate system, and can be obtained by mapping the position information of the target display area in the system coordinate system of the first operating system to the touch frame coordinate system of the first operating system .
- the position information of the target display area under the touch frame coordinate system of the first operating system may be determined by the relationship between the touch frame coordinate system and the system coordinate system, which is referred to as touch position information. In this way, when the display content of the second operating system is displayed in the target display area of the display interface of the first operating system, the problem of not knowing how to make the second operating system respond to the touch operation can be solved.
- the touch position information can be determined by the first operating system.
- the first operating system may send the data required for calculating the touch position information to the processing module, and the processing module will perform the processing. calculate.
- the touch position information can also be determined by the first operating system and sent to the processing module. Since the raw data of the touch frame is reported at a high frequency, the first operating system calculates the touch position information, which can avoid the situation that the processing module cannot keep up with the efficiency and performance due to the low computing power.
- the touch position information is updated after the display position of the target display area is adjusted, so as to ensure that the touch data can be correctly allocated and responded.
- the adjustment of the display position of the target display area may be an adjustment that causes the touch position information to change. For example, one or more operations of moving, zooming, maximizing an area, minimizing an area, closing an area, etc. may be performed on the target display area.
- the touch position information may change with the state of the window.
- the adjustment of the display position of the target display area may be an operation on the container of the content of the second operating system.
- the container can be SurfaceView, which controls the window drag/zoom, and can control the position of SurfaceView.
- how to determine the touch position information may be determined by the relationship between the touch frame coordinate system of the first operating system and the system coordinate system of the first operating system.
- the relationship between the touch frame coordinate system and the system coordinate system may include a conversion relationship between the position of the area under the touch frame coordinate system and the position of the area under the system coordinate system.
- the width and height of the touch frame coordinate system depend on the precision of the touch screen. For example, if the X-axis precision of the touch frame is 0-40000 and the Y-axis precision is 0-40000, the touch frame coordinate system can be as shown in Figure 5B.
- the width and height of the system coordinate system depend on the resolution of the screen, and the system coordinate system is determined by the orientation of the system, which in turn can be determined by the rotation direction of the device.
- the width and height of the screen coordinate system depend on the resolution of the screen, and the screen coordinate system is fixed and will not change the origin as the device rotates. For example, for example, the width of a 4k screen is 3840 and the height is 2160, and the screen coordinate system is as shown in Figure 5C.
- the relationship between the preset touch frame coordinate system and the system coordinate system may include: the relationship between the preset system coordinate system and the screen coordinate system, and the preset screen coordinate system and the touch frame coordinate system Relationship.
- the preset relationship between the system coordinate system and the screen coordinate system may include: a conversion relationship between the position of the region under the system coordinate system of the first operating system and the position of the region under the screen coordinate system of the first operating system.
- the preset relationship between the screen coordinate system and the touch frame coordinate system may include: a conversion relationship between the position of the area under the screen coordinate system of the first operating system and the position of the area under the touch frame coordinate system of the first operating system.
- the relationship between the touch frame coordinate system and the system coordinate system is determined by the relationship between the system coordinate system and the screen coordinate system, and the relationship between the screen coordinate system and the touch frame coordinate system, which can improve the accuracy of position conversion.
- FIG. 5D it is a flowchart of determining touch position information according to an exemplary embodiment of the present application.
- the method describes how to determine touch position information.
- the method may It includes the following steps 5001 to 5003:
- step 5001 the system position information of the target display area under the system coordinate system of the first operating system is obtained through the system interface;
- step 5002 according to the relationship between the preset system coordinate system and the screen coordinate system, the screen position information of the target display area in the screen coordinate system of the first operating system is obtained from the system position information mapping;
- step 5003 according to the preset relationship between the screen coordinate system and the touch frame coordinate system, the touch position information of the target display area in the touch frame coordinate system of the first operating system is obtained from the screen position information mapping.
- the position information of the target display area under different coordinate systems is named as different names.
- the location information of the target display area in the system coordinate system of the first operating system is called system location information
- the location information of the target display area in the screen coordinate system of the first operating system is called screen location information
- the position information of the display area in the touch frame coordinate system of the first operating system is called touch position information.
- system location information can be obtained through a system interface.
- the coordinates getX() and getY() where the target display area is located can be obtained, and the height and width of the target display area can be obtained by getHeight() and getWidth().
- FIG. 5E it is a schematic diagram of system location information according to an exemplary embodiment of the present application. Assuming that the system location information is represented by LEFT_system, TOP_system, RIGHT_system, and BOTTOM_system where the target display area is located, then:
- the relationship between the system coordinate system and the screen coordinate system includes: the relationship between the system coordinate system and the screen coordinate system after the touch screen is rotated by a specified angle in a specified direction.
- the specified angle includes 0 degree, 90 degree, 180 degree and 270 degree.
- Figure 5F it is a 4k system, a system coordinate system shown in four rotation directions.
- the target display area may be first mapped from the system coordinate system to the screen coordinate system.
- the screen position information (LEFT_screen, TOP_screen, RIGHT_screen, BOTTOM_screen) is the system position information (LEFT_system, TOP_system, RIGHT_system, BOTTOM_system).
- FIG. 5G it is a comparative schematic diagram of the target display area in different coordinate systems after the device according to an exemplary embodiment of the present application is rotated 90 degrees clockwise. If the system location information is still used as the screen location information, the error shown in Figure 5G will appear.
- the screen position information of the target display area in the screen coordinate system of the first operating system is obtained from the system position information mapping.
- the screen position information (LEFT_screen, TOP_screen, RIGHT_screen, BOTTOM_screen) can be:
- ScreenWidth' is the screen width when the screen is vertically screened.
- the screen resolution is 3840*2160.
- the width of the screen is 2160; when the screen is horizontal, the width of the screen is 3840.
- the system coordinate system When the device is rotated 180 degrees clockwise, the system coordinate system is rotated 180 degrees counterclockwise relative to the screen coordinate system, and the system coordinate system is not equal to the screen coordinate system.
- the screen position information of the target display area in the screen coordinate system of the first operating system is obtained from the system position information mapping.
- the screen position information (LEFT_screen, TOP_screen, RIGHT_screen, BOTTOM_screen) can be:
- ScreenWidth is the width of the screen when the screen is horizontal
- ScreenHeight is the height of the screen when the screen is horizontal.
- the system coordinate system When the device is rotated 270 degrees clockwise, the system coordinate system is rotated 270 degrees counterclockwise relative to the screen coordinate system, and the system coordinate system is not equal to the screen coordinate system.
- the screen position information of the target display area in the screen coordinate system of the first operating system is obtained from the system position information mapping.
- the screen position information (LEFT_screen, TOP_screen, RIGHT_screen, BOTTOM_screen) can be:
- ScreenHeight' is the height of the screen when the screen is vertically screened.
- the display content of the second operating system is displayed in the target display area of the display interface of the first operating system. If touch is to be realized, the position information in the screen coordinate system of the target display area can be mapped to the position in the touch frame coordinate system information. Define the touch accuracy of the X axis as TOUCH_X and the touch accuracy of the Y axis as TOUCH_Y. Since the screen coordinate system and the touch coordinate system are coincident, there is a proportional relationship:
- TOP_touch/TOUCH_Y TOP_screen/ScreenHeight
- LEFT_touch LEFT_screen*TOUCH_X/ScreenWidth
- TOP_touch TOP_screen*TOUCH_Y/ScreenHeight
- the conversion relationship between the touch position information and the system position information can be obtained, and the system position information can be obtained from the coordinates getX() and getY() where the target display area is located, as well as the height getHeight() and width getWidth of the target display area. ( ) to obtain, for this purpose, the conversion relationship between the touch position information and getX(), getY(), getHeight() and getWidth() can be obtained.
- the device selection of 0 degrees, 90 degrees, 180 degrees, and 270 degrees is still taken as an example for illustration.
- TOP_touch (ScreenWidth'-getX()-getWidth())*TOUCH_Y/ScreenHeight
- LEFT_touch (ScreenWidth-getX()-getWidth())*TOUCH_X/ScreenWidth
- TOP_touch (ScreenHeight-getY()-getHeight())*TOUCH_Y/ScreenHeight
- LEFT_touch (ScreenHeight'-getY()-getHeight())*TOUCH_X/ScreenWidth
- the touch position information can be obtained.
- the touch position information After the touch position information is obtained, if a touch operation on the display interface of the first operating system is detected, it can be determined whether the touch operation is displayed in the target display area according to the relationship between the touch operation and the touch position information content operation, so that it can be determined whether to send the touch data corresponding to the touch operation to the second operating system according to the judgment result, so that the second operating system can respond to the touch operation.
- the touch data may also include the ID of the touch point, the touch state, and the like, which is not limited herein.
- the judgment condition may include that the touch operation is an operation on the target display area.
- a touch area on the target display area may also be a touch area on the target display area that is only responsive to the first operating system.
- a floating window and/or controls for controlling the display state of the target display area may be displayed on the target display area.
- the floating window may be a button for controlling the target display area, or a button for controlling the display interface of the first operating system, or the like.
- the target display area display state may be an area size state, an area position state, and the like.
- the controls may be maximized controls, minimized controls, close controls, etc., which are fixed on the target display area. Touch operations acting on the floating window and controls are responded to by the first operating system. In this way, the area where the floating window and the controls are located can be used as a shielding area, and touch operations on the shielding area are not responded to by the second operating system.
- FIG. 5K is a schematic diagram of a window provided with a floating window according to an exemplary embodiment of the present application.
- This schematic diagram takes the first operating system as the Android main system, and the second system as the external channel system as an example, the display content of the external channel system is displayed in the window of the Android main system, and a close button is suspended on the window. It is expected that when the user clicks the button, the touch event is given to the Android main system, not to other external channel systems. The user can click the close button on the display interface of the Android main system to close the above-mentioned window without expecting the external channel system to respond to the touch data.
- the judging condition that the touch operation is an operation on the displayed content in the target display area includes: the touch operation is an operation on the target display area, and is not an operation on the shielding area, the shielding area is an operation on the target display area.
- the area where the floating window is located on the target display area and/or the area where the controls that control the display state of the target display area are located.
- the shielding area may be one area or multiple areas, which are specifically configured according to requirements. How to determine the position information of the shielded area is similar to how to obtain the touch position information.
- the shielded area is also an area under the touch frame coordinate system, which can be determined based on the relationship between the touch frame coordinate system and the system coordinate system.
- the shielding area may be calculated by the first operating system, or may be calculated by a processing module different from the first operating system. For the specific determination method, reference may be made to the determination method of the touch position information, which will not be repeated here.
- This embodiment uses two sub-conditions to define whether the touch operation is an operation on the displayed content in the target display area, which can deal with the situation that the first operating system is expected to respond but the second operating system is not expected to respond to the touch operation in the target display area.
- the judging condition for whether the touch operation is an operation on the displayed content in the target display area may also include other conditions, which are configured according to requirements, and will not be repeated here.
- the determination can be made according to whether the position of the touch operation is within the target display area.
- the judging process may: when it is detected that the touch object is performing a touch drop event, the touch position of the touch object is within the target display area; the touch drop event is that the touch object is performing a continuous touch operation events that occur in the process.
- a complete touch operation performed by the touch object may be continuous, which may often include: a touch down event (ACTION_DOWN), multiple consecutive touch move events (ACTION_MOVE), and a touch up event (ACTION_UP).
- ACTION_DOWN touch down event
- ACTION_MOVE multiple consecutive touch move events
- ACTION_UP touch up event
- the touch position of the touch object changes and corresponds to multiple touch positions. These touch positions refer to the touch points on the screen generated by the touch object in the process of touching and moving on the screen of the device. s position.
- the touch object may be an operating body that can be recognized by the screen, such as a finger, a palm, or a stylus. In this example, as long as the touch down event occurs in the target display area, it can be considered that the current continuous touch operations are all operations on the target display area.
- whether the current touch operation is an operation on the target display area may be determined by whether the touch position of the touch object is within the target display area when the touch object performs a touch drop event. If the touch position of the touch object is within the target display area when the touch object performs a touch down event, it is determined that the current touch operation is an operation on the target display area.
- the touch data may be touch data when a touch down event, a touch movement event or a touch up event of the same touch object is detected. In this way, the integrity of one continuous touch operation can be guaranteed.
- the touch data of the touch operation is sent to the first operating system
- the first operating system responds to the touch operation, so that the first operating system responds to the current touch operation when the touch operation is not an operation on the displayed content in the target display area.
- a piece of touch data is either sent to the first operating system for response, or sent to the second operating system for response.
- the touch operation is not an operation on the displayed content in the target display area, it may be that the touch operation is not an operation on the target display area; or, the touch operation is an operation on the target display area and an operation on the shielding area, the shielding area is The area on the target display area where the floating window is located and/or the area where the controls that control the display state of the target display area are located.
- the screen content of the writing software can be projected to the main system, the main system displays the writing software content in a windowed form, and the user can control the writing of the external system on the interface of the main system software.
- the external system needs to transfer the screen to the main system for display after responding to the touch data, it will be slower than displaying on the main system directly. Therefore, in order to improve the writing speed, the external system and the main system can write at the same time, that is, two The system also responds to touch data so that when you write, you can see the handwriting on the main system that displays faster.
- the touch data can be sent to the first operating system, that is, when the touch data of the current touch operation is obtained, The touch data of the touch operation is directly sent to the first operating system, and the first operating system determines whether to respond to the touch operation; in the case where the touch operation is an operation for displaying content in the target display area Next, whether the first operating system responds to the touch operation is determined based on a preset policy. It can be seen that whether to respond to the touch operation can be determined by the first operating system.
- the MCU as the processing module can copy the touch data into two copies, one copy is directly transmitted to the first operating system, and the other copy is based on the judgment condition (touch operation is displayed on the target Whether or not to send to the second operating system is determined based on whether the judgment condition for the operation of displaying the content in the area is satisfied. In this way, the requirements of the two systems responding at the same time can be met, and whether to respond at the same time can be determined by the first operating system.
- the touch data may be converted into data suitable for the second operating system and then sent to the second operating system.
- a complete touch operation may include operations on the target display area and operations on areas other than the target display area. In order to maintain the integrity of the touch data, a complete set of touch data needs to be from Down to Up.
- the touch position of the touch object is within the target display area, and the continuous In the process of the touch operation, the touch position is in the target display area, and according to the preset relationship between the touch frame coordinate system of the first operating system and the coordinate system of the second operating system, the touch position is located in the target display area.
- the touch data is converted into touch data in the coordinate system of the second operating system. If the touch position of the touch object is within the target display area when the touch object performs a touch drop event, and the touch position is not within the target display area during the continuous touch operation, the touch position is not within the target display area.
- the touch data in the target display area is converted into the touch data closest to the edge of the target display area, and the touch data after the first conversion is obtained; and according to the preset touch frame coordinate system of the first operating system and the second operating system
- the relationship of the coordinate system, the touch data after the first conversion is converted into the touch data in the coordinate system of the second operating system, and the touch data after the second conversion is obtained.
- the second operating system may or may not have a corresponding touch screen. Therefore, the touch data needs to be converted into data suitable for the second operating system for use by the second operating system.
- the conversion process of touch data can be completed by the first operating system, or a processing module different from the first operating system, so that the second operating system can directly respond accordingly according to the converted touch data.
- the conversion process of touch data can also be completed by the second operating system, the touch data is directly sent to the second operating system, and the second operating system converts the touch data into the coordinate system of the second operating system touch data, and respond according to the converted touch data, thereby reducing the burden of the processing module.
- the full-screen content of the second operating system is displayed in the target display area, the The touch data corresponding to the touch operation is sent to the second operating system, including:
- the touch position of the touch object is within the target display area when the touch object performs a touch drop event, and there is a touch position within the target display area during the continuous touch operation, according to the preset No.
- the touch position of the touch object is within the target display area when the touch object performs a touch drop event, and the touch position is not within the target display area during the continuous touch operation, the touch position is not within the target display area.
- the touch data in the target display area is converted into touch data closest to the edge of the target display area; and according to the preset relationship between the touch frame coordinate system of the first operating system and the coordinate system of the second operating system, the conversion into The touch data closest to the edge of the target display area is converted into touch data under the coordinate system of the second operating system;
- the converted touch data in the coordinate system of the second operating system is sent to the second operating system.
- the actual touch track is represented by a thick dashed line
- the track of the touch data actually received by the second operating system is represented by a thick solid line.
- the touch data of the touch operation is converted into the touch data closest to the edge of the target display area, which can not only ensure the integrity of the data, but also avoid false responses caused by the converted touch data. It should be understood that, in other embodiments, other conversion methods may also be used to ensure the integrity of the continuous touch operation, which is specifically set according to requirements.
- the conversion of the touch data may be determined by the scaling relationship between the target display area and the area where the displayed content is located in the coordinate system of the second operating system.
- the area where the displayed content is located is the full-screen area of the second operating system.
- x is the x-axis data reported by the touch data acquisition module
- y is the y-axis data reported by the touch data acquisition module
- x_source is the touch x-axis data that needs to be reported to the second operating system
- y_source is the touch data that needs to be reported to the second operating system.
- SourceWidth is the width of the screen resolution of the device where the second operating system is located
- SourceHeight is the height of the screen resolution of the device where the second operating system is located.
- x_source (x-LEFT_touch)*SourceWidth/(RIGHT_touch-LEFT_touch)
- y_source (y-TOP_touch)*SourceHeight/(BOTTOM_touch-TOP_touch)
- the calculated x_source and y_source can be input into the second operating system.
- the touch position of the touch operation is not within the target display area, you need to convert the touch data of the touch operation into the touch data closest to the edge of the target display area. For example, you can first convert x and y in the formula into x1. and y1 and then substituted into the formula, which will not be repeated here.
- this embodiment when the second operating system is not displaying full-screen content, but part of the screen content on the screen of the second operating system is displayed on the display interface of the first operating system, this embodiment is called a partial area;
- the relationship between the display area and the partial screen area can be obtained by x_source and y_source.
- the partial screen area is the area of the content to be shared in the second operating system.
- the touch position of the touch object is within the target display area when the touch object performs a touch drop event, and there is a touch position within the target display area during the continuous touch operation, according to the preset No.
- the relationship between the coordinate system of the touch frame of an operating system and the coordinate range of the local area converts the touch data whose touch position is in the target display area into the touch data in the coordinate system of the second operating system;
- the local area The coordinate range of the area refers to the coordinate range of the local area in the coordinate system of the screen corresponding to the second operating system;
- the touch position of the touch object is within the target display area when the touch object performs a touch drop event, and the touch position is not within the target display area during the continuous touch operation, the touch position is not within the target display area.
- the touch data in the target display area is converted into the touch data closest to the edge of the target display area; and according to the preset relationship between the touch frame coordinate system of the first operating system and the coordinate range of the local area, the conversion into The touch data closest to the edge of the target display area is converted into touch data under the coordinate system of the second operating system;
- the converted touch data in the coordinate system of the second operating system is sent to the second operating system.
- the full-screen content of the second operating system is displayed in the target display area, and is calculated by using the resolution SourceWidth/SourceHeight of the display screen of the second operating system.
- SourceWidth/SourceHeight cannot be directly calculated, and the coordinate range of the partial area on the second operating system screen needs to be known.
- the local area can be obtained by the second operating system and transmitted to the first operating system, and then transmitted to the MCU by the first operating system; the coordinate range of the local area on the screen of the second operating system can be composed of the following four coordinate values:
- x_source (x-LEFT_touch)*(RIGHT_system2_window-LEFT_system2_window)/(RIGHT_touch-LEFT_touch)+LEFT_system2_window
- y_source (y-TOP_touch)*(BOTTOM_system2_window-TOP_system2_window)/ (BOTTOM_touch-TOP_touch)+TOP_touch
- the calculated x_source and y_source can be input into the second operating system.
- Touch data whose touch position is within the target display area can be converted into touch data in the coordinate system of the second operating system.
- the touch position of the touch object is within the target display area, and if there is a touch position that is not within the target display area during the continuous touch operation, the above-mentioned implementation can be performed.
- the calculation method of the example is to convert the touch data of the touch operation into the touch data closest to the edge of the target display area, and then convert the converted touch data closest to the edge of the target display area into the touch data closest to the edge of the target display area according to the above calculation method. Touch data in the system's coordinate system.
- the window of the second operating system is further provided with a button area for switching external channel signals, for example: PC, Android box, HDMI, Type-c and so on.
- peripheral devices there may be a need to switch peripheral devices, and in view of this, the peripheral device can be switched to the first operating system or the second operating system through USB Hub control.
- External devices can be USB flash drives, cameras, microphones, and other devices.
- FIG. 3 is the same as the related technology in FIG. 5A . In order to save space, no detailed description is given in the embodiment of FIG. 3 .
- the first operating system may be the Android main system in the display device, and the second operating system may be the external channel system.
- the processing module may be an MCU, and the display content from the second operating system is displayed in a window of the display interface of the first operating system.
- the Android main system sends the calculated touch position information of the window to the MCU. After the MCU receives the touch data sent by the touch data acquisition module, it can determine whether the current touch operation is an operation of the content displayed in the window.
- the present application also provides embodiments of a touch data processing apparatus, an electronic device to which it is applied, and a computer storage medium.
- FIG. 7 is a block diagram of a touch data processing apparatus according to an exemplary embodiment of the present application
- the first operating system and the second operating system are operating systems in the same device or in different devices
- the apparatus includes :
- a data display module 72 configured to display the display content from the second operating system on the display interface of the first operating system, the display content being displayed in the target display area of the display interface of the first operating system;
- the operation determination module 74 is configured to, after the touch screen on the device where the first operating system is located is rotated by a preset angle, if a touch operation on the touch screen is detected, determine whether the touch operation is on the target display area The operation of displaying the content;
- the data processing module 76 is configured to send the touch data corresponding to the touch operation to the second operating system if the touch operation is an operation on the content displayed in the target display area, so that the second operating system can Respond to touch operations.
- the target display area is a partial area in the display interface of the first operating system.
- the display content is displayed on the display interface of the first operating system in a window display manner.
- the data display module 72 is further configured to:
- If the touch operation is an operation on the display content in the target display area, display the updated display content in the target display area;
- the updated display content is obtained by the second operating system updating the display content based on the touch operation, or the first operating system is obtained by updating the display content based on the touch operation.
- a floating window and/or controls for controlling the display state of the target display area are displayed on the target display area, and the touch operation acting on the floating window and the controls is responded by the first operating system .
- the data processing module 76 is further configured to:
- the touch operation is an operation on the display content in the adjusted target display area
- the touch data of the touch operation is sent to the second operating system, so that the second The operating system responds to the touch operation.
- FIG. 8 it is a block diagram of another touch data processing apparatus shown in the present application according to an exemplary embodiment.
- the first operating system and the second operating system are operating systems in the same device or in different devices. include:
- a content display module 82 configured to display the display content from the second operating system on the display interface of the first operating system, the display content being displayed in the target display area of the display interface of the first operating system;
- the position obtaining module 84 is configured to obtain touch position information of the target display area if the touch screen on the device where the first operating system is located is rotated by a preset angle, the touch position information is based on placing the target display area in the first position.
- the position information under the system coordinate system of an operating system is mapped to the touch frame coordinate system of the first operating system and obtained;
- the operation determination module 86 is configured to, when a touch operation for the display interface of the first operating system is detected, according to the relationship between the touch operation and the touch position information, determine whether the touch operation is to display the content in the target display area operation;
- the data processing module 88 is configured to send the touch data corresponding to the touch operation to the second operating system if the touch operation is an operation on the displayed content in the target display area, so that the second operating system can Respond to touch operations.
- the target display area is a partial area in the display interface of the first operating system.
- the display content is displayed on the display interface of the first operating system in a window display manner.
- the touch position information is updated after the display position of the target display area is adjusted.
- the apparatus further includes a position calculation module (not shown in FIG. 8 ) for:
- the screen position information of the target display area in the screen coordinate system of the first operating system is obtained from the system position information mapping
- the touch position information of the target display area in the touch frame coordinate system of the first operating system is obtained from the screen position information mapping.
- the touch frame coordinate system of the first operating system does not change the origin with the rotation of the touch screen, and the screen coordinate system of the first operating system does not change the origin with the rotation of the touch screen.
- the system coordinate system of the operating system will change the origin with the rotation of the touch screen; the touch screen is the touch screen on the device where the first operating system is located;
- the relationship between the system coordinate system and the screen coordinate system includes: the relationship between the system coordinate system and the screen coordinate system after the touch screen is rotated by a specified angle in a specified direction.
- a floating window and/or controls for controlling the display state of the target display area are displayed on the target display area, and the touch operation acting on the floating window and the controls is responded by the first operating system .
- the judgment condition for the operation determination module 86 to determine that the touch operation is an operation on the displayed content in the target display area includes:
- the touch operation is an operation on the target display area; or,
- the touch operation is an operation on the target display area, not an operation on a shielded area, where the shielded area is the area where the floating window is located on the target display area and/or the area where the controls that control the display state of the target display area are located .
- the process of determining that the touch operation is an operation on the target display area includes:
- the touch position of the touch object is within the target display area;
- the touch down event is an event that occurs when the touch object performs continuous touch operations.
- the data processing module 88 is specifically configured to: if the full-screen content of the second operating system is displayed in the target display area, execute:
- the touch position of the touch object is within the target display area when the touch object performs a touch drop event, and there is a touch position within the target display area during the continuous touch operation, according to the preset No.
- the touch position of the touch object is within the target display area when the touch object performs a touch drop event, and the touch position is not within the target display area during the continuous touch operation, the touch position is not within the target display area.
- the touch data in the target display area is converted into touch data closest to the edge of the target display area; and according to the preset relationship between the touch frame coordinate system of the first operating system and the coordinate system of the second operating system, the conversion into The touch data closest to the edge of the target display area is converted into touch data under the coordinate system of the second operating system;
- the converted touch data in the coordinate system of the second operating system is sent to the second operating system.
- the data processing module 88 is specifically configured to: if the content of the partial area of the screen of the second operating system is displayed in the target display area, execute:
- the touch position of the touch object is within the target display area when the touch object performs a touch drop event, and there is a touch position within the target display area during the continuous touch operation, according to the preset No.
- the relationship between the coordinate system of the touch frame of an operating system and the coordinate range of the local area converts the touch data whose touch position is in the target display area into the touch data in the coordinate system of the second operating system;
- the local area The coordinate range of the area refers to the coordinate range of the local area in the coordinate system of the screen corresponding to the second operating system;
- the touch position of the touch object is within the target display area when the touch object performs a touch drop event, and the touch position is not within the target display area during the continuous touch operation, the touch position is not within the target display area.
- the touch data in the target display area is converted into the touch data closest to the edge of the target display area; and according to the preset relationship between the touch frame coordinate system of the first operating system and the coordinate range of the local area, the conversion into The touch data closest to the edge of the target display area is converted into touch data under the coordinate system of the second operating system;
- the converted touch data in the coordinate system of the second operating system is sent to the second operating system.
- the data processing module 88 is further configured to:
- the touch operation is not an operation on the displayed content in the target display area
- the touch data of the touch operation is sent to the first operating system, and the first operating system responds to the touch operation; or,
- the touch data of the touch operation is directly sent to the first operating system, and the first operating system determines whether to respond to the touch operation; in the case where the touch operation is an operation for displaying content in the target display area Next, whether the first operating system responds to the touch operation is determined based on a preset policy.
- the apparatus further includes a device switching module (not shown in FIG. 8 ), used for: switching the external device to the first operating system or the second operating system through USB Hub control.
- a device switching module (not shown in FIG. 8 ), used for: switching the external device to the first operating system or the second operating system through USB Hub control.
- an embodiment of the present application further provides an electronic device, the device includes a memory and a processor; wherein, the memory stores a computer program, and the computer program is adapted to be loaded by the processor and execute any of the above Item's touch data processing method steps.
- the electronic device is an intelligent interactive tablet
- the first operating system is an Android system
- the second operating system is a Windows system or a system on an external device connected to the intelligent interactive tablet.
- an embodiment of the present application further provides a computer storage medium, where the computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing any of the above steps of the touch data processing method.
- the application may take the form of a computer program product embodied on one or more storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having program code embodied therein.
- Computer-usable storage media includes permanent and non-permanent, removable and non-removable media, and storage of information can be accomplished by any method or technology.
- Information may be computer readable instructions, data structures, modules of programs, or other data.
- Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cartridges, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.
- PRAM phase-change memory
- SRAM static random access memory
- DRAM dynamic random access memory
- RAM random access memory
- ROM read only memory
- EEPROM Electrically Erasable Programmable Read Only Memory
- Flash Memory or other memory technology
- CD-ROM Compact Disc Read Only Memory
- CD-ROM Compact Disc Read Only Memory
- DVD Digital Versatile Disc
- Magnetic tape cartridges magnetic tape magnetic disk storage or other magnetic storage devices or any other non-
- the embodiments of the touch data processing apparatus of the present application can be applied to electronic devices.
- the apparatus embodiment may be implemented by software, or may be implemented by hardware or a combination of software and hardware.
- a device in a logical sense is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory for operation by the processor of the electronic device where the device is located.
- FIG. 9 it is a hardware structure diagram of the electronic device where the touch data processing apparatus of the present application is located, except for the processor 910 , the network interface 920 , the memory 930 , and the non-volatile device shown in FIG. 9 .
- the electronic device where the apparatus is located in the embodiment may also include other hardware generally according to the actual function of the electronic device, which will not be repeated here.
Abstract
Description
Claims (21)
- 一种触摸数据处理方法,其中,第一操作系统和第二操作系统为相同设备或不同设备中的操作系统,所述方法包括:在第一操作系统的显示界面显示来自第二操作系统的显示内容,所述显示内容显示在所述第一操作系统的显示界面的目标显示区域中;在第一操作系统所在设备上的触摸屏按预设角度旋转后,若检测到针对所述触摸屏的触摸操作时,确定所述触摸操作是否为对所述目标显示区域内显示内容的操作;如果是,将所述触摸操作对应的触摸数据发送至第二操作系统,以使第二操作系统对所述触摸操作进行响应。
- 根据权利要求1任一项所述的方法,其中,所述方法还包括:如果所述触摸操作是对所述目标显示区域内显示内容的操作,在所述目标显示区域内显示更新后的显示内容;其中,所述更新后的显示内容是第二操作系统基于所述触摸操作更新所述显示内容所得,或是第一操作系统基于所述触摸操作更新所述显示内容所得。
- 根据权利要求1所述的方法,其中,在所述目标显示区域上显示有悬浮窗和/或控制目标显示区域显示状态的控件,作用在所述悬浮窗和控件上的触摸操作由第一操作系统响应。
- 根据权利要求1所述的方法,其中,所述方法还包括:当所述目标显示区域的显示位置调整后,若触摸操作是对调整后的目标显示区域内的显示内容的操作,则将所述触摸操作的触摸数据发送至第二操作系统,以使第二操作系统对所述触摸操作进行响应。
- 一种触摸数据处理方法,其中,第一操作系统和第二操作系统为相同设备或不同设备中的操作系统,所述方法包括:在第一操作系统的显示界面显示来自第二操作系统的显示内容,所述显示内容显示在所述第一操作系统的显示界面的目标显示区域中;若第一操作系统所在设备上的触摸屏按预设角度旋转,获取所述目标显示区域的触控位置信息,所述触控位置信息基于将所述目标显示区域在第一操作系统的系统坐标系下的位置信息映射到第一操作系统的触摸框坐标系下获得;当检测到针对第一操作系统的显示界面的触摸操作时,依据所述触摸操作与触控位置信息的关系,确定所述触摸操作是否为对所述目标显示区域内显示内容的操作;如果是,将所述触摸操作对应的触摸数据发送至第二操作系统,以使第二操作系统 对所述触摸操作进行响应。
- 根据权利要求5所述的方法,其中,所述触控位置信息在所述目标显示区域的显示位置调整后进行更新。
- 根据权利要求5任一项所述的方法,其中,所述触控位置信息的计算过程包括:通过系统接口获得所述目标显示区域在第一操作系统的系统坐标系下的系统位置信息;依据预设的系统坐标系与屏幕坐标系的关系,由所述系统位置信息映射获得所述目标显示区域在第一操作系统的屏幕坐标系中的屏幕位置信息;依据预设的屏幕坐标系与触摸框坐标系的关系,由所述屏幕位置信息映射获得所述目标显示区域在第一操作系统的触摸框坐标系中的触控位置信息。
- 根据权利要求7所述的方法,其中,第一操作系统的触摸框坐标系不会随着触摸屏的旋转而改变原点,第一操作系统的屏幕坐标系不会随着触摸屏的旋转而改变原点,第一操作系统的系统坐标系会随着触摸屏的旋转而改变原点;所述触摸屏为第一操作系统所在设备上的触摸屏;所述系统坐标系与屏幕坐标系的关系包括:在所述触摸屏沿指定方向旋转指定角度后系统坐标系与屏幕坐标系间的关系。
- 根据权利要求5所述的方法,其中,在所述目标显示区域上显示有悬浮窗和/或控制目标显示区域显示状态的控件,作用在所述悬浮窗和控件上的触摸操作由第一操作系统响应。
- 根据权利要求5或9所述的方法,其中,所述触摸操作是对所述目标显示区域内显示内容的操作的判断条件包括:所述触摸操作是对所述目标显示区域的操作;或,所述触摸操作是对所述目标显示区域的操作、且不是对屏蔽区域的操作,所述屏蔽区域是所述目标显示区域上悬浮窗所在区域和/或控制目标显示区域显示状态的控件所在区域。
- 根据权利要求10所述的方法,其中,所述触摸操作是对所述目标显示区域的操作的判断过程包括:检测到触摸对象在执行触摸落下事件时,所述触摸对象的触摸位置在所述目标显示区域内;所述触摸落下事件是触摸对象在执行连续的触摸操作过程中发生的事件。
- 根据权利要求11所述的方法,其中,若第二操作系统的全屏内容显示在所述目标显示区域中,所述将所述触摸操作对应的触摸数据发送至第二操作系统,包括:若所述触摸对象执行触摸落下事件时所述触摸对象的触摸位置在所述目标显示区域内、且所述连续的触摸操作过程中有触摸位置在所述目标显示区域内,依据预设的第一操作系统的触摸框坐标系和第二操作系统的坐标系的关系,将触摸位置在所述目标显示区域内的触摸数据转换成在第二操作系统的坐标系下的触摸数据;若所述触摸对象执行触摸落下事件时所述触摸对象的触摸位置在所述目标显示区域内、且所述连续的触摸操作过程中有触摸位置不在所述目标显示区域内,将触摸位置不在所述目标显示区域内的触摸数据转换成最接近目标显示区域边缘的触摸数据;并依据预设的第一操作系统的触摸框坐标系和第二操作系统的坐标系的关系,将所述转换成的最接近目标显示区域边缘的触摸数据转换成在第二操作系统的坐标系下的触摸数据;将所述转换成的在第二操作系统的坐标系下的触摸数据发送至第二操作系统。
- 根据权利要求11所述的方法,其中,若第二操作系统屏幕的局部区域的内容显示在所述目标显示区域中,所述将所述触摸操作对应的触摸数据发送至第二操作系统,包括:若所述触摸对象执行触摸落下事件时所述触摸对象的触摸位置在所述目标显示区域内、且所述连续的触摸操作过程中有触摸位置在所述目标显示区域内,依据预设的第一操作系统的触摸框坐标系和所述局部区域的坐标范围的关系,将触摸位置在所述目标显示区域内的触摸数据转换成在第二操作系统的坐标系下的触摸数据;所述局部区域的坐标范围是指所述局部区域在第二操作系统对应屏幕的坐标系中的坐标范围;若所述触摸对象执行触摸落下事件时所述触摸对象的触摸位置在所述目标显示区域内、且所述连续的触摸操作过程中有触摸位置不在所述目标显示区域内,将触摸位置不在所述目标显示区域内的触摸数据转换成最接近目标显示区域边缘的触摸数据;并依据预设的第一操作系统的触摸框坐标系和所述局部区域的坐标范围的关系,将所述转换成的最接近目标显示区域边缘的触摸数据转换成在第二操作系统的坐标系下的触摸数据;将所述转换成的在第二操作系统的坐标系下的触摸数据发送至第二操作系统。
- 根据权利要求5所述的方法,其中,所述方法还包括:若所述触摸操作不是对所述目标显示区域内显示内容的操作,则将所述触摸操作的触摸数据发送至第一操作系统,由第一操作系统对所述触摸操作进行响应;或,直接将所述触摸操作的触摸数据发送至第一操作系统,由第一操作系统确定是否对所述触摸操作进行响应;在所述触摸操作是对所述目标显示区域内显示内容的操作的情况下,第一操作系统是否响应触摸操作基于预设的策略决定。
- 根据权利要求5所述的方法,其中,所述方法还包括:通过USB Hub控制将外接设备切换至第一操作系统或第二操作系统。
- 一种触摸数据处理方法,其中,第一操作系统和第二操作系统为相同设备或不同设备中的操作系统,所述方法包括:第一操作系统所在设备在第一操作系统的显示界面显示来自第二操作系统的显示内容,所述显示内容显示在所述第一操作系统的显示界面的目标显示区域中;第一操作系统所在设备在第一操作系统所在设备上的触摸屏按预设角度旋转后,若检测到针对所述触摸屏的触摸操作时,确定所述触摸操作是否为对所述目标显示区域内显示内容的操作;如果是,将所述触摸操作对应的触摸数据发送至第二操作系统;第二操作系统所在设备依据接收到的触摸数据对所述触摸操作进行响应。
- 一种触摸数据处理装置,其中,第一操作系统和第二操作系统为相同设备或不同设备中的操作系统,所述装置包括:数据显示模块,用于在第一操作系统的显示界面显示来自第二操作系统的显示内容,所述显示内容显示在所述第一操作系统的显示界面的目标显示区域中;操作确定模块,用于在第一操作系统所在设备上的触摸屏按预设角度旋转后,若检测到针对所述触摸屏的触摸操作时,确定所述触摸操作是否为对所述目标显示区域内显示内容的操作;数据处理模块,用于如果所述触摸操作是对所述目标显示区域内显示内容的操作,将所述触摸操作对应的触摸数据发送至第二操作系统,以使第二操作系统对所述触摸操作进行响应。
- 一种触摸数据处理装置,其中,第一操作系统和第二操作系统为相同设备或不同设备中的操作系统,所述装置包括:内容显示模块,用于在第一操作系统的显示界面显示来自第二操作系统的显示内容,所述显示内容显示在所述第一操作系统的显示界面的目标显示区域中;位置获得模块,用于获得触控位置信息,所述触控位置信息基于将所述目标显示区域在第一操作系统的系统坐标系下的位置信息映射到第一操作系统的触摸框坐标系下获得;操作确定模块,用于当检测到针对第一操作系统的显示界面的触摸操作时,依据触摸操作与触控位置信息的关系,确定所述触摸操作是否为对所述目标显示区域内显示内容的操作;数据处理模块,用于如果所述触摸操作是对所述目标显示区域内显示内容的操作,将所述触摸操作对应的触摸数据发送至第二操作系统,以使第二操作系统对所述触摸操 作进行响应。
- 一种计算机存储介质,其中,所述计算机存储介质存储有多条指令,所述指令适于由处理器加载并执行如权利要求1至16任一项的方法步骤。
- 一种电子设备,其中,包括存储器和处理器;其中,所述存储器存储有计算机程序,所述计算机程序适于由所述处理器加载并执行如权利要求1至16任一项的方法步骤。
- 根据权利要求20所述的设备,其中,所述电子设备为智能交互平板,所述第一操作系统为Android系统,第二操作系统为Windows系统或与所述智能交互平板连接的外接设备上的系统。
Priority Applications (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
AU2020462903A AU2020462903A1 (en) | 2020-08-14 | 2020-12-30 | Touch data processing method and apparatus, and device and storage medium |
KR1020227025840A KR20220113541A (ko) | 2020-08-14 | 2020-12-30 | 터치 데이터 처리 방법, 장치, 디바이스 및 저장 매체 |
JP2022564176A JP2023522267A (ja) | 2020-08-14 | 2020-12-30 | タッチデータ処理方法、装置、デバイス及び記憶媒体 |
EP20949457.4A EP4116814A4 (en) | 2020-08-14 | 2020-12-30 | METHOD AND APPARATUS FOR PROCESSING TOUCH DATA, DEVICE AND RECORDING MEDIUM |
US17/953,331 US20230018519A1 (en) | 2020-08-14 | 2022-09-27 | Touch data processing method, apparatus, device and storage medium |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010821331.7 | 2020-08-14 | ||
CN202010821331.7A CN112035048B (zh) | 2020-08-14 | 2020-08-14 | 触摸数据处理方法、装置、设备及存储介质 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/953,331 Continuation US20230018519A1 (en) | 2020-08-14 | 2022-09-27 | Touch data processing method, apparatus, device and storage medium |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022032966A1 true WO2022032966A1 (zh) | 2022-02-17 |
Family
ID=73577917
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/139610 WO2022032966A1 (zh) | 2020-08-14 | 2020-12-30 | 触摸数据处理方法、装置、设备及存储介质 |
Country Status (7)
Country | Link |
---|---|
US (1) | US20230018519A1 (zh) |
EP (1) | EP4116814A4 (zh) |
JP (1) | JP2023522267A (zh) |
KR (1) | KR20220113541A (zh) |
CN (1) | CN112035048B (zh) |
AU (1) | AU2020462903A1 (zh) |
WO (1) | WO2022032966A1 (zh) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112035048B (zh) * | 2020-08-14 | 2022-03-25 | 广州视源电子科技股份有限公司 | 触摸数据处理方法、装置、设备及存储介质 |
CN115016697A (zh) * | 2021-09-08 | 2022-09-06 | 荣耀终端有限公司 | 投屏方法、计算机设备、可读存储介质和程序产品 |
CN113625889B (zh) * | 2021-10-11 | 2022-02-08 | 广东新空电子科技股份有限公司 | 一种基于触控屏的触控笔迹生成方法及装置 |
CN114489544A (zh) * | 2022-01-27 | 2022-05-13 | 京东方科技集团股份有限公司 | 一种显示方法、智能交互一体机、计算机设备和存储介质 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110377250A (zh) * | 2019-06-05 | 2019-10-25 | 华为技术有限公司 | 一种投屏场景下的触控方法及电子设备 |
CN110381195A (zh) * | 2019-06-05 | 2019-10-25 | 华为技术有限公司 | 一种投屏显示方法及电子设备 |
CN110662095A (zh) * | 2019-08-28 | 2020-01-07 | 北京小米移动软件有限公司 | 投屏处理方法、装置、终端及存储介质 |
US20200133475A1 (en) * | 2018-10-29 | 2020-04-30 | Asustek Computer Inc. | Control method, electronic device and non-transitory computer readable recording medium device |
CN112035048A (zh) * | 2020-08-14 | 2020-12-04 | 广州视源电子科技股份有限公司 | 触摸数据处理方法、装置、设备及存储介质 |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060155670A1 (en) * | 2005-01-13 | 2006-07-13 | Forlenza Randolph M | Method for queuing files to be sent to an application |
CN104020989B (zh) * | 2014-05-06 | 2017-10-03 | 深信服科技股份有限公司 | 基于远程应用的控制方法和系统 |
CN106095084A (zh) * | 2016-06-06 | 2016-11-09 | 乐视控股(北京)有限公司 | 投屏方法及装置 |
US10009933B2 (en) * | 2016-09-02 | 2018-06-26 | Brent Foster Morgan | Systems and methods for a supplemental display screen |
CN110597474B (zh) * | 2019-09-12 | 2022-03-25 | 联想(北京)有限公司 | 信息处理方法及电子设备 |
-
2020
- 2020-08-14 CN CN202010821331.7A patent/CN112035048B/zh active Active
- 2020-12-30 WO PCT/CN2020/139610 patent/WO2022032966A1/zh unknown
- 2020-12-30 JP JP2022564176A patent/JP2023522267A/ja active Pending
- 2020-12-30 KR KR1020227025840A patent/KR20220113541A/ko unknown
- 2020-12-30 AU AU2020462903A patent/AU2020462903A1/en active Pending
- 2020-12-30 EP EP20949457.4A patent/EP4116814A4/en active Pending
-
2022
- 2022-09-27 US US17/953,331 patent/US20230018519A1/en active Pending
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20200133475A1 (en) * | 2018-10-29 | 2020-04-30 | Asustek Computer Inc. | Control method, electronic device and non-transitory computer readable recording medium device |
CN110377250A (zh) * | 2019-06-05 | 2019-10-25 | 华为技术有限公司 | 一种投屏场景下的触控方法及电子设备 |
CN110381195A (zh) * | 2019-06-05 | 2019-10-25 | 华为技术有限公司 | 一种投屏显示方法及电子设备 |
CN110662095A (zh) * | 2019-08-28 | 2020-01-07 | 北京小米移动软件有限公司 | 投屏处理方法、装置、终端及存储介质 |
CN112035048A (zh) * | 2020-08-14 | 2020-12-04 | 广州视源电子科技股份有限公司 | 触摸数据处理方法、装置、设备及存储介质 |
Also Published As
Publication number | Publication date |
---|---|
JP2023522267A (ja) | 2023-05-29 |
US20230018519A1 (en) | 2023-01-19 |
CN112035048B (zh) | 2022-03-25 |
KR20220113541A (ko) | 2022-08-12 |
EP4116814A4 (en) | 2023-11-15 |
CN112035048A (zh) | 2020-12-04 |
AU2020462903A1 (en) | 2023-02-02 |
EP4116814A1 (en) | 2023-01-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2022032966A1 (zh) | 触摸数据处理方法、装置、设备及存储介质 | |
US10095316B2 (en) | Scrolling and zooming of a portable device display with device motion | |
US9335888B2 (en) | Full 3D interaction on mobile devices | |
CN110941383B (zh) | 双屏幕显示方法、装置、设备及存储介质 | |
US7486274B2 (en) | Method for stabilizing and precisely locating pointers generated by handheld direct pointing devices | |
US9223416B2 (en) | Display apparatus, remote controlling apparatus and control method thereof | |
US8144123B2 (en) | Dynamically controlling a cursor on a screen when using a video camera as a pointing device | |
US10585581B2 (en) | Controlling display object on display screen | |
US20140267049A1 (en) | Layered and split keyboard for full 3d interaction on mobile devices | |
KR102598082B1 (ko) | 영상 표시 장치, 모바일 장치 및 그 동작방법 | |
US10979700B2 (en) | Display control apparatus and control method | |
WO2021068365A1 (zh) | 智能交互平板的页面交互方法、装置、设备和存储介质 | |
US20060168528A1 (en) | Method for arranging user interface glyphs on displays | |
US8243101B2 (en) | Image display processing apparatus, image display processing method, and recording medium | |
US20090213067A1 (en) | Interacting with a computer via interaction with a projected image | |
WO2022218152A1 (zh) | 窗口切换方法、存储介质及相关设备 | |
CN110888581A (zh) | 元素传递方法、装置、设备及存储介质 | |
WO2022242013A1 (zh) | 笔迹擦除方法、装置、交互平板及存储介质 | |
WO2023065939A1 (zh) | 一种触控响应方法、装置、交互平板及存储介质 | |
US8941584B2 (en) | Apparatus, system, and method for simulating physical movement of a digital image | |
Procházka et al. | Mainstreaming gesture based interfaces | |
TWM464735U (zh) | 觸控螢幕之點擊操作結構 | |
TW201430680A (zh) | 觸控螢幕之點擊操作結構 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20949457 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 20227025840 Country of ref document: KR Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 2020949457 Country of ref document: EP Effective date: 20221005 |
|
ENP | Entry into the national phase |
Ref document number: 2022564176 Country of ref document: JP Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 2020462903 Country of ref document: AU Date of ref document: 20201230 Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |