WO2024037421A1 - Procédé de fonctionnement, dispositif électronique et support - Google Patents

Procédé de fonctionnement, dispositif électronique et support Download PDF

Info

Publication number
WO2024037421A1
WO2024037421A1 PCT/CN2023/112297 CN2023112297W WO2024037421A1 WO 2024037421 A1 WO2024037421 A1 WO 2024037421A1 CN 2023112297 W CN2023112297 W CN 2023112297W WO 2024037421 A1 WO2024037421 A1 WO 2024037421A1
Authority
WO
WIPO (PCT)
Prior art keywords
electronic device
information
video
interface
user
Prior art date
Application number
PCT/CN2023/112297
Other languages
English (en)
Chinese (zh)
Inventor
胡怡洁
严凯凯
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2024037421A1 publication Critical patent/WO2024037421A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04886—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B20/00—Signal processing not specific to the method of recording or reproducing; Circuits therefor
    • G11B20/10—Digital recording or reproducing

Definitions

  • the present application relates to the field of communication technology, and in particular to an operating method, electronic equipment and media.
  • electronic devices such as smartphones have been widely used in various customer groups.
  • electronic devices such as smartphones are also widely used in middle-aged and elderly people.
  • most users in the middle-aged and elderly people do some operations while using mobile phones. Not very familiar with the problem, or unable to perform some operations at all, such as not being able to connect to Bluetooth and WIFI, not being able to adjust the font display size of the mobile phone, etc.
  • family members are not around and cannot provide on-site guidance.
  • other users may not be able to use some operations, resulting in users being unable to perform certain operations.
  • FIG. 1a In the existing technology, some solutions are shown in Figure 1a.
  • User A generally clicks the screen recording control 101 of the mobile phone 100 to implement the process of performing the target operation, such as the process of performing some Bluetooth connection or WIFI connection operations in the settings application 102.
  • the video 103 is sent to the mobile phone 200 of user B who needs to learn the operation through communication software.
  • the user who needs to learn the operation can learn how to perform the target operation according to the video content.
  • user B easily forgets how to operate after watching the video, so he needs to watch the video repeatedly to learn. In this way, it is more cumbersome for users to implement the above operations and reduces the user experience.
  • embodiments of the present application provide an operating method, electronic equipment, and media.
  • embodiments of the present application provide an operating method applied to a first electronic device.
  • the method includes: obtaining a first video and first information associated with the first video, wherein the first information includes a screen
  • the interface information of at least one display interface recorded during the recording process and the identification information of the target control operated by the user on the at least one display interface; play the first video, and during the process of playing the first video, based on the perform the operation on the first information.
  • the first video may refer to the recorded video mentioned in the embodiment of this application.
  • the first electronic device can perform the same operation as the operation performed by the user on the electronic device in the recorded video, which can eliminate the need for the user to operate by himself and improve the convenience of operation and user experience.
  • the first information includes the identification information of the target control operated by the user on the display interface, where the identification information of the target space may be the layout ID information of the target space, since the identification information of each operation target is unique. Even if the interface settings of the first electronic device and the electronic device in the recorded video are different, the first electronic device can still accurately find the corresponding control according to the identification of the target control and perform the corresponding operation, effectively improving the accuracy of the operation.
  • the operation is consistent with the user's operation on the at least one display interface in the first video.
  • the first video and the first information are stored as the same file, or the first video and the first information are stored as different files.
  • the method includes: the first information also includes the user's operation mode of the target control on the at least one display interface.
  • performing an operation based on the first information includes: obtaining interface information of a first display interface in the first information, and determining the interface information of the first display interface based on the interface information of the first display interface.
  • the first to-be-operated interface of the electronic device obtains identification information of a target control operated by the user on the first display interface, and determines the first operation based on the identification information of the target control operated by the user on the first display interface.
  • the to-be-operated control on the to-be-operated interface obtains the user's operating mode for the target control on the first display interface, and controls the to-be-operated control based on the user's operating mode for the target control on the first display interface.
  • Controls perform corresponding operations; Obtain the interface information of the second display interface in the first information, and determine the second to-be-operated interface of the electronic device based on the interface information of the second display interface.
  • the first information when the operation mode of the target control is a multi-state operation mode, the first information further includes auxiliary information corresponding to the operation mode.
  • the multi-state operation mode includes a typing operation and an adjustment operation
  • the auxiliary information corresponding to the typing operation includes input content
  • the auxiliary information corresponding to the adjustment operation includes an adjustment state
  • the adjustment may include sliding operations, such as sliding progress bars, etc.
  • the second electronic device when the second electronic device performs operations with multiple states such as sliding and typing, it can accurately obtain the operation that should be performed on the target control based on the auxiliary information, thereby improving the accuracy of the operation.
  • the interface information includes an interface package name, an interface application name, and an interface activity name.
  • the first electronic device includes a first display area and a second display area
  • the first display area displays the playback screen of the first video
  • the second display area displays the first electronic device performing the operation. Operation process.
  • the first display area overlaps a partial area of the second display area, or the second display area overlaps a partial area of the first display area.
  • the second display area and the first display area are displayed in separate screens.
  • the third electronic device is connected to the first electronic device, the third electronic device displays the playback screen of the first video, and the first electronic device displays the first video.
  • An electronic device performs the operation process of the operation.
  • the third electronic device is connected to the first electronic device, the third electronic device displays the operation process of the first electronic device performing the operation, and the first electronic device The device displays the playback screen of the first video.
  • the third electronic device is connected to the first electronic device, and the first electronic device displays the operation process and/or the first video of the first electronic device performing the operation.
  • the playback screen; the third electronic device includes a third display area and a fourth display area, the third display area displays the display content of the first electronic device, and the fourth display area displays the operation process or The playback screen of the first video.
  • the method further includes: upon receiving an instruction to exit playback and detecting that the operation is not completed, sending prompt information to a display interface of the first electronic device.
  • the first video is marked with multiple time points, and each time point corresponds to part of the user's operation during the screen recording process.
  • the method further includes: fixing associated partial time points among the plurality of time points, so that videos corresponding to the associated partial time points are played continuously.
  • the associated time point may be a time point during the operation of executing any function, such as multiple time points during the operation of turning on the WIFI function in the embodiment of the present application. Fixing the associated time points prevents the electronic device from dragging the progress bar back and forth between the associated time points when playing a video, preventing the user from dragging the progress bar when recording a video that includes the execution of multiple functions. The user can perform subsequent viewing during the intermediate process of performing a certain function, causing the problem that the electronic device performs redundant operations or performs less operations.
  • the playback of the first video is stopped; After the operation corresponding to the target time point is completed, the first video continues to be played.
  • the method further includes: editing the first video to obtain a second video.
  • the editing processing includes cutting processing and/or splitting processing and/or reorganization processing.
  • the first electronic device stores the first video and the first information in MOV format, EXIF format or compressed package format.
  • performing an operation based on the first information includes: detecting a play instruction, playing the first video, and performing an operation on the first video.
  • the operation corresponding to the first time point is added to the task queue, and the operation in the task queue is executed based on the first information; when the first video is played to At the second time point, add the operation corresponding to the second time point to the task queue.
  • this application provides an operating method, applied to a second electronic device, which is characterized by including: detecting a screen recording instruction; starting screen recording, and generating a first video and first information, where the first information includes The interface information of at least one display interface recorded during the screen recording process and the identification information of the target control of the user's operation of the at least one display interface, wherein the first information can be used: when the electronic device plays the first video In the process, the corresponding operation is performed based on the first information.
  • the second electronic device sends the first video and the first information to the first electronic device.
  • the present application provides an electronic device, which is characterized in that it includes: a memory for storing instructions executed by one or more processors of the electronic device, and the processor is a processor of the electronic device.
  • the processor is a processor of the electronic device.
  • the present application provides a readable storage medium. Instructions are stored on the readable medium. When the instructions are executed on an electronic device, the electronic device causes the electronic device to perform the operation method mentioned in the present application.
  • the present application provides a computer program product, including: execution instructions, the execution instructions are stored in a readable storage medium, and at least one processor of an electronic device can read the execution instructions from the readable storage medium. Instructions, the at least one processor executes the execution instructions so that the electronic device implements the operation method mentioned in this application.
  • Figure 1a shows a schematic diagram of a screen recording process according to some embodiments of the prior art.
  • Figure 1b shows a schematic diagram of playing a recorded video according to some embodiments of the prior art.
  • Figure 2a shows a schematic diagram of a screen recording process according to some embodiments of the present application.
  • Figure 2b shows a schematic diagram of a screen recording process according to some embodiments of the present application.
  • Figure 3 shows a schematic diagram of a screen recording process according to some embodiments of the present application.
  • Figures 4a-4h show a schematic diagram of a screen recording process according to some embodiments of the present application.
  • Figure 5 shows a schematic structural diagram of an electronic device according to some embodiments of the present application.
  • Figure 6 shows a schematic flowchart of an operating method according to some embodiments of the present application.
  • Figures 7a-7c respectively show a schematic diagram of recording a video according to some embodiments of the present application.
  • Figure 7d shows a storage schematic diagram of recorded video and operation information respectively according to some embodiments of the present application.
  • Figure 8 shows a schematic diagram of playback of recorded video according to some embodiments of the present application.
  • Figures 9a-9b respectively show a schematic diagram of playback of a recorded video according to some embodiments of the present application.
  • Figure 10 shows a schematic diagram of playback of recorded video according to some embodiments of the present application.
  • Figure 11 shows a schematic diagram of playback of recorded video according to some embodiments of the present application.
  • Figure 12 shows a schematic diagram of playback of recorded video according to some embodiments of the present application.
  • Figure 13 shows a schematic diagram of a scene of exiting playback during playback of a recorded video according to some embodiments of the present application.
  • Figure 14 shows a schematic diagram of setting time points in a recorded video according to some embodiments of the present application.
  • Figure 15 shows a schematic diagram of associated fixation of time points in a recorded video according to some embodiments of the present application.
  • Figure 16 shows a schematic diagram of the generation of a task queue during the playback process of a recorded video according to some embodiments of the present application.
  • 17a-17b illustrate a schematic diagram of a video clip according to some embodiments of the present application.
  • Figure 18 shows a schematic diagram of the hardware structure of an electronic device according to some embodiments of the present application.
  • Illustrative embodiments of the present application include, but are not limited to, a method of operation, electronic equipment, and media.
  • an embodiment of the present application provides an operation method, which includes: after detecting the recording triggering operation, the first electronic device starts screen recording, and records the user's actions on the target controls triggered by each interface during the recording process.
  • Location information for example, as shown in Figure 2a, during the recording process, the user clicks on the WIFI control 4031 on the setting interface 403 of the mobile phone 100, then the electronic device can record the location information of the WIFI control 4031.
  • the location information of the WIFI control 4031 is "Set the second control row of the interface".
  • the first electronic device can send the recorded video and the corresponding position information of the target control of each interface to the second electronic device.
  • the second electronic device can automatically trigger the target control based on the position information of the target control on each interface, that is, to achieve the same Record actions consistent with the actions in the video. For example, during the recording process of the first electronic device, the user clicks on the WIFI control 4031 on the setting interface 403 of the mobile phone 100 . Then, while playing the video, the second electronic device will also trigger the WIFI control on the second electronic device based on the location information of the WIFI control 4031.
  • the above solution can automatically trigger corresponding operations while playing videos, simplifying the operation process.
  • the positions of different controls on different devices may change, for example, as shown in Figure 2a.
  • the location information of the WIFI control 4031 on the mobile phone 100 is "the second control row of the setting interface"
  • the location of the WIFI control 2011 on the mobile phone 200 is the third control row of the setting interface 201.
  • the mobile phone 200 plays the above recorded video, it will still trigger the second control row of the setting interface 201 of the mobile phone 200 based on the above location information, resulting in an incorrect operation.
  • the first electronic device can only record the location of the target control. Therefore, if there is a control with multiple state operations, such as multiple adjustment states, multiple typing states, etc., the second electronic device cannot determine the target control.
  • the specific status makes it impossible to perform specific operations. For example, as shown in Figure 2b, during the recording process, the user slides the brightness adjustment control 1001 on the shortcut interface 101 of the mobile phone 100.
  • the first electronic device can only record the location of the brightness adjustment control 1001. For example, in The last control row of the shortcut interface. However, the degree of brightness adjustment is not recorded. Therefore, based on the above solution, the second electronic device cannot determine the degree of brightness adjustment based on the position of the brightness adjustment control. Therefore, the second electronic device cannot adjust the brightness to the same level as the brightness in the recorded video.
  • an embodiment of the present application provides an operation method, which includes: the first electronic device detects a recording trigger operation. For example, as shown in Figure 3, the user clicks the operation recording control 4011 on the mobile phone 100, then Start screen recording, that is, record the change process of screen display caused by user operation, obtain the recorded video and the first information, and store the recorded video and the first information.
  • the first information may be the user's operation information on the first electronic device during the recording process.
  • the first information includes the interface information of each interface recorded during the screen recording process and the user's operation information on each interface.
  • the user's operation information on each interface may include the user's operation style, operation target and operation target.
  • the layout ID is the layout ID.
  • the user's operation information on each interface may also include information such as the coordinates of the operation target, the position of the control row where the operation target is located, the size of the control, and other information.
  • the operation target may refer to the target control operated by the user.
  • the recorded video and the first information can be stored as two separate files or as one file.
  • the first electronic device can send the recorded video and the corresponding first information to the second electronic device.
  • the second electronic device detects the user's playback operation of the recorded video, then plays the recorded video, and plays the recorded video.
  • a corresponding operation is performed on the second electronic device based on the first information, so that the second electronic device can perform the same operation as the user performed on the first electronic device in the video recording.
  • the user's operation on the first electronic device is to open the settings application, then the second electronic device can open the settings application based on the received first information.
  • the interface information of each interface may include the package name, application name, activity name, etc. of the interface.
  • the user's operation information on each interface may include the user's operation style (such as clicking, typing, filling information, sliding, selecting, typing, etc.), operation target, layout ID of the operation target, etc.
  • the layout ID of each operation target is unique. For example, if the layout ID of the WIFI control in the settings interface is "12345", then on any device, the layout ID of the WIFI control will be "12345". In this way, even if the interface settings of the second electronic device and the first electronic device are different, the electronic device can still accurately find the corresponding control according to the layout ID of the target control and perform the corresponding operation.
  • the first electronic device will record the interface information of the setting interface, including, for example, the application name "settings” and the package name "com. setting” and the activity name “setting interface”, and can record the user's operation information on the setting interface. For example, it can include: the user's operation style "click", the operation target "WIFI control", and the layout ID of "WIFI control" "12345” .
  • the operation information may also include auxiliary information corresponding to the operation style.
  • auxiliary information can also include the typing content
  • the sliding auxiliary information can also include the sliding position. , or adjust the position of the progress bar, etc.
  • the brightness adjustment degree can also be recorded, that is, the adjustment position of the brightness bar, for example, the percentage of the overall brightness bar. The position of thirty. In this way, when the second electronic device performs a brightness adjustment operation based on the operation information, the brightness can be accurately adjusted according to the adjustment position of the brightness bar.
  • the second electronic device when the second electronic device performs operations with multiple states such as sliding and typing, it can accurately obtain the operation that should be performed on the target control based on the auxiliary information, thereby improving the accuracy of the operation.
  • the index ID of the corresponding position of the content in the list can be used instead of the layout ID.
  • the first electronic device can send the recorded video and the corresponding first information to the second electronic device through any means such as communication software or Bluetooth transmission.
  • the way in which the second electronic device performs corresponding operations on the second electronic device based on the first information during the video playing process may be:
  • the second electronic device obtains the interface information and user operation information of the first interface in the received first information, enters the interface to be operated according to the interface information, and enters the interface to be operated according to the layout ID of the corresponding control in the user's operation information. Traverse the layout IDs of all controls to find the target control that is consistent with the layout ID of the operation target in the operation information of the first interface received, and perform corresponding operations on the target control according to the operation style in the operation information. After the operation on the first interface is completed, the interface information and user operation information of the next interface are obtained, and the above steps are performed to complete the corresponding operation on the next interface.
  • the user can obtain the recorded video and the user's first information about the first electronic device during the recording process by performing screen recording on the first electronic device, and record the video and the user's first information about the first electronic device during the recording process.
  • First information from an electronic device is sent to a second electronic device.
  • the second electronic device can perform the same operation as the operation performed by the user on the first electronic device in the recorded video, so that the user who receives the screen recording video and the first information does not need to do the operation himself.
  • the electronic device can perform corresponding operations based on the received recorded video and first information, thereby improving operation convenience and user experience.
  • the second electronic device can accurately identify the interface to be operated based on the first information. and corresponding controls to be operated to improve the accuracy of operations.
  • the operation process performed by the second electronic device can be displayed in the form of a floating window to facilitate viewing by the user.
  • the solution in the embodiment of the present application is explained below by taking a scenario in which the user performs screen recording of the process of turning on the WIFI function through the first electronic device and sends the recorded video and operation information to the second electronic device as an example.
  • the first electronic device may be a mobile phone 100
  • the second electronic device may be a mobile phone 200.
  • the mobile phone 100 will record the change process of the display state of the screen caused by the user A's operation, and record the corresponding actions of the user's operation.
  • first information and the mobile phone 100 can display the recording box 400 as shown in Figure 4b during the entire recording process.
  • the user records the WIFI connection operation on the mobile phone 100.
  • the user first clicks the setting application 4021 on the main interface 402.
  • the mobile phone 100 can record the interface information of the main interface 402 while recording the video, such as It includes the activity name "main interface” and can record the user's operation information on the main interface 402. For example, it can include: the user's operation style "click", the operation target "setting application”, and the layout ID "1234" of the setting application 4021.
  • the mobile phone 100 can display the setting interface 403.
  • the mobile phone 100 can record the setting interface 403 while recording the video.
  • Information for example, includes the application name "Settings", the package name "com.setting” and the activity name “Settings Interface”, as well as the user's operation information in the settings interface 403, for example, it may include: the user's operation style "click", operation target " WIFI Control", the layout ID of WIFI control 4031 is "12345".
  • the mobile phone 100 can display the WIFI interface 404.
  • the mobile phone 100 can record the interface information of the WIFI interface 404 while recording the video. For example, it includes the application name "Settings”, the package name "com.setting” and the activity name "WIFI interface”, as well as the user's operation information on the WIFI interface 404. For example, it can include: the user's operation style "click", the operation target "switch control" ", the layout ID of switch control 4041 is "12346".
  • the WIFI interface 404 of the mobile phone 100 can display a list of available WLANs. After the user finishes the operation, he can click the recording stop control 4001 in the recording box 400 to stop recording. At this time, the mobile phone 100 obtains the recorded video 500 of the entire operation process and the corresponding first information and stores it.
  • the recorded video can be sent through communication software as shown in Figure 4f.
  • Video 500 is sent to user B's mobile phone 200. It can be understood that while the recorded video 500 is being sent, the first information will also be sent to user B's mobile phone 200 at the same time.
  • user B can receive and display the recorded video 500 through communication software.
  • user B's electronic device can operate the mobile phone 200 based on the recorded information, and display the operation in the operation floating window 405 process.
  • the specific operation process may include: first triggering the setting control on the main interface and displaying the setting interface, and then triggering the WIFI control on the setting interface and displaying the WIFI interface. And trigger the switch control on the WIFI interface to turn on the WIFI function.
  • the electronic device is a first electronic device or a second electronic device.
  • Software systems of electronic devices can adopt layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture.
  • the embodiment of the present invention takes the Android system with a layered architecture as an example to illustrate the software structure of the electronic device.
  • FIG. 5 is a software structure block diagram of the electronic device according to the embodiment of the present invention.
  • the Android system is divided into four layers, from top to bottom: application layer, application framework layer, Android runtime and system libraries, and kernel layer.
  • the application layer can include a series of application packages. As shown in Figure 5, the application package can include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.
  • the application framework layer provides an application programming interface (API) and programming framework for applications in the application layer.
  • API application programming interface
  • the application framework layer includes some predefined functions.
  • the application framework layer can include window manager, content provider, view system, phone manager, resource manager, notification manager, etc.
  • a window manager is used to manage window programs.
  • the window manager can obtain the display size, determine whether there is a status bar, lock the screen, capture the screen, etc.
  • the window manager can respond to the user's playback operation of the recorded video and send a playback instruction to the corresponding application. For example, if the user clicks play in the photo album application, the window manager can send a playback instruction to the photo album. The application sends playback instructions.
  • the window manager can obtain the first information in response to the user's playback operation of the recorded video, and control the corresponding application program to perform corresponding operations on the second electronic device based on the first information.
  • the window manager can obtain the interface information and user operation information of the first interface in the received first information, determine the interface to be operated based on the interface information, and determine the interface to be operated based on the layout ID of the corresponding control in the user's operation information.
  • the interface traverses the layout IDs of all controls to find the target control that is consistent with the layout ID of the operation target in the operation information received from the first interface, and controls the corresponding application to perform operations on the target control according to the operation style in the operation information. Act accordingly.
  • the interface information and user operation information of the next interface are obtained, and the above steps are performed to complete the corresponding operation on the next interface.
  • Content providers are used to store and retrieve data and make this data accessible to applications.
  • Said data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.
  • the view system includes visual controls, such as controls that display text, controls that display pictures, etc.
  • a view system can be used to build applications.
  • the display interface can be composed of one or more views.
  • a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures.
  • Telephone managers are used to provide communication functions of electronic devices. For example, call status management (including connected, hung up, etc.).
  • the resource manager provides various resources to applications, such as localized strings, icons, pictures, layout files, video files, etc.
  • the notification manager allows applications to display notification information in the status bar, which can be used to convey notification-type messages and can automatically disappear after a short stay without user interaction.
  • the notification manager is used to notify download completion, message reminders, etc.
  • the notification manager can also be notifications that appear in the status bar at the top of the system in the form of charts or scroll bar text, such as notifications for applications running in the background, or notifications that appear on the screen in the form of conversation windows. For example, text information is prompted in the status bar, a beep sounds, the electronic device vibrates, the indicator light flashes, etc.
  • Android Runtime includes core libraries and virtual machines. Android runtime is responsible for the scheduling and management of the Android system.
  • the core library contains two parts: one is the functional functions that need to be called by the Java language, and the other is the core library of Android.
  • the application layer and application framework layer run in virtual machines.
  • the virtual machine executes the java files of the application layer and application framework layer into binary files.
  • the virtual machine is used to perform object life cycle management, stack management, thread management, security and exception management, and garbage collection and other functions.
  • System libraries can include multiple functional modules. For example: surface manager (surface manager), media libraries (Media Libraries), 3D graphics processing libraries (for example: OpenGL ES), 2D graphics engines (for example: SGL), etc.
  • the surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.
  • the media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc.
  • the media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.
  • the media library in response to the user's recording operation, can call a dedicated recording interface to perform screen recording, and store the obtained recorded video and the user's first information on the first electronic device during the recording process in Set a location, such as in the Photos app.
  • the media library may store the file containing the recorded video and the first information in the form of a compressed package in the photo album application of the electronic device.
  • the media library may store the entire file corresponding to the recorded video and the first information in a multi-track format, such as an Exchangeable Image File Format (EXIF) or MOV format.
  • EXIF Exchangeable Image File Format
  • MOV MOV format
  • the media library When the video is recorded and the first information is saved in the photo album or other folders of the electronic device via MOV format.
  • the media library will record the video and store it in the video track according to the format of the video.
  • the media library can output each interface information in the first information into a corresponding text file through the command line, and the operation information of each interface can also be output into the corresponding text file through the command line, and the interface information and the operation of each interface can be output
  • the text files corresponding to the information can be stored in the form of text through data tracks.
  • the video in MOV format supports arbitrary cropping and editing operations. Therefore, using the MOV format to store the recorded video and the first information can make the recorded video in an editable form, which facilitates post-processing.
  • the media library may bundle the recorded video and the first information and save them in the photo album application of the electronic device in MP4 format.
  • the media library can receive recorded videos and first information sent by other electronic devices.
  • the 3D graphics processing library is used to implement 3D graphics drawing, image rendering, composition, and layer processing.
  • 2D Graphics Engine is a drawing engine for 2D drawing.
  • the kernel layer is the layer between hardware and software.
  • the kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.
  • Figure 6 shows the interaction process between the first electronic device and the second electronic device according to the embodiment of the present application.
  • the operation method includes:
  • the first electronic device detects the recording trigger operation and starts screen recording.
  • the recording triggering operation may be an operation in which the user clicks on the recording control 4011 on the mobile phone 100 .
  • a recording box can be displayed on the display interface to remind the user that the recording is in progress and to prompt the user for the recording time.
  • a recording stop control may also be present in the recording box.
  • a recording box 400 may be displayed on the display interface of the mobile phone 100, and the recording box 400 may include a recording time and a recording stop control 4001.
  • the first electronic device obtains the recorded video and the first information.
  • the recorded video may include a video of changes in the screen display caused by user operations during the recording process.
  • the first information may include interface information of each interface recorded during the screen recording process and user operation information on each interface.
  • the interface information of each interface may include the package name, application name, activity name, etc. of the interface.
  • the user's operation information on each interface may include the user's operation style (such as clicking, typing, filling information, sliding, selecting, typing, etc.), operation target, layout ID of the operation target, etc.
  • the layout ID of each operation target is unique. For example, if the layout ID of the WIFI control in the settings interface is "12345", then on any device, the layout ID of the WIFI control will be "12345". In this way, even if the interface settings of the second electronic device and the first electronic device are different, the electronic device can still accurately find the corresponding control according to the layout ID of the target control and perform the corresponding operation.
  • the user clicked the WIFI control on the setting interface of the first electronic device, and the first electronic device
  • the device will record the interface information of the setting interface, such as the application name "Settings", the package name “com.setting” and the activity name “Settings Interface”, and can record the user's operation information in the setting interface, for example, it can include: the user's The operation style is "click”, the operation target is "WIFI control", and the layout ID of "WIFI control" is "12345".
  • the operation information may also include auxiliary information corresponding to the operation style.
  • auxiliary information can also include the typing content
  • the sliding auxiliary information can also include the sliding position. , or adjust the position of the progress bar, etc.
  • the brightness adjustment control 1001 on the shortcut interface 101 of the mobile phone 100.
  • the brightness adjustment degree can also be recorded, that is, the adjustment position of the brightness bar, for example, the position of 30% of the overall brightness bar. In this way, when the second electronic device performs a brightness adjustment operation based on the operation information, the brightness can be accurately adjusted according to the adjustment position of the brightness bar.
  • the second electronic device when the second electronic device performs operations with multiple states such as sliding and typing, it can accurately obtain the operation that should be performed on the target control based on the auxiliary information, thereby improving the accuracy of the operation.
  • the index ID of the corresponding position of the content in the list can be used instead of the layout ID.
  • the first electronic device can store the recorded video and the first information. There are various storage forms of the recorded video and the first information in the first electronic device.
  • the recorded video and the first information may be bundled and saved in a compressed package file format in a photo album application or other folder of the first-end electronic device.
  • the compressed package file containing the recorded video and the first information may be displayed in the photo album application in the form of a video plus an operation mark.
  • the presentation form of the compressed package file can be a video style as shown in Figure 7a, and a border highlighting operation mark 501 is added.
  • the presentation form of the compressed package file can also be a video style as shown in Figure 7b, and a corner mark is added.
  • Mark operation mark 502 the presentation form of the compressed package file can also be a video style as shown in Figure 7c, and add a masking layer prompt operation mark 503.
  • the recorded video and the first information may be saved in the photo album application of the first electronic device or in other folders in MP4 format.
  • the entire file corresponding to the recorded video and the first information can also be stored in a multi-track format, such as an exchangeable image file format (EXIF) or MOV format.
  • EXIF exchangeable image file format
  • MOV MOV format
  • the recorded video and the first information are saved in the photo album or other folder of the electronic device through the MOV format as shown in Figure 7d.
  • the recorded video can be stored in the video format, such as MP4 format, through the video track.
  • Each interface information in the first information can be output as a corresponding text file through the command line, and the operation information of each interface can also be output as a corresponding text file through the command line.
  • the interface information and the operation information of each interface correspond to Text files can be stored as text via data tracks.
  • the video in MOV format supports arbitrary cropping and editing operations. Therefore, using the MOV format to store the recorded video and the first information can make the recorded video in an editable form, which facilitates post-processing.
  • the first electronic device sends the recorded video and the corresponding first information to the second electronic device.
  • the first electronic device can send the recorded video and the file corresponding to the corresponding first information, such as a compressed package file, to the second electronic device through the communication software through the network.
  • the first electronic device may send the recorded video and the file corresponding to the corresponding first information to the second electronic device through Bluetooth transmission.
  • the first electronic device can also upload files corresponding to the recorded video and the corresponding first information to the cloud space, and the second electronic device can download the files corresponding to the recorded video and the corresponding first information from the cloud space.
  • the first electronic device will share the recorded video and the file corresponding to the corresponding first information through a link, password, etc. Share to the second electronic device, and the second electronic device obtains the recorded video and the corresponding first information by opening the sharing link or entering a password.
  • the first electronic device transmits the recorded video and the corresponding first information to the second electronic device
  • the first electronic device transmits the recorded video and the corresponding first information to the second electronic device.
  • the mode of the second electronic device may be any implementable mode.
  • the recorded video 500 can be sent to user B's mobile phone through the network and communication software as shown in Figure 4f. 200 for mobile phone. It can be understood that while the recorded video is being sent, the first information will also be sent to user B's mobile phone 200 at the same time.
  • the second electronic device receives the recorded video and the corresponding first information.
  • the second electronic device While receiving the recorded video, the second electronic device will also receive the first information at the same time.
  • user B can receive the recorded video 500 through communication software and display the recorded video in a video style.
  • the second electronic device can store the recorded video and the corresponding first information.
  • the recorded video and the first information may be stored in the second electronic device in a variety of storage forms.
  • the recorded video and the first information may be bundled and saved in a compressed package file format in a photo album application or other folder of the second electronic device.
  • the compressed package file containing the recorded video and the first information may be displayed in the photo album application in the form of a video plus an operation mark.
  • the presentation form of the compressed package file can be a video style as shown in the aforementioned Figure 7a, and a border highlighting operation mark 501 is added.
  • the presentation form of the compressed package file can also be a video style as shown in Figure 7b, with the addition
  • the corner mark operation mark 502 the presentation form of the compressed package file can also be a video style as shown in Figure 7c, and a masking layer prompt operation mark 503 is added.
  • the recorded video and the first information may be bundled and saved in MP4 format in a photo album application of the second electronic device or in other folders.
  • the recorded video and the first information may be bundled and saved in MP4 format in a photo album application of the second electronic device or in other folders.
  • the entire file corresponding to the recorded video and the first information can also be stored in a multi-track format, such as an exchangeable image file format (EXIF) or MOV format.
  • EXIF exchangeable image file format
  • MOV MOV format
  • the recorded video and the first information are saved in the photo album or other folder of the electronic device through the MOV format as shown in Figure 7d.
  • the recorded video can be stored in the video format, such as MP4 format, through the video track.
  • Each interface information in the first information can be output as a corresponding text file through the command line, and the operation information of each interface can also be output as a corresponding text file through the command line.
  • the interface information and the operation information of each interface correspond to Text files can be stored as text via data tracks.
  • the video in MOV format supports arbitrary cropping and editing operations. Therefore, using the MOV format to store the recorded video and the first information can make the recorded video in an editable form, which facilitates post-processing.
  • the second electronic device responds to the user's playback operation and performs a corresponding operation based on the first information.
  • the operation performed by the second electronic device based on the first information is consistent with the user's operation on the display interface in the recorded video.
  • the way in which the second electronic device performs the corresponding operation based on the first information may be:
  • the second electronic device obtains the information of the first interface and user operation information in the received first information, enters the interface to be operated according to the interface information, and displays the interface to be operated according to the layout ID of the corresponding control in the user's operation information.
  • the layout IDs of all controls are traversed to find the target control that is consistent with the layout ID of the operation target in the operation information of the first interface received, and corresponding operations are performed on the target control according to the operation style in the operation information.
  • the interface information and user operation information of the next interface are obtained, and the above steps are performed to complete the corresponding operation on the next interface.
  • the second electronic device performs the second electronic device on the second electronic device based on the first information during the video playback process. How electronic equipment performs corresponding operations:
  • the mobile phone 200 first obtains the interface information 402 of the main interface in the first information, including, for example, the activity name "main interface”, and Obtaining the user's operation information on the main interface 402 may include, for example: the user's operation style "click", the operation target "setting application”, and the layout ID "1234" of the setting application 4021.
  • the main interface of the mobile phone 200 is found as the interface to be operated.
  • the layout ID "1234" corresponding to the setting application 4021 in the user's operation information the layout IDs of all controls are traversed on the main interface of the mobile phone 200 to find the target control consistent with the layout ID "1234" of the setting application 4021.
  • the operation style "click" in the first information performs a click operation on the target control. After the click operation on the target control is completed, the interface information and user operation information of the setting interface 403 are obtained, and the above steps are performed to complete the corresponding operation on the setting interface of the mobile phone 200 .
  • the second electronic device in response to the user's playback operation, can play the recorded video in full screen or in a large window, and perform the operation in the background without displaying the operation process.
  • the second electronic device in response to the user's play operation, can play the recorded video in full screen or in a large window, and display the operation process in the operation floating window.
  • the recorded video 500 when the user clicks on the recorded video 500 received by the mobile phone 200 for video playback, as shown in Figure 4h, the recorded video can be played in full screen on the main interface of the mobile phone 200, and in the operation floating window 405 Show the operation process.
  • the second electronic device in response to the user's playback operation, can display the recorded video in a video operation floating window, and place the operation process in a large window for playback.
  • the operation process can be displayed on the main interface of the mobile phone 200, and the recorded video can be played in the video floating window 406.
  • the second electronic device may include a first display area and a second display area.
  • the first display area displays the recorded video
  • the second display area displays the operation process, that is, the recorded video and the operation process.
  • the second electronic device may have any number of other devices connected to each other. Then, the video can be played through any device among the multiple devices connected to each other, and any device among the other devices can control the second electronic device to perform the operation corresponding to the first information.
  • the above-mentioned mutual connection method can be any method such as Bluetooth connection, mobile phone NFC function pairing connection, scan code connection, wired connection, etc.
  • the third electronic device is a tablet 300
  • the second electronic device is a mobile phone 200.
  • the tablet 300 and the mobile phone 200 are paired and connected through the NFC function of the mobile phone, and the third electronic device can be used to play the recorded video 500
  • the second electronic device may be used to perform corresponding operations according to the first information.
  • the mobile phone 200 can be used to play the recorded video 500, and the tablet 300 can be used to control the second device to perform corresponding operations based on the first information, and display the operation process.
  • the tablet 300 can display the virtual interface 302 corresponding to the mobile phone 200.
  • the mobile phone 200 can display the operation process on the main interface of the mobile phone 200.
  • the virtual interface 302 of the tablet 300 can be consistent with the mobile phone. Show the operation process.
  • the play window 301 of the tablet 300 can play the recorded video at the same time.
  • the user when the user exits playback, if the second electronic device still has unfinished operations, the user may be reminded in the form of a pop-up window or other forms.
  • a prompt box 600 may be prompted that "the operation is not completed, do you want to continue?"
  • the user clicks the Yes control 601 in the prompt box 600 the video will continue to be played.
  • the user clicks the No control 602 in the prompt box 600 the video playback will be stopped.
  • the recorded video can maintain any functions of conventional video such as fast forward, rewind, window size adjustment, and pause. And the recorded video can be played multiple times.
  • the first electronic device can mark multiple time points in the recorded video. For example, in some embodiments, during the recording process, after all operations on each interface are completed, a time point can be marked before the next interface is displayed.
  • the following description takes the first electronic device as a mobile phone 100 and the mobile phone 100 recording the operation process of user A turning on the WIFI function as an example to describe the acquisition of the recorded information and the first information in the embodiment of the present application.
  • the user records the WIFI opening operation on the mobile phone 100.
  • the user first clicks the settings application 4021 on the main interface 402.
  • the mobile phone 100 records the interface information of the main interface 402 while recording the video, such as the activity name. "Main interface”, and can record the user's operation information on the main interface 402, which may include, for example: the user's operation style "click", the operation target "setting application", and the layout ID "1234" of the setting application 4021.
  • the mobile phone 100 can mark the first time point in the recorded video.
  • the mobile phone 100 can display the setting interface 403.
  • the mobile phone 100 can also record the interface information of the setting interface 403, for example, including The application name "Settings", the package name "com.setting” and the activity name “Settings Interface”, as well as the user's operation information on the settings interface 403, may include, for example: the user's operation style "click", the operation target "WIFI control", After the user clicks the layout ID "12345" of the WIFI control 401, the mobile phone 100 can mark the second time point in the recorded video.
  • the mobile phone 100 can display the WIFI interface 404.
  • the WIFI interface 404 of the mobile phone 100 can display a list of available WLANs.
  • the mobile phone 100 After the user finishes the operation, he can click the recording stop control 4001 in the recording box 400 to stop recording. At this time, the mobile phone 100 marks the third time point, which is the end time point. The mobile phone 100 obtains and stores the recorded video of the entire operation process and the corresponding first information.
  • the obtained recorded video 500 may include three time points as shown in FIG. 14 , where each time point may correspond to a subtask in the WIFI activation function task.
  • associated time points among multiple time points can be fixed to each other.
  • the associated time points can be time points during the operation of performing any function, such as performing the above Multiple time points during the operation of turning on the WIFI function. This prevents the first electronic device or the second electronic device from dragging the progress bar between associated time points back and forth when playing a video, thereby preventing the user from dragging the video when the video includes the execution of multiple functions. The progress bar is followed up until the execution of a certain function, causing the problem that the electronic device performs unnecessary operations or performs less operations.
  • the first time point in the recorded video 500 is the time point after the user clicks on the WIFI control 401 in the scene shown in Figure 4, and the second time point is after the user clicks on the WIFI control 401.
  • the third time point is the time point after the user clicks the switch control 4041.
  • the operation between the third time point and the fourth time point is the operation of returning to the main interface.
  • the fifth time point, The sixth time point and the seventh time point are respectively the time points in the recording process when the user performs the transfer function.
  • the first electronic device can transfer the first time point, the second time point and the third time point. Click to fix the association.
  • the fifth time point, the sixth time point and the seventh time point are also related and fixed. That is, when playing a video, the first electronic device or the second electronic device cannot drag the progress bar back and forth before the third time point between the starting times.
  • the content between the fixed time point and the previous time point is the operation content corresponding to the fixed time point. Therefore, when any time point is fixed, the content between the fixed time point and the previous time point will The content is also fixed. Because for the recorded video shown in Figure 15, when the second electronic device or the first electronic device plays the recorded video 500, the user cannot drag the progress bar between the starting time and the third time point. Nor can you drag the progress bar between the fourth time point and the seventh time point back and forth. That is, if the user wants to drag the progress bar, he can only drag it directly from the initial time point to the third time point, and directly play the video after the third time point. He cannot drag directly from the initial time point to the first time point. Or the second point in time. In this way, it can be effectively avoided that if the user wants to perform the operation of turning on the WIFI function, the electronic device will perform less functions. If the user wants to perform the operation of the transfer function, the electronic device will perform more operations of turning on the WIFI function.
  • the second electronic device can perform corresponding operations on the second electronic device based on the first information while starting to play the video.
  • the second electronic device can automatically pause the playback to wait for the operation before the point in time to be completed before playing.
  • the second electronic device While the second electronic device is playing the recorded video 500, if the recorded video 500 has been played to the second time point, but the second electronic device has not completed the operation of clicking the WIFI control on the setting interface, the second electronic device Video playback will stop and the video will be played again after the second electronic device completes the operation of clicking the WIFI control on the setting interface.
  • the second electronic device may not trigger the execution of the operation when starting to play the video. Instead, when the video is played to each time point, the second electronic device may trigger the execution of the operation task before that time point, and each operation task may be added. into the task queue, and the second device sequentially executes the above operation tasks in the order in which the operation tasks are added. That is, the second electronic device triggers the execution of the operation when the playback reaches the first time point.
  • the first operation task is triggered, that is, the first operation task is triggered.
  • Operations performed before a certain point in time such as clicking settings on the main interface Set the operation of the application and add the first operation task to the task queue;
  • the second task is triggered, that is, the operation before the second point in time, for example, to implement the Click the operation of the WIFI control on the setting interface, and add the second operation task to the task queue.
  • both the first electronic device and the second electronic device can edit the recorded video, such as cropping by mistake, splitting the operation to obtain multiple video files, etc. In this way, incorrect operations or redundant operations during the recording process can be effectively avoided. It can be understood that during the editing process of the recorded video by the electronic device, the first information will also be adjusted accordingly.
  • the electronic device after the electronic device performed the operation of clicking the WIFI control on the setting interface, it marked the second time point, and then accidentally pressed the volume button, the electronic device marked the third time point, so The operation corresponding to the third time point is a redundant operation. At this time, the user can crop out the video content between the second time point and the third time point as shown in Figure 17b. At the same time, the electronic device will automatically delete the second time point. and the first information corresponding to the video between the third time point.
  • the following describes the hardware structure of the electronic device provided by the embodiment of the present application, taking the mobile phone 10 as an example.
  • the mobile phone 10 may include a processor 110, a power module 140, a memory 180, a mobile communication module 130, a wireless communication module 120, a sensor module 190, an audio module 150, a camera 170, an interface module 160, a button 101 and a display. Screen 102 etc.
  • the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the mobile phone 10 .
  • the mobile phone 10 may include more or less components than shown in the figures, or some components may be combined, some components may be separated, or some components may be arranged differently.
  • the components illustrated may be implemented in hardware, software, or a combination of software and hardware.
  • the processor 110 may include one or more processing units, for example, may include a central processing unit (Central Processing Unit, CPU), a graphics processor (Graphics Processing Unit, GPU), a digital signal processor DSP, a microprocessor (Micro- Processing modules or processing circuits such as programmed Control Unit (MCU), artificial intelligence (Artificial Intelligence, AI) processor or programmable logic device (Field Programmable Gate Array, FPGA). Among them, different processing units can be independent devices or integrated in one or more processors.
  • a storage unit may be provided in the processor 110 for storing instructions and data. In some embodiments, the storage unit in the processor 110 is a cache memory 180, and the cache memory 180 can save instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to use the instructions or data again, it can be called directly from the memory.
  • Power module 140 may include a power supply, power management components, and the like.
  • the power source can be a battery.
  • the power management component is used to manage the charging of the power supply and the power supply from the power supply to other modules.
  • the power management component includes a charge management module and a power management module.
  • the charging management module is used to receive charging input from the charger; the power management module is used to connect the power supply, the charging management module and the processor 110 .
  • the power management module receives input from the power supply and/or charging management module and supplies power to the processor 110, the display screen 102, the camera 170, the wireless communication module 120, etc.
  • the mobile communication module 130 may include, but is not limited to, an antenna, a power amplifier, a filter, an LNA (Low noise amplifier), etc.
  • the mobile communication module 130 can provide wireless communication solutions including 2G/3G/4G/5G applied on the mobile phone 10 .
  • the mobile communication module 130 can receive electromagnetic waves through an antenna, perform filtering, amplification and other processing on the received electromagnetic waves, and transmit them to the modem processor for demodulation.
  • the mobile communication module 130 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves through the antenna for radiation.
  • at least part of the functional modules of the mobile communication module 130 may be disposed in the processor 110 .
  • at least part of the functional modules of the mobile communication module 130 may be provided in the same device as at least part of the modules of the processor 110 .
  • a modem processor may include a modulator and a demodulator.
  • the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal.
  • the demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal.
  • the demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing.
  • the application processor outputs sound signals through audio devices (not limited to speaker 170A, receiver 170B, etc.), or displays images or videos through display screen 102 .
  • the modem processor may be a stand-alone device. In other embodiments, the modem processor may be independent of the processing
  • the processor 110 is provided in the same device as the mobile communication module 130 or other functional modules.
  • the wireless communication module 120 may include an antenna, and implements the transmission and reception of electromagnetic waves via the antenna.
  • the wireless communication module 120 can provide applications on the mobile phone 10 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (bluetooth, BT), and global navigation satellite systems. (global navigation satellite system, GNSS), frequency modulation (FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions.
  • WLAN wireless local area networks
  • Wi-Fi wireless fidelity
  • Bluetooth bluetooth, BT
  • global navigation satellite systems global navigation satellite system
  • FM frequency modulation
  • NFC near field communication technology
  • infrared technology infrared, IR
  • the mobile phone 10 can communicate with the network and other devices through wireless communication technology.
  • the mobile communication module 130 and the wireless communication module 120 of the mobile phone 10 may also be located in the same module.
  • the display screen 102 is used to display human-computer interaction interfaces, images, videos, etc.
  • Display 102 includes a display panel.
  • the display panel can use a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (active-matrix organic light emitting diode).
  • LCD liquid crystal display
  • OLED organic light-emitting diode
  • AMOLED organic light-emitting diode
  • FLED flexible light-emitting diode
  • Miniled MicroLed, Micro-oLed, quantum dot light emitting diode (QLED), etc.
  • the sensor module 190 may include a proximity light sensor, a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.
  • the audio module 150 is used to convert digital audio information into an analog audio signal output, or convert an analog audio input into a digital audio signal. Audio module 150 may also be used to encode and decode audio signals. In some embodiments, the audio module 150 may be disposed in the processor 110 , or some functional modules of the audio module 150 may be disposed in the processor 110 . In some embodiments, audio module 150 may include speakers, earpieces, microphones, and headphone jacks.
  • Speakers also called “horns,” are used to convert audio electrical signals into sound signals.
  • a handset also called a “receiver” is used to convert audio electrical signals into sound signals.
  • Microphone also called “microphone” or “microphone” is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak through the mouth close to the microphone and input the sound signal into the microphone.
  • the electronic device 100 may be provided with at least one microphone 170C. In other embodiments, the electronic device 100 may be provided with two microphones 170C, which in addition to collecting sound signals, may also implement a noise reduction function. In other embodiments, the mobile phone 10 can also be equipped with three, four or more microphones to collect sound signals, reduce noise, identify sound sources, and implement directional recording functions, etc.
  • the headphone jack is used to connect wired headphones.
  • the headphone interface can be a USB interface, or it can be a 3.5mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.
  • OMTP open mobile terminal platform
  • CTIA cellular telecommunications industry association of the USA
  • Camera 170 is used to capture still images or video.
  • the object passes through the lens to produce an optical image that is projected onto the photosensitive element.
  • the photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to ISP (Image Signal Processing) to convert it into a digital image signal.
  • ISP Image Signal Processing
  • the mobile phone 10 can realize the shooting function through the ISP, camera 170, video codec, GPU (Graphic Processing Unit, graphics processor), display screen 102 and application processor.
  • the ISP is used to process the data fed back by the camera 193.
  • the shutter is opened, the light is transmitted to the camera sensor through the lens, the optical signal is converted into an electrical signal, and the camera sensor passes the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye.
  • ISP can also perform algorithm optimization on image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene.
  • the ISP may be provided in the camera 193.
  • Video codecs are used to compress or decompress digital video.
  • Electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as: MOVing picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.
  • MPEG picture experts group
  • MPEG2 MPEG2, MPEG3, MPEG4, etc.
  • the interface module 160 includes an external memory interface, a universal serial bus (universal serial bus, USB) interface, a subscriber identification module (subscriber identification module, SIM) card interface, and so on.
  • the external memory interface can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile phone 10.
  • the external memory card communicates with the processor 110 through the external memory interface to implement data storage functions.
  • the Universal Serial Bus interface is used by mobile phones 10 and other electronic devices to communicate.
  • the user identity module card interface is used to communicate with the SIM card installed in the mobile phone 1010, such as reading the phone number stored in the SIM card, or writing the phone number into the SIM card.
  • the mobile phone 10 also includes buttons 101, motors, indicators, etc.
  • the keys 101 may include volume keys, on/off keys, etc.
  • the motor can produce vibration cues.
  • the motor can be used to vibrate for incoming calls or for touch vibration feedback.
  • touch operations for different applications can correspond to different vibration feedback effects.
  • the motor can also respond to different vibration feedback effects for touch operations in different areas of the display screen 102 .
  • Different application scenarios such as time reminders, receiving information, alarm clocks, games, etc.
  • the touch vibration feedback effect can also be customized.
  • Indicators may include laser pointers, radio frequency indicators, LED indicators, etc.
  • Embodiments of the present application provide an electronic device, including: a memory, used to store instructions executed by one or more processors of the electronic device; and a processor, which is one of the one or more processors of the electronic device, used to execute The above operation method.
  • Embodiments of the present application provide a readable storage medium, in which instructions are stored on the readable storage medium.
  • the electronic device When the instructions are executed on an electronic device, the electronic device causes the electronic device to perform the above operation method.
  • Embodiments of the present application provide a computer program product, including: execution instructions.
  • the execution instructions are stored in a readable storage medium.
  • At least one processor of an electronic device can read the execution instructions from the readable storage medium.
  • the at least one processor executes the execution instructions.
  • the instructions enable the electronic device to implement the above operation method.
  • Embodiments disclosed in this application may be implemented in hardware, software, firmware, or a combination of these implementation methods.
  • Embodiments of the present application may be implemented as a computer program or program code executing on a programmable system including at least one processor, a storage system (including volatile and non-volatile memory and/or storage elements) , at least one input device and at least one output device.
  • Program code may be applied to input instructions to perform the functions described herein and to generate output information.
  • Output information can be applied to one or more output devices in a known manner.
  • a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.
  • DSP digital signal processor
  • ASIC application specific integrated circuit
  • Program code may be implemented in a high-level procedural language or an object-oriented programming language to communicate with the processing system.
  • assembly language or machine language can also be used to implement program code.
  • the mechanisms described in this application are not limited to the scope of any particular programming language. In either case, the language may be a compiled or interpreted language.
  • the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof.
  • the disclosed embodiments may also be implemented as instructions carried on or stored on one or more transitory or non-transitory machine-readable (e.g., computer-readable) storage media, which may be operated by one or more processors Read and execute.
  • instructions may be distributed over a network or through other computer-readable media.
  • machine-readable media may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including, but not limited to, floppy disks, optical disks, optical disks, read-only memories (CD-ROMs), magnetic Optical disk, read-only memory (ROM), random-access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic or optical card, flash memory, or Tangible machine-readable storage used to transmit information (e.g., carrier waves, infrared signals, digital signals, etc.) using electrical, optical, acoustic, or other forms of propagated signals over the Internet.
  • machine-readable media includes any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (eg, computer).
  • each unit/module mentioned in each device embodiment of this application is a logical unit/module.
  • a logical unit/module can be a physical unit/module, or it can be a physical unit/module.
  • Part of the module can also be implemented as a combination of multiple physical units/modules.
  • the physical implementation of these logical units/modules is not the most important.
  • the combination of functions implemented by these logical units/modules is what solves the problem of this application. Key technical issues raised.
  • the above-mentioned equipment embodiments of this application do not introduce units/modules that are not closely related to solving the technical problems raised by this application. This does not mean that the above-mentioned equipment embodiments do not exist. Other units/modules.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Signal Processing (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

La présente demande se rapporte au domaine technique des communications, et concerne un procédé de fonctionnement, un dispositif électronique et un support. Le procédé de fonctionnement comprend les étapes suivantes dans lesquelles : un premier dispositif électronique acquiert une première vidéo et des premières informations associées à la première vidéo, les premières informations comprenant des informations d'interface d'au moins une interface d'affichage enregistrée dans un processus d'enregistrement d'écran et des informations d'identification d'une commande cible actionnée par un utilisateur sur ladite interface d'affichage ; lit la première vidéo, et exécute une opération sur la base des premières informations dans le processus de lecture de la première vidéo. Selon la solution, dans un processus de lecture vidéo, le premier dispositif électronique peut exécuter la même opération que l'opération exécutée par un utilisateur sur un dispositif électronique pendant un enregistrement vidéo, de sorte que l'utilisateur n'a pas besoin de l'actionner par lui-même, ce qui permet d'améliorer la commodité d'opération et l'expérience d'utilisateur.
PCT/CN2023/112297 2022-08-18 2023-08-10 Procédé de fonctionnement, dispositif électronique et support WO2024037421A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210995317.8A CN117631919A (zh) 2022-08-18 2022-08-18 一种操作方法、电子设备及介质
CN202210995317.8 2022-08-18

Publications (1)

Publication Number Publication Date
WO2024037421A1 true WO2024037421A1 (fr) 2024-02-22

Family

ID=89940716

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/112297 WO2024037421A1 (fr) 2022-08-18 2023-08-10 Procédé de fonctionnement, dispositif électronique et support

Country Status (2)

Country Link
CN (1) CN117631919A (fr)
WO (1) WO2024037421A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108111903A (zh) * 2018-01-17 2018-06-01 广东欧珀移动通信有限公司 录屏文件播放方法、装置及终端
CN109561271A (zh) * 2018-11-19 2019-04-02 维沃移动通信有限公司 一种终端操作的指导方法、第一终端及第二终端
CN112286617A (zh) * 2020-10-30 2021-01-29 维沃移动通信有限公司 操作指导方法、装置及电子设备
EP3796318A1 (fr) * 2019-09-19 2021-03-24 Baidu Online Network Technology (Beijing) Co., Ltd. Procédé et dispositif de lecture vidéo, dispositif électronique et support d'informations lisible
CN113923461A (zh) * 2020-07-10 2022-01-11 华为技术有限公司 一种录屏方法和录屏系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108111903A (zh) * 2018-01-17 2018-06-01 广东欧珀移动通信有限公司 录屏文件播放方法、装置及终端
CN109561271A (zh) * 2018-11-19 2019-04-02 维沃移动通信有限公司 一种终端操作的指导方法、第一终端及第二终端
EP3796318A1 (fr) * 2019-09-19 2021-03-24 Baidu Online Network Technology (Beijing) Co., Ltd. Procédé et dispositif de lecture vidéo, dispositif électronique et support d'informations lisible
CN113923461A (zh) * 2020-07-10 2022-01-11 华为技术有限公司 一种录屏方法和录屏系统
CN112286617A (zh) * 2020-10-30 2021-01-29 维沃移动通信有限公司 操作指导方法、装置及电子设备

Also Published As

Publication number Publication date
CN117631919A (zh) 2024-03-01

Similar Documents

Publication Publication Date Title
CN110377250B (zh) 一种投屏场景下的触控方法及电子设备
US11922005B2 (en) Screen capture method and related device
JP7414842B2 (ja) コメント追加方法及び電子デバイス
JP7142783B2 (ja) 音声制御方法及び電子装置
US11722449B2 (en) Notification message preview method and electronic device
US20220342850A1 (en) Data transmission method and related device
WO2021036571A1 (fr) Procédé d'édition de bureau et dispositif électronique
CN107087101B (zh) 用于提供动态全景功能的装置和方法
KR102195314B1 (ko) 전자 장치 및 그 동작 방법
WO2021121052A1 (fr) Procédé et système de coopération à écrans multiples et dispositif électronique
KR102193704B1 (ko) 전자 장치의 락 스크린 디스플레이 방법 및 그 전자 장치
WO2022152024A1 (fr) Procédé d'affichage de gadgets logiciels et dispositif électronique
WO2022057852A1 (fr) Procédé d'interaction entre de multiples applications
WO2023236794A1 (fr) Procédé de marquage de piste audio et dispositif électronique
US20220366327A1 (en) Information sharing method for smart scene service and related apparatus
CN111176766A (zh) 一种通信终端及组件显示方法
WO2024037421A1 (fr) Procédé de fonctionnement, dispositif électronique et support
CN114449171A (zh) 控制摄像头的方法、终端设备、存储介质及程序产品
CN115002336A (zh) 视频信息的生成方法、电子设备及介质
CN112328135A (zh) 移动终端及其应用界面显示方法
US10924602B2 (en) Electronic device and method for controlling same
CN111324255A (zh) 一种基于双屏终端的应用处理方法及通信终端
CN111163220A (zh) 一种显示方法及通信终端
WO2024103894A1 (fr) Procédé de d'enregistrement d'écran et dispositif électronique
WO2023160455A1 (fr) Procédé de suppression d'objet et dispositif électronique

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23854322

Country of ref document: EP

Kind code of ref document: A1