WO2019210674A1 - Procédé, système et appareil pour dispositif intelligent d'opération manuelle de simulation - Google Patents

Procédé, système et appareil pour dispositif intelligent d'opération manuelle de simulation Download PDF

Info

Publication number
WO2019210674A1
WO2019210674A1 PCT/CN2018/115107 CN2018115107W WO2019210674A1 WO 2019210674 A1 WO2019210674 A1 WO 2019210674A1 CN 2018115107 W CN2018115107 W CN 2018115107W WO 2019210674 A1 WO2019210674 A1 WO 2019210674A1
Authority
WO
WIPO (PCT)
Prior art keywords
smart device
interface
target
interface element
module
Prior art date
Application number
PCT/CN2018/115107
Other languages
English (en)
Chinese (zh)
Inventor
康雨
张磊
董拯
夏海燕
Original Assignee
深圳市华夏双赢通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市华夏双赢通信有限公司 filed Critical 深圳市华夏双赢通信有限公司
Publication of WO2019210674A1 publication Critical patent/WO2019210674A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
    • G06F11/26—Functional testing
    • G06F11/263—Generation of test inputs, e.g. test vectors, patterns or sequences ; with adaptation of the tested hardware for testability with external testers

Definitions

  • the present invention relates to a method, system and apparatus for simulating a manually operated smart device.
  • a prior art device for simulating a manually operated smart device includes a screen click device that simulates a human finger clicking a touch screen through a mechanical device to perform a simple click action.
  • a single screen click device can not complete some complex multi-touch input; due to the physical characteristics and size limitations of the click head, only the limited space of the touch screen can be placed.
  • the movement of these mechanical click heads requires an external drive. When the click position changes rapidly, the response speed of multiple click heads needs to match the click requirement; and mechanical clicks produce noise.
  • the prior art system and apparatus for simulating a human-operated smart device also includes a method of simulating a human finger to click a touch screen or acquiring a touch coordinate point through a USB interface and a target touch screen connection, thereby completing a simple or complicated click action.
  • the desired screen click operation is realized by simulating the size of the touch screen and sending a mouse command.
  • the technical problem to be solved by the present invention is to avoid the above-mentioned deficiencies of the prior art, and to provide a method, system and method for simulating a manual operation intelligent device capable of being compatible with a plurality of input modes and selecting a corresponding analog operation conversion mode by connection type identification. Device.
  • the technical solution adopted by the present invention to solve the technical problem is a method for simulating a manually operated smart device, comprising: Step A0: Simulating a manual operation device to detect and obtain connection mode information between a target intelligent device to be operated and a simulated manual operation device; Step B0: The simulated manual operation device selects a corresponding analog conversion module according to the acquired connection mode information, converts the simulation operation requirement into an operation instruction of the operation target smart device, and transmits the operation instruction to the target smart device; Step C0: target smart device The simulated manual operation of the target smart device is performed according to the received operation instruction.
  • the connection manner between the target smart device and the simulated manual operation device includes: a wired electrical signal connection mode and a radio signal connection mode.
  • the analog conversion module is a first analog conversion module; and the first analog conversion module includes a target device end interface.
  • the target device side interface element acquisition module runs on the target smart device, and when the interface element analysis module runs on the simulated manual operation device, the target device end interface element acquisition module and the interface element analysis module communicate through the wireless WiFi connection; or the target device The interface element acquisition module and the interface element analysis module are both running on the target smart device.
  • the step B0 further includes the step of determining whether the current analog operation requirement requires the second analog conversion module; if the current simulation operation requirement is not The second analog conversion module is required, and the simulated manual operation device sends the simulated manual operation instruction directly to the target smart device according to the Bluetooth HID protocol in the manner of the Bluetooth HID external device, and the target smart device receives the operation instruction according to the Bluetooth HID protocol on the target smart device.
  • the operation is performed; if the current analog operation requirement requires the second analog conversion module, the second analog conversion module includes a target device interface element acquisition module and an interface element analysis module; and the target device interface element acquisition module collects the interface of the target smart device.
  • the information is transmitted to the interface element analysis module; the interface element analysis module outputs the simulation operation instruction to the target smart device according to the received interface information and the simulation operation requirement.
  • the target device-side interface element acquisition module runs on the target smart device, and when the interface element analysis module runs on the simulated manual operation device, the target device-side interface element acquisition module and the interface element analysis module communicate via Bluetooth connection; or the target device end Both the interface element acquisition module and the interface element analysis module run on the target smart device.
  • connection mode between the target smart device and the analog manual operation device is a USB cable electrical signal connection mode
  • step B0 the step of determining whether the current simulation operation requirement only requires the USB driver layer analog conversion module; If the current analog operation requirement only needs the driver layer analog conversion module; the analog manual operation device directly sends the analog human operation instruction of the USB driver layer to the target smart device by means of a USB external device; the target smart device according to the received USB driver layer
  • the simulated manual operation instructions operate on the target smart device.
  • the step B0 the step of determining whether the current simulation operation requirement only needs the USB application layer analog conversion module is included;
  • the USB application layer analog conversion module includes a target device side interface element acquisition module and an interface element analysis module; the target device end interface element acquisition module collects interface information of the target smart device, and The interface information is transmitted to the interface element analysis module; the interface element analysis module outputs the simulation operation instruction to the target smart device according to the received interface information and the simulation operation requirement.
  • the target device-side interface element acquisition module runs on the target smart device, and when the interface element analysis module runs on the simulated manual operation device, the target device-side interface element acquisition module and the interface element analysis module communicate through the USB wired electrical signal; or The target device side interface element acquisition module and the interface element analysis module are all running on the target smart device.
  • the simulation operation requirement is a configuration data file or a configuration command set
  • the simulated manual operation device acquires a configuration data file or a configuration command set from a cloud server or other web service device through a wired or wireless communication interface
  • the simulated manual operation device acquires a configuration data file or a configuration command set through an external input device input; or simulates the manual operation device to have a configuration data file or a configuration command set built therein.
  • the technical solution adopted by the present invention to solve the technical problem may also be a system for simulating a manually operated smart device, including a simulated manual operating device and a target intelligent device to be operated;
  • the simulated manual operating device includes: for a target smart device and a simulated artificial device
  • the operation device is connected to the interface identification sub-module identified by the interface type, and is used for the target intelligent device and the working mode selection sub-module for simulating the interaction mode selection of the manual operation device, and the main controller for the device main control;
  • the interface identification sub-module Detecting a connection manner between the target smart device and the simulated manual operation device, or the interface identification submodule selects a corresponding interface for simulating a connection between the manual operation device and the target smart device according to an interface selection instruction from the outside;
  • the working mode selection sub-module selects a connection mode between the target smart device and the simulated manual operation device according to the detected connection mode of the target smart device and the simulated manual operation device, or the working mode
  • the connection between the target smart device and the analog manual operating device includes a wired electrical signal connection method and a radio signal connection method.
  • the system for simulating the manual operation smart device further includes a first analog conversion module; the first analog conversion module includes a target device interface element Obtaining a module and an interface element analysis module; the target device interface element acquisition module collects interface information of the target smart device, and transmits the interface information to the interface element analysis module; the interface element analysis module outputs the simulation according to the received interface information and the simulation operation requirement Operation instructions to the target smart device.
  • the target device-side interface element acquisition module runs on the target smart device, and when the interface element analysis module runs on the simulated manual operation device, the target device-side interface element acquisition module and the interface element analysis module communicate via a radio connection; or the target device end Both the interface element acquisition module and the interface element analysis module run on the target smart device.
  • the first analog conversion module further includes a simulation operation requirement acquisition module running on the target smart device, and the simulation operation requirement acquisition module passes the target intelligence
  • the communication interface on the device is connected to the external device to obtain the analog operation requirements.
  • the system for simulating a manually operated smart device further includes a second analog conversion module; the second analog conversion module includes a target device interface element acquisition module and an interface element analysis module; and when the target smart device and the simulated manual operation device
  • the radio signal connection mode is a Bluetooth connection
  • the analog manual operation device determines whether the current analog operation requirement requires a second analog conversion module; if the current analog operation requirement does not require the second analog conversion module, the simulated manual operation device uses Bluetooth
  • the HID external device directly sends the simulated manual operation instruction to the target smart device according to the Bluetooth HID protocol; the target smart device operates on the target smart device according to the operation instruction received by the Bluetooth HID protocol; if the current simulation operation requirement requires the second simulation
  • the conversion module, the target device interface element acquisition module collects the interface information of the target smart device, and transmits the interface information to the interface element analysis module; the interface element analysis module outputs the simulation operation instruction according to the received interface information and the simulation operation requirement.
  • Target smart devices When the radio signal connection mode is a Bluetooth connection;
  • the target device-side interface element acquisition module runs on the target smart device, and when the interface element analysis module runs on the simulated manual operation device, the target device-side interface element acquisition module and the interface element analysis module communicate via Bluetooth connection; or the target device end Both the interface element acquisition module and the interface element analysis module run on the target smart device.
  • the system for simulating a manually operated smart device further includes a USB driver layer analog conversion module; when the connection mode between the target smart device and the simulated manual operation device is a USB cable electrical signal connection mode; the simulated manual operation device determines Whether the current analog operation requires only the USB driver layer analog conversion module; if the current analog operation requires only the USB driver layer analog conversion module; the analog manual operation device sends the USB driver layer directly to the target smart device by means of a USB external device.
  • the manual operation instruction is simulated; the target smart device operates on the target smart device according to the received USB drive layer simulation manual operation instruction.
  • the system for simulating a manually operated smart device further includes a USB application layer analog conversion module; the USB application layer analog conversion module includes a running target device interface element acquisition module and an interface element analysis module; when the target smart device and the simulated artificial When the connection mode between the operating devices is the USB cable electrical signal connection mode; the analog manual operation device determines whether the current analog operation requirement only requires the USB application layer analog conversion module; if the current analog operation requirement requires a USB application layer analog conversion
  • the module, the target device interface element acquisition module collects the interface information of the target smart device, and transmits the interface information to the interface element analysis module; the interface element analysis module outputs the simulation operation instruction to the target smart device according to the received interface information and the simulation operation requirement.
  • the target device-side interface element acquisition module runs on the target smart device, and when the interface element analysis module runs on the simulated manual operation device, the target device-side interface element acquisition module and the interface element analysis module communicate through the USB wired electrical signal; or The target device side interface element acquisition module and the interface element analysis module are all running on the target smart device.
  • the simulated operation requirement is a configuration data file or a configuration command set; the simulated manual operation device wired or wireless communication interface acquires a configuration data file or a configuration command set from the cloud server or other web service device; or simulates the manual operation device through the external input device. Enter the configuration data file or configuration command set; or simulate the manual operation device built-in configuration data file or configuration command set.
  • the technical solution adopted by the present invention to solve the technical problem may also be a simulated manual operation device for simulating a manually operated smart device, comprising: a first interface module for simulating the connection between the manual operation device and the target intelligent device to be operated, An interface identification sub-module for simulating a manual operation device and a target smart device connection interface type identification, a working mode selection sub-module for simulating a manual operation device and a target intelligent device interaction mode selection, and a main function for simulating the main control of the manual operation device a controller; the interface identification submodule directly detects a type of a connection interface that identifies the simulated manual operation device and the target smart device, or the interface identification submodule selects the simulated manual operation device and the target smart device according to an interface selection instruction from the outside The type of the connection interface; the working mode selection sub-module according to the detected type of the simulated manual operation device and the target smart device connection interface, or the interface identification sub-module according to the interface selection instruction in the received external control
  • the simulation conversion module includes an interface element analysis module; the interface element information acquired from the target smart device is transmitted to the interface element analysis module in the simulated manual operation device through the first interface module; the interface element analysis module is based on the simulation operation requirement and the interface element
  • the information calculation analyzes the interface elements that need to be operated, and sends corresponding simulation operation instructions to the target smart device.
  • the first interface module includes a Bluetooth protocol interface sub-module; the Bluetooth protocol interface sub-module enables the analog manual operation device to directly send a simulated manual operation instruction to the target smart device according to the Bluetooth HID protocol in the manner of a Bluetooth HID external device.
  • the first interface module includes a Bluetooth protocol interface sub-module; the interface element analysis module acquires interface information of the target smart device through the Bluetooth protocol interface sub-module; the interface element analysis module according to the received interface information of the target smart device and the simulated operation demand The target smart device outputs an analog operation command.
  • the first interface module includes a USB protocol interface sub-module; the analog manual operation device directly sends a USB driver layer simulation manual operation instruction to the target smart device in a manner of a USB external device.
  • the first interface module includes a USB protocol interface sub-module; the interface element analysis module obtains interface information of the target smart device through the USB protocol interface sub-module; the interface element analysis module according to the received interface information of the target smart device and the simulated operation requirement The target smart device outputs an analog operation command.
  • the first interface module includes a WiFi protocol interface sub-module; the simulated manual operation device acquires interface element information from the target smart device through the WiFi protocol interface sub-module, and the interface element analysis module calculates and analyzes the operation required according to the simulation operation requirement and the interface element information. Interface elements and send corresponding simulation operation instructions to the target smart device.
  • the simulated manual operating device further includes a second interface module for acquiring external control information and outputting information to an external human-machine interface;
  • the second interface module includes a network interface; and the interface identifying the sub-module from the second interface module Obtaining an interface selection instruction, and selecting, according to the interface selection instruction, a corresponding interface in the first interface module to connect with the target intelligent device;
  • the working mode selection submodule acquiring a working mode selection instruction from the second interface module, and setting the target intelligence according to the interface selection instruction
  • the beneficial effects of the present invention are: in the method, system and device for simulating a manual operation smart device designed by the present invention, the method and system for simulating the manual operation device and the target smart device can be automatically selected and connected.
  • the method is adapted to convert the analog operation requirements into analog operation instructions; the applicable scenarios are rich, not only compatible with wireless connection methods such as wifi and Bluetooth, but also compatible with wired connection, so that an analog manual operation device can be applied to many different types.
  • the application scenario greatly improves the test efficiency in the production test process, and can be used in a test scenario of a plurality of target smart devices, and can be flexibly adapted according to the interface provided by the target smart device. Match.
  • FIG. 1 is a functional block diagram of a preferred embodiment of a simulated manual operation device, in which a power management module is provided;
  • Figure 2 is a functional block diagram of a preferred embodiment of a simulated manual operating device
  • FIG. 3 is a schematic structural diagram of a preferred embodiment of a system for simulating a manually operated smart device
  • FIG. 4 is a second schematic structural diagram of a system for simulating a manually operated smart device
  • FIG. 5 is a schematic diagram showing the workflow of a method for simulating a manually operated smart device
  • FIG. 6 is a schematic diagram of a detailed workflow in the working mode matching pool of FIG. 5;
  • FIG. 7 is a schematic diagram of a workflow when a Bluetooth connection is simulated between a manual operation device and a target smart device;
  • FIG. 8 is a schematic diagram of a workflow when simulating a wireless Wifi mode connection between a manual operation device and a target smart device;
  • FIG. 9 is a schematic diagram showing the workflow of the wired USB connection between the simulated manual operation device and the target smart device.
  • an embodiment of a simulated manual operating device and system thereof includes: an interface identification sub-module for target smart device and analog human-operated device connection interface type identification, for target smart device and simulation The working mode selection sub-module selected by the manual operation device interaction mode, and the main controller for the device main control, the first interface module for simulating the connection between the manual operation device and the target smart device, for simulating the manual operation device and the external An interactive second interface module, a power module for simulating the power management of the manual operating device, and a battery not shown.
  • the simulated manual operation device further includes a power module and a battery for power supply management; and the simulation manual operation device further includes configuration information.
  • the first interface module includes a USB interface, a WiFi interface, a Bluetooth interface, or other network communication interface.
  • the interface identification sub-module directly detects the type of the connection target smart device and the simulated manual operation device connection interface, or the interface identification sub-module according to the interface selection instruction in the external control information acquired from the second interface module. Select the corresponding interface in the first interface module to connect with the target smart device, and output the connection interface type to the working mode selection submodule.
  • the working mode selection sub-module sets an interaction working mode between the target smart device and the system according to the type of the target smart device and the simulated manual operation device connection interface detected by the interface identification sub-module; the working mode selection sub-module can also be controlled according to external control.
  • the working mode selection instruction and the simulation operation requirement in the information set the interactive working mode of the target smart device and the system.
  • the interface identification sub-module and the working mode selection sub-module may be directly wired or radio signal connections, or may be connected to each other through the main controller.
  • an external input device interface and a network communication module are equivalent to the second interface module in FIGS. 2 and 3, and the two interfaces provide a way to input information.
  • the second interface module can modify the configuration information of the simulated manual operation device;
  • the external input device interface can be multiplexed with the interface of the smart device accessing the simulated manual operation device, that is, the first interface module and the second interface module can Sharing part of the interface; specifically, the type of interface that can be shared is mainly related to the interface used by the main controller. According to the interface type of the main controller, the interfaces that can be shared are different.
  • the network communication module is used to enable the analog human operation device to communicate with the public network and has the basic conditions for communication with the cloud server.
  • the simulated manual operating device includes an interface element analysis module.
  • the interface element information acquired from the target smart device is transmitted to the simulated manual operation device through the first interface module; the interface element analysis module calculates and analyzes the interface element that needs to be operated according to the configuration information and the interface element information, and sends the corresponding simulation operation instruction to the target.
  • the smart device, the target smart device performs simulation operations according to the simulation operation program installed inside it.
  • the simulated manual operation device selects a corresponding analog conversion module according to the acquired connection mode information of the target smart device and the simulated manual operation device, converts the simulation operation requirement in the simulated manual operation device into an instruction of the operation target intelligent device, and operates the operation target
  • the instructions of the smart device are transmitted to the target smart device.
  • the connection between the target smart device and the analog manual operation device is different.
  • the analog conversion module will have different forms.
  • the analog conversion module in the wireless connection mode, the analog conversion module will be embodied as an application running on the target smart device, which is responsible for the target.
  • the intelligent device communicates with the simulated manual operation device to complete the conversion of the relevant instructions, and actually performs the configured operation on the target smart device end, such as moving to a specified coordinate position, clicking or dragging; and in the wired mode, mainly USB wired connection mode, through the establishment of ADB (Android debug bridge) communication channel to complete the connection between the target smart device and the simulated manual operation device, more specifically, according to the logical channel finally adopted by the device in the target intelligent device system
  • ADB Android debug bridge
  • the analog conversion module will be embodied as an interface running on the target smart device system (target smart device driver layer) or an application running on the target smart device (target smart device application layer) .
  • the USB driver layer analog conversion module is the underlying driver that comes with the USB interface;
  • the USB application layer layer analog conversion module is an application running on the underlying driver.
  • the driver layer and the application layer referred to in the USB driver layer analog conversion module and the USB application layer layer analog conversion module are corresponding to the driver layer and the application layer of the Android operating system.
  • the analog conversion module when the connection mode between the target smart device and the analog manual operation device is a radio signal connection mode, the analog conversion module is a first analog conversion module; the first analog conversion module is not shown in the figure;
  • An analog conversion module includes a target device-side interface element acquisition module running on the target smart device and an interface element analysis module running on the simulated manual operation device; the target device-side interface element acquisition module collects interface information of the target smart device, and The interface information is transmitted to the interface element analysis module; the interface element analysis module outputs the simulation operation instruction to the target smart device according to the received interface information and the simulation operation requirement.
  • the interface element analysis module in various interface states, in the case where an analog conversion module is required, can be run on the simulated manual operation device, or can be run on the target smart device, or even run. Other external devices, such as cloud servers, that have a communication connection with the target smart device or the simulated human operating device. Especially when the computational complexity of the interface element analysis module is higher, it can be distributedly set on multiple computing nodes in the network.
  • the interface element analysis module referred to in the present invention is a typical interface element analysis module in the prior art, and includes a general and classic interface element analysis method.
  • the simulated manual operating device is connected to the target smart device by wire or wirelessly; the analog operation conversion module is connected to the simulated manual operating device and the target smart device. Between; simulate the manual operation device and the cloud server connection, while simulating the manual operation device also has other input methods.
  • the simulation operation requirement is a configuration data file or a configuration command set; the simulated manual operation device is connected from the cloud server or other web service device through a wired or wireless communication interface.
  • a method of simulating a manually operated smart device includes:
  • Step 10 Simulate that the manual operation device is powered on or turned on, and initialize each port of the device;
  • Step 30 Read the recently saved configuration information, that is, the simulation operation requirement
  • Step 40 Determine whether it is in the configuration state. If it is not in the configuration state, the configuration information currently stored in the simulated manual operation device is used as the simulation operation requirement; if it is in the configuration state, the configuration is completed, and the newly obtained configuration information is used as the simulation. Operational requirements;
  • step 40 if it is in the configuration state, the configuration information is synchronized from the cloud server in the device networking state; or the configuration information is input through an external keyboard connected to the simulated manual operation device; or the communication interface such as wifi or USB is in accordance with a certain The communication protocol transmits configuration information from another smart device;
  • Step 50 After the configuration synchronization or input is completed, the simulated manual operation device will receive the request that the configuration is required to be saved, and after the configuration is saved, the process proceeds to the next step.
  • Step 60 Enter a working mode matching pool to perform work mode matching
  • the simulated manual operation device detects the connection mode information between the target intelligent device and the simulated manual operation device, and simulates the manual operation device according to the acquired connection mode information and the current simulation operation configuration information to be executed. Perform pattern matching screening;
  • Step 70 According to the processing result of the working mode matching pool, select a working mode corresponding to different access modes; in different working modes, the interactive working mode of the target smart device and the simulated manual operating device are different, including: simulating the manual operating device and The target smart devices interact with each other through a Bluetooth protocol; the simulated human operating device interacts with the target smart device by interacting with a simulated operating program installed inside the smart device;
  • Step 80 The simulated manual operation device uses the identified working mode in the step 70 and the simulated operation requirement in the configuration information to select a subsequent specific working step;
  • the alternative paths include:
  • the analog operation instruction is directly sent to the target smart device by the analog manual operation device;
  • step 90 If an analog conversion module is required, then go to step 90;
  • Step 90 The simulation operation conversion module collects and analyzes the interface elements of the target smart device by adapting the detection of the target smart device; the configured simulation operation requirement is to trigger the action according to the specific element of the interface, and obtain the corresponding interface element information. Then, the determined connection and simulation mode are converted into specific actual simulation operation instructions generated on the target smart device, and sent to the target smart device; that is, the simulated manual operation device simulation interface analysis sub-module according to the target device interface element Information analysis, and output simulation operation instructions according to configuration information;
  • Step 100 The simulated manual operation device transmits the simulation operation instruction generated in step 90 to the target smart device; the target smart device executes the instruction.
  • the connection manner between the target smart device and the simulated manual operation device includes: a wired electrical signal connection method and a radio signal connection method.
  • the working mode matches the content in the pool, and the matching mode in the method architecture can be continuously expanded according to actual needs;
  • the working mode matching pool function is that the hardware connection manner of the target smart device access device is different, such as Bluetooth.
  • the connection, the WiFi connection, the USB connection, and the already loaded configuration information determine the subsequent simulated execution path.
  • the method further includes the step of determining whether the current analog operation requirement requires the second analog conversion module;
  • the simulated manual operation device directly sends the simulated manual operation instruction to the target smart device according to the Bluetooth HID (human interface device) protocol; the target smart device receives the operation according to the Bluetooth HID protocol.
  • the instructions operate on the target smart device.
  • the analog manual operation device is paired with the target smart phone in the form of a Bluetooth HID peripheral such as a Bluetooth mouse keyboard or the like (the conventional steps of how to implement the Bluetooth HID device are not redundant), after the connection is completed,
  • the identification sub-module in the simulated manual operation device recognizes that the specific interface mode is a Bluetooth connection mode, and the working mode selection sub-module selects a corresponding working mode according to the simulation operation requirement, that is, the configuration information, and simulates the manual operation.
  • the device enters the corresponding working mode to interact with the target smart device.
  • the second analog conversion module includes a target device interface element acquisition module running on the target smart device and an interface element running on the simulated manual operation device.
  • the analysis module; the target device interface element acquisition module and the interface element analysis module communicate through the Bluetooth connection; the target device interface element acquisition module collects the interface information of the target smart device, and transmits the interface information to the interface element analysis module; the interface element
  • the analysis module outputs the simulated operation instruction to the target smart device according to the received interface information and the simulation operation requirement.
  • the working mode When the working mode selects the sub-module to select the working mode, it checks the current configuration information, that is, the simulation operation requirement, and determines whether the current simulated operation requirement to be valid must depend on an application that exists on the target smart device, that is, whether it depends on
  • the target device interface information acquisition module and the simulation operation module are set on the target smart device; generally, the target device interface element acquisition module and the simulation operation module on the target smart device are set in an application form; that is, in the target An application is installed on the smart device, and the application can complete the function of the target device-side interface element acquisition module and the simulation operation module; the application program is a simulation operation conversion function of the system, and is not originally owned by the smart device system.
  • the working mode selects the sub-module to select the working mode
  • the instruction is directly sent to the target smart device for the simulation operation; For example, there is no requirement for the location of the movement and click operation, and only the movement and the click action need to be completed, and the device only needs to perform the corresponding movement and click action determined by the configuration according to the execution mode of the conventional Bluetooth mouse.
  • the working mode selection sub-module selects the working mode, if the current configuration information indicates that the operation depends on a simulated operation conversion that is extracted by the application entity, it is checked whether the application is installed or has been run and is in a ready state. If the application is not ready, install and run the application on the target smart device first.
  • the target smart device and the simulated manual operation device are connected by Bluetooth communication, and the application on the target smart device acquires the information of the mobile interface element.
  • the analog manual operation device is sent to simulate the coordinate position of the manual operation device combined with the configured content conversion, and the trigger action and the trigger condition.
  • the simulated manual operation device acquires the interface element information of the target to the application in the target smart device, and simulates the manual operation device to perform parameter filling according to the interface information required for the current configuration, and supplements the interface.
  • the operation instruction of the information parameter is sent to the target smart device, and the analog operation module on the target smart device operates the target smart device according to the received analog operation instruction.
  • the analog conversion module is a first analog conversion module; and the first analog conversion module includes running on a target smart
  • the module collects the interface information of the target smart device, and transmits the interface information to the interface element analysis module; the interface element analysis module outputs the simulated operation instruction to the target smart device according to the received interface information and the simulated operation requirement.
  • the first analog conversion module further includes a simulation operation requirement acquisition module running on the target smart device, the simulation operation requirement acquisition module is not shown in the drawing, and the simulation operation requirement acquisition module passes the communication interface on the target smart device and External device connections for analog operation needs.
  • the actual simulation operation execution depends on the analog operation conversion program running on the target smart device; in fact, in this mode, the simulation operation conversion program is implemented.
  • the device itself In the collection and final execution of the information, the device itself only stores the transfer parameters or the auxiliary analysis in this way. Therefore, in the case where the program exists, the configuration information can be obtained directly from the target smart device and connected to the cloud server. Then complete the subsequent operations.
  • the simulated manual operation device checks whether the analog operation conversion program of the target smart device side is installed or has been running and is in a ready state according to the current configuration information. If the application is not ready, install and run the application on the target smart device first.
  • the device communicates with the program of the target smart device side (based on socket communication), and when it is confirmed that the simulation operation conversion program is ready, the simulated manual operation device acquires the interface element of the target from the application in the target smart device. Information, the simulated manual operation device performs parameter filling according to the interface information required by the current configuration, and sends an operation instruction supplementing the interface information parameter to the target smart device, and the analog operation module on the target smart device according to the received analog operation instruction Operate the target smart device.
  • the final implementation of the simulation operation conversion program is based on the Android AccessiblityService helper class.
  • the method actually corresponds to a specific implementation method under the form of wifi and the application layer of usb. In both cases, the method is dominant to form an actual operation action, and in other cases, the analog device is dominant to pass parameters or Call the interface on the target system to implement the actual action.
  • connection mode between the target smart device and the analog manual operation device is a USB cable electrical signal connection mode; further includes the step of determining whether the current analog operation requirement only requires the USB driver layer analog conversion module; The current analog operation requires only the USB driver layer analog conversion module; the analog manual operation device directly sends a USB driver layer simulation manual operation instruction to the target smart device; the target smart device simulates the manual operation instruction according to the received USB driver layer in the target intelligence. Operate on the device.
  • a USB driver layer analog conversion module is built in the target smart device; the USB driver layer analog conversion module can be a separate driver layer analog conversion module, or a sub-module integrated in the underlying operating system, or integrated in the operating system. Part of the completed USB drive function.
  • the USB driver layer analog conversion module can have different expressions in different operating systems USB driver layer analog conversion modules.
  • the specific process of the USB driver layer analog conversion module is described as follows: the USB path is established between the target smart device and the analog manual operation device, and then the ADB debugging path is established on the USB path, that is, the Android operating system driver layer.
  • the target smart device is the ADB server
  • the analog manual device is the ADB client
  • the ADB service is running on the driver layer at the target smart device, so the USB driver layer
  • the analog conversion module is an analog manual operation device that directly transmits to the target smart device in the form of an ADB command, thereby directly affecting the simulation operation of the device from the driver layer of the target smart device operating system.
  • the USB connection and the ADB debugging path enable the analog manual operation device to send commands or data communication to the driver layer or the application layer of the smart device Android system through the ADB command.
  • connection mode between the target smart device and the analog manual operation device is the USB cable electrical signal connection mode; further includes the step of determining whether the current analog operation requirement only needs the USB application layer analog conversion module; if the current simulation operation requirement requires USB
  • the application layer analog conversion module runs at the target smart device application layer, and the USB application layer analog conversion module is embodied as an application App program, which can be installed in advance or pushed from the simulated manual operation device to the target smart device; USB application layer simulation
  • the conversion module includes a target device-side interface element acquisition module running on the target smart device and an interface element analysis module running on the simulated manual operation device; the target device-side interface element acquisition module collects interface information of the target smart device, and the interface information
  • the USB cable electrical signal connection method is transmitted to the interface element analysis module; the interface element analysis module outputs the simulation operation instruction to the target smart device according to the received interface information and the simulation operation requirement.
  • the USB application layer analog conversion module is an app application.
  • the reason why the app can perform simulated manual operations on the mobile phone is to find the interface element through the accessibility service class and the app application by calling the accessibility service class of the Android operating system. Implement analog clicks.
  • the entire complete path is to simulate the manual operation device through the ADB and the target intelligent device such as the communication terminal. If the simulation operation to be executed requires the app layer to cooperate, first check whether the app exists, if it does not exist, you need to install this.
  • the app can be installed by the user or can be pushed by the device to the target smart device for installation. After the app is run, the interface element is obtained, and the device transmits the final operation parameter to the socket communication through the ADB channel. To the app, the app is responsible for completing the final execution.
  • the device program logic When the target smart device is accessed in the form of USB, the device program logic enters the processing flow of the USB access mode by the identification of the working mode matching pool.
  • the current configuration information and the target smart device information are checked first, and the configuration information of the current final effective operation and the simulation method suitable for the target smart device are determined, and are classified according to different levels of execution on the target smart device system.
  • Drive simulation and application layer simulation where the driver simulation does not need to rely on an additional program to be installed to interact with each other, while application layer simulation requires additional installation or caching of temporarily running programs to match.
  • the driving simulation mode is entered, and the device controller sends a command to complete the interface element information acquisition to perform a specific simulation operation.
  • the driving mode is specifically implemented, the device establishes an ADB connection with the smart device, intercepts the screen of the smart device by using a screen capture command, analyzes the information of the intercepted screen, determines the simulated operation to be performed in combination with the configuration, and sends an input event x, y command combination, thereby Perform the appropriate action at the specified location.
  • the device obtains the interface element of the target from the program.
  • the application layer is specifically implemented, and the ADB connection is established between the device and the smart device.
  • the analog conversion program is represented as an App, and the interaction between the device and the analog conversion program is completed through socket communication, and the simulation conversion program itself performs the actual execution action.
  • the actual execution action is based on the Android AccessiblityService helper class.
  • the simulated manual operation device can be connected to the target intelligent device by wire or wirelessly, thereby realizing a preset simulated manual operation on the target smart device;
  • the design work pattern matching and analog operation function conversion module can play the role of multi-channel selection conversion of the simulation operation strategy, and can be connected according to the target intelligent device and device, including hardware connection methods such as wired connection, wireless connection, software strategy mode such as driving simulation.
  • the application layer simulation and the like to select an appropriate simulation operation command generation method, thereby converting the preset simulation operation configuration information or instructions in the device into actual operations that can be effective on the target smart device;
  • the simulated manual operation device can Input via an external input device Acquire or update the configuration by the access connection in the form of a cloud server.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Selective Calling Equipment (AREA)
  • Telephone Function (AREA)

Abstract

L'invention concerne un procédé pour un dispositif intelligent d'opération manuelle de simulation, comportant les étapes suivantes: étape A0: un appareil d'opération manuelle de simulation détecte et acquiert des informations de méthode de connexion entre un dispositif intelligent cible à faire fonctionner et l'appareil d'opération manuelle de simulation; étape B0: d'après les informations acquises de méthode de connexion, l'appareil d'opération manuelle de simulation sélectionne un module correspondant de conversion de simulation de façon à convertir une exigence d'opération de simulation en une instruction d'opération pour faire fonctionner le dispositif intelligent cible et à envoyer l'instruction d'opération au dispositif intelligent cible; étape C0: selon l'instruction d'opération reçue, le dispositif intelligent cible effectue une opération manuelle de simulation du dispositif intelligent cible. Un procédé adaptatif peut être automatiquement sélectionné d'après la méthode de connexion entre un appareil d'opération manuelle de simulation et un dispositif intelligent cible de façon à convertir une exigence d'opération de simulation en une instruction d'opération de simulation, ce qui le rend compatible avec les méthodes de connexion par Wi-Fi, Bluetooth et par câble USB et applicable à une pluralité de différents scénarios d'application; par conséquent, le rendement des essais pendant les essais en production est considérablement amélioré.
PCT/CN2018/115107 2018-05-03 2018-11-12 Procédé, système et appareil pour dispositif intelligent d'opération manuelle de simulation WO2019210674A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810414681.4 2018-05-03
CN201810414681.4A CN109144807A (zh) 2018-05-03 2018-05-03 模拟人工操作智能设备的方法、系统和装置

Publications (1)

Publication Number Publication Date
WO2019210674A1 true WO2019210674A1 (fr) 2019-11-07

Family

ID=64801673

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/115107 WO2019210674A1 (fr) 2018-05-03 2018-11-12 Procédé, système et appareil pour dispositif intelligent d'opération manuelle de simulation

Country Status (2)

Country Link
CN (1) CN109144807A (fr)
WO (1) WO2019210674A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116501220A (zh) * 2023-06-08 2023-07-28 北京鹅厂科技有限公司 一种应用人工智能的方法与装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106021095A (zh) * 2016-05-06 2016-10-12 北京邮电大学 一种基于推送机制的Android应用自动化测试方法
CN106250307A (zh) * 2016-07-20 2016-12-21 福建省农村信用社联合社 一种银行手机应用自动化测试方法及系统
CN106874206A (zh) * 2017-02-27 2017-06-20 网易（杭州）网络有限公司 用于跨平台自动测试的方法、装置及电子设备
US20170262130A1 (en) * 2016-03-11 2017-09-14 Spirent Communications, Inc. Performance test application sequence script
CN107528959A (zh) * 2017-08-29 2017-12-29 苏州惠邦科信息技术有限公司 利用控制盒检测手机的方法

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103577320A (zh) * 2012-08-07 2014-02-12 卓望数码技术（深圳）有限公司 远程终端测试方法及系统
CN103079019A (zh) * 2012-12-21 2013-05-01 康佳集团股份有限公司 一种通过移动设备控制智能终端的控制方法及系统
CN104571913A (zh) * 2014-12-25 2015-04-29 青岛歌尔声学科技有限公司 一种模拟鼠标控制的系统和方法
CN105046900A (zh) * 2015-07-06 2015-11-11 深圳市新众玩网络科技有限公司 一种用于智能显示设备的交互式操控系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170262130A1 (en) * 2016-03-11 2017-09-14 Spirent Communications, Inc. Performance test application sequence script
CN106021095A (zh) * 2016-05-06 2016-10-12 北京邮电大学 一种基于推送机制的Android应用自动化测试方法
CN106250307A (zh) * 2016-07-20 2016-12-21 福建省农村信用社联合社 一种银行手机应用自动化测试方法及系统
CN106874206A (zh) * 2017-02-27 2017-06-20 网易（杭州）网络有限公司 用于跨平台自动测试的方法、装置及电子设备
CN107528959A (zh) * 2017-08-29 2017-12-29 苏州惠邦科信息技术有限公司 利用控制盒检测手机的方法

Also Published As

Publication number Publication date
CN109144807A (zh) 2019-01-04

Similar Documents

Publication Publication Date Title
CN108804302B (zh) 一种远程测试方法、系统及相关设备
CN102270168A (zh) 自动检测串口信息的方法
CN105302364B (zh) 遥控安卓设备的方法及系统
CN114237676B (zh) 一种fpga逻辑更新方法、装置、设备及可读存储介质
JP5740634B2 (ja) 自動操作システム及び操作自動化方法
CN111190605A (zh) 嵌入式平台的部署方法、设备及存储介质
CN108293199A (zh) 提供事件管理服务的电子装置和方法
WO2019210674A1 (fr) Procédé, système et appareil pour dispositif intelligent d'opération manuelle de simulation
CN107283423A (zh) 智能型测试机器人系统
WO2017215671A1 (fr) Procédé, appareil et système de commande de dispositif d'accès
US11128527B2 (en) Installation support device and method for installation process support for an automation system
CN102750143A (zh) 基于matlab com组件调用的dsp开发方法
CN109189452A (zh) 一种电子设备的软件更新方法、装置及存储介质
CN112099410A (zh) 一种软件自动化控制系统和方法
CN102929159B (zh) 一种仿真模型状态控制方法与装置
CN104022933B (zh) 一种基于多模信号转换的车辆总线调试系统
CN109725621B (zh) 一种基于1553b总线和can总线的二级总线程序在线烧写方法
CN102467372A (zh) 一种测量资源控制系统和测量资源控制方法
WO2021027852A1 (fr) Système de signal de train et procédé de liaison associé
CN103701662A (zh) 自动化测试终端仿真方法及仿真系统
CN103576019A (zh) 电器设备的测试工装、测试系统和测试方法
CN103473170A (zh) 测试系统及测试方法
KR100918840B1 (ko) 센서 네트워크를 구성하기 위한 타겟 센서 노드를 시험하는장치 및 그 방법
US20210018888A1 (en) Automated programming of a programmable-logic controller (plc) of a microcontroller using an expert system
US11762364B2 (en) Automated programming of a programmable-logic controller (PLC) of a microcontroller using an expert system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18917407

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18917407

Country of ref document: EP

Kind code of ref document: A1