TWM584537U - Smart application program installation test system - Google Patents

Abstract

The new model is an intelligent application installation test system, which includes: an image capture unit, which is arranged above a mobile device, for capturing the icon position of the mobile device to generate more than one capture screen; a control module, It is electrically connected with the image capture unit, used to calculate the position of the icon in the capture screen and generate a target information, and store an installation program, multiple installation step screens; a mechanical arm, and the control module. According to the coordinate information, touch the icon and button on the display interface; the new model determines the icon position according to the icon's timepiece capture screen, and the robot arm automatically clicks the icon position and the button position to install and test the application Program to increase testing efficiency.

Description

Smart application installation test system

An application installation system, in particular, an installation system for automatically installing and installing applications using a graphical method.

With the popularity of mobile devices such as mobile phones and tablets, more and more people own mobile devices, and the number of applications (apps) used for mobile devices has also increased. Therefore, many software engineers are constantly developing more functional and convenient applications to meet the needs of the public.

In the process of developing an application, in addition to testing the code of the application itself for defects, you must further install and test it on a mobile device to see if it works properly to ensure that users can use it normally. The existing application test method is to prepare multiple mobile devices of different brands, manually install the application on the mobile device one by one, and open the test application one by one to find whether there is code Vulnerabilities and enhancements.

However, there are many types of mobile phones now, and each major mobile phone manufacturer will launch new mobile phones after a period of time. The high mobile phone replacement rate makes it possible for one application to operate normally on this mobile device and on another updated operation. Device operation often crashes. If you manually install test applications one by one, it will cause a significant increase in labor and time costs for testing, which is not conducive to application testing.

In order to improve the efficiency of installation and testing of applications on mobile devices, the present invention proposes a smart application installation testing system. The image capture unit captures the phone screen, and the control module determines the position of the icons in the screen and controls the robotic arm. Step-by-step procedures for installing and installing applications, increase the testing efficiency when installing and testing applications in an automated manner.

In order to achieve the above purpose, the smart application installation test system provided by the new model includes:
An image capture unit is disposed above a mobile device, and is used to capture a display interface of the mobile device to generate a capture screen, in which more than one icon and a plurality of keys are displayed on the display interface;
A control module electrically connected to the image capturing unit for calculating the position of the icon and the plurality of keys in the capturing screen and generating coordinate information belonging to the icon and one of the plurality of keys, the control module An installation program, a plurality of installation step screens, and a specified application name are stored; wherein the control module compares the captured screen with each installation step screen to determine the installation steps of the specified application, the icon, and the plurality of keys. The coordinate information;
A robotic arm is electrically connected to the control module, wherein the control module judges the installation steps of the designated application based on the captured screen and the plurality of installation step screens, and controls the robotic arm to follow the plurality of installation step screens and For the coordinate information, click the one or more icons and the plurality of buttons.

In the new model, the image capture unit is used to capture the picture of the icon on the mobile device, and the control module calculates the coordinate information of the icon and stores each picture displayed by the mobile device during the installation of the application program. When the control module determines that the captured image of the icon matches one of the stored captured image installation steps, the control module controls the robotic arm to click the corresponding icon to complete the application installation, whether it is a mobile phone, tablet, etc. The device, or a computer (vehicle machine) or even the Internet of Things can be applied, which can greatly shorten the testing time, and does not require manual testing of each mobile device, improving the convenience of testing.

The new type of image capture unit can use a camera with a depth of field lens to determine the distance between the mobile device and the image capture unit by using infrared or laser to correct the depth of the mobile device when the mobile device clicks on the device. The mobile device cannot be tested due to its different thickness, for example: it is too far away from the robotic arm and not touched by the robotic arm, or because the mobile device is too close to the robotic arm and the force of the robotic arm is too large, the The mobile device is damaged and the freedom of testing is high.

In addition, the new method uses coordinates to locate the icon position. The coordinates of the icon can still be calculated when the mobile device is not squared, and then the correct icon can be clicked correctly, which can also increase the freedom of testing.

Referring to FIG. 1 and FIG. 2, the present invention provides a smart application installation test system that can automatically operate the application installation process and correctly install the application on a mobile device 40. The smart application installation test system includes: an image capture unit 10, a control module 20, and a robot arm 30.

The image capturing unit 10 can be mounted on a bracket 60 and located above the mobile device 40 to capture the mobile device 40 and capture a picture of a display interface of the mobile device 40 to generate more than one captured image. The display interface of the mobile device 40 includes more than one icon and a plurality of buttons. In the preferred embodiment of the present invention, the image capturing unit 10 is a camera with a depth-of-field lens, and the distance between the mobile device 40 and the image capturing unit 10 can be determined by means of infrared or laser.

The control module 20 is electrically connected to the image capturing unit 10, and is configured to calculate the positions of the one or more icons and the plurality of keys in the capturing screen and generate the icons or one of the plurality of keys. Coordinate information. The control module 20 simultaneously establishes an installation program, a plurality of installation step screens, and more than one designated application program name, wherein each installation step screen is a display of each step on the mobile device 40 when installing the designated application program. Take the picture. The control module 20 may further include a display screen 21. The display screen 21 may be used to display the multiple installation step screens, the capture screens, and instructions for the installation program.

When the control module 20 executes the installation program, the control module 20 controls the robotic arm 30 to operate when comparing the captured screen with the plurality of installation step screens and determining that the captured screen matches one of the installation step screens The mobile device 40 installs a designated program. In a preferred embodiment of the present invention, the coordinate information indicates the positions of the one or more icons and the plurality of keys by using XYZ three-axis coordinates.

The robot arm 30 is electrically connected to the control module 20 and includes a movable arm 31 and a touch lever 33, wherein the touch lever 33 is disposed on the movable arm 31 to move with the movable arm 31. The robot arm 30 is used to contact the icon and the plurality of keys displayed on the display interface using the touch lever 33 according to the coordinate information, instead of manual clicking, to perform application installation on the mobile device 40 in an automated manner. step.

With the above-mentioned components, the following describes the operation flow of the control module 20 of the present invention for executing the installation program.

Step1: Enter the app store. Referring to FIG. 3, the image capture unit 10 captures the capture screen of the display interface 41 of the mobile device 40, and an application store icon 411 is displayed on the display interface 41. The control module 20 receives the captured screen and compares the plurality of installation step screens. When the control module 20 determines that the captured screen is consistent with one of the installation step screens, it determines that this step is a step of entering the application store. The control module 20 controls the touch lever 33 of the robot arm 30 to click the application store icon 411 of the display interface 41.

Step2: Search for the specified application name. Referring to FIG. 4, after the touch bar 33 clicks the application mall icon 411, the display interface 41 then displays a search field 412. The image capture unit 10 captures the capture screen with the search field 412 and compares the multiple installation step screens; when the control module 20 determines that the capture screen with the search field 412 and one of the installation steps When the screens match, it is deemed that this step is a step for searching the specified application name, and the touch bar 33 clicks the search field 412.

Step3: Enter the name of the specified application. Referring to FIG. 5, after the touch bar 33 clicks the search field 412, the display interface 41 then displays an input keyboard 413. The control module 20 controls the touch bar 33 and clicks the input keyboard 413 to input the specified application name according to the preset application name to search for a specified application corresponding to one of the specified application names.

Step4: Install the specified application. Referring to FIG. 6, after the specified application is found, the display interface 41 displays an installation interface of the specified application, and an icon 414 of the specified application and an installation key 415 are displayed on the installation interface. The control module 20 operates the touch lever 33 and clicks the installation key 415 to enable the mobile device 40 to install the designated application.

Step5: The specified application is installed. Referring to FIG. 7, after the designated application is installed, the display interface 41 displays the installation completion interface, and displays a remove button 416 and an open button 417.

Step6: Return to the desktop. Referring to FIG. 8, the control module 20 operates the touch lever 33 and clicks a return key 418 of the display interface 41 to return the display interface 41 to the desktop screen. At the same time, an icon 414 of the designated application is displayed on the display interface 41 that presents the desktop.

Step7: Check whether the specified application is installed correctly. The control module 20 operates the touch lever 33 to click the icon 414 of the designated application on the desktop, and then the mobile device 40 displays a screen after the designated application is opened. The image capturing unit 10 captures the screen of the specified application and compares the screens of the multiple installation steps with the control module 20; if the screen of the specified application opens with one of the installation steps, the control Module 20 determines that the specified application is successfully installed; if the screen after the specified application is opened does not match all the installation step screens, the control module 20 determines that the specified application fails to be installed, and the control module 20 restarts Execute the installer.

In the second preferred embodiment of the present invention, the control module 20 can execute a deep learning mode, wherein the control module 20 can learn to record the appearance of the icon on the captured screen through the continuously acquired captured screen. And text. When using different mobile phones for installation and testing, the control module 20 can automatically recognize the appearance and position of the icon, and execute the installation process of the new model, and send the coordinates back to the arm for clicking or sliding, etc. The captured images are marked. Furthermore, the control module 20 can also set a test schedule time in advance, and control the robotic arm 10 to test various mobile devices 40 at a specific time.

10‧‧‧Image Acquisition Unit

20‧‧‧Control Module
21‧‧‧display

30‧‧‧ robotic arm
31‧‧‧ movable arm

33‧‧‧touch lever
40‧‧‧ mobile device

41‧‧‧display interface
411‧‧‧App store icon

412‧‧‧Search field
413‧‧‧Input keyboard

414‧‧‧Specified application icon
415‧‧‧install key

416‧‧‧Remove key
417‧‧‧open key

418‧‧‧Back key
60‧‧‧Scaffold

Figure 1: Circuit block diagram of the new model.
Figure 2: Schematic diagram of the actual test screen of the new model.
Figure 3 ~ Figure 8: Schematic diagram of the steps for installing the new application on a mobile device.

Claims (7)

A smart application installation test system including:
An image capture unit is disposed above a mobile device, and is used to capture a display interface of the mobile device to generate a capture screen, in which more than one icon and a plurality of buttons are displayed on the display interface;
A control module electrically connected to the image capturing unit for calculating the position of the icon and the plurality of keys in the capturing screen and generating coordinate information belonging to the icon and one of the plurality of keys, the control module An installation program, a plurality of installation step screens, and a specified application name are stored; wherein the control module compares the captured screen with each installation step screen to determine the installation steps of the specified application and the icon and the plurality of keys The coordinate information;
A robotic arm is electrically connected to the control module, wherein the control module judges the installation steps of the designated application program according to the captured screen and the multiple installation step screens, and controls the robotic arm to follow the multiple installation step screens and For the coordinate information, click the one or more icons and the plurality of buttons. According to the smart application installation test system described in claim 1, the image capture unit is a camera with an infrared or laser depth of field lens. The smart application installation test system according to claim 2, the robotic arm includes a movable arm and a touch bar, wherein the touch bar is arranged on the movable arm, and the touch bar is used to touch the display The icon displayed on the interface. According to the smart application installation test system described in claim 3, the one or more icons include an application store icon and a designated application icon. According to the smart application installation test system described in claim 4, the plurality of keys includes a search field, an input keyboard, an installation key, a return key, and an open key. According to the smart application installation test system described in claim 5, the image capturing unit is mounted on a bracket and is arranged above the mobile device. According to the smart application installation test system described in claim 6, the control module further executes a deep learning mode, wherein the control module learns to record the appearance of the icon on the captured screen through the continuously acquired captured screen. And text.