CN110231885B - Touch simulation device, control method and method for controlling intelligent equipment - Google Patents
Touch simulation device, control method and method for controlling intelligent equipment Download PDFInfo
- Publication number
- CN110231885B CN110231885B CN201910498026.6A CN201910498026A CN110231885B CN 110231885 B CN110231885 B CN 110231885B CN 201910498026 A CN201910498026 A CN 201910498026A CN 110231885 B CN110231885 B CN 110231885B
- Authority
- CN
- China
- Prior art keywords
- touch
- strip
- control
- touch control
- processor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/0412—Digitisers structurally integrated in a display
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/0416—Control or interface arrangements specially adapted for digitisers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/044—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by capacitive means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Position Input By Displaying (AREA)
Abstract
The invention relates to a touch control simulation device, a control method and a method for controlling intelligent equipment, which are mainly suitable for controlling and testing an intelligent terminal with a capacitive sensing touch screen, such as a PC (personal computer), a mobile phone or a tablet personal computer. According to the touch control principle of the capacitive sensing type touch screen, the touch control of the touch screen is simulated through the cooperation of the transverse touch control strips and the longitudinal touch control strips, the traditional manual touch control operation can be replaced, the touch control simulation device and the control method thereof are high in touch control accuracy and operation efficiency, and the touch control simulation device and the control method thereof can also be used for complex simulated touch control operation; according to the method for controlling the intelligent equipment, disclosed by the invention, the operation instruction matched with the touch control simulation device can be generated according to the image data information of the software, and the closed-loop control is formed, so that the method can be used for automatically testing the software.
Description
Technical Field
The invention relates to a touch control simulation device, a control method and a method for controlling intelligent equipment, which are mainly suitable for controlling and testing an intelligent terminal with a capacitive sensing touch screen, such as a PC (personal computer), a mobile phone or a tablet personal computer.
Background
With the rapid development of communication technology, the functions of the terminal are also more and more powerful. In order to meet the requirement that a user can simply, conveniently and naturally perform human-computer interaction with a terminal, more and more terminals are provided with touch screens as display screens. Touch screen (touch screen), also known as touch screen or touch panel, is an inductive liquid crystal display device capable of receiving input signals such as touch, when touching graphic buttons on the screen, the touch feedback system on the screen can drive various connecting devices according to pre-programmed programs, and can be used to replace mechanical button panels and produce vivid video and audio effects by means of liquid crystal display pictures. According to the working principle and the medium for transmitting information, the touch screen can be divided into four different types, namely a resistance type, a capacitance induction type, an infrared type and a surface acoustic wave type.
As shown in fig. 1, in the capacitive sensing touch screen, different ITO conductive circuit modules are etched on two layers of ITO conductive glass coatings. The etched patterns on the two modules are perpendicular to each other as shown in fig. 1, and can be considered as slides with continuously changing X and Y directions. Since X, Y is structured on different surfaces, their intersections form a capacitance node. One slide may be defined as the drive line a1 and the other slide as the sense line b 1. When current passes through one of the conductors in drive line a1, a change in capacitance node on the other conductor will be caused if a signal is externally asserted that the capacitance has changed. The variation of the detected capacitance can be measured by the electronic circuit connected with the capacitance, and then converted into digital signals by the A/D controller to make the computer perform calculation processing to obtain the position of the (X, Y) axis, thereby achieving the purpose of positioning. The controller supplies current to the driving line a1, so that a specific electric field is formed between each node and the wire. Then, the line-by-line scanning sensing lines measure the capacitance variation between the electrodes, thereby achieving multi-point positioning. When the finger or the touch medium approaches, the controller rapidly detects the capacitance change between the touch node and the wire, and then confirms the touch position.
As human-computer interaction software, before the software is put into commercial operation formally, usability, functional aspects, pictures, performance, required configuration and the like of the software are required to be tested, and for testing the human-computer interaction performance and the control effect of the software, a software developer usually adopts an internal test or public test mode to test the running conditions of a large number of users by using more ginseng and tests. However, due to the influence of various factors, many software developers either do not do test planning and test design at all, or complete test planning and design quickly just before starting to execute the test, in this case, the test only verifies the correctness of the program, but not the functions and the use effects of the whole system, or even if the test is performed by adopting a large amount of manual participation, the subjective factors during manual control are too much, such as the operation mode, the operation skill and the like cannot meet the predetermined requirements, and the test evaluation with higher requirements cannot be performed.
Disclosure of Invention
The invention aims to provide a touch control simulation device, a control method and a method for controlling intelligent equipment, and solves the technical problems that a touch screen cannot be accurately controlled and manual touch control cannot be automatically simulated during human-computer interaction software testing.
The technical scheme adopted by the invention for solving the problems is as follows: a touch simulation device is used for a capacitive sensing touch screen, the touch screen comprises driving circuits and detecting circuits which are arranged in a criss-cross array mode, the touch simulation device comprises four groups of metal touch strips, namely a first touch strip, a second touch strip, a third touch strip and a fourth touch strip, and each touch strip is formed by arranging a plurality of metal contact pieces with equal length at equal intervals; the first touch strip and the second touch strip are respectively covered at the two transverse ends of the touch screen and are electrically coupled with the driving circuit of the touch screen so as to acquire a current mutual inductance signal of the driving circuit, and the metal contact pieces of the first touch strip and the second touch strip are correspondingly arranged in a staggered manner in the transverse direction; the third touch strip and the fourth touch strip are respectively covered at the two longitudinal ends of the touch screen and are electrically coupled with the detection circuit of the touch screen so as to output a simulation current mutual inductance signal of the detection circuit, and the metal contact pieces of the third touch strip and the fourth touch strip are correspondingly arranged in a staggered manner in the longitudinal direction; the touch control simulation device further comprises a first processing circuit, wherein the first processing circuit comprises an analog signal processing circuit and a first processor, the analog signal processing circuit is connected with the first processor, the analog signal processing circuit comprises a multi-path input analog switch, an analog signal level conversion circuit and a multi-path output analog switch which are sequentially connected, metal contact pieces of a first touch control strip and a second touch control strip are respectively connected with the input end of the multi-path input analog switch, and metal contact pieces of a third touch control strip and a fourth touch control strip are respectively connected with the output end of the multi-path output analog switch; the first processor is used for receiving a touch operation instruction, controlling the touch sheet corresponding to the multi-channel input analog switch to acquire a current mutual inductance signal of a line at a corresponding position on the driving line, and outputting an analog current mutual inductance signal at a corresponding position on the detection line through the touch sheet corresponding to the multi-channel output analog switch after being processed by the analog signal level conversion circuit so as to simulate the touch operation on the touch position.
Through the scheme, according to the touch control principle of the capacitive sensing type touch screen, the touch control of the touch screen is simulated through the cooperation between the transverse touch control strips and the longitudinal touch control strips. The touch control strips are arranged into four groups, each group of touch control strips consists of a plurality of metal contact pieces, and the metal contact pieces on each corresponding touch control strip are correspondingly and staggeredly arranged on the four sides of the touch screen, so that the simulated touch operation of all areas on the touch screen in a square range formed by the four groups of touch control strips can be realized. Each touch strip is composed of a plurality of equilong metal contacts which are arranged at equal intervals, the metal contacts can be pure gold or metal contacts adopting a gold immersion process, current mutual inductance signal connection can be well formed between the driving circuit and the detection circuit, and a coordinate system can be established for a simulation operation area by combining an array arrangement form of the metal contacts by adopting a scheme that the equilong metal contacts are arranged at equal intervals, so that correspondence with an image coordinate system of a software interface is conveniently formed. The density of touch control points within the touch range can be controlled by varying the size of the metal touch pads. After the scheme is adopted, automatic simulation touch operation control in different modes can be realized by setting different touch operation instructions, and the control precision is high.
According to the touch simulation device of the above scheme, the operation method of one touch action may be: the first processor controls the analog signal processing circuit to control a metal contact on the first touch strip or the second touch strip to collect a current mutual inductance signal of the driving circuit, and the analog current mutual inductance signal of the detection circuit is output by controlling a metal contact at a corresponding coordinate position on the third touch strip or the fourth touch strip to control single-point analog touch operation in a certain functional area.
Further, the arrangement mode of each metal contact can adopt the following scheme: the metal contact pieces of the first touch strip and the second touch strip are correspondingly staggered in the transverse direction and are arranged at intervals, and the metal contact pieces of the third touch strip and the fourth touch strip are correspondingly staggered in the longitudinal direction and are arranged at intervals. The arrangement mode is simple and convenient, and for the simulation operation of the touch point, one metal contact on the first touch strip or the second touch strip can be controlled to acquire the current mutual inductance signal of the driving circuit, and simultaneously one metal contact on the third touch strip or the fourth touch strip corresponding to the coordinate position is controlled to output the simulation current mutual inductance signal of the detection circuit, so that the single-point simulation touch operation in a certain functional area can be controlled.
Further, in the invention, the metal contact pieces of the first touch strip and the second touch strip are correspondingly staggered and partially overlapped in the transverse direction, and the metal contact pieces of the third touch strip and the fourth touch strip are correspondingly staggered and partially overlapped in the longitudinal direction. By adopting the scheme, compared with the staggered and spaced arrangement mode, the density of the touch control points can be effectively improved in the touch range under the condition of the metal contact pieces with the same size.
Furthermore, the first processing circuit of the invention is provided with a plurality of groups of analog signal processing circuits, and each group of analog signal processing circuits is connected with the first processor; and the metal contact pieces of the third touch strip and the fourth touch strip are respectively connected with the output ends of the multi-path output analog switches of the analog signal processing circuits.
According to the touch simulation device of the above-mentioned scheme, the operation method of one touch action may also be: the first processor simultaneously controls a plurality of groups of analog signal processing circuits to simultaneously control the analog touch operation of a plurality of single points in a certain functional area respectively, so as to form single-point analog touch operation in a chip area.
According to the touch simulation device of the above scheme, the operation method of one touch action may further include: the first processor simultaneously controls a plurality of groups of analog signal processing circuits to simultaneously control the analog touch operation of a plurality of single points in a plurality of different functional areas respectively, so as to form multi-point analog touch operation.
A method for controlling an intelligent device comprises a second processing circuit and an AI processor, wherein the second processing circuit comprises an image acquisition module and a second processor, and the control method comprises the following steps:
(1) image acquisition: and image information acquired through an image output port of the image sensor or the intelligent equipment is transmitted to the AI processor through the second processor.
(2) Image recognition and motion detection: and (3) performing feature extraction and image recognition on the image obtained in the step (1) by adopting a convolutional neural network technology through an AI processor to obtain the position and category information of each object in the image.
(3) Generating a control command: the AI processor continuously generates the position and category information, when the occurrence of a certain event in the image is monitored, the position and category information is reported to the second processor, a control program preset by the second processor is used for generating a control command signal corresponding to the next event according to the current event, and the event is a phenomenon that a certain type or a certain plurality of types of objects appear or disappear in the image and a phenomenon that the position of the certain type or the certain plurality of types of objects relatively or absolutely changes.
(4) Generating and sending a touch operation instruction: the control command signal is generated by the second processor into a touch operation instruction which can be identified by the touch simulation device and is sent to the touch simulation device.
(5) Executing a touch operation instruction: and after receiving the touch operation instruction, the touch control simulation device controls software running on the intelligent equipment to perform corresponding operation control in a touch operation simulation mode.
(6) And (3) after the touch operation instruction in the step (5) is executed, continuously repeating the control process of the steps (1) to (3), comparing the collected new event information with the next event in the step (3) by the AI processor to generate new event information, correspondingly generating a new control command signal, and continuously repeating the control process of the steps (4) to (5) to form closed-loop control.
In the step (3), the control command signal is: and the software control program changes the image after the current event occurs into the image after the next event occurs.
In the step (4), the touch operation instruction is used for simulating touch operation on the intelligent device, the simulated touch operation is a behavior of performing touch on a functional area of a corresponding image on the touch screen, a coordinate position of a touch control point corresponding to the touch operation instruction is set corresponding to a position of the functional area on the image, and a coordinate system adopted by the coordinate position is a coordinate network formed by a driving line and a detection line.
The method for controlling the intelligent equipment obtains the data information of a software interface through image acquisition, identifies and moves detection data of an image by adopting a convolutional neural network technology through an AI processor, generates a control command through the data information, and executes the control command by adopting a touch simulation device so as to realize the operation control of the software, and the whole control process forms closed-loop control and can be used for automatic test of the software operation function.
Compared with the prior art, the invention has the following advantages and effects: the touch control simulation device and the control method thereof disclosed by the invention can replace the traditional manual touch control operation, have high touch control precision and high operation efficiency, and can also be used for complex operation; according to the method for controlling the intelligent equipment, disclosed by the invention, the operation instruction matched with the touch control simulation device can be generated according to the image data information of the software, and the closed-loop control is formed, so that the method can be used for automatically testing the software.
Drawings
Fig. 1 is a schematic structural diagram of a driving circuit and a detecting circuit of a capacitive sensing touch screen.
Fig. 2 is a first schematic view of an arrangement structure of the touch strip according to the present invention.
Fig. 3 is a second schematic view of the layout structure of the touch strip according to the present invention.
Fig. 4 is a third schematic view of the layout structure of the touch strip according to the present invention.
Fig. 5 is a fourth schematic view of the arrangement structure of the touch strip according to the present invention.
FIG. 6 is a first connection diagram of the analog signal processing circuit according to the present invention.
Fig. 7 is a second connection diagram of the analog signal processing circuit according to the present invention.
Fig. 8 is a schematic diagram of simulating touch points according to an embodiment of the invention.
Fig. 9 is a control flow chart of the control smart device of the present invention.
In the drawings: a1 is a driving circuit, b1 is a detecting circuit, 1 is a first touch bar, 2 is a second touch bar, 3 is a third touch bar, 4 is a fourth touch bar, 5 is a metal contact, 6 is an analog signal processing circuit, 61 is a multi-input analog switch, 62 is an analog signal level converting circuit, and 63 is a multi-output analog switch.
Detailed Description
The present invention will be described in further detail below by way of examples with reference to the accompanying drawings, which are illustrative of the present invention and are not to be construed as limiting the present invention.
As shown in fig. 2-5, the touch simulation apparatus of the present embodiment includes four sets of metal touch bars, which are a first touch bar 1, a second touch bar 2, a third touch bar 3, and a fourth touch bar 4, and each touch bar is composed of a plurality of metal touch pads 5 with equal length arranged at equal intervals.
The first touch strip 1 and the second touch strip 2 cover the two transverse ends of the touch screen respectively and are electrically coupled with a driving circuit of the touch screen to acquire a current mutual inductance signal of the driving circuit, and the metal contact pieces 5 of the first touch strip 1 and the second touch strip 2 are correspondingly arranged in a staggered manner in the transverse direction; the third touch strip 3 and the fourth touch strip 4 respectively cover the longitudinal two ends of the touch screen and are electrically coupled with the detection lines of the touch screen to output analog current mutual inductance signals of the detection lines, and the metal contact pieces 5 of the third touch strip 3 and the fourth touch strip 4 are correspondingly arranged in a staggered mode in the longitudinal direction.
The arrangement of the metal contact pieces 5 in this embodiment may adopt the following scheme: as shown in fig. 2, the metal contact pads 5 of the first touch bar 1 and the second touch bar 2 are correspondingly staggered and partially overlapped in the transverse direction, and the metal contact pads 5 of the third touch bar 3 and the fourth touch bar 4 are correspondingly staggered and partially overlapped in the longitudinal direction. Of course, the following scheme may also be selected: the metal contact pieces 5 of the first touch strip 1 and the second touch strip 2 are correspondingly staggered in the transverse direction and are arranged at intervals, and the metal contact pieces 5 of the third touch strip 3 and the fourth touch strip 4 are correspondingly staggered in the longitudinal direction and are arranged at intervals, as shown in fig. 3.
To explain the operation and effects of the embodiments of the present embodiment, the applicant designates the embodiment shown in fig. 2 as embodiment one and the embodiment shown in fig. 3 as embodiment two. Further assume a third solution, which only includes the first touch bar 1 and the third touch bar 3, as shown in fig. 4-5, the first touch bar 1 and the third touch bar 3 in fig. 4 are arranged in the same manner as the first touch bar 1 and the third touch bar 3 in fig. 2, and the first touch bar 1 and the third touch bar 3 in fig. 5 are arranged in the same manner as the first touch bar 1 and the third touch bar 3 in fig. 3. The metal contact pieces 5 in fig. 2 to 5 have the same size, the driving lines and the detecting lines corresponding to the centers of the metal contact pieces 5 are connected to form a plurality of intersections, the number of the intersections can represent the distribution density of the analog touch points, and the touch point density in each figure sequentially from small to large: FIG. 5 < FIG. 4 < FIG. 3 < FIG. 2.
As shown in fig. 6, the touch-control simulation device further includes a first processing circuit, the first processing circuit includes a plurality of sets of analog signal processing circuits 6 and a first processor, each set of analog signal processing circuits 6 is connected to the first processor, each set of analog signal processing circuits 6 includes a multi-input analog switch 61, an analog signal level conversion circuit 62 and a multi-output analog switch 63, which are connected in sequence, each metal contact 5 of the first touch bar 1 and the second touch bar 2 is connected to an input terminal of the multi-input analog switch 61 of each set of analog signal processing circuits 6, and each metal contact 5 of the third touch bar 3 and the fourth touch bar 4 is connected to an output terminal of the multi-output analog switch 63 of each set of analog signal processing circuits 6. A set of analog signal processing circuits 6 may also be provided in the first processing circuit, as shown in fig. 7, multiple sets of analog signal processing circuits 6 are adopted in this embodiment.
As shown in fig. 8, the control method of the touch simulation apparatus of the present embodiment is as follows:
single point simulated touch action in a certain functional area: the first processor receives a touch operation instruction of a point A to be simulated, controls the multi-input analog switch 61 of one group of analog signal processing circuits 6, acquires a current mutual inductance signal on a current driving line through the c6 metal contact 5, simultaneously controls the multi-output analog switch 63 of the group of analog signal processing circuits 6, and outputs an analog current mutual inductance signal corresponding to a detection line through the d1 metal contact 5 so as to realize the operation of the point A to be simulated.
Multi-point simulated touch actions in different functional areas: the first processor receives a touch operation instruction for simulating touch points A and B simultaneously, controls the multi-path input analog switch 61 of one group of analog signal processing circuits 6, acquires a current mutual inductance signal on a current driving line through the c6 metal contact 5, controls the multi-path output analog switch 63 of the group of analog signal processing circuits 6 simultaneously, and outputs an analog current mutual inductance signal corresponding to a detection line through the d1 metal contact 5; meanwhile, the first processor controls the multi-input analog switch 61 of another group of analog signal processing circuits 6, acquires current mutual inductance signals on the current driving circuit through the c5 metal contact 5, controls the multi-output analog switch 63 of the group of analog signal processing circuits 6 at the same time, and outputs analog current mutual inductance signals corresponding to the detection circuit through the d2 metal contact 5; so as to realize the simulation of the touch of the point A and the point B at the same time.
A single point of a region in a certain functional region simulates touch operation: the first processor receives a touch operation instruction for simulating touch points C, D, E and F at the same time, controls a multi-path input analog switch 61 of one group of analog signal processing circuits 6, acquires current mutual inductance signals on a current driving circuit through an E3 metal contact 5, controls a multi-path output analog switch 63 of the group of analog signal processing circuits 6 at the same time, and respectively outputs analog current mutual inductance signals corresponding to a detection circuit through a D5 metal contact 5 and a D12 metal contact 5; meanwhile, the first processor controls the multi-input analog switch 61 of another group of analog signal processing circuits 6, acquires current mutual inductance signals on the current driving circuit through the e8 metal contact 5, controls the multi-output analog switch 63 of the group of analog signal processing circuits 6, and respectively outputs analog current mutual inductance signals corresponding to the detection circuit through the d5 metal contact 5 and the d12 metal contact 5; so as to realize the single-point simulation touch operation of the area formed by C, D, E, F points together.
The analog signal processing circuit 6 in this embodiment includes an input amplifying circuit, a digital amplitude modulation circuit and a positive and negative amplitude limiting circuit, because the mutual induction current of the capacitive screen collected by the metal contact 5 of the signal receiving end is very small, the capacitive screen needs to be amplified by the amplifying circuit, the amplification factor of the part of the circuit is generally fixed, the amplification factor can be controlled by software according to actual requirements, the amplification factor of the digital amplitude modulation circuit can be controlled by software at the rear end, the total amplification factor of the signal is the front end fixed amplification factor multiplied by the rear end digital amplitude modulation amplification factor, the amplified signal may generate a level signal with a high peak value due to noise and other reasons, the signal is easily identified as noise so as to ignore the analog touch, and in addition, the excessively high current mutual induction signal is easily broken through the capacitive screen to cause equipment damage, the back-end amplitude-limited circuit generally limits the signal peak value to be not more than plus or minus 12 volts.
The method for controlling the intelligent device in the embodiment includes a second processing circuit and an AI processor, the second processing circuit includes an image acquisition module and a second processor, and the control method includes:
(1) image acquisition: and image information is acquired through an image output port of the image sensor or the intelligent device and is transmitted to the second processor, and the second processor transmits the image information to the AI processor. The captured video or picture, if encoded and compressed, needs to be decoded first. The captured content, if in video format, requires time slicing of the video into a series of pictures.
The image acquisition mode comprises a wired mode and a wireless mode. Wired for devices with video output ports, which typically include HDMI, DVI, VGA, USB, TYPE-C, MHL, Lighting, etc.; the wireless mode adopts wireless image transmission technologies such as WiFi and Bluetooth; or may receive the video signal directly with an image sensor, etc.
(2) Image recognition and motion detection: and (3) performing feature extraction and image recognition on the image obtained in the step (1) by adopting a convolutional neural network technology through an AI processor to obtain the position and category information of each object in the image.
The process and principle of using the convolutional neural network technology to identify the pictures of the acquired images are as follows: the acquired image is first divided into a plurality of overlapping independent small blocks, for example, into 77 small pictures with the same size. Each individual patch is then input to a small neural network that has been trained to determine whether a picture belongs to a certain class, which outputs a feature array.
After all the independent small blocks are input into the small neural network, each output characteristic array is arranged according to the relative positions of the 77 independent small blocks in the first step to obtain a new array. The same calculation and weight sharing were performed on the 77 same-sized small pictures by the small neural network.
For data sets such as images, the values of the local arrays are often highly correlated, which can create unique local features that are easily detected; local statistical features of images and other signals are less relevant to their location, and can occur anywhere if the feature map can appear in one part of the picture. The cells in different locations share the same weight and detect the same pattern in different parts of the array. The filtering operation performed by a signature graph is a discrete convolution.
And after the convolution step is finished, reducing the pixel sampling array by using an algorithm, dividing the characteristic matrix according to 2 multiplied by 2, only keeping the maximum array in each divided grid, and discarding other arrays to obtain the maximum pooling array.
The maximum pooling array is then used as an input to another neural network, which ultimately calculates whether the map meets the desired criteria.
In practical application, the convolution, the maximum pooling and the full-connection neural network calculation can be repeated for many times, and the general idea is to continuously compress a large picture until a single value is output. With more convolution steps, the neural network can process and learn more features.
(3) Generating a control command: the AI processor continuously generates the position and category information, when the occurrence of a certain event in the image is monitored, the position and category information is reported to the second processor, a control program preset by the second processor is used for generating a control command signal corresponding to the next event according to the current event, and the event is a phenomenon that a certain type or a certain plurality of types of objects appear or disappear in the image and a phenomenon that the position of the certain type or the certain plurality of types of objects appears relatively or absolutely changes. The control command signals of this embodiment are: and the software control program changes the image after the current event occurs into the image after the next event occurs. The software control program is a preset control program, is a function operation control program required to be made after certain events occur, can be simple command waiting, and can also be a command after operation.
(4) Generating and sending a touch operation instruction: the control command signal is generated into a touch operation instruction which can be identified by the touch simulation device by the second processor and is sent to the touch simulation device; the touch operation command refers to a series of machine control level signals.
The touch operation instruction is used for simulating touch operation of the intelligent device, the simulating touch operation is a behavior of performing touch on a functional area of a corresponding image on a touch screen, a coordinate position of a touch control point corresponding to the touch operation instruction is set corresponding to the position of the functional area on the image, and a coordinate system adopted by the coordinate position is a coordinate network formed by a driving circuit and a sensing circuit.
(5) Executing a touch operation instruction: and after receiving the touch operation instruction, the touch control simulation device controls the software running on the intelligent equipment to perform corresponding operation control in a touch operation simulation mode.
(6) And (4) after the touch operation instruction in the step (5) is executed, continuing to repeat the control processes of the steps (1) - (3), comparing the collected new event information with the next event in the step (3) by the AI processor to generate new event information, correspondingly generating a new control command signal, and continuing to repeat the control processes of the steps (4) - (5) to form closed-loop control.
It should be noted that, the first processor and the second processing circuit mentioned in the present invention may adopt an ARM processor or a higher-level performance processor, and the specific software program that controls the first processing circuit and the like is implemented by the conventional technical means according to the disclosure of the present invention by those skilled in the art, and the protection scope of the present invention is not limited by the control program and the circuit itself; the AI processor used for image recognition, motion detection, generation of control commands, etc. mentioned in the present invention may be a chip of a commercially available NPU or TPU architecture or a chip with higher performance, and the specific software algorithm and program thereof are implemented by those skilled in the art by conventional technical means according to the disclosure of the present invention, and the protection scope of the present invention is not limited by the control program and circuit thereof.
Although the present invention has been described with reference to the above embodiments, it should be understood that the scope of the present invention is not limited thereto, and that various changes and modifications can be made by those skilled in the art without departing from the spirit and scope of the present invention.
Claims (6)
1. A method for controlling an intelligent device is characterized in that the control method comprises the following steps:
(1) image acquisition: image information is collected through an image output port of the image sensor or the intelligent equipment and is transmitted to the AI processor through the second processor;
(2) image recognition and motion detection: performing feature extraction and image recognition on the image obtained in the step (1) by adopting a convolutional neural network technology through an AI processor to obtain the position and category information of each object in the image;
(3) generating a control command: the AI processor continuously generates the position and category information, when monitoring that a certain event occurs in the image, the information is reported to the second processor, a preset control program is used for generating a control command signal corresponding to the next event according to the current event through the second processor, and the event is a phenomenon that a certain type or a certain plurality of types of objects appear or disappear in the image and a phenomenon that the position of the certain type or the certain plurality of types of objects relatively or absolutely changes;
(4) generating and sending a touch operation instruction: the control command signal is generated into a touch operation instruction which can be identified by the touch simulation device by the second processor and is sent to the touch simulation device;
(5) executing a touch operation instruction: after receiving the touch operation instruction, the touch control simulation device controls software running on the intelligent equipment to perform corresponding operation control in a touch operation simulation mode;
(6) after the touch operation instruction in the step (5) is executed, continuing to repeat the control process of the steps (1) - (3), comparing the acquired new event information with the next event in the step (3) by the AI processor to generate new event information, correspondingly generating a new control command signal, and continuing to repeat the control process of the steps (4) - (5) to form closed-loop control;
the touch control simulation device in the step (4) is used for a capacitance induction type touch screen, the touch screen comprises driving circuits and detecting circuits which are arranged in a criss-cross array manner, the touch control simulation device comprises four groups of metal touch control strips, namely a first touch control strip, a second touch control strip, a third touch control strip and a fourth touch control strip, and each touch control strip is formed by arranging a plurality of metal contact pieces with equal length at equal intervals;
the first touch strip and the second touch strip are respectively covered at the two transverse ends of the touch screen and are electrically coupled with the driving circuit of the touch screen so as to acquire a current mutual inductance signal of the driving circuit, and the metal contact pieces of the first touch strip and the second touch strip are correspondingly arranged in a staggered manner in the transverse direction;
the third touch strip and the fourth touch strip are respectively covered at the two longitudinal ends of the touch screen and are electrically coupled with the detection circuit of the touch screen so as to output a simulation current mutual inductance signal of the detection circuit, and the metal contact pieces of the third touch strip and the fourth touch strip are correspondingly arranged in a staggered manner in the longitudinal direction;
the touch control simulation device further comprises a first processing circuit, wherein the first processing circuit comprises an analog signal processing circuit and a first processor, the analog signal processing circuit is connected with the first processor, the analog signal processing circuit comprises a multi-path input analog switch, an analog signal level conversion circuit and a multi-path output analog switch which are sequentially connected, metal contact pieces of a first touch control strip and a second touch control strip are respectively connected with the input end of the multi-path input analog switch, and metal contact pieces of a third touch control strip and a fourth touch control strip are respectively connected with the output end of the multi-path output analog switch; the first processor is used for receiving a touch operation instruction, controlling the touch sheet corresponding to the multi-channel input analog switch to acquire a current mutual inductance signal of a line at a corresponding position on the driving line, and outputting an analog current mutual inductance signal at a corresponding position on the detection line through the touch sheet corresponding to the multi-channel output analog switch after being processed by the analog signal level conversion circuit so as to realize the simulation of executing touch operation on a touch position.
2. The method of controlling a smart device of claim 1, wherein: the metal contact pieces of the first touch control strip and the second touch control strip are correspondingly staggered in the transverse direction and are arranged at intervals, and the metal contact pieces of the third touch control strip and the fourth touch control strip are correspondingly staggered in the longitudinal direction and are arranged at intervals.
3. The method of controlling a smart device of claim 1, wherein: the metal contact pieces of the first touch control strip and the second touch control strip are correspondingly staggered in the transverse direction and are arranged in a partially overlapped mode, and the metal contact pieces of the third touch control strip and the fourth touch control strip are correspondingly staggered in the longitudinal direction and are arranged in a partially overlapped mode.
4. The method of controlling a smart device of claim 1, wherein: the first processing circuit is provided with a plurality of groups of analog signal processing circuits, and each group of analog signal processing circuits is connected with the first processor; and the metal contact pieces of the third touch strip and the fourth touch strip are respectively connected with the output ends of the multi-path output analog switches of the analog signal processing circuits.
5. The method of controlling a smart device of claim 1, wherein in the step (3), the control command signal is: and the software control program changes the image after the current event occurs into the image after the next event occurs.
6. The method according to claim 1, wherein in the step (4), the touch operation instruction is used to simulate a touch operation on the smart device, the simulated touch operation is a behavior of touching a functional area of a corresponding image on the touch screen, a coordinate position of a touch control point corresponding to the touch operation instruction is set corresponding to a position of the functional area on the image, and a coordinate system adopted by the coordinate position is a coordinate network formed by a driving line and a detecting line.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910498026.6A CN110231885B (en) | 2019-06-10 | 2019-06-10 | Touch simulation device, control method and method for controlling intelligent equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910498026.6A CN110231885B (en) | 2019-06-10 | 2019-06-10 | Touch simulation device, control method and method for controlling intelligent equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110231885A CN110231885A (en) | 2019-09-13 |
| CN110231885B true CN110231885B (en) | 2022-09-23 |
Family
ID=67859377
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910498026.6A Active CN110231885B (en) | 2019-06-10 | 2019-06-10 | Touch simulation device, control method and method for controlling intelligent equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110231885B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110737363B (en) * | 2019-10-31 | 2023-08-04 | 广东晶科电子股份有限公司 | Resistance type touch screen simulation control device and method |
| CN112100076B (en) * | 2020-09-24 | 2022-11-11 | 腾讯科技（深圳）有限公司 | Game testing method, related device, equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN203689492U (en) * | 2013-12-20 | 2014-07-02 | 锐达互动科技股份有限公司 | Capacitive touch and electromagnetic induction composite electronic whiteboard device |
| CN205334405U (en) * | 2015-12-03 | 2016-06-22 | 深圳磨石科技有限公司 | Touching device , touch display apparatus and electronic equipment |
| CN107797648A (en) * | 2017-11-09 | 2018-03-13 | 安徽大学 | Virtual touch system and image recognition localization method, computer-readable recording medium |
| CN109145809A (en) * | 2018-08-17 | 2019-01-04 | 腾讯科技（深圳）有限公司 | A kind of note spectrum processing method and device and computer readable storage medium |
| CN109189278A (en) * | 2017-10-27 | 2019-01-11 | 上海飞智电子科技有限公司 | Touch control method, touch device and touch-control system applied to capacitance touch screen |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI519155B (en) * | 2012-02-24 | 2016-01-21 | 宏達國際電子股份有限公司 | Burst image capture method and image capture system thereof |
-
2019
- 2019-06-10 CN CN201910498026.6A patent/CN110231885B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN203689492U (en) * | 2013-12-20 | 2014-07-02 | 锐达互动科技股份有限公司 | Capacitive touch and electromagnetic induction composite electronic whiteboard device |
| CN205334405U (en) * | 2015-12-03 | 2016-06-22 | 深圳磨石科技有限公司 | Touching device , touch display apparatus and electronic equipment |
| CN109189278A (en) * | 2017-10-27 | 2019-01-11 | 上海飞智电子科技有限公司 | Touch control method, touch device and touch-control system applied to capacitance touch screen |
| CN107797648A (en) * | 2017-11-09 | 2018-03-13 | 安徽大学 | Virtual touch system and image recognition localization method, computer-readable recording medium |
| CN109145809A (en) * | 2018-08-17 | 2019-01-04 | 腾讯科技（深圳）有限公司 | A kind of note spectrum processing method and device and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110231885A (en) | 2019-09-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104142766B (en) | Control point sensing method and device applied to capacitive panel | |
| JP5451181B2 (en) | Sensor device for detecting contact or proximity of an object | |
| US9547399B2 (en) | Injected touch noise analysis | |
| CN110231885B (en) | Touch simulation device, control method and method for controlling intelligent equipment | |
| CN104317466B (en) | Touch-screen control method and touch panel device | |
| JP2011530078A (en) | Electronic circuit and multi-point touch detection sensor for acquiring and analyzing multi-point touch detection sensor using bisection method and multi-point touch detection sensor acquisition and analysis method using bisection principle | |
| CN111078552A (en) | Method and device for detecting page display abnormity and storage medium | |
| CN102866322A (en) | Touch device detection method | |
| CN109444650B (en) | Open-short circuit testing method and device for single film of oversized capacitive screen | |
| US20180210606A1 (en) | Touch control panel, terminal device and method for detecting touch point | |
| US20140354310A1 (en) | Capacitive Sensor Testing | |
| CN108089754A (en) | A kind of touch-control simulator | |
| CN206281915U (en) | The test device and system of a kind of capacitance type touch key | |
| CN1945513A (en) | Cursor controlling device and method | |
| CN101903851A (en) | Multicontact tactile sensor with single-contact idle mode | |
| CN107131982B (en) | The touch-control pressure detection method of touch panel | |
| CN102591516A (en) | Control methods for sensing devices | |
| CN109964134A (en) | The test macro and method of touch control device anti-interference ability | |
| CN110162210A (en) | Capacitive sensing device, the preparation method of touch threshold and its bearing calibration | |
| CN107678662A (en) | A kind of control method based on button, device and intelligent terminal | |
| WO2022017135A1 (en) | Capacitance measurement circuit and method, and electronic device | |
| CN103064549B (en) | touch communication system | |
| CN203405807U (en) | Simulator for interface of touch screen | |
| CN110715792B (en) | Display panel, test method and tool thereof and display device | |
| CN211318564U (en) | System for multi-path detection of electrical parameter values by universal meter |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |