CN111273798B - Method and system for executing macro instruction of mouse, and method and device for executing macro instruction - Google Patents
Method and system for executing macro instruction of mouse, and method and device for executing macro instruction Download PDFInfo
- Publication number
- CN111273798B CN111273798B CN202010047897.9A CN202010047897A CN111273798B CN 111273798 B CN111273798 B CN 111273798B CN 202010047897 A CN202010047897 A CN 202010047897A CN 111273798 B CN111273798 B CN 111273798B
- Authority
- CN
- China
- Prior art keywords
- mouse
- target image
- executing
- image
- macro
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 238000013136 deep learning model Methods 0.000 claims abstract description 73
- 238000012549 training Methods 0.000 claims description 26
- 238000012216 screening Methods 0.000 claims description 7
- 238000013507 mapping Methods 0.000 claims description 4
- 230000001960 triggered effect Effects 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 12
- 238000001514 detection method Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000002372 labelling Methods 0.000 description 6
- 238000012544 monitoring process Methods 0.000 description 6
- 238000004458 analytical method Methods 0.000 description 5
- 238000003709 image segmentation Methods 0.000 description 5
- 230000003993 interaction Effects 0.000 description 5
- 238000012916 structural analysis Methods 0.000 description 5
- 206010016256 fatigue Diseases 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 238000004080 punching Methods 0.000 description 3
- 206010063385 Intellectualisation Diseases 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 208000019914 Mental Fatigue Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/038—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Image Analysis (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses a method for executing a macro mouse instruction, which is applied to a mouse operating system and comprises the steps of detecting whether a current screen image is a target image preset in a deep learning model or not; and executing a mouse macro instruction matched with the target image when the current screen image is the target image. The invention also discloses a system for executing the macro instruction of the mouse, and by the method and the system disclosed by the invention, different macro instructions can be automatically triggered according to different image pictures, so that the system is more intelligent, and the experience of a user and the service life of the mouse are increased.
Description
Technical Field
The present invention relates to the field of mouse macros, and in particular, to a method and system for executing a mouse macro, and a method and apparatus for executing a macro.
Background
The mouse macro is a series of system code combinations for simplifying computer operation in the mouse application process, and is commonly used for accelerating daily editing and format setting, combining a plurality of commands, enabling options in a dialog box to be easier to access, enabling a series of complex tasks to be automatically executed, and the like.
At present, the application mode of the mouse macro is mainly that the burning macro instruction enters a mouse chip and is triggered by a key, but the number of the macro instructions bound with the mouse is limited, the switching is inconvenient, the key position of the mouse is occupied, the burning macro instruction enters the mouse chip for a certain time, and the mouse is at risk of damage after the certain burning time is reached.
Disclosure of Invention
The invention aims to solve the technical problem of providing a method and a device for executing a macro instruction of a mouse, which can solve the problem that the service life of the mouse is influenced by burning the macro of the mouse in advance every time in the prior art, can automatically trigger different macro instructions according to different image pictures, is more intelligent, and increases the experience of a user and the service life of the mouse.
In order to solve the above technical problems, a first aspect of the present invention discloses a method for executing a macro instruction of a mouse, where the method is applied to an operating system of the mouse, and the method includes:
detecting whether the current screen image is a target image preset in a deep learning model;
and when the current screen image is a target image, executing a mouse macro instruction matched with the target image.
In some embodiments, executing a mouse macro that matches the target image includes:
receiving all mouse macros associated with the target image;
and screening all the mouse macros, and executing the mouse macros with the highest matching degree.
In some embodiments, executing a mouse macro that matches the target image includes:
performing image recognition on the target image to generate a recognition degree parameter;
judging the recognition degree parameters according to a preset recognition degree threshold value;
and executing a mouse macro instruction matched with the target image when the recognition degree parameter is higher than the recognition degree threshold value.
In some embodiments, the method further comprises training the deep learning model based on the target image when the current screen image is not a target image.
In some embodiments, training the deep learning model from the target image includes: setting up a plurality of mouse macros under different applications; and configuring images with mapping relation for the mouse macro instruction.
The second aspect of the present invention discloses a method for executing macro instructions, the method is applied to a camera monitoring system, and the method comprises:
detecting whether a current screen image is a target image preset in a deep learning model or not through a camera;
and executing a macro instruction matched with the target image when the current screen image is the target image.
In a third aspect, the invention discloses a system for executing a mouse macro, comprising:
the image recognition module is used for judging whether the current screen image is a target image preset in the deep learning model or not;
and the execution module is used for executing the mouse macro instruction matched with the target image when the current screen image is the target image.
In a fourth aspect, the invention discloses an apparatus for executing macro instructions, comprising:
the external device is used for detecting whether the current screen image is a target image preset in the deep learning model;
and the execution device is used for executing the macro instruction matched with the target image when the current screen image is the target image.
In a fifth aspect, the present invention discloses an apparatus for executing a macro mouse instruction, the apparatus comprising:
a memory storing executable program code;
a processor coupled to the memory;
the processor invokes the executable program code stored in the memory to execute the method of executing the mouse macro according to any one of the first aspect of the present invention.
A sixth aspect of the invention discloses a computer storage medium storing computer instructions that, when invoked, are adapted to perform the method of executing a mouse macro according to any of the first aspects of the invention.
Compared with the prior art, the invention has the beneficial effects that:
the implementation of the invention can realize the automatic detection of the current image by combining the novel AI deep learning model with the mouse macro instruction, acquire the mouse macro instruction matched with the image and execute the mouse macro instruction, thereby improving the working efficiency, being more humanized and intelligent, having more pertinence and saving the service life of the mouse.
Drawings
FIG. 1 is a flow chart of a method for executing a macro mouse command according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating another method for executing a macro mouse command according to an embodiment of the present invention;
FIG. 3 is a flow chart of a method for executing macro instructions according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a system for executing a macro mouse command according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of an apparatus for executing macro instructions according to an embodiment of the present invention;
fig. 6 is a schematic diagram of an apparatus for executing a macro instruction of a mouse according to an embodiment of the present invention.
Detailed Description
For a better understanding and implementation, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or modules is not necessarily limited to those steps or modules that are expressly listed or inherent to such process, method, article, or apparatus.
The embodiment of the invention discloses a method and a system for executing a mouse macro instruction, which can automatically acquire and execute the mouse macro instruction matched with an image by combining a novel AI deep learning model with the mouse macro instruction and detecting a current image, thereby improving the working efficiency, being more humanized and intelligent, having more pertinence and saving the service life of a mouse.
Example 1
Referring to fig. 1, fig. 1 is a flowchart illustrating a method for executing a macro mouse command according to an embodiment of the invention. The method described in fig. 1 may be applied to a mouse operating system, specifically, a mouse operating system with a function of setting a macro mouse, which is not limited in the embodiment of the present invention. As shown in fig. 1, the method for executing the mouse macro may include the following operations:
101. it is detected whether the current screen image is a target image preset in the deep learning model.
The method for capturing the current screen image comprises the steps that the current interface using the mouse can be captured through a display screen connected with the mouse, the mouse stays or passes through, and then the captured image is identified based on a deep learning model to judge whether an approximate image exists in the deep learning model.
It should be noted that, the method for detecting whether the current screen image is the target image according to the present invention may also be performed by conventional image recognition and color finding recognition, which is not limited herein.
Further, for a deep learning model, the training content of the model is mainly training learning image frames. Common categories for learning include: human-machine interaction, object recognition, image segmentation, face recognition, motion tracking, robots, motion analysis, machine vision, structural analysis and safe driving of automobiles. Under different application scenes, different mouse macros are set according to different images. For example, in the current application, according to the requirement of the user, the macro instruction set by the mouse is used for quickly identifying and labeling the image units in the table, so that the object to be learned in the deep learning model is the image to be labeled (such as a product document of a certain company, a specified product to be labeled), and the training is performed after the image material to be trained (the image to be labeled) is labeled. Then in the current interface, e.g., in an on-screen document, a quick identification of the image therein is made to determine if the image is image material stored in the deep learning model.
102. And when the current screen image is the target image, executing a mouse macro instruction matched with the target image.
Because a large number of training images are stored in the deep learning model, different matched mouse macros can exist for some similar images, all the mouse macros associated with the target image are received, and then all the mouse macros are screened by combining the current interface and application scene, wherein the screening mode can be realized through intelligent priority matching in the prior art, so that the mouse macros with the highest matching degree are executed.
For example, in the case where the application scenario where the target image is located is a batch of excel tables, since a large number of images contained in the excel tables are not numbers, the labeling operation cannot be performed by the quick search function of the excel, and therefore, an operator is required to manually distinguish and label the target image (commonly found in the statistical products of the system in the factory). However, since the self-contained quick search function label cannot be used for the pure numbers and the characters, a mouse macro instruction of 'one mouse clicking a table unit and then executing a label shortcut key combination' set in the deep learning model can be utilized at the moment, and after the target images to be distinguished are identified through the deep learning model, the mouse is used for quickly clicking and executing the key macro combination to label. Thus, the working efficiency of the staff is greatly improved, and the fatigue feeling of the mechanical operation of the staff can be greatly reduced.
As another illustration of this embodiment, the application scenario where the target image is located is a game interface, and for some games with complex operations, for example, in the game of "magic beast world", because of the diversity of the games, the fictional 13 kinds of 11 major occupations and various expression actions in the game are very complicated, so that it is very difficult to perform the complex operations, for example, to activate more than 2 kinds of laws simultaneously, send information by using the chat system in the game while applying the laws, and so on. The macro command in wow has been developed from game operations to many types of macro commands, and the macro commands can be played by a key by a small input command, and can be used in cooperation with various skills and states. Then, the current screen image is detected in a mode based on a deep learning model, and when the target image is detected, a mouse macro instruction or macro combination bound with the target image can be executed, so that the game operation of a user can be simplified, and the fatigue of the hand operation and the mental fatigue caused by the high-difficulty operation can be properly reduced. In a preferred embodiment, executing the mouse macro matching the target image further comprises:
and carrying out image recognition on the target image to generate a recognition degree parameter, wherein the recognition degree parameter is a comparison result of the target image and an image which is output in the deep learning model and is similar to the target image, and judging the recognition degree parameter according to a preset recognition degree threshold value, wherein the preset recognition degree threshold value is set according to the precision requirement of a user. For example, for some mouse macros requiring high accuracy recognition matching, the recognition threshold may be set to 95%, and for some commonly required mouse macros with recognition accuracy, 80% similarity may be sufficiently accurate.
Further, when the recognition degree parameter is higher than the recognition degree threshold value, executing a mouse macro instruction matched with the target image. The implementation mode is as follows: and sending the mouse macro instruction to an execution end corresponding to the mouse, and automatically executing the mouse macro instruction matched with the target image.
According to the method for executing the mouse macro instruction disclosed by the embodiment, the novel AI deep learning model is combined with the mouse macro instruction, so that the automatic detection of the current image is realized, the mouse macro instruction matched with the image is obtained and executed, the working efficiency is improved, the method is more humanized and intelligent, the pertinence is also realized, and the service life of the mouse is saved.
Example two
Referring to fig. 2, fig. 2 is a flowchart illustrating another method for executing a macro mouse command according to an embodiment of the invention. The method described in fig. 2 may be applied to a mouse operating system and a mouse operating system with a function of setting a macro mouse, which is not limited in the embodiment of the present invention. As shown in fig. 2, the method for executing the mouse macro may include the following operations:
201. it is detected whether the current screen image is a target image preset in the deep learning model.
The method for capturing the current screen image comprises the steps that the current interface using the mouse can be captured through a display screen connected with the mouse, the mouse stays or passes through, and then the captured image is identified based on a deep learning model to judge whether an approximate image exists in the deep learning model.
It should be noted that, the method for detecting whether the current screen image is the target image according to the present invention may also be performed by conventional image recognition and color finding recognition, which is not limited herein.
Further, for a deep learning model, the training content of the model is mainly training learning image frames. Common categories for learning include: human-machine interaction, object recognition, image segmentation, face recognition, motion tracking, robots, motion analysis, machine vision, structural analysis and safe driving of automobiles. Under different application scenes, different mouse macros are set according to different images. For example, in the current application, according to the requirement of the user, the macro instruction set by the mouse is used for quickly identifying and labeling the image units in the table, so that the object to be learned in the deep learning model is the image to be labeled (such as a product document of a certain company, a specified product to be labeled), and the training is performed after the image material to be trained (the image to be labeled) is labeled. Then in the current interface, e.g., in an on-screen document, a quick identification of the image therein is made to determine if the image is image material stored in the deep learning model.
202. And when the current screen image is the target image, executing a mouse macro instruction matched with the target image.
Because a large number of training images are stored in the deep learning model, different matched mouse macros can exist for some similar images, all the mouse macros associated with the target image are received, and then all the mouse macros are screened by combining the current interface and application scene, wherein the screening mode can be realized through intelligent priority matching in the prior art, so that the mouse macros with the highest matching degree are executed.
For example, in the case where the application scenario where the target image is located is a batch of excel tables, since a large number of images contained in the excel tables are not numbers, the labeling operation cannot be performed by the quick search function of the excel, and therefore, an operator is required to manually distinguish and label the target image (commonly found in the statistical products of the system in the factory). However, since the self-contained quick search function label cannot be used for the pure numbers and the characters, a mouse macro instruction of 'one mouse clicking a table unit and then executing a label shortcut key combination' set in the deep learning model can be utilized at the moment, and after the target images to be distinguished are identified through the deep learning model, the mouse is used for quickly clicking and executing the key macro combination to label. Thus, the working efficiency of the staff is greatly improved, and the fatigue feeling of the mechanical operation of the staff can be greatly reduced.
In a preferred embodiment, executing the mouse macro matching the target image further comprises:
and carrying out image recognition on the target image to generate a recognition degree parameter, wherein the recognition degree parameter is a comparison result of the target image and an image which is output in the deep learning model and is similar to the target image, and judging the recognition degree parameter according to a preset recognition degree threshold value, wherein the preset recognition degree threshold value is set according to the precision requirement of a user. For example, for some mouse macros requiring high accuracy recognition matching, the recognition threshold may be set to 95%, and for some commonly required mouse macros with recognition accuracy, 80% similarity may be sufficiently accurate.
Further, when the recognition degree parameter is higher than the recognition degree threshold value, executing a mouse macro instruction matched with the target image. The implementation mode is as follows: and sending the mouse macro instruction to an execution end corresponding to the mouse, and automatically executing the mouse macro instruction matched with the target image.
203. When the current screen image is not the target image, training the deep learning model according to the target image.
Training the deep learning model from the target image, comprising: the method for constructing the mouse macro instructions under different applications can be constructed according to a deep learning knowledge base, wherein the deep learning knowledge base is in the prior art, and the construction mode can be formed by searching the types of the mouse macro instructions in the database or automatically recording the mouse and key operations according to the user requirements. After the macro is recorded, a memory chip on the mouse board can be burnt, and if the macro is too huge, the macro can be stored in a local computer in the form of an encrypted document. And executing by immediate calling of the deep learning model.
And then configuring an image with a mapping relation for the mouse macro instruction, wherein the configuration mode of the image is the main learning content of the deep learning model.
According to the method for executing the mouse macro instruction disclosed by the embodiment, the novel AI deep learning model is combined with the mouse macro instruction, so that the automatic detection of the current image is realized, the mouse macro instruction matched with the image is obtained and executed, the working efficiency is improved, the method is more humanized and intelligent, the pertinence is also realized, and the service life of the mouse is saved.
Example III
Referring to fig. 3, fig. 3 is a flowchart illustrating a method for executing macro instructions according to an embodiment of the present invention. The system described in fig. 3 may be implemented as a camera operating system, a monitoring system, a card punching system, or other systems with image recognition and macro instruction combination, which are all within the scope of the present invention, and the embodiment of the present invention is not limited thereto. As shown in fig. 4, the method for executing the mouse macro may include the following operations:
301. and detecting whether the current screen image is a target image preset in the deep learning model through a camera.
For some applications of macros that assist in night theft assistance monitoring or personal privacy protection, etc., external devices may be connected to make the application more versatile. In this embodiment, the macro instruction using the external device as the camera is mainly described, and in other modes, the present invention is not limited.
The detection mode can be used for capturing a picture or a portrait moving in the camera, identifying the captured image based on the image in the deep learning model, and judging whether an approximate image exists in the deep learning model.
Further, for a deep learning model, the training content of the model is mainly training learning image frames. Common categories for learning include: human-machine interaction, object recognition, image segmentation, face recognition, motion tracking, robots, motion analysis, machine vision, structural analysis and safe driving of automobiles. Under different application scenes, different macro instructions are set according to different images. For example, if the requirement for a certain application scenario is whether unattended or assisted supervision is monitored by a camera, the human body or the human figure can be correspondingly trained in the deep learning model, and monitoring assistance is performed to a certain extent. Further, after the computer camera is connected with the computer camera, the human face of a computer owner can be trained through the deep learning model, automatic locking, unlocking and the like of the computer can be completed, and the computer can be used for privacy protection after leaving a station in work. Further, the method can also be applied to an internet bar system, and when an internet client leaves, the method can be automatically locked so as to achieve the effect of protecting privacy. Further, the method can be applied to functions of office work automatic identification, check-in and card punching, network game identification, simple on-hook operation and the like.
302. When the current screen image is the target image, executing the macro instruction matched with the target image.
Because a large number of training images are stored in the deep learning model, different macro instructions matched with some similar images can exist, all macro instructions associated with the target image are received, and then all macro instructions are screened by combining the current interface and application scene, wherein the screening mode can be realized through intelligent priority matching in the prior art, so that the macro instruction with the highest matching degree is executed.
By way of illustration of this embodiment, for the application scenario in which the target image is located, it is applied to personal privacy protection of a PC computer, and then the target image learned by the deep learning model is a face image of an owner of the PC computer, and the macros matched with the application and the image are for automatically unlocking and unlocking the computer, so as to automatically lock the computer before leaving the computer according to the identified face image, and automatically unlock the computer before returning to the computer.
In a preferred embodiment, executing the macro instruction matching the target image further comprises:
and carrying out image recognition on the target image to generate a recognition degree parameter, wherein the recognition degree parameter is a comparison result of the target image and an image which is output in the deep learning model and is similar to the target image, and judging the recognition degree parameter according to a preset recognition degree threshold value, wherein the preset recognition degree threshold value is set according to the precision requirement of a user. For example, for some mouse macros requiring high accuracy recognition matching, the recognition threshold may be set to 95%, and for some commonly required mouse macros with recognition accuracy, 80% similarity may be sufficiently accurate.
Further, when the recognition parameter is higher than the recognition threshold, executing the macro instruction matched with the target image. The implementation mode is as follows: and sending the macro instruction to an execution end corresponding to the mouse, and automatically executing the macro instruction matched with the target image.
According to the method disclosed by the embodiment, the further expansion can be realized on the basis of the deep learning model, so that the application range is wider, the working efficiency is improved, the humanization and the intellectualization are realized, and the pertinence is realized.
Example IV
Referring to fig. 4, fig. 4 is a schematic diagram of a system for executing a macro instruction of a mouse according to an embodiment of the invention. The method described in fig. 4 may be applied to a mouse operating system, specifically, a mouse operating system with a function of setting a macro mouse, which is not limited in the embodiment of the present invention. As shown in fig. 4, the system for executing the macro mouse instruction may include:
an image recognition module 401, configured to detect whether a current screen image is a target image preset in a deep learning model;
and the execution module 402 is used for executing the macro mouse instruction matched with the target image when the current screen image is the target image.
As an implementation manner, the image recognition module 401 may capture an image through a display screen connected to the mouse on an interface where the mouse is currently used, where the mouse stays or passes, and then compare the captured image with an image in the deep learning model to determine whether an approximate image exists in the deep learning model.
Further, for a deep learning model, the training content of the model is mainly training learning image frames. Common categories for learning include: human-machine interaction, object recognition, image segmentation, face recognition, motion tracking, robots, motion analysis, machine vision, structural analysis and safe driving of automobiles. Under different application scenes, different mouse macros are set according to different images. For example, in the current application, according to the requirement of the user, the macro instruction set by the mouse is used for quickly identifying and labeling the image units in the table, so that the object to be learned in the deep learning model is the image to be labeled (such as a product document of a certain company, a specified product to be labeled), and the training is performed after the image material to be trained (the image to be labeled) is labeled. Then in the current interface, e.g., in an on-screen document, a quick identification of the image therein is made to determine if the image is image material stored in the deep learning model.
Because a large number of training images are stored in the deep learning model, and different matched mouse macros may exist for some similar images, the execution module 402 receives all the mouse macros associated with the target image, and screens all the mouse macros in combination with the current interface and application scene, wherein the screening mode can be realized through intelligent priority matching in the prior art, so that the mouse macros with the highest matching degree are executed.
For example, in the case where the application scenario where the target image is located is a batch of excel tables, since a large number of images contained in the excel tables are not numbers, the labeling operation cannot be performed by the quick search function of the excel, and therefore, an operator is required to manually distinguish and label the target image (commonly found in the statistical products of the system in the factory). However, since the self-contained quick search function label cannot be used for the pure numbers and the characters, a mouse macro instruction of 'one mouse clicking a table unit and then executing a label shortcut key combination' set in the deep learning model can be utilized at the moment, and after the target images to be distinguished are identified through the deep learning model, the mouse is used for quickly clicking and executing the key macro combination to label. Thus, the working efficiency of the staff is greatly improved, and the fatigue feeling of the mechanical operation of the staff can be greatly reduced.
In a preferred embodiment, execution module 40 further comprises:
and carrying out image recognition on the target image to generate a recognition degree parameter, wherein the recognition degree parameter is a comparison result of the target image and an image which is output in the deep learning model and is similar to the target image, and judging the recognition degree parameter according to a preset recognition degree threshold value, wherein the preset recognition degree threshold value is set according to the precision requirement of a user. For example, for some mouse macros requiring high accuracy recognition matching, the recognition threshold may be set to 95%, and for some commonly required mouse macros with recognition accuracy, 80% similarity may be sufficiently accurate.
Further, when the recognition degree parameter is higher than the recognition degree threshold value, executing a mouse macro instruction matched with the target image. The implementation mode is as follows: and sending the mouse macro instruction to an execution end corresponding to the mouse, and automatically executing the mouse macro instruction matched with the target image.
According to the system for executing the mouse macro instruction disclosed by the embodiment, the novel AI deep learning model is combined with the mouse macro instruction, so that the automatic detection of the current image is realized, the mouse macro instruction matched with the image is obtained and executed, the working efficiency is improved, the system is more humanized and intelligent, the pertinence is also realized, and the service life of the mouse is saved.
Example five
Referring to fig. 5, fig. 5 is a schematic structural diagram of an apparatus for executing macro instructions according to an embodiment of the present invention. The device described in fig. 5 may be applied to a mouse operating system, specifically, a mouse operating system with a function of setting a macro mouse, which is not limited in the embodiment of the present invention. As shown in fig. 5, the apparatus for executing macro instruction may include:
an external device 501 for detecting whether the current screen image is a target image preset in the deep learning model;
and an execution device 502 for executing a macro instruction matching the target image when the current screen image is the target image.
For some applications of macros that assist in night theft assistance monitoring or personal privacy protection, the external device 501 may be connected to allow application in more scenarios, as an embodiment. In this embodiment, the macro instruction using the external device as the camera is mainly described, and in other modes, the present invention is not limited.
The detection mode can be used for capturing a picture or a portrait moving in the camera, comparing the captured image with an image in the deep learning model, and judging whether an approximate image exists in the deep learning model.
It should be noted that, the external device 501 of the present invention is not limited to a camera, and other devices capable of capturing images and recognizing images are all within the scope of the present invention.
Further, for a deep learning model, the training content of the model is mainly training learning image frames. Common categories for learning include: human-machine interaction, object recognition, image segmentation, face recognition, motion tracking, robots, motion analysis, machine vision, structural analysis and safe driving of automobiles. Under different application scenes, different macro instructions are set according to different images. For example, if the requirement for a certain application scenario is whether unattended or assisted supervision is monitored by a camera, the human body or the human figure can be correspondingly trained in the deep learning model, and monitoring assistance is performed to a certain extent. Further, after the computer camera is connected with the computer camera, the human face of a computer owner can be trained through the deep learning model, automatic locking, unlocking and the like of the computer can be completed, and the computer can be used for privacy protection after leaving a station in work. Further, the method can also be applied to an internet bar system, and when an internet client leaves, the method can be automatically locked so as to achieve the effect of protecting privacy. Further, the method can be applied to functions of office work automatic identification, check-in and card punching, network game identification, simple on-hook operation and the like.
Because a large number of training images are stored in the deep learning model, different macro instructions matched with some similar images can exist, all macro instructions associated with the target image are received, and then all macro instructions are screened by combining the current interface and application scene, wherein the screening mode can be realized through intelligent priority matching in the prior art, so that the macro instruction with the highest matching degree is executed.
By way of illustration of this embodiment, for the application scenario in which the target image is located, it is applied to personal privacy protection of a PC computer, and then the target image learned by the deep learning model is a face image of an owner of the PC computer, and the macros matched with the application and the image are for automatically unlocking and unlocking the computer, so as to automatically lock the computer before leaving the computer according to the identified face image, and automatically unlock the computer before returning to the computer.
In a preferred embodiment, the execution device 502 further comprises:
and carrying out image recognition on the target image to generate a recognition degree parameter, wherein the recognition degree parameter is a comparison result of the target image and an image which is output in the deep learning model and is similar to the target image, and judging the recognition degree parameter according to a preset recognition degree threshold value, wherein the preset recognition degree threshold value is set according to the precision requirement of a user. For example, for some mouse macros requiring high accuracy recognition matching, the recognition threshold may be set to 95%, and for some commonly required mouse macros with recognition accuracy, 80% similarity may be sufficiently accurate.
Further, when the recognition parameter is higher than the recognition threshold, executing the macro instruction matched with the target image. The implementation mode is as follows: and sending the macro instruction to an execution end corresponding to the mouse, and automatically executing the macro instruction matched with the target image.
According to the method disclosed by the embodiment, the further expansion can be realized on the basis of the deep learning model, so that the application range is wider, the working efficiency is improved, the humanization and the intellectualization are realized, and the pertinence is realized.
Example six
Referring to fig. 6, fig. 6 is a schematic structural diagram of an apparatus for executing a macro mouse command according to an embodiment of the invention. The apparatus depicted in fig. 6 may be, but is not limited to, an embodiment of the present invention. As shown in fig. 6, the apparatus may include:
a memory 601 in which executable program codes are stored;
a processor 602 coupled to the memory 601;
the processor 602 invokes executable program code stored in the memory 601 for performing the method of executing a mouse macro as described in embodiment one or embodiment two.
Example seven
The embodiment of the invention discloses a computer readable storage medium storing a computer program for electronic data exchange, wherein the computer program causes a computer to execute the method for executing the macro mouse instruction described in the first embodiment or the second embodiment.
The embodiments described above are merely illustrative, wherein the modules illustrated as separate components may or may not be physically separate, and the components shown as modules may or may not be physical, i.e., may be located in one place, or may be distributed over a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above detailed description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course by means of hardware. Based on such understanding, the foregoing technical solutions may be embodied essentially or in part in the form of a software product that may be stored in a computer-readable storage medium including Read-Only Memory (ROM), random access Memory (RandomAccess Memory, RAM), programmable Read-Only Memory (Programmable Read-Only Memory, PROM), erasable programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), one-time programmable Read-Only Memory (One-time Programmable Read-OnlyMemory, OTPROM), electrically erasable rewritable Read-Only Memory (EEPROM), compact disc Read-Only Memory (Compact Disc Read-Only Memory, CD-ROM) or other optical disc Memory, magnetic disk Memory, tape Memory, or any other medium that can be used to carry or store data that is readable by a computer.
Finally, it should be noted that: the disclosed method and system for executing a macro instruction of a mouse are only preferred embodiments of the present invention, and are only used for illustrating the technical scheme of the present invention, but not limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art will understand that; the technical scheme recorded in the various embodiments can be modified or part of technical features in the technical scheme can be replaced equivalently; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions.
Claims (6)
1. A method for executing a macro instruction of a mouse, the method being applied to a mouse operating system, the method comprising:
detecting whether the current screen image is a target image preset in a deep learning model;
when the current screen image is a target image, executing a mouse macro instruction matched with the target image;
when the current screen image is not a target image, training the deep learning model according to the target image; training the deep learning model according to the target image, including:
setting up a plurality of mouse macros under different applications;
and configuring images with mapping relation for the mouse macro instruction.
2. The method of executing a mouse macro according to claim 1, wherein said executing a mouse macro that matches the target image comprises:
receiving all mouse macros associated with the target image;
and screening all the mouse macros, and executing the mouse macros with the highest matching degree.
3. The method of executing a mouse macro of claim 1, wherein executing a mouse macro that matches the target image comprises:
performing image recognition on the target image to generate a recognition degree parameter;
judging the recognition degree parameters according to a preset recognition degree threshold value;
and executing a mouse macro instruction matched with the target image when the recognition degree parameter is higher than the recognition degree threshold value.
4. A system for executing a mouse macro, the system comprising:
the image recognition module is used for detecting whether the current screen image is a target image preset in the deep learning model;
the execution module is used for executing a mouse macro instruction matched with the target image when the current screen image is the target image; training the deep learning model according to the target image when the current screen image is not the target image; training the deep learning model according to the target image, including:
setting up a plurality of mouse macros under different applications;
and configuring images with mapping relation for the mouse macro instruction.
5. An apparatus for executing a mouse macro, the apparatus comprising:
a memory storing executable program code;
a processor coupled to the memory;
the processor invokes the executable program code stored in the memory to perform the method of any one of claims 1-3.
6. Computer storage medium, characterized in that it stores computer instructions for executing the method according to any of claims 1-3 when called.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010047897.9A CN111273798B (en) | 2020-01-16 | 2020-01-16 | Method and system for executing macro instruction of mouse, and method and device for executing macro instruction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010047897.9A CN111273798B (en) | 2020-01-16 | 2020-01-16 | Method and system for executing macro instruction of mouse, and method and device for executing macro instruction |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111273798A CN111273798A (en) | 2020-06-12 |
| CN111273798B true CN111273798B (en) | 2024-02-06 |
Family
ID=71001658
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010047897.9A Active CN111273798B (en) | 2020-01-16 | 2020-01-16 | Method and system for executing macro instruction of mouse, and method and device for executing macro instruction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111273798B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002041199A (en) * | 2000-05-15 | 2002-02-08 | Pasukaru:Kk | Operation processing method for computer device using shortcut symbol, and shortcut processing system |
| CN102142004A (en) * | 2010-08-20 | 2011-08-03 | 华为技术有限公司 | Service processing method and service engine |
| CN105631282A (en) * | 2015-07-31 | 2016-06-01 | 宇龙计算机通信科技(深圳)有限公司 | Operation method and device of electronic equipment |
| CN108156668A (en) * | 2017-11-22 | 2018-06-12 | 努比亚技术有限公司 | A kind of communication interaction method, terminal and computer readable storage medium |
| WO2019120029A1 (en) * | 2017-12-20 | 2019-06-27 | Oppo广东移动通信有限公司 | Intelligent screen brightness adjustment method and apparatus, and storage medium and mobile terminal |
| CN110210194A (en) * | 2019-04-18 | 2019-09-06 | 深圳壹账通智能科技有限公司 | Electronic contract display methods, device, electronic equipment and storage medium |
| CN110297709A (en) * | 2018-03-22 | 2019-10-01 | 东莞宝德电子有限公司 | Tool shares the computer input and its computer input method of operand function |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080282160A1 (en) * | 2007-04-06 | 2008-11-13 | James Ian Tonnison | Designated screen capturing and automatic image exporting |
| US10281893B2 (en) * | 2009-06-25 | 2019-05-07 | Universal Electronics Inc. | System and method for configuration of macro commands in a controlling device |
| US10235192B2 (en) * | 2017-06-23 | 2019-03-19 | Accenture Global Solutions Limited | Self-learning robotic process automation |
-
2020
- 2020-01-16 CN CN202010047897.9A patent/CN111273798B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002041199A (en) * | 2000-05-15 | 2002-02-08 | Pasukaru:Kk | Operation processing method for computer device using shortcut symbol, and shortcut processing system |
| CN102142004A (en) * | 2010-08-20 | 2011-08-03 | 华为技术有限公司 | Service processing method and service engine |
| CN105631282A (en) * | 2015-07-31 | 2016-06-01 | 宇龙计算机通信科技(深圳)有限公司 | Operation method and device of electronic equipment |
| CN108156668A (en) * | 2017-11-22 | 2018-06-12 | 努比亚技术有限公司 | A kind of communication interaction method, terminal and computer readable storage medium |
| WO2019120029A1 (en) * | 2017-12-20 | 2019-06-27 | Oppo广东移动通信有限公司 | Intelligent screen brightness adjustment method and apparatus, and storage medium and mobile terminal |
| CN110297709A (en) * | 2018-03-22 | 2019-10-01 | 东莞宝德电子有限公司 | Tool shares the computer input and its computer input method of operand function |
| CN110210194A (en) * | 2019-04-18 | 2019-09-06 | 深圳壹账通智能科技有限公司 | Electronic contract display methods, device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111273798A (en) | 2020-06-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6893606B2 (en) | Image tagging methods, devices and electronics | |
| CN109727275B (en) | Object detection method, device, system and computer readable storage medium | |
| CN114742509B (en) | Cargo access system, device and cargo access method | |
| WO2015098144A1 (en) | Information processing device, information processing program, recording medium, and information processing method | |
| CN108647633B (en) | Identification tracking method, identification tracking device and robot | |
| CN109086725B (en) | Hand tracking method and machine-readable storage medium | |
| US20210127071A1 (en) | Method, system and computer program product for object-initiated redaction of surveillance video | |
| CN111191207A (en) | Electronic file control method and device, computer equipment and storage medium | |
| US20070257794A1 (en) | Action recognition and interpretation using a precision positioning system | |
| Bekel et al. | Adaptive computer vision: Online learning for object recognition | |
| CN111273798B (en) | Method and system for executing macro instruction of mouse, and method and device for executing macro instruction | |
| WO2019242156A1 (en) | Method and device for controlling application in terminal, and computer readable storage medium | |
| CN115858049B (en) | RPA flow componentization arrangement method, device, equipment and medium | |
| CN111539390A (en) | Small target image identification method, equipment and system based on Yolov3 | |
| CN109635688B (en) | Method and system for managing books on bookshelf based on image recognition | |
| CN106407386B (en) | Method and device for improving topic searching efficiency | |
| CN114764897A (en) | Behavior recognition method, behavior recognition device, terminal equipment and storage medium | |
| CN111368726B (en) | Construction site operation face personnel number statistics method, system, storage medium and device | |
| CN115086527B (en) | Household video tracking and monitoring method, device, equipment and storage medium | |
| CN112183284A (en) | Safety information verification and designated driving order receiving control method and device | |
| Linan et al. | Development of a Network Camera with a Built-in Mask and QR Code Recognition Function | |
| CN111461103B (en) | Method and device for eliminating interference of click-to-read scene, electronic equipment and storage medium | |
| CN116738411B (en) | Multi-mode registration method and identity recognition method based on biological feature recognition | |
| CN117112034A (en) | Drive control method, drive control device, electronic equipment and computer storage medium | |
| JPH0935058A (en) | Image recognizing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |