CN110286825A - A kind of mechanical automatic mouse action device based on machine vision - Google Patents
A kind of mechanical automatic mouse action device based on machine vision Download PDFInfo
- Publication number
- CN110286825A CN110286825A CN201910560105.5A CN201910560105A CN110286825A CN 110286825 A CN110286825 A CN 110286825A CN 201910560105 A CN201910560105 A CN 201910560105A CN 110286825 A CN110286825 A CN 110286825A
- Authority
- CN
- China
- Prior art keywords
- mouse
- machine vision
- region
- image
- device based
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000009471 action Effects 0.000 title claims abstract description 26
- 238000004458 analytical method Methods 0.000 claims abstract description 33
- 238000000034 method Methods 0.000 claims abstract description 12
- 230000033001 locomotion Effects 0.000 claims description 10
- 238000009826 distribution Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 6
- 230000011218 segmentation Effects 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 5
- 238000009434 installation Methods 0.000 claims description 4
- 238000004737 colorimetric analysis Methods 0.000 claims description 3
- 230000008859 change Effects 0.000 claims description 2
- 230000005611 electricity Effects 0.000 claims description 2
- 238000001514 detection method Methods 0.000 abstract description 3
- 230000000007 visual effect Effects 0.000 abstract description 2
- 238000005516 engineering process Methods 0.000 description 10
- 230000010354 integration Effects 0.000 description 5
- 238000013139 quantization Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 230000003252 repetitive effect Effects 0.000 description 2
- 230000003321 amplification Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 239000000571 coke Substances 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses a kind of mechanical automatic mouse action device based on machine vision, including video acquisition system, machine vision analysis system and mechanical control system.The present invention is based on computer machine Visual analysis techniques, it can be according to the requirement of machine vision analysis system, result according to video analysis, realize mouse state tracking, detection automatically, intelligent, it can be in the mode of program setting, time, range, according to the route of setting, mode, the automatic round-trip various operations for carrying out mouse, it can be used in the occasion in various appliance computers, manual intervention can be largely saved, further save human cost.
Description
Technical field
The invention belongs to technical field of automatic control, and in particular to a kind of mechanical automatic mouse based on machine vision
Mark operating device.
Background technique
Currently, a main problem in computer use is exactly: existing computer system occurs in screen mostly
When variation, need artificially to carry out intervention control, such as typewrited, key, by carriage return, by space bar, mouse-click, mouse
Mark the operation such as double-click, mouse rollovers.
Present machine vision, artificial intelligence and electromechanical integration technology provides skill for the full automatic working of computer
A possibility that in art.Machine vision technique, can carry out OCR character recognition at this stage, carry out standard square, circle, ellipse
Identification.But, at this stage, computer based Machine Vision Recognition, it is main using computer carry out be paper media, video,
The analysis of the materials such as image；And computer vision recognition technology is utilized, directly carry out the content recognition of computer-oriented display screen
And its application, it is also necessary to the mouse of several ancillary equipments --- full automatic working, the keyboard of full automatic working, full automatic working
Handwriting pad, the loudspeaker of full automatic working etc..
The existing existing main problem of the technology is:
The operation of most computer equipments at this stage, still in such a way that operator is manually-operated based on；
It much typewrites, draw on the computer screen, filling in a form etc. operation, still cannot achieve full-automatic identification and operation, need
Computer and its software are wanted, electromechanical integration the relevant technologies are relied on, realizes the unattended full automatic working of mouse, but at present should
Technique direction is not also to be paid attention to very much；
Computer and its software are needed, relies on electromechanical integration the relevant technologies, realizes the unattended full-automatic behaviour of keyboard
Make；
Computer and its software are needed, relies on electromechanical integration the relevant technologies, realizes the full automatic working of handwriting pad；
The ancillary equipment more than relying on is needed, within the preset various situation ranges of program, realizes and relies on computer, machine
The computer full automatic working of device vision, electromechanical integration technology.
Summary of the invention
Goal of the invention of the invention is: real in order to further decrease the manual intervention in computer operation mode at this stage
Higher automation, the intelligence of existing computer operation, the invention proposes a kind of based on computer control, machine vision auxiliary
Analysis, can full automatic working, mechanization mouse action device.
The technical scheme is that a kind of mechanical automatic mouse action device based on machine vision, including view
Frequency acquisition system, machine vision analysis system and mechanical control system；
The video acquisition system is used to for computer screen being divided into several net regions, and uses multiple cameras
Matrix carries out Image Acquisition to each net region respectively, and the image data of acquisition is transmitted to machine vision analysis system；
The machine vision analysis system is obtained for successively carrying out target identification to the image data of each camera acquisition
Mouse domain of the existence is taken, carries out mouse tracking in mouse domain of the existence, letter is prompted to dialog box, the dialog box of mouse position
Breath and input information are identified, are generated the mobile control signal of mouse and mouse action control signal and are transmitted to mechanical control
System；
The mechanical control system is used to be moved according to the mobile control signal control mouse of mouse, is grasped according to mouse
Make control signal control mouse and carries out corresponding operating.
Further, the machine vision analysis system is according to camera serial number, successively for the acquisition of each camera
Image data carries out video frame reading, determines mouse domain of the existence；When mouse is motion state, then image processing algorithm is used
Mouse position extraction is carried out, when mouse is stationary state, then image is amplified, recycles Feature Points Matching and color special
Sign carries out template matching；After obtaining mouse domain of the existence, mouse tracking is carried out in mouse domain of the existence.
Further, the machine vision analysis system is located at the coincidence area in multiple camera collection image regions to mouse
When domain, delay setting time is carried out, then respectively to there are each camera collection image regions of overlapping region to carry out mouse knowledge
Not, after obtaining image-region existing for mouse, mouse tracking is carried out in the mouse domain of the existence.
Further, the machine vision analysis system is located at the coincidence area in multiple camera collection image regions to mouse
When domain, image mosaic is carried out by the way of finding characteristic point in each camera collection image region there are overlapping region,
Mouse tracking is carried out in the image-region again.
Further, the machine vision analysis system identifies the dialog box of mouse position specifically:
To mouse domain of the existence, sample areas acquisition is carried out according to the acquisition parameter of setting；
The sample areas of acquisition is numbered, pair of each logical number Yu mouse domain of the existence physical location is established
Answer mapping relations；
Feature extraction is carried out to the sample areas of acquisition, calculates the color difference directional gradient vector rotational difference of sample areas
Value；
Label of the color difference directional gradient vector rotational difference in upper lower threshold value be by lower threshold value in setting；
The cluster based on mean value for carrying out default cluster number, merges the region after cluster, obtains based on cluster
Multiple regions segmentation；
According to the state of obtained clustering distribution, the edge segmentation of image is carried out, the dialog box of mouse position is obtained
Boundary.
Further, the color difference directional gradient vector rotational difference for calculating sample areas specifically:
Eight directional calculating is carried out according to gradient information direction, is calculated according still further to the color difference in sample region, other side
Vectorized process is carried out to gradient, calculates separately the gradient in each direction in four orientation up and down, then carry out Vector modulation；Then
Sample region is carried out to 90 degree of rotation, then carries out the Colorimetry in a sample region, line direction gradient of going forward side by side
Vectorized process；The color difference directional gradient vector value of finally comparison first time and secondary sample region, in conjunction with ladder
Information is spent, the color difference directional gradient vector rotational difference of sample areas is calculated according to default weight expression formula.
Further, the machine vision analysis system is according to logical coordinates of the mouse in computer screen image and mouse
Biaxial stress structure relationship between target logical coordinates and physical coordinates, working region model of the combined mouse in computer screen image
It encloses, generates the mobile control signal of mouse.
Further, the mechanical control system includes the fixation slide rail for being set to mouse moving boundary, installation
The Y-direction that X-direction moving leader and installation between the fixed slide rail of X-direction are fixed in the Y direction between slide rail is moved
Dynamic guide rod, mouse are set to the intersection position of X-direction moving leader and Y-direction moving leader.
Further, in the mechanical control system, the both ends of X-direction moving leader are respectively arranged with the first stepping electricity
Machine and second stepper motor, the first stepper motor and second stepper motor control X-direction moving leader are led in the fixed sliding of X-direction
It is moved in rail；Y-direction moving leader is provided with third stepper motor and the 4th stepper motor, third stepper motor and the 4th stepping
Motor control Y-direction moving leader is fixed in slide rail move in the Y direction.
Further, in the mechanical control system, left button, right button and the scroll wheel positions of mouse are respectively arranged with the 5th
Stepper motor, the 6th stepper motor and the 7th stepper motor, the 5th step motor control left mouse button carry out upwardly or downwardly side
To movement, the 6th step motor control right mouse button carries out upwardly or downwardly direction and moves, the rolling of the 7th step motor control mouse
Wheel carries out forward and backward directions move.
The invention has the following advantages:
(1) replaced manually by machine, save in computer regular job and put into a large amount of people in routine, repetitive operation
Power；, in mouse using in this link, human cost can be saved during computer entirely works；
(2) it may replace manual operation, realize the full-automatic of the mouse operation and control under PLC technology, it is ensured that mouse automation
The manipulation of operation and the specific position during manipulation, can be according to the requirement of programming Control, in the state that program is default, root
According to various requirement, the continuous routine of various modes, repetitive operation are realized；
(3) mouse of PLC technology according to the operating mode of program setting, action and working line, can be realized certainly
The contents such as dynamic input, detection, output；
(4) can secondary development, in conjunction with statistics with optimization method, realize mouse use in full-automation, improve work
Efficiency is increased economic efficiency.
Detailed description of the invention
Fig. 1 is the mechanical automatic mouse action apparatus structure schematic diagram of the invention based on machine vision.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to the accompanying drawings and embodiments, right
The present invention is further elaborated.It should be appreciated that described herein, specific examples are only used to explain the present invention, not
For limiting the present invention.
As shown in Figure 1, being the mechanical automatic mouse action apparatus structure schematic diagram of the invention based on machine vision；
A kind of mechanical automatic mouse action device based on machine vision, including video acquisition system, machine vision analysis system
With mechanical control system；
The video acquisition system is used to for computer screen being divided into several net regions, and uses multiple cameras
Matrix carries out Image Acquisition to each net region respectively, and the image data of acquisition is transmitted to machine vision analysis system；
The machine vision analysis system is obtained for successively carrying out target identification to the image data of each camera acquisition
Mouse domain of the existence is taken, carries out mouse tracking in mouse domain of the existence, letter is prompted to dialog box, the dialog box of mouse position
Breath and input information are identified, are generated the mobile control signal of mouse and mouse action control signal and are transmitted to mechanical control
System；
The mechanical control system is used to be moved according to the mobile control signal control mouse of mouse, is grasped according to mouse
Make control signal control mouse and carries out corresponding operating.
In an alternate embodiment of the present invention where, above-mentioned video acquisition system is carried out respective using multiple camera matrixes
It is responsible for the Image Acquisition in region, i.e., computer screen is divided into several net regions, each net region is by mixing up coke
Away from camera carry out Image Acquisition；The approximate region of each camera acquisition is computer screen area/camera sum, example
If camera is 5 rows, every row 7, then being exactly 35 cameras, each camera is responsible for acquiring area being computer screen face
Product adds each camera boundary repeating part area divided by camera number；Particularly, in order to further increase at image
Speed is managed, the present invention can be used low repetition system etc., carry out the reduction of image data.
In an alternate embodiment of the present invention where, above-mentioned machine vision analysis system is to the collected figure of each camera
Picture carries out target identification, until finding mouse domain of the existence according to camera serial number.In this process, the mesh of video tracking
Mark is mouse images, to find target operation region, carries out subsequent tracking.
First, in accordance with camera serial number, video frame reading successively is carried out for the image data of each camera acquisition, from 1
Number, No. 2, No. 3 ... ..., to No. 35；
Which number working region the judgement for carrying out mouse performance region again specifies mouse in: mouse domain of the existence, and the such as the 6th
Number camera obtains region.Such as, mouse is motion state, then can use optical flow algorithm, background subtraction, frame differential method
Deng, carry out the extraction of mouse position, e.g., mouse be it is static, then amplified using reversed low repetition system, then utilize characteristic point
Matching carries out template matching with color characteristic；
After being eventually found mouse domain of the existence, it is timed tracking.
When the present invention carries out target following to mouse domain of the existence, multiple camera collection image regions are located at for mouse
Overlapping region, such as: No. 3, No. 4, No. 10, No. 11 area coincidence regions carry out delay setting time, then respectively to exist be overlapped
Each camera collection image region in region, i.e., No. 3, No. 4, No. 10, No. 11 area images progress mouse identifications, obtains mouse
After existing image-region, mouse tracking is carried out in the mouse domain of the existence.
In addition, can also exist when the present invention is located at the overlapping region in multiple camera collection image regions to mouse
Each camera collection image region of overlapping region carries out image mosaic by the way of finding characteristic point, then in the image district
Domain carries out mouse tracking.
In order to accurately identify mouse position belong to can input content dialog box or dialog box prompt information,
Mouse has input what content etc., and the present invention is identified using the elongated identification block dialog box algorithm of the autonomous super-resolution frame that engages in the dialogue,
After identifying dialog box, the frame prompt information that engages in the dialogue around dialog box identification；Mouse input letter is carried out in dialog box
Breath identification.To text, existing OCR identification technology can be used and identified.
Above-mentioned machine vision analysis system identifies the dialog box of mouse position specifically:
To mouse domain of the existence, sample areas acquisition is carried out according to the acquisition parameter of setting；Acquisition parameter is set as acquiring
Line number, the number of each row, sample areas area and disabled sequence etc., such as 16 row of region preparation for acquiring, 28 samples of every row
Region, each region are 2*2 perhaps 4*4 perhaps 6*6 or 8*8；
16*28 sample areas of acquisition is numbered, while establishing each logical number and mouse domain of the existence
The correspondence mappings relationship of physical location on image；The physical parameter in the target operation region is obtained, such as long 10cm, wide 6cm, target
Sampling area number is 16*28, then the interval in each destination sample region is 10/28,6/10.
Feature extraction, such as 16*28 are carried out to the sample areas of acquisition, calculate the color difference direction gradient arrow of sample areas
Quantify rotational difference, specifically:
Upper and lower, left and right, upper left, upper right, lower left, lower right eight directional calculating are carried out according to gradient information direction, according still further to
The color difference in sample region is calculated, and is carried out vectorized process to direction gradient, is decoupled according to RGB, calculates separately up and down
The gradient in the left and right each direction in four orientation, then carry out Vector modulation；Then 16*28 sample region is carried out to 90 degree of rotation
Turn, then carries out the Colorimetry in a sample region, line direction gradient vectorization of going forward side by side processing；Finally comparison for the first time and
The color difference directional gradient vector value in secondary sample region, in conjunction with gradient information, according to default weight expression formula meter
Calculate the color difference directional gradient vector rotational difference of sample areas, the color difference direction ladder in 16*28 obtained sample region
Vector quantization rotational difference is spent, the color difference direction gradient that color difference directional gradient vector rotational difference is equal to first time sample areas is sweared
The color difference directional gradient vectorization that quantization rotation value subtracts sample areas is worth after rotating.
Label of the color difference directional gradient vector rotational difference in upper lower threshold value be by lower threshold value in setting；It will
Gradient information, first time color difference directional gradient vector value, second of color difference directional gradient vector value, twice color difference direction ladder
Vector quantization difference is spent, secondary vector quantization is carried out, as the auxiliary information clustered；
The cluster based on k- mean value for carrying out default cluster number m, merges the region after cluster, in this way can be with
Obtain a m region segmentation based on cluster；Have at frame, cluster areas area is smaller, more dense, and each cluster area
Domain shape is more irregular, and cluster mean value difference is bigger；At Rimless, clustering distribution region area is bigger, and cluster mean value difference is more
It is small；
According to the state of obtained clustering distribution, the edge segmentation of image is carried out, the dialog box of mouse position is obtained
Boundary.The status image feature of the clustering distribution is dialog box boundary due to gradient information difference, color difference direction gradient before rotating
The difference of color difference directional gradient vector value, different in conjunction with clustering distribution shape, it can to obtain dialog box after vector value and rotation
Effective working region, dialog box.Particularly, after can according to need the amplification and diminution that carry out image, then row processing.
Machine vision analysis system passes through the logical coordinates determined mouse in computer screen image and in electromechanical
Physical coordinates in networked control systems establish the biaxial stress structure between the logical coordinates of mouse and physical coordinates, form computer
The biaxial stress structure of screen picture regional scope and mouse physical motion range, so as to carry out mouse action according to image, move
The accurate control of physical operations such as move, click, double-clicking.
The logical coordinates of logical coordinates and mouse of the machine vision analysis system according to mouse in computer screen image
Biaxial stress structure relationship between physical coordinates, combined mouse generate mouse in the working region range of computer screen image
Mobile control signal.Machine vision analysis system is according to preset identification sample, according to preset requirement, carry out target with
The accurate positioning in track region, the input and state that mouse content is carried out on accurate position change.
In an alternate embodiment of the present invention where, mechanical control system for realizing mouse various operations.Control
After system is connected to the various instructions of mouse action, by stepper motor, carried out in conjunction with camera using the feedback of machine vision
High-precision mouse action --- translation (upper and lower, left and right, it is left it is tiltedly upper, left obliquely downward, it is right it is tiltedly upper, right obliquely downward)；Left mouse button
It clicks, double-click；Right mouse button is clicked, is double-clicked；Mouse roller such as moves up and down at the operation.
Mechanical control system includes the fixation slide rail for being set to mouse moving boundary, is mounted on the fixed cunning of X-direction
X-direction moving leader and installation between dynamic guide rail fix the Y-direction moving leader between slide rail in the Y direction, and mouse is set
It is placed in the intersection position of X-direction moving leader and Y-direction moving leader.
In mechanical control system, the both ends of X-direction moving leader are respectively arranged with the first stepper motor and second step
Into motor, the first stepper motor and second stepper motor control X-direction moving leader move in the fixed slide rail of X-direction；Y
Direction moving leader is provided with third stepper motor and the 4th stepper motor, third stepper motor and the 4th step motor control Y
Direction moving leader is fixed in slide rail move in the Y direction.
In mechanical control system, left button, right button and the scroll wheel positions of mouse are respectively arranged with the 5th stepper motor,
Six stepper motors and the 7th stepper motor, the 5th step motor control left mouse button carry out upwardly or downwardly direction and move, and the 6th
Step motor control right mouse button carry out upwardly or downwardly direction move, the 7th step motor control mouse roller carry out forward and
Backward directions movement.
The present invention is based on computer machine Visual analysis techniques, can be according to the requirement of machine vision analysis system, foundation
Video analysis as a result, presetting in conjunction with software program, realize automatically, the tracking of the mouse state of intelligence, detection, can
In the mode of program setting, time, range, according to the route of setting, mode, the automatic round-trip various operations for carrying out mouse,
It can be used in the occasion in various appliance computers, manual intervention can be largely saved, further save human cost.
Those of ordinary skill in the art will understand that the embodiments described herein, which is to help reader, understands this hair
Bright principle, it should be understood that protection scope of the present invention is not limited to such specific embodiments and embodiments.This field
Those of ordinary skill disclosed the technical disclosures can make according to the present invention and various not depart from the other each of essence of the invention
The specific variations and combinations of kind, these variations and combinations are still within the scope of the present invention.
Claims (10)
1. a kind of mechanical automatic mouse action device based on machine vision, which is characterized in that including video acquisition system,
Machine vision analysis system and mechanical control system；
The video acquisition system is used to for computer screen being divided into several net regions, and uses multiple camera matrixes
Image Acquisition is carried out to each net region respectively, the image data of acquisition is transmitted to machine vision analysis system；
The machine vision analysis system obtains mouse for successively carrying out target identification to the image data of each camera acquisition
Mark domain of the existence, mouse domain of the existence carry out mouse tracking, to the dialog box of mouse position, dialog box prompt information and
Input information is identified, is generated the mobile control signal of mouse and mouse action control signal and is transmitted to mechanical control system
System；
The mechanical control system is used to be carried out using mechanical control mode according to the mobile control signal control mouse of mouse
Movement controls signal control mouse progress corresponding operating according to mouse action.
2. the mechanical automatic mouse action device based on machine vision as described in claim 1, which is characterized in that described
Machine vision analysis system successively carries out video frame reading for the image data of each camera acquisition according to camera serial number
It takes, determines mouse domain of the existence；When mouse is motion state, then mouse position extraction is carried out using image processing algorithm, when
When mouse is stationary state, then image is amplified, Feature Points Matching and color characteristic is recycled to carry out template matching；It obtains
After mouse domain of the existence, mouse tracking is carried out in mouse domain of the existence.
3. the mechanical automatic mouse action device based on machine vision as claimed in claim 2, which is characterized in that described
When machine vision analysis system is located at the overlapping region in multiple camera collection image regions to mouse, when carrying out delay setting
Between, then respectively to there are each camera collection image regions of overlapping region to carry out mouse identification, obtain the existing figure of mouse
As carrying out mouse tracking in the mouse domain of the existence behind region.
4. the mechanical automatic mouse action device based on machine vision as claimed in claim 3, which is characterized in that described
When machine vision analysis system is located at the overlapping region in multiple camera collection image regions to mouse, there are overlapping regions
Each camera collection image region carries out image mosaic by the way of finding characteristic point, then carries out mouse in the image-region
Tracking.
5. the mechanical automatic mouse action device based on machine vision as claimed in claim 4, which is characterized in that described
Machine vision analysis system identifies the dialog box of mouse position specifically:
To mouse domain of the existence, sample areas acquisition is carried out according to the acquisition parameter of setting；
The sample areas of acquisition is numbered, corresponding the reflecting of each logical number with mouse domain of the existence physical location is established
Penetrate relationship；
Feature extraction is carried out to the sample areas of acquisition, calculates the color difference directional gradient vector rotational difference of sample areas；
Label of the color difference directional gradient vector rotational difference in upper lower threshold value be by lower threshold value in setting；
The cluster based on mean value for carrying out default cluster number, merges the region after cluster, obtains based on the more of cluster
A region segmentation；
According to the state of obtained clustering distribution, the edge segmentation of image is carried out, the dialog box boundary of mouse position is obtained.
6. the mechanical automatic mouse action device based on machine vision as claimed in claim 5, which is characterized in that described
Calculate the color difference directional gradient vector rotational difference of sample areas specifically:
Eight directional calculating is carried out according to gradient information direction, is calculated according still further to the color difference in sample region, to direction ladder
Degree carries out vectorized process, calculates separately the gradient in each direction in four orientation up and down, then carry out Vector modulation；Then it will adopt
All one's respective areas carry out 90 degree of rotation, then carry out the Colorimetry in a sample region, line direction gradient vector of going forward side by side
Change processing；The color difference directional gradient vector value of finally comparison first time and secondary sample region, believes in conjunction with gradient
Breath calculates the color difference directional gradient vector rotational difference of sample areas according to default weight expression formula.
7. the mechanical automatic mouse action device based on machine vision as claimed in claim 6, which is characterized in that described
The logical coordinates and physics of logical coordinates and mouse of the machine vision analysis system according to mouse in computer screen image are sat
Biaxial stress structure relationship between mark, combined mouse generate the mobile control of mouse in the working region range of computer screen image
Signal.
8. the mechanical automatic mouse action device based on machine vision as claimed in claim 7, which is characterized in that described
Mechanical control system include the fixation slide rail for being set to mouse moving boundary, be mounted on the fixed slide rail of X-direction it
Between X-direction moving leader and installation fix the Y-direction moving leader between slide rail in the Y direction, mouse is set to X-direction
The intersection position of moving leader and Y-direction moving leader.
9. the mechanical automatic mouse action device based on machine vision as claimed in claim 8, which is characterized in that described
In mechanical control system, the both ends of X-direction moving leader are respectively arranged with the first stepper motor and second stepper motor, and first
Stepper motor and second stepper motor control X-direction moving leader move in the fixed slide rail of X-direction；Y-direction movement is led
Bar is provided with third stepper motor and the 4th stepper motor, and third stepper motor and the movement of the 4th step motor control Y-direction are led
Bar is fixed in slide rail move in the Y direction.
10. the mechanical automatic mouse action device based on machine vision as claimed in claim 9, which is characterized in that institute
It states in mechanical control system, left button, right button and the scroll wheel positions of mouse are respectively arranged with the 5th stepper motor, the 6th stepping electricity
Machine and the 7th stepper motor, the 5th step motor control left mouse button carry out upwardly or downwardly direction and move, the 6th stepper motor
It controls right mouse button and carries out upwardly or downwardly direction and move, the 7th step motor control mouse roller carries out forward and backward directions
Movement.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910560105.5A CN110286825B (en) | 2019-06-26 | 2019-06-26 | Mechanical full-automatic mouse operating device based on machine vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910560105.5A CN110286825B (en) | 2019-06-26 | 2019-06-26 | Mechanical full-automatic mouse operating device based on machine vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110286825A true CN110286825A (en) | 2019-09-27 |
| CN110286825B CN110286825B (en) | 2020-09-08 |
Family
ID=68005927
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910560105.5A Expired - Fee Related CN110286825B (en) | 2019-06-26 | 2019-06-26 | Mechanical full-automatic mouse operating device based on machine vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110286825B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111652225A (en) * | 2020-04-29 | 2020-09-11 | 浙江省北大信息技术高等研究院 | Non-invasive camera reading method and system based on deep learning |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001034385A (en) * | 1999-07-26 | 2001-02-09 | Toyo Commun Equip Co Ltd | Method and device for displaying mouse cursor |
| CN2728726Y (en) * | 2004-04-02 | 2005-09-28 | 李朝辉 | Collected biological information used for controlling motion of robot teaching programing |
| US20050216844A1 (en) * | 2004-03-03 | 2005-09-29 | Error Brett M | Delayed transmission of website usage data |
| CN201262736Y (en) * | 2008-07-08 | 2009-06-24 | 钟泰佑 | Palm optical mouse |
| CN101901095A (en) * | 2009-05-27 | 2010-12-01 | 东莞伍联电子科技有限公司 | Method for operating mouse wheel of computer by differentiating position of mouse cursor |
| CN103124949A (en) * | 2010-08-13 | 2013-05-29 | 许洪 | Input method, input system and input device of vision directing type mouse using monocular camera calibration technique |
| CN104608120A (en) * | 2015-02-05 | 2015-05-13 | 天津市康凯特软件科技有限公司 | Manipulator device for automatic testing of mobile phone and use method |
| US20160188021A1 (en) * | 2014-12-24 | 2016-06-30 | Samsung Electronics Co., Ltd. | Method of receiving user input by detecting movement of user and apparatus therefor |
| CN107908296A (en) * | 2017-11-28 | 2018-04-13 | 深圳市东微智能科技股份有限公司 | KVM control methods, device, storage medium and computer equipment |
| CN108027655A (en) * | 2016-02-25 | 2018-05-11 | 日本电气株式会社 | Information processing system, information processing equipment, control method and program |
| CN108227970A (en) * | 2018-03-08 | 2018-06-29 | 李慕浩 | A kind of remote mouse based on image identification |
-
2019
- 2019-06-26 CN CN201910560105.5A patent/CN110286825B/en not_active Expired - Fee Related
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001034385A (en) * | 1999-07-26 | 2001-02-09 | Toyo Commun Equip Co Ltd | Method and device for displaying mouse cursor |
| US20050216844A1 (en) * | 2004-03-03 | 2005-09-29 | Error Brett M | Delayed transmission of website usage data |
| CN2728726Y (en) * | 2004-04-02 | 2005-09-28 | 李朝辉 | Collected biological information used for controlling motion of robot teaching programing |
| CN201262736Y (en) * | 2008-07-08 | 2009-06-24 | 钟泰佑 | Palm optical mouse |
| CN101901095A (en) * | 2009-05-27 | 2010-12-01 | 东莞伍联电子科技有限公司 | Method for operating mouse wheel of computer by differentiating position of mouse cursor |
| CN103124949A (en) * | 2010-08-13 | 2013-05-29 | 许洪 | Input method, input system and input device of vision directing type mouse using monocular camera calibration technique |
| US20160188021A1 (en) * | 2014-12-24 | 2016-06-30 | Samsung Electronics Co., Ltd. | Method of receiving user input by detecting movement of user and apparatus therefor |
| CN104608120A (en) * | 2015-02-05 | 2015-05-13 | 天津市康凯特软件科技有限公司 | Manipulator device for automatic testing of mobile phone and use method |
| CN108027655A (en) * | 2016-02-25 | 2018-05-11 | 日本电气株式会社 | Information processing system, information processing equipment, control method and program |
| CN107908296A (en) * | 2017-11-28 | 2018-04-13 | 深圳市东微智能科技股份有限公司 | KVM control methods, device, storage medium and computer equipment |
| CN108227970A (en) * | 2018-03-08 | 2018-06-29 | 李慕浩 | A kind of remote mouse based on image identification |
Non-Patent Citations (1)
| Title |
|---|
| 张志杰等: "主-从PTZ视频检测软件模块化设计及架构", 《计算机技术与发展》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111652225A (en) * | 2020-04-29 | 2020-09-11 | 浙江省北大信息技术高等研究院 | Non-invasive camera reading method and system based on deep learning |
| CN111652225B (en) * | 2020-04-29 | 2024-02-27 | 杭州未名信科科技有限公司 | Non-invasive camera shooting and reading method and system based on deep learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110286825B (en) | 2020-09-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106127780A (en) | A kind of curved surface defect automatic testing method and device thereof | |
| CN106408015A (en) | Road fork identification and depth estimation method based on convolutional neural network | |
| CN105262991B (en) | A kind of substation equipment object identifying method based on Quick Response Code | |
| CN109858480A (en) | Digital instrument identification method | |
| CN104202547A (en) | Method for extracting target object in projection picture, projection interaction method and system thereof | |
| EP3822830B1 (en) | Feature processing method and device for motion trajectory, and computer storage medium | |
| CN110991360B (en) | Robot inspection point position intelligent configuration method based on visual algorithm | |
| CN108628267B (en) | A kind of separate type of object space scanning imaging system, distributed control method | |
| CN115797811A (en) | Agricultural product detection method and system based on vision | |
| CN114241469A (en) | Information identification method and device for electricity meter rotation process | |
| CN115424265A (en) | Point cloud semantic segmentation and labeling method and system | |
| CN110286825A (en) | A kind of mechanical automatic mouse action device based on machine vision | |
| CN110807416A (en) | Digital instrument intelligent recognition device and method suitable for mobile detection device | |
| CN114694130A (en) | Method and device for detecting telegraph poles and pole numbers along railway based on deep learning | |
| CN110400335A (en) | Texture image quality estimation method based on deep learning | |
| CN110286780A (en) | A kind of electro-mechanical mixed unattended computer-operated device based on machine vision | |
| Reiterer et al. | A 3D optical deformation measurement system supported by knowledge-based and learning techniques | |
| CN110275628A (en) | A kind of full-automatic mouse action device of electromechanical based on machine vision | |
| CN110308747A (en) | A kind of full-automatic computer-operated device of electronic type based on machine vision | |
| CN110377640A (en) | One kind is based on deep learning ballot statistical method | |
| CN113971627B (en) | License plate picture generation method and device | |
| CN114898215A (en) | Automatic arrangement method of sound barrier | |
| CN110286777A (en) | A kind of full-automatic keyboard operation device of electromechanical based on machine vision | |
| CN113673534A (en) | RGB-D image fruit detection method based on fast RCNN | |
| CN110516551A (en) | A kind of line walking positional shift identifying system, method and the unmanned plane of view-based access control model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20200908 |