TWM635698U - virtual mouse controller - Google Patents

Abstract

A virtual mouse control device, including a camera and an image processing device; wherein, the camera is connected with the image processing device and a computer to photograph the user's face and hands, and the captured facial images and hand The entire image is sent to the impact processing device for recognition and judgment; the image processor is coupled with the camera and includes an image recognition module, the image recognition module has a face recognition unit, a face database, a hand recognition unit and The hand database is used to store and identify the face images and hand images transmitted by the camera; the image recognition module is coupled with an image processing module, and the image processing module is used to identify and obtain observation points in the image In addition, the image processing module is coupled with an image analysis module, the image analysis module is used to analyze and judge the change of the two-dimensional coordinates of the observation point in the image, and form a cursor control command, which can control A cursor for a computer; thereby, a virtual mouse control device that can save costs, improve identification accuracy and practicality of cursor control is provided.

Description

virtual mouse controller

This creation is a virtual mouse control device, especially a virtual mouse control device that can save costs, improve identification accuracy and practicality of cursor control

Since today is the era of network technology, many information and documents are mostly processed, edited and calculated through the computer, and the functions of controlling the computer, program processing, data input, ... and so on are achieved through the mouse and keyboard; In other words, due to the limited mouse operation methods and functional limitations, the shapes of commercially available mice of various brands do not deviate from the basic shape and basic size of the ellipse.

However, because each user's palm size, curvature, and grasping range are different, and under the factors of long-term use of the mouse and poor posture, the user's hand cannot completely cover the back of the mouse. The heel of the palm is compressed, leading to inflammation of the median nerve at the wrist and causing carpal tunnel syndrome, and the pressure on the median nerve at the wrist causes numbness or tingling in the thumb, index finger, middle finger and ring finger, and in severe cases, the hand will Weakness occurs, and even pain extends to the arms, shoulders and neck, making the user suffer from carpal tunnel syndrome.

In order to solve the above-mentioned shortcomings, some companies develop and design a virtual mouse one after another, use a depth camera to shoot hands, and recognize hand movements to judge the manipulation behavior, and then control and operate the computer cursor according to the recognized manipulation behavior; the above-mentioned method, Although the mouse cursor can be controlled without a physical mouse, because this kind of virtual mouse must use depth photography, it must recognize and judge hand movements in a three-dimensional mode. Video cameras are expensive, and at the same time, it takes a long time to perform 3D identification and calculation judgments to achieve correct control of the computer cursor, which is inconvenient to use and relatively expensive to improve.

The purpose of this creation is to improve the above-mentioned defects, so as to provide a virtual mouse control device that can save costs, improve identification accuracy and practicality of cursor control.

In order to achieve the above purpose, the virtual mouse control device of this creation includes a camera and an image processing device; wherein: the camera is connected with the image processing device and a computer to photograph the user's face and hands, and the captured The face image and hand image are sent to the influence processing device for recognition and judgment; the image processor is coupled with the camera and includes an image recognition module. The image recognition module has a face recognition unit, a face database, The hand recognition unit and the hand database are used to store and recognize the facial images and hand images transmitted by the camera; the image recognition module is coupled with an image processing module, which is used for recognition and Obtain the two-dimensional coordinates of the observation point in the image; in addition, the image processing module is coupled with an image analysis module, which is used to analyze and judge the change of the two-dimensional coordinates of the observation point in the image, and form a cursor control The instruction can control the cursor of the computer; with the above-mentioned structure, the invention can indeed provide a virtual mouse control device that can save costs, improve identification accuracy and practicality of cursor control.

10: Camera

20: Image processor

21: Image recognition module

211: Facial recognition unit

212: Face database

213: Hand recognition unit

214: hand database

22: Image processing module

23: Image analysis module

30: computer

〔Figure 1〕is the installation system diagram of this creation.

[Fig. 2] is the flow chart of the steps of this creation.

[Picture 3] is a schematic diagram of the observation points for obtaining hand images in this creation.

Regarding the technical means and the structure thereof used for achieving the purpose of this work, please cooperate with the embodiment shown in Fig. 1 to Fig. 3 again, and describe in detail as follows: As shown in Fig. 1, the virtual mouse control device in the embodiment, Comprising camera 10, image processing device 20; Wherein: camera 10 (see also shown in Figure 1), is connected with image processing device 20 and computer 30, in order to photograph user's face and hand, and the photographed The face image and hand image are sent to the influence processing device 20 for identification and judgment.

The image processor 20 (please refer to FIG. 1 ), is coupled with the camera 10, and includes an image recognition module 21. The image recognition module 21 has a face recognition unit 211, a face database 212, and a hand recognition module. The unit 213 and the hand database 214 are used to store and identify the facial images and hand images transmitted by the camera 10; the image recognition module 21 is coupled with an image processing module 22, and the image processing module 22 is used To identify and obtain the two-dimensional coordinates of the observation point in the image; and the image processing module 22 is coupled to an image analysis module 23, which is used to analyze and judge the two-dimensional coordinates of the observation point in the image change, and form a cursor control instruction, the cursor of the computer 30 can be controlled.

Based on the above, the image processor 20 can be a central processing unit (CPU), a microcontroller unit (MCU), or other logic machines capable of executing computer programs and calculations.

Based on the above, in a preferred embodiment, the camera 10 can be provided with two cameras 10 according to the requirement, which are used to take pictures of the face and hands respectively to obtain images.

As shown in FIG. 2 , the processing steps of the program related to this creation are as follows: start the camera: start the camera 10 to prepare for shooting the user's face and hand images.

Capturing facial images: the camera 10 captures facial images of users, and sends the facial images to the image recognition module 21 of the image processor 20 .

Facial image recognition: through the facial recognition unit 211 in the image recognition module 21 and the facial database 212 for storing facial images, the facial images captured by the camera 10 are recognized and interpreted; if the user's face If part of the image does not conform to the pre-input facial features, the face recognition unit 211 will generate an error signal, causing the image processor 20 to stop the cursor control program, and the cursor of the computer 30 connected to the image processor 20 will not be controlled. , the personnel cannot operate the computer 30; and when the facial image matches the correct facial features of the user, the video camera 10 can capture the user's hand image for the next procedure.

Shooting hand images: the camera 10 shoots the user's hand, and sends the captured hand images to the image recognition module 21 of the image processor 20 .

Hand image recognition and observation point acquisition: the continuous hand images captured by the camera 10 are sent to the hand recognition unit 213, and the position of the hand observation point (such as fingertips) is identified and grasped by the hand database 214. be marked (as shown in Figure 3).

Obtaining the two-dimensional coordinates of the observation point: the image processing module 22 of the image processor 20 obtains the two-dimensional coordinates (X, Y) of the observation point of the hand in the hand image. At this time, the camera 10 will continue to capture the hand image, and continuously transmit the image to the image processor 20 to capture the observation point of the hand and obtain the two-dimensional coordinates.

Observation point coordinate movement judgment: when the user clicks the left button (or right button) and the fingertip moves, the camera 10 sends the image of the fingertip moving hand to the image processor 20, At this time, when the finger selects a case, the movement of the observation point of the index finger tip will obviously produce a change in the Y variation on the two-dimensional coordinates of the image. The image analysis module 23 can identify and judge the two-dimensional coordinates of the fingertip observation point Therefore, the image analysis module 23 can determine that it is in the point-and-click control mode.

When the finger moves the mouse wheel, its index finger and middle finger (or middle finger, ring finger) will bend with the finger, so that the position of the fingertip will change, and the camera 10 will continue to shoot images of the hand, and these fingertips will observe A point on the two-dimensional coordinates of the image will obviously produce a Y change. The image analysis module 23 can identify and judge the change in the two-dimensional coordinates of the fingertip observation point. Therefore, the image analysis module 23 can determine that it is a page scrolling control mode. .

When wanting to move the cursor, the user's index finger (or hand) can move left and right. At this time, since the user's fingertip does not click or bend, the camera 10 continues to shoot the hand image. Some fingertip observation points will obviously produce X variation on the two-dimensional coordinates of the image, but will not produce Y variation, so the image analysis module 23 can determine that it is a cursor movement mode, rather than a click and scroll operation mode .

Computer cursor control: According to the control mode obtained by the image analysis module in the above observation point coordinate movement judgment step, the image processor 20 can transmit the control signal to the coupled computer 30, and the user can smoothly control the computer 30 Cursor, to achieve the control program of clicking, page scrolling and cursor movement.

Therefore, as can be seen from the above, this creation has the following advantages:

(1) Since this creation is a virtual mouse to control the computer 30 cursors, the user does not need to operate a physical mouse with his hands, so it can indeed reduce wrist injuries and avoid suffering from carpal tunnel syndrome.

(2) Since this creation system uses the camera 10 to identify and judge the operation mode with finger movements for hand photography, and then achieve computer cursor control, it does not require a physical mouse, so it can save desktop space and avoid mouse button sounds , At the same time, there is no risk of hand contact with the mouse, reducing the chance of infection.

(3) Also, since this creation is to set the fingertip observation point and two-dimensional coordinates (X, Y) in the hand image first, no matter when the fingertip observation point is clicked, scrolled or moved, the fingertip observation point The two-dimensional coordinates (X, Y) will have a moving amount and will cause coordinate changes. By identifying and judging the change pattern of the coordinates, the identification of the cursor control mode can be completed, and then the computer cursor can be controlled.

(4) Furthermore, since this creation uses the image processor 20 to identify and judge the two-dimensional coordinates of the fingertip observation point, without depth photography or deep learning, it is only necessary to set up a general camera instead of a depth camera or 3D camera, this creation can indeed reduce the setup cost of the virtual mouse.

(5) In addition, since this creation uses the coordinate changes of the two-dimensional coordinates of the fingertip observation point for identification and judgment, even when the user's fingers overlap, this creation can also identify and judge the fingertip observation point; compared with the depth In the way of photography, the superimposed fingers will present a mass of white images after being converted into a binary image, which cannot be recognized and judged. This creation is indeed more accurate and practical.

(6) Also, since the face recognition unit 211 and the face database 212 are set in this creation, it can be used to identify whether the person who wants to operate the computer is the correct user, if it is not the correct user stored in the face database 212 , then the image processor 20 of this invention interrupts the cursor control program immediately, and the risk of personnel accidentally touching or avoiding data leakage can actually improve safety and practicability.

Therefore, from the above, it can be seen that compared with the previous technology, this creation is indeed significantly progressive, and its structure is indeed unprecedented, and it has met the requirements of a new patent. I file a patent application in accordance with the law and pray for the grant of a patent. Pray, I am most grateful.

However, the above is only a feasible embodiment of this creation. This embodiment is mainly used to illustrate the technical means and structure used by this creation to achieve the purpose, so it cannot limit the scope of protection of this creation. For example, all equivalent changes or modifications made in accordance with the description of this creation and the scope of the patent application shall still fall within the scope of protection covered by this creation.

10: Camera

20: Image processor

21: Image recognition module

211: Facial recognition unit

212: Face database

213: Hand recognition unit

214: hand database

22: Image processing module

23: Image analysis module

30: computer

Claims (2)

A virtual mouse control device, including a camera and an image processing device; wherein: The camera is connected with the image processing device and the computer to take pictures of the user's face and hands, and transmit the captured face and hand images to the processing device for identification and judgment; The image processor is coupled with the camera and includes an image recognition module. The image recognition module has a face recognition unit, a face database, a hand recognition unit and a hand database for storing and recognizing The transmitted facial image and hand image; the image recognition module is coupled with an image processing module, and the image processing module is used to identify and obtain the two-dimensional coordinates of observation points in the image; and the image processing module Coupled with an image analysis module, the image analysis module is used to analyze and judge the change of the two-dimensional coordinates of the observation point in the image, and form a cursor control command to control the cursor of the computer. The virtual mouse control device as described in Claim 1, wherein the camera is provided with two cameras, and it is best to take pictures of the face and the hand respectively.

Images