WO2021004505A1

WO2021004505A1 - Air mouse, air mouse system, image processing method and control method

Info

Publication number: WO2021004505A1
Application number: PCT/CN2020/101074
Authority: WO
Inventors: 周海涛
Original assignee: 周海涛
Priority date: 2019-07-10
Filing date: 2020-07-09
Publication date: 2021-01-14
Also published as: CN112214117A

Abstract

The present invention discloses an air mouse, an air mouse system, an image processing method and a control method, wherein the image processing method includes the following steps: (a) obtaining a positioning image with a positioning mark image; and (b) determining a feature code and image position data of the positioning mark image by reading the pixel value of the pixel in the positioning image. The air mouse using the image processing method can increase the moving length by adding positioning marks.

Description

Air mouse, air mouse system, image processing method and control method

Technical field

The present invention relates to the field of computer and electronic game control, in particular to an image processing method for an air mouse, a chip adopting the image processing method, an air mouse, an air mouse system, an air mouse control system, and a device for intelligent mobile equipment Air mouse, air mouse driver and computer system using the program, infrared sensor strip providing positioning mark for air mouse, cloud game system, image processing method for computer.

Background technique

The mouse is the most common computer and electronic game control device. Its main purpose is to control the movement of the cursor on the screen of display devices such as televisions and monitors. The cursor refers to various icons that move with the movement of the mouse, such as the mouse pointer on the screen, the weapon crosshairs in video games, and the characters.

Movement accuracy and movement length are two important aspects to measure the performance of a mouse. The movement accuracy refers to the minimum number of pixels that the mouse can control the cursor on the screen by one-way movement; the movement length refers to the one-way movement that can be controlled by the mouse once. The maximum number of pixels the cursor moves on the screen.

The common optical mouse has a high level of movement accuracy and movement length. A slight movement of the optical mouse on the desktop can make the cursor move only 1 pixel on the screen; if it keeps moving in one direction, the cursor can be moved from One edge of the screen moves to the opposite edge. The optical mouse is used to control a screen with a resolution of 1920×1080 or higher, which can have both movement precision and movement length.

The air mouse is another device used to control the cursor on the screen. Unlike an optical mouse, an air mouse controls the cursor on the screen to move by moving it in the air.

Chinese Patent Application No. 201210319834.X discloses an air mouse using an infrared image recognition scheme. The air mouse emits infrared light through an infrared light emitter, and the infrared light is sensed by the infrared image sensor on the body to form a A series of images, in which there are light spots formed by infrared light emitters. When the user holds the main body and moves, the position of the light spot changes in the series of images formed. The main body sends position change data to a controlled device according to the change of the position of the light spot in the adjacent image, and the controlled device controls the cursor on the driven screen to move according to the position change data.

Compared with an optical mouse, an air mouse using an infrared image recognition solution cannot meet the requirements of mainstream display devices while ensuring a higher movement accuracy.

Due to the limitation of image processing performance, the image resolution output by the infrared image sensor of the air mouse is relatively low, such as 320×240. In the case of a movement accuracy of 1 (that is, a single movement can control the cursor on the screen to move 1 pixel at least), the body moves once in a single direction, and the horizontal movement length of the cursor on the screen does not exceed 320, and the vertical movement length does not exceed 240 . The existing mainstream display devices such as LCDs and TVs have a resolution of 1920×1080 and above. When the air mouse of the prior art is used in this type of display device, a horizontal movement can control the cursor to move up to a sixth of the distance on the screen; a vertical movement can control the cursor to move up to a quarter of the distance on the screen. . Far from meeting the needs. In order to meet the needs of mainstream display devices, the body part can increase the moving length by enlarging the output ratio of the position change data. For example, if the change in the position of the light spot in two adjacent frames of images is measured to be 1 pixel, the screen on the screen is controlled to move 6 pixels. At this time, the lateral movement length can reach about 1920, which can meet the needs of use. But the movement accuracy is reduced from 1 to 6, that is, the body moves once, and the minimum distance for the cursor to move on the screen is 6 pixels. Such movement accuracy will lead to a very poor experience. Therefore, the existing air mouse using the infrared image recognition solution cannot meet the requirements of movement accuracy and movement length at the same time when used in mainstream resolution display devices, which greatly restricts the application of the air mouse.

The infrared light transmitter is equivalent to providing a positioning mark for the body of the air mouse, so that it can determine its position in the air at any time. However, only one infrared light transmitter is used, which is equivalent to only one positioning mark, so that the air mouse cannot maintain a high level of movement accuracy and movement length at the same time. To increase the movement length, the movement accuracy must be reduced.

If you want to increase the length of the movement while maintaining the accuracy of the movement, it is necessary to increase the number of positioning marks. However, the prior art air mouse cannot increase the moving length by increasing the number of positioning marks. This is limited by the image processing method used by the air mouse. The image processing method outputs position change data according to the position change of the image formed by positioning marks (such as infrared light source) in two adjacent frames of images. When there is only one positioning mark, the two images for position comparison are the same positioning mark. The image's position change data can reflect the moving direction and distance of the air mouse; when there are 2 or more positioning marks, the two images to be compared may be formed by different positioning marks, and the position of the image changes The data cannot correctly reflect the moving direction and distance of the air mouse. Controlling the movement of the cursor on the screen according to the position change data may cause the moving direction and distance to be different from the actual moving direction and distance of the air mouse.

Nintendo has released an infrared sensor strip, including a long strip of bracket and 4 infrared light sources arranged on the bracket. The problem with using this infrared sensor bar to provide positioning marks for the air mouse is that the four infrared light sources arranged on the bracket are the same, which will cause the images output by the infrared image sensor of the air mouse to be all it's the same. Therefore, although the infrared sensor strip can provide 4 positioning marks, it still cannot increase the moving length of the air mouse.

Smart mobile devices such as smartphones and tablet computers are mainly operated by touch. This limits the development of mobile games, because many types of games are suitable for mouse operation, such as FPS, MOBA, etc. When this part of the game is operated on a smart phone or tablet computer, the touch screen operation method is far from the mouse operation method that players are accustomed to, and the cursor cannot be controlled quickly and accurately, resulting in a poor experience.

In the prior art, there is another solution to control the cursor on the screen by processing infrared images. This solution uses an air mouse with infrared image capture function to capture infrared images, and then sends the infrared images to a controlled device ( electronic calculator). The controlled device performs image processing on the received infrared image and controls the cursor on the screen to move according to the result of the image processing. Compared with the aforementioned solution, the main body of image processing has changed from an air mouse to a controlled device, but the image processing method is the same, so there are still the same shortcomings when performing cursor control.

The present invention was produced under this background.

Summary of the invention

An object of the present invention is to provide an image processing method, wherein the image processing method is suitable for an air mouse. The air mouse adopting the image processing method can increase the moving length by adding positioning marks.

Another object of the present invention is to provide an image processing method, wherein the image processing method is used for an air mouse. The air mouse using this image processing method can increase the moving length without reducing the moving accuracy.

Another object of the present invention is to provide an air mouse, which can increase the moving length by adding positioning marks.

Another object of the present invention is to provide an air mouse system with high moving accuracy and length.

Another object of the present invention is to provide an air mouse system that can meet the requirements of mainstream display devices (screen resolution of about 1920×1080) in terms of movement accuracy and movement length.

Another object of the present invention is to provide an image processing chip that can be used for an air mouse, so that the air mouse can increase the moving length by adding positioning marks.

Another object of the present invention is to provide an air mouse control system for controlling a cursor on a display screen, which can increase the moving length of the cursor on the screen by adding positioning marks.

Another object of the present invention is to provide an air mouse control system for controlling the cursor on a display screen, which can be used in conjunction with multiple (two or more) positioning marks, while achieving a higher moving length and movement Accuracy.

Another object of the present invention is to provide an air mouse driver, which is installed on a controlled device connected to the air mouse, and can control the cursor movement on the screen according to the cursor control signal provided by the air mouse.

Another object of the present invention is to provide a computer system capable of controlling the movement of the cursor on the controlled screen according to the cursor control signal provided by the air mouse.

Another object of the present invention is to provide an air mouse for smart mobile devices, which can quickly and accurately control the cursor and games on the smart mobile device screen.

Another object of the present invention is to provide an infrared sensor strip for providing positioning marks for the air mouse, which can increase the moving length of the air mouse without reducing the movement accuracy.

Another object of the present invention is to provide an infrared sensor strip that can provide positioning marks for the air mouse, so that the air mouse can have both moving length and accuracy.

Another object of the present invention is to provide a cloud game system in which the air mouse can increase the moving length by adding positioning marks.

Another object of the present invention is to provide an air mouse. When the positioning mark changes during the movement of the air mouse, the movement of the air mouse can also be reflected on the display screen.

Another object of the present invention is to provide a driver for an air mouse, which enables the movement of the air mouse to be reflected on the display screen when the positioning mark changes during the movement of the air mouse.

Another object of the present invention is to provide an image processing method for a computer. The computer adopting the image processing method can control the cursor on the driven screen to move according to the positioning image.

Another object of the present invention is to provide a computer system capable of controlling the movement of the cursor on the driven screen according to the positioning image.

To achieve the above objective, the present invention provides an image processing method for air mouse. The method is characterized by

When a frame of positioning image is processed, the feature code of the positioning mark image and the image position data are obtained, wherein the feature code is determined according to the image feature of the positioning mark image.

In an embodiment of the image processing method of the present invention, after the feature code and image position data are obtained from the positioning image, the feature code obtained from the previous frame of the positioning image is searched for the same.

In an embodiment of the image processing method of the present invention, after the feature code and image location data are obtained from the positioning image, the image location data corresponding to the same feature code in the current frame and the previous frame of the positioning image are compared. Yes, and get the image displacement data.

In an embodiment of the image processing method of the present invention, when a frame of positioning image is processed, the feature code and image position data of one of the positioning mark images are acquired.

In an embodiment of the image processing method of the present invention, when one frame of positioning image is processed, the feature codes and image position data of all positioning mark images therein are acquired.

The invention also provides an air mouse. The air mouse includes a housing, an image acquisition unit, and a processing unit. The image acquisition unit is used to photograph a scene including the positioning mark and output a positioning image with the positioning mark image; the processing unit performs image processing on the positioning image, and sends a cursor control signal to the controlled device according to the image processing result. When image processing is performed on the positioning image, it includes obtaining the feature code of the positioning mark image and the image location data, wherein the feature code is determined according to the feature of the positioning mark image.

In an embodiment of the air mouse of the present invention, the processing unit sends the feature code and image position data to the controlled device.

In an embodiment of the air mouse of the present invention, after the processing unit obtains the feature code and image position data from one frame of positioning image, it searches for the same feature code in the feature code obtained from the previous frame of positioning image . If there are the same, the image location data corresponding to the feature code is sent to the controlled device; if there is no the same, the rebranding signal is sent to the controlled device.

In an embodiment of the air mouse of the present invention, the processing unit obtains image displacement data according to the displacement of the positioning mark images with the same image characteristics in the two previous and subsequent frames of positioning images, and sends a displacement signal to the controlled device according to the image displacement data.

In an embodiment of the air mouse of the present invention, after the processing unit obtains the feature code and image position data from one frame of positioning image, it searches for the same feature code in the feature code obtained from the previous frame of positioning image . If there are the same, compare the corresponding image position data of the feature code in the two frames of positioning images, and send the displacement signal to the controlled device according to the obtained image displacement data; if there is no the same, then locate the previous frame The displacement signal generated after image processing is sent to the controlled device.

In an embodiment of the air mouse of the present invention, it further includes a button unit. The key unit is connected with the processing unit, and the key signal generated by the key unit is sent to the controlled device through the processing unit.

In an embodiment of the air mouse of the present invention, the air mouse includes a housing, an image acquisition unit, an image processing unit, a button unit, and a main control unit. The image acquisition unit is used to photograph the scene including the positioning mark and output the positioning image with the positioning mark image; the image processing unit performs image processing on the positioning image and sends the image processing result to the main control unit; the main control unit according to the image processing result The cursor control signal is sent to the controlled device; the key signal generated by the key unit is sent to the controlled device through the main control unit; when the image processing unit processes the positioning image, the image processing method provided by the present invention is adopted.

In an embodiment of the air mouse of the present invention, after the processing unit obtains the feature code and image position data from one frame of positioning image, it searches for the same feature code in the feature code obtained from the previous frame of positioning image ; If there are the same, the image location data corresponding to the feature code is sent to the controlled device; if there is no the same, the rebranding signal is sent to the controlled device.

In an embodiment of the air mouse of the present invention, after the processing unit obtains the feature code and image position data from one frame of positioning image, it searches for the same feature code in the feature code obtained from the previous frame of positioning image ; If there are the same, compare the corresponding image position data of the feature code in the two frames of positioning images, and send the displacement signal to the controlled device according to the obtained image displacement data; if there is no the same, the previous frame The displacement signal generated after positioning image processing is sent to the controlled device.

In an embodiment of the air mouse of the present invention, the air mouse includes a housing, an image acquisition unit, an image processing unit, a button unit, and a main control unit; the image acquisition unit is used to photograph a scene including a positioning mark and output the positioning The positioning image of the logo image; the image processing unit performs image processing on the positioning image and sends the image processing result to the main control unit; the main control unit sends the cursor control signal to the controlled device according to the image processing result; the key signal generated by the key unit Sending to the controlled device through the main control unit; when the image processing unit processes the positioning image, it includes acquiring the feature code of the positioning mark image and the image position data, wherein the feature code is determined according to the feature of the positioning mark image.

The present invention also provides an air mouse system including the air mouse of the present invention and at least two positioning marks with different characteristics. The positioning mark images formed by the positioning marks with different characteristics in the positioning image have different image characteristics.

In an embodiment of the air mouse system of the present invention, the positioning mark images formed by positioning marks with different characteristics in the positioning image have different pixel values of the constituent pixels.

In an embodiment of the air mouse system of the present invention, the positioning mark images formed by positioning marks with different characteristics in the positioning image have different arrangements of the constituent pixels.

In an embodiment of the air mouse system of the present invention, the positioning mark images formed by positioning marks with different characteristics in the positioning image have different numbers of pixels.

The invention also provides an air mouse control system, which includes the air mouse and controlled equipment of the invention. The air mouse sends a cursor control signal to the controlled device, and the controlled device controls the movement of the cursor on the driven screen according to the cursor control signal.

In an embodiment of the air mouse control system of the present invention, the cursor control signal sent by the air mouse to the controlled device is image data (including feature code and image position data). When the driver of the controlled device receives a group of image data, it controls the movement of the cursor on the driven screen according to the change of the image position data corresponding to the same feature code in the group and the previous group of image data.

In an embodiment of the air mouse control system of the present invention, the cursor control signal sent by the air mouse to the controlled device is image position data and a rebrand signal. The driver of the controlled device controls the movement of the cursor on the driven screen after obtaining the image displacement data by processing the image position data and the conversion signal.

In an embodiment of the air mouse control system of the present invention, the cursor control signal sent by the air mouse to the controlled device is a displacement signal, and the controlled device controls the movement of the cursor on the screen of the driven display device according to the displacement signal.

In an embodiment of the air mouse control system of the present invention, the positioning mark images formed by the positioning marks with different characteristics in the positioning image have different arrangement of the constituent pixels.

In an embodiment of the air mouse control system of the present invention, the air mouse control system includes an air mouse and a controlled device. The air mouse includes a housing, an image acquisition unit, and a processing unit; the image acquisition unit is used for To photograph a scene including a positioning mark, and output a positioning image with a positioning mark image; the processing unit performs image processing on the positioning image, and sends a cursor control signal to the controlled device according to the image processing result; When the image is processed, it includes obtaining the feature code of the positioning mark image and the image position data, wherein the feature code is determined according to the feature of the positioning mark image; the controlled device controls the cursor movement on the driven screen according to the cursor control signal.

In an embodiment of the air mouse control system of the present invention, the cursor control signal sent by the air mouse to the controlled device is image data (including feature codes and image position data); the driver of the controlled device When the program receives a group of image data, it controls the movement of the cursor on the driven screen according to the change of the image position data corresponding to the same feature code in the group and the previous group of image data.

In an embodiment of the air mouse control system of the present invention, the cursor control signal sent by the air mouse to the controlled device is image position data and a rebranding signal; The position data and the conversion signal are processed to obtain the image displacement data and then control the cursor movement on the driven screen.

In an embodiment of the air mouse control system of the present invention, the cursor control signal sent by the air mouse to the controlled device is a displacement signal, and the controlled device controls the cursor on the screen of the driven display device according to the displacement signal mobile.

The present invention also provides an infrared sensor strip for providing positioning marks for an air mouse, which includes a bracket and at least two infrared light sources mounted on the bracket, and the two infrared light sources have different characteristics.

In an embodiment of the infrared sensor strip of the present invention, the brightness of the two infrared light sources is different.

In an embodiment of the infrared sensor strip of the present invention, the light-emitting areas of the two infrared light sources are different.

In an embodiment of the infrared sensor strip of the present invention, the two infrared light sources have different shapes.

In an embodiment of the infrared sensor strip of the present invention, the bracket is a hollow structure.

The present invention provides an image processing method used in a computer for processing a positioning image including a positioning mark image, which includes the following steps:

(a) Obtain the feature code and image location data of the location mark image; where the feature code is determined according to the image feature of the location mark image;

(b) Compare the obtained feature code with the feature code obtained in the previous frame of the positioning image, and if they are the same, compare the image position data corresponding to the feature code.

The present invention provides a method for a computer to control the movement of a cursor on a driven screen, which includes the following steps:

(a) Receive two frames of positioning images at intervals, and the two frames of positioning images have positioning mark images;

(b) Determine whether the positioning mark images in the two frames of the positioning image have the same characteristics, if so, compare the positions of the positioning mark images in the two frames of the positioning image, and if an image displacement occurs Data, the cursor on the screen driven by the image displacement data is controlled to move.

In the method for controlling the movement of a cursor on a screen driven by a computer of the present invention, the step (b) includes the following steps: after processing the positioning image for each frame, if the image displacement data is generated, then The cursor movement on the driven screen is controlled according to the image displacement data; if the image displacement data is not generated, the cursor movement on the driven screen is controlled according to the image displacement data generated after the positioning image processing of the previous frame.

The invention also provides an image processing chip for processing digital images. When the image processing chip processes digital images, the image processing method of the present invention is adopted.

The invention also provides a computer system, which includes a controlled device and a driver of the air mouse.

The present invention also provides an infrared sensor strip, which includes a bracket and at least two infrared light sources with different characteristics.

In an embodiment of the infrared sensor strip of the present invention, adjacent infrared light sources have different brightness.

In an embodiment of the infrared sensor strip of the present invention, adjacent infrared light sources have different light-emitting areas.

In an embodiment of the infrared sensor strip of the present invention, adjacent infrared light sources have different shapes.

In an embodiment of the infrared sensor strip of the present invention, the bracket is hollow to conveniently accommodate the infrared light source and the battery.

The present invention also provides an air mouse for smart mobile devices. The air mouse includes a housing, an image acquisition unit, and a processing unit. The image acquisition unit is used to photograph a scene including a positioning mark and output a positioning image containing the positioning mark image; the processing unit obtains a cursor control signal by image processing the positioning image, and sends the cursor control signal to the controlled intelligent mobile device (Smartphone or tablet) to send. When the processing unit performs image processing on the positioning image, the image processing method of the present invention is adopted.

The controlled devices in the air mouse control system of the present invention are various types of electronic computer devices, including but not limited to PCs, video game consoles, smart TVs, set-top boxes, tablet computers, smart phones, and VR devices.

Compared with the image processing method of the air mouse in the prior art, the image processing method of the air mouse of the present invention further acquires the feature code according to the image characteristics on the basis of acquiring the location data of the positioning mark image in each frame of the positioning image. The air mouse to which the image processing method is applied can be used in conjunction with two or more positioning marks with different characteristics to achieve the purpose of increasing the moving length without reducing the moving accuracy.

The image processing chip of the present invention adopts the image processing method of the present invention. Therefore, the air mouse adopting the image processing chip can increase the moving length by adding positioning marks.

The air mouse system of the present invention uses two or more positioning marks. The movement length is increased without reducing the movement accuracy. When the air mouse is used in mainstream display devices, it has good effects in both the length and accuracy of the cursor movement, and it is truly practical. While the cursor performance is greatly improved, the increased cost is very small (only positioning marks are added to the hardware), so while achieving higher cursor control performance, it can still maintain a lower cost, which is conducive to popularization and application.

The air mouse control system of the present invention is used to control the movement of the cursor on the screen of the display device driven by the controlled device, and the movement length is greatly increased without reducing the cursor movement accuracy. It meets the needs of mainstream display devices (screen resolution of about 1920×1020).

In the infrared sensor strip of the present invention, adjacent infrared light sources have different characteristics, so that the positioning mark image formed in the positioning image has different image characteristics. When used in conjunction with an air mouse, it can make the air mouse have a higher moving length and moving accuracy.

According to one aspect of the present invention, the present invention further provides an air mouse suitable for controlling a controlled device, which includes:

A shell

An image acquisition unit, wherein the image acquisition unit is arranged on the housing, and the image acquisition unit outputs a positioning image with a positioning mark image;

A processing unit, wherein the processing unit is arranged in the housing, the processing unit is communicably connected to the image acquisition unit, and the processing unit is based on a feature code and a feature code of the positioning mark image The image position data processes the positioning image, and sends a cursor control signal to the controlled device to control the controlled device.

According to an embodiment of the present invention, the air mouse further includes a button unit, wherein the button unit is disposed on the housing, and the button unit is communicably connected to the processing unit.

According to an embodiment of the present invention, the air mouse further includes a cursor movement control key, wherein the cursor movement control key is provided on the housing, and the cursor movement control key is communicatively connected to the processor. unit.

According to an embodiment of the present invention, the image acquisition unit includes a lens and an image sensor module, wherein the lens is communicably connected to the image sensor module.

According to one aspect of the present invention, the present invention further provides an air mouse system suitable for controlling a controlled device, which includes:

An air mouse, wherein the air mouse includes a housing, an image acquisition unit, and a processing unit, wherein the image acquisition unit is disposed on the housing, and the image acquisition unit outputs an image with a positioning mark A positioning image, wherein the processing unit is provided in the housing, the processing unit is communicably connected to the image acquisition unit, and the processing unit is based on a feature code and a feature code of the positioning mark image The image position data processes the positioning image and sends a cursor control signal to the controlled device; and

At least one positioning mark, wherein the positioning mark can form the positioning image with the positioning mark image, and the position of the air mouse can be determined by using the positioning mark.

According to an embodiment of the present invention, the positioning marks of the air mouse system are implemented as at least two, and the two positioning marks have different characteristics, and the characteristics of each positioning mark can be recorded in The positioning image.

According to an embodiment of the present invention, the positioning mark image formed by the two positioning marks in the positioning image has different pixel values for its constituent pixels.

According to an embodiment of the present invention, the positioning mark images formed by the two positioning marks in the positioning image are composed of different numbers of pixels.

According to an embodiment of the present invention, the positioning mark is implemented as an infrared light source.

According to an embodiment of the present invention, the positioning mark is implemented as a visible light source.

According to an embodiment of the present invention, the distance between the air mouse and the positioning mark is maintained to be greater than or equal to 50 cm and less than or equal to 100 cm.

According to an embodiment of the present invention, the air mouse system further includes a bracket, wherein the positioning mark is laterally arranged on the bracket.

According to one aspect of the present invention, the present invention further provides an image processing method. The image processing method includes the following steps:

(a) Obtain a positioning image with a positioning mark image; and

(b) Determine a feature code and image position data of the positioning image by reading the pixel value of the pixel in the positioning image.

According to an embodiment of the present invention, in the step (b), the step (c) reads the pixel value of each pixel in the positioning image one by one, and finds the first pixel value greater than a preset pixel value In the case of pixels, the pixel values and coordinates of the pixels are recorded.

According to an embodiment of the present invention, the image processing method further includes the step (d) comparing the feature codes of the positioning mark images of the two positioning images acquired first.

According to an embodiment of the present invention, after the step (b), a step (e) is included to compare the positioning mark images of the two sets of positioning images with the same feature code, and obtain an image displacement data.

According to an embodiment of the present invention, the step (b) further includes:

(I) Read the pixel value of each pixel one by one in a preselected area until a pixel with a pixel value greater than a preset pixel value is found;

(Ii) Find all pixels in a preset pixel range centered on the pixel whose pixel value is greater than the preset pixel value; and

(Iii) Determine the feature code according to the number of all pixels found, and find the pixel with the smallest coordinate value among all the pixels whose pixel value is greater than the preset pixel value, and record the coordinate of this pixel as the image position data.

According to an embodiment of the present invention, the step (b) further includes the following steps:

(I) Read the pixel values of all pixels in a preselected area, and find out all the pixels whose pixel values are greater than a preset pixel value as a candidate pixel;

(Ii) Select a pixel with the smallest coordinate value from the candidate pixels, find out all pixels with a pixel value greater than the preset pixel value in a preset pixel range centered on the pixel, and define these pixels as One component pixel;

(Iii) Record the total number of the constituent pixels as the feature code, find a pixel with the smallest coordinate value among the constituent pixels, and record the coordinate of the pixel as the image position data corresponding to the feature code.

According to an embodiment of the present invention, in the step (b), it further includes the step (iv) to remove the constituent pixels from the candidate pixels, and if the candidate pixels remain, proceed from step (Ii) Start processing, if there are no candidate pixels, the image processing process ends.

According to an embodiment of the present invention, in the step (b), further comprising the following steps:

(I) In a preselected area, read the pixel value of each pixel according to the order from left to right and from top to bottom until a pixel with a pixel value greater than a preset pixel value is found;

(Ii) Find out all pixels with a pixel value greater than the preset pixel value within a preset pixel range centered on the pixel;

(Iii) If the column coordinates of all the pixels found are the same, record the feature code as 1; if the row coordinates are the same, record the feature code as 2; and

(Iv) Among all the found pixels, find the pixel with the smallest coordinate value, and record the coordinate of the pixel as the image position data corresponding to the feature code.

(Iii) If the column coordinates of all the constituent pixels found are the same, record the feature code as 1; if the row coordinates of all constituent pixels are the same, record the feature code as 2; and among all the constituent pixels, find one The pixel with the smallest coordinate value, and the coordinate of the pixel is recorded as the image position data; and

(Iv) Among the candidate pixels, remove the pixels with the same coordinates as the constituent pixels. If the candidate pixels are still left, then start processing from step (ii); if there are no candidate pixels, Then the step of acquiring image data ends.

According to one aspect of the present invention, the present invention provides a cursor control method. The cursor control method includes the following steps:

(A) receiving at least two sets of image data at intervals, wherein the image data includes a feature code and an image position data corresponding to the feature code; and

(B) Determine whether to generate image displacement data according to the feature code of the image data and the image position data, and if the image position data is generated, control the movement of a cursor according to the image displacement data.

According to an embodiment of the present invention, the step (B) includes the following steps:

(B.1) Searching for the feature code of the latter group of image data in the received previous group of image data;

(B.2) The positioning mark images of the two sets of positioning images that are the same as the feature code, and obtaining the image displacement data; and

(B.3) Control the movement of a cursor according to the image displacement data.

According to an embodiment of the present invention, the step (B) includes the following step: after the positioning image is processed for each frame, if the image displacement data is generated, then the image displacement data is sent to the The controlled device sends the displacement signal; if the image displacement data is not generated, the controlled device sends the displacement signal generated after processing the positioning image of the previous frame.

Description of the drawings

Figure 1 shows an embodiment of the air mouse of the present invention.

Figure 2 shows the air mouse of the present invention and its usage.

Figure 3 shows an implementation of the air mouse driver of the present invention.

Figure 4 shows the signal flow of the air mouse and air mouse control system of the present invention.

Figure 5 shows an embodiment of the infrared sensor strip of the present invention.

Figure 6 shows an embodiment of the infrared sensor strip of the present invention.

FIG. 7 shows an embodiment of the image processing method of the present invention.

Figure 8 shows an embodiment of the infrared sensor strip of the present invention.

Figure 9 shows an embodiment of the infrared sensor strip of the present invention.

Figure 10 shows an embodiment of the infrared sensor strip of the present invention.

Figure 11 shows an embodiment of the infrared sensor strip of the present invention.

Fig. 12 shows an embodiment of the image processing method of the present invention.

Figure 13 shows an embodiment of the infrared sensor strip of the present invention.

Figure 14 shows an embodiment of the infrared sensor strip of the present invention.

Fig. 15 shows an embodiment of the image processing method of the present invention.

Figure 16 shows an embodiment of the air mouse driver of the present invention

FIG. 17 shows an embodiment of the image processing method of the present invention.

Figure 18 shows an embodiment of the air mouse and air mouse control system of the present invention.

Figure 19 shows an embodiment of the air mouse and air mouse control system of the present invention.

Detailed ways

The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments in the following description are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, modifications, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the present invention.

Those skilled in the art should understand that, in the disclosure of the present invention, the terms "longitudinal", "lateral", "upper", "lower", "front", "rear", "left", "right", " The orientation or positional relationship indicated by "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. are based on the orientation or positional relationship shown in the drawings, which is only for the convenience of describing the present invention and The description is simplified, rather than indicating or implying that the pointed device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore the above terms should not be construed as limiting the present invention.

It can be understood that the term "a" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in other embodiments, The number can be multiple, and the term "one" cannot be understood as a restriction on the number.

Hereinafter, the present invention will be further described through specific embodiments in conjunction with the drawings.

Example one

This embodiment provides an air mouse. As shown in FIG. 1, the air mouse includes a housing 200, buttons 201 arranged on the upper surface of the housing 200, a lens 202 arranged at the middle position of the front end of the housing 200, and a circuit part in the housing 200. The circuit part includes a button circuit connected to the button 201, an image sensor module connected to the lens 202, and a processing unit.

The lens 202 and the image sensor module together form an image acquisition unit, where the lens 202 is a filter lens that can filter out visible light and only allows infrared light to pass through. The image sensor module is an OV7620 image sensor module, which integrates a CMOS image sensor and a DSP chip, and a lens 202 to form a complete infrared digital camera module, which can capture infrared light-emitting objects and output digital images. The button 201 and the button circuit together form a button unit, and the button signal generated by the button unit is sent to a controlled device after passing through the processing unit. The key signal is used to realize the key control function and belongs to the prior art.

The processing unit uses STMicroelectronics' STM32F407 chip. The output end of the OV7620 image sensor module is electrically connected to the input pin of the STM32F407 chip, and the button circuit is electrically connected to the input pin of the STM32F407. The processing unit is also used to bridge the controlled device. In this embodiment, the controlled device is a PC, which is connected to the STM32F407 via a USB cable. The controlled device drives a display device. In this embodiment, the display device is a liquid crystal display with a resolution of 1920×1080. The processing unit can also connect and communicate with the controlled device wirelessly.

In order to realize the cursor control function, the air mouse needs at least one infrared light source as a positioning mark to help the air mouse determine its position in the air at any time. The infrared light source may be various objects capable of providing infrared light, including various infrared light emitters, candles and the like.

When in use, the OV7620 image sensor module shoots through the lens 202, and continuously outputs the formed image in the form of a digital signal to the processing unit (STM32F407). The output image contains the image formed by the positioning mark (infrared light source), which is called positioning Logo image. The image with the positioning mark image is called the positioning image. The processing unit performs image processing on the positioning image, and sends a control signal to the controlled device according to the image processing result.

The OV7620 image sensor module is set to output 50 frames of grayscale images with a resolution of 320*240 per second. Gray image is a kind of digital image. In each frame of gray image, the information of a 320*240 pixel matrix is recorded. The pixel matrix has 240 rows and 320 columns, from top to bottom from row 0 to row 239, and from left to right from column 0 to column 319. The position of each pixel in the matrix is marked by a unique coordinate. The first pixel in the upper left corner has coordinates (0, 0). Indicates that it is located in row 0 and column 0, where the first 0 is the row coordinate, indicating the number of rows it is in; the second 0 is the column coordinate, indicating the number of columns it is in. Each pixel in the pixel matrix has a pixel value, represented by a number from 0 to 255. Among them, 255 represents white, 0 represents black, and the remaining numbers represent varying degrees of gray.

The air mouse and two positioning marks form an air mouse system, which has better performance. The two positioning marks have different characteristics, and the characteristics can be recorded in the positioning image.

In this embodiment, the image acquisition unit may also be an infrared image sensor, as long as it can output digital images.

This embodiment also provides an infrared sensor strip, which is used to provide a reference for the displacement of the air mouse of this embodiment in the air.

As shown in FIG. 2, the infrared sensor strip includes a bracket 300 and an infrared light source A301 arranged at the left end of the front of the bracket 300, and an infrared light source B302 arranged at the right end of the front of the bracket 300. Preferably, the bracket 300 is rectangular, with a length of 25 cm, a height of 2 cm, and a thickness of 2 cm. The distance between the two infrared light sources is 20 cm. Preferably, the two infrared light sources are infrared light-emitting diodes and are powered by an external power source. The bracket 300 may also be hollow, with a battery compartment for installing batteries to supply power to the infrared light source. The two infrared light sources on the bracket 300 serve as two positioning marks, and together with the air mouse form an air mouse control system. The infrared light source A301 is the positioning mark A, and the infrared light source B302 is the positioning mark B. When in use, place the infrared sensor strip in front of the user at a distance of about 50cm.

Two infrared light sources emit infrared light of different brightness. For example, the luminous flux of the infrared light source A301 is 5 lumens; the luminous flux of the infrared light source B302 is 10 lumens. The positioning mark image formed by the infrared light source in the positioning image has different image characteristics, which is reflected in different pixel values. For example, in this embodiment, the image formed by the infrared light source A301 in the positioning image has a pixel value of 50 for each pixel constituting the image; the image formed by the infrared light source B302 in the positioning image has the value of each pixel constituting the image The pixel value is 100.

Because in the positioning image, only the pixel value of the constituent pixels of the positioning mark image is above 50, so it can be determined whether the positioning mark image is found based on this information. In this embodiment, when the processing unit searches for the positioning mark image in the pixel matrix, it finds a pixel with a pixel value above 40, that is, it is determined that the positioning mark image is found.

The processing unit of the air mouse performs image processing on the positioning image output by the image acquisition unit, and its purpose is to obtain the feature code and image position data of the positioning mark image. The specific method in this embodiment is:

Read the pixel value of each pixel in the positioning image one by one, and record the pixel value and coordinates of the pixel when the first pixel value is 40 or more. The pixel value is a feature code, and the coordinates are image location data corresponding to the feature code. For example, when reading a pixel with coordinates (0, 5), it is found that the pixel value is 50, and 50 and (0, 5) are recorded. Wherein 50 is the feature code, (0, 5) is the image location data corresponding to the feature code. The image position data is actually the coordinates of one of the constituent pixels of a positioning mark image in the positioning image, and is used to represent the position of the positioning mark image in the positioning image.

Each time a frame of positioning image is processed, the processing unit sends a set of image data to the connected PC, including feature codes and image location data.

This embodiment also provides a driver program, used on the controlled device of the air mouse (PC in this embodiment), for controlling the movement of the cursor on the driven screen according to the image data.

When the driver receives a group of image data, the processing method adopted is:

According to the feature code, find the same in the last group of image data received. If the same is found, the image position data corresponding to the feature code in the two sets of image data is compared to obtain the image displacement data, and the cursor movement on the screen is controlled according to the image displacement data; if the same is not found, The processing of this group of image data ends. The processing method is shown in Figure 3.

The composition and signal transmission mode of the air mouse and the air mouse control system in this embodiment are shown in FIG. 4.

Taking the horizontal movement of the air mouse as an example, the air mouse and the image processing method of this embodiment will be described in detail below.

The driver on the PC side presets a recording area to record the image data (including feature code and image position data) from the air mouse, and update the data in the recording area at the end of each group of image data processing .

The user holds the air mouse and points the lens 202 toward the positioning mark A (infrared light source A301). The image acquisition unit starts to take and output the first frame of positioning image. In this frame of positioning image, the four pixels with coordinates (120,318), (120,319), (121,318), (121,319) The value is 50, and the pixel value of the remaining pixels is 0. The image composed of these four pixels is formed by the infrared light source A and is called the positioning mark A image.

This frame positioning image is transmitted to the processing unit in the form of a digital signal, and the processing unit performs the following image processing:

Read the pixel value of each pixel in the pixel matrix one by one in the order from left to right and top to bottom. The specific method is to start from the pixel with coordinates (0, 0), first read the pixel value of each pixel in the first row of pixels in the pixel matrix in the order from left to right, and then the second row, third row, until the last line.

According to the above sequence, when the pixel with coordinates (120, 318) is read, the read pixel value is 50. Record 50 and (120, 318) (because the pixel value of the pixel is greater than 40), where 50 is the feature code, and (120, 318) is the image location data corresponding to the feature code.

The processing unit sends 50 and (120, 318) to the connected PC, and the driver on the PC side searches for the same in the recording area according to the feature code 50. Because this is the first set of image data processed, there is no record in the recording area. After storing 50 and (120, 318) in the recording area, the entire image processing process ends.

The user holds the air mouse and continues to move to the right. In the second frame of the positioning image output by the image acquisition unit, the coordinates are (120,317), (120,318), (121,317), (121,318). The pixel value of the pixel is 50, and the pixel value of the remaining pixels is 0.

The image data obtained by the processing unit from this frame positioning image is 50 and (120, 317).

The driver found the same in the recording area according to the feature code 50. Comparing (120,317) with (120,318), it is found that the value of the column coordinate is reduced by 1, and the driver obtains the image displacement data. The cursor on the LCD screen driven by the PC is moved 1 pixel to the right (the screen faces the user).

The user holds the air mouse and continues to move to the right. The image acquisition unit continues to capture and output positioning images. In the output series of positioning images, the position of the positioning mark A image gradually moves to the left, and the processing unit continues to output image data according to the image processing results , The PC controls the cursor on the LCD screen to move continuously to the right according to the image data.

When the image acquisition unit outputs the nth frame of the positioning image, the processing unit obtains the feature code 50 and the corresponding image position data (120, 0) from the frame of the positioning image. At this time, the positioning mark A has reached the visible range of the lens 202 edge. After the frame positioning image processing is completed, the data in the recording area of the driver is 50, (120, 0).

After the user holds the air mouse and moves to the right for a certain distance, there is only the positioning mark B in the lens 202. At this time, the image acquisition unit outputs a positioning image of the n+1th frame, in which four pixels have a pixel value of 100, and the remaining pixels have a pixel value of 0. The four pixels constitute the positioning mark B image, and the coordinates of the four pixels are (120, 317), (120, 318), (121, 317), (121, 318), respectively.

The processing unit obtains the feature code 100 and the corresponding image position data (120, 317) in the frame positioning image. The driver on the PC side does not find the same in the recording area according to the feature code 100. After updating the data in the recording area to 100, (120, 317). The processing of this group of image data ends.

After the user holds the air mouse and continues to move a small distance to the right, in a frame of positioning image output by the image acquisition unit, 4 pixels have a pixel value of 100 and the remaining pixels have a pixel value of 0. The coordinates of the 4 pixels are (120, 316), (120, 317), (121, 316), (121, 317), respectively. After the processing unit performs image processing on the frame positioning image, the image data sent to the PC is 100 and (120, 316). According to the feature code 100, the driver finds the same in the recording area, and changes (120, 316) Comparing with (120, 317), the obtained image displacement data is that the value of the column coordinate is reduced by 1, and the cursor on the LCD screen driven by the PC control moves 1 pixel to the right. The user holds the air mouse and continues to move to the right, and the cursor on the screen also continues to move to the right, until the positioning mark B moves out of the visible range of the lens 202.

In the air mouse of this embodiment, there is a positioning mark within the visual range of the lens 202 to control the cursor. The length of the movement can be increased by adding positioning marks, and how much the movement length increases depends on the distance between the two positioning marks. A more reasonable positioning mark placement distance is that when the air mouse is in the middle of the two positioning marks, both positioning marks are within the visible range of the lens 202, but both are located at the edge, so that the cursor can be guaranteed during the horizontal movement. The continuity of control can also have the longest movement length. At this time, adding 1 positioning mark can increase the movement length close to 320 pixels.

The beneficial effect of the air mouse in this embodiment comes from the image processing method used. In the process of moving from left to right, the air mouse uses positioning mark A and positioning mark B as reference objects to determine its own position in the air. When the reference object is switched from positioning mark A to positioning mark B, the cursor on the screen has a very short pause, and then continues to move to the right. If it is an air mouse in the prior art, it simply compares the positions of the positioning mark images in the two frames of the positioning image before and after. When the reference object is switched, the cursor on the screen will move in the opposite direction to the air mouse. For example, when positioning mark A is used as the reference object, the image position data of the last frame of positioning image generated is (120, 0); after moving slightly to the right, the first frame of positioning image generated using positioning mark B as the reference object, The image position data is (120, 317). After comparing the two image position data and outputting a displacement signal, the cursor on the screen will move 317 pixels to the left.

When the air mouse of this embodiment performs image processing on each frame of positioning image, on the basis of obtaining the image position data, it further obtains a feature code according to the image feature. The driver on the PC side obtains the necessary information, and only compares the position of the same positioning mark in the two positioning images, making it possible to increase the moving length by adding positioning marks.

When the driver compares the image position data, it needs to compare the row coordinates and column coordinates separately to find out the changes of the values. There are four types of changes: row coordinates increase, row coordinates decrease, column coordinates increase, and column coordinates decrease. small. The driver of this embodiment controls the cursor movement on the screen according to the image displacement data obtained after the image position data comparison. The rule adopted is: if the value of the row coordinate increases by n, the cursor on the screen is controlled to move up n pixels ; If the value of the row coordinate decreases by n, the cursor on the control screen moves down by n pixels. If the value of the column coordinate increases by n, the cursor on the control screen moves n pixels to the left; if the value of the column coordinate decreases by n, the cursor on the control screen moves n pixels to the right; if the row coordinate and column If the values of the coordinates have changed, control the cursor to move correspondingly in two directions. For example, if the measured values of row coordinates and column coordinates are increased by 1, the cursor on the control screen will move up by 1 pixel, and then move to the left by 1 pixel.

Using the above rules can ensure that the moving direction of the cursor on the screen is consistent with the moving direction of the air mouse. According to needs, the rule can also be changed: in the obtained image displacement data, if the value of the column coordinate increases, the cursor is controlled to move to the left; the value of the column coordinate decreases, and the cursor is controlled to move to the right. In this way, when moving horizontally, the moving direction of the cursor on the screen is opposite to that of the air mouse. The direction of longitudinal movement can also be modified in the same way.

In an implementation of the driver program of this embodiment, the driver program multiplies the image displacement data by 2 and outputs it. For example, if the value of the measured column coordinates is reduced by 1, the cursor on the screen is moved 2 pixels to the right . In this way, the air mouse can control the cursor on the screen to move a longer distance with a shorter moving distance.

After multiplying the image displacement data by 3, the air mouse system consisting of an air mouse and two positioning marks can basically meet the needs of controlling mainstream resolution screens. At this time, the horizontal movement length of the air mouse can be close to 1920, and the movement accuracy is 3. If you want to increase the moving length of the air mouse in the vertical direction, place the infrared sensor strip upright so that the two infrared light sources are arranged up and down.

In addition to panning, turning the air mouse in the air can also control the cursor movement on the screen, turning left or right can control the cursor on the screen to move horizontally; turning up or down can control the cursor on the screen to move vertically . Because the rotation can also change the position of the positioning mark image in the positioning image output by the image acquisition unit.

In the present invention, the distance between the air mouse and the infrared light source is more suitable to be 50cm to 100cm. Too close will make the image of the positioning mark in the image determined to be too large and the image to be positioned.

In an implementation of the infrared sensor strip of this embodiment, the length of the rectangular bracket is 45 cm, and three infrared light sources are arranged on the front of the bracket from left to right. The 3 infrared light sources are arranged in a row, and the distance between each light source is 20cm. The brightness of the infrared light source on the left and right sides is the same, while the brightness of the light source in the middle is different. For example, the luminous flux of the infrared light source on the left and right sides is 5 lumens, and the luminous flux of the infrared light source in the middle is 10 lumens. The 3 infrared light sources are equivalent to 3 positioning marks, and form an air mouse system with the air mouse, which can further increase the moving length of the air mouse.

In an implementation of the infrared sensor strip of this embodiment, the bracket 300 is rectangular, 25 cm in length and 8 cm in height. There are 4 infrared light sources with different brightness in 4 corners of the bracket, as shown in Figure 5. Preferably, the luminous fluxes of the four infrared light sources are 5 lumens, 10 lumens, 15 lumens, and 20 lumens, respectively. It is equivalent to providing 4 positioning marks for the air mouse, and the positioning mark images formed by any two adjacent positioning marks in the positioning image have different characteristics (pixel values), compared to only one positioning mark in the prior art After the air mouse system is composed of the 4 infrared light sources and the air mouse, the air mouse can be moved horizontally, vertically and diagonally in the air, and the moving length is increased. The bracket 300 in this embodiment may also have a frame-shaped structure, with one infrared light source at each of the four corners, and four infrared light sources with four kinds of brightness. As shown in Figure 6. Preferably, the width of each side of the frame is 2CM.

In an implementation of the air mouse of this embodiment, the processing unit uses the following image processing method for the positioning image:

S1. Read the pixel values of all pixels in the positioning image, and use all pixels with a pixel value greater than 40 as candidate pixels.

S2. From all candidate pixels, select a pixel with the smallest coordinate value, record the pixel value of this pixel as a feature code, and record the coordinate as image position data. The method of selecting the pixel with the smallest coordinate value is to select the pixel with the smallest row coordinate value among all the candidate pixels. If there is only one, then the pixel is the pixel with the smallest coordinate value; if there are more than one, select the column among the multiple The smallest coordinate value.

S3. Remove all pixels with the same pixel value from the smallest pixel with the same coordinate value and the smallest pixel with the same coordinate value from the candidate pixels. If there are still candidate pixels, continue processing from S2; if not, process the positioning image in this frame The process is over.

After processing one frame of positioning image according to the image processing method, the processing unit can obtain the image data of all the positioning mark images. The image processing method is shown in Figure 7. After the image processing of each frame of the positioning image is completed, the processing unit sends all the acquired image codes and image positions as a set of image data to the connected PC.

At this time, each group of image data received by the driver on the PC side may have one or more feature codes and corresponding image position data. In the case of more than one feature code, extract one of the feature codes randomly, search for the same in the recording area, and find the same, then the image position data corresponding to the feature code corresponds to the feature code in the recording area The position data of the image is compared, and the cursor on the screen is controlled to move according to the result of the comparison; if the same is not found, another feature code is extracted and the search continues until the same is found or all feature codes have been searched.

The following examples illustrate the image processing method and driver.

When the image acquisition unit captures infrared light sources A and B at the same time, it outputs the first frame positioning image. There are 4 pixels with a pixel value of 50, the coordinates are (120, 0), (120, 1), (121, 0), (121, 1); there are 4 pixels with a pixel value of 100, the coordinates They are (120,318), (120,319), (121,318), (121,319).

The processing unit performs the following image processing on this frame positioning image:

S1. Read the pixel values of all pixels in the positioning image, and find a total of 8 spare pixels. The coordinates are (120, 0), (120, 1), (121, 0), (121, 1) (120, 318), (120, 319), (121, 318), (121, 319), respectively.

S2. Select a pixel with the smallest coordinate value among the candidate pixels, and its coordinate is (120, 0). The feature code 50 and the image position data (120, 0) are recorded in the image data.

S3. After removing this pixel and all pixels with a pixel value of 50 from the candidate pixels, there are still 4 candidate pixels. The pixel values of the 4 candidate pixels are 100, and the coordinates are (120, 318 ), (120,319), (121,318), (121,319).

Going back to S2, select a pixel with the smallest coordinate value from the remaining 4 candidate pixels, whose coordinates are (120, 318), and record the feature code 100 and pixel position data (120, 318) into the image data.

After removing this pixel and all pixels with a pixel value of 100 from the candidate pixels, there are no candidate pixels, and the image processing process of the positioning image of this frame ends. The processing unit sends the first group of image data to the connected PC, including feature code 50 and corresponding image location data (120, 0), and feature code 100 and corresponding image location data (120, 318).

After the PC driver receives the group of image data, it first searches the recording area according to the feature code 50. Did not find the same; then search in the recording area according to feature code 100. If the same is not found, the process ends, and the group of image data is recorded in the recording area.

The image acquisition unit sends out the second frame positioning image, and the processing unit sends the second group of image data to the PC after image processing. There are feature code 50 and corresponding image displacement data (120, 1), and feature code 100 and corresponding image displacement data (120, 319). The driver finds the same in the recording area according to the feature code 50, compares (120, 1) with (120, 0), and finds that the value of the column coordinate has increased by 1. Then the cursor on the control screen moves 1 pixel to the left to end the processing process, and at the same time change the data in the recording area to the second group of image data.

In an implementation of the air mouse of this embodiment, the image acquisition unit can capture visible light images, for example, the lens matched with the image sensor module is changed to a lens that can pass visible light. The visible light source can also be used as a positioning mark. For example, light-emitting diodes can be used as positioning marks.

Including the following methods, using two light-emitting diodes as positioning signs, one of which has a luminous flux of 5 lumens and the other has a luminous flux of 10 lumens.

Use 3 light-emitting diodes as positioning marks, one of which has a luminous flux of 5 lumens and the other two has a luminous flux of 10 lumens. When in use, put out a row with 5 lumens in the middle.

Use 4 light-emitting diodes with different brightnesses as positioning marks, and place them according to the placement method in Figure 6. For example, place 4 light-emitting diodes on the 4 corners of a monitor. When using, the air mouse is aligned with the monitor OK.

The image processing unit of the air mouse of the present invention can process higher-resolution images. For example, the OV7620 image sensor outputs a black and white digital image with a resolution of 640×480 at 30 frames per second. Processing the positioning image of this resolution can make the air mouse have a higher moving length without changing the moving accuracy.

In an implementation of the air mouse of this embodiment, it further includes a cursor movement control key, which is arranged on the surface of the housing and is connected to the processing unit through a circuit. It is used to control whether the cursor on the screen moves with the air mouse. For example, the user often needs to move the air mouse to the middle of the effective area and does not want the cursor on the screen to follow the movement. At this time, when the cursor movement control key is pressed, the processing unit stops outputting the cursor control signal, and when the key is released, the processing unit resumes outputting the cursor control signal. Preferably, the key is arranged at a position suitable for thumb pressing on the housing.

In this manual, the cursor controlled by the air mouse refers to various icons that move with the movement of the air mouse, such as the mouse pointer on the screen, the weapon crosshairs in electronic games, and characters. The cursor can be of different sizes, and even include the entire screen. For example, using an air mouse to control the field of view of a character in a game, the entire screen may be the field of view. When the air mouse moves, the display of the entire screen will change.

The air mouse and the controlled device in this embodiment jointly constitute an air mouse control system. The controlled devices are various computer devices, including but not limited to PCs, video game consoles, smart TVs, set-top boxes, tablet computers, smart phones, and VR devices. The air mouse outputs image data (including code features and image position data), and the controlled device controls the cursor on the display device screen driven by the image data to move.

When the air mouse of the present invention is used to control a smart phone or a tablet computer, the processing unit outputs image data, and the smart phone or tablet computer controls the cursor movement on the screen according to the image data.

Example two

This embodiment provides an air mouse, which is different from the air mouse in the first embodiment in the method of acquiring image data. The processing unit of the air mouse in this embodiment adopts the following processing methods for the positioning image:

Read the pixel value of each pixel one by one in the preselected area until a pixel with a pixel value above 40 is found. Then take this pixel as the center, read the pixel values of all pixels in the range of 7 pixels×7 pixels, and find out the pixels with all the pixel values above 40.

For example, if the first pixel with a pixel value of 40 or more is found and its coordinates are (X, Y), the pixel values of all pixels whose coordinates meet the following requirements are read: the row coordinates are from X-3 to X+3, and The column coordinates are from Y-3 to Y+3. The purpose of this step is to find all the pixels that make up the positioning mark image.

Then the feature code is determined according to the number of all pixels found. For example, if there are 4 pixels with a pixel value above 40, record the feature code as 4; if there are 9 pixels with a pixel value above 40, record the feature code as 9.

Then, among all the pixels with pixel values above 40, find the pixel with the smallest coordinate value, and record the coordinate of this pixel as image position data. The method for determining the pixel with the smallest coordinate value is: among all the pixels with coordinate values above 40, select the pixel with the smallest row coordinate value. If there is only one pixel, the pixel is the pixel with the smallest coordinate value. If there are more than one, the pixel with the smallest column coordinate value is selected from the multiple as the pixel with the smallest coordinate value.

The preselected area mentioned above refers to an area divided in the positioning image. The method of dividing the positioning area is: in a matrix composed of 320×240 pixels, the first 3 rows, the last 3 rows, and the first 3 columns After removing the pixels in the last three columns, the remaining pixels constitute the preselected area of the positioning image. That is, all the pixels whose row coordinates are 3 to 236 and column coordinates 3 to 316 constitute the preselected area.

The preselected regions involved in the embodiments in this specification are all within the same selection range. In this embodiment, the positioning marks with different characteristics and the positioning mark images formed in the positioning images have different numbers of constituent pixels, and the number of constituent pixels reflects the image characteristics of the positioning mark images. If the image of the light source is incomplete, it may affect the determination of the image feature, which may cause an error in the feature code obtained from the image feature. And for the positioning mark image found in the preselected area, all its constituent pixels must be in the entire 320×240 pixel matrix. This is the purpose of setting the preselected area.

If the positioning mark image in the positioning image is larger, there are more pixels. The preselected area can be re-divided. For example, after removing the pixels in the first 5 rows, the last 5 rows, the first 5 columns, and the last 5 columns in the pixel matrix, the remaining pixel set is used as the preselected area.

This embodiment also provides an infrared sensor strip, which includes a bracket 300 and two infrared light sources. As shown in FIG. 8, the bracket 300 is rectangular, 25 cm long, 2 cm high, and 2 cm thick. There is a square hole on the left and right ends of the front, and the side length is 2 mm and 3 mm respectively. There is an infrared light-emitting diode in each hole, and the brightness of the infrared light-emitting diode is the same. The two holes that can emit infrared light are infrared light sources. The hole with a side length of 2 mm has a light-emitting area of 4 square millimeters, and the hole with a side length of 3 mm has a light-emitting area of 9 square millimeters. The inside of the bracket 300 may be hollow, which is convenient for accommodating the infrared light-emitting diodes.

These two infrared light sources can be used as two positioning marks to form an air mouse system together with the air mouse of this embodiment. The images formed by the two infrared light sources in the positioning image have different composition numbers. For example, when the distance between the air mouse and the infrared light source is about 1M, the image formed by the infrared light source with a light-emitting area of 4 square millimeters is composed of 4 pixels; the image formed by the infrared light source with a light-emitting area of 9 square millimeters is composed of 9 pixels. The processing unit determines the feature code according to the number of pixels in the positioning mark image.

The method for obtaining feature codes and image position data by the air mouse of this embodiment will be specifically described below.

In one frame of positioning image output by the image acquisition unit, 4 pixels have pixel values of 100, and the coordinates are (120, 3), (120, 4), (121, 3), (121, 4), respectively.

The processing unit first reads the pixel value of each pixel in the preselected area in the order from left to right and top to bottom. When it reads the pixel with coordinates (120, 3), it reads the first one. Pixels with pixel values above 40.

With this pixel as the center, the pixel values of 49 pixels in the surrounding 7 pixels×7 pixels range are read. That is, the row coordinates are 117 to 123, and the column coordinates are the pixel values of all pixels in the range of 0 to 6. It is found that the pixel value of 4 pixels is 100, and the coordinates are (120, 3), (120, 4), (121, 3), (121, 4).

Because there are 4 pixels with a pixel value above 40, the feature code is recorded as 4. At the same time, the image position data is recorded as (120, 3) according to the coordinates of the pixel with the smallest coordinate value.

When the image processing ends, the processing unit sends the obtained feature code and corresponding image position data to the controlled device.

Another implementation of the infrared sensor strip of this embodiment is shown in FIG. 9, a longitudinal rectangular hole is opened on the left end of the front of the bracket 300. There are two longitudinal rectangular holes at the right end, and the two rectangular holes are 1 mm apart. The height of the three rectangular holes is 4 mm, and the width is 1 mm. An infrared light emitting diode with the same brightness is arranged behind each rectangular hole, and the two rectangular holes at the right end can share one infrared light emitting diode. The three holes can be used as two positioning marks, wherein the hole at the left end serves as a positioning mark, and the two holes at the right end together serve as a positioning mark. The images formed by the two positioning marks in the positioning image are composed of different numbers of pixels.

In an implementation of the infrared sensor strip of this embodiment, it includes a bracket and three infrared light sources. The bracket is rectangular, 45cm long, 2cm high, and 2cm thick. There are 3 square holes on the front of the bracket, one at each end near the end and one at the middle. The square holes at both ends have a side length of 2 mm; the middle hole has a side length of 3 mm. There are 3 infrared light-emitting diodes with the same brightness behind the 3 holes. Preferably, the luminous flux of the three infrared light-emitting diodes is 10 lumens. The three square holes that can emit infrared light are three infrared light sources, which can be used as three positioning marks and an air mouse to form an air mouse system. Compared with the solution of two positioning marks, it can further increase the moving length of the air mouse.

In an implementation of the infrared sensor strip of this embodiment, it includes a bracket and 4 infrared light sources. Preferably, the bracket is rectangular with a length of 25 cm and a height of 8 cm. A square hole is opened at each of the four corners of the rectangular bracket. Each hole has a different side length. The hole in the upper left corner has a side length of 1 mm, the hole in the lower left corner has a side length of 2 mm, the hole in the upper right corner has a side length of 3 mm, and the hole in the lower right corner has a side length of 4 mm. There is an infrared light-emitting diode behind each hole. Preferably, the luminous flux of the 4 infrared light-emitting diodes is 10 lumens. The four holes that can emit infrared light are the infrared light sources, which can be used as four positioning marks to form an air mouse system with an air mouse. Compared with the solution of 2 positioning marks, the air mouse can increase the moving length in the horizontal, vertical and diagonal directions at the same time. In the infrared sensor strip of this embodiment, as shown in FIG. 10, four infrared light sources can be arranged on a rectangular frame, which can reduce the weight and increase the aesthetics.

In an implementation of this embodiment, when processing a frame of positioning image, the processing unit obtains the feature codes and image position data of all positioning mark images therein. The image processing methods used are:

S1. Read the pixel values of all pixels in the preselected area, and find all pixels with a pixel value above 40 as candidate pixels.

S2. Select a pixel with the smallest coordinate value from the candidate pixels, take this pixel as the center, read the pixel values of all the pixels in the surrounding 7 pixels × 7 pixels, and find all the pixels with pixel values above 40. These pixels Called component pixels.

S3. Record the total number of constituent pixels as a feature code; then, among constituent pixels, find a pixel with the smallest coordinate value, and record the coordinate of the pixel as the image position data corresponding to the feature code.

S4. Remove the constituent pixels from the candidate pixels. If there are still candidate pixels, start processing from the second step; if there are no candidate pixels, the image processing process ends.

The following example illustrates the image processing method.

In the 1 frame of positioning image output by the image acquisition unit, there are a total of 5 pixels with a pixel value of 100, and their coordinates are (5, 6), (10, 12), (10, 13), (11, 12), ( 11, 13). The pixel value of the remaining pixels is 0. Among them, the pixel (5, 6) is the location mark A image, and the remaining pixels form the location mark B image. The processing unit performs the following image processing:

S1. Read the pixel values of all pixels in the preselected area, and find 5 pixels with a pixel value above 40 as candidate pixels. The coordinates are (5, 6), (10, 12), (10, 13), ( 11, 12), (11, 13).

S2. Find the pixel with the smallest coordinate value (5, 6). After reading the pixel values of all pixels in the range of 7 pixels×7 pixels around the pixel, the found component pixel is (5, 6).

S3. The constituent pixel is 1, so the feature code is recorded as 1; the image position data is recorded as (5, 6).

S4. After removing the constituent pixels (5, 6), candidate pixels (10, 12), (10, 13), (11, 12), (11, 13) are left.

Return to S2, find the pixel with the smallest coordinate value (10, 12), read the pixel values of all pixels in the range of 7 pixels × 7 pixels around the pixel, and find the constituent pixels (10, 12), (10, 13), ( 11, 12), (11, 13). The feature code is recorded as 4, and the image position data is recorded as (10, 12). After removing the constituent pixels from the candidate pixels, there are no remaining candidate pixels.

The image processing process ends, and the acquired image data are: feature code 1 and corresponding image position data (5, 6);

Feature code and corresponding image location data (10, 12).

In the infrared sensor strips in this embodiment, the infrared light-emitting diodes are replaced with light-emitting diodes that emit visible light. It can also provide positioning marks for the air mouse. At this time, the image acquisition unit of the air mouse is an image acquisition unit that can shoot visible light. Preferably, the luminous flux of the light emitting diode is 10 lumens.

Example three

This embodiment provides an air mouse, which is different from the air mouse in the first embodiment in the method of acquiring image data. The processing unit of the air mouse in this embodiment adopts the following image processing method for the positioning image:

S1. In the preselected area, read the pixel value of each pixel in the order from left to right and top to bottom. Until you find a pixel with a pixel value above 40.

S2. With this pixel as the center, read the pixel values of a total of 49 pixels in the 7×7 range, and find all the pixels with pixel values above 40. If the column coordinates of all the pixels found are the same, record the feature code as 1; if the row coordinates are the same, record the feature code as 2. Then, among all the found pixels, the pixel with the smallest coordinate value is found, and the coordinate of the pixel is recorded as the image position record corresponding to the feature code.

This embodiment provides an infrared sensor strip. The infrared sensor strip is shown in Figure 11 and includes a bracket and two infrared light sources. The bracket 300 is rectangular, with a rectangular hole at each end of the front surface. The length of the two rectangular holes is 4 mm, and the width is 1 mm. The left end is a longitudinal hole, and the right end is a horizontal hole. There is an infrared light-emitting diode behind each hole. Preferably, the luminous flux of the two infrared light-emitting diodes is 10 lumens. The two rectangular holes that can emit infrared light are two infrared light sources. It can be used as two positioning marks to form an air mouse system with the air mouse of this embodiment.

The following example illustrates the image processing method of the air mouse in this embodiment:

In the 1 frame of positioning image output by the image acquisition unit, the 4 pixels with coordinates (119,316), (120,316), (121,316), (122,316) have a pixel value of 100, and the pixels of the remaining pixels The value is 0. The processing unit uses the following methods to obtain feature codes and image location data.

S1. In the preselected area, read the pixel value of each pixel one by one in the order from left to right and top to bottom. When the pixel with coordinates (119, 316) is read, the first pixel value is found More than 40 pixels.

S2. With this pixel as the center, read the pixel values of 49 pixels in the surrounding 7 pixels×7 pixels range. Read row coordinates

It is 116 to 122, and the column coordinates are the pixel values of all pixels in the range of 313 to 319. The result is that there are 4 pixels with pixel values above 40, and the coordinates are (119, 316), (120, 316), (121, 316), (122, 316).

Because the column coordinates of the 4 pixels found are the same, the feature code is recorded as 1. Among the 4 pixels found, the coordinate of the pixel with the smallest coordinate value is (119, 316), which is the image position data corresponding to feature code 1.

The above-mentioned processing method obtains the feature code and image position data of one positioning mark image in one positioning image. In an implementation of this embodiment, when the processing unit processes one frame of positioning image, it acquires the feature codes and image position data of all the positioning mark images therein. The specific method is as follows:

S2. Select a pixel with the smallest coordinate value from the candidate pixels, take this pixel as the center, read the pixel values of all pixels in the surrounding 7 pixels × 7 pixels, and find all the pixels with pixel values above 20. These pixels Called component pixels.

S3. If the column coordinates of all the constituent pixels found are the same, the image code is recorded as 1; if the row coordinates of all the constituent pixels are the same, the image code is recorded as 2. Then, among all the constituent pixels, find a pixel with the smallest coordinate value, and record the coordinate of this pixel as an image position.

S4. Among the candidate pixels, remove the pixels with the same coordinates as the constituent pixels (there are a few count as several), if there are still candidate pixels, start processing from the second step; if there are no candidate pixels, Then the step of acquiring image data ends.

In this embodiment, the image processing method in the step of acquiring image data is shown in FIG. 12.

The following examples illustrate the processing steps.

In the 1 frame of positioning image output by the image acquisition unit, there are 8 pixels with a pixel value of 100. The coordinates are (119, 3), (120, 3), (121, 3), (122, 3), (120). , 315), (120,316), (120,317), (120,318). The first 4 pixels form the location mark A image, and the last 4 pixels form the location mark B image.

The processing unit uses the following methods to obtain feature codes and image location data:

S1. Read the pixel values of all pixels in the preselected area, and find 6 pixels with a pixel value above 40 as candidate pixels. The coordinates are (119, 3), (120, 3), (121, 3), (122,3), (120,315), (120,316).

S2. Select a pixel with the smallest coordinate value among the candidate pixels, and its coordinate is (119, 3). With this pixel as the center, read the pixel values of all pixels in the surrounding 7 pixels × 7 pixels range, and find 4 pixels with a pixel value of 40 or more as constituent pixels (the 4 pixels are all the pixels that make up the image of the positioning mark A) , The coordinates are (119, 3), (120, 3), (121, 3), (122, 3).

S3.4 The column coordinates of the constituent pixels are the same, and the feature code is recorded as 1. Find a pixel with the smallest coordinate value among the 4 constituent pixels, and its coordinate is (119, 3). Record the coordinates as the image position data corresponding to feature code 1.

S4. In the candidate pixels, after removing the constituent pixels. There are two candidate pixels left, the coordinates are (120, 315), (120, 316), so proceed as follows:

Among the candidate pixels, the pixel with the smallest coordinate value is selected, and its coordinate is (120,315). With this pixel as the center, read the pixel values of all pixels in the surrounding 7 pixels × 7 pixels range, and find 4 pixels with a pixel value of 20 or more as the constituent pixels (the 4 pixels are all the pixels that make up the positioning mark B image) , The coordinates are (120,315), (120,316), (120,317), (120,318).

The row coordinates of the 4 constituent pixels are the same, and the feature code is recorded as 2. Find a pixel with the smallest coordinate value among the 4 constituent pixels, and its coordinate is (120,315), and record this coordinate as the corresponding feature code 2 Image location record.

In the 2 candidate pixels, after removing the constituent pixels, there are no remaining candidate pixels, the image processing process. The acquired image data includes feature code 1 and corresponding image location data (119, 3); feature code 2 and corresponding image location data (120, 315).

In an implementation of the infrared sensor strip of this embodiment, it includes a bracket and three infrared light sources. As shown in Figure 13, the bracket is rectangular, 45 cm long, 2 cm high, and 2 cm thick. There are 3 rectangular holes on the front of the rectangular bracket, one at each end near the end and one at the middle. The length of the three rectangular holes is 4 mm, and the width is 1 mm. The rectangular holes at both ends are vertical; the middle rectangular hole is horizontal. The distance between the holes at both ends and the middle hole is 20cm. There are 3 infrared light-emitting diodes with the same brightness behind the 3 holes. Preferably, the luminous flux of the three infrared light-emitting diodes is 10 lumens. The three rectangular holes capable of emitting infrared light are three infrared light sources, which can be used as three positioning marks to form an air mouse system with the air mouse of this embodiment. Compared with the solution of two positioning marks, it can further increase the moving length of the air mouse.

In an implementation of the infrared sensor strip of this embodiment, it includes a bracket and multiple infrared light sources. As shown in Fig. 14, the bracket 300 is rectangular, 25 cm long and 8 cm high. There is a longitudinal rectangular hole in the upper left corner with a length of 4 mm and a width of 1 mm; the upper right corner has a horizontal rectangular hole with a length of 4 mm and a width of 1 mm; the lower left corner has two longitudinal rectangular holes, the two rectangular holes Arranged in parallel, with a distance of 1 mm, a length of 2 mm, and a width of 1 mm; there are two horizontal rectangular holes in the lower right corner, the two rectangular holes are arranged in parallel, with a distance of 1 mm, the length is 2 mm, and the width is both It is 1 mm. There are light-emitting diodes behind each hole. Preferably, the luminous flux of the infrared light-emitting diode is 10 lumens. The rectangular hole on the bracket that can emit infrared light can be used as a positioning mark. The vertical rectangular hole in the upper left corner is used as one; the horizontal rectangular hole in the upper right corner is used as one; the two longitudinal rectangular holes in the lower left corner are used as one; the two in the lower right corner are used as one. As one horizontal rectangular hole, there are 4 positioning marks in total.

When the infrared sensor bar provides positioning marks for the air mouse, in the positioning image output by the image acquisition unit, the image of each positioning mark is composed of 4 pixels, and the arrangement methods are different. Among them, the image formed by the positioning mark in the upper left corner is composed of 4 pixels vertically arranged in a row; the image formed by the positioning mark in the upper right corner is composed of 4 pixels horizontally arranged in a row; the positioning mark in the lower left corner is composed of 4 pixels Arranged in two rows longitudinally, each row has 2 pixels, the distance between the two rows is 1 pixel; the positioning mark at the lower right corner has an image of 4 pixels arranged horizontally in two rows, each row has 2 pixels, and the distance between the two rows The distance is 2 pixels; for example, the positioning mark in the lower left corner, the coordinates of the 4 pixels that make up the image are (0, 0), (0, 2), (1, 0), (1, 2); the lower right corner The positioning mark of, the coordinates of the 4 pixels that make up the image are (0, 0), (0, 1), (2, 0), (2, 1). The rule for the processing unit to determine the feature code is that if the column coordinates of all pixels found with the pixel value of 40 are the same, the feature code is recorded as 1; if the row coordinates are the same, the feature code is recorded as 2; if the column coordinates are the same in pairs , And two pixels with the same column coordinate, the row coordinate value is adjacent, then the feature code is recorded as 3; if the column coordinate is the same two by two, and the column coordinate is the same two pixels, the row coordinate value If it is not a contiguous number, record the feature code as 4.

In the several infrared sensor strips in this embodiment, the infrared light-emitting diodes are replaced with light-emitting diodes, which can also provide positioning marks for the air mouse. At this time, the image acquisition unit of the air mouse is an image acquisition unit that can photograph visible light scenes. Preferably, the luminous flux of the light emitting diode is 10 lumens.

Example four

This embodiment provides an air mouse, and the difference from the air mouse in the first embodiment lies in the processing method of the positioning image. When the processing unit of the air mouse of this embodiment processes each frame of positioning image, it first performs image processing, and then sends image position data or a rebranding signal to the controlled device according to the image processing result.

Image processing consists of two steps:

The first step is to obtain the feature code and image location data of the positioning mark image.

In the second step, according to the acquired characteristic codes, search for the same characteristic codes acquired in the processed previous frame positioning image, and after the results are obtained, the image processing process ends.

According to different results of image processing, the type of data sent by the processing unit to the controlled device is also different. If the same feature code is found, the processing unit sends the image position data corresponding to the feature code; if the same feature code is not found, the processing unit sends a rebrand signal. The processing method of the positioning image by the processing unit is shown in FIG. 15.

This embodiment also provides a driver program installed on the controlled device of the air mouse for controlling the movement of the cursor on the driven screen according to the image position data or the conversion signal.

The driver has a preset recording area for recording image position data. When the driver receives the image position data, if there is image position data in the recording area, compare the received image position data with that in the recording area, and control the cursor movement on the screen according to the obtained image displacement data, and Update the data in the recording area to the image location data just received; if there is no image location data in the recording area, store the image location data just received in the recording area. When the driver receives the rebranding signal, it will clear the recording area.

The processing method of the driver of this embodiment when the image position data is received is as shown in FIG. 16.

The air mouse in this embodiment adopts various methods described in Embodiment 1, Embodiment 2, and Embodiment 3 when acquiring feature codes and image position data from a positioning image.

This embodiment also provides an air mouse system, which is composed of the air mouse in this embodiment and at least two positioning marks with different characteristics. The positioning marks include various positioning marks recorded in the first, second, and third embodiments.

The following examples illustrate the processing method and driver of this embodiment.

The image acquisition unit outputs the first frame positioning image, and the processing unit acquires the feature code 50 and corresponding image position data (120, 318) from it. Because it is the first frame, there is no record of the previous frame. Therefore, the number 1 is sent to the controlled device, and the number 1 is the standard change signal. The driver of the controlled device will clear the data in the recording area after receiving the rebranding signal.

The image acquisition unit outputs the second frame positioning image, from which the processing unit obtains the feature code 50 and the corresponding image position data (120, 317). According to the feature code 50, the same is found in the feature code obtained from the positioning image of the previous frame. Then (120,317) is sent to the controlled device. After receiving the image displacement data, the driver of the controlled device will record (120,317) in the recording area because the recording area is empty.

The image acquisition unit outputs the third frame positioning image, from which the processing unit obtains the feature code 50 and the corresponding image position data (120, 315). According to the feature code 50, the same is found in the feature code obtained from the positioning image of the previous frame. Then send (120,315) to the controlled device. After receiving the image displacement data, the driver program of the controlled device compares (120,315) with (120,317), and finds that the value of the column coordinate is reduced by 2. The cursor on the control screen has moved 2 pixels to the right. Then update the data in the recording area to (120,,315)

The image acquisition unit outputs the fourth frame positioning image, and the processing unit acquires the feature code 100 and corresponding image position data (120, 312) from it. According to the feature code 100, the same is not found in the feature code of the positioning image of the previous frame, so the number 1 is sent to the controlled device. The driver of the controlled device will clear the data in the recording area.

The image acquisition unit outputs the fifth frame positioning image, and the processing unit obtains the feature code 100 and corresponding image location data (120, 310) from it. According to the feature code 100, the same is found in the feature code of the previous frame of positioning image. (120, 310) is sent to the controlled device, and the driver of the controlled device writes it into the recording area.

The image acquisition unit outputs the sixth frame positioning image, and the processing unit obtains the feature code 100 and the corresponding image location data (120, 308) from it. According to the feature code 100, the same is found in the feature code of the previous frame of the positioning image. (120,308) is sent to the controlled device, the driver compares (120,308) with (120,310), and finds that the value of the column coordinate is less than 2, and the cursor on the control screen moves to the right by 2 Pixels.

After comparing the image position data, the driver controls the movement of the cursor on the screen according to the comparison result. The following rules are adopted: if the value of the row coordinate increases by n, the cursor on the screen is controlled to move up n pixels; if the row coordinate If the value of is reduced by n, the cursor on the control screen moves down by n pixels. If the value of the column coordinate increases by n, the cursor on the control screen moves n pixels to the left; if the value of the column coordinate decreases by n, the cursor on the control screen moves n pixels to the right; if the row coordinate and column If the values of the coordinates have changed, control the cursor to move correspondingly in two directions. For example, if the measured values of row coordinates and column coordinates are increased by 1, the cursor on the control screen will move up by 1 pixel, and then move to the left by 1 pixel.

In this embodiment and the first to third embodiments, the driver and the controlled device constitute a computer system.

It can be seen from the above process that the cursor on the screen controlled by the air mouse does not move when the positioning mark used as the air position reference object is replaced. After the reference object is replaced, the cursor continues to move in the original direction. The air mouse of this embodiment can also increase the moving length by adding positioning marks.

Example five

This embodiment provides an air mouse, and the difference from the air mouse in the first embodiment lies in the processing method of the positioning image. When the processing unit of the air mouse of this embodiment processes each frame of positioning image, it obtains image displacement data through image processing.

The processing unit of the air mouse of this embodiment includes two steps when performing image processing on each frame of positioning image:

Step 1: Obtain the feature code and image location data of the positioning mark image. The specific methods include the various methods described in Embodiment 1, Embodiment 2, and Embodiment 3.

Step 2: Obtain image displacement data.

The specific method is as follows: according to the acquired characteristic codes, look for the same in the characteristic codes acquired in the processed previous frame positioning image. If there are the same, the corresponding image position data of the feature code in the two frames of positioning images are compared, and the image displacement data is obtained according to the comparison result; if there is no the same, the image processing of the positioning image of this frame ends.

The image processing method is shown in FIG. 17.

After the image processing process ends, if image displacement data is generated, the processing unit sends a displacement signal to the controlled device according to the image displacement data, and the controlled device controls the cursor on the driven screen to move correspondingly according to the displacement signal.

In the following, the air mouse moves from left to right in the air as an example to illustrate the air mouse and image processing method of this embodiment.

The air mouse and two positioning marks form an air mouse system. The two positioning marks are two infrared light sources. The implementation is the same as that shown in Figure 2. The two infrared light sources are arranged horizontally as positioning mark A and positioning mark B respectively.

The image processing program used by the processing unit must first establish a recording area for recording the data (including feature codes and image position data) obtained from the positioning image, and update the record at the end of each frame positioning image processing process Data in the area.

The user holds the air mouse and points the lens 202 toward the positioning mark A. The image acquisition unit starts to capture and output the first frame of positioning image. In this frame of positioning image, the four pixels with coordinates (120,318), (120,319), (121,318), (121,319) The value is 50, and the pixel value of the remaining pixels is 0. These four pixels form the positioning mark A image. The image data obtained by the processing unit from this frame positioning image is 50 and (120, 318).

The processing unit searches for the same in the recording area according to the feature code 50. Because this is the first frame positioning image processed, there is no data in the recording area. After storing 50 and (120, 318) in the recording area, the first frame positioning image processing ends.

The user holds the air mouse and continues to move to the right. In the second frame of the positioning image output by the image acquisition unit, the coordinates are (120,317), (120,318), (121,317), (121,318). The pixel value of the pixel is 50, and the pixel value of the remaining pixels is 0. The processing unit obtains the feature code 50 and image location data (120, 317) from this frame positioning image.

The processing unit found the same in the recording area according to the feature code 50. Comparing (120,317) with (120,318), it is found that the value of the column coordinate is reduced by 1. This is the image displacement data.

According to the image displacement data, the processing unit sends a displacement signal to the controlled device, and the information contained in the displacement signal is 1 to the right. After updating the data in the recording area to 50, (120, 317), the frame positioning image processing ends. The controlled device controls the cursor on the driven screen to move 1 pixel to the right (the screen faces the user) according to the displacement signal.

The user holds the air mouse and continues to move to the right. The image acquisition unit continues to capture and output positioning images. In the output series of positioning images, the position of the positioning mark A image gradually moves to the left, and the processing unit continues to output displacement signals according to the image processing results , The controlled device controls the cursor on the screen to continuously move to the right according to the displacement signal.

When the image acquisition unit outputs the nth frame of the positioning image, the processing unit obtains the feature code 50 and the corresponding image position data (120, 0) from the positioning image. At this time, the positioning mark A has reached the visibility of the lens of the image acquisition unit The edge of the range. After the frame positioning image processing is completed, the data in the recording area is 50, (120, 0).

After the user holds the air mouse and moves to the right for a certain distance, there is only the positioning mark B in the lens. At this time, the image acquisition unit outputs a positioning image of the n+1th frame, in which four pixels have a pixel value of 100, and the remaining pixels have a pixel value of 0. The four pixels constitute the positioning mark B image, and the coordinates of the four pixels are (120, 317), (120, 318), (121, 317), (121, 318), respectively.

The image data obtained by the processing unit in the frame positioning image is 100 and (120, 317). According to the feature code 100, the same is not found in the recording area. After updating the data in the recording area to 100, (120, 317). The image processing process of this frame positioning image ends.

After the user holds the air mouse and continues to move a small distance to the right, in a frame of positioning image output by the image acquisition unit, 4 pixels have a pixel value of 100 and the remaining pixels have a pixel value of 0. The coordinates of the 4 pixels are (120, 316), (120, 317), (121, 316), (121, 317), respectively. After the processing unit performs image processing on the frame positioning image, it sends a displacement signal moving 1 to the right to the controlled device, and the controlled device controls the cursor on the screen to move 1 pixel to the right. The air mouse continues to move to the right, and the cursor on the screen also continues to move to the right, until the positioning mark B moves out of the visual range of the lens.

In this embodiment, the method of obtaining the image displacement data according to the feature code and the image position data is the same as the method of obtaining the image displacement data according to the feature code and the image position data by the driver in the first embodiment. The obtained image displacement data is also used as the basis for controlling the cursor movement on the screen, so the beneficial effects are also the same.

In this embodiment, when the image position data is compared, the row coordinates and column coordinates are compared separately to find out the changes in the values. The comparison result includes four situations: row coordinates increase, row coordinates decrease, column coordinates increase, and column coordinates decrease. The air mouse of this embodiment sends a displacement signal according to the image displacement data. The rule adopted is: if the value of the row coordinate increases by n, it outputs a displacement signal that moves upward by n; if the value of the row coordinate decreases by n, it outputs to Move down the displacement signal of n. If the value of the column coordinate increases by n, it outputs a displacement signal that moves to the left by n; if the value of the column coordinate decreases by n, it outputs a displacement signal that moves to the right by n; the controlled device controls according to the received displacement signal The cursor on the driven screen moves n pixels in the corresponding direction. If the values of row coordinates and column coordinates have changed, they must be reflected in the displacement signal. For example, if the measured values of the row coordinate and the column coordinate both increase by 1, the information contained in the output displacement signal is 1 move up and 1 left. The control device receives this displacement signal and controls the cursor on the screen to move up by 1 pixel, and then move to the left by 1 pixel. The specific format of the displacement signal can refer to the data format of the optical mouse.

In an implementation of the air mouse of this embodiment, the rule for sending displacement signals according to the image displacement data is: when the value of the column coordinate increases, the displacement signal that moves to the left is output; when the value of the column coordinate decreases, the output is Displacement signal for right movement. In this way, when moving horizontally, the moving direction of the cursor on the screen is opposite to that of the air mouse. The direction of longitudinal movement can also be modified in the same way.

In an implementation of the air mouse of this embodiment, the processing unit multiplies the image displacement data by 2 and outputs it. For example, the measured value of the column coordinate is reduced by 1, and the displacement signal of 2 to the right is output. base on needs. You can also multiply the image displacement data by 3, 4 or more and output it.

Example Six

This embodiment provides an air mouse. As shown in FIG. 18, the air mouse includes a housing, an image acquisition unit, and a processing unit. The image acquisition unit is used to photograph a scene including the positioning mark and output a positioning image with the positioning mark image; the processing unit performs image processing on the positioning image, and sends a cursor control signal to the controlled device according to the image processing result. When performing image processing on the positioning image, it includes obtaining the feature code of the positioning mark image and the image position data.

The air mouse of this embodiment is equivalent to removing the button unit on the basis of the air mouse of the first to fifth embodiments.

When the processing unit of the air mouse in this embodiment performs image processing on a frame of positioning image, various image processing methods recorded in Embodiment 1 to Embodiment 5 can be used.

Because the air mouse of this embodiment has no button unit, the volume can be made relatively small. For example, a cube with a side length of 5 cm is used as the housing, and a hole is opened on one surface to place the lens of the image capture unit.

The air mouse of this embodiment is particularly suitable for smart mobile devices, and the smart mobile devices include smart phones and tablet computers. When using a smart phone, the player holds the phone with the screen facing the player. Paste the air mouse on the back of the phone. The lens is located on the side opposite to the sticking surface, that is, the lens faces the front of the player (the same as the back of the phone) The processing unit is connected to the mobile phone through a data cable. In front of the player, place a positioning mark (one or more can be placed as needed) where the camera can shoot. Players can control the cursor movement on the screen by moving their mobile phone in the air.

Example Seven

This embodiment provides an air mouse. Based on the air mouse in the first to fifth embodiments, the processing unit is divided into an image processing unit and a main control unit. As shown in Figure 19, the air mouse of this embodiment includes a housing, an image acquisition unit, an image processing unit, a button unit, and a main control unit. The image acquisition unit is used to photograph the scene including the positioning mark and output the positioning image with the positioning mark image; the image processing unit performs image processing on the positioning image and sends the image processing result to the main control unit; the main control unit according to the image processing result The cursor control signal is sent to the controlled device; the key signal generated by the key unit is sent to the controlled device through the main control unit; when the image processing unit processes the positioning image, the various processing units used in the first to fifth embodiments are used Image processing method.

When the image processing method adopted by the image processing unit is any one of Embodiment 1, Embodiment 2 or Implementation 3, a set of image data (including feature codes and image position data) will be obtained every time a frame of positioning image is processed. The image data is sent to the main control unit and sent to the controlled device by the main control unit.

When the image processing method adopted by the image processing unit is any one of the fourth embodiment, every time a frame of positioning image is processed, the obtained image position data or conversion signal will be sent to the controlled device through the main control unit.

When the image processing method adopted by the image processing unit is any one of the fifth embodiment, each frame of positioning image is processed, the obtained image displacement data is sent to the main control unit. The main control unit sends a displacement signal to the controlled device according to the image displacement data. The rules for sending the displacement signal according to the image displacement data are the same as those in the fifth embodiment.

The image processing unit of the air mouse in this embodiment can be served by an image processing chip. The image processing chip can be a general-purpose microprocessor (such as a single-chip microcomputer) or a digital signal processor (DSP). Preferably, the image processing chip is a STMicroelectronics STM32F407 chip, and the input terminal is electrically connected with the image acquisition unit, and the output terminal is electrically connected with the main control unit. The image processing chip may also use other microprocessors, such as a single-chip microcomputer or DSP.

The image processing chip in this embodiment adopts the image processing method of the present invention. The air mouse using the image processing chip can increase the moving length by adding positioning marks.

Example eight

This embodiment provides a cloud game system. Including air mouse, client device and cloud server. The air mouse includes a housing, an image acquisition unit, a key unit, and a processing unit; the image acquisition unit is used to shoot scenes including positioning marks and output positioning images with positioning marks; the key unit generates key signals and sends them to the client through the processing unit Device sending; the processing unit performs image processing on the positioning image, and sends the cursor control signal to the client device according to the image processing result. The processing unit processes the positioning images using various image processing methods in the first to fifth embodiments of the present invention.

The client device sends the cursor control signal and the key signal to the cloud server. The cloud server is used to run the game program, generate screen display data according to the cursor key signal and the key signal, and send it to the client device. The client device controls the display content on the screen of the driven display device according to the screen display data to make corresponding changes.

Using different image processing methods, the cursor control signals output by the air mouse are also different. When the received cursor control signal is a displacement signal, the cloud server generates screen display data according to the displacement signal. When the received cursor control signal is a feature code and image position data, the cloud server is provided with the driver program described in the first embodiment, the image displacement data is obtained through the driver program, and the screen display data is generated according to the displacement data. When the received cursor control signal is the image position data and the conversion signal, the cloud server is provided with the driver program described in the fourth embodiment, the image displacement data is obtained through the driver program, and the screen display data is generated according to the image displacement data.

The following example illustrates that the client device is a PC that drives a liquid crystal display. The cloud server runs a shooting game, and the air mouse controls the front sight on the screen. The air mouse sends a displacement signal to the PC (the information contained is 10 pixels to the right), the PC sends the signal to the cloud server, and the cloud server generates a video data packet through calculation (the video data packet includes the screen All data required for 1 frame of screen is displayed on the screen). The video data packet is sent to the PC, and the PC drives the liquid crystal display based on the data in the video data packet, and the liquid crystal display displays the picture after the crosshair moves 10 pixels to the right. The air mouse continuously sends out displacement signals, and the picture on the screen also changes continuously.

The air mouse sends a button signal to the PC, and the button signal contains the command to change the magazine. The key signal is sent to the cloud server through the PC. The cloud server generates 50 video data packets through calculations and sends them to the PC continuously. Each video data packet includes the data needed to display a frame of picture. Every time the PC receives a video data packet, it drives the LCD to display one frame. The picture, 50 frames of pictures connected together is the continuous action of a character in the game changing magazines.

The displacement signal sent by the air mouse of the present invention can be in the following format: 1, 2, 3, and 4 are used to represent the four moving directions of up, down, left, and right, and a number is added to indicate the number of pixels moved. The cloud server outputs video data packets according to the received displacement signal, and controls the display content on the screen. For example, when receiving 2+5, the cloud server controls the screen driven by the PC to display the screen after the cursor moves down 5 pixels; when receiving 3+2, it controls the screen to display the screen after moving the cursor 2 pixels to the left.

The cloud server controls the screen display content according to the image displacement data, and adopts the following rules: when row coordinates increase, row coordinates decrease, column coordinates increase, and column coordinates decrease, the cursor on the screen is controlled up, down, left, and Moving in 4 directions to the right, the value of the movement is equal to the value of the coordinate change. For example, the image displacement data adds 5 to the row coordinate, and the cloud server controls the screen driven by the PC to display the screen after the cursor has moved up by 5 pixels.

The controlled devices of the cloud game system in this embodiment are various types of electronic computer devices, including but not limited to PCs, video game consoles, smart TVs, tablet computers, and smart phones.

In this embodiment, the driver and the cloud server form a computer system.

In the cloud game system of this embodiment, the moving length of the air mouse can be increased by adding a positioning mark, so that the air mouse can have a higher moving length and moving accuracy at the same time, thereby increasing the gaming experience.

Example 9

This embodiment provides an air mouse, which is improved on the basis of the air mouse in the fifth embodiment. The difference lies in the method that the processing unit sends a displacement signal to the controlled device after processing each frame of the positioning image.

In the air mouse in this embodiment, after processing each frame of positioning image, if image displacement data is generated, the processing unit sends a displacement signal to the controlled device according to the image displacement data; if no image displacement data is generated, the processing unit The control device sends the displacement signal generated after the positioning image processing of the previous frame (if the displacement signal is generated after the positioning image processing of the previous frame).

The following example illustrates the air mouse of this embodiment.

The user holds the air mouse and moves it to the right. The nth frame of positioning image is output during the process of the air mouse moving to the right. After the image processing is completed, the displacement signal sent by the processing unit to the controlled device is X+10 (moving to the right by 10 on the screen). Pixels). The air mouse continues to move to the right and outputs the n+1th frame positioning image. The feature code obtained by the processing unit from the positioning image of this frame changes, and no image displacement data is generated after the positioning image processing of this frame is completed. At this time, the processing unit sends a displacement signal X+10 to the controlled unit (the same as the displacement signal output after processing the n-th frame positioning image).

During the movement of the air mouse in the fifth embodiment, if the displacement reference object (positioning mark) is changed, in the first frame of positioning image obtained after the change, the feature code of the positioning mark image is the same as the one in the previous frame of positioning image. are different. No image displacement data is generated, and no displacement signal is generated. That is, in this time period between the output of the two frames of positioning images, the movement of the air mouse cannot be reflected on the display screen of the controlled device, which affects the accuracy of cursor control. During the movement of the air mouse, if the time between the output of two positioning images is regarded as a time period, the moving direction and speed of the air mouse in the two adjacent time periods are very close in most cases. Therefore, the air mouse of this embodiment sends the displacement signal generated after the positioning image processing of the previous frame to the controlled device when there is no displacement data generated. During the movement, when the positioning reference object is switched, the direction of the air mouse movement and The distance can be correctly reflected on the display screen, which increases the accuracy of control.

Example ten

This embodiment provides a driver program, which is improved on the basis of the driver program in Embodiment 1. The processing method adopted by the driver program in this embodiment when each group of image data is received is:

According to the feature code, find the same in the last group of image data received. If the same is found, the image position data corresponding to the feature code in the two sets of image data are compared to obtain image displacement data, and the movement of the cursor on the screen is controlled according to the image displacement data. If the same is not found, the movement of the cursor on the screen is controlled according to the image displacement data obtained after processing the last set of image data received (if the data exists).

For example, the driver is installed on the PC, and the user holds the air mouse to move laterally. The nth group of image data received by the driver includes feature code 1 and an image position data. The feature code is the same as that in the n-1th group of image data. After comparing the two image position data, the obtained image displacement is that the column coordinate is reduced by 10, then the cursor on the control screen moves 10 cursors to the right; The obtained (n+1)th set of displacement data includes feature code 2 and an image position data. At this time, the cursor on the screen is still controlled to move 10 cursors to the right. Then start processing the n+2 group of image data.

The beneficial effect of the driver program of this embodiment is that the movement of the matching air mouse when the displacement reference object (positioning mark) is switched can also be reflected on the display screen, which increases the accuracy of control. In the driving program in the first embodiment, when the displacement reference object changes during the movement of the air mouse, there will be a certain distance of movement that cannot be reflected on the screen.

Example 11

This embodiment provides a driver program, which is improved on the basis of the driver program provided in the fourth embodiment.

The air mouse in the fourth embodiment will send a set of data to the controlled device after processing a frame of positioning image. This set of data is image position data or a rebranding signal, and the driver on the controlled device controls according to the set of data The movement of the cursor on the screen.

The driver of this embodiment will set a recording area for recording image position data. When receiving a group of data sent by the air mouse, use the following method to process:

In the first step, when the driver receives the image position data, if there is image position data in the recording area, compare the received image position data with that in the recording area, and control the screen based on the obtained image displacement data. The cursor moves, and the data in the recording area is updated to the image position data just received; if there is no image position data in the recording area, the image position data just received is stored in the recording area. When the driver receives the rebranding signal, it will clear the recording area.

In the second step, if the image displacement data is generated in the first step, the processing of this group of data ends. If the image displacement data is not generated in the first step, the cursor on the screen is controlled to move in the same direction and distance according to the result of the movement of the cursor on the screen after the processing of the last set of data received.

For example, the user holds the air mouse to move laterally, and sends a set of data to the controlled device every time a frame of positioning image is processed. When the driver on the controlled device receives the nth group of data, this group of data is the image position data. After the driver has finished processing, the cursor on the control screen has moved 5 pixels to the right. When the driver receives the n+1th group of data, this group of data is a rebranding signal. After the first step is processed, there is no image displacement data, then in the second step of processing, it is processed according to the previous group (nth group) data After finishing the result, control the cursor on the screen to move 5 pixels to the right. The driver receives the n+2 group of data, which is the image position data. The first step is processed and no image displacement data is generated, then in the second step, it is processed according to the previous group (n+1 group) data After finishing the result, control the cursor on the screen to move 5 pixels to the right. When the driver receives the n+3 group of data, this group of data is image position data. After the first step of processing is completed, the image displacement data is generated. The driver controls the cursor on the screen to move correspondingly according to this data.

According to the processing method of the driver in the fourth embodiment, when the reference object (location mark) changes during the movement of the air mouse, the air mouse has a certain moving distance (the distance moved within the time of outputting 2 frames of positioning images). Reflected on the display screen, affecting the accuracy of cursor control. The driver program of this embodiment completely compensates for this shortcoming and improves the accuracy of cursor control.

Embodiment 12

This embodiment provides an image processing method for a computer. It is used for image processing of the positioning image on the computer side, and the computer controls the movement of the cursor on the screen of the driven display device according to the image processing result.

In an implementation of this embodiment, the computer is a PC, which drives a liquid crystal display. The PC is connected to an air mouse. The composition of the air mouse is the same as the air mouse recorded in the first embodiment. However, the processing unit of the air mouse sends the positioning image to the connected device after receiving the positioning image output by the image acquisition unit. PC. An image processing software is installed on the PC, and the image processing software performs image processing on the received positioning image. The image processing method is the same as the image processing method adopted by the air mouse in the fifth embodiment. After the image processing software processes a frame of positioning image, if the image displacement data is obtained, the cursor on the LCD screen driven by this data is controlled to move accordingly. To control the cursor movement on the screen according to the image displacement data, the rules adopted are the same as those recorded in the first embodiment.

The air mouse held by the user continuously sends positioning images to the connected PC. The image processing software used by the computer in this embodiment obtains the image displacement data according to the displacement of the positioning mark images with the same image characteristics in the two frames of the positioning images. And according to the image displacement data, the cursor on the screen is controlled to move accordingly.

In an implementation of the image processing method of this embodiment, after the image processing software processes the positioning image for each frame, if image displacement data is generated, the processing unit controls the cursor on the screen to move accordingly according to the image displacement data; if If no image displacement data is generated, the cursor on the screen is controlled to move in the same direction and distance according to the result of the cursor movement after the positioning image processing of the previous frame.

The following example illustrates this embodiment.

The user holds the air mouse and moves it to the right. During the movement of the air mouse, the n-th frame positioning image is output. After the image processing software on the PC is finished, the cursor on the screen is controlled to move 10 pixels to the right. After the air mouse continues to move to the right, it outputs the n+1th frame positioning image. The feature code obtained by the image processing software from the positioning image of this frame has changed (the displacement reference object has changed at this time). Image displacement data is generated. At this time, the image processing software controls the cursor on the screen to move 10 pixels to the right (the direction and distance of the cursor movement after the positioning image processing of the nth frame is the same). When the positioning reference object of the air mouse changes during the movement, this embodiment can accurately reflect the moving direction and distance of the air mouse on the display screen, thereby increasing the accuracy of cursor control.

The computers in this embodiment are various electronic computer equipment, including but not limited to PCs, video game consoles, smart TVs, and cloud game servers.

In this embodiment, the computer and the image processing software used constitute a computer system, which can control the movement of the cursor on the driven screen according to the positioning image.

In an implementation of the computer system of this embodiment, the computer is a PC.

In an implementation of the computer system of this embodiment, the computer is a video game machine, and the installed image processing software performs image processing on the received positioning image, and controls the cursor on the driven screen to move according to the processing result .

In an implementation of the computer system of this embodiment, the computer is a smart TV, and the installed image processing software performs image processing on the received positioning image, and controls the cursor on the TV screen to move according to the processing result.

In an implementation of the computer system of this embodiment, the computer is a cloud game server. The positioning image is sent to the server, and the image processing software on the server performs image processing, and then controls the cursor movement on the driven screen according to the processing result.

The image processing method in this embodiment can enable a computer using the image processing method to increase the moving length by adding positioning marks.

The computer system in this embodiment, because of the image processing software used, can increase the moving length of the cursor on the driven screen by adding positioning marks.

The foregoing descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are all within the scope of the present invention. Inside.

Those skilled in the art can understand that the above embodiments are only examples, in which the features of different embodiments can be combined with each other to obtain implementations that are easily conceivable according to the disclosure of the present invention but are not clearly indicated in the drawings.

Those skilled in the art should understand that the above description and the embodiments of the present invention shown in the accompanying drawings are only examples and do not limit the present invention. The purpose of the present invention has been completely and effectively achieved. The functions and structural principles of the present invention have been shown and explained in the embodiments. Without departing from the principles, the embodiments of the present invention may have any deformation or modification.

Claims

An image processing method for an air mouse is characterized in that when image processing is performed on a positioning image, the feature code and image position data of the positioning mark image are obtained; wherein the feature code is determined according to the image characteristics of the positioning mark image.
5. The image processing method according to claim 1, wherein after the feature code and image location data are obtained from the positioning image, the feature code is compared with the feature code obtained in the previous frame of the positioning image.
The image processing method of claim 1, wherein after the feature code and image position data are obtained from the positioning image, the feature code and image position data obtained in the previous frame of the positioning image are compared, and the image displacement is further obtained data.
8. The image processing method of claim 1, wherein when processing the positioning image, the feature code and image position data of one of the positioning mark images are obtained.
5. The image processing method of claim 1, wherein when the positioning image is processed, feature codes and image position data of all the positioning mark images are acquired.
An air mouse, which is characterized by comprising a housing, an image acquisition unit, and a processing unit, wherein the image acquisition unit is used to photograph a scene including a positioning mark and output a positioning image with a positioning mark image; the processing unit is The positioning image is image processed, and the cursor control signal is sent to a controlled device according to the image processing result; when the positioning image is processed, it includes acquiring the feature code of the positioning mark image and the image position data, wherein the feature code is based on the positioning To determine the characteristics of the logo image.
8. The air mouse according to claim 6, wherein the processing unit sends the feature code and image position data to the controlled device.
7. The air mouse of claim 6, wherein after the processing unit obtains the feature code and image position data from a frame of positioning image, it searches for the feature code obtained from the previous frame of positioning image for the same feature code; if If there is the same, the image location data corresponding to the feature code is sent to the controlled device; if there is no the same, the rebranding signal is sent to the controlled device.
7. The air mouse according to claim 6, wherein the processing unit obtains image displacement data according to the displacement of the positioning mark images with the same image characteristics in the two previous and subsequent frames of positioning images, and sends a displacement signal to the controlled device according to the image displacement data.
7. The air mouse of claim 6, wherein after the processing unit obtains the feature code and image position data from a frame of positioning image, it searches for the feature code obtained from the previous frame of positioning image for the same feature code; if If there are the same, compare the corresponding image position data of the feature code in the two frames of positioning images, and send the displacement signal to the controlled device according to the obtained image displacement data; if there is no the same, the previous frame of positioning image The displacement signal generated after processing is sent to the controlled device.
8. The air mouse according to claim 6, further comprising a button unit for inputting a button signal, and the button signal is sent to the controlled device through the processing unit.
An air mouse, which is characterized by comprising a housing, an image acquisition unit, an image processing unit, a key unit, and a main control unit, wherein the image acquisition unit is used to photograph a scene including a positioning mark and output a positioning image with the positioning mark image , Wherein the image processing unit performs image processing on the positioning image and sends the image processing result to the main control unit. The main control unit sends the cursor control signal to the controlled device according to the image processing result, and the key signal generated by the key unit passes the main control unit. The control unit sends to the controlled device, wherein when the image processing unit processes the positioning image, it includes acquiring the feature code of the positioning mark image and the image position data, wherein the feature code is determined according to the feature of the positioning mark image.
An air mouse system, which is characterized by comprising an air mouse and at least two positioning marks with different characteristics, wherein the air mouse includes a housing, an image acquisition unit, a button unit, and a processing unit, wherein the image acquisition unit is used for For shooting a scene including a positioning mark and outputting a positioning image with a positioning mark image, wherein the key signal input by the key unit is sent to the controlled device through the processing unit, and the processing unit performs image processing on the positioning image , And send the cursor control signal to the controlled device according to the image processing result, the positioning mark with different characteristics, the positioning mark image formed in the positioning image has different image characteristics.
The air mouse system according to claim 13, wherein the positioning mark images formed by the positioning marks with different characteristics in the positioning image have different pixel values.
The air mouse system according to claim 13, wherein the positioning mark images formed by the positioning marks with different characteristics in the positioning image are composed of different numbers of pixels.
The air mouse system according to claim 13, wherein the positioning mark images formed by the positioning marks with different characteristics in the positioning image have different arrangement of the constituent pixels.
An air mouse control system, which is characterized by comprising an air mouse and a controlled device, the air mouse including a housing, an image acquisition unit, and a processing unit, wherein the image acquisition unit is used to photograph a scene including a positioning mark , And output a positioning image with a positioning mark image, wherein the processing unit performs image processing on the positioning image, and sends a cursor control signal to the controlled device according to the image processing result. When the positioning image is processed, It includes acquiring the feature code of the positioning mark image and the image position data, wherein the feature code is determined according to the feature of the positioning mark image, wherein the controlled device controls the movement of the cursor on the driven screen according to the cursor control signal.
The air mouse control system of claim 17, wherein the cursor control signal sent by the air mouse to the controlled device is image data, wherein the image data includes feature code and image position data, wherein the controlled device When the device driver receives a group of image data, it controls the movement of the cursor on the driven screen according to the change of the image position data corresponding to the same feature code in the group and the previous group of image data.
The air mouse control system according to claim 17, wherein the cursor control signal sent by the air mouse to the controlled device is image position data and a rebranding signal, and the driver program of the controlled device determines the position of the image by The data and the conversion signal are processed to obtain the image displacement data and then control the cursor movement on the driven screen.
The air mouse control system according to claim 17, wherein the cursor control signal sent by the air mouse to the controlled device is a displacement signal, and the controlled device controls the movement of the cursor on the screen of the driven display device according to the displacement signal.
An infrared sensor strip for providing positioning marks for an air mouse is characterized in that it comprises a bracket and at least two infrared light sources installed on the bracket, and the two infrared light sources have different characteristics.
The infrared sensor strip according to claim 21, wherein the brightness of the two infrared light sources is different.
The infrared sensor strip of claim 21, wherein the light-emitting areas of the two infrared light sources are different.
The infrared sensor strip of claim 21, wherein the two infrared light sources have different shapes.
The infrared sensor strip according to claim 21, wherein the bracket is a hollow structure.
An image processing method used in a computer for processing a positioning image including a positioning mark image, characterized in that it includes the following steps:

(a) Obtain the feature code and image location data of the location mark image, where the feature code is determined based on the image feature of the location mark image; and

(b) Compare the obtained feature code with the feature code obtained in the previous frame of the positioning image, and if they are the same, compare the image position data corresponding to the feature code.
A method for controlling the movement of a cursor on a screen driven by a computer is characterized in that it comprises the following steps:

(a) Receiving two frames of positioning images at intervals, in which there are positioning mark images; and

(b) Determine whether the positioning mark images in the two frames of the positioning image have the same characteristics, if so, compare the positions of the positioning mark images in the two frames of the positioning image, and if an image displacement occurs Data, the cursor on the screen driven by the image displacement data is controlled to move.
The method for controlling the movement of a cursor on a driven screen by a computer according to claim 27, wherein said step (b) comprises the following step: after processing each frame of said positioning image, if said image displacement occurs Data, the cursor movement on the driven screen is controlled according to the image displacement data; if the image displacement data is not generated, the cursor movement on the driven screen is controlled according to the image displacement data generated after the positioning image processing of the previous frame .
An air mouse, suitable for controlling a controlled device, is characterized by including:

A shell

An image acquisition unit, wherein the image acquisition unit is disposed on the housing, and the image acquisition unit outputs a positioning image with a positioning mark image; and

A processing unit, wherein the processing unit is arranged in the housing, the processing unit is communicably connected to the image acquisition unit, and the processing unit is based on a feature code and a feature code of the positioning mark image The image position data processes the positioning image, and sends a cursor control signal to the controlled device to control the controlled device.
The air mouse according to claim 29, further comprising a button unit, wherein the button unit is disposed on the housing, and the button unit is communicably connected to the processing unit.
The air mouse according to claim 29, further comprising a cursor movement control key, wherein the cursor movement control key is provided on the housing, and the cursor movement control key is communicably connected to the processing unit.
28. The air mouse of claim 29, wherein the image acquisition unit includes a lens and an image sensor module, wherein the lens is communicably connected to the image sensor module.
An air mouse system, suitable for controlling a controlled device, is characterized by including:

An air mouse, wherein the air mouse includes a housing, an image acquisition unit, and a processing unit, wherein the image acquisition unit is disposed on the housing, and the image acquisition unit outputs an image with a positioning mark A positioning image, wherein the processing unit is provided in the housing, the processing unit is communicably connected to the image acquisition unit, and the processing unit is based on a feature code and a feature code of the positioning mark image The image position data processes the positioning image and sends a cursor control signal to the controlled device; and

At least one positioning mark, wherein the positioning mark can form the positioning image with the positioning mark image, and the position of the air mouse can be determined by using the positioning mark.
The air mouse system according to claim 33, wherein the positioning marks of the air mouse system are implemented as at least two, and the two positioning marks have different characteristics, and the characteristics of each positioning mark It can be recorded in the positioning image.
The air mouse system according to claim 34, wherein the positioning mark images formed by the two positioning marks in the positioning image have different pixel values.
The air mouse system according to claim 34, wherein the positioning mark images formed by the two positioning marks in the positioning image are composed of different numbers of pixels.
The air mouse system according to claim 34, wherein the positioning mark is implemented as an infrared light source.
The air mouse system according to claim 34, wherein the positioning mark is implemented as a visible light source.
The air mouse system according to claim 34, wherein the distance between the air mouse and the positioning mark is maintained to be greater than or equal to 50 cm and less than or equal to 100 cm.
The air mouse system of claim 34, further comprising a bracket, wherein the positioning mark is laterally arranged on the bracket.
The air mouse system of claim 34, further comprising a bracket, wherein the positioning mark is laterally arranged on the bracket.
An image processing method, characterized in that the image processing method includes the following steps:

(a) Obtain a positioning image with a positioning mark image; and

(b) Determine a feature code and image position data of the positioning mark image by reading the pixel value of the pixel in the positioning image.
The image processing method according to claim 42, wherein the step (b) includes the step (c) reading the pixel value of each pixel in the positioning image one by one, and finding the first pixel value greater than a predetermined value When a pixel with a pixel value is set, the pixel value and coordinates of the pixel are recorded.
The image processing method according to claim 42, further comprising step (d) comparing the feature codes of the positioning mark images of the two positioning images obtained one after another.
The image processing method according to claim 44, wherein after the step (b), it comprises the step (e) comparing the positioning mark images of the two sets of positioning images with the same feature code, and obtaining an image displacement data.
The image processing method according to claim 42, wherein said step (b) further comprises:

(I) Read the pixel value of each pixel one by one in a preselected area until a pixel with a pixel value greater than a preset pixel value is found;

(Ii) Find all pixels in a preset pixel range centered on the pixel whose pixel value is greater than the preset pixel value; and

(Iii) Determine the feature code according to the number of all pixels found, and find the pixel with the smallest coordinate value among all the pixels whose pixel value is greater than the preset pixel value, and record the coordinate of this pixel as the image position data.
The image processing method according to claim 42, wherein said step (b) further comprises the following steps:

(I) Read the pixel values of all pixels in a preselected area, and find out all the pixels whose pixel values are greater than a preset pixel value as a candidate pixel;

(Ii) Select a pixel with the smallest coordinate value from the candidate pixels, find out all pixels with a pixel value greater than the preset pixel value in a preset pixel range centered on the pixel, and define these pixels as A component pixel; and

(Iii) Record the total number of the constituent pixels as the feature code, find a pixel with the smallest coordinate value among the constituent pixels, and record the coordinate of the pixel as the image position data corresponding to the feature code.
The image processing method according to claim 47, wherein in said step (b), it further comprises a step (iv) of removing the constituent pixels from the candidate pixels, if there is still the candidate pixel , Then start processing from step (ii), if there are no candidate pixels, the image processing process ends.
The image processing method according to claim 42, wherein said step (b) further comprises the following steps:

(I) In a preselected area, read the pixel value of each pixel according to the order from left to right and from top to bottom until a pixel with a pixel value greater than a preset pixel value is found;

(Ii) Find out all pixels with a pixel value greater than the preset pixel value within a preset pixel range centered on the pixel;

(Iii) If the column coordinates of all the pixels found are the same, record the feature code as 1; if the row coordinates are the same, record the feature code as 2; and

(Iv) Among all the found pixels, find the pixel with the smallest coordinate value, and record the coordinate of the pixel as the image position data corresponding to the feature code.
The image processing method according to claim 42, wherein said step (b) further comprises the following steps:

(I) Read the pixel values of all pixels in a preselected area, and find out all the pixels whose pixel values are greater than a preset pixel value as a candidate pixel;

(Ii) Select a pixel with the smallest coordinate value from the candidate pixels, find out all pixels with a pixel value greater than the preset pixel value in a preset pixel range centered on the pixel, and define these pixels as One component pixel;

(Iii) If the column coordinates of all the constituent pixels found are the same, record the feature code as 1; if the row coordinates of all constituent pixels are the same, record the feature code as 2; and among all the constituent pixels, find one The pixel with the smallest coordinate value, and the coordinate of the pixel is recorded as the image position data; and

(Iv) Among the candidate pixels, remove the pixels with the same coordinates as the constituent pixels. If the candidate pixels are still left, then start processing from step (ii); if there are no candidate pixels, Then the step of acquiring image data ends.
A control method, characterized in that the control method includes the following steps:

(A) receiving at least two sets of a positioning image with a positioning mark image at intervals, wherein the positioning mark image includes a feature code and an image position data corresponding to the feature code; and

(B) Determine whether to generate an image displacement data according to the feature code of the positioning mark image and the image position data. If the image position data is generated, send a displacement signal to a target according to the image displacement data.控设备。 Control equipment.
The control method according to claim 51, wherein said step (B) includes the following steps:

(B.1) Searching for the feature code of the latter group of image data in the received previous group of image data;

(B.2) Compare the positioning mark images of the two sets of positioning images with the same feature code, and obtain the image displacement data; and

(B.3) Control the movement of a cursor according to the image displacement data.
The control method according to claim 51, wherein said step (B) comprises the following step: after processing each frame of said positioning image, if said image displacement data is generated, then according to the image displacement data to The controlled device sends the displacement signal; if the image displacement data is not generated, the controlled device sends the displacement signal generated after processing the positioning image of the previous frame.