WO2021004505A1 - Air mouse, air mouse system, image processing method and control method - Google Patents
Air mouse, air mouse system, image processing method and control method Download PDFInfo
- Publication number
- WO2021004505A1 WO2021004505A1 PCT/CN2020/101074 CN2020101074W WO2021004505A1 WO 2021004505 A1 WO2021004505 A1 WO 2021004505A1 CN 2020101074 W CN2020101074 W CN 2020101074W WO 2021004505 A1 WO2021004505 A1 WO 2021004505A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- positioning
- air mouse
- feature code
- pixel
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
Definitions
- the present invention relates to the field of computer and electronic game control, in particular to an image processing method for an air mouse, a chip adopting the image processing method, an air mouse, an air mouse system, an air mouse control system, and a device for intelligent mobile equipment Air mouse, air mouse driver and computer system using the program, infrared sensor strip providing positioning mark for air mouse, cloud game system, image processing method for computer.
- the mouse is the most common computer and electronic game control device. Its main purpose is to control the movement of the cursor on the screen of display devices such as televisions and monitors.
- the cursor refers to various icons that move with the movement of the mouse, such as the mouse pointer on the screen, the weapon crosshairs in video games, and the characters.
- Movement accuracy and movement length are two important aspects to measure the performance of a mouse.
- the movement accuracy refers to the minimum number of pixels that the mouse can control the cursor on the screen by one-way movement; the movement length refers to the one-way movement that can be controlled by the mouse once. The maximum number of pixels the cursor moves on the screen.
- the common optical mouse has a high level of movement accuracy and movement length. A slight movement of the optical mouse on the desktop can make the cursor move only 1 pixel on the screen; if it keeps moving in one direction, the cursor can be moved from One edge of the screen moves to the opposite edge.
- the optical mouse is used to control a screen with a resolution of 1920 ⁇ 1080 or higher, which can have both movement precision and movement length.
- the air mouse is another device used to control the cursor on the screen. Unlike an optical mouse, an air mouse controls the cursor on the screen to move by moving it in the air.
- Chinese Patent Application No. 201210319834.X discloses an air mouse using an infrared image recognition scheme.
- the air mouse emits infrared light through an infrared light emitter, and the infrared light is sensed by the infrared image sensor on the body to form a A series of images, in which there are light spots formed by infrared light emitters.
- the position of the light spot changes in the series of images formed.
- the main body sends position change data to a controlled device according to the change of the position of the light spot in the adjacent image, and the controlled device controls the cursor on the driven screen to move according to the position change data.
- an air mouse using an infrared image recognition solution cannot meet the requirements of mainstream display devices while ensuring a higher movement accuracy.
- the image resolution output by the infrared image sensor of the air mouse is relatively low, such as 320 ⁇ 240.
- a movement accuracy of 1 that is, a single movement can control the cursor on the screen to move 1 pixel at least
- the body moves once in a single direction, and the horizontal movement length of the cursor on the screen does not exceed 320, and the vertical movement length does not exceed 240 .
- the existing mainstream display devices such as LCDs and TVs have a resolution of 1920 ⁇ 1080 and above.
- the body part can increase the moving length by enlarging the output ratio of the position change data. For example, if the change in the position of the light spot in two adjacent frames of images is measured to be 1 pixel, the screen on the screen is controlled to move 6 pixels. At this time, the lateral movement length can reach about 1920, which can meet the needs of use.
- the movement accuracy is reduced from 1 to 6, that is, the body moves once, and the minimum distance for the cursor to move on the screen is 6 pixels.
- Such movement accuracy will lead to a very poor experience. Therefore, the existing air mouse using the infrared image recognition solution cannot meet the requirements of movement accuracy and movement length at the same time when used in mainstream resolution display devices, which greatly restricts the application of the air mouse.
- the infrared light transmitter is equivalent to providing a positioning mark for the body of the air mouse, so that it can determine its position in the air at any time.
- only one infrared light transmitter is used, which is equivalent to only one positioning mark, so that the air mouse cannot maintain a high level of movement accuracy and movement length at the same time. To increase the movement length, the movement accuracy must be reduced.
- the image processing method outputs position change data according to the position change of the image formed by positioning marks (such as infrared light source) in two adjacent frames of images. When there is only one positioning mark, the two images for position comparison are the same positioning mark.
- the image's position change data can reflect the moving direction and distance of the air mouse; when there are 2 or more positioning marks, the two images to be compared may be formed by different positioning marks, and the position of the image changes The data cannot correctly reflect the moving direction and distance of the air mouse. Controlling the movement of the cursor on the screen according to the position change data may cause the moving direction and distance to be different from the actual moving direction and distance of the air mouse.
- Smart mobile devices such as smartphones and tablet computers are mainly operated by touch. This limits the development of mobile games, because many types of games are suitable for mouse operation, such as FPS, MOBA, etc.
- FPS field-programmable gate array
- MOBA multi-reliable and low-power bidirectional keyboard
- the touch screen operation method is far from the mouse operation method that players are accustomed to, and the cursor cannot be controlled quickly and accurately, resulting in a poor experience.
- the present invention was produced under this background.
- An object of the present invention is to provide an image processing method, wherein the image processing method is suitable for an air mouse.
- the air mouse adopting the image processing method can increase the moving length by adding positioning marks.
- Another object of the present invention is to provide an image processing method, wherein the image processing method is used for an air mouse.
- the air mouse using this image processing method can increase the moving length without reducing the moving accuracy.
- Another object of the present invention is to provide an air mouse, which can increase the moving length by adding positioning marks.
- Another object of the present invention is to provide an air mouse system with high moving accuracy and length.
- Another object of the present invention is to provide an air mouse system that can meet the requirements of mainstream display devices (screen resolution of about 1920 ⁇ 1080) in terms of movement accuracy and movement length.
- Another object of the present invention is to provide an image processing chip that can be used for an air mouse, so that the air mouse can increase the moving length by adding positioning marks.
- Another object of the present invention is to provide an air mouse control system for controlling a cursor on a display screen, which can increase the moving length of the cursor on the screen by adding positioning marks.
- Another object of the present invention is to provide an air mouse control system for controlling the cursor on a display screen, which can be used in conjunction with multiple (two or more) positioning marks, while achieving a higher moving length and movement Accuracy.
- Another object of the present invention is to provide an air mouse driver, which is installed on a controlled device connected to the air mouse, and can control the cursor movement on the screen according to the cursor control signal provided by the air mouse.
- Another object of the present invention is to provide a computer system capable of controlling the movement of the cursor on the controlled screen according to the cursor control signal provided by the air mouse.
- Another object of the present invention is to provide an air mouse for smart mobile devices, which can quickly and accurately control the cursor and games on the smart mobile device screen.
- Another object of the present invention is to provide an infrared sensor strip for providing positioning marks for the air mouse, which can increase the moving length of the air mouse without reducing the movement accuracy.
- Another object of the present invention is to provide an infrared sensor strip that can provide positioning marks for the air mouse, so that the air mouse can have both moving length and accuracy.
- Another object of the present invention is to provide a cloud game system in which the air mouse can increase the moving length by adding positioning marks.
- Another object of the present invention is to provide an air mouse.
- the positioning mark changes during the movement of the air mouse
- the movement of the air mouse can also be reflected on the display screen.
- Another object of the present invention is to provide a driver for an air mouse, which enables the movement of the air mouse to be reflected on the display screen when the positioning mark changes during the movement of the air mouse.
- Another object of the present invention is to provide an image processing method for a computer.
- the computer adopting the image processing method can control the cursor on the driven screen to move according to the positioning image.
- Another object of the present invention is to provide a computer system capable of controlling the movement of the cursor on the driven screen according to the positioning image.
- the present invention provides an image processing method for air mouse.
- the method is characterized by
- the feature code of the positioning mark image and the image position data are obtained, wherein the feature code is determined according to the image feature of the positioning mark image.
- the feature code obtained from the previous frame of the positioning image is searched for the same.
- the image location data corresponding to the same feature code in the current frame and the previous frame of the positioning image are compared. Yes, and get the image displacement data.
- the feature code and image position data of one of the positioning mark images are acquired.
- the feature codes and image position data of all positioning mark images therein are acquired.
- the invention also provides an air mouse.
- the air mouse includes a housing, an image acquisition unit, and a processing unit.
- the image acquisition unit is used to photograph a scene including the positioning mark and output a positioning image with the positioning mark image; the processing unit performs image processing on the positioning image, and sends a cursor control signal to the controlled device according to the image processing result.
- image processing is performed on the positioning image, it includes obtaining the feature code of the positioning mark image and the image location data, wherein the feature code is determined according to the feature of the positioning mark image.
- the processing unit sends the feature code and image position data to the controlled device.
- the processing unit after the processing unit obtains the feature code and image position data from one frame of positioning image, it searches for the same feature code in the feature code obtained from the previous frame of positioning image . If there are the same, the image location data corresponding to the feature code is sent to the controlled device; if there is no the same, the rebranding signal is sent to the controlled device.
- the processing unit obtains image displacement data according to the displacement of the positioning mark images with the same image characteristics in the two previous and subsequent frames of positioning images, and sends a displacement signal to the controlled device according to the image displacement data.
- the processing unit after the processing unit obtains the feature code and image position data from one frame of positioning image, it searches for the same feature code in the feature code obtained from the previous frame of positioning image . If there are the same, compare the corresponding image position data of the feature code in the two frames of positioning images, and send the displacement signal to the controlled device according to the obtained image displacement data; if there is no the same, then locate the previous frame The displacement signal generated after image processing is sent to the controlled device.
- the air mouse of the present invention further includes a button unit.
- the key unit is connected with the processing unit, and the key signal generated by the key unit is sent to the controlled device through the processing unit.
- the air mouse includes a housing, an image acquisition unit, an image processing unit, a button unit, and a main control unit.
- the image acquisition unit is used to photograph the scene including the positioning mark and output the positioning image with the positioning mark image; the image processing unit performs image processing on the positioning image and sends the image processing result to the main control unit; the main control unit according to the image processing result
- the cursor control signal is sent to the controlled device; the key signal generated by the key unit is sent to the controlled device through the main control unit; when the image processing unit processes the positioning image, the image processing method provided by the present invention is adopted.
- the processing unit sends the feature code and image position data to the controlled device.
- the processing unit after the processing unit obtains the feature code and image position data from one frame of positioning image, it searches for the same feature code in the feature code obtained from the previous frame of positioning image ; If there are the same, the image location data corresponding to the feature code is sent to the controlled device; if there is no the same, the rebranding signal is sent to the controlled device.
- the processing unit obtains image displacement data according to the displacement of the positioning mark images with the same image characteristics in the two previous and subsequent frames of positioning images, and sends a displacement signal to the controlled device according to the image displacement data.
- the processing unit after the processing unit obtains the feature code and image position data from one frame of positioning image, it searches for the same feature code in the feature code obtained from the previous frame of positioning image ; If there are the same, compare the corresponding image position data of the feature code in the two frames of positioning images, and send the displacement signal to the controlled device according to the obtained image displacement data; if there is no the same, the previous frame The displacement signal generated after positioning image processing is sent to the controlled device.
- the air mouse includes a housing, an image acquisition unit, an image processing unit, a button unit, and a main control unit;
- the image acquisition unit is used to photograph a scene including a positioning mark and output the positioning The positioning image of the logo image;
- the image processing unit performs image processing on the positioning image and sends the image processing result to the main control unit;
- the main control unit sends the cursor control signal to the controlled device according to the image processing result;
- the present invention also provides an air mouse system including the air mouse of the present invention and at least two positioning marks with different characteristics.
- the positioning mark images formed by the positioning marks with different characteristics in the positioning image have different image characteristics.
- the positioning mark images formed by positioning marks with different characteristics in the positioning image have different pixel values of the constituent pixels.
- the positioning mark images formed by positioning marks with different characteristics in the positioning image have different arrangements of the constituent pixels.
- the positioning mark images formed by positioning marks with different characteristics in the positioning image have different numbers of pixels.
- the invention also provides an air mouse control system, which includes the air mouse and controlled equipment of the invention.
- the air mouse sends a cursor control signal to the controlled device, and the controlled device controls the movement of the cursor on the driven screen according to the cursor control signal.
- the cursor control signal sent by the air mouse to the controlled device is image data (including feature code and image position data).
- the driver of the controlled device receives a group of image data, it controls the movement of the cursor on the driven screen according to the change of the image position data corresponding to the same feature code in the group and the previous group of image data.
- the cursor control signal sent by the air mouse to the controlled device is image position data and a rebrand signal.
- the driver of the controlled device controls the movement of the cursor on the driven screen after obtaining the image displacement data by processing the image position data and the conversion signal.
- the cursor control signal sent by the air mouse to the controlled device is a displacement signal
- the controlled device controls the movement of the cursor on the screen of the driven display device according to the displacement signal
- the positioning mark images formed by the positioning marks with different characteristics in the positioning image have different arrangement of the constituent pixels.
- the air mouse control system includes an air mouse and a controlled device.
- the air mouse includes a housing, an image acquisition unit, and a processing unit; the image acquisition unit is used for To photograph a scene including a positioning mark, and output a positioning image with a positioning mark image; the processing unit performs image processing on the positioning image, and sends a cursor control signal to the controlled device according to the image processing result;
- the image includes obtaining the feature code of the positioning mark image and the image position data, wherein the feature code is determined according to the feature of the positioning mark image; the controlled device controls the cursor movement on the driven screen according to the cursor control signal.
- the cursor control signal sent by the air mouse to the controlled device is image data (including feature codes and image position data); the driver of the controlled device
- the program receives a group of image data, it controls the movement of the cursor on the driven screen according to the change of the image position data corresponding to the same feature code in the group and the previous group of image data.
- the cursor control signal sent by the air mouse to the controlled device is image position data and a rebranding signal;
- the position data and the conversion signal are processed to obtain the image displacement data and then control the cursor movement on the driven screen.
- the cursor control signal sent by the air mouse to the controlled device is a displacement signal
- the controlled device controls the cursor on the screen of the driven display device according to the displacement signal mobile.
- the present invention also provides an infrared sensor strip for providing positioning marks for an air mouse, which includes a bracket and at least two infrared light sources mounted on the bracket, and the two infrared light sources have different characteristics.
- the brightness of the two infrared light sources is different.
- the light-emitting areas of the two infrared light sources are different.
- the two infrared light sources have different shapes.
- the bracket is a hollow structure.
- the present invention provides an image processing method used in a computer for processing a positioning image including a positioning mark image, which includes the following steps:
- the present invention provides a method for a computer to control the movement of a cursor on a driven screen, which includes the following steps:
- the step (b) includes the following steps: after processing the positioning image for each frame, if the image displacement data is generated, then The cursor movement on the driven screen is controlled according to the image displacement data; if the image displacement data is not generated, the cursor movement on the driven screen is controlled according to the image displacement data generated after the positioning image processing of the previous frame.
- the invention also provides an image processing chip for processing digital images.
- the image processing chip processes digital images
- the image processing method of the present invention is adopted.
- the invention also provides a computer system, which includes a controlled device and a driver of the air mouse.
- the present invention also provides an infrared sensor strip, which includes a bracket and at least two infrared light sources with different characteristics.
- adjacent infrared light sources have different brightness.
- adjacent infrared light sources have different light-emitting areas.
- adjacent infrared light sources have different shapes.
- the bracket is hollow to conveniently accommodate the infrared light source and the battery.
- the present invention also provides an air mouse for smart mobile devices.
- the air mouse includes a housing, an image acquisition unit, and a processing unit.
- the image acquisition unit is used to photograph a scene including a positioning mark and output a positioning image containing the positioning mark image; the processing unit obtains a cursor control signal by image processing the positioning image, and sends the cursor control signal to the controlled intelligent mobile device (Smartphone or tablet) to send.
- the processing unit performs image processing on the positioning image, the image processing method of the present invention is adopted.
- the controlled devices in the air mouse control system of the present invention are various types of electronic computer devices, including but not limited to PCs, video game consoles, smart TVs, set-top boxes, tablet computers, smart phones, and VR devices.
- the image processing method of the air mouse of the present invention further acquires the feature code according to the image characteristics on the basis of acquiring the location data of the positioning mark image in each frame of the positioning image.
- the air mouse to which the image processing method is applied can be used in conjunction with two or more positioning marks with different characteristics to achieve the purpose of increasing the moving length without reducing the moving accuracy.
- the image processing chip of the present invention adopts the image processing method of the present invention. Therefore, the air mouse adopting the image processing chip can increase the moving length by adding positioning marks.
- the air mouse system of the present invention uses two or more positioning marks.
- the movement length is increased without reducing the movement accuracy.
- the air mouse When the air mouse is used in mainstream display devices, it has good effects in both the length and accuracy of the cursor movement, and it is truly practical. While the cursor performance is greatly improved, the increased cost is very small (only positioning marks are added to the hardware), so while achieving higher cursor control performance, it can still maintain a lower cost, which is conducive to popularization and application.
- the air mouse control system of the present invention is used to control the movement of the cursor on the screen of the display device driven by the controlled device, and the movement length is greatly increased without reducing the cursor movement accuracy. It meets the needs of mainstream display devices (screen resolution of about 1920 ⁇ 1020).
- adjacent infrared light sources have different characteristics, so that the positioning mark image formed in the positioning image has different image characteristics.
- it can make the air mouse have a higher moving length and moving accuracy.
- the present invention further provides an air mouse suitable for controlling a controlled device, which includes:
- An image acquisition unit wherein the image acquisition unit is arranged on the housing, and the image acquisition unit outputs a positioning image with a positioning mark image;
- a processing unit wherein the processing unit is arranged in the housing, the processing unit is communicably connected to the image acquisition unit, and the processing unit is based on a feature code and a feature code of the positioning mark image
- the image position data processes the positioning image, and sends a cursor control signal to the controlled device to control the controlled device.
- the air mouse further includes a button unit, wherein the button unit is disposed on the housing, and the button unit is communicably connected to the processing unit.
- the air mouse further includes a cursor movement control key, wherein the cursor movement control key is provided on the housing, and the cursor movement control key is communicatively connected to the processor. unit.
- the image acquisition unit includes a lens and an image sensor module, wherein the lens is communicably connected to the image sensor module.
- the present invention further provides an air mouse system suitable for controlling a controlled device, which includes:
- the air mouse includes a housing, an image acquisition unit, and a processing unit, wherein the image acquisition unit is disposed on the housing, and the image acquisition unit outputs an image with a positioning mark A positioning image, wherein the processing unit is provided in the housing, the processing unit is communicably connected to the image acquisition unit, and the processing unit is based on a feature code and a feature code of the positioning mark image
- the image position data processes the positioning image and sends a cursor control signal to the controlled device;
- At least one positioning mark wherein the positioning mark can form the positioning image with the positioning mark image, and the position of the air mouse can be determined by using the positioning mark.
- the positioning marks of the air mouse system are implemented as at least two, and the two positioning marks have different characteristics, and the characteristics of each positioning mark can be recorded in The positioning image.
- the positioning mark image formed by the two positioning marks in the positioning image has different pixel values for its constituent pixels.
- the positioning mark images formed by the two positioning marks in the positioning image are composed of different numbers of pixels.
- the positioning mark is implemented as an infrared light source.
- the positioning mark is implemented as a visible light source.
- the distance between the air mouse and the positioning mark is maintained to be greater than or equal to 50 cm and less than or equal to 100 cm.
- the air mouse system further includes a bracket, wherein the positioning mark is laterally arranged on the bracket.
- the air mouse system further includes a bracket, wherein the positioning mark is laterally arranged on the bracket.
- the present invention further provides an image processing method.
- the image processing method includes the following steps:
- the step (c) reads the pixel value of each pixel in the positioning image one by one, and finds the first pixel value greater than a preset pixel value In the case of pixels, the pixel values and coordinates of the pixels are recorded.
- the image processing method further includes the step (d) comparing the feature codes of the positioning mark images of the two positioning images acquired first.
- a step (e) is included to compare the positioning mark images of the two sets of positioning images with the same feature code, and obtain an image displacement data.
- the step (b) further includes:
- the step (b) further includes the following steps:
- step (b) in the step (b), it further includes the step (iv) to remove the constituent pixels from the candidate pixels, and if the candidate pixels remain, proceed from step (Ii) Start processing, if there are no candidate pixels, the image processing process ends.
- step (b) further comprising the following steps:
- step (b) further comprising the following steps:
- step (Iv) Among the candidate pixels, remove the pixels with the same coordinates as the constituent pixels. If the candidate pixels are still left, then start processing from step (ii); if there are no candidate pixels, Then the step of acquiring image data ends.
- the present invention provides a cursor control method.
- the cursor control method includes the following steps:
- the step (B) includes the following steps:
- the step (B) includes the following step: after the positioning image is processed for each frame, if the image displacement data is generated, then the image displacement data is sent to the The controlled device sends the displacement signal; if the image displacement data is not generated, the controlled device sends the displacement signal generated after processing the positioning image of the previous frame.
- Figure 1 shows an embodiment of the air mouse of the present invention.
- Figure 2 shows the air mouse of the present invention and its usage.
- FIG. 3 shows an implementation of the air mouse driver of the present invention.
- Figure 4 shows the signal flow of the air mouse and air mouse control system of the present invention.
- Figure 5 shows an embodiment of the infrared sensor strip of the present invention.
- Figure 6 shows an embodiment of the infrared sensor strip of the present invention.
- FIG. 7 shows an embodiment of the image processing method of the present invention.
- Figure 8 shows an embodiment of the infrared sensor strip of the present invention.
- Figure 9 shows an embodiment of the infrared sensor strip of the present invention.
- Figure 10 shows an embodiment of the infrared sensor strip of the present invention.
- Figure 11 shows an embodiment of the infrared sensor strip of the present invention.
- Fig. 12 shows an embodiment of the image processing method of the present invention.
- Figure 13 shows an embodiment of the infrared sensor strip of the present invention.
- Figure 14 shows an embodiment of the infrared sensor strip of the present invention.
- Fig. 15 shows an embodiment of the image processing method of the present invention.
- Figure 16 shows an embodiment of the air mouse driver of the present invention
- FIG. 17 shows an embodiment of the image processing method of the present invention.
- Figure 18 shows an embodiment of the air mouse and air mouse control system of the present invention.
- Figure 19 shows an embodiment of the air mouse and air mouse control system of the present invention.
- the air mouse includes a housing 200, buttons 201 arranged on the upper surface of the housing 200, a lens 202 arranged at the middle position of the front end of the housing 200, and a circuit part in the housing 200.
- the circuit part includes a button circuit connected to the button 201, an image sensor module connected to the lens 202, and a processing unit.
- the lens 202 and the image sensor module together form an image acquisition unit, where the lens 202 is a filter lens that can filter out visible light and only allows infrared light to pass through.
- the image sensor module is an OV7620 image sensor module, which integrates a CMOS image sensor and a DSP chip, and a lens 202 to form a complete infrared digital camera module, which can capture infrared light-emitting objects and output digital images.
- the button 201 and the button circuit together form a button unit, and the button signal generated by the button unit is sent to a controlled device after passing through the processing unit.
- the key signal is used to realize the key control function and belongs to the prior art.
- the processing unit uses STMicroelectronics' STM32F407 chip.
- the output end of the OV7620 image sensor module is electrically connected to the input pin of the STM32F407 chip, and the button circuit is electrically connected to the input pin of the STM32F407.
- the processing unit is also used to bridge the controlled device.
- the controlled device is a PC, which is connected to the STM32F407 via a USB cable.
- the controlled device drives a display device.
- the display device is a liquid crystal display with a resolution of 1920 ⁇ 1080.
- the processing unit can also connect and communicate with the controlled device wirelessly.
- the air mouse needs at least one infrared light source as a positioning mark to help the air mouse determine its position in the air at any time.
- the infrared light source may be various objects capable of providing infrared light, including various infrared light emitters, candles and the like.
- the OV7620 image sensor module shoots through the lens 202, and continuously outputs the formed image in the form of a digital signal to the processing unit (STM32F407).
- the output image contains the image formed by the positioning mark (infrared light source), which is called positioning Logo image.
- the image with the positioning mark image is called the positioning image.
- the processing unit performs image processing on the positioning image, and sends a control signal to the controlled device according to the image processing result.
- the OV7620 image sensor module is set to output 50 frames of grayscale images with a resolution of 320*240 per second.
- Gray image is a kind of digital image.
- the pixel matrix has 240 rows and 320 columns, from top to bottom from row 0 to row 239, and from left to right from column 0 to column 319.
- the position of each pixel in the matrix is marked by a unique coordinate.
- the first pixel in the upper left corner has coordinates (0, 0). Indicates that it is located in row 0 and column 0, where the first 0 is the row coordinate, indicating the number of rows it is in; the second 0 is the column coordinate, indicating the number of columns it is in.
- Each pixel in the pixel matrix has a pixel value, represented by a number from 0 to 255. Among them, 255 represents white, 0 represents black, and the remaining numbers represent varying degrees of gray.
- the air mouse and two positioning marks form an air mouse system, which has better performance.
- the two positioning marks have different characteristics, and the characteristics can be recorded in the positioning image.
- the image acquisition unit may also be an infrared image sensor, as long as it can output digital images.
- This embodiment also provides an infrared sensor strip, which is used to provide a reference for the displacement of the air mouse of this embodiment in the air.
- the infrared sensor strip includes a bracket 300 and an infrared light source A301 arranged at the left end of the front of the bracket 300, and an infrared light source B302 arranged at the right end of the front of the bracket 300.
- the bracket 300 is rectangular, with a length of 25 cm, a height of 2 cm, and a thickness of 2 cm. The distance between the two infrared light sources is 20 cm.
- the two infrared light sources are infrared light-emitting diodes and are powered by an external power source.
- the bracket 300 may also be hollow, with a battery compartment for installing batteries to supply power to the infrared light source.
- the two infrared light sources on the bracket 300 serve as two positioning marks, and together with the air mouse form an air mouse control system.
- the infrared light source A301 is the positioning mark A
- the infrared light source B302 is the positioning mark B.
- Two infrared light sources emit infrared light of different brightness.
- the luminous flux of the infrared light source A301 is 5 lumens; the luminous flux of the infrared light source B302 is 10 lumens.
- the positioning mark image formed by the infrared light source in the positioning image has different image characteristics, which is reflected in different pixel values.
- the image formed by the infrared light source A301 in the positioning image has a pixel value of 50 for each pixel constituting the image; the image formed by the infrared light source B302 in the positioning image has the value of each pixel constituting the image The pixel value is 100.
- the processing unit searches for the positioning mark image in the pixel matrix, it finds a pixel with a pixel value above 40, that is, it is determined that the positioning mark image is found.
- the processing unit of the air mouse performs image processing on the positioning image output by the image acquisition unit, and its purpose is to obtain the feature code and image position data of the positioning mark image.
- the specific method in this embodiment is:
- the pixel value is a feature code
- the coordinates are image location data corresponding to the feature code. For example, when reading a pixel with coordinates (0, 5), it is found that the pixel value is 50, and 50 and (0, 5) are recorded. Wherein 50 is the feature code, (0, 5) is the image location data corresponding to the feature code.
- the image position data is actually the coordinates of one of the constituent pixels of a positioning mark image in the positioning image, and is used to represent the position of the positioning mark image in the positioning image.
- the processing unit sends a set of image data to the connected PC, including feature codes and image location data.
- This embodiment also provides a driver program, used on the controlled device of the air mouse (PC in this embodiment), for controlling the movement of the cursor on the driven screen according to the image data.
- a driver program used on the controlled device of the air mouse (PC in this embodiment), for controlling the movement of the cursor on the driven screen according to the image data.
- the processing method adopted is:
- the image position data corresponding to the feature code in the two sets of image data is compared to obtain the image displacement data, and the cursor movement on the screen is controlled according to the image displacement data; if the same is not found, The processing of this group of image data ends.
- the processing method is shown in Figure 3.
- composition and signal transmission mode of the air mouse and the air mouse control system in this embodiment are shown in FIG. 4.
- the driver on the PC side presets a recording area to record the image data (including feature code and image position data) from the air mouse, and update the data in the recording area at the end of each group of image data processing .
- the image acquisition unit starts to take and output the first frame of positioning image.
- the four pixels with coordinates (120,318), (120,319), (121,318), (121,319)
- the value is 50, and the pixel value of the remaining pixels is 0.
- the image composed of these four pixels is formed by the infrared light source A and is called the positioning mark A image.
- This frame positioning image is transmitted to the processing unit in the form of a digital signal, and the processing unit performs the following image processing:
- the specific method is to start from the pixel with coordinates (0, 0), first read the pixel value of each pixel in the first row of pixels in the pixel matrix in the order from left to right, and then the second row, third row, until the last line.
- the read pixel value is 50.
- the processing unit sends 50 and (120, 318) to the connected PC, and the driver on the PC side searches for the same in the recording area according to the feature code 50. Because this is the first set of image data processed, there is no record in the recording area. After storing 50 and (120, 318) in the recording area, the entire image processing process ends.
- the coordinates are (120,317), (120,318), (121,317), (121,318).
- the pixel value of the pixel is 50, and the pixel value of the remaining pixels is 0.
- the image data obtained by the processing unit from this frame positioning image is 50 and (120, 317).
- the driver found the same in the recording area according to the feature code 50. Comparing (120,317) with (120,318), it is found that the value of the column coordinate is reduced by 1, and the driver obtains the image displacement data.
- the cursor on the LCD screen driven by the PC is moved 1 pixel to the right (the screen faces the user).
- the image acquisition unit continues to capture and output positioning images.
- the position of the positioning mark A image gradually moves to the left, and the processing unit continues to output image data according to the image processing results ,
- the PC controls the cursor on the LCD screen to move continuously to the right according to the image data.
- the processing unit obtains the feature code 50 and the corresponding image position data (120, 0) from the frame of the positioning image. At this time, the positioning mark A has reached the visible range of the lens 202 edge. After the frame positioning image processing is completed, the data in the recording area of the driver is 50, (120, 0).
- the image acquisition unit After the user holds the air mouse and moves to the right for a certain distance, there is only the positioning mark B in the lens 202. At this time, the image acquisition unit outputs a positioning image of the n+1th frame, in which four pixels have a pixel value of 100, and the remaining pixels have a pixel value of 0.
- the four pixels constitute the positioning mark B image, and the coordinates of the four pixels are (120, 317), (120, 318), (121, 317), (121, 318), respectively.
- the processing unit obtains the feature code 100 and the corresponding image position data (120, 317) in the frame positioning image.
- the driver on the PC side does not find the same in the recording area according to the feature code 100.
- the processing of this group of image data ends.
- the image data sent to the PC is 100 and (120, 316).
- the driver finds the same in the recording area, and changes (120, 316) Comparing with (120, 317), the obtained image displacement data is that the value of the column coordinate is reduced by 1, and the cursor on the LCD screen driven by the PC control moves 1 pixel to the right.
- the user holds the air mouse and continues to move to the right, and the cursor on the screen also continues to move to the right, until the positioning mark B moves out of the visible range of the lens 202.
- the air mouse of this embodiment there is a positioning mark within the visual range of the lens 202 to control the cursor.
- the length of the movement can be increased by adding positioning marks, and how much the movement length increases depends on the distance between the two positioning marks.
- a more reasonable positioning mark placement distance is that when the air mouse is in the middle of the two positioning marks, both positioning marks are within the visible range of the lens 202, but both are located at the edge, so that the cursor can be guaranteed during the horizontal movement.
- the continuity of control can also have the longest movement length. At this time, adding 1 positioning mark can increase the movement length close to 320 pixels.
- the beneficial effect of the air mouse in this embodiment comes from the image processing method used.
- the air mouse uses positioning mark A and positioning mark B as reference objects to determine its own position in the air.
- the reference object is switched from positioning mark A to positioning mark B, the cursor on the screen has a very short pause, and then continues to move to the right. If it is an air mouse in the prior art, it simply compares the positions of the positioning mark images in the two frames of the positioning image before and after. When the reference object is switched, the cursor on the screen will move in the opposite direction to the air mouse.
- the image position data of the last frame of positioning image generated is (120, 0); after moving slightly to the right, the first frame of positioning image generated using positioning mark B as the reference object, The image position data is (120, 317). After comparing the two image position data and outputting a displacement signal, the cursor on the screen will move 317 pixels to the left.
- the air mouse of this embodiment When the air mouse of this embodiment performs image processing on each frame of positioning image, on the basis of obtaining the image position data, it further obtains a feature code according to the image feature.
- the driver on the PC side obtains the necessary information, and only compares the position of the same positioning mark in the two positioning images, making it possible to increase the moving length by adding positioning marks.
- the driver When the driver compares the image position data, it needs to compare the row coordinates and column coordinates separately to find out the changes of the values. There are four types of changes: row coordinates increase, row coordinates decrease, column coordinates increase, and column coordinates decrease. small.
- the driver of this embodiment controls the cursor movement on the screen according to the image displacement data obtained after the image position data comparison. The rule adopted is: if the value of the row coordinate increases by n, the cursor on the screen is controlled to move up n pixels ; If the value of the row coordinate decreases by n, the cursor on the control screen moves down by n pixels.
- the cursor on the control screen moves n pixels to the left; if the value of the column coordinate decreases by n, the cursor on the control screen moves n pixels to the right; if the row coordinate and column If the values of the coordinates have changed, control the cursor to move correspondingly in two directions. For example, if the measured values of row coordinates and column coordinates are increased by 1, the cursor on the control screen will move up by 1 pixel, and then move to the left by 1 pixel.
- the rule can also be changed: in the obtained image displacement data, if the value of the column coordinate increases, the cursor is controlled to move to the left; the value of the column coordinate decreases, and the cursor is controlled to move to the right. In this way, when moving horizontally, the moving direction of the cursor on the screen is opposite to that of the air mouse.
- the direction of longitudinal movement can also be modified in the same way.
- the driver program multiplies the image displacement data by 2 and outputs it. For example, if the value of the measured column coordinates is reduced by 1, the cursor on the screen is moved 2 pixels to the right . In this way, the air mouse can control the cursor on the screen to move a longer distance with a shorter moving distance.
- the air mouse system consisting of an air mouse and two positioning marks can basically meet the needs of controlling mainstream resolution screens.
- the horizontal movement length of the air mouse can be close to 1920, and the movement accuracy is 3. If you want to increase the moving length of the air mouse in the vertical direction, place the infrared sensor strip upright so that the two infrared light sources are arranged up and down.
- turning the air mouse in the air can also control the cursor movement on the screen, turning left or right can control the cursor on the screen to move horizontally; turning up or down can control the cursor on the screen to move vertically . Because the rotation can also change the position of the positioning mark image in the positioning image output by the image acquisition unit.
- the distance between the air mouse and the infrared light source is more suitable to be 50cm to 100cm. Too close will make the image of the positioning mark in the image determined to be too large and the image to be positioned.
- the length of the rectangular bracket is 45 cm, and three infrared light sources are arranged on the front of the bracket from left to right.
- the 3 infrared light sources are arranged in a row, and the distance between each light source is 20cm.
- the brightness of the infrared light source on the left and right sides is the same, while the brightness of the light source in the middle is different.
- the luminous flux of the infrared light source on the left and right sides is 5 lumens
- the luminous flux of the infrared light source in the middle is 10 lumens.
- the 3 infrared light sources are equivalent to 3 positioning marks, and form an air mouse system with the air mouse, which can further increase the moving length of the air mouse.
- the bracket 300 is rectangular, 25 cm in length and 8 cm in height.
- the luminous fluxes of the four infrared light sources are 5 lumens, 10 lumens, 15 lumens, and 20 lumens, respectively. It is equivalent to providing 4 positioning marks for the air mouse, and the positioning mark images formed by any two adjacent positioning marks in the positioning image have different characteristics (pixel values), compared to only one positioning mark in the prior art After the air mouse system is composed of the 4 infrared light sources and the air mouse, the air mouse can be moved horizontally, vertically and diagonally in the air, and the moving length is increased.
- the bracket 300 in this embodiment may also have a frame-shaped structure, with one infrared light source at each of the four corners, and four infrared light sources with four kinds of brightness. As shown in Figure 6.
- the width of each side of the frame is 2CM.
- the processing unit uses the following image processing method for the positioning image:
- the method of selecting the pixel with the smallest coordinate value is to select the pixel with the smallest row coordinate value among all the candidate pixels. If there is only one, then the pixel is the pixel with the smallest coordinate value; if there are more than one, select the column among the multiple The smallest coordinate value.
- the processing unit After processing one frame of positioning image according to the image processing method, the processing unit can obtain the image data of all the positioning mark images.
- the image processing method is shown in Figure 7. After the image processing of each frame of the positioning image is completed, the processing unit sends all the acquired image codes and image positions as a set of image data to the connected PC.
- each group of image data received by the driver on the PC side may have one or more feature codes and corresponding image position data.
- extract one of the feature codes randomly search for the same in the recording area, and find the same, then the image position data corresponding to the feature code corresponds to the feature code in the recording area
- the position data of the image is compared, and the cursor on the screen is controlled to move according to the result of the comparison; if the same is not found, another feature code is extracted and the search continues until the same is found or all feature codes have been searched.
- the image acquisition unit When the image acquisition unit captures infrared light sources A and B at the same time, it outputs the first frame positioning image. There are 4 pixels with a pixel value of 50, the coordinates are (120, 0), (120, 1), (121, 0), (121, 1); there are 4 pixels with a pixel value of 100, the coordinates They are (120,318), (120,319), (121,318), (121,319).
- the processing unit performs the following image processing on this frame positioning image:
- the coordinates are (120, 0), (120, 1), (121, 0), (121, 1) (120, 318), (120, 319), (121, 318), (121, 319), respectively.
- the processing unit After removing this pixel and all pixels with a pixel value of 100 from the candidate pixels, there are no candidate pixels, and the image processing process of the positioning image of this frame ends.
- the processing unit sends the first group of image data to the connected PC, including feature code 50 and corresponding image location data (120, 0), and feature code 100 and corresponding image location data (120, 318).
- the PC driver After the PC driver receives the group of image data, it first searches the recording area according to the feature code 50. Did not find the same; then search in the recording area according to feature code 100. If the same is not found, the process ends, and the group of image data is recorded in the recording area.
- the image acquisition unit sends out the second frame positioning image
- the processing unit sends the second group of image data to the PC after image processing.
- the driver finds the same in the recording area according to the feature code 50, compares (120, 1) with (120, 0), and finds that the value of the column coordinate has increased by 1. Then the cursor on the control screen moves 1 pixel to the left to end the processing process, and at the same time change the data in the recording area to the second group of image data.
- the image acquisition unit can capture visible light images, for example, the lens matched with the image sensor module is changed to a lens that can pass visible light.
- the visible light source can also be used as a positioning mark.
- light-emitting diodes can be used as positioning marks.
- Use 3 light-emitting diodes as positioning marks one of which has a luminous flux of 5 lumens and the other two has a luminous flux of 10 lumens.
- When in use put out a row with 5 lumens in the middle.
- the image processing unit of the air mouse of the present invention can process higher-resolution images.
- the OV7620 image sensor outputs a black and white digital image with a resolution of 640 ⁇ 480 at 30 frames per second. Processing the positioning image of this resolution can make the air mouse have a higher moving length without changing the moving accuracy.
- the air mouse of this embodiment further includes a cursor movement control key, which is arranged on the surface of the housing and is connected to the processing unit through a circuit. It is used to control whether the cursor on the screen moves with the air mouse. For example, the user often needs to move the air mouse to the middle of the effective area and does not want the cursor on the screen to follow the movement.
- the processing unit stops outputting the cursor control signal, and when the key is released, the processing unit resumes outputting the cursor control signal.
- the key is arranged at a position suitable for thumb pressing on the housing.
- the cursor controlled by the air mouse refers to various icons that move with the movement of the air mouse, such as the mouse pointer on the screen, the weapon crosshairs in electronic games, and characters.
- the cursor can be of different sizes, and even include the entire screen. For example, using an air mouse to control the field of view of a character in a game, the entire screen may be the field of view. When the air mouse moves, the display of the entire screen will change.
- the air mouse and the controlled device in this embodiment jointly constitute an air mouse control system.
- the controlled devices are various computer devices, including but not limited to PCs, video game consoles, smart TVs, set-top boxes, tablet computers, smart phones, and VR devices.
- the air mouse outputs image data (including code features and image position data), and the controlled device controls the cursor on the display device screen driven by the image data to move.
- the processing unit When the air mouse of the present invention is used to control a smart phone or a tablet computer, the processing unit outputs image data, and the smart phone or tablet computer controls the cursor movement on the screen according to the image data.
- This embodiment provides an air mouse, which is different from the air mouse in the first embodiment in the method of acquiring image data.
- the processing unit of the air mouse in this embodiment adopts the following processing methods for the positioning image:
- the pixel values of all pixels whose coordinates meet the following requirements are read: the row coordinates are from X-3 to X+3, and The column coordinates are from Y-3 to Y+3.
- the purpose of this step is to find all the pixels that make up the positioning mark image.
- the feature code is determined according to the number of all pixels found. For example, if there are 4 pixels with a pixel value above 40, record the feature code as 4; if there are 9 pixels with a pixel value above 40, record the feature code as 9.
- the method for determining the pixel with the smallest coordinate value is: among all the pixels with coordinate values above 40, select the pixel with the smallest row coordinate value. If there is only one pixel, the pixel is the pixel with the smallest coordinate value. If there are more than one, the pixel with the smallest column coordinate value is selected from the multiple as the pixel with the smallest coordinate value.
- the preselected area mentioned above refers to an area divided in the positioning image.
- the method of dividing the positioning area is: in a matrix composed of 320 ⁇ 240 pixels, the first 3 rows, the last 3 rows, and the first 3 columns After removing the pixels in the last three columns, the remaining pixels constitute the preselected area of the positioning image. That is, all the pixels whose row coordinates are 3 to 236 and column coordinates 3 to 316 constitute the preselected area.
- the positioning marks with different characteristics and the positioning mark images formed in the positioning images have different numbers of constituent pixels, and the number of constituent pixels reflects the image characteristics of the positioning mark images. If the image of the light source is incomplete, it may affect the determination of the image feature, which may cause an error in the feature code obtained from the image feature. And for the positioning mark image found in the preselected area, all its constituent pixels must be in the entire 320 ⁇ 240 pixel matrix. This is the purpose of setting the preselected area.
- the preselected area can be re-divided. For example, after removing the pixels in the first 5 rows, the last 5 rows, the first 5 columns, and the last 5 columns in the pixel matrix, the remaining pixel set is used as the preselected area.
- This embodiment also provides an infrared sensor strip, which includes a bracket 300 and two infrared light sources.
- the bracket 300 is rectangular, 25 cm long, 2 cm high, and 2 cm thick.
- the two holes that can emit infrared light are infrared light sources.
- the hole with a side length of 2 mm has a light-emitting area of 4 square millimeters, and the hole with a side length of 3 mm has a light-emitting area of 9 square millimeters.
- the inside of the bracket 300 may be hollow, which is convenient for accommodating the infrared light-emitting diodes.
- the images formed by the two infrared light sources in the positioning image have different composition numbers. For example, when the distance between the air mouse and the infrared light source is about 1M, the image formed by the infrared light source with a light-emitting area of 4 square millimeters is composed of 4 pixels; the image formed by the infrared light source with a light-emitting area of 9 square millimeters is composed of 9 pixels.
- the processing unit determines the feature code according to the number of pixels in the positioning mark image.
- 4 pixels have pixel values of 100, and the coordinates are (120, 3), (120, 4), (121, 3), (121, 4), respectively.
- the processing unit first reads the pixel value of each pixel in the preselected area in the order from left to right and top to bottom. When it reads the pixel with coordinates (120, 3), it reads the first one. Pixels with pixel values above 40.
- the pixel values of 49 pixels in the surrounding 7 pixels ⁇ 7 pixels range are read. That is, the row coordinates are 117 to 123, and the column coordinates are the pixel values of all pixels in the range of 0 to 6. It is found that the pixel value of 4 pixels is 100, and the coordinates are (120, 3), (120, 4), (121, 3), (121, 4).
- the feature code is recorded as 4.
- the image position data is recorded as (120, 3) according to the coordinates of the pixel with the smallest coordinate value.
- the processing unit sends the obtained feature code and corresponding image position data to the controlled device.
- FIG. 9 Another implementation of the infrared sensor strip of this embodiment is shown in FIG. 9, a longitudinal rectangular hole is opened on the left end of the front of the bracket 300. There are two longitudinal rectangular holes at the right end, and the two rectangular holes are 1 mm apart. The height of the three rectangular holes is 4 mm, and the width is 1 mm. An infrared light emitting diode with the same brightness is arranged behind each rectangular hole, and the two rectangular holes at the right end can share one infrared light emitting diode.
- the three holes can be used as two positioning marks, wherein the hole at the left end serves as a positioning mark, and the two holes at the right end together serve as a positioning mark.
- the images formed by the two positioning marks in the positioning image are composed of different numbers of pixels.
- the infrared sensor strip of this embodiment includes a bracket and three infrared light sources.
- the bracket is rectangular, 45cm long, 2cm high, and 2cm thick.
- the square holes at both ends have a side length of 2 mm; the middle hole has a side length of 3 mm.
- the luminous flux of the three infrared light-emitting diodes is 10 lumens.
- the three square holes that can emit infrared light are three infrared light sources, which can be used as three positioning marks and an air mouse to form an air mouse system. Compared with the solution of two positioning marks, it can further increase the moving length of the air mouse.
- the infrared sensor strip of this embodiment includes a bracket and 4 infrared light sources.
- the bracket is rectangular with a length of 25 cm and a height of 8 cm.
- a square hole is opened at each of the four corners of the rectangular bracket. Each hole has a different side length.
- the hole in the upper left corner has a side length of 1 mm
- the hole in the lower left corner has a side length of 2 mm
- the hole in the upper right corner has a side length of 3 mm
- the hole in the lower right corner has a side length of 4 mm.
- the luminous flux of the 4 infrared light-emitting diodes is 10 lumens.
- the four holes that can emit infrared light are the infrared light sources, which can be used as four positioning marks to form an air mouse system with an air mouse. Compared with the solution of 2 positioning marks, the air mouse can increase the moving length in the horizontal, vertical and diagonal directions at the same time.
- four infrared light sources can be arranged on a rectangular frame, which can reduce the weight and increase the aesthetics.
- the processing unit when processing a frame of positioning image, the processing unit obtains the feature codes and image position data of all positioning mark images therein.
- the image processing methods used are:
- the following example illustrates the image processing method.
- the processing unit performs the following image processing:
- the constituent pixel is 1, so the feature code is recorded as 1; the image position data is recorded as (5, 6).
- candidate pixels (10, 12), (10, 13), (11, 12), (11, 13) are left.
- the image processing process ends, and the acquired image data are: feature code 1 and corresponding image position data (5, 6);
- the infrared light-emitting diodes are replaced with light-emitting diodes that emit visible light. It can also provide positioning marks for the air mouse.
- the image acquisition unit of the air mouse is an image acquisition unit that can shoot visible light.
- the luminous flux of the light emitting diode is 10 lumens.
- This embodiment provides an air mouse, which is different from the air mouse in the first embodiment in the method of acquiring image data.
- the processing unit of the air mouse in this embodiment adopts the following image processing method for the positioning image:
- the infrared sensor strip is shown in Figure 11 and includes a bracket and two infrared light sources.
- the bracket 300 is rectangular, with a rectangular hole at each end of the front surface.
- the length of the two rectangular holes is 4 mm, and the width is 1 mm.
- the left end is a longitudinal hole, and the right end is a horizontal hole.
- the luminous flux of the two infrared light-emitting diodes is 10 lumens.
- the two rectangular holes that can emit infrared light are two infrared light sources. It can be used as two positioning marks to form an air mouse system with the air mouse of this embodiment.
- the 4 pixels with coordinates (119,316), (120,316), (121,316), (122,316) have a pixel value of 100, and the pixels of the remaining pixels The value is 0.
- the processing unit uses the following methods to obtain feature codes and image location data.
- the column coordinates are the pixel values of all pixels in the range of 313 to 319.
- the result is that there are 4 pixels with pixel values above 40, and the coordinates are (119, 316), (120, 316), (121, 316), (122, 316).
- the feature code is recorded as 1.
- the coordinate of the pixel with the smallest coordinate value is (119, 316), which is the image position data corresponding to feature code 1.
- the above-mentioned processing method obtains the feature code and image position data of one positioning mark image in one positioning image.
- the processing unit when it processes one frame of positioning image, it acquires the feature codes and image position data of all the positioning mark images therein.
- the specific method is as follows:
- the image code is recorded as 1; if the row coordinates of all the constituent pixels are the same, the image code is recorded as 2. Then, among all the constituent pixels, find a pixel with the smallest coordinate value, and record the coordinate of this pixel as an image position.
- the image processing method in the step of acquiring image data is shown in FIG. 12.
- the image acquisition unit there are 8 pixels with a pixel value of 100.
- the coordinates are (119, 3), (120, 3), (121, 3), (122, 3), (120). , 315), (120,316), (120,317), (120,318).
- the first 4 pixels form the location mark A image
- the last 4 pixels form the location mark B image.
- the processing unit uses the following methods to obtain feature codes and image location data:
- the pixel with the smallest coordinate value is selected, and its coordinate is (120,315).
- This pixel is the center, read the pixel values of all pixels in the surrounding 7 pixels ⁇ 7 pixels range, and find 4 pixels with a pixel value of 20 or more as the constituent pixels (the 4 pixels are all the pixels that make up the positioning mark B image) ,
- the coordinates are (120,315), (120,316), (120,317), (120,318).
- the row coordinates of the 4 constituent pixels are the same, and the feature code is recorded as 2. Find a pixel with the smallest coordinate value among the 4 constituent pixels, and its coordinate is (120,315), and record this coordinate as the corresponding feature code 2 Image location record.
- the acquired image data includes feature code 1 and corresponding image location data (119, 3); feature code 2 and corresponding image location data (120, 315).
- the infrared sensor strip of this embodiment includes a bracket and three infrared light sources.
- the bracket is rectangular, 45 cm long, 2 cm high, and 2 cm thick.
- the length of the three rectangular holes is 4 mm, and the width is 1 mm.
- the rectangular holes at both ends are vertical; the middle rectangular hole is horizontal.
- the distance between the holes at both ends and the middle hole is 20cm.
- the luminous flux of the three infrared light-emitting diodes is 10 lumens.
- the three rectangular holes capable of emitting infrared light are three infrared light sources, which can be used as three positioning marks to form an air mouse system with the air mouse of this embodiment. Compared with the solution of two positioning marks, it can further increase the moving length of the air mouse.
- the infrared sensor strip of this embodiment includes a bracket and multiple infrared light sources.
- the bracket 300 is rectangular, 25 cm long and 8 cm high.
- There is a longitudinal rectangular hole in the upper left corner with a length of 4 mm and a width of 1 mm;
- the upper right corner has a horizontal rectangular hole with a length of 4 mm and a width of 1 mm;
- the lower left corner has two longitudinal rectangular holes, the two rectangular holes Arranged in parallel, with a distance of 1 mm, a length of 2 mm, and a width of 1 mm;
- there are two horizontal rectangular holes in the lower right corner the two rectangular holes are arranged in parallel, with a distance of 1 mm, the length is 2 mm, and the width is both It is 1 mm.
- the luminous flux of the infrared light-emitting diode is 10 lumens.
- the rectangular hole on the bracket that can emit infrared light can be used as a positioning mark.
- the vertical rectangular hole in the upper left corner is used as one; the horizontal rectangular hole in the upper right corner is used as one; the two longitudinal rectangular holes in the lower left corner are used as one; the two in the lower right corner are used as one.
- As one horizontal rectangular hole there are 4 positioning marks in total.
- the image of each positioning mark is composed of 4 pixels, and the arrangement methods are different.
- the image formed by the positioning mark in the upper left corner is composed of 4 pixels vertically arranged in a row
- the image formed by the positioning mark in the upper right corner is composed of 4 pixels horizontally arranged in a row
- the positioning mark in the lower left corner is composed of 4 pixels Arranged in two rows longitudinally, each row has 2 pixels, the distance between the two rows is 1 pixel
- the positioning mark at the lower right corner has an image of 4 pixels arranged horizontally in two rows, each row has 2 pixels, and the distance between the two rows
- the distance is 2 pixels; for example, the positioning mark in the lower left corner, the coordinates of the 4 pixels that make up the image are (0, 0), (0, 2), (1, 0), (1, 2); the lower right corner
- the positioning mark of, the coordinates of the 4 pixels that make up the image are (0, 0), (0, 1)
- the rule for the processing unit to determine the feature code is that if the column coordinates of all pixels found with the pixel value of 40 are the same, the feature code is recorded as 1; if the row coordinates are the same, the feature code is recorded as 2; if the column coordinates are the same in pairs , And two pixels with the same column coordinate, the row coordinate value is adjacent, then the feature code is recorded as 3; if the column coordinate is the same two by two, and the column coordinate is the same two pixels, the row coordinate value If it is not a contiguous number, record the feature code as 4.
- the infrared light-emitting diodes are replaced with light-emitting diodes, which can also provide positioning marks for the air mouse.
- the image acquisition unit of the air mouse is an image acquisition unit that can photograph visible light scenes.
- the luminous flux of the light emitting diode is 10 lumens.
- This embodiment provides an air mouse, and the difference from the air mouse in the first embodiment lies in the processing method of the positioning image.
- the processing unit of the air mouse of this embodiment processes each frame of positioning image, it first performs image processing, and then sends image position data or a rebranding signal to the controlled device according to the image processing result.
- Image processing consists of two steps:
- the first step is to obtain the feature code and image location data of the positioning mark image.
- the image processing process ends.
- the type of data sent by the processing unit to the controlled device is also different. If the same feature code is found, the processing unit sends the image position data corresponding to the feature code; if the same feature code is not found, the processing unit sends a rebrand signal.
- the processing method of the positioning image by the processing unit is shown in FIG. 15.
- This embodiment also provides a driver program installed on the controlled device of the air mouse for controlling the movement of the cursor on the driven screen according to the image position data or the conversion signal.
- the driver has a preset recording area for recording image position data.
- receives the image position data if there is image position data in the recording area, compare the received image position data with that in the recording area, and control the cursor movement on the screen according to the obtained image displacement data, and Update the data in the recording area to the image location data just received; if there is no image location data in the recording area, store the image location data just received in the recording area.
- receives the rebranding signal it will clear the recording area.
- the processing method of the driver of this embodiment when the image position data is received is as shown in FIG. 16.
- the air mouse in this embodiment adopts various methods described in Embodiment 1, Embodiment 2, and Embodiment 3 when acquiring feature codes and image position data from a positioning image.
- This embodiment also provides an air mouse system, which is composed of the air mouse in this embodiment and at least two positioning marks with different characteristics.
- the positioning marks include various positioning marks recorded in the first, second, and third embodiments.
- the image acquisition unit outputs the first frame positioning image, and the processing unit acquires the feature code 50 and corresponding image position data (120, 318) from it. Because it is the first frame, there is no record of the previous frame. Therefore, the number 1 is sent to the controlled device, and the number 1 is the standard change signal. The driver of the controlled device will clear the data in the recording area after receiving the rebranding signal.
- the image acquisition unit outputs the second frame positioning image, from which the processing unit obtains the feature code 50 and the corresponding image position data (120, 317). According to the feature code 50, the same is found in the feature code obtained from the positioning image of the previous frame. Then (120,317) is sent to the controlled device. After receiving the image displacement data, the driver of the controlled device will record (120,317) in the recording area because the recording area is empty.
- the image acquisition unit outputs the third frame positioning image, from which the processing unit obtains the feature code 50 and the corresponding image position data (120, 315). According to the feature code 50, the same is found in the feature code obtained from the positioning image of the previous frame. Then send (120,315) to the controlled device. After receiving the image displacement data, the driver program of the controlled device compares (120,315) with (120,317), and finds that the value of the column coordinate is reduced by 2. The cursor on the control screen has moved 2 pixels to the right. Then update the data in the recording area to (120,,315)
- the image acquisition unit outputs the fourth frame positioning image, and the processing unit acquires the feature code 100 and corresponding image position data (120, 312) from it. According to the feature code 100, the same is not found in the feature code of the positioning image of the previous frame, so the number 1 is sent to the controlled device. The driver of the controlled device will clear the data in the recording area.
- the image acquisition unit outputs the fifth frame positioning image, and the processing unit obtains the feature code 100 and corresponding image location data (120, 310) from it. According to the feature code 100, the same is found in the feature code of the previous frame of positioning image. (120, 310) is sent to the controlled device, and the driver of the controlled device writes it into the recording area.
- the image acquisition unit outputs the sixth frame positioning image, and the processing unit obtains the feature code 100 and the corresponding image location data (120, 308) from it. According to the feature code 100, the same is found in the feature code of the previous frame of the positioning image. (120,308) is sent to the controlled device, the driver compares (120,308) with (120,310), and finds that the value of the column coordinate is less than 2, and the cursor on the control screen moves to the right by 2 Pixels.
- the driver controls the movement of the cursor on the screen according to the comparison result.
- the following rules are adopted: if the value of the row coordinate increases by n, the cursor on the screen is controlled to move up n pixels; if the row coordinate If the value of is reduced by n, the cursor on the control screen moves down by n pixels. If the value of the column coordinate increases by n, the cursor on the control screen moves n pixels to the left; if the value of the column coordinate decreases by n, the cursor on the control screen moves n pixels to the right; if the row coordinate and column If the values of the coordinates have changed, control the cursor to move correspondingly in two directions. For example, if the measured values of row coordinates and column coordinates are increased by 1, the cursor on the control screen will move up by 1 pixel, and then move to the left by 1 pixel.
- the driver and the controlled device constitute a computer system.
- the cursor on the screen controlled by the air mouse does not move when the positioning mark used as the air position reference object is replaced. After the reference object is replaced, the cursor continues to move in the original direction.
- the air mouse of this embodiment can also increase the moving length by adding positioning marks.
- This embodiment provides an air mouse, and the difference from the air mouse in the first embodiment lies in the processing method of the positioning image.
- the processing unit of the air mouse of this embodiment processes each frame of positioning image, it obtains image displacement data through image processing.
- Step 1 Obtain the feature code and image location data of the positioning mark image.
- the specific methods include the various methods described in Embodiment 1, Embodiment 2, and Embodiment 3.
- Step 2 Obtain image displacement data.
- the specific method is as follows: according to the acquired characteristic codes, look for the same in the characteristic codes acquired in the processed previous frame positioning image. If there are the same, the corresponding image position data of the feature code in the two frames of positioning images are compared, and the image displacement data is obtained according to the comparison result; if there is no the same, the image processing of the positioning image of this frame ends.
- the image processing method is shown in FIG. 17.
- the processing unit sends a displacement signal to the controlled device according to the image displacement data, and the controlled device controls the cursor on the driven screen to move correspondingly according to the displacement signal.
- the air mouse moves from left to right in the air as an example to illustrate the air mouse and image processing method of this embodiment.
- the air mouse and two positioning marks form an air mouse system.
- the two positioning marks are two infrared light sources.
- the implementation is the same as that shown in Figure 2.
- the two infrared light sources are arranged horizontally as positioning mark A and positioning mark B respectively.
- the image processing program used by the processing unit must first establish a recording area for recording the data (including feature codes and image position data) obtained from the positioning image, and update the record at the end of each frame positioning image processing process Data in the area.
- the image acquisition unit starts to capture and output the first frame of positioning image.
- the four pixels with coordinates (120,318), (120,319), (121,318), (121,319)
- the value is 50
- the pixel value of the remaining pixels is 0.
- the image data obtained by the processing unit from this frame positioning image is 50 and (120, 318).
- the processing unit searches for the same in the recording area according to the feature code 50. Because this is the first frame positioning image processed, there is no data in the recording area. After storing 50 and (120, 318) in the recording area, the first frame positioning image processing ends.
- the processing unit obtains the feature code 50 and image location data (120, 317) from this frame positioning image.
- the processing unit found the same in the recording area according to the feature code 50. Comparing (120,317) with (120,318), it is found that the value of the column coordinate is reduced by 1. This is the image displacement data.
- the processing unit sends a displacement signal to the controlled device, and the information contained in the displacement signal is 1 to the right.
- the frame positioning image processing ends.
- the controlled device controls the cursor on the driven screen to move 1 pixel to the right (the screen faces the user) according to the displacement signal.
- the image acquisition unit continues to capture and output positioning images.
- the position of the positioning mark A image gradually moves to the left, and the processing unit continues to output displacement signals according to the image processing results ,
- the controlled device controls the cursor on the screen to continuously move to the right according to the displacement signal.
- the processing unit obtains the feature code 50 and the corresponding image position data (120, 0) from the positioning image. At this time, the positioning mark A has reached the visibility of the lens of the image acquisition unit The edge of the range. After the frame positioning image processing is completed, the data in the recording area is 50, (120, 0).
- the image acquisition unit After the user holds the air mouse and moves to the right for a certain distance, there is only the positioning mark B in the lens. At this time, the image acquisition unit outputs a positioning image of the n+1th frame, in which four pixels have a pixel value of 100, and the remaining pixels have a pixel value of 0.
- the four pixels constitute the positioning mark B image, and the coordinates of the four pixels are (120, 317), (120, 318), (121, 317), (121, 318), respectively.
- the image data obtained by the processing unit in the frame positioning image is 100 and (120, 317). According to the feature code 100, the same is not found in the recording area. After updating the data in the recording area to 100, (120, 317). The image processing process of this frame positioning image ends.
- 4 pixels After the user holds the air mouse and continues to move a small distance to the right, in a frame of positioning image output by the image acquisition unit, 4 pixels have a pixel value of 100 and the remaining pixels have a pixel value of 0.
- the coordinates of the 4 pixels are (120, 316), (120, 317), (121, 316), (121, 317), respectively.
- the processing unit After the processing unit performs image processing on the frame positioning image, it sends a displacement signal moving 1 to the right to the controlled device, and the controlled device controls the cursor on the screen to move 1 pixel to the right.
- the air mouse continues to move to the right, and the cursor on the screen also continues to move to the right, until the positioning mark B moves out of the visual range of the lens.
- the method of obtaining the image displacement data according to the feature code and the image position data is the same as the method of obtaining the image displacement data according to the feature code and the image position data by the driver in the first embodiment.
- the obtained image displacement data is also used as the basis for controlling the cursor movement on the screen, so the beneficial effects are also the same.
- the row coordinates and column coordinates are compared separately to find out the changes in the values.
- the comparison result includes four situations: row coordinates increase, row coordinates decrease, column coordinates increase, and column coordinates decrease.
- the air mouse of this embodiment sends a displacement signal according to the image displacement data.
- the rule adopted is: if the value of the row coordinate increases by n, it outputs a displacement signal that moves upward by n; if the value of the row coordinate decreases by n, it outputs to Move down the displacement signal of n.
- the controlled device controls according to the received displacement signal
- the cursor on the driven screen moves n pixels in the corresponding direction. If the values of row coordinates and column coordinates have changed, they must be reflected in the displacement signal. For example, if the measured values of the row coordinate and the column coordinate both increase by 1, the information contained in the output displacement signal is 1 move up and 1 left.
- the control device receives this displacement signal and controls the cursor on the screen to move up by 1 pixel, and then move to the left by 1 pixel.
- the specific format of the displacement signal can refer to the data format of the optical mouse.
- the rule for sending displacement signals according to the image displacement data is: when the value of the column coordinate increases, the displacement signal that moves to the left is output; when the value of the column coordinate decreases, the output is Displacement signal for right movement.
- the moving direction of the cursor on the screen is opposite to that of the air mouse.
- the direction of longitudinal movement can also be modified in the same way.
- the processing unit multiplies the image displacement data by 2 and outputs it. For example, the measured value of the column coordinate is reduced by 1, and the displacement signal of 2 to the right is output. base on needs. You can also multiply the image displacement data by 3, 4 or more and output it.
- the air mouse includes a housing, an image acquisition unit, and a processing unit.
- the image acquisition unit is used to photograph a scene including the positioning mark and output a positioning image with the positioning mark image; the processing unit performs image processing on the positioning image, and sends a cursor control signal to the controlled device according to the image processing result.
- the processing unit performs image processing on the positioning image, and sends a cursor control signal to the controlled device according to the image processing result.
- it includes obtaining the feature code of the positioning mark image and the image position data.
- the air mouse of this embodiment is equivalent to removing the button unit on the basis of the air mouse of the first to fifth embodiments.
- Embodiment 1 When the processing unit of the air mouse in this embodiment performs image processing on a frame of positioning image, various image processing methods recorded in Embodiment 1 to Embodiment 5 can be used.
- the volume can be made relatively small.
- a cube with a side length of 5 cm is used as the housing, and a hole is opened on one surface to place the lens of the image capture unit.
- the air mouse of this embodiment is particularly suitable for smart mobile devices, and the smart mobile devices include smart phones and tablet computers.
- the player holds the phone with the screen facing the player.
- the lens is located on the side opposite to the sticking surface, that is, the lens faces the front of the player (the same as the back of the phone)
- the processing unit is connected to the mobile phone through a data cable. In front of the player, place a positioning mark (one or more can be placed as needed) where the camera can shoot.
- Players can control the cursor movement on the screen by moving their mobile phone in the air.
- the processing unit is divided into an image processing unit and a main control unit.
- the air mouse of this embodiment includes a housing, an image acquisition unit, an image processing unit, a button unit, and a main control unit.
- the image acquisition unit is used to photograph the scene including the positioning mark and output the positioning image with the positioning mark image; the image processing unit performs image processing on the positioning image and sends the image processing result to the main control unit; the main control unit according to the image processing result
- the cursor control signal is sent to the controlled device; the key signal generated by the key unit is sent to the controlled device through the main control unit; when the image processing unit processes the positioning image, the various processing units used in the first to fifth embodiments are used Image processing method.
- Embodiment 1 When the image processing method adopted by the image processing unit is any one of Embodiment 1, Embodiment 2 or Implementation 3, a set of image data (including feature codes and image position data) will be obtained every time a frame of positioning image is processed.
- the image data is sent to the main control unit and sent to the controlled device by the main control unit.
- the image processing method adopted by the image processing unit is any one of the fourth embodiment, every time a frame of positioning image is processed, the obtained image position data or conversion signal will be sent to the controlled device through the main control unit.
- each frame of positioning image is processed, the obtained image displacement data is sent to the main control unit.
- the main control unit sends a displacement signal to the controlled device according to the image displacement data.
- the rules for sending the displacement signal according to the image displacement data are the same as those in the fifth embodiment.
- the image processing unit of the air mouse in this embodiment can be served by an image processing chip.
- the image processing chip can be a general-purpose microprocessor (such as a single-chip microcomputer) or a digital signal processor (DSP).
- the image processing chip is a STMicroelectronics STM32F407 chip, and the input terminal is electrically connected with the image acquisition unit, and the output terminal is electrically connected with the main control unit.
- the image processing chip may also use other microprocessors, such as a single-chip microcomputer or DSP.
- the image processing chip in this embodiment adopts the image processing method of the present invention.
- the air mouse using the image processing chip can increase the moving length by adding positioning marks.
- This embodiment provides a cloud game system.
- the air mouse includes a housing, an image acquisition unit, a key unit, and a processing unit; the image acquisition unit is used to shoot scenes including positioning marks and output positioning images with positioning marks; the key unit generates key signals and sends them to the client through the processing unit Device sending; the processing unit performs image processing on the positioning image, and sends the cursor control signal to the client device according to the image processing result.
- the processing unit processes the positioning images using various image processing methods in the first to fifth embodiments of the present invention.
- the client device sends the cursor control signal and the key signal to the cloud server.
- the cloud server is used to run the game program, generate screen display data according to the cursor key signal and the key signal, and send it to the client device.
- the client device controls the display content on the screen of the driven display device according to the screen display data to make corresponding changes.
- the cursor control signals output by the air mouse are also different.
- the cloud server When the received cursor control signal is a displacement signal, the cloud server generates screen display data according to the displacement signal.
- the cloud server is provided with the driver program described in the first embodiment, the image displacement data is obtained through the driver program, and the screen display data is generated according to the displacement data.
- the cloud server is provided with the driver program described in the fourth embodiment, the image displacement data is obtained through the driver program, and the screen display data is generated according to the image displacement data.
- the client device is a PC that drives a liquid crystal display.
- the cloud server runs a shooting game, and the air mouse controls the front sight on the screen.
- the air mouse sends a displacement signal to the PC (the information contained is 10 pixels to the right), the PC sends the signal to the cloud server, and the cloud server generates a video data packet through calculation (the video data packet includes the screen All data required for 1 frame of screen is displayed on the screen).
- the video data packet is sent to the PC, and the PC drives the liquid crystal display based on the data in the video data packet, and the liquid crystal display displays the picture after the crosshair moves 10 pixels to the right.
- the air mouse continuously sends out displacement signals, and the picture on the screen also changes continuously.
- the air mouse sends a button signal to the PC, and the button signal contains the command to change the magazine.
- the key signal is sent to the cloud server through the PC.
- the cloud server generates 50 video data packets through calculations and sends them to the PC continuously. Each video data packet includes the data needed to display a frame of picture. Every time the PC receives a video data packet, it drives the LCD to display one frame.
- the picture, 50 frames of pictures connected together is the continuous action of a character in the game changing magazines.
- the displacement signal sent by the air mouse of the present invention can be in the following format: 1, 2, 3, and 4 are used to represent the four moving directions of up, down, left, and right, and a number is added to indicate the number of pixels moved.
- the cloud server outputs video data packets according to the received displacement signal, and controls the display content on the screen. For example, when receiving 2+5, the cloud server controls the screen driven by the PC to display the screen after the cursor moves down 5 pixels; when receiving 3+2, it controls the screen to display the screen after moving the cursor 2 pixels to the left.
- the cloud server controls the screen display content according to the image displacement data, and adopts the following rules: when row coordinates increase, row coordinates decrease, column coordinates increase, and column coordinates decrease, the cursor on the screen is controlled up, down, left, and Moving in 4 directions to the right, the value of the movement is equal to the value of the coordinate change.
- the image displacement data adds 5 to the row coordinate
- the cloud server controls the screen driven by the PC to display the screen after the cursor has moved up by 5 pixels.
- the controlled devices of the cloud game system in this embodiment are various types of electronic computer devices, including but not limited to PCs, video game consoles, smart TVs, tablet computers, and smart phones.
- the driver and the cloud server form a computer system.
- the moving length of the air mouse can be increased by adding a positioning mark, so that the air mouse can have a higher moving length and moving accuracy at the same time, thereby increasing the gaming experience.
- This embodiment provides an air mouse, which is improved on the basis of the air mouse in the fifth embodiment.
- the difference lies in the method that the processing unit sends a displacement signal to the controlled device after processing each frame of the positioning image.
- the processing unit after processing each frame of positioning image, if image displacement data is generated, the processing unit sends a displacement signal to the controlled device according to the image displacement data; if no image displacement data is generated, the processing unit The control device sends the displacement signal generated after the positioning image processing of the previous frame (if the displacement signal is generated after the positioning image processing of the previous frame).
- the nth frame of positioning image is output during the process of the air mouse moving to the right.
- the displacement signal sent by the processing unit to the controlled device is X+10 (moving to the right by 10 on the screen). Pixels).
- the air mouse continues to move to the right and outputs the n+1th frame positioning image.
- the feature code obtained by the processing unit from the positioning image of this frame changes, and no image displacement data is generated after the positioning image processing of this frame is completed.
- the processing unit sends a displacement signal X+10 to the controlled unit (the same as the displacement signal output after processing the n-th frame positioning image).
- the displacement reference object positioning mark
- the feature code of the positioning mark image is the same as the one in the previous frame of positioning image. are different. No image displacement data is generated, and no displacement signal is generated. That is, in this time period between the output of the two frames of positioning images, the movement of the air mouse cannot be reflected on the display screen of the controlled device, which affects the accuracy of cursor control.
- the time between the output of two positioning images is regarded as a time period, the moving direction and speed of the air mouse in the two adjacent time periods are very close in most cases.
- the air mouse of this embodiment sends the displacement signal generated after the positioning image processing of the previous frame to the controlled device when there is no displacement data generated.
- the positioning reference object is switched, the direction of the air mouse movement and The distance can be correctly reflected on the display screen, which increases the accuracy of control.
- This embodiment provides a driver program, which is improved on the basis of the driver program in Embodiment 1.
- the processing method adopted by the driver program in this embodiment when each group of image data is received is:
- the feature code find the same in the last group of image data received. If the same is found, the image position data corresponding to the feature code in the two sets of image data are compared to obtain image displacement data, and the movement of the cursor on the screen is controlled according to the image displacement data. If the same is not found, the movement of the cursor on the screen is controlled according to the image displacement data obtained after processing the last set of image data received (if the data exists).
- the driver is installed on the PC, and the user holds the air mouse to move laterally.
- the nth group of image data received by the driver includes feature code 1 and an image position data.
- the feature code is the same as that in the n-1th group of image data.
- the obtained image displacement is that the column coordinate is reduced by 10
- the cursor on the control screen moves 10 cursors to the right;
- the obtained (n+1)th set of displacement data includes feature code 2 and an image position data.
- the cursor on the screen is still controlled to move 10 cursors to the right. Then start processing the n+2 group of image data.
- the beneficial effect of the driver program of this embodiment is that the movement of the matching air mouse when the displacement reference object (positioning mark) is switched can also be reflected on the display screen, which increases the accuracy of control.
- the displacement reference object changes during the movement of the air mouse, there will be a certain distance of movement that cannot be reflected on the screen.
- This embodiment provides a driver program, which is improved on the basis of the driver program provided in the fourth embodiment.
- the air mouse in the fourth embodiment will send a set of data to the controlled device after processing a frame of positioning image.
- This set of data is image position data or a rebranding signal, and the driver on the controlled device controls according to the set of data The movement of the cursor on the screen.
- the driver of this embodiment will set a recording area for recording image position data.
- the driver when the driver receives the image position data, if there is image position data in the recording area, compare the received image position data with that in the recording area, and control the screen based on the obtained image displacement data.
- the cursor moves, and the data in the recording area is updated to the image position data just received; if there is no image position data in the recording area, the image position data just received is stored in the recording area.
- the driver receives the rebranding signal, it will clear the recording area.
- the processing of this group of data ends. If the image displacement data is not generated in the first step, the cursor on the screen is controlled to move in the same direction and distance according to the result of the movement of the cursor on the screen after the processing of the last set of data received.
- the user holds the air mouse to move laterally, and sends a set of data to the controlled device every time a frame of positioning image is processed.
- this group of data is the image position data.
- the cursor on the control screen has moved 5 pixels to the right.
- this group of data is a rebranding signal.
- the driver receives the n+2 group of data, which is the image position data.
- the first step is processed and no image displacement data is generated, then in the second step, it is processed according to the previous group (n+1 group) data After finishing the result, control the cursor on the screen to move 5 pixels to the right.
- this group of data is image position data.
- the image displacement data is generated. The driver controls the cursor on the screen to move correspondingly according to this data.
- the air mouse when the reference object (location mark) changes during the movement of the air mouse, the air mouse has a certain moving distance (the distance moved within the time of outputting 2 frames of positioning images). Reflected on the display screen, affecting the accuracy of cursor control.
- the driver program of this embodiment completely compensates for this shortcoming and improves the accuracy of cursor control.
- This embodiment provides an image processing method for a computer. It is used for image processing of the positioning image on the computer side, and the computer controls the movement of the cursor on the screen of the driven display device according to the image processing result.
- the computer is a PC, which drives a liquid crystal display.
- the PC is connected to an air mouse.
- the composition of the air mouse is the same as the air mouse recorded in the first embodiment.
- the processing unit of the air mouse sends the positioning image to the connected device after receiving the positioning image output by the image acquisition unit.
- An image processing software is installed on the PC, and the image processing software performs image processing on the received positioning image.
- the image processing method is the same as the image processing method adopted by the air mouse in the fifth embodiment. After the image processing software processes a frame of positioning image, if the image displacement data is obtained, the cursor on the LCD screen driven by this data is controlled to move accordingly. To control the cursor movement on the screen according to the image displacement data, the rules adopted are the same as those recorded in the first embodiment.
- the air mouse held by the user continuously sends positioning images to the connected PC.
- the image processing software used by the computer in this embodiment obtains the image displacement data according to the displacement of the positioning mark images with the same image characteristics in the two frames of the positioning images. And according to the image displacement data, the cursor on the screen is controlled to move accordingly.
- the processing unit controls the cursor on the screen to move accordingly according to the image displacement data; if If no image displacement data is generated, the cursor on the screen is controlled to move in the same direction and distance according to the result of the cursor movement after the positioning image processing of the previous frame.
- the user holds the air mouse and moves it to the right.
- the n-th frame positioning image is output.
- the cursor on the screen is controlled to move 10 pixels to the right.
- the air mouse continues to move to the right, it outputs the n+1th frame positioning image.
- the feature code obtained by the image processing software from the positioning image of this frame has changed (the displacement reference object has changed at this time).
- Image displacement data is generated.
- the image processing software controls the cursor on the screen to move 10 pixels to the right (the direction and distance of the cursor movement after the positioning image processing of the nth frame is the same).
- the computers in this embodiment are various electronic computer equipment, including but not limited to PCs, video game consoles, smart TVs, and cloud game servers.
- the computer and the image processing software used constitute a computer system, which can control the movement of the cursor on the driven screen according to the positioning image.
- the computer is a PC.
- the computer is a video game machine
- the installed image processing software performs image processing on the received positioning image, and controls the cursor on the driven screen to move according to the processing result .
- the computer is a smart TV
- the installed image processing software performs image processing on the received positioning image, and controls the cursor on the TV screen to move according to the processing result.
- the computer is a cloud game server.
- the positioning image is sent to the server, and the image processing software on the server performs image processing, and then controls the cursor movement on the driven screen according to the processing result.
- the image processing method in this embodiment can enable a computer using the image processing method to increase the moving length by adding positioning marks.
- the computer system in this embodiment because of the image processing software used, can increase the moving length of the cursor on the driven screen by adding positioning marks.
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Position Input By Displaying (AREA)
Abstract
The present invention discloses an air mouse, an air mouse system, an image processing method and a control method, wherein the image processing method includes the following steps: (a) obtaining a positioning image with a positioning mark image; and (b) determining a feature code and image position data of the positioning mark image by reading the pixel value of the pixel in the positioning image. The air mouse using the image processing method can increase the moving length by adding positioning marks.
Description
本发明涉及计算机及电子游戏控制领域,特别是一种用于空中鼠标的图像处理方法、采用了该图像处理方法的芯片、空中鼠标、空中鼠标系统、空中鼠标控制系统、用于智能移动设备的空中鼠标、空中鼠标的驱动程序及应用了该程序的计算机系统、为空中鼠标提供定位标志的红外感应条、云游戏系统、用于计算机的图像处理方法。The present invention relates to the field of computer and electronic game control, in particular to an image processing method for an air mouse, a chip adopting the image processing method, an air mouse, an air mouse system, an air mouse control system, and a device for intelligent mobile equipment Air mouse, air mouse driver and computer system using the program, infrared sensor strip providing positioning mark for air mouse, cloud game system, image processing method for computer.
鼠标是最常见的计算机及电子游戏控制设备,其主要的用途是控制电视机、显示器等显示设备屏幕上光标的移动。光标泛指屏幕上的鼠标指针、电子游戏中的武器准星、人物角色等各种跟随鼠标的移动而移动的图标。The mouse is the most common computer and electronic game control device. Its main purpose is to control the movement of the cursor on the screen of display devices such as televisions and monitors. The cursor refers to various icons that move with the movement of the mouse, such as the mouse pointer on the screen, the weapon crosshairs in video games, and the characters.
移动精度和移动长度是衡量鼠标性能的两个重要方面,其中移动精度指的是鼠标单向移动一次能够控制屏幕上的光标移动的最小像素数;移动长度指的是鼠标单向移动一次能够控制屏幕上的光标移动的最大像素数。Movement accuracy and movement length are two important aspects to measure the performance of a mouse. The movement accuracy refers to the minimum number of pixels that the mouse can control the cursor on the screen by one-way movement; the movement length refers to the one-way movement that can be controlled by the mouse once. The maximum number of pixels the cursor moves on the screen.
常见的光学鼠标在移动精度和移动长度方面都具备较高的水平,光学鼠标在桌面上作一次轻微移动,可以使光标在屏幕上只移动1个像素;朝一个方向一直移动,可以将光标从屏幕的一个边缘移动到相对的另一个边缘。光学鼠标用于控制1920×1080或更高分辨率的屏幕,都能够兼具移动精度和移动长度。The common optical mouse has a high level of movement accuracy and movement length. A slight movement of the optical mouse on the desktop can make the cursor move only 1 pixel on the screen; if it keeps moving in one direction, the cursor can be moved from One edge of the screen moves to the opposite edge. The optical mouse is used to control a screen with a resolution of 1920×1080 or higher, which can have both movement precision and movement length.
空中鼠标是另一种用于对屏幕上的光标进行控制的设备。和光学鼠标不同的是,空中鼠标通过在自身在空中的移动来控制屏幕上的光标进行移动。The air mouse is another device used to control the cursor on the screen. Unlike an optical mouse, an air mouse controls the cursor on the screen to move by moving it in the air.
中国专利申请号201210319834.X公布了一种采用了红外图像识别方案的空中鼠标,该空中鼠标通过一个红外光发射器发出红外光,在本体处通过红外图像感应器对红外光进行感应并形成一系列图像,图像中有红外光发射器形成的光点。使用者手持本体移动时,在形成的一系列图像中,光点的位置会产生变化。本体根据相邻图像中光点位置的变化向一被控设备发出位置变化数据,被控设备根据该位置变化数据控制所驱动的屏幕上的光标进行移动。Chinese Patent Application No. 201210319834.X discloses an air mouse using an infrared image recognition scheme. The air mouse emits infrared light through an infrared light emitter, and the infrared light is sensed by the infrared image sensor on the body to form a A series of images, in which there are light spots formed by infrared light emitters. When the user holds the main body and moves, the position of the light spot changes in the series of images formed. The main body sends position change data to a controlled device according to the change of the position of the light spot in the adjacent image, and the controlled device controls the cursor on the driven screen to move according to the position change data.
相比光学鼠标,采用红外图像识别方案的空中鼠标在保证较高移动精度的时候,移动长度不能满足主流显示设备的需求。Compared with an optical mouse, an air mouse using an infrared image recognition solution cannot meet the requirements of mainstream display devices while ensuring a higher movement accuracy.
受图像处理性能的限制,空中鼠标的红外图像感应器输出的图像分辨率较低,例如320×240。在移动精度为1的情况下(即单次移动最少可以控制屏幕上的光标移动1个像素),本体单向移动一次,光标在屏幕上的横向移动长度不超过320,纵向移动长度不超过240。而现有的液晶显示器及电视等主流显示设备,其分辨率在1920×1080及以上。现有技术的空中鼠标在用于此类显示设备时,横向移动一次最多能控制光标在屏幕上移动六分之一的距离;纵向移动一次最多能控制光标在屏幕上移动四分之一的距离。远远不能满足需要。而为了满足主流显示设备的需求,本体部分可以通过放大位置变化数据的输出比例的方式增加移动长度。例如测得光点在相邻两帧图像中位置的变化为1个像素,则控制屏幕上的屏幕移动6个像素。此时横向移动长度可以达到1920左右,能够满足使用需求。但移动精度由1降低到了6,即本体移动一次,屏幕上光标移动的最小距离为6个像素,这样的移动精度会导致极差的体验。所以现有的采用红外图像识别方案的空中鼠标,在用于主流分辨率的显示设备时,无法同时在移动精度及移动长度方面满足需求,这就使得空中鼠标的应用受到了极大的限制。Due to the limitation of image processing performance, the image resolution output by the infrared image sensor of the air mouse is relatively low, such as 320×240. In the case of a movement accuracy of 1 (that is, a single movement can control the cursor on the screen to move 1 pixel at least), the body moves once in a single direction, and the horizontal movement length of the cursor on the screen does not exceed 320, and the vertical movement length does not exceed 240 . The existing mainstream display devices such as LCDs and TVs have a resolution of 1920×1080 and above. When the air mouse of the prior art is used in this type of display device, a horizontal movement can control the cursor to move up to a sixth of the distance on the screen; a vertical movement can control the cursor to move up to a quarter of the distance on the screen. . Far from meeting the needs. In order to meet the needs of mainstream display devices, the body part can increase the moving length by enlarging the output ratio of the position change data. For example, if the change in the position of the light spot in two adjacent frames of images is measured to be 1 pixel, the screen on the screen is controlled to move 6 pixels. At this time, the lateral movement length can reach about 1920, which can meet the needs of use. But the movement accuracy is reduced from 1 to 6, that is, the body moves once, and the minimum distance for the cursor to move on the screen is 6 pixels. Such movement accuracy will lead to a very poor experience. Therefore, the existing air mouse using the infrared image recognition solution cannot meet the requirements of movement accuracy and movement length at the same time when used in mainstream resolution display devices, which greatly restricts the application of the air mouse.
红外光发射器相当于为空中鼠标的本体提供了一个定位标志,使其能够随时确定自身在空中的位置。但仅采用一个红外光发射器,相当于只有一个定位标志,使得空中鼠标无法同时在移动精度和移动长度两方面保持高水平,要增加移动长度就要降低移动精度。The infrared light transmitter is equivalent to providing a positioning mark for the body of the air mouse, so that it can determine its position in the air at any time. However, only one infrared light transmitter is used, which is equivalent to only one positioning mark, so that the air mouse cannot maintain a high level of movement accuracy and movement length at the same time. To increase the movement length, the movement accuracy must be reduced.
想要在保持移动精度的情况下增加移动长度,增加定位标志的数量是必须的。但现有技术的空中鼠标无法通过增加定位标志数量的方式来增加移动长度。这是受到空中鼠标采用的图像处理方法的限制。该图像处理方法根据相邻两帧图像中定位标志(例如红外光源)形成的影像的位置变化输出位置变化数据,在只有1个定位标志时,进行位置比较的两个影像是同一个定位标志所形成的,影像的位置变化数据能够反映空中鼠标的移动方向和距离;在有2个及以上的定位标志时,进行比较的两个影像就有可能是不同的定位标志形成的,影像的位置变化数据不能正确反映空中鼠标的移动方向和距离,根据该位置变化数据控制屏幕上光标的移动,就可能造成移动方向及距离同空中鼠标实际移动方向和距离不相同。If you want to increase the length of the movement while maintaining the accuracy of the movement, it is necessary to increase the number of positioning marks. However, the prior art air mouse cannot increase the moving length by increasing the number of positioning marks. This is limited by the image processing method used by the air mouse. The image processing method outputs position change data according to the position change of the image formed by positioning marks (such as infrared light source) in two adjacent frames of images. When there is only one positioning mark, the two images for position comparison are the same positioning mark. The image's position change data can reflect the moving direction and distance of the air mouse; when there are 2 or more positioning marks, the two images to be compared may be formed by different positioning marks, and the position of the image changes The data cannot correctly reflect the moving direction and distance of the air mouse. Controlling the movement of the cursor on the screen according to the position change data may cause the moving direction and distance to be different from the actual moving direction and distance of the air mouse.
任天堂公司发布了一种红外感应条,包括一个长条状的支架及设置在支架上的4个红外光源。用该红外感应条为空中鼠标提供定位标志的问题在于,支架上 排列的4个红外光源是相同的,会导致空中鼠标的红外图像感应器输出的图像中,该4个红外光源形成的影像都是一样的。所以该红外感应条虽然可提供4个定位标志,仍无法增加空中鼠标的移动长度。Nintendo has released an infrared sensor strip, including a long strip of bracket and 4 infrared light sources arranged on the bracket. The problem with using this infrared sensor bar to provide positioning marks for the air mouse is that the four infrared light sources arranged on the bracket are the same, which will cause the images output by the infrared image sensor of the air mouse to be all it's the same. Therefore, although the infrared sensor strip can provide 4 positioning marks, it still cannot increase the moving length of the air mouse.
智能手机及平板电脑等智能移动设备,主要用触摸的方式来操作。这对移动游戏的发展形成了限制,因为很多类型的游戏是适合于鼠标操作的,例如FPS、MOBA等。这部分游戏在智能手机或平板电脑上操作时,因触屏操作的方式和玩家习惯的鼠标操作方式相差较远,也不能迅速精确的控制光标,导致体验较差。Smart mobile devices such as smartphones and tablet computers are mainly operated by touch. This limits the development of mobile games, because many types of games are suitable for mouse operation, such as FPS, MOBA, etc. When this part of the game is operated on a smart phone or tablet computer, the touch screen operation method is far from the mouse operation method that players are accustomed to, and the cursor cannot be controlled quickly and accurately, resulting in a poor experience.
现有技术中,通过对红外图像的处理来控制屏幕上的光标还有另一种方案,该方案利用具备红外图像捕捉功能的空中鼠标拍摄红外图像,然后将红外图像发送到一被控设备(电子计算机)。被控设备对收到的红外图像进行图像处理,并根据图像处理的结果控制屏幕上的光标进行移动。和前述方案相比,图像处理的主体由空中鼠标变成了被控设备,但图像处理方法相同,所以在进行光标控制时,仍有相同的缺点。In the prior art, there is another solution to control the cursor on the screen by processing infrared images. This solution uses an air mouse with infrared image capture function to capture infrared images, and then sends the infrared images to a controlled device ( electronic calculator). The controlled device performs image processing on the received infrared image and controls the cursor on the screen to move according to the result of the image processing. Compared with the aforementioned solution, the main body of image processing has changed from an air mouse to a controlled device, but the image processing method is the same, so there are still the same shortcomings when performing cursor control.
本发明在这样的背景下产生。The present invention was produced under this background.
发明内容Summary of the invention
本发明的一个目的是提供一种图像处理方法,其中所述图像处理方法适用于空中鼠标。采用了该图像处理方法的空中鼠标,能够通过增加定位标志的方式来增加移动长度。An object of the present invention is to provide an image processing method, wherein the image processing method is suitable for an air mouse. The air mouse adopting the image processing method can increase the moving length by adding positioning marks.
本发明的另一个目的是提供一种图像处理方法,其中所述图像处理方法用于空中鼠标。采用了该图像处理方法的空中鼠标,能够在不降低移动精度的情况下增加移动长度。Another object of the present invention is to provide an image processing method, wherein the image processing method is used for an air mouse. The air mouse using this image processing method can increase the moving length without reducing the moving accuracy.
本发明的另一个目的是提供一种空中鼠标,能够用增加定位标志的方式来增加移动长度。Another object of the present invention is to provide an air mouse, which can increase the moving length by adding positioning marks.
本发明的另一个目的是提供一种空中鼠标系统,同时具备较高的移动精度及长度。Another object of the present invention is to provide an air mouse system with high moving accuracy and length.
本发明的另一个目的是提供一种空中鼠标系统,能够在移动精度和移动长度两方面都满足主流显示设备(屏幕分辨率在1920×1080左右)的要求。Another object of the present invention is to provide an air mouse system that can meet the requirements of mainstream display devices (screen resolution of about 1920×1080) in terms of movement accuracy and movement length.
本发明的另一个目的是提供一种图像处理芯片,该图像处理芯片可以用于空中鼠标,使得空中鼠标可以通过增加定位标志的方式来增加移动长度。Another object of the present invention is to provide an image processing chip that can be used for an air mouse, so that the air mouse can increase the moving length by adding positioning marks.
本发明的另一个目的是提供一种空中鼠标控制系统,用于控制一显示屏幕上的光标,能够用增加定位标志的方式增加光标在屏幕上的移动长度。Another object of the present invention is to provide an air mouse control system for controlling a cursor on a display screen, which can increase the moving length of the cursor on the screen by adding positioning marks.
本发明的另一个目的是提供一种空中鼠标控制系统,用于控制一显示屏幕上的光标,能够和多个(两个及以上)的定位标志搭配使用,同时实现较高的移动长度和移动精度。Another object of the present invention is to provide an air mouse control system for controlling the cursor on a display screen, which can be used in conjunction with multiple (two or more) positioning marks, while achieving a higher moving length and movement Accuracy.
本发明的另一个目的是提供一种空中鼠标的驱动程序,安装在和空中鼠标连接的被控设备上,能够根据空中鼠标提供的光标控制信号控制屏幕上的光标移动。Another object of the present invention is to provide an air mouse driver, which is installed on a controlled device connected to the air mouse, and can control the cursor movement on the screen according to the cursor control signal provided by the air mouse.
本发明的另一个目的是提供一种计算机系统,能够根据空中鼠标提供的光标控制信号控制被控屏幕上光标的移动。Another object of the present invention is to provide a computer system capable of controlling the movement of the cursor on the controlled screen according to the cursor control signal provided by the air mouse.
本发明的另一个目的是提供一种用于智能移动设备的空中鼠标,能够迅速精确的控制智能移动设备屏幕上的光标及游戏。Another object of the present invention is to provide an air mouse for smart mobile devices, which can quickly and accurately control the cursor and games on the smart mobile device screen.
本发明的另一个目的是提供一种红外感应条,用于为空中鼠标提供定位标志,能够使空中鼠标在不降低移动精度的情况下增加移动长度。Another object of the present invention is to provide an infrared sensor strip for providing positioning marks for the air mouse, which can increase the moving length of the air mouse without reducing the movement accuracy.
本发明的另一个目的是提供一种红外感应条,能够为空中鼠标提供定位标志,使得空中鼠标能够兼具移动长度及精度。Another object of the present invention is to provide an infrared sensor strip that can provide positioning marks for the air mouse, so that the air mouse can have both moving length and accuracy.
本发明的另一个目的是提供一种云游戏系统,其中的空中鼠标能够通过增加定位标志的方式增加移动长度。Another object of the present invention is to provide a cloud game system in which the air mouse can increase the moving length by adding positioning marks.
本发明的另一个目的是提供一种空中鼠标,空中鼠标移动过程中定位标志发生改变的时候,该空中鼠标的移动也能反映在显示屏幕上。Another object of the present invention is to provide an air mouse. When the positioning mark changes during the movement of the air mouse, the movement of the air mouse can also be reflected on the display screen.
本发明的另一个目的是提供一种空中鼠标的驱动程序,能够使空中鼠标移动过程中定位标志发生改变的时候,该空中鼠标的移动也能反映在显示屏幕上。Another object of the present invention is to provide a driver for an air mouse, which enables the movement of the air mouse to be reflected on the display screen when the positioning mark changes during the movement of the air mouse.
本发明的另一个目的是提供一种用于计算机的图像处理方法,采用了该图像处理方法的计算机,能够根据定位图像来控制所驱动的屏幕上的光标进行移动。Another object of the present invention is to provide an image processing method for a computer. The computer adopting the image processing method can control the cursor on the driven screen to move according to the positioning image.
本发明的另一个目的是提供一种计算机系统,能够根据定位图像控制所驱动的屏幕上的光标移动。Another object of the present invention is to provide a computer system capable of controlling the movement of the cursor on the driven screen according to the positioning image.
为实现上述目的,本发明提供了一种图像处理方法,用于空中鼠标。该方法的特征在于,To achieve the above objective, the present invention provides an image processing method for air mouse. The method is characterized by
对一帧定位图像进行处理时,获取其中定位标志影像的特征代码及影像位置数据,其中特征代码根据定位标志影像的影像特征确定。When a frame of positioning image is processed, the feature code of the positioning mark image and the image position data are obtained, wherein the feature code is determined according to the image feature of the positioning mark image.
在本发明的图像处理方法的一种实施方式中,从定位图像中获取到特征代码及影像位置数据后,在前一帧定位图像所获取的特征代码中寻找是否有相同的。In an embodiment of the image processing method of the present invention, after the feature code and image position data are obtained from the positioning image, the feature code obtained from the previous frame of the positioning image is searched for the same.
在本发明的图像处理方法的一种实施方式中,从定位图像中获取到特征代码及影像位置数据后,将本帧和上一帧定位图像中相同的特征代码所对应的影像位置数据进行比对,并获得影像位移数据。In an embodiment of the image processing method of the present invention, after the feature code and image location data are obtained from the positioning image, the image location data corresponding to the same feature code in the current frame and the previous frame of the positioning image are compared. Yes, and get the image displacement data.
在本发明的图像处理方法的一种实施方式中,在对一帧定位图像进行处理时,获取其中一个定位标志影像的特征代码及影像位置数据。In an embodiment of the image processing method of the present invention, when a frame of positioning image is processed, the feature code and image position data of one of the positioning mark images are acquired.
在本发明的图像处理方法的一种实施方式中,在对一帧定位图像进行处理时,获取其中全部定位标志影像的特征代码及影像位置数据。In an embodiment of the image processing method of the present invention, when one frame of positioning image is processed, the feature codes and image position data of all positioning mark images therein are acquired.
本发明还提供了一种空中鼠标。该空中鼠标包括壳体、图像采集单元、处理单元。图像采集单元用于拍摄包括定位标志的景物,并输出有定位标志影像的定位图像;处理单元对所述定位图像进行图像处理,并根据图像处理结果将光标控制信号向被控设备发送。对定位图像进行图像处理时,包括获取其中定位标志影像的特征代码及影像位置数据,其中特征代码根据定位标志影像的特征来确定。The invention also provides an air mouse. The air mouse includes a housing, an image acquisition unit, and a processing unit. The image acquisition unit is used to photograph a scene including the positioning mark and output a positioning image with the positioning mark image; the processing unit performs image processing on the positioning image, and sends a cursor control signal to the controlled device according to the image processing result. When image processing is performed on the positioning image, it includes obtaining the feature code of the positioning mark image and the image location data, wherein the feature code is determined according to the feature of the positioning mark image.
在本发明的空中鼠标的一种实施方式中,处理单元将特征代码和影像位置数据向被控设备发送。In an embodiment of the air mouse of the present invention, the processing unit sends the feature code and image position data to the controlled device.
在本发明的空中鼠标的一种实施方式中,处理单元从一帧定位图像中获取到特征代码及影像位置数据后,在前一帧定位图像所获取的特征代码中寻找是否有相同的特征代码。如果有相同的,则将该特征代码所对应的影像位置数据向被控设备发送;如果没有相同的,则将换标信号向被控设备发送。In an embodiment of the air mouse of the present invention, after the processing unit obtains the feature code and image position data from one frame of positioning image, it searches for the same feature code in the feature code obtained from the previous frame of positioning image . If there are the same, the image location data corresponding to the feature code is sent to the controlled device; if there is no the same, the rebranding signal is sent to the controlled device.
在本发明的空中鼠标的一种实施方式中,处理单元根据前后两帧定位图像中具有相同影像特征的定位标志影像的位移获得影像位移数据,并根据影像位移数据向被控设备发送位移信号。In an embodiment of the air mouse of the present invention, the processing unit obtains image displacement data according to the displacement of the positioning mark images with the same image characteristics in the two previous and subsequent frames of positioning images, and sends a displacement signal to the controlled device according to the image displacement data.
在本发明的空中鼠标的一种实施方式中,处理单元从一帧定位图像中获取到特征代码及影像位置数据后,在前一帧定位图像所获取的特征代码中寻找是否有相同的特征代码。如果有相同的,则将该特征代码在两帧定位图像中对应的影像位置数据进行比对,根据获得的影像位移数据向被控设备发送位移信号;如果没有相同的,则将前一帧定位图像处理后产生的位移信号向被控设备发送。In an embodiment of the air mouse of the present invention, after the processing unit obtains the feature code and image position data from one frame of positioning image, it searches for the same feature code in the feature code obtained from the previous frame of positioning image . If there are the same, compare the corresponding image position data of the feature code in the two frames of positioning images, and send the displacement signal to the controlled device according to the obtained image displacement data; if there is no the same, then locate the previous frame The displacement signal generated after image processing is sent to the controlled device.
在本发明的空中鼠标的一种实施方式中,还包括按键单元。按键单元和处理单元连接,按键单元产生的按键信号通过处理单元向被控设备发送。In an embodiment of the air mouse of the present invention, it further includes a button unit. The key unit is connected with the processing unit, and the key signal generated by the key unit is sent to the controlled device through the processing unit.
在本发明的空中鼠标的一种实施方式中,空中鼠标包括壳体、图像采集单元、图像处理单元、按键单元、主控单元。图像采集单元用于拍摄包括定位标志的景物,并输出有定位标志影像的定位图像;图像处理单元对定位图像进行图像处理,并将图像处理结果发送到主控单元;主控单元根据图像处理结果将光标控制信号向被控设备发送;按键单元产生的按键信号通过主控单元向被控设备发送;图像处理单元对定位图像进行处理时,采用了本发明提供的图像处理方法。In an embodiment of the air mouse of the present invention, the air mouse includes a housing, an image acquisition unit, an image processing unit, a button unit, and a main control unit. The image acquisition unit is used to photograph the scene including the positioning mark and output the positioning image with the positioning mark image; the image processing unit performs image processing on the positioning image and sends the image processing result to the main control unit; the main control unit according to the image processing result The cursor control signal is sent to the controlled device; the key signal generated by the key unit is sent to the controlled device through the main control unit; when the image processing unit processes the positioning image, the image processing method provided by the present invention is adopted.
在本发明的空中鼠标的一种实施方式中,处理单元将特征代码和影像位置数据向所述被控设备发送。In an embodiment of the air mouse of the present invention, the processing unit sends the feature code and image position data to the controlled device.
在本发明的空中鼠标的一种实施方式中,处理单元从一帧定位图像中获取到特征代码及影像位置数据后,在前一帧定位图像所获取的特征代码中寻找是否有相同的特征代码;如果有相同的,则将该特征代码所对应的影像位置数据向被控设备发送;如果没有相同的,则将换标信号向所述被控设备发送。In an embodiment of the air mouse of the present invention, after the processing unit obtains the feature code and image position data from one frame of positioning image, it searches for the same feature code in the feature code obtained from the previous frame of positioning image ; If there are the same, the image location data corresponding to the feature code is sent to the controlled device; if there is no the same, the rebranding signal is sent to the controlled device.
在本发明的空中鼠标的一种实施方式中,处理单元根据前后两帧定位图像中具有相同影像特征的定位标志影像的位移获得影像位移数据,并根据影像位移数据向被控设备发送位移信号。In an embodiment of the air mouse of the present invention, the processing unit obtains image displacement data according to the displacement of the positioning mark images with the same image characteristics in the two previous and subsequent frames of positioning images, and sends a displacement signal to the controlled device according to the image displacement data.
在本发明的空中鼠标的一种实施方式中,处理单元从一帧定位图像中获取到特征代码及影像位置数据后,在前一帧定位图像所获取的特征代码中寻找是否有相同的特征代码;如果有相同的,则将该特征代码在两帧定位图像中对应的影像位置数据进行比对,根据获得的影像位移数据向被控设备发送位移信号;如果没有相同的,则将前一帧定位图像处理后产生的位移信号向被控设备发送。In an embodiment of the air mouse of the present invention, after the processing unit obtains the feature code and image position data from one frame of positioning image, it searches for the same feature code in the feature code obtained from the previous frame of positioning image ; If there are the same, compare the corresponding image position data of the feature code in the two frames of positioning images, and send the displacement signal to the controlled device according to the obtained image displacement data; if there is no the same, the previous frame The displacement signal generated after positioning image processing is sent to the controlled device.
在本发明的空中鼠标的一种实施方式中,空中鼠标包括壳体、图像采集单元、图像处理单元、按键单元、主控单元;图像采集单元用于拍摄包括定位标志的景物,并输出有定位标志影像的定位图像;图像处理单元对定位图像进行图像处理,并将图像处理结果发送到主控单元;主控单元根据图像处理结果将光标控制信号向被控设备发送;按键单元产生的按键信号通过主控单元向被控设备发送;所述图像处理单元对定位图像进行处理时,包括获取其中定位标志影像的特征代码及影像位置数据,其中特征代码根据定位标志影像的特征来确定。In an embodiment of the air mouse of the present invention, the air mouse includes a housing, an image acquisition unit, an image processing unit, a button unit, and a main control unit; the image acquisition unit is used to photograph a scene including a positioning mark and output the positioning The positioning image of the logo image; the image processing unit performs image processing on the positioning image and sends the image processing result to the main control unit; the main control unit sends the cursor control signal to the controlled device according to the image processing result; the key signal generated by the key unit Sending to the controlled device through the main control unit; when the image processing unit processes the positioning image, it includes acquiring the feature code of the positioning mark image and the image position data, wherein the feature code is determined according to the feature of the positioning mark image.
本发明还提供了一种空中鼠标系统,包括本发明的空中鼠标及至少两个具有不同特征的定位标志,不同特征的定位标志在定位图像中形成的定位标志影像,具有不同的影像特征。The present invention also provides an air mouse system including the air mouse of the present invention and at least two positioning marks with different characteristics. The positioning mark images formed by the positioning marks with different characteristics in the positioning image have different image characteristics.
在本发明的空中鼠标系统的一种实施方式中,具有不同特征的定位标志在定位图像中形成的定位标志影像,其组成像素的像素值不同。In an embodiment of the air mouse system of the present invention, the positioning mark images formed by positioning marks with different characteristics in the positioning image have different pixel values of the constituent pixels.
在本发明的空中鼠标系统的一种实施方式中,具有不同特征的定位标志在定位图像中形成的定位标志影像,其组成像素的排列方式不同。In an embodiment of the air mouse system of the present invention, the positioning mark images formed by positioning marks with different characteristics in the positioning image have different arrangements of the constituent pixels.
在本发明的空中鼠标系统的一种实施方式中,具有不同特征的定位标志在定位图像中形成的定位标志影像,其组成像素的数量不同。In an embodiment of the air mouse system of the present invention, the positioning mark images formed by positioning marks with different characteristics in the positioning image have different numbers of pixels.
本发明还提供了一种空中鼠标控制系统,包括本发明的空中鼠标和被控设备。空中鼠标向被控设备发送光标控制信号,被控设备根据光标控制信号控制所驱动的屏幕上的光标移动。The invention also provides an air mouse control system, which includes the air mouse and controlled equipment of the invention. The air mouse sends a cursor control signal to the controlled device, and the controlled device controls the movement of the cursor on the driven screen according to the cursor control signal.
在本发明的空中鼠标控制系统的一种实施方式中,空中鼠标向被控设备发送的光标控制信号为影像数据(包括特征代码和影像位置数据)。被控设备端的驱动程序接收到一组影像数据时,根据该组和前一组影像数据中相同的特征代码所对应的影像位置数据的变化控制所驱动的屏幕上光标的移动。In an embodiment of the air mouse control system of the present invention, the cursor control signal sent by the air mouse to the controlled device is image data (including feature code and image position data). When the driver of the controlled device receives a group of image data, it controls the movement of the cursor on the driven screen according to the change of the image position data corresponding to the same feature code in the group and the previous group of image data.
在本发明的空中鼠标控制系统的一种实施方式中,空中鼠标向被控设备发送的光标控制信号为影像位置数据及换标信号。被控设备端的驱动程序通过对影像位置数据及换标信号进行处理获得影像位移数据后控制所驱动的屏幕上的光标移动。In an embodiment of the air mouse control system of the present invention, the cursor control signal sent by the air mouse to the controlled device is image position data and a rebrand signal. The driver of the controlled device controls the movement of the cursor on the driven screen after obtaining the image displacement data by processing the image position data and the conversion signal.
在本发明的空中鼠标控制系统的一种实施方式中,空中鼠标向被控设备发送的光标控制信号为位移信号,被控设备根据位移信号控制所驱动的显示设备屏幕上的光标移动。In an embodiment of the air mouse control system of the present invention, the cursor control signal sent by the air mouse to the controlled device is a displacement signal, and the controlled device controls the movement of the cursor on the screen of the driven display device according to the displacement signal.
在本发明的空中鼠标控制系统的一种实施方式中,所述具有不同特征的定位标志在定位图像中形成的定位标志影像,其组成像素的排列方式不同。In an embodiment of the air mouse control system of the present invention, the positioning mark images formed by the positioning marks with different characteristics in the positioning image have different arrangement of the constituent pixels.
在本发明的空中鼠标控制系统的一种实施方式中,空中鼠标控制系统包括一空中鼠标和一被控设备,所述空中鼠标包括壳体、图像采集单元、处理单元;所述图像采集单元用于拍摄包括定位标志的景物,并输出有定位标志影像的定位图像;所述处理单元对所述定位图像进行图像处理,并根据图像处理结果将光标控制信号向所述被控设备发送;对定位图像进行处理时,包括获取其中定位标志影像的特征代码及影像位置数据,其中特征代码根据定位标志影像的特征来确定;所述被控设备根据光标控制信号控制所驱动的屏幕上的光标移动。In an embodiment of the air mouse control system of the present invention, the air mouse control system includes an air mouse and a controlled device. The air mouse includes a housing, an image acquisition unit, and a processing unit; the image acquisition unit is used for To photograph a scene including a positioning mark, and output a positioning image with a positioning mark image; the processing unit performs image processing on the positioning image, and sends a cursor control signal to the controlled device according to the image processing result; When the image is processed, it includes obtaining the feature code of the positioning mark image and the image position data, wherein the feature code is determined according to the feature of the positioning mark image; the controlled device controls the cursor movement on the driven screen according to the cursor control signal.
在本发明的空中鼠标控制系统的一种实施方式中,所述空中鼠标向所述被控 设备发送的光标控制信号为影像数据(包括特征代码和影像位置数据);所述被控设备端的驱动程序接收到一组影像数据时,根据该组和前一组影像数据中相同的特征代码所对应的影像位置数据的变化控制所驱动的屏幕上光标的移动。In an embodiment of the air mouse control system of the present invention, the cursor control signal sent by the air mouse to the controlled device is image data (including feature codes and image position data); the driver of the controlled device When the program receives a group of image data, it controls the movement of the cursor on the driven screen according to the change of the image position data corresponding to the same feature code in the group and the previous group of image data.
在本发明的空中鼠标控制系统的一种实施方式中,所述空中鼠标向所述被控设备发送的光标控制信号为影像位置数据及换标信号;所述被控设备端的驱动程序通过对影像位置数据及换标信号进行处理获得影像位移数据后控制所驱动的屏幕上的光标移动。In an embodiment of the air mouse control system of the present invention, the cursor control signal sent by the air mouse to the controlled device is image position data and a rebranding signal; The position data and the conversion signal are processed to obtain the image displacement data and then control the cursor movement on the driven screen.
在本发明的空中鼠标控制系统的一种实施方式中,所述空中鼠标向所述被控设备发送的光标控制信号为位移信号,被控设备根据位移信号控制所驱动的显示设备屏幕上的光标移动。In an embodiment of the air mouse control system of the present invention, the cursor control signal sent by the air mouse to the controlled device is a displacement signal, and the controlled device controls the cursor on the screen of the driven display device according to the displacement signal mobile.
本发明还提供了一种红外感应条,用于为空中鼠标提供定位标志,其包括一支架和至少两个安装在支架上的红外光源,所述两个红外光源有不同的特征。The present invention also provides an infrared sensor strip for providing positioning marks for an air mouse, which includes a bracket and at least two infrared light sources mounted on the bracket, and the two infrared light sources have different characteristics.
在本发明的红外感应条的一种实施方式中,所述两个红外光源的亮度不同。In an embodiment of the infrared sensor strip of the present invention, the brightness of the two infrared light sources is different.
在本发明的红外感应条的一种实施方式中,所述两个红外光源的发光面积不同。In an embodiment of the infrared sensor strip of the present invention, the light-emitting areas of the two infrared light sources are different.
在本发明的红外感应条的一种实施方式中,所述两个红外光源的形状不同。In an embodiment of the infrared sensor strip of the present invention, the two infrared light sources have different shapes.
在本发明的红外感应条的一种实施方式中,所述支架为中空结构。In an embodiment of the infrared sensor strip of the present invention, the bracket is a hollow structure.
本发明提供一种用于计算机的图像处理方法,用于处理包括定位标志影像的定位图像,其包括如下步骤:The present invention provides an image processing method used in a computer for processing a positioning image including a positioning mark image, which includes the following steps:
(a)获取其中定位标志影像的特征代码及影像位置数据;其中特征代码根据定位标志影像的影像特征确定;(a) Obtain the feature code and image location data of the location mark image; where the feature code is determined according to the image feature of the location mark image;
(b)将获得的特征代码和上一帧定位图像中获得的特征代码相比较,如果相同,则将特征代码所对应的影像位置数据进行比对。(b) Compare the obtained feature code with the feature code obtained in the previous frame of the positioning image, and if they are the same, compare the image position data corresponding to the feature code.
本发明提供一种计算机控制所驱动的屏幕上光标移动的方法,其包括如下步骤:The present invention provides a method for a computer to control the movement of a cursor on a driven screen, which includes the following steps:
(a)间隔的接收两帧定位图像,所述两帧定位图像中有定位标志影像;(a) Receive two frames of positioning images at intervals, and the two frames of positioning images have positioning mark images;
(b)判断两帧所述定位图像中的所述定位标志影像是否有相同的特征,如果有,则将两帧所述定位图像中所述定位标志影像的位置进行比较,如果产生了影像位移数据,则根据该影像位移数据控制所驱动的屏幕上光标进行移动。(b) Determine whether the positioning mark images in the two frames of the positioning image have the same characteristics, if so, compare the positions of the positioning mark images in the two frames of the positioning image, and if an image displacement occurs Data, the cursor on the screen driven by the image displacement data is controlled to move.
在本发明的计算机控制所驱动的屏幕上光标移动的方法中,在所述步骤(b) 中包括如下步骤:在对每帧所述定位图像处理后,如果产生了所述影像位移数据,则根据该影像位移数据控制所驱动的屏幕上的光标移动;如果没有产生所述影像位移数据,则根据前一帧定位图像处理后产生的影像位移数据控制所驱动的屏幕上的光标移动。In the method for controlling the movement of a cursor on a screen driven by a computer of the present invention, the step (b) includes the following steps: after processing the positioning image for each frame, if the image displacement data is generated, then The cursor movement on the driven screen is controlled according to the image displacement data; if the image displacement data is not generated, the cursor movement on the driven screen is controlled according to the image displacement data generated after the positioning image processing of the previous frame.
本发明还提供了一种图像处理芯片,用于处理数字图像。所述图像处理芯片对数字图像进行处理时,采用本发明的图像处理方法。The invention also provides an image processing chip for processing digital images. When the image processing chip processes digital images, the image processing method of the present invention is adopted.
本发明还提供了一种计算机系统,包括空中鼠标的被控设备及驱动程序。The invention also provides a computer system, which includes a controlled device and a driver of the air mouse.
本发明还提供了一种红外感应条,包括支架和至少两个具有不同特征的红外光源。The present invention also provides an infrared sensor strip, which includes a bracket and at least two infrared light sources with different characteristics.
在本发明的红外感应条的一种实施方式中,相邻红外光源的亮度不同。In an embodiment of the infrared sensor strip of the present invention, adjacent infrared light sources have different brightness.
在本发明的红外感应条的一种实施方式中,相邻红外光源的发光面积不同。In an embodiment of the infrared sensor strip of the present invention, adjacent infrared light sources have different light-emitting areas.
在本发明的红外感应条的一种实施方式中,相邻红外光源的形状不同。In an embodiment of the infrared sensor strip of the present invention, adjacent infrared light sources have different shapes.
在本发明的红外感应条的一种实施方式中,支架为中空的,方便容纳红外光源及电池。In an embodiment of the infrared sensor strip of the present invention, the bracket is hollow to conveniently accommodate the infrared light source and the battery.
本发明还提供了一种用于智能移动设备的空中鼠标,该空中鼠标包括壳体、图像采集单元、处理单元。图像采集单元用于拍摄包括定位标志的景物,并输出含有定位标志影像的定位图像;处理单元通过对所述定位图像进行图像处理获得光标控制信号,并将光标控制信号向所控的智能移动设备(智能手机或平板电脑)发送。处理单元对定位图像进行图像处理时,采用了本发明的图像处理方法。The present invention also provides an air mouse for smart mobile devices. The air mouse includes a housing, an image acquisition unit, and a processing unit. The image acquisition unit is used to photograph a scene including a positioning mark and output a positioning image containing the positioning mark image; the processing unit obtains a cursor control signal by image processing the positioning image, and sends the cursor control signal to the controlled intelligent mobile device (Smartphone or tablet) to send. When the processing unit performs image processing on the positioning image, the image processing method of the present invention is adopted.
本发明的空中鼠标控制系统中的被控设备为各种类型的电子计算机设备、包括但不限于PC、电视游戏机、智能电视、机顶盒、平板电脑、智能手机、VR设备。The controlled devices in the air mouse control system of the present invention are various types of electronic computer devices, including but not limited to PCs, video game consoles, smart TVs, set-top boxes, tablet computers, smart phones, and VR devices.
本发明的空中鼠标的图像处理方法,和现有技术中空中鼠标的图像处理方法相比,在获取每帧定位图像中定位标志影像位置数据的基础上,进一步根据影像特征获取特征代码。使得应用了该图像处理方法的空中鼠标,能够和两个及两个以上具有不同特征的定位标志配合使用,达到在不降低移动精度的情况下增加移动长度的目的。Compared with the image processing method of the air mouse in the prior art, the image processing method of the air mouse of the present invention further acquires the feature code according to the image characteristics on the basis of acquiring the location data of the positioning mark image in each frame of the positioning image. The air mouse to which the image processing method is applied can be used in conjunction with two or more positioning marks with different characteristics to achieve the purpose of increasing the moving length without reducing the moving accuracy.
本发明的图像处理芯片,采用了本发明的图像处理方法。使得采用了该图像处理芯片的空中鼠标,能够通过增加定位标志的方式增加移动长度。The image processing chip of the present invention adopts the image processing method of the present invention. Therefore, the air mouse adopting the image processing chip can increase the moving length by adding positioning marks.
本发明的空中鼠标系统,采用了两个及两个以上的定位标志。在不降低移动 精度的情况下增加了移动长度。使得空中鼠标用于主流显示设备时,在光标移动的长度和精度两方面都有较好的效果,真正具备了实用性。在光标性能大幅提升的同时,增加的成本却很少(硬件上仅增加了定位标志),所以在实现较高的光标控制性能的同时,仍能保持较低的成本,有利于普及和应用。The air mouse system of the present invention uses two or more positioning marks. The movement length is increased without reducing the movement accuracy. When the air mouse is used in mainstream display devices, it has good effects in both the length and accuracy of the cursor movement, and it is truly practical. While the cursor performance is greatly improved, the increased cost is very small (only positioning marks are added to the hardware), so while achieving higher cursor control performance, it can still maintain a lower cost, which is conducive to popularization and application.
本发明的空中鼠标控制系统,用于控制被控设备驱动的显示设备屏幕上的光标进行移动,在不降低光标移动精度的情况下,大幅提高了移动长度。满足了主流显示设备(屏幕分辨率1920×1020左右)的需要。The air mouse control system of the present invention is used to control the movement of the cursor on the screen of the display device driven by the controlled device, and the movement length is greatly increased without reducing the cursor movement accuracy. It meets the needs of mainstream display devices (screen resolution of about 1920×1020).
本发明的红外感应条,相邻的红外光源具有不同的特征,使得在定位图像中形成定位标志影像有不同的影像特征。和空中鼠标配合使用时,能够使空中鼠标同时具备较高的移动长度和移动精度。In the infrared sensor strip of the present invention, adjacent infrared light sources have different characteristics, so that the positioning mark image formed in the positioning image has different image characteristics. When used in conjunction with an air mouse, it can make the air mouse have a higher moving length and moving accuracy.
依本发明的一个方面,本发明进一步提供一空中鼠标,适用于控制一被控设备,其包括:According to one aspect of the present invention, the present invention further provides an air mouse suitable for controlling a controlled device, which includes:
一壳体;A shell
一图像采集单元,其中所述图像采集单元被设置于所述壳体,其中所述图像采集单元输出具有一定位标志影像的一定位图像;An image acquisition unit, wherein the image acquisition unit is arranged on the housing, and the image acquisition unit outputs a positioning image with a positioning mark image;
一处理单元,其中所述处理单元被设置于所述壳体,所述处理单元被可通信地连接于所述图像采集单元,其中所述处理单元根据所述定位标志影像的一特征代码和一影像位置数据处理所述定位图像,并发送一光标控制信号于所述被控设备,以控制所述被控设备。A processing unit, wherein the processing unit is arranged in the housing, the processing unit is communicably connected to the image acquisition unit, and the processing unit is based on a feature code and a feature code of the positioning mark image The image position data processes the positioning image, and sends a cursor control signal to the controlled device to control the controlled device.
根据本发明的一个实施例,所述空中鼠标进一步包括一按键单元,其中所述按键单元被设置于所述壳体,所述按键单元被可通信地连接于所述处理单元。According to an embodiment of the present invention, the air mouse further includes a button unit, wherein the button unit is disposed on the housing, and the button unit is communicably connected to the processing unit.
根据本发明的一个实施例,所述空中鼠标进一步包括一光标移动控制键,其中所述光标移动控制键被设置于所述壳体,所述光标移动控制键被可通信地连接于所述处理单元。According to an embodiment of the present invention, the air mouse further includes a cursor movement control key, wherein the cursor movement control key is provided on the housing, and the cursor movement control key is communicatively connected to the processor. unit.
根据本发明的一个实施例,所述图像采集单元包括一镜头和一图像传感器模块,其中所述镜头被可通信地连接于所述图像传感模块。According to an embodiment of the present invention, the image acquisition unit includes a lens and an image sensor module, wherein the lens is communicably connected to the image sensor module.
依本发明的一个方面,本发明进一步提供一空中鼠标系统,适用于控制一被控设备,其包括:According to one aspect of the present invention, the present invention further provides an air mouse system suitable for controlling a controlled device, which includes:
一空中鼠标,其中所述空中鼠标包括一壳体、一图像采集单元以及一处理单元,其中所述图像采集单元被设置于所述壳体,其中所述图像采集单元输出具有 一定位标志影像的一定位图像,其中所述处理单元被设置于所述壳体,所述处理单元被可通信地连接于所述图像采集单元,其中所述处理单元根据所述定位标志影像的一特征代码和一影像位置数据处理所述定位图像,并发送一光标控制信号于所述被控设备;和An air mouse, wherein the air mouse includes a housing, an image acquisition unit, and a processing unit, wherein the image acquisition unit is disposed on the housing, and the image acquisition unit outputs an image with a positioning mark A positioning image, wherein the processing unit is provided in the housing, the processing unit is communicably connected to the image acquisition unit, and the processing unit is based on a feature code and a feature code of the positioning mark image The image position data processes the positioning image and sends a cursor control signal to the controlled device; and
至少一定位标志,其中所述定位标志能够形成具有所述定位标志影像的所述定位图像,利用所述定位标志能够确定所述空中鼠标的位置。At least one positioning mark, wherein the positioning mark can form the positioning image with the positioning mark image, and the position of the air mouse can be determined by using the positioning mark.
根据本发明的一个实施例,所述空中鼠标系统的所述定位标志被实施为至少两个,且两个所述定位标志具有不同的特征,且每个所述定位标志的特征能够被记录于所述定位图像中。According to an embodiment of the present invention, the positioning marks of the air mouse system are implemented as at least two, and the two positioning marks have different characteristics, and the characteristics of each positioning mark can be recorded in The positioning image.
根据本发明的一个实施例,两个所述定位标志在所述定位图像中形成的所述定位标志影像,其组成像素有不同的像素值。According to an embodiment of the present invention, the positioning mark image formed by the two positioning marks in the positioning image has different pixel values for its constituent pixels.
根据本发明的一个实施例,两个所述定位标志在所述定位图像中形成的所述定位标志影像,由不同数量的像素组成。According to an embodiment of the present invention, the positioning mark images formed by the two positioning marks in the positioning image are composed of different numbers of pixels.
根据本发明的一个实施例,所述定位标志被实施为一红外光源。According to an embodiment of the present invention, the positioning mark is implemented as an infrared light source.
根据本发明的一个实施例,所述定位标志被实施为一可见光源。According to an embodiment of the present invention, the positioning mark is implemented as a visible light source.
根据本发明的一个实施例,所述空中鼠标和所述定位标志的距离被保持于大于等于50cm小于等于100cm。According to an embodiment of the present invention, the distance between the air mouse and the positioning mark is maintained to be greater than or equal to 50 cm and less than or equal to 100 cm.
根据本发明的一个实施例,所述空中鼠标系统进一步包括一支架,其中所述定位标志被横向地设置于所述支架。According to an embodiment of the present invention, the air mouse system further includes a bracket, wherein the positioning mark is laterally arranged on the bracket.
根据本发明的一个实施例,所述空中鼠标系统进一步包括一支架,其中所述定位标志被横向地设置于所述支架。According to an embodiment of the present invention, the air mouse system further includes a bracket, wherein the positioning mark is laterally arranged on the bracket.
依本发明的一个方面,本发明进一步提供一图像处理方法,所述图像处理方法包括如下步骤:According to one aspect of the present invention, the present invention further provides an image processing method. The image processing method includes the following steps:
(a)获取具有一定位标志影像的一定位图像;和(a) Obtain a positioning image with a positioning mark image; and
(b)通过读取所述定位图像中的像素的像素值的方式确定所述定位影像的一特征代码和一影像位置数据。(b) Determine a feature code and image position data of the positioning image by reading the pixel value of the pixel in the positioning image.
根据本发明的一个实施例,在所述步骤(b)中,包括步骤(c)逐个读取所述定位图像中每个像素的像素值,找到第1个像素值大于一预设像素值的像素时,记录所述像素的像素值及坐标。According to an embodiment of the present invention, in the step (b), the step (c) reads the pixel value of each pixel in the positioning image one by one, and finds the first pixel value greater than a preset pixel value In the case of pixels, the pixel values and coordinates of the pixels are recorded.
根据本发明的一个实施例,所述图像处理方法进一步包括步骤(d)对比先 后获取的两个定位图像的所述定位标志影像的所述特征代码。According to an embodiment of the present invention, the image processing method further includes the step (d) comparing the feature codes of the positioning mark images of the two positioning images acquired first.
根据本发明的一个实施例,在所述步骤(b)之后包括步骤(e)对比所述特征代码相同的两组所述定位图像的所述定位标志影像,并得到一影像位移数据。According to an embodiment of the present invention, after the step (b), a step (e) is included to compare the positioning mark images of the two sets of positioning images with the same feature code, and obtain an image displacement data.
根据本发明的一个实施例,在所述步骤(b)中进一步包括:According to an embodiment of the present invention, the step (b) further includes:
(ⅰ)在一预选区域中逐个读取每个像素的像素值,直到找到像素值大于一预设像素值的一个像素;(I) Read the pixel value of each pixel one by one in a preselected area until a pixel with a pixel value greater than a preset pixel value is found;
(ⅱ)找出以所述像素为中心的一预设像素范围内像素值大于所述预设像素值的全部像素;以及(Ii) Find all pixels in a preset pixel range centered on the pixel whose pixel value is greater than the preset pixel value; and
(ⅲ)根据找出的全部像素的数量确定特征代码,并在所有像素值大于所述预设像素值以上的像素中,找出坐标值最小像素,将该像素的坐标记录为所述影像位置数据。(Iii) Determine the feature code according to the number of all pixels found, and find the pixel with the smallest coordinate value among all the pixels whose pixel value is greater than the preset pixel value, and record the coordinate of this pixel as the image position data.
根据本发明的一个实施例,在所述步骤(b)中进一步包括如下步骤:According to an embodiment of the present invention, the step (b) further includes the following steps:
(ⅰ)读取一预选区域内所有像素的像素值,找出其中全部像素值大于一预设像素值的像素作为一备选像素;(I) Read the pixel values of all pixels in a preselected area, and find out all the pixels whose pixel values are greater than a preset pixel value as a candidate pixel;
(ⅱ)从所述备选像素中选择一个坐标值最小像素,找出以所述像素为中心的一预设像素范围内像素值大于所述预设像素值的全部像素,定义这些像素称为一组成像素;(Ii) Select a pixel with the smallest coordinate value from the candidate pixels, find out all pixels with a pixel value greater than the preset pixel value in a preset pixel range centered on the pixel, and define these pixels as One component pixel;
(ⅲ)将所述组成像素的总数记录为所述特征代码,并在所述组成像素中,找出一个坐标值最小像素,将该像素的坐标记录为所述特征代码对应的影像位置数据。(Iii) Record the total number of the constituent pixels as the feature code, find a pixel with the smallest coordinate value among the constituent pixels, and record the coordinate of the pixel as the image position data corresponding to the feature code.
根据本发明的一个实施例,在所述步骤(b)中,进一步包括步骤(ⅳ)在所述备选像素中去掉所述组成像素,如果还剩下有所述备选像素,则从步骤(ⅱ)开始处理,如果没有备选像素了,则图像处理过程结束。According to an embodiment of the present invention, in the step (b), it further includes the step (iv) to remove the constituent pixels from the candidate pixels, and if the candidate pixels remain, proceed from step (Ii) Start processing, if there are no candidate pixels, the image processing process ends.
根据本发明的一个实施例,在所述步骤(b)中,进一步包括如下步骤:According to an embodiment of the present invention, in the step (b), further comprising the following steps:
(ⅰ)在一预选区域内,根据由左至右,由上至下的顺序,读取每个像素的像素值,直到找到像素值大于一预设像素值的一个像素;(I) In a preselected area, read the pixel value of each pixel according to the order from left to right and from top to bottom until a pixel with a pixel value greater than a preset pixel value is found;
(ⅱ)找出以所述像素为中心的一预设像素范围内像素值大于所述预设像素值的全部像素;(Ii) Find out all pixels with a pixel value greater than the preset pixel value within a preset pixel range centered on the pixel;
(ⅲ)如果找到的全部像素的列坐标相同,将特征代码记录为1;如果行坐标相同,将特征代码记录为2;以及(Iii) If the column coordinates of all the pixels found are the same, record the feature code as 1; if the row coordinates are the same, record the feature code as 2; and
(ⅳ)在找到的全部像素中,找出坐标值最小像素,将该像素的坐标记录为所述特征代码所对应的所述影像位置数据。(Iv) Among all the found pixels, find the pixel with the smallest coordinate value, and record the coordinate of the pixel as the image position data corresponding to the feature code.
根据本发明的一个实施例,在所述步骤(b)中,进一步包括如下步骤:According to an embodiment of the present invention, in the step (b), further comprising the following steps:
(ⅰ)读取一预选区域内所有像素的像素值,找出其中全部像素值大于一预设像素值的像素作为一备选像素;(I) Read the pixel values of all pixels in a preselected area, and find out all the pixels whose pixel values are greater than a preset pixel value as a candidate pixel;
(ⅱ)从所述备选像素中选择一个坐标值最小像素,找出以所述像素为中心的一预设像素范围内像素值大于所述预设像素值的全部像素,定义这些像素称为一组成像素;(Ii) Select a pixel with the smallest coordinate value from the candidate pixels, find out all pixels with a pixel value greater than the preset pixel value in a preset pixel range centered on the pixel, and define these pixels as One component pixel;
(ⅲ)如果找出的全部组成像素的列坐标相同,则将特征代码记录为1;如果全部组成像素的行坐标相同,则将特征代码记录为2;并在全部组成像素中,找出一个坐标值最小像素,将该像素的坐标记录为所述影像位置数据;和(Iii) If the column coordinates of all the constituent pixels found are the same, record the feature code as 1; if the row coordinates of all constituent pixels are the same, record the feature code as 2; and among all the constituent pixels, find one The pixel with the smallest coordinate value, and the coordinate of the pixel is recorded as the image position data; and
(ⅳ)在所述备选像素中,去掉和所述组成像素相同坐标的像素,如果还剩下有所述备选像素,则又从步骤(ⅱ)开始处理;如果没有备选像素了,则获取影像数据的步骤结束。(Iv) Among the candidate pixels, remove the pixels with the same coordinates as the constituent pixels. If the candidate pixels are still left, then start processing from step (ii); if there are no candidate pixels, Then the step of acquiring image data ends.
依本发明的一个方面,本发明提供一光标控制方法,所述光标控制方法包括如下步骤:According to one aspect of the present invention, the present invention provides a cursor control method. The cursor control method includes the following steps:
(A)间隔地接收至少两组影像数据,其中所述影像数据包括一特征代码和对应于所述特征代码的一影像位置数据;和(A) receiving at least two sets of image data at intervals, wherein the image data includes a feature code and an image position data corresponding to the feature code; and
(B)根据所述影像数据的所述特征代码和所述影像位置数据判断是否产生一影像位移数据,若产生所述影像位置数据,则根据所述影像位移数据控制一光标的移动。(B) Determine whether to generate image displacement data according to the feature code of the image data and the image position data, and if the image position data is generated, control the movement of a cursor according to the image displacement data.
根据本发明的一个实施例,在所述步骤(B)中包括如下步骤:According to an embodiment of the present invention, the step (B) includes the following steps:
(B.1)在接收的前一组所述影像数据中查找后一组所述影像数据的所述特征代码;(B.1) Searching for the feature code of the latter group of image data in the received previous group of image data;
(B.2)比所述特征代码相同的两组所述定位图像的所述定位标志影像,并得到所述影像位移数据;以及(B.2) The positioning mark images of the two sets of positioning images that are the same as the feature code, and obtaining the image displacement data; and
(B.3)根据所述影像位移数据控制一光标的移动。(B.3) Control the movement of a cursor according to the image displacement data.
根据本发明的一个实施例,其中在所述步骤(B)中包括如下步骤:在对每帧所述定位图像处理后,如果产生了所述影像位移数据,则根据该影像位移数据向所述被控设备发送所述位移信号;如果没有产生所述影像位移数据,则向所述 被控设备发送对前一帧所述定位图像处理后产生的所述位移信号。According to an embodiment of the present invention, the step (B) includes the following step: after the positioning image is processed for each frame, if the image displacement data is generated, then the image displacement data is sent to the The controlled device sends the displacement signal; if the image displacement data is not generated, the controlled device sends the displacement signal generated after processing the positioning image of the previous frame.
图1展示的是本发明的空中鼠标的一种实施方式。Figure 1 shows an embodiment of the air mouse of the present invention.
图2展示的是本发明的空中鼠标及使用方式。Figure 2 shows the air mouse of the present invention and its usage.
图3展示的是本发明的空中鼠标的驱动程序的一种实施方式。Figure 3 shows an implementation of the air mouse driver of the present invention.
图4展示的是本发明的空中鼠标及空中鼠标控制系统的信号流动方式。Figure 4 shows the signal flow of the air mouse and air mouse control system of the present invention.
图5展示的是本发明的红外感应条的一种实施方式。Figure 5 shows an embodiment of the infrared sensor strip of the present invention.
图6展示的是本发明的红外感应条的一种实施方式。Figure 6 shows an embodiment of the infrared sensor strip of the present invention.
图7展示的是本发明的图像处理方法的一种实施方式。FIG. 7 shows an embodiment of the image processing method of the present invention.
图8展示的是本发明的红外感应条的一种实施方式。Figure 8 shows an embodiment of the infrared sensor strip of the present invention.
图9展示的是本发明的红外感应条的一种实施方式。Figure 9 shows an embodiment of the infrared sensor strip of the present invention.
图10展示的是本发明的红外感应条的一种实施方式。Figure 10 shows an embodiment of the infrared sensor strip of the present invention.
图11展示的是本发明的红外感应条的一种实施方式。Figure 11 shows an embodiment of the infrared sensor strip of the present invention.
图12展示的是本发明的图像处理方法的一种实施方式。Fig. 12 shows an embodiment of the image processing method of the present invention.
图13展示的是本发明的红外感应条的一种实施方式。Figure 13 shows an embodiment of the infrared sensor strip of the present invention.
图14展示的是本发明的红外感应条的一种实施方式。Figure 14 shows an embodiment of the infrared sensor strip of the present invention.
图15展示的是本发明的图像处理方法的一种实施方式。Fig. 15 shows an embodiment of the image processing method of the present invention.
图16展示的是本发明的空中鼠标的驱动程序的一种实施方式Figure 16 shows an embodiment of the air mouse driver of the present invention
图17展示的是本发明的图像处理方法的一种实施方式。FIG. 17 shows an embodiment of the image processing method of the present invention.
图18展示的是本发明的空中鼠标及空中鼠标控制系统的一种实施方式。Figure 18 shows an embodiment of the air mouse and air mouse control system of the present invention.
图19展示的是本发明的空中鼠标及空中鼠标控制系统的一种实施方式。Figure 19 shows an embodiment of the air mouse and air mouse control system of the present invention.
以下描述用于揭露本发明以使本领域技术人员能够实现本发明。以下描述中的优选实施例只作为举例,本领域技术人员可以想到其他显而易见的变型。在以下描述中界定的本发明的基本原理可以应用于其他实施方案、变形方案、改进方案、等同方案以及没有背离本发明的精神和范围的其他技术方案。The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments in the following description are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, modifications, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the present invention.
本领域技术人员应理解的是,在本发明的揭露中,术语“纵向”、“横向”、“上”、“下”、“前”、“后”、“左”、“右”、“竖直”、“水平”、“顶”、“底”“内”、 “外”等指示的方位或位置关系是基于附图所示的方位或位置关系,其仅是为了便于描述本发明和简化描述,而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作,因此上述术语不能理解为对本发明的限制。Those skilled in the art should understand that, in the disclosure of the present invention, the terms "longitudinal", "lateral", "upper", "lower", "front", "rear", "left", "right", " The orientation or positional relationship indicated by "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. are based on the orientation or positional relationship shown in the drawings, which is only for the convenience of describing the present invention and The description is simplified, rather than indicating or implying that the pointed device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore the above terms should not be construed as limiting the present invention.
可以理解的是,术语“一”应理解为“至少一”或“一个或多个”,即在一个实施例中,一个元件的数量可以为一个,而在另外的实施例中,该元件的数量可以为多个,术语“一”不能理解为对数量的限制。It can be understood that the term "a" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in other embodiments, The number can be multiple, and the term "one" cannot be understood as a restriction on the number.
下面,将结合附图通过具体的实施例对本发明作进一步说明。Hereinafter, the present invention will be further described through specific embodiments in conjunction with the drawings.
实施例一Example one
本实施例提供了一种空中鼠标。如图1所示,空中鼠标包括壳体200、设置于壳体200上表面的按键201、设置于壳体200前端中间位置的镜头202、以及壳体200内的电路部分。电路部分包括和按键201相连的按键电路、和镜头202相连的图像传感器模块、处理单元。This embodiment provides an air mouse. As shown in FIG. 1, the air mouse includes a housing 200, buttons 201 arranged on the upper surface of the housing 200, a lens 202 arranged at the middle position of the front end of the housing 200, and a circuit part in the housing 200. The circuit part includes a button circuit connected to the button 201, an image sensor module connected to the lens 202, and a processing unit.
镜头202和所述图像传感器模块共同组成图像采集单元,其中镜头202为滤光镜头,能够滤除可见光,只允许红外光通过。所述图像传感器模块为OV7620图像传感器模块,集成了CMOS图像传感器及DSP芯片,和镜头202构成完整的红外数字摄像模块,能够拍摄到红外发光物体并输出数字图像。按键201及按键电路共同组成按键单元,按键单元产生的按键信号通过处理单元后发送到一被控设备。按键信号用于实现按键控制功能,属于现有技术。The lens 202 and the image sensor module together form an image acquisition unit, where the lens 202 is a filter lens that can filter out visible light and only allows infrared light to pass through. The image sensor module is an OV7620 image sensor module, which integrates a CMOS image sensor and a DSP chip, and a lens 202 to form a complete infrared digital camera module, which can capture infrared light-emitting objects and output digital images. The button 201 and the button circuit together form a button unit, and the button signal generated by the button unit is sent to a controlled device after passing through the processing unit. The key signal is used to realize the key control function and belongs to the prior art.
处理单元选用意法半导体的STM32F407芯片。OV7620图像传感器模块的输出端和STM32F407芯片的输入引脚电连接,按键电路和STM32F407的输入引脚电连接。处理单元还用于桥接被控设备。本实施例中,被控设备为一台PC,通过USB电缆的和STM32F407连接。被控设备驱动一台显示设备,本实施例中,所述显示设备为一台分辨率为1920×1080的液晶显示器。处理单元也可以通过无线的方式和被控设备连接并通讯。The processing unit uses STMicroelectronics' STM32F407 chip. The output end of the OV7620 image sensor module is electrically connected to the input pin of the STM32F407 chip, and the button circuit is electrically connected to the input pin of the STM32F407. The processing unit is also used to bridge the controlled device. In this embodiment, the controlled device is a PC, which is connected to the STM32F407 via a USB cable. The controlled device drives a display device. In this embodiment, the display device is a liquid crystal display with a resolution of 1920×1080. The processing unit can also connect and communicate with the controlled device wirelessly.
为实现光标控制功能,空中鼠标需要至少1个红外光源作为定位标志来帮助空中鼠标随时确定自身在空中的位置。所述红外光源可以是各种能够提供红外光的物体,包括各种红外光发射器,蜡烛等。In order to realize the cursor control function, the air mouse needs at least one infrared light source as a positioning mark to help the air mouse determine its position in the air at any time. The infrared light source may be various objects capable of providing infrared light, including various infrared light emitters, candles and the like.
使用时,OV7620图像传感器模块通过镜头202进行拍摄,并将形成的图像以数字信号的形式连续输出到处理单元(STM32F407),输出的图像中有定位标志(红外光源)形成的影像,称为定位标志影像。有定位标志影像的图像称为定 位图像。处理单元对定位图像进行图像处理,根据图像处理结果向被控设备发送控制信号。When in use, the OV7620 image sensor module shoots through the lens 202, and continuously outputs the formed image in the form of a digital signal to the processing unit (STM32F407). The output image contains the image formed by the positioning mark (infrared light source), which is called positioning Logo image. The image with the positioning mark image is called the positioning image. The processing unit performs image processing on the positioning image, and sends a control signal to the controlled device according to the image processing result.
OV7620图像传感器模块设定为每秒钟输出50帧分辨率为320*240的灰度图像。灰度图像是数字图像的一种,每一帧灰度图像中,记录的是一个320*240的像素矩阵的信息。该像素矩阵有240行,320列,由上至下为第0行至第239行,由左至右分别为第0列至第319列。矩阵中的每个像素的位置由一个唯一的坐标来标示。左上角的第一个像素,其坐标为(0,0)。表示其位于第0行第0列,其中第一个0为行坐标,表示其所在的行数;第二个0为列坐标,表示其所在的列数。像素矩阵中的每个像素都有一个像素值,由0至255的数字来表示。其中,255代表白色,0代表黑色,其余数字代表了不同程度的灰色。The OV7620 image sensor module is set to output 50 frames of grayscale images with a resolution of 320*240 per second. Gray image is a kind of digital image. In each frame of gray image, the information of a 320*240 pixel matrix is recorded. The pixel matrix has 240 rows and 320 columns, from top to bottom from row 0 to row 239, and from left to right from column 0 to column 319. The position of each pixel in the matrix is marked by a unique coordinate. The first pixel in the upper left corner has coordinates (0, 0). Indicates that it is located in row 0 and column 0, where the first 0 is the row coordinate, indicating the number of rows it is in; the second 0 is the column coordinate, indicating the number of columns it is in. Each pixel in the pixel matrix has a pixel value, represented by a number from 0 to 255. Among them, 255 represents white, 0 represents black, and the remaining numbers represent varying degrees of gray.
所述的空中鼠标和两个定位标志组成空中鼠标系统,有更好的性能表现。所述的两个定位标志有不同的特征,且该特征能够被记录到定位图像中。The air mouse and two positioning marks form an air mouse system, which has better performance. The two positioning marks have different characteristics, and the characteristics can be recorded in the positioning image.
本实施例中,图像采集单元也可以是红外图像感应器,只要能输出数字图像即可。In this embodiment, the image acquisition unit may also be an infrared image sensor, as long as it can output digital images.
本实施例还提供了一种红外感应条,用于为本实施例的空中鼠标在空中的位移提供参照。This embodiment also provides an infrared sensor strip, which is used to provide a reference for the displacement of the air mouse of this embodiment in the air.
如图2所示,该红外感应条包括一个支架300及设置在支架300正面左端的红外光源A301,设置在支架300正面右端的红外光源B302。优选的,支架300为矩形,长度为25CM,高度为2CM,厚度为2CM,2个红外光源之间的距离为20CM。优选的,该2个红外光源为红外发光二极管,外接电源供电。支架300也可以是中空的,设有电池仓,用于加装电池为红外光源供电。所述支架300上的2个红外光源作为2个定位标志,和空中鼠标一起组成了空中鼠标控制系统。其中红外光源A301为定位标志A,红外光源B302为定位标志B。使用时,将该红外感应条放置在使用者前方,距离50CM左右。As shown in FIG. 2, the infrared sensor strip includes a bracket 300 and an infrared light source A301 arranged at the left end of the front of the bracket 300, and an infrared light source B302 arranged at the right end of the front of the bracket 300. Preferably, the bracket 300 is rectangular, with a length of 25 cm, a height of 2 cm, and a thickness of 2 cm. The distance between the two infrared light sources is 20 cm. Preferably, the two infrared light sources are infrared light-emitting diodes and are powered by an external power source. The bracket 300 may also be hollow, with a battery compartment for installing batteries to supply power to the infrared light source. The two infrared light sources on the bracket 300 serve as two positioning marks, and together with the air mouse form an air mouse control system. The infrared light source A301 is the positioning mark A, and the infrared light source B302 is the positioning mark B. When in use, place the infrared sensor strip in front of the user at a distance of about 50cm.
两个红外光源发出不同亮度的红外光。例如,红外光源A301的光通量为5流明;红外光源B302的光通量为10流明。红外光源在定位图像中形成的定位标志影像,有不同的影像特征,体现为像素值不同。例如,本实施例中,红外光源A301在定位图像中形成的影像,组成该影像的每个像素的像素值为50;红外光源B302在定位图像中形成的影像,组成该影像的每个像素的像素值为100。Two infrared light sources emit infrared light of different brightness. For example, the luminous flux of the infrared light source A301 is 5 lumens; the luminous flux of the infrared light source B302 is 10 lumens. The positioning mark image formed by the infrared light source in the positioning image has different image characteristics, which is reflected in different pixel values. For example, in this embodiment, the image formed by the infrared light source A301 in the positioning image has a pixel value of 50 for each pixel constituting the image; the image formed by the infrared light source B302 in the positioning image has the value of each pixel constituting the image The pixel value is 100.
因为定位图像中,仅有定位标志影像的组成像素的像素值在50以上,所以, 可以根据这个信息来判断是否找到定位标志影像。本实施例中,处理单元在像素矩阵中查找定位标志影像时,找到像素值在40以上的像素,即判断为找到了定位标志影像。Because in the positioning image, only the pixel value of the constituent pixels of the positioning mark image is above 50, so it can be determined whether the positioning mark image is found based on this information. In this embodiment, when the processing unit searches for the positioning mark image in the pixel matrix, it finds a pixel with a pixel value above 40, that is, it is determined that the positioning mark image is found.
空中鼠标的处理单元对图像采集单元输出的定位图像进行图像处理,其目的是获取其中定位标志影像的特征代码及影像位置数据,本实施例中的具体方法为:The processing unit of the air mouse performs image processing on the positioning image output by the image acquisition unit, and its purpose is to obtain the feature code and image position data of the positioning mark image. The specific method in this embodiment is:
逐个读取定位图像中每个像素的像素值,找到第1个像素值为40以上的像素时,将该像素的像素值及坐标记录。所述像素值为特征代码,坐标为该特征代码对应的影像位置数据。例如,读取到坐标为(0,5)的像素时,发现像素值为50,将50和(0,5)记录。其中50为特征代码,(0,5)是该特征代码对应的影像位置数据。该影像位置数据,实际是定位图像中1个定位标志影像的其中1个组成像素的坐标,用于代表该定位标志影像在定位图像中的位置。Read the pixel value of each pixel in the positioning image one by one, and record the pixel value and coordinates of the pixel when the first pixel value is 40 or more. The pixel value is a feature code, and the coordinates are image location data corresponding to the feature code. For example, when reading a pixel with coordinates (0, 5), it is found that the pixel value is 50, and 50 and (0, 5) are recorded. Wherein 50 is the feature code, (0, 5) is the image location data corresponding to the feature code. The image position data is actually the coordinates of one of the constituent pixels of a positioning mark image in the positioning image, and is used to represent the position of the positioning mark image in the positioning image.
每处理一帧定位图像,处理单元都会向连接的PC发送一组影像数据,其中包括特征代码和影像位置数据。Each time a frame of positioning image is processed, the processing unit sends a set of image data to the connected PC, including feature codes and image location data.
本实施例还提供了一种驱动程序,用在空中鼠标的被控设备(本实施例为PC)上,用于根据影像数据控制所驱动的屏幕上光标的移动。This embodiment also provides a driver program, used on the controlled device of the air mouse (PC in this embodiment), for controlling the movement of the cursor on the driven screen according to the image data.
该驱动程序接收到一组影像数据时,采用的处理方法为:When the driver receives a group of image data, the processing method adopted is:
根据其中的特征代码,在收到的上一组影像数据中查找相同的。如果找到相同的,则将此两组影像数据中,该特征代码所对应的影像位置数据进行比对后获得影像位移数据,并根据影像位移数据控制屏幕上光标的移动;如果没有找到相同的,本组影像数据的处理过程结束。其处理方法如图3所示。According to the feature code, find the same in the last group of image data received. If the same is found, the image position data corresponding to the feature code in the two sets of image data is compared to obtain the image displacement data, and the cursor movement on the screen is controlled according to the image displacement data; if the same is not found, The processing of this group of image data ends. The processing method is shown in Figure 3.
本实施例中的空中鼠标及空中鼠标控制系统的组成及信号传输方式,如图4所示。The composition and signal transmission mode of the air mouse and the air mouse control system in this embodiment are shown in FIG. 4.
下面以空中鼠标横向移动为例,详细说明本实施例的空中鼠标及图像处理方法。Taking the horizontal movement of the air mouse as an example, the air mouse and the image processing method of this embodiment will be described in detail below.
PC端的驱动程序预设有一个记录区,用于记录从空中鼠标传来的影像数据(包括特征代码和影像位置数据),并在每一组影像数据处理结束时,更新该记录区中的数据。The driver on the PC side presets a recording area to record the image data (including feature code and image position data) from the air mouse, and update the data in the recording area at the end of each group of image data processing .
使用者手持空中鼠标将镜头202朝向定位标志A(红外光源A301)。图像采集单元开始拍摄并输出第1帧定位图像,该帧定位图像中,坐标为(120,318)、 (120,319)、(121,318)、(121,319)的四个像素的像素值为50,其余像素的像素值为0。这四个像素组成的影像即是红外光源A形成的,称为定位标志A影像。The user holds the air mouse and points the lens 202 toward the positioning mark A (infrared light source A301). The image acquisition unit starts to take and output the first frame of positioning image. In this frame of positioning image, the four pixels with coordinates (120,318), (120,319), (121,318), (121,319) The value is 50, and the pixel value of the remaining pixels is 0. The image composed of these four pixels is formed by the infrared light source A and is called the positioning mark A image.
此帧定位图像以数字信号的形式传送给处理单元,处理单元进行如下图像处理:This frame positioning image is transmitted to the processing unit in the form of a digital signal, and the processing unit performs the following image processing:
按照由左至右,由上至下的顺序,逐个读取像素矩阵中每个像素的像素值。具体方法是从坐标为(0,0)的像素开始,首先按照由左至右的顺序读取像素矩阵中第一行像素中每个像素的像素值,然后第二行、第三行、直到最后一行。Read the pixel value of each pixel in the pixel matrix one by one in the order from left to right and top to bottom. The specific method is to start from the pixel with coordinates (0, 0), first read the pixel value of each pixel in the first row of pixels in the pixel matrix in the order from left to right, and then the second row, third row, until the last line.
按照上述顺序,读取到坐标为(120,318)的像素时,读取到的像素值为50。则将50和(120,318)记录下来(因为该像素的像素值大于40),其中50为特征代码,(120,318)为该特征代码对应的影像位置数据。According to the above sequence, when the pixel with coordinates (120, 318) is read, the read pixel value is 50. Record 50 and (120, 318) (because the pixel value of the pixel is greater than 40), where 50 is the feature code, and (120, 318) is the image location data corresponding to the feature code.
处理单元将50和(120,318)发送到连接的PC,PC端的驱动程序根据特征代码50,在记录区中寻找相同的。因为这是处理的第一组影像数据,记录区中并无记录,将50和(120,318)存入记录区后,整个图像处理过程结束。The processing unit sends 50 and (120, 318) to the connected PC, and the driver on the PC side searches for the same in the recording area according to the feature code 50. Because this is the first set of image data processed, there is no record in the recording area. After storing 50 and (120, 318) in the recording area, the entire image processing process ends.
使用者手持空中鼠标继续向右移动,图像采集单元输出的第二帧定位图像中,坐标为(120,317)、(120,318)、(121,317)、(121,318)的四个像素的像素值为50,其余像素的像素值为0。The user holds the air mouse and continues to move to the right. In the second frame of the positioning image output by the image acquisition unit, the coordinates are (120,317), (120,318), (121,317), (121,318). The pixel value of the pixel is 50, and the pixel value of the remaining pixels is 0.
处理单元从此帧定位图像中获取到的影像数据为50及(120,317)。The image data obtained by the processing unit from this frame positioning image is 50 and (120, 317).
驱动程序根据特征代码50,在记录区中找到了相同的。将(120,317)同(120,318)进行比对,发现列坐标的数值减小了1,驱动程序获得了影像位移数据。PC控制所驱动的液晶显示器屏幕上的光标向右(屏幕面对使用者)移动1个像素。The driver found the same in the recording area according to the feature code 50. Comparing (120,317) with (120,318), it is found that the value of the column coordinate is reduced by 1, and the driver obtains the image displacement data. The cursor on the LCD screen driven by the PC is moved 1 pixel to the right (the screen faces the user).
使用者手持空中鼠标继续向右移动,图像采集单元持续拍摄并输出定位图像,在输出的一系列定位图像中,定位标志A影像的位置逐渐向左移动,处理单元根据图像处理结果持续输出影像数据,PC根据该影像数据控制液晶显示器屏幕上的光标持续向右移动。The user holds the air mouse and continues to move to the right. The image acquisition unit continues to capture and output positioning images. In the output series of positioning images, the position of the positioning mark A image gradually moves to the left, and the processing unit continues to output image data according to the image processing results , The PC controls the cursor on the LCD screen to move continuously to the right according to the image data.
图像采集单元输出第n帧定位图像时,处理单元从该帧定位图像中获得特征代码50及对应的影像位置数据(120,0),此时,定位标志A已经到达镜头202的可视范围的边缘。此帧定位图像处理完成后,驱动程序的记录区中的数据为50、(120,0)。When the image acquisition unit outputs the nth frame of the positioning image, the processing unit obtains the feature code 50 and the corresponding image position data (120, 0) from the frame of the positioning image. At this time, the positioning mark A has reached the visible range of the lens 202 edge. After the frame positioning image processing is completed, the data in the recording area of the driver is 50, (120, 0).
使用者手持空中鼠标向右继续移动一段距离后,镜头202中只有定位标志B。此时图像采集单元输出第n+1帧定位图像,其中有四个像素的像素值为100,其余像素的像素值为0。该四个像素组成了定位标志B影像,该四个像素的坐标分别是(120,317)、(120,318)、(121,317)、(121,318)。After the user holds the air mouse and moves to the right for a certain distance, there is only the positioning mark B in the lens 202. At this time, the image acquisition unit outputs a positioning image of the n+1th frame, in which four pixels have a pixel value of 100, and the remaining pixels have a pixel value of 0. The four pixels constitute the positioning mark B image, and the coordinates of the four pixels are (120, 317), (120, 318), (121, 317), (121, 318), respectively.
处理单元在该帧定位图像中获取到特征代码100及对应的影像位置数据(120,317)。PC端的驱动程序根据特征代码100在记录区中没有找到相同的。将记录区中的数据更新为100、(120,317)后。本组影像数据的处理结束。The processing unit obtains the feature code 100 and the corresponding image position data (120, 317) in the frame positioning image. The driver on the PC side does not find the same in the recording area according to the feature code 100. After updating the data in the recording area to 100, (120, 317). The processing of this group of image data ends.
使用者手持空中鼠标继续向右移动一段微小的距离后,图像采集单元输出的一帧定位图像中,有4个像素的像素值为100,其余像素的像素值为0。该4个像素的坐标分别是:(120,316)、(120,317)、(121,316)、(121,317)。处理单元对该帧定位图像进行图像处理后,向PC端发送的影像数据为100及(120,316),驱动程序根据特征代码100,在记录区中发现了相同的,将(120,316)同(120,317)进行比对,获得的影像位移数据是列坐标的数值减小了1,PC控制所驱动的液晶显示器屏幕上的光标向右移动1个像素。使用者手持空中鼠标向右继续移动,屏幕上的光标也向右继续移动,直到定位标志B移出了镜头202的可视范围。After the user holds the air mouse and continues to move a small distance to the right, in a frame of positioning image output by the image acquisition unit, 4 pixels have a pixel value of 100 and the remaining pixels have a pixel value of 0. The coordinates of the 4 pixels are (120, 316), (120, 317), (121, 316), (121, 317), respectively. After the processing unit performs image processing on the frame positioning image, the image data sent to the PC is 100 and (120, 316). According to the feature code 100, the driver finds the same in the recording area, and changes (120, 316) Comparing with (120, 317), the obtained image displacement data is that the value of the column coordinate is reduced by 1, and the cursor on the LCD screen driven by the PC control moves 1 pixel to the right. The user holds the air mouse and continues to move to the right, and the cursor on the screen also continues to move to the right, until the positioning mark B moves out of the visible range of the lens 202.
本实施例的空中鼠标,有1个定位标志在镜头202的可视范围内就能够实现对光标的控制。增加定位标志就可以增加移动长度,移动长度增加的多少取决于两个定位标志之间的距离。较合理的定位标志摆放距离是,空中鼠标位于两个定位标志中间时,两个定位标志都在镜头202的可视范围内,但都位于边缘,这样在横向移动过程中,即可以保证光标控制的连续性,又能有最长的移动长度,此时,增加1个定位标志,就可以增加接近320个像素的移动长度。In the air mouse of this embodiment, there is a positioning mark within the visual range of the lens 202 to control the cursor. The length of the movement can be increased by adding positioning marks, and how much the movement length increases depends on the distance between the two positioning marks. A more reasonable positioning mark placement distance is that when the air mouse is in the middle of the two positioning marks, both positioning marks are within the visible range of the lens 202, but both are located at the edge, so that the cursor can be guaranteed during the horizontal movement. The continuity of control can also have the longest movement length. At this time, adding 1 positioning mark can increase the movement length close to 320 pixels.
本实施例的空中鼠标的有益效果来自于所采用的图像处理方法。在从左向右移动的过程中,空中鼠标先后以定位标志A和定位标志B作为参照物确定自身在空中的位置。在参照物从定位标志A切换到定位标志B时,屏幕上的光标有一个极短暂的停顿,然后继续向右移动。如果是现有技术的空中鼠标,只是简单的对前后两帧定位图像中定位标志影像的位置进行比对,在参照物切换时,屏幕上的光标会和空中鼠标的移动方向相反。例如,以定位标志A为参照物时,产生的最后一帧定位图像,影像位置数据是(120,0);向右轻微移动后,以定位标志B为参照物产生的第一帧定位图像,影像位置数据是(120,317),将两个 影像位置数据进行比对后输出位移信号,屏幕上的光标就会向左移动317个像素。The beneficial effect of the air mouse in this embodiment comes from the image processing method used. In the process of moving from left to right, the air mouse uses positioning mark A and positioning mark B as reference objects to determine its own position in the air. When the reference object is switched from positioning mark A to positioning mark B, the cursor on the screen has a very short pause, and then continues to move to the right. If it is an air mouse in the prior art, it simply compares the positions of the positioning mark images in the two frames of the positioning image before and after. When the reference object is switched, the cursor on the screen will move in the opposite direction to the air mouse. For example, when positioning mark A is used as the reference object, the image position data of the last frame of positioning image generated is (120, 0); after moving slightly to the right, the first frame of positioning image generated using positioning mark B as the reference object, The image position data is (120, 317). After comparing the two image position data and outputting a displacement signal, the cursor on the screen will move 317 pixels to the left.
本实施例的空中鼠标在对每一帧定位图像进行图像处理时,在获取影像位置数据的基础上,进一步根据影像特征获得特征代码。使得PC端的驱动程序获得了必要的信息,只将同一定位标志在两帧定位图像中的位置进行比对,使得通过增加定位标志的方式来增加移动长度成为可能。When the air mouse of this embodiment performs image processing on each frame of positioning image, on the basis of obtaining the image position data, it further obtains a feature code according to the image feature. The driver on the PC side obtains the necessary information, and only compares the position of the same positioning mark in the two positioning images, making it possible to increase the moving length by adding positioning marks.
驱动程序将影像位置数据进行比对时,要将行坐标和列坐标分别进行比对,找出数值的变化,包括四种变化,行坐标增加、行坐标减小、列坐标增加、列坐标减小。本实施例的驱动程序根据影像位置数据比对后获得的影像位移数据控制屏幕上的光标移动,采用的规则是:如果行坐标的数值增加了n,则控制屏幕上的光标向上移动n个像素;如果行坐标的数值减小了n,则控制屏幕上的光标向下移动n个像素。如果列坐标的数值增加了n,则控制屏幕上的光标向左移动n个像素;如果列坐标的数值减小了n,则控制屏幕上的光标向右移动n个像素;如果行坐标和列坐标的数值都有变化,则控制光标在两个方向上作相应的移动。例如,测得行坐标及列坐标的数值都增加了1,则控制屏幕上的光标向上移动1个像素,再向左移动1个像素。When the driver compares the image position data, it needs to compare the row coordinates and column coordinates separately to find out the changes of the values. There are four types of changes: row coordinates increase, row coordinates decrease, column coordinates increase, and column coordinates decrease. small. The driver of this embodiment controls the cursor movement on the screen according to the image displacement data obtained after the image position data comparison. The rule adopted is: if the value of the row coordinate increases by n, the cursor on the screen is controlled to move up n pixels ; If the value of the row coordinate decreases by n, the cursor on the control screen moves down by n pixels. If the value of the column coordinate increases by n, the cursor on the control screen moves n pixels to the left; if the value of the column coordinate decreases by n, the cursor on the control screen moves n pixels to the right; if the row coordinate and column If the values of the coordinates have changed, control the cursor to move correspondingly in two directions. For example, if the measured values of row coordinates and column coordinates are increased by 1, the cursor on the control screen will move up by 1 pixel, and then move to the left by 1 pixel.
采用上述规则可以保证屏幕上光标的移动方向和空中鼠标的移动方向一致。根据需要,也可以将规则改为:获得的影像位移数据中,列坐标的数值增加,则控制光标向左移动;列坐标的数值减小,则控制光标向右移动。这样在横向移动时,屏幕上光标的移动方向和空中鼠标的移动方向相反。纵向移动的方向,也可以根据同样的方法修改。Using the above rules can ensure that the moving direction of the cursor on the screen is consistent with the moving direction of the air mouse. According to needs, the rule can also be changed: in the obtained image displacement data, if the value of the column coordinate increases, the cursor is controlled to move to the left; the value of the column coordinate decreases, and the cursor is controlled to move to the right. In this way, when moving horizontally, the moving direction of the cursor on the screen is opposite to that of the air mouse. The direction of longitudinal movement can also be modified in the same way.
在本实施例的驱动程序的一种实施方式中,驱动程序将影像位移数据乘2后输出,例如,测得列坐标的数值减小了1,则将屏幕上的光标向右移动2个像素。这样空中鼠标能够以较短的移动距离控制屏幕上的光标作较长距离的移动。In an implementation of the driver program of this embodiment, the driver program multiplies the image displacement data by 2 and outputs it. For example, if the value of the measured column coordinates is reduced by 1, the cursor on the screen is moved 2 pixels to the right . In this way, the air mouse can control the cursor on the screen to move a longer distance with a shorter moving distance.
将影像位移数据乘3后输出,则由空中鼠标和两个定位标志构成的空中鼠标系统,就能够基本满足控制主流分辨率屏幕的需要。此时,空中鼠标的横向移动长度可以接近1920,移动精度为3。如果要增加空中鼠标在纵向上的移动长度,将红外感应条竖立放置,使得两个红外光源上下排列即可。After multiplying the image displacement data by 3, the air mouse system consisting of an air mouse and two positioning marks can basically meet the needs of controlling mainstream resolution screens. At this time, the horizontal movement length of the air mouse can be close to 1920, and the movement accuracy is 3. If you want to increase the moving length of the air mouse in the vertical direction, place the infrared sensor strip upright so that the two infrared light sources are arranged up and down.
除了平移,空中鼠标在空中的转动也可以控制屏幕上的光标移动,朝左或朝右转动可以控制屏幕上的光标作横向移动;朝上或朝下转动可以控制屏幕上的光 标作纵向移动。因为转动同样可以使图像采集单元输出的定位图像中定位标志影像的位置产生变化。In addition to panning, turning the air mouse in the air can also control the cursor movement on the screen, turning left or right can control the cursor on the screen to move horizontally; turning up or down can control the cursor on the screen to move vertically . Because the rotation can also change the position of the positioning mark image in the positioning image output by the image acquisition unit.
本发明中,空中鼠标和红外光源之间的距离在50CM至100CM较为合适,太近会使得定为图像中定位标志影像过大从而影像到定位。In the present invention, the distance between the air mouse and the infrared light source is more suitable to be 50cm to 100cm. Too close will make the image of the positioning mark in the image determined to be too large and the image to be positioned.
在本实施例的红外感应条的一种实施方式中,矩形支架的长度为45CM,在支架的正面从左至右设置3个红外光源。3个红外光源排列成一行,每个光源之间的距离为20CM。左右两边的红外光源亮度相同,而中间的光源亮度不同。例如左右两边的红外光源光通量为5流明,中间的红外光源光通量为10流明。该3个红外光源相当于3个定位标志,和空中鼠标组成空中鼠标系统,能够进一步增加空中鼠标的移动长度。In an implementation of the infrared sensor strip of this embodiment, the length of the rectangular bracket is 45 cm, and three infrared light sources are arranged on the front of the bracket from left to right. The 3 infrared light sources are arranged in a row, and the distance between each light source is 20cm. The brightness of the infrared light source on the left and right sides is the same, while the brightness of the light source in the middle is different. For example, the luminous flux of the infrared light source on the left and right sides is 5 lumens, and the luminous flux of the infrared light source in the middle is 10 lumens. The 3 infrared light sources are equivalent to 3 positioning marks, and form an air mouse system with the air mouse, which can further increase the moving length of the air mouse.
在本实施例的红外感应条的一种实施方式中,支架300为矩形,长25CM,高8CM。支架的4个角落设有4个亮度不同的红外光源,如图5所示。优选的,4个红外光源的光通量分别为5流明、10流明、15流明、20流明。等于为空中鼠标提供了4个定位标志,且任意两个相邻的定位标志在定位图像中形成的定位标志影像都有不同的特征(像素值),相对于现有技术中只有1个定位标志的空中鼠标,该4个红外光源和空中鼠标构成空中鼠标系统后,可以使空中鼠标在空中横向、纵向、斜向移动时,移动长度都得到增加。该实施方式中的支架300也可以是框形的结构,4个角落各有1个红外光源,4个红外光源有4种亮度。如图6所示。优选的,框的每条边的宽度为2CM。In an implementation of the infrared sensor strip of this embodiment, the bracket 300 is rectangular, 25 cm in length and 8 cm in height. There are 4 infrared light sources with different brightness in 4 corners of the bracket, as shown in Figure 5. Preferably, the luminous fluxes of the four infrared light sources are 5 lumens, 10 lumens, 15 lumens, and 20 lumens, respectively. It is equivalent to providing 4 positioning marks for the air mouse, and the positioning mark images formed by any two adjacent positioning marks in the positioning image have different characteristics (pixel values), compared to only one positioning mark in the prior art After the air mouse system is composed of the 4 infrared light sources and the air mouse, the air mouse can be moved horizontally, vertically and diagonally in the air, and the moving length is increased. The bracket 300 in this embodiment may also have a frame-shaped structure, with one infrared light source at each of the four corners, and four infrared light sources with four kinds of brightness. As shown in Figure 6. Preferably, the width of each side of the frame is 2CM.
在本实施例的空中鼠标一种实施方式中,处理单元对定位图像采用如下图像处理方法:In an implementation of the air mouse of this embodiment, the processing unit uses the following image processing method for the positioning image:
S1.读取定位图像中全部像素的像素值,将其中所有像素值大于40的像素作为备选像素。S1. Read the pixel values of all pixels in the positioning image, and use all pixels with a pixel value greater than 40 as candidate pixels.
S2.在全部备选像素中,选取一个坐标值最小像素,将该像素的像素值记录为特征代码,坐标记录为影像位置数据。坐标值最小像素的选取方法是,在所有备选像素中选取行坐标值最小的像素,如果只有1个,则该像素为坐标值最小像素;如果有多个,则在这多个中间选取列坐标值最小的。S2. From all candidate pixels, select a pixel with the smallest coordinate value, record the pixel value of this pixel as a feature code, and record the coordinate as image position data. The method of selecting the pixel with the smallest coordinate value is to select the pixel with the smallest row coordinate value among all the candidate pixels. If there is only one, then the pixel is the pixel with the smallest coordinate value; if there are more than one, select the column among the multiple The smallest coordinate value.
S3.将坐标值最小像素和同坐标值最小像素相同像素值的全部像素从备选像素中去掉,如果还有备选像素,则又从S2开始继续处理;如果没有,此帧定位图像的处理过程结束。S3. Remove all pixels with the same pixel value from the smallest pixel with the same coordinate value and the smallest pixel with the same coordinate value from the candidate pixels. If there are still candidate pixels, continue processing from S2; if not, process the positioning image in this frame The process is over.
根据该图像处理方法对1帧定位图像处理完成后,处理单元可以获取其中全部定位标志影像的影像数据。图像处理方法如图7所示。每帧定位图像的图像处理结束后,处理单元将获取的全部影像代码和影像位置作为一组影像数据向连接的PC发送。After processing one frame of positioning image according to the image processing method, the processing unit can obtain the image data of all the positioning mark images. The image processing method is shown in Figure 7. After the image processing of each frame of the positioning image is completed, the processing unit sends all the acquired image codes and image positions as a set of image data to the connected PC.
此时PC端的驱动程序接收到的每组影像数据中,可能有1个或多个特征代码及对应的影像位置数据。在有1个以上特征代码的情况下,则从特征代码中任意提取一个,在记录区中查找相同的,找到相同的则将该特征代码对应的影像位置数据同记录区中该特征代码所对应的影像位置数据进行比对,并根据比对的结果控制屏幕上的光标进行移动;如果没找到相同的则再提取一个特征代码继续查找,直到找到相同的或是全部特征代码都查找过。At this time, each group of image data received by the driver on the PC side may have one or more feature codes and corresponding image position data. In the case of more than one feature code, extract one of the feature codes randomly, search for the same in the recording area, and find the same, then the image position data corresponding to the feature code corresponds to the feature code in the recording area The position data of the image is compared, and the cursor on the screen is controlled to move according to the result of the comparison; if the same is not found, another feature code is extracted and the search continues until the same is found or all feature codes have been searched.
下面举例说明该图像处理方法及驱动程序。The following examples illustrate the image processing method and driver.
图像采集单元同时拍摄到红外光源A和B时,输出第1帧定位图像。其中有4个像素值为50的像素,坐标分别是(120,0)、(120,1)、(121,0)、(121,1);还有4个像素值为100的像素,坐标分别是(120,318)、(120,319)、(121,318)、(121,319)。When the image acquisition unit captures infrared light sources A and B at the same time, it outputs the first frame positioning image. There are 4 pixels with a pixel value of 50, the coordinates are (120, 0), (120, 1), (121, 0), (121, 1); there are 4 pixels with a pixel value of 100, the coordinates They are (120,318), (120,319), (121,318), (121,319).
处理单元对此帧定位图像进行如下图像处理:The processing unit performs the following image processing on this frame positioning image:
S1.读取定位图像中全部像素的像素值,共找到8个备用像素。坐标分别是(120,0)、(120,1)、(121,0)、(121,1)(120,318)、(120,319)、(121,318)、(121,319)。S1. Read the pixel values of all pixels in the positioning image, and find a total of 8 spare pixels. The coordinates are (120, 0), (120, 1), (121, 0), (121, 1) (120, 318), (120, 319), (121, 318), (121, 319), respectively.
S2.在备选像素中选出一个坐标值最小像素,其坐标为(120,0)。将特征代码50及影像位置数据(120,0)记入影像数据。S2. Select a pixel with the smallest coordinate value among the candidate pixels, and its coordinate is (120, 0). The feature code 50 and the image position data (120, 0) are recorded in the image data.
S3.将该像素及所有像素值为50的像素从备选像素中去掉后,还剩下4个备选像素,所述4个备选像素的像素值为100,坐标分别是(120,318)、(120,319)、(121,318)、(121,319)。S3. After removing this pixel and all pixels with a pixel value of 50 from the candidate pixels, there are still 4 candidate pixels. The pixel values of the 4 candidate pixels are 100, and the coordinates are (120, 318 ), (120,319), (121,318), (121,319).
回到S2处,从剩下的4个备选像素中选出一个坐标值最小像素,其坐标为(120,318),将特征代码100及像素位置数据(120,318)记入影像数据。Going back to S2, select a pixel with the smallest coordinate value from the remaining 4 candidate pixels, whose coordinates are (120, 318), and record the feature code 100 and pixel position data (120, 318) into the image data.
将该像素及所有像素值为100的像素从备选像素中去掉后,已无备选像素,本帧定位图像的图像处理过程结束。处理单元将第1组影像数据发送到连接的PC,其中有特征代码50及对应的影像位置数据(120,0)、还有特征代码100及对应的影像位置数据(120,318)。After removing this pixel and all pixels with a pixel value of 100 from the candidate pixels, there are no candidate pixels, and the image processing process of the positioning image of this frame ends. The processing unit sends the first group of image data to the connected PC, including feature code 50 and corresponding image location data (120, 0), and feature code 100 and corresponding image location data (120, 318).
PC端驱动程序接收到该组影像数据后,首先根据特征代码50在记录区中查找。并未找到相同的;然后根据特征代码100在记录区中查找。未找到相同的,处理过程结束,同时该组影像数据被记入记录区。After the PC driver receives the group of image data, it first searches the recording area according to the feature code 50. Did not find the same; then search in the recording area according to feature code 100. If the same is not found, the process ends, and the group of image data is recorded in the recording area.
图像采集单元发出第2帧定位图像,处理单元进行图像处理后将第2组影像数据发送到PC。其中有特征代码50及对应的影像位移数据(120,1)、还有特征代码100及对应的影像位移数据(120,319)。驱动程序根据特征代码50在记录区中找到了相同的,将(120,1)同(120,0)进行比对,发现列坐标的数值增加了1。则控制屏幕上的光标向左移动1个像素,结束处理过程,同时将记录区中的数据改为第2组影像数据。The image acquisition unit sends out the second frame positioning image, and the processing unit sends the second group of image data to the PC after image processing. There are feature code 50 and corresponding image displacement data (120, 1), and feature code 100 and corresponding image displacement data (120, 319). The driver finds the same in the recording area according to the feature code 50, compares (120, 1) with (120, 0), and finds that the value of the column coordinate has increased by 1. Then the cursor on the control screen moves 1 pixel to the left to end the processing process, and at the same time change the data in the recording area to the second group of image data.
在本实施例的空中鼠标的一种实施方式中,图像采集单元可以拍摄可见光图像,例如将图像传感器模块配套的镜头改为能够通过可见光的镜头。则可见光光源也能够作为定位标志。例如,可以用发光二极管作为定位标志。In an implementation of the air mouse of this embodiment, the image acquisition unit can capture visible light images, for example, the lens matched with the image sensor module is changed to a lens that can pass visible light. The visible light source can also be used as a positioning mark. For example, light-emitting diodes can be used as positioning marks.
包括下面几种做法,用两个发光二极管作为定位标志,其中一个的光通量为5流明,另一个的光通量为10流明。Including the following methods, using two light-emitting diodes as positioning signs, one of which has a luminous flux of 5 lumens and the other has a luminous flux of 10 lumens.
用3个发光二极管作为定位标志,其中一个的光通量为5流明,另两个的光通量为10流明。使用时摆出一排,光通量5流明的放在中间。Use 3 light-emitting diodes as positioning marks, one of which has a luminous flux of 5 lumens and the other two has a luminous flux of 10 lumens. When in use, put out a row with 5 lumens in the middle.
用4个具有不同亮度的发光二极管作为定位标志,摆放时按照图6中的摆放方式,例如把4个发光二极管放置在1台显示器的4个角上,使用时,空中鼠标对准显示器即可。Use 4 light-emitting diodes with different brightnesses as positioning marks, and place them according to the placement method in Figure 6. For example, place 4 light-emitting diodes on the 4 corners of a monitor. When using, the air mouse is aligned with the monitor OK.
本发明的空中鼠标的图像处理单元可以处理更高分辨率的图像,例如OV7620图像传感输出分辨率为640×480的黑白数字图像,每秒30帧。处理该分辨率的定位图像,可以使空中鼠标在移动精度不变的情况下有更高的移动长度。The image processing unit of the air mouse of the present invention can process higher-resolution images. For example, the OV7620 image sensor outputs a black and white digital image with a resolution of 640×480 at 30 frames per second. Processing the positioning image of this resolution can make the air mouse have a higher moving length without changing the moving accuracy.
在本实施例的空中鼠标的一种实施方式中,还包括光标移动控制键,该光标移动控制键设置在壳体表面,通过电路和处理单元连接。用于控制屏幕上的光标是否跟随空中鼠标移动,例如,使用者经常会需要将空中鼠标移动到有效区域的中间位置,又不想屏幕上的光标跟随移动。此时按下光标移动控制键,处理单元即停止输出光标控制信号,释放该键,处理单元则恢复输出光标控制信号。优选的,该键设置在壳体上适合大拇指按压的位置。In an implementation of the air mouse of this embodiment, it further includes a cursor movement control key, which is arranged on the surface of the housing and is connected to the processing unit through a circuit. It is used to control whether the cursor on the screen moves with the air mouse. For example, the user often needs to move the air mouse to the middle of the effective area and does not want the cursor on the screen to follow the movement. At this time, when the cursor movement control key is pressed, the processing unit stops outputting the cursor control signal, and when the key is released, the processing unit resumes outputting the cursor control signal. Preferably, the key is arranged at a position suitable for thumb pressing on the housing.
本说明书中,空中鼠标控制的光标泛指屏幕上的鼠标指针、电子游戏中的武 器准星、人物角色等各种跟随空中鼠标的移动而移动的图标。光标可以是不同大小,甚至包括整个屏幕,例如用空中鼠标控制游戏中人物的视野,可能整个屏幕都是视野范围,空中鼠标移动时,整个屏幕的显示都会发生变化。In this manual, the cursor controlled by the air mouse refers to various icons that move with the movement of the air mouse, such as the mouse pointer on the screen, the weapon crosshairs in electronic games, and characters. The cursor can be of different sizes, and even include the entire screen. For example, using an air mouse to control the field of view of a character in a game, the entire screen may be the field of view. When the air mouse moves, the display of the entire screen will change.
本实施例中的空中鼠标和被控设备共同组成空中鼠标控制系统。所述被控设备为各种计算机设备,包括但不限于PC、电视游戏机、智能电视、机顶盒、平板电脑、智能手机、VR设备。空中鼠标输出影像数据(包括代码特征和影像位置数据),被控设备根据影像数据控制所驱动的显示设备屏幕上的光标进行移动。The air mouse and the controlled device in this embodiment jointly constitute an air mouse control system. The controlled devices are various computer devices, including but not limited to PCs, video game consoles, smart TVs, set-top boxes, tablet computers, smart phones, and VR devices. The air mouse outputs image data (including code features and image position data), and the controlled device controls the cursor on the display device screen driven by the image data to move.
本发明的空中鼠标在用于对智能手机或平板电脑进行控制时,处理单元输出影像数据,智能手机或平板电脑根据该影像数据控制屏幕上的光标移动。When the air mouse of the present invention is used to control a smart phone or a tablet computer, the processing unit outputs image data, and the smart phone or tablet computer controls the cursor movement on the screen according to the image data.
实施例二Example two
本实施例提供了一种空中鼠标,和实施例一中的空中鼠标区别在于获取影像数据的方法。本实施例的空中鼠标的处理单元对定位图像采用如下处理方法:This embodiment provides an air mouse, which is different from the air mouse in the first embodiment in the method of acquiring image data. The processing unit of the air mouse in this embodiment adopts the following processing methods for the positioning image:
在预选区域中逐个读取每个像素的像素值,直到找到像素值在40以上的像素。然后以该像素为中心,读取7像素×7像素范围内全部像素的像素值,找出其中全部像素值在40以上的像素。Read the pixel value of each pixel one by one in the preselected area until a pixel with a pixel value above 40 is found. Then take this pixel as the center, read the pixel values of all pixels in the range of 7 pixels×7 pixels, and find out the pixels with all the pixel values above 40.
例如,找到的第1个像素值在40以上的像素,其坐标为(X,Y),则读取坐标符合下列要求的全部像素的像素值:行坐标为X-3至X+3,且列坐标为Y-3至Y+3。本步骤的目的是找出组成该定位标志影像的全部像素。For example, if the first pixel with a pixel value of 40 or more is found and its coordinates are (X, Y), the pixel values of all pixels whose coordinates meet the following requirements are read: the row coordinates are from X-3 to X+3, and The column coordinates are from Y-3 to Y+3. The purpose of this step is to find all the pixels that make up the positioning mark image.
然后根据找出的全部像素的数量确定特征代码。例如:像素值在40以上的像素有4个,则将特征代码记录为4;像素值在40以上的像素有9个,则将特征代码记录为9。Then the feature code is determined according to the number of all pixels found. For example, if there are 4 pixels with a pixel value above 40, record the feature code as 4; if there are 9 pixels with a pixel value above 40, record the feature code as 9.
然后在所有像素值在40以上的像素中,找出坐标值最小像素,将该像素的坐标记录为影像位置数据。确定坐标值最小像素的方法是:在全部坐标值在40以上的像素中,选择行坐标的值最小的像素,如果只有1个,则该像素为坐标值最小像素。如果有多个,在这多个中选出列坐标值最小的像素作为坐标值最小像素。Then, among all the pixels with pixel values above 40, find the pixel with the smallest coordinate value, and record the coordinate of this pixel as image position data. The method for determining the pixel with the smallest coordinate value is: among all the pixels with coordinate values above 40, select the pixel with the smallest row coordinate value. If there is only one pixel, the pixel is the pixel with the smallest coordinate value. If there are more than one, the pixel with the smallest column coordinate value is selected from the multiple as the pixel with the smallest coordinate value.
前面所述的预选区域,指的是在定位图像中划分出来的一个区域,定位区域的划分方法是:在320×240个像素构成的矩阵中,将前3行、后3行、前3列、后3列的像素去掉后,剩下的像素组成了定位图像的预选区域。也就是说,行坐标为3至236,且列坐标为3至316的全部像素组成了预选区域。The preselected area mentioned above refers to an area divided in the positioning image. The method of dividing the positioning area is: in a matrix composed of 320×240 pixels, the first 3 rows, the last 3 rows, and the first 3 columns After removing the pixels in the last three columns, the remaining pixels constitute the preselected area of the positioning image. That is, all the pixels whose row coordinates are 3 to 236 and column coordinates 3 to 316 constitute the preselected area.
本说明书中的各实施例中涉及到的预选区域,都是相同的选取范围。本实施例中不同特征的定位标志,在定位图像中形成的定位标志影像,其组成像素有不同的数量,组成像素的数量体现定位标志影像的影像特征。如果光源影像不完整,有可能会影响到影像特征的判定,从而导致根据影像特征来获得的特征代码出现错误。而在预选区域中找到的定位标志影像,其全部的组成像素肯定都在整个320×240的像素矩阵内。这是设置预选区域的目的。The preselected regions involved in the embodiments in this specification are all within the same selection range. In this embodiment, the positioning marks with different characteristics and the positioning mark images formed in the positioning images have different numbers of constituent pixels, and the number of constituent pixels reflects the image characteristics of the positioning mark images. If the image of the light source is incomplete, it may affect the determination of the image feature, which may cause an error in the feature code obtained from the image feature. And for the positioning mark image found in the preselected area, all its constituent pixels must be in the entire 320×240 pixel matrix. This is the purpose of setting the preselected area.
如果定位图像中定位标志影像较大,即组成像素较多。可以重新划分预选区域,例如,在像素矩阵中将将前面5行、后5行、前5列、后5列的像素去掉后,剩下的像素集合作为预选区域。If the positioning mark image in the positioning image is larger, there are more pixels. The preselected area can be re-divided. For example, after removing the pixels in the first 5 rows, the last 5 rows, the first 5 columns, and the last 5 columns in the pixel matrix, the remaining pixel set is used as the preselected area.
本实施例还提供了一种红外感应条,包括支架300及2个红外光源。如图8所示,支架300为矩形,长25CM,高2CM,厚2CM。正面的左右两端各有1个正方形的孔,其边长分别为2毫米及3毫米。孔内各有1个红外发光二极管,红外发光二极管的亮度相同。这两个能发出红外光的孔就是红外光源,其中边长2毫米的孔具有4平方毫米的发光面积,而边长3毫米的孔有9平方毫米的发光面积。支架300内部可以是中空的,方便容纳红外发光二极管。This embodiment also provides an infrared sensor strip, which includes a bracket 300 and two infrared light sources. As shown in FIG. 8, the bracket 300 is rectangular, 25 cm long, 2 cm high, and 2 cm thick. There is a square hole on the left and right ends of the front, and the side length is 2 mm and 3 mm respectively. There is an infrared light-emitting diode in each hole, and the brightness of the infrared light-emitting diode is the same. The two holes that can emit infrared light are infrared light sources. The hole with a side length of 2 mm has a light-emitting area of 4 square millimeters, and the hole with a side length of 3 mm has a light-emitting area of 9 square millimeters. The inside of the bracket 300 may be hollow, which is convenient for accommodating the infrared light-emitting diodes.
这2个红外光源可以作为2个定位标志,和本实施例的空中鼠标一起组成空中鼠标系统。2个红外光源在定位图像中形成的影像,有不同的组成数量。例如,空中鼠标和红外光源相距1M左右时,发光面积为4平方毫米的红外光源形成的影像,由4个像素组成;发光面积为9平方毫米的红外光源形成的影像,由9个像素组成。处理单元根据定位标志影像所具有的像素数量来确定特征代码。These two infrared light sources can be used as two positioning marks to form an air mouse system together with the air mouse of this embodiment. The images formed by the two infrared light sources in the positioning image have different composition numbers. For example, when the distance between the air mouse and the infrared light source is about 1M, the image formed by the infrared light source with a light-emitting area of 4 square millimeters is composed of 4 pixels; the image formed by the infrared light source with a light-emitting area of 9 square millimeters is composed of 9 pixels. The processing unit determines the feature code according to the number of pixels in the positioning mark image.
下面具体说明本实施例的空中鼠标获取特征代码和影像位置数据的方法。The method for obtaining feature codes and image position data by the air mouse of this embodiment will be specifically described below.
图像采集单元输出的1帧定位图像中,有4个像素的像素值为100,坐标分别是为(120,3)、(120,4)、(121,3)、(121,4)。In one frame of positioning image output by the image acquisition unit, 4 pixels have pixel values of 100, and the coordinates are (120, 3), (120, 4), (121, 3), (121, 4), respectively.
处理单元首先在预选区域中,按照由左至右,由上至下的顺序,逐个读取每个像素的像素值,读到坐标为(120,3)的像素时,读取到第1个像素值在40以上的像素。The processing unit first reads the pixel value of each pixel in the preselected area in the order from left to right and top to bottom. When it reads the pixel with coordinates (120, 3), it reads the first one. Pixels with pixel values above 40.
以该像素为中心,读取周围7像素×7像素范围内49个像素的像素值。即读取行坐标为117至123,且列坐标为0至6范围内全部像素的像素值。发现有4个像素的像素值为100,坐标分别为(120,3)、(120,4)、(121,3)、(121,4)。With this pixel as the center, the pixel values of 49 pixels in the surrounding 7 pixels×7 pixels range are read. That is, the row coordinates are 117 to 123, and the column coordinates are the pixel values of all pixels in the range of 0 to 6. It is found that the pixel value of 4 pixels is 100, and the coordinates are (120, 3), (120, 4), (121, 3), (121, 4).
因为找出的像素值在40以上的像素共有4个,所以将特征代码记录为4;同时,根据坐标值最小像素的坐标,将影像位置数据记录为(120,3)。Because there are 4 pixels with a pixel value above 40, the feature code is recorded as 4. At the same time, the image position data is recorded as (120, 3) according to the coordinates of the pixel with the smallest coordinate value.
图像处理结束,处理单元将获得的特征代码及对应的影像位置数据向被控设备发送。When the image processing ends, the processing unit sends the obtained feature code and corresponding image position data to the controlled device.
本实施例的红外感应条的另一种实施方式如图9所示,支架300的正面的左端开有1个纵向的矩形孔。右端开有两个纵向的矩形孔,两个矩形孔相距1毫米。3个矩形孔的高度都是4毫米,宽度都是1毫米。每个矩形孔后面设置有亮度相同的红外发光二极管,右端的两个矩形孔可以共用1个红外发光二极管。该3个孔可以作为两个定位标志,其中左端的孔作为一个定位标志,右端的两个孔共同作为一个定位标志。该两个定位标志在定位图像中形成的影像,由不同数量的像素组成。Another implementation of the infrared sensor strip of this embodiment is shown in FIG. 9, a longitudinal rectangular hole is opened on the left end of the front of the bracket 300. There are two longitudinal rectangular holes at the right end, and the two rectangular holes are 1 mm apart. The height of the three rectangular holes is 4 mm, and the width is 1 mm. An infrared light emitting diode with the same brightness is arranged behind each rectangular hole, and the two rectangular holes at the right end can share one infrared light emitting diode. The three holes can be used as two positioning marks, wherein the hole at the left end serves as a positioning mark, and the two holes at the right end together serve as a positioning mark. The images formed by the two positioning marks in the positioning image are composed of different numbers of pixels.
在本实施例的红外感应条的一种实施方式中,包括一个支架及3个红外光源。该支架为矩形,长45CM,高2CM,厚2CM。支架的正面开有3个正方形的孔,两端靠近端头的位置各一个,正中间的位置一个。两端的正方孔,边长为2毫米;中间的孔,边长为3毫米。3个孔后面设有3个具有相同亮度的红外发光二极管。优选的,该3个红外发光二级管的光通量为10流明。该3个能发出红外光的正方形孔就是3个红外光源,能作为3个定位标志和空中鼠标构成空中鼠标系统。相对于两个定位标志的方案,能进一步增加空中鼠标的移动长度。In an implementation of the infrared sensor strip of this embodiment, it includes a bracket and three infrared light sources. The bracket is rectangular, 45cm long, 2cm high, and 2cm thick. There are 3 square holes on the front of the bracket, one at each end near the end and one at the middle. The square holes at both ends have a side length of 2 mm; the middle hole has a side length of 3 mm. There are 3 infrared light-emitting diodes with the same brightness behind the 3 holes. Preferably, the luminous flux of the three infrared light-emitting diodes is 10 lumens. The three square holes that can emit infrared light are three infrared light sources, which can be used as three positioning marks and an air mouse to form an air mouse system. Compared with the solution of two positioning marks, it can further increase the moving length of the air mouse.
在本实施例的红外感应条的一种实施方式中,包括一个支架及4个红外光源。优选的,该支架为矩形,长25CM,高8CM。在矩形支架的四个角,各开有一个正方形的孔。每个孔有不同的边长。左上角的孔边长为1毫米,左下角的孔边长为2毫米,右上角的孔边长为3毫米,右下角的孔,边长为4毫米。每个孔的后面都有一个红外发光二级管。优选的,该4个红外发光二极管的光通量为10流明。该4个能够发出红外光的孔就是所述的红外光源,能够作为4个定位标志,和空中鼠标组成空中鼠标系统。和2个定位标志的方案相比,空中鼠标能够同时增加在横向、纵向及斜向上的移动长度。本实施方式的红外感应条,可以如图10所示,将4个红外光源设置在一个矩形的框上,这样能够减小重量,并增加美观度。In an implementation of the infrared sensor strip of this embodiment, it includes a bracket and 4 infrared light sources. Preferably, the bracket is rectangular with a length of 25 cm and a height of 8 cm. A square hole is opened at each of the four corners of the rectangular bracket. Each hole has a different side length. The hole in the upper left corner has a side length of 1 mm, the hole in the lower left corner has a side length of 2 mm, the hole in the upper right corner has a side length of 3 mm, and the hole in the lower right corner has a side length of 4 mm. There is an infrared light-emitting diode behind each hole. Preferably, the luminous flux of the 4 infrared light-emitting diodes is 10 lumens. The four holes that can emit infrared light are the infrared light sources, which can be used as four positioning marks to form an air mouse system with an air mouse. Compared with the solution of 2 positioning marks, the air mouse can increase the moving length in the horizontal, vertical and diagonal directions at the same time. In the infrared sensor strip of this embodiment, as shown in FIG. 10, four infrared light sources can be arranged on a rectangular frame, which can reduce the weight and increase the aesthetics.
在本实施例的一种实施方式中,处理单元在对一帧定位图像进行处理时,获取其中全部定位标志影像的特征代码及影像位置数据。采用的图像处理方法是:In an implementation of this embodiment, when processing a frame of positioning image, the processing unit obtains the feature codes and image position data of all positioning mark images therein. The image processing methods used are:
S1.读取预选区域内所有像素的像素值,找出其中全部像素值在40以上的像素作为备选像素。S1. Read the pixel values of all pixels in the preselected area, and find all pixels with a pixel value above 40 as candidate pixels.
S2.从备选像素中选择一个坐标值最小像素,以该像素为中心,读取周围7像素×7像素范围内全部像素的像素值,找出其中全部像素值在40以上的像素,这些像素称为组成像素。S2. Select a pixel with the smallest coordinate value from the candidate pixels, take this pixel as the center, read the pixel values of all the pixels in the surrounding 7 pixels × 7 pixels, and find all the pixels with pixel values above 40. These pixels Called component pixels.
S3.将组成像素的总数记录为特征代码;然后在组成像素中,找出一个坐标值最小像素,将该像素的坐标记录为该特征代码对应的影像位置数据。S3. Record the total number of constituent pixels as a feature code; then, among constituent pixels, find a pixel with the smallest coordinate value, and record the coordinate of the pixel as the image position data corresponding to the feature code.
S4.在备选像素中去掉组成像素,如果还剩下有备选像素,则又从第二步,开始处理;如果没有备选像素了,则图像处理过程结束。S4. Remove the constituent pixels from the candidate pixels. If there are still candidate pixels, start processing from the second step; if there are no candidate pixels, the image processing process ends.
下面举例说明该图像处理方法。The following example illustrates the image processing method.
图像采集单元输出的1帧定位图像中,共有5个像素值为100的像素,其坐标分别是(5,6)、(10,12)、(10、13)、(11,12)、(11,13)。其余像素的像素值为0。其中像素(5,6)为定位标志A影像,其余几个像素组成定位标志B影像。处理单元进行如下图像处理:In the 1 frame of positioning image output by the image acquisition unit, there are a total of 5 pixels with a pixel value of 100, and their coordinates are (5, 6), (10, 12), (10, 13), (11, 12), ( 11, 13). The pixel value of the remaining pixels is 0. Among them, the pixel (5, 6) is the location mark A image, and the remaining pixels form the location mark B image. The processing unit performs the following image processing:
S1.读取预选区域内所有像素的像素值,找到5个像素值在40以上的像素作为备选像素,坐标分别是(5,6)、(10,12)、(10、13)、(11,12)、(11,13)。S1. Read the pixel values of all pixels in the preselected area, and find 5 pixels with a pixel value above 40 as candidate pixels. The coordinates are (5, 6), (10, 12), (10, 13), ( 11, 12), (11, 13).
S2.找到坐标值最小像素(5,6),读取该像素周围7像素×7像素范围内全部像素的像素值后,找到的组成像素是(5,6)。S2. Find the pixel with the smallest coordinate value (5, 6). After reading the pixel values of all pixels in the range of 7 pixels×7 pixels around the pixel, the found component pixel is (5, 6).
S3.组成像素为1,所以将特征代码记录为1;将影像位置数据记录为(5,6)。S3. The constituent pixel is 1, so the feature code is recorded as 1; the image position data is recorded as (5, 6).
S4.去掉组成像素(5,6)后,剩下备选像素(10,12)、(10、13)、(11,12)、(11,13)。S4. After removing the constituent pixels (5, 6), candidate pixels (10, 12), (10, 13), (11, 12), (11, 13) are left.
返回到S2,找到坐标值最小像素(10,12),读取该像素周围7像素×7像素范围内全部像素的像素值后,找到组成像素(10,12)、(10、13)、(11,12)、(11,13)。将特征代码记录为4,影像位置数据记录为(10,12)。在备选像素中去掉组成像素后,无剩余的备选像素。Return to S2, find the pixel with the smallest coordinate value (10, 12), read the pixel values of all pixels in the range of 7 pixels × 7 pixels around the pixel, and find the constituent pixels (10, 12), (10, 13), ( 11, 12), (11, 13). The feature code is recorded as 4, and the image position data is recorded as (10, 12). After removing the constituent pixels from the candidate pixels, there are no remaining candidate pixels.
图像处理过程结束,获取到的影像数据为:特征代码1及对应的影像位置数据(5,6);The image processing process ends, and the acquired image data are: feature code 1 and corresponding image position data (5, 6);
特征代码及对应的影像位置数据(10,12)。Feature code and corresponding image location data (10, 12).
该实施例中的几种红外感应条,将其中的红外发光二极管更换为发出可见光的发光二级管。也能够为空中鼠标提供定位标志,此时空中鼠标的图像采集单元 为可以拍摄可见光的图像采集单元。优选的,发光二极管的光通量为10流明。In the infrared sensor strips in this embodiment, the infrared light-emitting diodes are replaced with light-emitting diodes that emit visible light. It can also provide positioning marks for the air mouse. At this time, the image acquisition unit of the air mouse is an image acquisition unit that can shoot visible light. Preferably, the luminous flux of the light emitting diode is 10 lumens.
实施例三Example three
本实施例提供了一种空中鼠标,和实施例一中的空中鼠标区别在于获取影像数据的方法。本实施例的空中鼠标的处理单元对定位图像采用如下图像处理方法:This embodiment provides an air mouse, which is different from the air mouse in the first embodiment in the method of acquiring image data. The processing unit of the air mouse in this embodiment adopts the following image processing method for the positioning image:
S1.在预选区域内,根据由左至右,由上至下的顺序读取每个像素的像素值。直到找到像素值在40以上的像素。S1. In the preselected area, read the pixel value of each pixel in the order from left to right and top to bottom. Until you find a pixel with a pixel value above 40.
S2.以该像素为中心,读取7×7范围内总共49个像素的像素值,找出其中全部像素值在40以上的像素。如果找到的全部像素的列坐标相同,将特征代码记录为1;如果行坐标相同,将特征代码记录为2。然后在找到的全部像素中,找出坐标值最小像素,将该像素的坐标记录为特征代码所对应的影像位置记录。S2. With this pixel as the center, read the pixel values of a total of 49 pixels in the 7×7 range, and find all the pixels with pixel values above 40. If the column coordinates of all the pixels found are the same, record the feature code as 1; if the row coordinates are the same, record the feature code as 2. Then, among all the found pixels, the pixel with the smallest coordinate value is found, and the coordinate of the pixel is recorded as the image position record corresponding to the feature code.
本实施例提供了一种红外感应条。该红外感应条如图11所示,包括一个支架及两个红外光源。支架300为矩形,正面的两端各开有一个矩形孔。两个矩形孔的长度都是4毫米,宽度都是1毫米。位于左端的为纵向的孔,位于右端的为横向的孔。每个孔后面都有1个红外发光二级管,优选的,两个红外发光二级管的光通量都是10流明。这两个能发出红外光的矩形孔就是两个红外光源。能够作为两个定位标志,和本实施例的空中鼠标构成空中鼠标系统。This embodiment provides an infrared sensor strip. The infrared sensor strip is shown in Figure 11 and includes a bracket and two infrared light sources. The bracket 300 is rectangular, with a rectangular hole at each end of the front surface. The length of the two rectangular holes is 4 mm, and the width is 1 mm. The left end is a longitudinal hole, and the right end is a horizontal hole. There is an infrared light-emitting diode behind each hole. Preferably, the luminous flux of the two infrared light-emitting diodes is 10 lumens. The two rectangular holes that can emit infrared light are two infrared light sources. It can be used as two positioning marks to form an air mouse system with the air mouse of this embodiment.
下面举例说明本实施例的空中鼠标的图像处理方法:The following example illustrates the image processing method of the air mouse in this embodiment:
图像采集单元输出的1帧定位图像中,坐标为(119,316)、(120,316)、(121,316)、(122,316)的4个像素的像素值为100,其余像素的像素值为0。处理单元采用如下方法获取特征代码及影像位置数据。In the 1 frame of positioning image output by the image acquisition unit, the 4 pixels with coordinates (119,316), (120,316), (121,316), (122,316) have a pixel value of 100, and the pixels of the remaining pixels The value is 0. The processing unit uses the following methods to obtain feature codes and image location data.
S1.在预选区域中,按照由左至右,由上至下的顺序,逐个读取每个像素的像素值,读到坐标为(119,316)的像素时,找到第1个像素值在40以上的像素。S1. In the preselected area, read the pixel value of each pixel one by one in the order from left to right and top to bottom. When the pixel with coordinates (119, 316) is read, the first pixel value is found More than 40 pixels.
S2.以该像素为中心,读取周围7像素×7像素范围内49个像素的像素值。即读取行坐标S2. With this pixel as the center, read the pixel values of 49 pixels in the surrounding 7 pixels×7 pixels range. Read row coordinates
为116至122,且列坐标为313至319范围内全部像素的像素值。结果为:有4个像素的像素值在40以上,坐标分别为(119,316)、(120,316)、(121,316)、(122,316)。It is 116 to 122, and the column coordinates are the pixel values of all pixels in the range of 313 to 319. The result is that there are 4 pixels with pixel values above 40, and the coordinates are (119, 316), (120, 316), (121, 316), (122, 316).
因为找出的4个像素的列坐标相同,所以将特征代码记录为1。在找出的4 个像素中,坐标值最小像素的坐标为(119,316),该坐标就是特征代码1所对应的影像位置数据。Because the column coordinates of the 4 pixels found are the same, the feature code is recorded as 1. Among the 4 pixels found, the coordinate of the pixel with the smallest coordinate value is (119, 316), which is the image position data corresponding to feature code 1.
上述处理方法,在1帧定位图像中获取1个定位标志影像的特征代码及影像位置数据。在本实施例的一种实施方式中,处理单元对1帧定位图像进行处理时,获取其中全部定位标志影像的特征代码及影像位置数据,具体方法如下:The above-mentioned processing method obtains the feature code and image position data of one positioning mark image in one positioning image. In an implementation of this embodiment, when the processing unit processes one frame of positioning image, it acquires the feature codes and image position data of all the positioning mark images therein. The specific method is as follows:
S1.读取预选区域内所有像素的像素值,找出其中全部像素值在40以上的像素作为备选像素。S1. Read the pixel values of all pixels in the preselected area, and find all pixels with a pixel value above 40 as candidate pixels.
S2.从备选像素中选择一个坐标值最小像素,以该像素为中心,读取周围7像素×7像素范围内全部像素的像素值,找出其中全部像素值在20以上的像素,这些像素称为组成像素。S2. Select a pixel with the smallest coordinate value from the candidate pixels, take this pixel as the center, read the pixel values of all pixels in the surrounding 7 pixels × 7 pixels, and find all the pixels with pixel values above 20. These pixels Called component pixels.
S3.如果找出的全部组成像素的列坐标相同,则将影像代码记录为1;如果全部组成像素的行坐标相同,则将影像代码记录为2。然后在全部组成像素中,找出一个坐标值最小像素,将该像素的坐标记录为影像位置。S3. If the column coordinates of all the constituent pixels found are the same, the image code is recorded as 1; if the row coordinates of all the constituent pixels are the same, the image code is recorded as 2. Then, among all the constituent pixels, find a pixel with the smallest coordinate value, and record the coordinate of this pixel as an image position.
S4.在备选像素中,去掉和组成像素相同坐标的像素(有几个算几个),如果还剩下有备选像素,则又从第二步开始处理;如果没有备选像素了,则获取影像数据的步骤结束。S4. Among the candidate pixels, remove the pixels with the same coordinates as the constituent pixels (there are a few count as several), if there are still candidate pixels, start processing from the second step; if there are no candidate pixels, Then the step of acquiring image data ends.
本实施方式中,获取影像数据步骤的图像处理方法如图12所示。In this embodiment, the image processing method in the step of acquiring image data is shown in FIG. 12.
下面举例说明处理步骤。The following examples illustrate the processing steps.
图像采集单元输出的1帧定位图像中,有8个像素值为100的像素,坐标分别是(119,3)、(120,3)、(121,3)、(122,3)、(120,315)、(120,316)、(120,317)、(120,318)。其中前4个像素组成了定位标志A影像,后4个像素组成了定位标志B影像。In the 1 frame of positioning image output by the image acquisition unit, there are 8 pixels with a pixel value of 100. The coordinates are (119, 3), (120, 3), (121, 3), (122, 3), (120). , 315), (120,316), (120,317), (120,318). The first 4 pixels form the location mark A image, and the last 4 pixels form the location mark B image.
处理单元采用如下方法获取特征代码及影像位置数据:The processing unit uses the following methods to obtain feature codes and image location data:
S1.读取预选区域内所有像素的像素值,找到6个像素值在40以上的像素作为备选像素,其坐标分别是(119,3)、(120,3)、(121,3)、(122,3)、(120,315)、(120,316)。S1. Read the pixel values of all pixels in the preselected area, and find 6 pixels with a pixel value above 40 as candidate pixels. The coordinates are (119, 3), (120, 3), (121, 3), (122,3), (120,315), (120,316).
S2.在备选像素中选出一个坐标值最小像素,其坐标是(119,3)。以该像素为中心,读取周围7像素×7像素范围内全部像素的像素值,找到4个像素值在40以上的像素作为组成像素(该4个像素是组成定位标志A影像的全部像素),坐标分别是(119,3)、(120,3)、(121,3)、(122,3)。S2. Select a pixel with the smallest coordinate value among the candidate pixels, and its coordinate is (119, 3). With this pixel as the center, read the pixel values of all pixels in the surrounding 7 pixels × 7 pixels range, and find 4 pixels with a pixel value of 40 or more as constituent pixels (the 4 pixels are all the pixels that make up the image of the positioning mark A) , The coordinates are (119, 3), (120, 3), (121, 3), (122, 3).
S3.4个组成像素的列坐标相同,将特征代码记录为1;在4个组成像素中找出一个坐标值最小像素,其坐标为(119,3)。将该坐标记录为特征代码1所对应的影像位置数据。S3.4 The column coordinates of the constituent pixels are the same, and the feature code is recorded as 1. Find a pixel with the smallest coordinate value among the 4 constituent pixels, and its coordinate is (119, 3). Record the coordinates as the image position data corresponding to feature code 1.
S4.在备选像素中,去掉组成像素以后。还剩下两个备选像素,坐标分别为(120、315)、(120,316),所以继续进行如下操作:S4. In the candidate pixels, after removing the constituent pixels. There are two candidate pixels left, the coordinates are (120, 315), (120, 316), so proceed as follows:
在备选像素中选出坐标值最小像素,其坐标是(120,315)。以该像素为中心,读取周围7像素×7像素范围内全部像素的像素值,找到4个像素值在20以上的像素作为组成像素(该4个像素是组成定位标志B影像的全部像素),坐标分别是(120,315)、(120,316)、(120,317)、(120,318)。Among the candidate pixels, the pixel with the smallest coordinate value is selected, and its coordinate is (120,315). With this pixel as the center, read the pixel values of all pixels in the surrounding 7 pixels × 7 pixels range, and find 4 pixels with a pixel value of 20 or more as the constituent pixels (the 4 pixels are all the pixels that make up the positioning mark B image) , The coordinates are (120,315), (120,316), (120,317), (120,318).
4个组成像素的行坐标相同,将特征代码记录为2;在4个组成像素中找出一个坐标值最小像素,其坐标为(120,315),将该坐标记录为特征代码2所对应的影像位置记录。The row coordinates of the 4 constituent pixels are the same, and the feature code is recorded as 2. Find a pixel with the smallest coordinate value among the 4 constituent pixels, and its coordinate is (120,315), and record this coordinate as the corresponding feature code 2 Image location record.
在2个备选像素中,去掉组成像素后,没有剩余的备选像素,图像处理过程。获取到的影像数据包括特征代码1及对应的影像位置数据(119,3);特征代码2及对应的影像位置数据(120,315)。In the 2 candidate pixels, after removing the constituent pixels, there are no remaining candidate pixels, the image processing process. The acquired image data includes feature code 1 and corresponding image location data (119, 3); feature code 2 and corresponding image location data (120, 315).
在本实施例的红外感应条的一种实施方式中,包括一个支架和3个红外光源。如图13所示,该支架为矩形,长45CM,高2CM,厚2CM。矩形支架的正面开有3个矩形孔,两端靠近端头的位置各一个,正中间的位置一个。3个矩形孔的长度都是4毫米,宽度都是1毫米两端的矩形孔为纵向;中间矩形孔为横向。两端的孔同中间的孔距离为20CM。3个孔后面设有3个具有相同亮度的红外发光二极管。优选的,该3个红外发光二级管的光通量为10流明。该3个能发出红外光的矩形孔就是3个红外光源,能作为3个定位标志和本实施例的空中鼠标构成空中鼠标系统。相对于两个定位标志的方案,能进一步增加空中鼠标的移动长度。In an implementation of the infrared sensor strip of this embodiment, it includes a bracket and three infrared light sources. As shown in Figure 13, the bracket is rectangular, 45 cm long, 2 cm high, and 2 cm thick. There are 3 rectangular holes on the front of the rectangular bracket, one at each end near the end and one at the middle. The length of the three rectangular holes is 4 mm, and the width is 1 mm. The rectangular holes at both ends are vertical; the middle rectangular hole is horizontal. The distance between the holes at both ends and the middle hole is 20cm. There are 3 infrared light-emitting diodes with the same brightness behind the 3 holes. Preferably, the luminous flux of the three infrared light-emitting diodes is 10 lumens. The three rectangular holes capable of emitting infrared light are three infrared light sources, which can be used as three positioning marks to form an air mouse system with the air mouse of this embodiment. Compared with the solution of two positioning marks, it can further increase the moving length of the air mouse.
在本实施例的红外感应条的一种实施方式中,包括一个支架和多个红外光源。如图14所示,支架300为矩形,长25CM,高8CM。左上角有一个纵向的矩形孔,长度4毫米,宽度1毫米;右上角有1个横向的矩形孔,长度4毫米,宽度1毫米;左下角有两个纵向的矩形孔,该两个矩形孔平行排列,相距1毫米,长度都是2毫米,宽度都是1毫米;在右下角有两个横向的矩形孔,该两个矩形孔平行排列,相距1毫米,长度都是2毫米,宽度都是1毫米。每个孔的后面都 有发光二极管,优选的,红外发光二极管的光通量为10流明。支架上的能够发出红外光的矩形孔能够作为定位标志,其中左上角的纵向矩形孔作为一个;右上角的横向矩形孔作为一个;左下角的两个纵向矩形孔作为一个;右下角的两个横向矩形孔作为一个,一共是4个定位标志。In an implementation of the infrared sensor strip of this embodiment, it includes a bracket and multiple infrared light sources. As shown in Fig. 14, the bracket 300 is rectangular, 25 cm long and 8 cm high. There is a longitudinal rectangular hole in the upper left corner with a length of 4 mm and a width of 1 mm; the upper right corner has a horizontal rectangular hole with a length of 4 mm and a width of 1 mm; the lower left corner has two longitudinal rectangular holes, the two rectangular holes Arranged in parallel, with a distance of 1 mm, a length of 2 mm, and a width of 1 mm; there are two horizontal rectangular holes in the lower right corner, the two rectangular holes are arranged in parallel, with a distance of 1 mm, the length is 2 mm, and the width is both It is 1 mm. There are light-emitting diodes behind each hole. Preferably, the luminous flux of the infrared light-emitting diode is 10 lumens. The rectangular hole on the bracket that can emit infrared light can be used as a positioning mark. The vertical rectangular hole in the upper left corner is used as one; the horizontal rectangular hole in the upper right corner is used as one; the two longitudinal rectangular holes in the lower left corner are used as one; the two in the lower right corner are used as one. As one horizontal rectangular hole, there are 4 positioning marks in total.
该红外感应条为空中鼠标提供定位标志时,图像采集单元输出的定位图像中,各定位标志的影像都由4个像素组成,排列方式各不相同。其中,左上角的定位标志形成的影像,由4个像素纵向排成一排;右上角的定位标志形成的影像,由4个像素横向排成一排;左下角的定位标志,由4个像素纵向排成两排,每排2个像素,两排间的距离为1个像素;右下角的定位标志,其影像由4个像素横向排成两排,每排2个像素,两排间的距离为2个像素;例如,左下角的定位标志,组成其影像的4个像素坐标分别为(0,0)、(0,2)、(1,0)、(1,2);右下角的定位标志,组成其影像的4个像素坐标分别为(0,0)、(0,1)、(2,0)、(2,1)。处理单元确定特征代码的规则是,如果找到的像素值为40的全部像素的列坐标相同,将特征代码记录为1;如果行坐标相同,将特征代码记录为2;如果列坐标为两两相同,且列坐标相同的两个像素,其行坐标的值为相邻数,则将特征代码记录为3;如果列坐标为两两相同,且列坐标相同的两个像素,其行坐标的值不是相邻数,则将特征代码记录为4。When the infrared sensor bar provides positioning marks for the air mouse, in the positioning image output by the image acquisition unit, the image of each positioning mark is composed of 4 pixels, and the arrangement methods are different. Among them, the image formed by the positioning mark in the upper left corner is composed of 4 pixels vertically arranged in a row; the image formed by the positioning mark in the upper right corner is composed of 4 pixels horizontally arranged in a row; the positioning mark in the lower left corner is composed of 4 pixels Arranged in two rows longitudinally, each row has 2 pixels, the distance between the two rows is 1 pixel; the positioning mark at the lower right corner has an image of 4 pixels arranged horizontally in two rows, each row has 2 pixels, and the distance between the two rows The distance is 2 pixels; for example, the positioning mark in the lower left corner, the coordinates of the 4 pixels that make up the image are (0, 0), (0, 2), (1, 0), (1, 2); the lower right corner The positioning mark of, the coordinates of the 4 pixels that make up the image are (0, 0), (0, 1), (2, 0), (2, 1). The rule for the processing unit to determine the feature code is that if the column coordinates of all pixels found with the pixel value of 40 are the same, the feature code is recorded as 1; if the row coordinates are the same, the feature code is recorded as 2; if the column coordinates are the same in pairs , And two pixels with the same column coordinate, the row coordinate value is adjacent, then the feature code is recorded as 3; if the column coordinate is the same two by two, and the column coordinate is the same two pixels, the row coordinate value If it is not a contiguous number, record the feature code as 4.
本实施例中的几种红外感应条,将红外发光二极管替换为发光二极管,也可以为空中鼠标提供定位标志,此时空中鼠标的图像采集单元为可以拍摄可见光景物的图像采集单元。优选的,发光二极管的光通量为10流明。In the several infrared sensor strips in this embodiment, the infrared light-emitting diodes are replaced with light-emitting diodes, which can also provide positioning marks for the air mouse. At this time, the image acquisition unit of the air mouse is an image acquisition unit that can photograph visible light scenes. Preferably, the luminous flux of the light emitting diode is 10 lumens.
实施例四Example four
本实施例提供了一种空中鼠标,和实施例一中的空中鼠标的区别在于对定位图像的处理方式。本实施例的空中鼠标的处理单元对每帧定位图像进行处理时,首先进行图像处理,然后根据图像处理结果向被控设备发送影像位置数据或换标信号。This embodiment provides an air mouse, and the difference from the air mouse in the first embodiment lies in the processing method of the positioning image. When the processing unit of the air mouse of this embodiment processes each frame of positioning image, it first performs image processing, and then sends image position data or a rebranding signal to the controlled device according to the image processing result.
图像处理包括两个步骤:Image processing consists of two steps:
第一步,获取其中定位标志影像的特征代码和影像位置数据。The first step is to obtain the feature code and image location data of the positioning mark image.
第二步,根据所获取的特征代码,在所处理的前一帧定位图像所获取的特征代码中查找相同的,获得结果后,图像处理过程结束。In the second step, according to the acquired characteristic codes, search for the same characteristic codes acquired in the processed previous frame positioning image, and after the results are obtained, the image processing process ends.
根据图像处理的不同结果,处理单元向被控设备发送的数据类型也不同。如 果找到了相同的特征代码,处理单元发送的是该特征代码所对应的影像位置数据;如果没找到相同的特征代码,处理单元发送的是换标信号。处理单元对定位图像的处理方式如图15所示。According to different results of image processing, the type of data sent by the processing unit to the controlled device is also different. If the same feature code is found, the processing unit sends the image position data corresponding to the feature code; if the same feature code is not found, the processing unit sends a rebrand signal. The processing method of the positioning image by the processing unit is shown in FIG. 15.
本实施例还提供了一种驱动程序,安装在空中鼠标的被控设备上,用于根据影像位置数据或换标信号控制所驱动的屏幕上的光标移动。This embodiment also provides a driver program installed on the controlled device of the air mouse for controlling the movement of the cursor on the driven screen according to the image position data or the conversion signal.
驱动程序预设有记录区,用于记录影像位置数据。驱动程序接收到影像位置数据时,记录区中如果有影像位置数据,则将接收到的影像位置数据同记录区中的进行比对,并根据得到的影像位移数据控制屏幕上的光标移动,并将记录区中的数据更新为刚接到的影像位置数据;记录区中如果没有影像位置数据,则将刚接到的影像位置数据存入记录区。驱动程序接收到换标信号时,将记录区清空。The driver has a preset recording area for recording image position data. When the driver receives the image position data, if there is image position data in the recording area, compare the received image position data with that in the recording area, and control the cursor movement on the screen according to the obtained image displacement data, and Update the data in the recording area to the image location data just received; if there is no image location data in the recording area, store the image location data just received in the recording area. When the driver receives the rebranding signal, it will clear the recording area.
本实施例的驱动程序,在接收到影像位置数据时的处理方法,如图16所示。The processing method of the driver of this embodiment when the image position data is received is as shown in FIG. 16.
本实施例中的空中鼠标,在从定位图像中获取特征代码和影像位置数据时,采用实施例一、实施例二、实施例三中说明的各种方法。The air mouse in this embodiment adopts various methods described in Embodiment 1, Embodiment 2, and Embodiment 3 when acquiring feature codes and image position data from a positioning image.
本实施例还提供了一种空中鼠标系统,由本实施例中的空中鼠标和至少两个具有不同特征的定位标志组成。定位标志包括实施例一、实施二、实施三中记录的各种定位标志。This embodiment also provides an air mouse system, which is composed of the air mouse in this embodiment and at least two positioning marks with different characteristics. The positioning marks include various positioning marks recorded in the first, second, and third embodiments.
下面举例说明本实施例的处理方法和驱动程序。The following examples illustrate the processing method and driver of this embodiment.
图像采集单元输出第1帧定位图像,处理单元从中获取到特征代码50及对应的影像位置数据(120,318)。因为是第1帧,并无前一帧的记录。所以向被控设备发送数字1,该数字1即为换标信号。被控设备端的驱动程序收到换标信号后,将记录区中的数据清空。The image acquisition unit outputs the first frame positioning image, and the processing unit acquires the feature code 50 and corresponding image position data (120, 318) from it. Because it is the first frame, there is no record of the previous frame. Therefore, the number 1 is sent to the controlled device, and the number 1 is the standard change signal. The driver of the controlled device will clear the data in the recording area after receiving the rebranding signal.
图像采集单元输出第2帧定位图像,处理单元从中获取到特征代码50及对应的影像位置数据(120,317)。根据特征代码50,在前一帧定位图像所获取的特征代码中找到了相同的。则将(120,317)向被控设备发送。被控设备端的驱动程序在接到此影像位移数据后,因为记录区中为空,则将(120,317)记入记录区。The image acquisition unit outputs the second frame positioning image, from which the processing unit obtains the feature code 50 and the corresponding image position data (120, 317). According to the feature code 50, the same is found in the feature code obtained from the positioning image of the previous frame. Then (120,317) is sent to the controlled device. After receiving the image displacement data, the driver of the controlled device will record (120,317) in the recording area because the recording area is empty.
图像采集单元输出第3帧定位图像,处理单元从中获取到特征代码50及对应的影像位置数据(120,315)。根据特征代码50,在前一帧定位图像所获取的特征代码中找到了相同的。则将(120,315)向被控设备发送。被控设备端的驱动程序在接到此影像位移数据后,将(120,315)同(120,317)进行比对,发 现列坐标的数值减小了2。则控制屏幕上的光标向右移动了2个像素。然后将记录区中的数据更新为(120,,315)The image acquisition unit outputs the third frame positioning image, from which the processing unit obtains the feature code 50 and the corresponding image position data (120, 315). According to the feature code 50, the same is found in the feature code obtained from the positioning image of the previous frame. Then send (120,315) to the controlled device. After receiving the image displacement data, the driver program of the controlled device compares (120,315) with (120,317), and finds that the value of the column coordinate is reduced by 2. The cursor on the control screen has moved 2 pixels to the right. Then update the data in the recording area to (120,,315)
图像采集单元输出第4帧定位图像,处理单元从中获取到特征代码100及对应的影像位置数据(120,312)。根据特征代码100,在前一帧定位图像的特征代码中没有找到相同的,所以向被控设备发送了数字1。被控设备端的驱动程序将记录区中的数据清空。The image acquisition unit outputs the fourth frame positioning image, and the processing unit acquires the feature code 100 and corresponding image position data (120, 312) from it. According to the feature code 100, the same is not found in the feature code of the positioning image of the previous frame, so the number 1 is sent to the controlled device. The driver of the controlled device will clear the data in the recording area.
图像采集单元输出第5帧定位图像,处理单元从中获取到特征代码100及对应的影像位置数据(120,310),根据特征代码100,在上一帧定位图像的特征代码中找到了相同的。(120,310)被发送到被控设备,被控设备端的驱动程序将其记入记录区。The image acquisition unit outputs the fifth frame positioning image, and the processing unit obtains the feature code 100 and corresponding image location data (120, 310) from it. According to the feature code 100, the same is found in the feature code of the previous frame of positioning image. (120, 310) is sent to the controlled device, and the driver of the controlled device writes it into the recording area.
图像采集单元输出第6帧定位图像,处理单元从中获取到特征代码100及对应的影像位置数据(120,308),根据特征代码100,在上一帧定位图像的特征代码中找到了相同的。(120,308)被发送到被控设备,驱动程序将(120,308)同(120,310)进行比对,发现列坐标的数值少了2,则控制屏幕上的光标向右移动了2个像素点。The image acquisition unit outputs the sixth frame positioning image, and the processing unit obtains the feature code 100 and the corresponding image location data (120, 308) from it. According to the feature code 100, the same is found in the feature code of the previous frame of the positioning image. (120,308) is sent to the controlled device, the driver compares (120,308) with (120,310), and finds that the value of the column coordinate is less than 2, and the cursor on the control screen moves to the right by 2 Pixels.
驱动程序将影像位置数据进行比对后,根据比对结果控制屏幕上光标的移动,采用如下规则:如果行坐标的数值增加了n,则控制屏幕上的光标向上移动n个像素;如果行坐标的数值减小了n,则控制屏幕上的光标向下移动n个像素。如果列坐标的数值增加了n,则控制屏幕上的光标向左移动n个像素;如果列坐标的数值减小了n,则控制屏幕上的光标向右移动n个像素;如果行坐标和列坐标的数值都有变化,则控制光标在两个方向上作相应的移动。例如,测得行坐标及列坐标的数值都增加了1,则控制屏幕上的光标向上移动1个像素,再向左移动1个像素。After comparing the image position data, the driver controls the movement of the cursor on the screen according to the comparison result. The following rules are adopted: if the value of the row coordinate increases by n, the cursor on the screen is controlled to move up n pixels; if the row coordinate If the value of is reduced by n, the cursor on the control screen moves down by n pixels. If the value of the column coordinate increases by n, the cursor on the control screen moves n pixels to the left; if the value of the column coordinate decreases by n, the cursor on the control screen moves n pixels to the right; if the row coordinate and column If the values of the coordinates have changed, control the cursor to move correspondingly in two directions. For example, if the measured values of row coordinates and column coordinates are increased by 1, the cursor on the control screen will move up by 1 pixel, and then move to the left by 1 pixel.
本实施例及实施例一至三中,驱动程序和被控设备组成了计算机系统。In this embodiment and the first to third embodiments, the driver and the controlled device constitute a computer system.
从上述过程中可以看到,在作为空中位置参照物的定位标志更换时,空中鼠标控制的屏幕上的光标是不移动的,在参照物更换后,光标继续向原方向移动。本实施例的空中鼠标,也能通过增加定位标志的方式增加移动长度。It can be seen from the above process that the cursor on the screen controlled by the air mouse does not move when the positioning mark used as the air position reference object is replaced. After the reference object is replaced, the cursor continues to move in the original direction. The air mouse of this embodiment can also increase the moving length by adding positioning marks.
实施例五Example five
本实施例提供了一种空中鼠标,和实施例一中的空中鼠标的区别在于对定位图像的处理方式。本实施例的空中鼠标的处理单元对每帧定位图像进行处理时, 通过图像处理获得影像位移数据。This embodiment provides an air mouse, and the difference from the air mouse in the first embodiment lies in the processing method of the positioning image. When the processing unit of the air mouse of this embodiment processes each frame of positioning image, it obtains image displacement data through image processing.
本实施例的空中鼠标的处理单元对每帧定位图像进行图像处理时包括两个步骤:The processing unit of the air mouse of this embodiment includes two steps when performing image processing on each frame of positioning image:
步骤一,获取其中定位标志影像的特征代码及影像位置数据。具体方法包括了实施例一、实施例二及实施例三中说明的各种方法。Step 1: Obtain the feature code and image location data of the positioning mark image. The specific methods include the various methods described in Embodiment 1, Embodiment 2, and Embodiment 3.
步骤二,获取影像位移数据。Step 2: Obtain image displacement data.
具体方法为:根据获取的特征代码,在处理的前一帧定位图像所获取的特征代码中寻找是否有相同的。如果有相同的,则将该特征代码在两帧定位图像中对应的影像位置数据进行比对,根据比对结果获得影像位移数据;如果没有相同的,则本帧定位图像的图像处理结束。The specific method is as follows: according to the acquired characteristic codes, look for the same in the characteristic codes acquired in the processed previous frame positioning image. If there are the same, the corresponding image position data of the feature code in the two frames of positioning images are compared, and the image displacement data is obtained according to the comparison result; if there is no the same, the image processing of the positioning image of this frame ends.
所述图像处理方法如图17所示。The image processing method is shown in FIG. 17.
图像处理过程结束后,如果产生了影像位移数据,处理单元根据该影像位移数据向被控设备发送位移信号,被控设备根据该位移信号控制所驱动的屏幕上的光标作相应的移动。After the image processing process ends, if image displacement data is generated, the processing unit sends a displacement signal to the controlled device according to the image displacement data, and the controlled device controls the cursor on the driven screen to move correspondingly according to the displacement signal.
下面仍以空中鼠标在空中从左向右移动为例,说明本实施例的空中鼠标及图像处理方法。In the following, the air mouse moves from left to right in the air as an example to illustrate the air mouse and image processing method of this embodiment.
空中鼠标和两个定位标志组成了空中鼠标系统,两个定位标志为两个红外光源,实施方式和图2中展示的一样,两个红外光源横向排列,分别作为定位标志A及定位标志B。The air mouse and two positioning marks form an air mouse system. The two positioning marks are two infrared light sources. The implementation is the same as that shown in Figure 2. The two infrared light sources are arranged horizontally as positioning mark A and positioning mark B respectively.
处理单元所采用的图像处理程序要先建立一个记录区,用于记录从定位图像中获得的数据(包括特征代码和影像位置数据),并在每一帧定位图像处理过程结束时,更新该记录区中的数据。The image processing program used by the processing unit must first establish a recording area for recording the data (including feature codes and image position data) obtained from the positioning image, and update the record at the end of each frame positioning image processing process Data in the area.
使用者手持空中鼠标将镜头202朝向定位标志A。图像采集单元开始拍摄并输出第1帧定位图像,该帧定位图像中,坐标为(120,318)、(120,319)、(121,318)、(121,319)的四个像素的像素值为50,其余像素的像素值为0。这四个像素组成了定位标志A影像。处理单元从此帧定位图像中获取的影像数据为50和(120,318)。The user holds the air mouse and points the lens 202 toward the positioning mark A. The image acquisition unit starts to capture and output the first frame of positioning image. In this frame of positioning image, the four pixels with coordinates (120,318), (120,319), (121,318), (121,319) The value is 50, and the pixel value of the remaining pixels is 0. These four pixels form the positioning mark A image. The image data obtained by the processing unit from this frame positioning image is 50 and (120, 318).
处理单元根据特征代码50,在记录区中寻找相同的。因为这是处理的第一帧定位图像,记录区中并无数据,将50和(120,318)存入记录区后,第1帧定位图像处理结束。The processing unit searches for the same in the recording area according to the feature code 50. Because this is the first frame positioning image processed, there is no data in the recording area. After storing 50 and (120, 318) in the recording area, the first frame positioning image processing ends.
使用者手持空中鼠标继续向右移动,图像采集单元输出的第二帧定位图像中,坐标为(120,317)、(120,318)、(121,317)、(121,318)的四个像素的像素值为50,其余像素的像素值为0。处理单元从此帧定位图像中获取了特征代码50及影像位置数据(120,317)。The user holds the air mouse and continues to move to the right. In the second frame of the positioning image output by the image acquisition unit, the coordinates are (120,317), (120,318), (121,317), (121,318). The pixel value of the pixel is 50, and the pixel value of the remaining pixels is 0. The processing unit obtains the feature code 50 and image location data (120, 317) from this frame positioning image.
处理单元根据特征代码50,在记录区中找到了相同的。将(120,317)同(120,318)进行比对,发现列坐标的数值减小了1。这个就是影像位移数据。The processing unit found the same in the recording area according to the feature code 50. Comparing (120,317) with (120,318), it is found that the value of the column coordinate is reduced by 1. This is the image displacement data.
根据此影像位移数据,处理单元向被控设备发送位移信号,位移信号中包含的信息是向右移动1。将记录区中的数据更新为50、(120,317)后,本帧定位图像处理结束。被控设备根据位移信号控制所驱动的屏幕上的光标向右(屏幕面对使用者)移动了1个像素。According to the image displacement data, the processing unit sends a displacement signal to the controlled device, and the information contained in the displacement signal is 1 to the right. After updating the data in the recording area to 50, (120, 317), the frame positioning image processing ends. The controlled device controls the cursor on the driven screen to move 1 pixel to the right (the screen faces the user) according to the displacement signal.
使用者手持空中鼠标继续向右移动,图像采集单元持续拍摄并输出定位图像,在输出的一系列定位图像中,定位标志A影像的位置逐渐向左移动,处理单元根据图像处理结果持续输出位移信号,被控设备根据该位移信号控制屏幕上的光标持续向右移动。The user holds the air mouse and continues to move to the right. The image acquisition unit continues to capture and output positioning images. In the output series of positioning images, the position of the positioning mark A image gradually moves to the left, and the processing unit continues to output displacement signals according to the image processing results , The controlled device controls the cursor on the screen to continuously move to the right according to the displacement signal.
图像采集单元输出第n帧定位图像时,处理单元从该定位图像中获得特征代码50及对应的影像位置数据(120,0),此时,定位标志A已经到达图像采集单元的镜头的可视范围的边缘。此帧定位图像处理完成后,记录区中的数据为50、(120,0)。When the image acquisition unit outputs the nth frame of the positioning image, the processing unit obtains the feature code 50 and the corresponding image position data (120, 0) from the positioning image. At this time, the positioning mark A has reached the visibility of the lens of the image acquisition unit The edge of the range. After the frame positioning image processing is completed, the data in the recording area is 50, (120, 0).
使用者手持空中鼠标向右继续移动一段距离后,镜头中只有定位标志B。此时图像采集单元输出第n+1帧定位图像,其中有四个像素的像素值为100,其余像素的像素值为0。该四个像素组成了定位标志B影像,该四个像素的坐标分别是(120,317)、(120,318)、(121,317)、(121,318)。After the user holds the air mouse and moves to the right for a certain distance, there is only the positioning mark B in the lens. At this time, the image acquisition unit outputs a positioning image of the n+1th frame, in which four pixels have a pixel value of 100, and the remaining pixels have a pixel value of 0. The four pixels constitute the positioning mark B image, and the coordinates of the four pixels are (120, 317), (120, 318), (121, 317), (121, 318), respectively.
处理单元在该帧定位图像中获取到的影像数据是100及(120,317)。根据特征代码100,在记录区中没有找到相同的。将记录区中的数据更新为100、(120,317)后。本帧定位图像的图像处理过程结束。The image data obtained by the processing unit in the frame positioning image is 100 and (120, 317). According to the feature code 100, the same is not found in the recording area. After updating the data in the recording area to 100, (120, 317). The image processing process of this frame positioning image ends.
使用者手持空中鼠标继续向右移动一段微小的距离后,图像采集单元输出的一帧定位图像中,有4个像素的像素值为100,其余像素的像素值为0。该4个像素的坐标分别是:(120,316)、(120,317)、(121,316)、(121,317)。处理单元对该帧定位图像进行图像处理后,向被控设备发送向右移动1的位移信号,被控设备控制屏幕上的光标向右移动了1个像素。空中鼠标向右持续移动,屏幕 上的光标也向右持续移动,直到定位标志B移出了镜头的可视范围。After the user holds the air mouse and continues to move a small distance to the right, in a frame of positioning image output by the image acquisition unit, 4 pixels have a pixel value of 100 and the remaining pixels have a pixel value of 0. The coordinates of the 4 pixels are (120, 316), (120, 317), (121, 316), (121, 317), respectively. After the processing unit performs image processing on the frame positioning image, it sends a displacement signal moving 1 to the right to the controlled device, and the controlled device controls the cursor on the screen to move 1 pixel to the right. The air mouse continues to move to the right, and the cursor on the screen also continues to move to the right, until the positioning mark B moves out of the visual range of the lens.
本实施例中,根据特征代码和影像位置数据获得影像位移数据的方法,和实施例一中的驱动程序根据特征代码和影像位置数据获得影像位移数据的方法是一样的。所获得的影像位移数据也都作为控制屏幕上光标移动的依据,所以有益效果也是一样的。In this embodiment, the method of obtaining the image displacement data according to the feature code and the image position data is the same as the method of obtaining the image displacement data according to the feature code and the image position data by the driver in the first embodiment. The obtained image displacement data is also used as the basis for controlling the cursor movement on the screen, so the beneficial effects are also the same.
本实施例中,影像位置数据进行比对时,要将行坐标和列坐标分别进行比对,找出数值的变化。比对结果包括四种情况,行坐标增加、行坐标减小、列坐标增加、列坐标减小。本实施例的空中鼠标根据影像位移数据发送位移信号,采用的规则是:如果行坐标的数值增加了n,则输出向上移动n的位移信号;如果行坐标的数值减小了n,则输出向下移动n的位移信号。如果列坐标的数值增加了n,则输出向左移动n的位移信号;如果列坐标的数值减小了n,则输出向右移动n的位移信号;被控设备根据接收到的位移信号,控制所驱动的屏幕上的光标向相应的方向移动n个像素。如果行坐标和列坐标的数值都有变化,则位移信号中都要有体现。例如,测得行坐标及列坐标的数值都增加了1,则输出的位移信号中包含的信息为向上移动1和向左移动1。控制设备接收到此位移信号,控制屏幕上的光标向上移动1个像素,再向左移动1个像素。位移信号的具体格式可以参照光学鼠标的数据格式。In this embodiment, when the image position data is compared, the row coordinates and column coordinates are compared separately to find out the changes in the values. The comparison result includes four situations: row coordinates increase, row coordinates decrease, column coordinates increase, and column coordinates decrease. The air mouse of this embodiment sends a displacement signal according to the image displacement data. The rule adopted is: if the value of the row coordinate increases by n, it outputs a displacement signal that moves upward by n; if the value of the row coordinate decreases by n, it outputs to Move down the displacement signal of n. If the value of the column coordinate increases by n, it outputs a displacement signal that moves to the left by n; if the value of the column coordinate decreases by n, it outputs a displacement signal that moves to the right by n; the controlled device controls according to the received displacement signal The cursor on the driven screen moves n pixels in the corresponding direction. If the values of row coordinates and column coordinates have changed, they must be reflected in the displacement signal. For example, if the measured values of the row coordinate and the column coordinate both increase by 1, the information contained in the output displacement signal is 1 move up and 1 left. The control device receives this displacement signal and controls the cursor on the screen to move up by 1 pixel, and then move to the left by 1 pixel. The specific format of the displacement signal can refer to the data format of the optical mouse.
在本实施例的空中鼠标的一种实施方式中,根据影像位移数据发送位移信号的规则为:列坐标的数值增加,则输出向左移动的位移信号;列坐标的数值减小,则输出向右移动的位移信号。这样在横向移动时,屏幕上光标的移动方向和空中鼠标的移动方向相反。纵向移动的方向,也可以根据同样的方法修改。In an implementation of the air mouse of this embodiment, the rule for sending displacement signals according to the image displacement data is: when the value of the column coordinate increases, the displacement signal that moves to the left is output; when the value of the column coordinate decreases, the output is Displacement signal for right movement. In this way, when moving horizontally, the moving direction of the cursor on the screen is opposite to that of the air mouse. The direction of longitudinal movement can also be modified in the same way.
在本实施例的空中鼠标的一种实施方式中,处理单元将影像位移数据乘以2后输出。例如测得列坐标的数值减小了1,输出向右移动2的位移信号。根据需要。也可以将影像位移数据乘以3、4或更大后输出。In an implementation of the air mouse of this embodiment, the processing unit multiplies the image displacement data by 2 and outputs it. For example, the measured value of the column coordinate is reduced by 1, and the displacement signal of 2 to the right is output. base on needs. You can also multiply the image displacement data by 3, 4 or more and output it.
实施例六Example Six
本实施例提供了一种空中鼠标,如图18所示该空中鼠标包括壳体、图像采集单元、处理单元。图像采集单元用于拍摄包括定位标志的景物,并输出有定位标志影像的定位图像;处理单元对所述定位图像进行图像处理,并根据图像处理结果将光标控制信号向被控设备发送。对定位图像进行图像处理时,包括获取其中定位标志影像的特征代码及影像位置数据。This embodiment provides an air mouse. As shown in FIG. 18, the air mouse includes a housing, an image acquisition unit, and a processing unit. The image acquisition unit is used to photograph a scene including the positioning mark and output a positioning image with the positioning mark image; the processing unit performs image processing on the positioning image, and sends a cursor control signal to the controlled device according to the image processing result. When performing image processing on the positioning image, it includes obtaining the feature code of the positioning mark image and the image position data.
本实施例的空中鼠标,等于在实施例一至五的空中鼠标的基础上去掉了按键单元。The air mouse of this embodiment is equivalent to removing the button unit on the basis of the air mouse of the first to fifth embodiments.
本实施例中空中鼠标的处理单元对一帧定位图像进行图像处理时,可以采用实施例一至实施例五中记录的各种图像处理方法。When the processing unit of the air mouse in this embodiment performs image processing on a frame of positioning image, various image processing methods recorded in Embodiment 1 to Embodiment 5 can be used.
本实施例的空中鼠标,因为没有按键单元,体积可以做得比较小。例如用一个边长为5厘米的正方体作为壳体,在其中1个面上开一个孔放置图像采集单元的镜头。Because the air mouse of this embodiment has no button unit, the volume can be made relatively small. For example, a cube with a side length of 5 cm is used as the housing, and a hole is opened on one surface to place the lens of the image capture unit.
本实施例的空中鼠标,特别适用于智能移动设备,所述的智能移动设备包括智能手机及平板电脑。用于智能手机时,玩家手持手机,屏幕朝向玩家。将空中鼠标粘贴于手机的背面,镜头位于和粘贴面相对的一面,即镜头朝向玩家的前方(和手机背部镜头朝向一致)。处理单元通过一根数据线和手机连接。在玩家前方,镜头能够拍摄到的位置放置定位标志(根据需要,可放置1个或多个)。玩家手持手机在空中移动,即可控制屏幕上的光标移动。The air mouse of this embodiment is particularly suitable for smart mobile devices, and the smart mobile devices include smart phones and tablet computers. When using a smart phone, the player holds the phone with the screen facing the player. Paste the air mouse on the back of the phone. The lens is located on the side opposite to the sticking surface, that is, the lens faces the front of the player (the same as the back of the phone) The processing unit is connected to the mobile phone through a data cable. In front of the player, place a positioning mark (one or more can be placed as needed) where the camera can shoot. Players can control the cursor movement on the screen by moving their mobile phone in the air.
实施例七Example Seven
本实施例提供了一种空中鼠标,在实施例一至实施例五中的空中鼠标的基础上,将处理单元分为了图像处理单元和主控单元。本实施例的空中鼠标如图19所示,包括壳体、图像采集单元、图像处理单元、按键单元、主控单元。图像采集单元用于拍摄包括定位标志的景物,并输出有定位标志影像的定位图像;图像处理单元对定位图像进行图像处理,并将图像处理结果发送到主控单元;主控单元根据图像处理结果将光标控制信号向被控设备发送;按键单元产生的按键信号通过主控单元向被控设备发送;图像处理单元对定位图像进行处理时,采用了实施例一至五中处理单元所采用的各种图像处理方法。This embodiment provides an air mouse. Based on the air mouse in the first to fifth embodiments, the processing unit is divided into an image processing unit and a main control unit. As shown in Figure 19, the air mouse of this embodiment includes a housing, an image acquisition unit, an image processing unit, a button unit, and a main control unit. The image acquisition unit is used to photograph the scene including the positioning mark and output the positioning image with the positioning mark image; the image processing unit performs image processing on the positioning image and sends the image processing result to the main control unit; the main control unit according to the image processing result The cursor control signal is sent to the controlled device; the key signal generated by the key unit is sent to the controlled device through the main control unit; when the image processing unit processes the positioning image, the various processing units used in the first to fifth embodiments are used Image processing method.
图像处理单元采用的图像处理方法为实施例一、实施例二或实施三中的任何一种时,每处理一帧定位图像都会获得一组影像数据(包括特征代码和影像位置数据),该组影像数据被发送到主控单元,并被主控单元发送到被控设备。When the image processing method adopted by the image processing unit is any one of Embodiment 1, Embodiment 2 or Implementation 3, a set of image data (including feature codes and image position data) will be obtained every time a frame of positioning image is processed. The image data is sent to the main control unit and sent to the controlled device by the main control unit.
图像处理单元采用的图像处理方法为实施例四中的任何一种时,每处理一帧定位图像,会将获得的影像位置数据或换标信号通过主控单元向被控设备发送。When the image processing method adopted by the image processing unit is any one of the fourth embodiment, every time a frame of positioning image is processed, the obtained image position data or conversion signal will be sent to the controlled device through the main control unit.
图像处理单元采用的图像处理方法为实施例五中的任何一种时,每处理一帧定位图像,会将获得的影像位移数据向主控单元发送。主控单元根据该影像位移数据向被控设备发送位移信号。根据影像位移数据发送位移信号,其规则和实施 例五中的一样。When the image processing method adopted by the image processing unit is any one of the fifth embodiment, each frame of positioning image is processed, the obtained image displacement data is sent to the main control unit. The main control unit sends a displacement signal to the controlled device according to the image displacement data. The rules for sending the displacement signal according to the image displacement data are the same as those in the fifth embodiment.
本实施例的空中鼠标的图像处理单元,可以由一个图像处理芯片来担任。该图像处理芯片可以是通用微处理器(如单片机)或数字信号处理器(DSP)。优选的,该图像处理芯片选用意法半导体的STM32F407芯片,其输入端和图像采集单元电连接,输出端和主控单元电连接。所述图像处理芯片也可以选用其他微处理器,例如单片机或DSP。The image processing unit of the air mouse in this embodiment can be served by an image processing chip. The image processing chip can be a general-purpose microprocessor (such as a single-chip microcomputer) or a digital signal processor (DSP). Preferably, the image processing chip is a STMicroelectronics STM32F407 chip, and the input terminal is electrically connected with the image acquisition unit, and the output terminal is electrically connected with the main control unit. The image processing chip may also use other microprocessors, such as a single-chip microcomputer or DSP.
本实施例中的图像处理芯片,采用了本发明的图像处理方法。使得使用了该图像处理芯片的空中鼠标,可以通过增加定位标志的方式增加移动长度。The image processing chip in this embodiment adopts the image processing method of the present invention. The air mouse using the image processing chip can increase the moving length by adding positioning marks.
实施例八Example eight
本实施例提供了一种云游戏系统。包括空中鼠标、客户端设备及云服务器。空中鼠标包括壳体、图像采集单元、按键单元、处理单元;图像采集单元用于拍摄包括定位标志的景物,并输出有定位标志影像的定位图像;按键单元产生按键信号并通过处理单元向客户端设备发送;处理单元对定位图像进行图像处理,并根据图像处理结果将光标控制信号向客户端设备发送。处理单元对定位图像的处理采用了本发明实施例一至实施例五中的各种图像处理方法。This embodiment provides a cloud game system. Including air mouse, client device and cloud server. The air mouse includes a housing, an image acquisition unit, a key unit, and a processing unit; the image acquisition unit is used to shoot scenes including positioning marks and output positioning images with positioning marks; the key unit generates key signals and sends them to the client through the processing unit Device sending; the processing unit performs image processing on the positioning image, and sends the cursor control signal to the client device according to the image processing result. The processing unit processes the positioning images using various image processing methods in the first to fifth embodiments of the present invention.
客户端设备将光标控制信号和按键信号向云服务器发送。云服务器用于运行游戏程序,根据光标按键信号和按键信号产生屏幕显示数据发送到客户端设备,客户端设备根据屏幕显示数据控制所驱动的显示设备屏幕上的显示内容作相应改变。The client device sends the cursor control signal and the key signal to the cloud server. The cloud server is used to run the game program, generate screen display data according to the cursor key signal and the key signal, and send it to the client device. The client device controls the display content on the screen of the driven display device according to the screen display data to make corresponding changes.
采用不同的图像处理方法,空中鼠标输出的光标控制信号也不同。收到的光标控制信号为位移信号时,云服务器根据位移信号产生屏幕显示数据。收到的光标控制信号为特征代码和影像位置数据时,云服务器设有实施例一中所述的驱动程序,通过驱动程序得到影像位移数据,根据该位移数据产生屏幕显示数据。收到的光标控制信号为影像位置数据和换标信号时,云服务器设有实施例四中所述的驱动程序,通过驱动程序得到影像位移数据,根据该影像位移数据产生屏幕显示数据。Using different image processing methods, the cursor control signals output by the air mouse are also different. When the received cursor control signal is a displacement signal, the cloud server generates screen display data according to the displacement signal. When the received cursor control signal is a feature code and image position data, the cloud server is provided with the driver program described in the first embodiment, the image displacement data is obtained through the driver program, and the screen display data is generated according to the displacement data. When the received cursor control signal is the image position data and the conversion signal, the cloud server is provided with the driver program described in the fourth embodiment, the image displacement data is obtained through the driver program, and the screen display data is generated according to the image displacement data.
下面举例说明,客户端设备为一台PC,该PC驱动一台液晶显示器。云服务器运行一个射击游戏,空中鼠标控制屏幕上的准星。空中鼠标向该PC发出一个位移信号(包含的信息为向右移动10个像素),PC将该信号发送到云服务器,云服务器通过运算产生了一个视频数据包(该视频数据包包括了在屏幕上显示1 帧画面需要的全部数据)。该视频数据包发送到PC,PC根该视频数据包中的数据驱动液晶显示器,液晶显示器上显示出准星向右移动10个像素后的画面。空中鼠标连续发出位移信号,则屏幕上的画面也产生连续的变化。The following example illustrates that the client device is a PC that drives a liquid crystal display. The cloud server runs a shooting game, and the air mouse controls the front sight on the screen. The air mouse sends a displacement signal to the PC (the information contained is 10 pixels to the right), the PC sends the signal to the cloud server, and the cloud server generates a video data packet through calculation (the video data packet includes the screen All data required for 1 frame of screen is displayed on the screen). The video data packet is sent to the PC, and the PC drives the liquid crystal display based on the data in the video data packet, and the liquid crystal display displays the picture after the crosshair moves 10 pixels to the right. The air mouse continuously sends out displacement signals, and the picture on the screen also changes continuously.
空中鼠标向PC发出一个按键信号,该按键信号包含的指令是换弹夹。该按键信号通过PC发送到了云服务器。云服务器通过运算产生了50个的视频数据包并向PC连续发生,每个视频数据包都包括了显示一帧画面需要的数据,PC每接到一个视频数据包,都驱动液晶显示器显示一帧画面,50帧画面连起来就是一个游戏中的人物换弹夹的连续动作。The air mouse sends a button signal to the PC, and the button signal contains the command to change the magazine. The key signal is sent to the cloud server through the PC. The cloud server generates 50 video data packets through calculations and sends them to the PC continuously. Each video data packet includes the data needed to display a frame of picture. Every time the PC receives a video data packet, it drives the LCD to display one frame. The picture, 50 frames of pictures connected together is the continuous action of a character in the game changing magazines.
本发明的空中鼠标发出的位移信号可以是如下格式:用1、2、3、4分别代表上、下、左、右4个移动方向,后面加上数字表示移动的像素数。云服务器根据收到的位移信号输出视频数据包,控制屏幕上的显示内容。例如,收到2+5,云服务器就控制PC所驱动的屏幕显示光标向下移动5个像素后的画面;收到3+2,就控制屏幕显示光标向左移动2个像素后的画面。The displacement signal sent by the air mouse of the present invention can be in the following format: 1, 2, 3, and 4 are used to represent the four moving directions of up, down, left, and right, and a number is added to indicate the number of pixels moved. The cloud server outputs video data packets according to the received displacement signal, and controls the display content on the screen. For example, when receiving 2+5, the cloud server controls the screen driven by the PC to display the screen after the cursor moves down 5 pixels; when receiving 3+2, it controls the screen to display the screen after moving the cursor 2 pixels to the left.
云服务器根据影像位移数据控制屏幕显示内容,采用如下规则:行坐标增加、行坐标减小、列坐标增加、列坐标减小四种不同情况时,分别控制屏幕上的光标向上、下、左、右4个方向移动,移动的数值等于坐标变化的数值。例如,影像位移数据为行坐标增加了5,云服务器则控制PC所驱动的屏幕显示光标向上移动了5个像素后的画面。The cloud server controls the screen display content according to the image displacement data, and adopts the following rules: when row coordinates increase, row coordinates decrease, column coordinates increase, and column coordinates decrease, the cursor on the screen is controlled up, down, left, and Moving in 4 directions to the right, the value of the movement is equal to the value of the coordinate change. For example, the image displacement data adds 5 to the row coordinate, and the cloud server controls the screen driven by the PC to display the screen after the cursor has moved up by 5 pixels.
本实施例的云游戏系统的被控设备为各种类型的电子计算机设备、包括但不限于PC、电视游戏机、智能电视、平板电脑、智能手机。The controlled devices of the cloud game system in this embodiment are various types of electronic computer devices, including but not limited to PCs, video game consoles, smart TVs, tablet computers, and smart phones.
本实施例中,驱动程序和云服务器组成了计算机系统。In this embodiment, the driver and the cloud server form a computer system.
本实施例的云游戏系统,可以通过增加定位标志的方式增加空中鼠标的移动长度,使得空中鼠标能够同时具备较高的移动长度和移动精度,增加了游戏体验。In the cloud game system of this embodiment, the moving length of the air mouse can be increased by adding a positioning mark, so that the air mouse can have a higher moving length and moving accuracy at the same time, thereby increasing the gaming experience.
实施例九Example 9
本实施例提供了一种空中鼠标,在实施例五的空中鼠标的基础上改进而来。区别在于处理单元对每帧定位图像处理完后,向被控设备发送位移信号的方法。This embodiment provides an air mouse, which is improved on the basis of the air mouse in the fifth embodiment. The difference lies in the method that the processing unit sends a displacement signal to the controlled device after processing each frame of the positioning image.
本实施例中的空中鼠标,在对每帧定位图像处理后,如果产生了影像位移数据,处理单元根据该影像位移数据向被控设备发送位移信号;如果没有产生影像位移数据,处理单元向被控设备发送前一帧定位图像处理后产生的位移信号(如果前一帧定位图像处理后产生过位移信号)。In the air mouse in this embodiment, after processing each frame of positioning image, if image displacement data is generated, the processing unit sends a displacement signal to the controlled device according to the image displacement data; if no image displacement data is generated, the processing unit The control device sends the displacement signal generated after the positioning image processing of the previous frame (if the displacement signal is generated after the positioning image processing of the previous frame).
下面举例说明本实施例的空中鼠标。The following example illustrates the air mouse of this embodiment.
使用者手持空中鼠标向右移动,空中鼠标向右移动过程中输出第n帧定位图像,图像处理完成后,处理单元向被控设备发出的位移信号为X+10(在屏幕上向右移动10个像素)。空中鼠标继续向右移动后输出第n+1帧定位图像,处理单元从此帧定位图像中获取的特征代码发生了改变,则此帧定位图像处理完成后没有影像位移数据产生。此时处理单元向被控单元发出位移信号X+10(和第n帧定位图像处理后输出的位移信号相同)。The user holds the air mouse and moves it to the right. The nth frame of positioning image is output during the process of the air mouse moving to the right. After the image processing is completed, the displacement signal sent by the processing unit to the controlled device is X+10 (moving to the right by 10 on the screen). Pixels). The air mouse continues to move to the right and outputs the n+1th frame positioning image. The feature code obtained by the processing unit from the positioning image of this frame changes, and no image displacement data is generated after the positioning image processing of this frame is completed. At this time, the processing unit sends a displacement signal X+10 to the controlled unit (the same as the displacement signal output after processing the n-th frame positioning image).
实施例五中的空中鼠标在移动过程中,如果位移参照物(定位标志)发生了改变,在改变后获取的第一帧定位图像中,定位标志影像的特征代码和前一帧定位图像中的是不一样的。不会有影像位移数据产生,也就不会有位移信号产生。即这两帧定位图像输出之间的这个时间段,空中鼠标的移动并不能反映在被控设备的显示屏幕上,影响光标控制的准确性。在空中鼠标的移动过程中,如果把两帧定位图像输出之间的时间作为一个时间段,相邻两个时间段内空中鼠标的移动方向和速度在绝大多数情况下是很接近的。所以本实施例的空中鼠标在没有位移数据产生时,向被控设备发出前一帧定位图像处理后产生的位移信号,在移动过程中,当定位参照物发生切换时,空中鼠标移动的方向和距离能正确的反映到显示屏幕上,增加了控制的准确性。During the movement of the air mouse in the fifth embodiment, if the displacement reference object (positioning mark) is changed, in the first frame of positioning image obtained after the change, the feature code of the positioning mark image is the same as the one in the previous frame of positioning image. are different. No image displacement data is generated, and no displacement signal is generated. That is, in this time period between the output of the two frames of positioning images, the movement of the air mouse cannot be reflected on the display screen of the controlled device, which affects the accuracy of cursor control. During the movement of the air mouse, if the time between the output of two positioning images is regarded as a time period, the moving direction and speed of the air mouse in the two adjacent time periods are very close in most cases. Therefore, the air mouse of this embodiment sends the displacement signal generated after the positioning image processing of the previous frame to the controlled device when there is no displacement data generated. During the movement, when the positioning reference object is switched, the direction of the air mouse movement and The distance can be correctly reflected on the display screen, which increases the accuracy of control.
实施例十Example ten
本实施例提供了一种驱动程序,在实施例一中的驱动程序的基础上改进而来,本实施例中的驱动程序在接收到每一组影像数据时采用的处理方法为:This embodiment provides a driver program, which is improved on the basis of the driver program in Embodiment 1. The processing method adopted by the driver program in this embodiment when each group of image data is received is:
根据其中的特征代码,在收到的上一组影像数据中查找相同的。如果找到相同的,则将此两组影像数据中,该特征代码所对应的影像位置数据进行比对后获得影像位移数据,并根据影像位移数据控制屏幕上光标的移动。如果没有找到相同的,则根据收到的上一组影像数据处理后获得的影像位移数据控制屏幕上光标的移动(如果该数据是存在的)。According to the feature code, find the same in the last group of image data received. If the same is found, the image position data corresponding to the feature code in the two sets of image data are compared to obtain image displacement data, and the movement of the cursor on the screen is controlled according to the image displacement data. If the same is not found, the movement of the cursor on the screen is controlled according to the image displacement data obtained after processing the last set of image data received (if the data exists).
例如,该驱动程序安装在PC端,使用者手持空中鼠标横向移动。驱动程序收到的第n组影像数据中包括特征代码1及一个影像位置数据。特征代码和第n-1组影像数据中的相同,将两个影像位置数据比对后,得到的影像位移为列坐标减小了10,则控制屏幕上的光标向右移动10个光标;接收到的第n+1组位移数据包括特征代码2及一个影像位置数据。此时仍控制屏幕上的光标向右移动 10个光标。然后开始处理第n+2组影像数据。For example, the driver is installed on the PC, and the user holds the air mouse to move laterally. The nth group of image data received by the driver includes feature code 1 and an image position data. The feature code is the same as that in the n-1th group of image data. After comparing the two image position data, the obtained image displacement is that the column coordinate is reduced by 10, then the cursor on the control screen moves 10 cursors to the right; The obtained (n+1)th set of displacement data includes feature code 2 and an image position data. At this time, the cursor on the screen is still controlled to move 10 cursors to the right. Then start processing the n+2 group of image data.
本实施例的驱动程序的有益效果在于,可以使配套的空中鼠标在位移参照物(定位标志)切换时的移动也能反映在显示屏幕上,增加了控制的准确性。而实施例一中的驱动程序,在空中鼠标移动过程中位移参照物发生变化时,会有一段距离的移动不能反映到屏幕上。The beneficial effect of the driver program of this embodiment is that the movement of the matching air mouse when the displacement reference object (positioning mark) is switched can also be reflected on the display screen, which increases the accuracy of control. In the driving program in the first embodiment, when the displacement reference object changes during the movement of the air mouse, there will be a certain distance of movement that cannot be reflected on the screen.
实施例十一Example 11
本实施例提供了一种驱动程序,在实施例四中提供的驱动程序的基础上改进而来。This embodiment provides a driver program, which is improved on the basis of the driver program provided in the fourth embodiment.
实施例四中的空中鼠标,每处理完一帧定位图像,都会向被控设备发送一组数据,该组数据为影像位置数据或换标信号,被控设备上的驱动程序根据该组数据控制屏幕上光标的移动。The air mouse in the fourth embodiment will send a set of data to the controlled device after processing a frame of positioning image. This set of data is image position data or a rebranding signal, and the driver on the controlled device controls according to the set of data The movement of the cursor on the screen.
本实施例的驱动程序,会设置一个记录区用于记录影像位置数据。在接收到空中鼠标发送的一组数据时,用如下方法处理:The driver of this embodiment will set a recording area for recording image position data. When receiving a group of data sent by the air mouse, use the following method to process:
第一步,驱动程序接收到影像位置数据时,记录区中如果有影像位置数据,则将接收到的影像位置数据同记录区中的进行比对,并根据得到的影像位移数据控制屏幕上的光标移动,并将记录区中的数据更新为刚接到的影像位置数据;记录区中如果没有影像位置数据,则将刚接到的影像位置数据存入记录区。驱动程序接收到换标信号时,将记录区清空。In the first step, when the driver receives the image position data, if there is image position data in the recording area, compare the received image position data with that in the recording area, and control the screen based on the obtained image displacement data. The cursor moves, and the data in the recording area is updated to the image position data just received; if there is no image position data in the recording area, the image position data just received is stored in the recording area. When the driver receives the rebranding signal, it will clear the recording area.
第二步,如果第一步产生了影像位移数据,则本组数据处理过程结束。如果第一步没有产生影像位移数据,则根据接收到的上一组数据处理结束后屏幕上光标的移动结果,控制屏幕上的光标作相同方向和距离的移动。In the second step, if the image displacement data is generated in the first step, the processing of this group of data ends. If the image displacement data is not generated in the first step, the cursor on the screen is controlled to move in the same direction and distance according to the result of the movement of the cursor on the screen after the processing of the last set of data received.
例如,使用者手持空中鼠标横向移动,每处理一帧定位图像就向被控设备发送一组数据。被控设备上的驱动程接到第n组数据时,该组数据为影像位置数据。驱动程序处理完成后控制屏幕上的光标向右移动了5个像素。驱动程序接到第n+1组数据时,该组数据为换标信号,第一步处理完成后没有影像位移数据,则在第二步处理时,根据上一组(第n组)数据处理完成后的结果,控制屏幕上的光标向右移动5个像素。驱动程序接到第n+2组数据,该组数据为影像位置数据,第一步处理完成没有影像位移数据产生,则在第二步时,根据上一组(第n+1组)数据处理完成后的结果,控制屏幕上的光标向右移动5个像素。驱动程序接到第n+3组数据时,该组数据为影像位置数据,经第一步处理完成后产生 了影像位移数据,驱动程序根据此数据控制屏幕上的光标作相应的移动。For example, the user holds the air mouse to move laterally, and sends a set of data to the controlled device every time a frame of positioning image is processed. When the driver on the controlled device receives the nth group of data, this group of data is the image position data. After the driver has finished processing, the cursor on the control screen has moved 5 pixels to the right. When the driver receives the n+1th group of data, this group of data is a rebranding signal. After the first step is processed, there is no image displacement data, then in the second step of processing, it is processed according to the previous group (nth group) data After finishing the result, control the cursor on the screen to move 5 pixels to the right. The driver receives the n+2 group of data, which is the image position data. The first step is processed and no image displacement data is generated, then in the second step, it is processed according to the previous group (n+1 group) data After finishing the result, control the cursor on the screen to move 5 pixels to the right. When the driver receives the n+3 group of data, this group of data is image position data. After the first step of processing is completed, the image displacement data is generated. The driver controls the cursor on the screen to move correspondingly according to this data.
根据实施例四中的驱动程序的处理方法,在空中鼠标移动过程中参照物(定位标志)发生变化时,空中鼠标有一段移动距离(输出2帧定位图像的时间内移动过的距离)是没有反映在显示屏幕上的,影响到光标控制的准确性。本实施例的驱动程序,完全弥补了这个缺点,提高了光标控制时的准确性。According to the processing method of the driver in the fourth embodiment, when the reference object (location mark) changes during the movement of the air mouse, the air mouse has a certain moving distance (the distance moved within the time of outputting 2 frames of positioning images). Reflected on the display screen, affecting the accuracy of cursor control. The driver program of this embodiment completely compensates for this shortcoming and improves the accuracy of cursor control.
实施例十二Embodiment 12
本实施例提供了一种用于计算机的图像处理方法。用于在计算机端对定位图像进行图像处理,计算机根据图像处理结果控制所驱动的显示设备屏幕上光标的移动。This embodiment provides an image processing method for a computer. It is used for image processing of the positioning image on the computer side, and the computer controls the movement of the cursor on the screen of the driven display device according to the image processing result.
本实施例的一种实施方式中,所述计算机为一台PC,驱动一台液晶显示器。该PC和一空中鼠标连接,该空中鼠标的构成和实施例一中记录的空中鼠标一样,但该空中鼠标的处理单元在接收到图像采集单元输出的定位图像后,将定位图像发送到连接的PC。PC上安装有图像处理软件,该图像处理软件对接收到的定位图像进行图像处理,图像处理方法和实施例五中空中鼠标所采用的图像处理方法一样。图像处理软件每处理完一帧定位图像,如果获得了影像位移数据,则根据此数据控制所驱动的液晶显示器屏幕上的光标进行相应的移动。根据影像位移数据控制屏幕上的光标移动,所采用的规则和实施例一中记录的一样。In an implementation of this embodiment, the computer is a PC, which drives a liquid crystal display. The PC is connected to an air mouse. The composition of the air mouse is the same as the air mouse recorded in the first embodiment. However, the processing unit of the air mouse sends the positioning image to the connected device after receiving the positioning image output by the image acquisition unit. PC. An image processing software is installed on the PC, and the image processing software performs image processing on the received positioning image. The image processing method is the same as the image processing method adopted by the air mouse in the fifth embodiment. After the image processing software processes a frame of positioning image, if the image displacement data is obtained, the cursor on the LCD screen driven by this data is controlled to move accordingly. To control the cursor movement on the screen according to the image displacement data, the rules adopted are the same as those recorded in the first embodiment.
使用者手持的空中鼠标连续向所连接的PC发送定位图像,本实施例中的计算机采用的图像处理软件,根据前后两帧定位图像中具有相同影像特征的定位标志影像的位移获得影像位移数据,并根据影像位移数据控制屏幕上的光标进行相应的移动。The air mouse held by the user continuously sends positioning images to the connected PC. The image processing software used by the computer in this embodiment obtains the image displacement data according to the displacement of the positioning mark images with the same image characteristics in the two frames of the positioning images. And according to the image displacement data, the cursor on the screen is controlled to move accordingly.
在本实施例的图像处理方法的一种实施方式中,图像处理软件对每帧定位图像处理后,如果产生了影像位移数据,处理单元根据该影像位移数据控制屏幕上的光标进行相应移动;如果没有产生影像位移数据,则根据前一帧定位图像处理后光标的移动结果,控制屏幕上的光标进行相同方向和距离的移动。In an implementation of the image processing method of this embodiment, after the image processing software processes the positioning image for each frame, if image displacement data is generated, the processing unit controls the cursor on the screen to move accordingly according to the image displacement data; if If no image displacement data is generated, the cursor on the screen is controlled to move in the same direction and distance according to the result of the cursor movement after the positioning image processing of the previous frame.
下面举例说明该实施方式。The following example illustrates this embodiment.
使用者手持空中鼠标向右移动,空中鼠标移动过程中输出第n帧定位图像,PC端的图像处理软件处理完成后,控制屏幕上的光标向右移动10个像素。空中鼠标继续向右移动后输出第n+1帧定位图像,图像处理软件从此帧定位图像中获取的特征代码发生了改变(此时位移参照物变了),则此帧定位图像处理完成 后没有影像位移数据产生。此时图像处理软件控制屏幕上的光标向右移动10个像素(和第n帧定位图像处理后光标移动的方向和距离相同)。空中鼠标在移动过程中定位参照物发生变化时,本实施方式能够使空中鼠标移动的方向和距离正确的反映到显示屏幕上,增加光标控制的准确性。The user holds the air mouse and moves it to the right. During the movement of the air mouse, the n-th frame positioning image is output. After the image processing software on the PC is finished, the cursor on the screen is controlled to move 10 pixels to the right. After the air mouse continues to move to the right, it outputs the n+1th frame positioning image. The feature code obtained by the image processing software from the positioning image of this frame has changed (the displacement reference object has changed at this time). Image displacement data is generated. At this time, the image processing software controls the cursor on the screen to move 10 pixels to the right (the direction and distance of the cursor movement after the positioning image processing of the nth frame is the same). When the positioning reference object of the air mouse changes during the movement, this embodiment can accurately reflect the moving direction and distance of the air mouse on the display screen, thereby increasing the accuracy of cursor control.
本实施例中的计算机为各种电子计算机设备,包括但不限于PC、视频游戏机、智能电视、云游戏服务器。The computers in this embodiment are various electronic computer equipment, including but not limited to PCs, video game consoles, smart TVs, and cloud game servers.
本实施例中,计算机及所采用的图像处理软件组成了计算机系统,该系统能够根据定位图像来控制所驱动的屏幕上的光标的移动。In this embodiment, the computer and the image processing software used constitute a computer system, which can control the movement of the cursor on the driven screen according to the positioning image.
在本实施例的计算机系统的一种实施方式中,计算机为PC。In an implementation of the computer system of this embodiment, the computer is a PC.
在本实施例的计算机系统的一种实施方式中,计算机为视频游戏机,所安装的图像处理软件对接收到的定位图像进行图像处理,并根据处理结果控制所驱动的屏幕上的光标进行移动。In an implementation of the computer system of this embodiment, the computer is a video game machine, and the installed image processing software performs image processing on the received positioning image, and controls the cursor on the driven screen to move according to the processing result .
在本实施例的计算机系统的一种实施方式中,计算机为智能电视,所安装的图像处理软件对接收到的定位图像进行图像处理,并根据处理结果控制电视屏幕上的光标进行移动。In an implementation of the computer system of this embodiment, the computer is a smart TV, and the installed image processing software performs image processing on the received positioning image, and controls the cursor on the TV screen to move according to the processing result.
在本实施例的计算机系统的一种实施方式中,计算机为云游戏服务器。定位图像发送到服务器端,服务器端的图像处理软件进行图像处理后,根据处理结果控制所驱动的屏幕上的光标移动。In an implementation of the computer system of this embodiment, the computer is a cloud game server. The positioning image is sent to the server, and the image processing software on the server performs image processing, and then controls the cursor movement on the driven screen according to the processing result.
本实施例中的图像处理方法,能够使采用了该图像处理方法的计算机能够通过增加定位标志的方式增加移动长度。The image processing method in this embodiment can enable a computer using the image processing method to increase the moving length by adding positioning marks.
本实施例中的计算机系统,因为所采用的图像处理软件的作用,可以通过增加定位标志的方式增加所驱动的屏幕上光标的移动长度。The computer system in this embodiment, because of the image processing software used, can increase the moving length of the cursor on the driven screen by adding positioning marks.
以上所述仅为本发明的较佳实施例,并不用以限制本发明,凡在本发明的精神和原则之内,所作的任何修改、等同替换、改进等,均在本发明的包含范围之内。The foregoing descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are all within the scope of the present invention. Inside.
本领域的技术人员可以理解的是,以上实施例仅为举例,其中不同实施例的特征可以相互组合,以得到根据本发明揭露的内容很容易想到但是在附图中没有明确指出的实施方式。Those skilled in the art can understand that the above embodiments are only examples, in which the features of different embodiments can be combined with each other to obtain implementations that are easily conceivable according to the disclosure of the present invention but are not clearly indicated in the drawings.
本领域的技术人员应理解,上述描述及附图中所示的本发明的实施例只作为举例而并不限制本发明。本发明的目的已经完整并有效地实现。本发明的功能及 结构原理已在实施例中展示和说明,在没有背离所述原理下,本发明的实施方式可以有任何变形或修改。Those skilled in the art should understand that the above description and the embodiments of the present invention shown in the accompanying drawings are only examples and do not limit the present invention. The purpose of the present invention has been completely and effectively achieved. The functions and structural principles of the present invention have been shown and explained in the embodiments. Without departing from the principles, the embodiments of the present invention may have any deformation or modification.
Claims (53)
- 一种图像处理方法,用于空中鼠标,其特征在于,对定位图像进行图像处理时,获取其中定位标志影像的特征代码及影像位置数据;其中特征代码根据定位标志影像的影像特征确定。An image processing method for an air mouse is characterized in that when image processing is performed on a positioning image, the feature code and image position data of the positioning mark image are obtained; wherein the feature code is determined according to the image characteristics of the positioning mark image.
- 如权利要求1所述的图像处理方法,其中从定位图像中获取到特征代码及影像位置数据后,将特征代码同上一帧定位图像中获取的特征代码进行比对。5. The image processing method according to claim 1, wherein after the feature code and image location data are obtained from the positioning image, the feature code is compared with the feature code obtained in the previous frame of the positioning image.
- 如权利要求1所述的图像处理方法,其中从定位图像中获取到特征代码及影像位置数据后,同上一帧定位图像中获取的特征代码及影像位置数据进行比对,并进一步获取到影像位移数据。The image processing method of claim 1, wherein after the feature code and image position data are obtained from the positioning image, the feature code and image position data obtained in the previous frame of the positioning image are compared, and the image displacement is further obtained data.
- 如权利要求1所述的图像处理方法,其中在对定位图像进行处理时,获取其中一个定位标志影像的特征代码及影像位置数据。8. The image processing method of claim 1, wherein when processing the positioning image, the feature code and image position data of one of the positioning mark images are obtained.
- 如权利要求1所述的图像处理方法,其中在对定位图像进行处理时,获取其中全部定位标志影像的特征代码及影像位置数据。5. The image processing method of claim 1, wherein when the positioning image is processed, feature codes and image position data of all the positioning mark images are acquired.
- 一种空中鼠标,其特征在于,包括壳体、图像采集单元、处理单元,其中所述图像采集单元用于拍摄包括定位标志的景物,并输出有定位标志影像的定位图像;所述处理单元对所述定位图像进行图像处理,并根据图像处理结果将光标控制信号向一被控设备发送;对定位图像进行处理时,包括获取其中定位标志影像的特征代码及影像位置数据,其中特征代码根据定位标志影像的特征来确定。An air mouse, which is characterized by comprising a housing, an image acquisition unit, and a processing unit, wherein the image acquisition unit is used to photograph a scene including a positioning mark and output a positioning image with a positioning mark image; the processing unit is The positioning image is image processed, and the cursor control signal is sent to a controlled device according to the image processing result; when the positioning image is processed, it includes acquiring the feature code of the positioning mark image and the image position data, wherein the feature code is based on the positioning To determine the characteristics of the logo image.
- 如权利要求6所述的空中鼠标,其中处理单元将特征代码和影像位置数据向所述被控设备发送。8. The air mouse according to claim 6, wherein the processing unit sends the feature code and image position data to the controlled device.
- 如权利要求6所述的空中鼠标,其中处理单元从一帧定位图像中获取到特征代码及影像位置数据后,在前一帧定位图像所获取的特征代码中寻找是否有相同的特征代码;如果有相同的,则将该特征代码所对应的影像位置数据向被控设备发送;如果没有相同的,则将换标信号向所述被控设备发送。7. The air mouse of claim 6, wherein after the processing unit obtains the feature code and image position data from a frame of positioning image, it searches for the feature code obtained from the previous frame of positioning image for the same feature code; if If there is the same, the image location data corresponding to the feature code is sent to the controlled device; if there is no the same, the rebranding signal is sent to the controlled device.
- 如权利要求6所述的空中鼠标,其中处理单元根据前后两帧定位图像中具有相同影像特征的定位标志影像的位移获得影像位移数据,并根据影像位移数据向被控设备发送位移信号。7. The air mouse according to claim 6, wherein the processing unit obtains image displacement data according to the displacement of the positioning mark images with the same image characteristics in the two previous and subsequent frames of positioning images, and sends a displacement signal to the controlled device according to the image displacement data.
- 如权利要求6所述的空中鼠标,其中处理单元从一帧定位图像中获取到特征代码及影像位置数据后,在前一帧定位图像所获取的特征代码中寻找是否有相同的特征代码;如果有相同的,则将该特征代码在两帧定位图像中对应的影像位置数据进行比对,根据获得的影像位移数据向被控设备发送位移信号;如果没有相同的,则将前一帧定位图像处理后产生的位移信号向被控设备发送。7. The air mouse of claim 6, wherein after the processing unit obtains the feature code and image position data from a frame of positioning image, it searches for the feature code obtained from the previous frame of positioning image for the same feature code; if If there are the same, compare the corresponding image position data of the feature code in the two frames of positioning images, and send the displacement signal to the controlled device according to the obtained image displacement data; if there is no the same, the previous frame of positioning image The displacement signal generated after processing is sent to the controlled device.
- 如权利要求6所述的空中鼠标,还包括按键单元,所述按键单元用于输入按键信号,所述按键信号通过所述处理单元向所述被控设备发送。8. The air mouse according to claim 6, further comprising a button unit for inputting a button signal, and the button signal is sent to the controlled device through the processing unit.
- 一种空中鼠标,其特征在于,包括壳体、图像采集单元、图像处理单元、按键单元、主控单元,其中图像采集单元用于拍摄包括定位标志的景物,并输出有定位标志影像的定位图像,其中图像处理单元对定位图像进行图像处理,并将图像处理结果发送到主控单元,其中主控单元根据图像处理结果将光标控制信号向被控设备发送,其中按键单元产生的按键信号通过主控单元向被控设备发送,其中所述图像处理单元对定位图像进行处理时,包括获取其中定位标志影像的特征代码及影像位置数据,其中特征代码根据定位标志影像的特征来确定。An air mouse, which is characterized by comprising a housing, an image acquisition unit, an image processing unit, a key unit, and a main control unit, wherein the image acquisition unit is used to photograph a scene including a positioning mark and output a positioning image with the positioning mark image , Wherein the image processing unit performs image processing on the positioning image and sends the image processing result to the main control unit. The main control unit sends the cursor control signal to the controlled device according to the image processing result, and the key signal generated by the key unit passes the main control unit. The control unit sends to the controlled device, wherein when the image processing unit processes the positioning image, it includes acquiring the feature code of the positioning mark image and the image position data, wherein the feature code is determined according to the feature of the positioning mark image.
- 一种空中鼠标系统,其特征在于,包括空中鼠标和至少两个具有不同特征的定位标志,其中所述空中鼠标包括壳体、图像采集单元、按键单元、处理单元,其中所述图像采集单元用于拍摄包括定位标志的景物并输出有定位标志影像的定位图像,其中所述按键单元输入的按键信号通过所述处理单元向被控设备发送,其中所述处理单元对所述定位图像进行图像处理,并根据图像处理结果将光标控制信号向被控设备发送,所述具有不同特征的定位标志,在定位图像中形成的定位标志影像有不同的影像特征。An air mouse system, which is characterized by comprising an air mouse and at least two positioning marks with different characteristics, wherein the air mouse includes a housing, an image acquisition unit, a button unit, and a processing unit, wherein the image acquisition unit is used for For shooting a scene including a positioning mark and outputting a positioning image with a positioning mark image, wherein the key signal input by the key unit is sent to the controlled device through the processing unit, and the processing unit performs image processing on the positioning image , And send the cursor control signal to the controlled device according to the image processing result, the positioning mark with different characteristics, the positioning mark image formed in the positioning image has different image characteristics.
- 如权利要求13所述的空中鼠标系统,其中所述具有不同的特征的定位标志在定位图像中形成的定位标志影像,其组成像素有不同的像素值。The air mouse system according to claim 13, wherein the positioning mark images formed by the positioning marks with different characteristics in the positioning image have different pixel values.
- 如权利要求13所述的空中鼠标系统,其中所述具有不同的特征的定位标志在定位图像中形成的定位标志影像,由不同数量的像素组成。The air mouse system according to claim 13, wherein the positioning mark images formed by the positioning marks with different characteristics in the positioning image are composed of different numbers of pixels.
- 如权利要求13所述的空中鼠标系统,其中所述具有不同特征的定位标志在定位图像中形成的定位标志影像,其组成像素的排列方式不同。The air mouse system according to claim 13, wherein the positioning mark images formed by the positioning marks with different characteristics in the positioning image have different arrangement of the constituent pixels.
- 一种空中鼠标控制系统,其特征在于,包括一空中鼠标和一被控设备,所述空中鼠标包括壳体、图像采集单元、处理单元,其中所述图像采集单元用于拍摄包括定位标志的景物,并输出有定位标志影像的定位图像,其中所述处理单元对所述定位图像进行图像处理,并根据图像处理结果将光标控制信号向所述被控设备发送,其中对定位图像进行处理时,包括获取其中定位标志影像的特征代码及影像位置数据,其中特征代码根据定位标志影像的特征来确定,其中所述被控设备根据光标控制信号控制所驱动的屏幕上的光标移动。An air mouse control system, which is characterized by comprising an air mouse and a controlled device, the air mouse including a housing, an image acquisition unit, and a processing unit, wherein the image acquisition unit is used to photograph a scene including a positioning mark , And output a positioning image with a positioning mark image, wherein the processing unit performs image processing on the positioning image, and sends a cursor control signal to the controlled device according to the image processing result. When the positioning image is processed, It includes acquiring the feature code of the positioning mark image and the image position data, wherein the feature code is determined according to the feature of the positioning mark image, wherein the controlled device controls the movement of the cursor on the driven screen according to the cursor control signal.
- 如权利要求17所述的空中鼠标控制系统,其中所述空中鼠标向所述被控设备发送的光标控制信号为影像数据,其中所述影像数据包括特征代码和影像位置数据,其中所述被控设备端的驱动程序接收到一组影像数据时,根据该组和前一组影像数据中相同的特征代码所对应的影像位置数据的变化控制所驱动的屏幕上光标的移动。The air mouse control system of claim 17, wherein the cursor control signal sent by the air mouse to the controlled device is image data, wherein the image data includes feature code and image position data, wherein the controlled device When the device driver receives a group of image data, it controls the movement of the cursor on the driven screen according to the change of the image position data corresponding to the same feature code in the group and the previous group of image data.
- 如权利要求17所述的空中鼠标控制系统,其中所述空中鼠标向所述被控设备发送的光标控制信号为影像位置数据及换标信号,其中所述被控设备端的驱动程序通过对影像位置数据及换标信号进行处理获得影像位移数据后控制所驱动的屏幕上的光标移动。The air mouse control system according to claim 17, wherein the cursor control signal sent by the air mouse to the controlled device is image position data and a rebranding signal, and the driver program of the controlled device determines the position of the image by The data and the conversion signal are processed to obtain the image displacement data and then control the cursor movement on the driven screen.
- 如权利要求17所述的空中鼠标控制系统,其中所述空中鼠标向所述被控设备发送的光标控制信号为位移信号,被控设备根据位移信号控制所驱动的显示设备屏幕上的光标移动。The air mouse control system according to claim 17, wherein the cursor control signal sent by the air mouse to the controlled device is a displacement signal, and the controlled device controls the movement of the cursor on the screen of the driven display device according to the displacement signal.
- 一种红外感应条,用于为空中鼠标提供定位标志,其特征在于,包括一支架和至少两个安装在支架上的红外光源,所述两个红外光源有不同的特征。An infrared sensor strip for providing positioning marks for an air mouse is characterized in that it comprises a bracket and at least two infrared light sources installed on the bracket, and the two infrared light sources have different characteristics.
- 如权利要求21所述的红外感应条,其中所述两个红外光源的亮度不同。The infrared sensor strip according to claim 21, wherein the brightness of the two infrared light sources is different.
- 如权利要求21所述的红外感应条,其中所述两个红外光源的发光面积不同。The infrared sensor strip of claim 21, wherein the light-emitting areas of the two infrared light sources are different.
- 如权利要求21所述的红外感应条,其中所述两个红外光源的形状不同。The infrared sensor strip of claim 21, wherein the two infrared light sources have different shapes.
- 如权利要求21所述的红外感应条,其中所述支架为中空结构。The infrared sensor strip according to claim 21, wherein the bracket is a hollow structure.
- 一种用于计算机的图像处理方法,用于处理包括定位标志影像的定位图像,其特征在于,包括如下步骤:An image processing method used in a computer for processing a positioning image including a positioning mark image, characterized in that it includes the following steps:(a)获取其中定位标志影像的特征代码及影像位置数据,其中特征代码根据定位标志影像的影像特征确定;和(a) Obtain the feature code and image location data of the location mark image, where the feature code is determined based on the image feature of the location mark image; and(b)将获得的特征代码和上一帧定位图像中获得的特征代码相比较,如果相同,则将特征代码所对应的影像位置数据进行比对。(b) Compare the obtained feature code with the feature code obtained in the previous frame of the positioning image, and if they are the same, compare the image position data corresponding to the feature code.
- 一种计算机控制所驱动的屏幕上光标移动的方法,其特征在于,包括如下步骤:A method for controlling the movement of a cursor on a screen driven by a computer is characterized in that it comprises the following steps:(a)间隔的接收两帧定位图像,所述两帧定位图像中有定位标志影像;和(a) Receiving two frames of positioning images at intervals, in which there are positioning mark images; and(b)判断两帧所述定位图像中的所述定位标志影像是否有相同的特征,如果有,则将两帧所述定位图像中所述定位标志影像的位置进行比较,如果产生了影像位移数据,则根据该影像位移数据控制所驱动的屏幕上光标进行移动。(b) Determine whether the positioning mark images in the two frames of the positioning image have the same characteristics, if so, compare the positions of the positioning mark images in the two frames of the positioning image, and if an image displacement occurs Data, the cursor on the screen driven by the image displacement data is controlled to move.
- 如权利要求27所述的计算机控制所驱动的屏幕上光标移动的方法,其中在所述步骤(b)中包括如下步骤:在对每帧所述定位图像处理后,如果产生了所述影像位移数据,则根据该影像位移数据控制所驱动的屏幕上的光标移动;如果没有产生所述影像位移数据,则根据前一帧定位图像处理后产生的影像位移数据控制所驱动的屏幕上的光标移动。The method for controlling the movement of a cursor on a driven screen by a computer according to claim 27, wherein said step (b) comprises the following step: after processing each frame of said positioning image, if said image displacement occurs Data, the cursor movement on the driven screen is controlled according to the image displacement data; if the image displacement data is not generated, the cursor movement on the driven screen is controlled according to the image displacement data generated after the positioning image processing of the previous frame .
- 一空中鼠标,适用于控制一被控设备,其特征在于,包括:An air mouse, suitable for controlling a controlled device, is characterized by including:一壳体;A shell一图像采集单元,其中所述图像采集单元被设置于所述壳体,其中所述图像采集单元输出具有一定位标志影像的一定位图像;以及An image acquisition unit, wherein the image acquisition unit is disposed on the housing, and the image acquisition unit outputs a positioning image with a positioning mark image; and一处理单元,其中所述处理单元被设置于所述壳体,所述处理单元被可通信地连接于所 述图像采集单元,其中所述处理单元根据所述定位标志影像的一特征代码和一影像位置数据处理所述定位图像,并发送一光标控制信号于所述被控设备,以控制所述被控设备。A processing unit, wherein the processing unit is arranged in the housing, the processing unit is communicably connected to the image acquisition unit, and the processing unit is based on a feature code and a feature code of the positioning mark image The image position data processes the positioning image, and sends a cursor control signal to the controlled device to control the controlled device.
- 根据权利要求29所述的空中鼠标,进一步包括一按键单元,其中所述按键单元被设置于所述壳体,所述按键单元被可通信地连接于所述处理单元。The air mouse according to claim 29, further comprising a button unit, wherein the button unit is disposed on the housing, and the button unit is communicably connected to the processing unit.
- 根据权利要求29所述的空中鼠标,进一步包括一光标移动控制键,其中所述光标移动控制键被设置于所述壳体,所述光标移动控制键被可通信地连接于所述处理单元。The air mouse according to claim 29, further comprising a cursor movement control key, wherein the cursor movement control key is provided on the housing, and the cursor movement control key is communicably connected to the processing unit.
- 根据权利要求29所述的空中鼠标,其中所述图像采集单元包括一镜头和一图像传感器模块,其中所述镜头被可通信地连接于所述图像传感模块。28. The air mouse of claim 29, wherein the image acquisition unit includes a lens and an image sensor module, wherein the lens is communicably connected to the image sensor module.
- 一空中鼠标系统,适用于控制一被控设备,其特征在于,包括:An air mouse system, suitable for controlling a controlled device, is characterized by including:一空中鼠标,其中所述空中鼠标包括一壳体、一图像采集单元以及一处理单元,其中所述图像采集单元被设置于所述壳体,其中所述图像采集单元输出具有一定位标志影像的一定位图像,其中所述处理单元被设置于所述壳体,所述处理单元被可通信地连接于所述图像采集单元,其中所述处理单元根据所述定位标志影像的一特征代码和一影像位置数据处理所述定位图像,并发送一光标控制信号于所述被控设备;和An air mouse, wherein the air mouse includes a housing, an image acquisition unit, and a processing unit, wherein the image acquisition unit is disposed on the housing, and the image acquisition unit outputs an image with a positioning mark A positioning image, wherein the processing unit is provided in the housing, the processing unit is communicably connected to the image acquisition unit, and the processing unit is based on a feature code and a feature code of the positioning mark image The image position data processes the positioning image and sends a cursor control signal to the controlled device; and至少一定位标志,其中所述定位标志能够形成具有所述定位标志影像的所述定位图像,利用所述定位标志能够确定所述空中鼠标的位置。At least one positioning mark, wherein the positioning mark can form the positioning image with the positioning mark image, and the position of the air mouse can be determined by using the positioning mark.
- 根据权利要求33所述的空中鼠标系统,其中所述空中鼠标系统的所述定位标志被实施为至少两个,且两个所述定位标志具有不同的特征,且每个所述定位标志的特征能够被记录于所述定位图像中。The air mouse system according to claim 33, wherein the positioning marks of the air mouse system are implemented as at least two, and the two positioning marks have different characteristics, and the characteristics of each positioning mark It can be recorded in the positioning image.
- 根据权利要求34所述的空中鼠标系统,其中两个所述定位标志在所述定位图像中形成的所述定位标志影像,其组成像素有不同的像素值。The air mouse system according to claim 34, wherein the positioning mark images formed by the two positioning marks in the positioning image have different pixel values.
- 根据权利要求34所述的空中鼠标系统,其中两个所述定位标志在所述定位图像中形成的所述定位标志影像,由不同数量的像素组成。The air mouse system according to claim 34, wherein the positioning mark images formed by the two positioning marks in the positioning image are composed of different numbers of pixels.
- 根据权利要求34所述的空中鼠标系统,其中所述定位标志被实施为一红外光源。The air mouse system according to claim 34, wherein the positioning mark is implemented as an infrared light source.
- 根据权利要求34所述的空中鼠标系统,其中所述定位标志被实施为一可见光源。The air mouse system according to claim 34, wherein the positioning mark is implemented as a visible light source.
- 根据权利要求34所述的空中鼠标系统,其中所述空中鼠标和所述定位标志的距离被保持于大于等于50cm小于等于100cm。The air mouse system according to claim 34, wherein the distance between the air mouse and the positioning mark is maintained to be greater than or equal to 50 cm and less than or equal to 100 cm.
- 根据权利要求34所述的空中鼠标系统,进一步包括一支架,其中所述定位标志被横向地设置于所述支架。The air mouse system of claim 34, further comprising a bracket, wherein the positioning mark is laterally arranged on the bracket.
- 根据权利要求34所述的空中鼠标系统,进一步包括一支架,其中所述定位标志被横向地设置于所述支架。The air mouse system of claim 34, further comprising a bracket, wherein the positioning mark is laterally arranged on the bracket.
- 一图像处理方法,其特征在于,所述图像处理方法包括如下步骤:An image processing method, characterized in that the image processing method includes the following steps:(a)获取具有一定位标志影像的一定位图像;和(a) Obtain a positioning image with a positioning mark image; and(b)通过读取所述定位图像中的像素的像素值的方式确定所述定位标志影像的一特征代码和一影像位置数据。(b) Determine a feature code and image position data of the positioning mark image by reading the pixel value of the pixel in the positioning image.
- 根据权利要求42所述的图像处理方法,其中在所述步骤(b)中,包括步骤(c)逐个读取所述定位图像中每个像素的像素值,找到第1个像素值大于一预设像素值的像素时,记录所述像素的像素值及坐标。The image processing method according to claim 42, wherein the step (b) includes the step (c) reading the pixel value of each pixel in the positioning image one by one, and finding the first pixel value greater than a predetermined value When a pixel with a pixel value is set, the pixel value and coordinates of the pixel are recorded.
- 根据权利要求42所述的图像处理方法,进一步包括步骤(d)对比先后获取的两个定位图像的所述定位标志影像的所述特征代码。The image processing method according to claim 42, further comprising step (d) comparing the feature codes of the positioning mark images of the two positioning images obtained one after another.
- 根据权利要求44所述的图像处理方法,其中在所述步骤(b)之后包括步骤(e)对比所述特征代码相同的两组所述定位图像的所述定位标志影像,并得到一影像位移数据。The image processing method according to claim 44, wherein after the step (b), it comprises the step (e) comparing the positioning mark images of the two sets of positioning images with the same feature code, and obtaining an image displacement data.
- 根据权利要求42所述的图像处理方法,其中在所述步骤(b)中进一步包括:The image processing method according to claim 42, wherein said step (b) further comprises:(ⅰ)在一预选区域中逐个读取每个像素的像素值,直到找到像素值大于一预设像素值的一个像素;(I) Read the pixel value of each pixel one by one in a preselected area until a pixel with a pixel value greater than a preset pixel value is found;(ⅱ)找出以所述像素为中心的一预设像素范围内像素值大于所述预设像素值的全部像素;以及(Ii) Find all pixels in a preset pixel range centered on the pixel whose pixel value is greater than the preset pixel value; and(ⅲ)根据找出的全部像素的数量确定特征代码,并在所有像素值大于所述预设像素值以上的像素中,找出坐标值最小像素,将该像素的坐标记录为所述影像位置数据。(Iii) Determine the feature code according to the number of all pixels found, and find the pixel with the smallest coordinate value among all the pixels whose pixel value is greater than the preset pixel value, and record the coordinate of this pixel as the image position data.
- 根据权利要求42所述的图像处理方法,其中在所述步骤(b)中进一步包括如下步骤:The image processing method according to claim 42, wherein said step (b) further comprises the following steps:(ⅰ)读取一预选区域内所有像素的像素值,找出其中全部像素值大于一预设像素值的像素作为一备选像素;(I) Read the pixel values of all pixels in a preselected area, and find out all the pixels whose pixel values are greater than a preset pixel value as a candidate pixel;(ⅱ)从所述备选像素中选择一个坐标值最小像素,找出以所述像素为中心的一预设像素范围内像素值大于所述预设像素值的全部像素,定义这些像素称为一组成像素;以及(Ii) Select a pixel with the smallest coordinate value from the candidate pixels, find out all pixels with a pixel value greater than the preset pixel value in a preset pixel range centered on the pixel, and define these pixels as A component pixel; and(ⅲ)将所述组成像素的总数记录为所述特征代码,并在所述组成像素中,找出一个坐标值最小像素,将该像素的坐标记录为所述特征代码对应的影像位置数据。(Iii) Record the total number of the constituent pixels as the feature code, find a pixel with the smallest coordinate value among the constituent pixels, and record the coordinate of the pixel as the image position data corresponding to the feature code.
- 根据权利要求47所述的图像处理方法,其中在所述步骤(b)中,进一步包括步骤(ⅳ)在所述备选像素中去掉所述组成像素,如果还剩下有所述备选像素,则从步骤(ⅱ)开始处理,如果没有备选像素了,则图像处理过程结束。The image processing method according to claim 47, wherein in said step (b), it further comprises a step (iv) of removing the constituent pixels from the candidate pixels, if there is still the candidate pixel , Then start processing from step (ii), if there are no candidate pixels, the image processing process ends.
- 根据权利要求42所述的图像处理方法,其中在所述步骤(b)中,进一步包括如下步骤:The image processing method according to claim 42, wherein said step (b) further comprises the following steps:(ⅰ)在一预选区域内,根据由左至右,由上至下的顺序,读取每个像素的像素值,直到找到像素值大于一预设像素值的一个像素;(I) In a preselected area, read the pixel value of each pixel according to the order from left to right and from top to bottom until a pixel with a pixel value greater than a preset pixel value is found;(ⅱ)找出以所述像素为中心的一预设像素范围内像素值大于所述预设像素值的全部像素;(Ii) Find out all pixels with a pixel value greater than the preset pixel value within a preset pixel range centered on the pixel;(ⅲ)如果找到的全部像素的列坐标相同,将特征代码记录为1;如果行坐标相同,将特征代码记录为2;以及(Iii) If the column coordinates of all the pixels found are the same, record the feature code as 1; if the row coordinates are the same, record the feature code as 2; and(ⅳ)在找到的全部像素中,找出坐标值最小像素,将该像素的坐标记录为所述特征代码所对应的所述影像位置数据。(Iv) Among all the found pixels, find the pixel with the smallest coordinate value, and record the coordinate of the pixel as the image position data corresponding to the feature code.
- 根据权利要求42所述的图像处理方法,其中在所述步骤(b)中,进一步包括如下步骤:The image processing method according to claim 42, wherein said step (b) further comprises the following steps:(ⅰ)读取一预选区域内所有像素的像素值,找出其中全部像素值大于一预设像素值的像素作为一备选像素;(I) Read the pixel values of all pixels in a preselected area, and find out all the pixels whose pixel values are greater than a preset pixel value as a candidate pixel;(ⅱ)从所述备选像素中选择一个坐标值最小像素,找出以所述像素为中心的一预设像素范围内像素值大于所述预设像素值的全部像素,定义这些像素称为一组成像素;(Ii) Select a pixel with the smallest coordinate value from the candidate pixels, find out all pixels with a pixel value greater than the preset pixel value in a preset pixel range centered on the pixel, and define these pixels as One component pixel;(ⅲ)如果找出的全部组成像素的列坐标相同,则将特征代码记录为1;如果全部组成像素的行坐标相同,则将特征代码记录为2;并在全部组成像素中,找出一个坐标值最小像素,将该像素的坐标记录为所述影像位置数据;以及(Iii) If the column coordinates of all the constituent pixels found are the same, record the feature code as 1; if the row coordinates of all constituent pixels are the same, record the feature code as 2; and among all the constituent pixels, find one The pixel with the smallest coordinate value, and the coordinate of the pixel is recorded as the image position data; and(ⅳ)在所述备选像素中,去掉和所述组成像素相同坐标的像素,如果还剩下有所述备选像素,则又从步骤(ⅱ)开始处理;如果没有备选像素了,则获取影像数据的步骤结束。(Iv) Among the candidate pixels, remove the pixels with the same coordinates as the constituent pixels. If the candidate pixels are still left, then start processing from step (ii); if there are no candidate pixels, Then the step of acquiring image data ends.
- 一控制方法,其特征在于,所述控制方法包括如下步骤:A control method, characterized in that the control method includes the following steps:(A)间隔地接收至少两组具有一定位标志影像的一定位图像,其中所述定位标志影像包括一特征代码和对应于所述特征代码的一影像位置数据;和(A) receiving at least two sets of a positioning image with a positioning mark image at intervals, wherein the positioning mark image includes a feature code and an image position data corresponding to the feature code; and(B)根据所述定位标志影像的所述特征代码和所述影像位置数据判断是否产生一影像位移数据,若产生所述影像位置数据,则根据所述影像位移数据发送一位移信号于一被控设备。(B) Determine whether to generate an image displacement data according to the feature code of the positioning mark image and the image position data. If the image position data is generated, send a displacement signal to a target according to the image displacement data.控设备。 Control equipment.
- 根据权利要求51所述的控制方法,其中在所述步骤(B)中包括如下步骤:The control method according to claim 51, wherein said step (B) includes the following steps:(B.1)在接收的前一组所述影像数据中查找后一组所述影像数据的所述特征代码;(B.1) Searching for the feature code of the latter group of image data in the received previous group of image data;(B.2)对比所述特征代码相同的两组所述定位图像的所述定位标志影像,并得到所述影像位移数据;以及(B.2) Compare the positioning mark images of the two sets of positioning images with the same feature code, and obtain the image displacement data; and(B.3)根据所述影像位移数据控制一光标的移动。(B.3) Control the movement of a cursor according to the image displacement data.
- 根据权利要求51所述的控制方法,其中在所述步骤(B)中包括如下步骤:在对每帧所述定位图像处理后,如果产生了所述影像位移数据,则根据该影像位移数据向所述被控 设备发送所述位移信号;如果没有产生所述影像位移数据,则向所述被控设备发送对前一帧所述定位图像处理后产生的所述位移信号。The control method according to claim 51, wherein said step (B) comprises the following step: after processing each frame of said positioning image, if said image displacement data is generated, then according to the image displacement data to The controlled device sends the displacement signal; if the image displacement data is not generated, the controlled device sends the displacement signal generated after processing the positioning image of the previous frame.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910618882.0A CN112214117A (en) | 2019-07-10 | 2019-07-10 | Image processing method and chip of air mouse, air mouse and system |
CN201910618882.0 | 2019-07-10 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2021004505A1 true WO2021004505A1 (en) | 2021-01-14 |
Family
ID=74047094
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/101074 WO2021004505A1 (en) | 2019-07-10 | 2020-07-09 | Air mouse, air mouse system, image processing method and control method |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN112214117A (en) |
WO (1) | WO2021004505A1 (en) |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2005346145A (en) * | 2004-05-31 | 2005-12-15 | Olympus Corp | Picture processor, method and program for processing picture, and recording medium |
CN102508565A (en) * | 2011-11-17 | 2012-06-20 | Tcl集团股份有限公司 | Remote control cursor positioning method and device, remote control and cursor positioning system |
TW201443705A (en) * | 2013-05-01 | 2014-11-16 | Unigrand Ltd | Cursor locating device and cursor locating method |
CN104731373A (en) * | 2013-12-18 | 2015-06-24 | 原相科技股份有限公司 | Handheld pointing device and cursor locating method thereof |
-
2019
- 2019-07-10 CN CN201910618882.0A patent/CN112214117A/en active Pending
-
2020
- 2020-07-09 WO PCT/CN2020/101074 patent/WO2021004505A1/en active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2005346145A (en) * | 2004-05-31 | 2005-12-15 | Olympus Corp | Picture processor, method and program for processing picture, and recording medium |
CN102508565A (en) * | 2011-11-17 | 2012-06-20 | Tcl集团股份有限公司 | Remote control cursor positioning method and device, remote control and cursor positioning system |
TW201443705A (en) * | 2013-05-01 | 2014-11-16 | Unigrand Ltd | Cursor locating device and cursor locating method |
CN104731373A (en) * | 2013-12-18 | 2015-06-24 | 原相科技股份有限公司 | Handheld pointing device and cursor locating method thereof |
Also Published As
Publication number | Publication date |
---|---|
CN112214117A (en) | 2021-01-12 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7161596B2 (en) | Display location calculation means | |
US20170154570A1 (en) | Organic light emitting diode (oled) display apparatus having light sensing function | |
US7975243B2 (en) | System and method for television control using hand gestures | |
JP5232478B2 (en) | Information processing program, information processing apparatus, information processing system, and information processing method | |
JP2009050701A (en) | Interactive picture system, interactive apparatus, and its operation control method | |
US10537814B2 (en) | Screen coding methods and camera based game controller for video shoot game | |
EP3479881A1 (en) | Display apparatus and control method thereof | |
JP5037842B2 (en) | Information processing apparatus and program | |
US20140132725A1 (en) | Electronic device and method for determining depth of 3d object image in a 3d environment image | |
US9058064B2 (en) | Interactive image system, interactive control device and operation method thereof | |
JP5379385B2 (en) | Program information display program and program information display device | |
US8184211B2 (en) | Quasi analog knob control method and appartus using the same | |
JP2024029047A (en) | Display detecting apparatus, method therefor, and computer readable medium | |
KR100820573B1 (en) | Computer input device utilizing a camera to recognize position and twinkling compare laser pointing image with computer display picture | |
US20050107160A1 (en) | [photographic pointer positioning system and its operation process] | |
WO2021004505A1 (en) | Air mouse, air mouse system, image processing method and control method | |
KR20180023723A (en) | Display apparatus and the control method thereof | |
CN108139203A (en) | Information processing equipment and location information acquisition method | |
TWI766417B (en) | Color change of information elements | |
CN114764275A (en) | Image processing method for air mouse, air mouse and system | |
US20230310980A1 (en) | Game controller with displacement detecting device and displacement detecting device thereof | |
CN114115776B (en) | Display control method, display control device, electronic equipment and storage medium | |
WO2023275920A1 (en) | Trigger signal generation device and mobile terminal | |
CN213276698U (en) | Translation pen | |
KR100799766B1 (en) | Apparatus for controlling moving of pointer |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20837664 Country of ref document: EP Kind code of ref document: A1 |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20837664 Country of ref document: EP Kind code of ref document: A1 |