KR100720647B1 - Ring-style pointing device - Google Patents

Abstract

Disclosed is a ring pointing device. The apparatus of the present invention comprises: an imaging element; An optical system for forming an image on the image pickup device; An image processor which receives an image from an image pickup device and generates a displacement vector; An up-down button switch; A detector for activating / deactivating the image processor or detecting a click / double click event of the button switch and generating a corresponding message according to the state of the button switch; A communication unit for transferring the displacement vector, the click message and the double click message to a GUI subsystem of the computer; A power supply unit; It is characterized in that the ring-shaped body that each component is installed is provided. According to the present invention, it is easy to use by using a ring shape, it can be clicked / double-clicked only with the thumb, and the GUI cursor can be operated only by the movement of the wrist or the arm. This allows both fast input of large displacements and fine input of small displacements.

Rings, Pointing Devices, Wearable Computers, Imaging Devices, Optical Systems, Button Switches

Description

Ring-style pointing device

1A and 1B are schematic views for explaining a ring-shaped pointing device according to the present invention;

2A is a schematic diagram for explaining an image pickup device and an optical system in the ring-shaped pointing device according to FIG. 1A;

Figure 2b is a state transition diagram for explaining the operation of the sensing unit in the ring-shaped pointing device according to Figure 1a;

2C is a flowchart for explaining a processing algorithm of an image processing unit in the ring-shaped pointing device according to FIG. 1A; And

3a and 3b are real pictures of the ring-shaped pointing device according to Figure 1a.

TECHNICAL FIELD The present invention relates to a ring pointing device, and more particularly, to a ring pointing device for a wearable computer.

Representative examples of human interface device technologies include mice, trackballs, joysticks, touch pads, keyboards, keypads, and the like. Pointing device technologies for wearable computers include small trackballs, small joysticks, trackpoints, gyroscope-based mice, and gesture recognition. A globe etc. are mentioned.

However, in the case of a pointing device, such as a small trackball, a small joystick, a gyroscope-based mouse, and the like, which has been developed for a conventional wearable computer, it is inconvenient to hold and release it by hand. In the case of the gesture recognition glove, there is no such burden because it is always worn, and there is a problem in that it is inconvenient to perform a routine work while wearing the glove.

Therefore, there is a need for a pointing device capable of minimizing the burden on wearing and living as a pointing device for a wearable computer.

In addition, in the case of a small joystick, a trackpoint, a gesture recognition glove, and the like, the operation radius is small, so that the displacement of the operation is inevitably mapped to the speed on the graphical user interface (GUI).

Therefore, there is a need for an efficient pointing device with less constraint on the operating radius.

Therefore, the technical problem to be achieved by the present invention is to provide a ring-type pointing device that can minimize the burden on the wearable and wearable life, easy to operate.

Another object of the present invention is to provide a ring pointing device capable of providing a small but large operating radius to enable fast input of large displacement and fine input of small displacement.

Ring-shaped pointing device of the present invention for solving the above technical problem: an imaging device; An optical system for forming an image of a front background on the image pickup device; An image processor which receives the image from the image pickup device and calculates displacement between successive images to generate a displacement vector; A button switch up-down by a user's operation; A detector for activating or deactivating the image processor or detecting a click or double click event of the button switch and generating a corresponding message according to the state of the button switch; A communication unit for communicating the displacement vector, the click message and the double click message to a GUI subsystem of a computer; A power supply unit supplying power to each of the components; The image pickup device, the optical system, and an image processing unit, a button switch, a sensing unit, a communication unit and a power supply unit is provided with a ring-shaped body, respectively.

In this case, the detection unit generates the click message when the user releases the pressing within a predetermined time while pressing the button switch, and activates the image processing unit when the button switch is pressed for the predetermined time or more, and the image processing unit When the button switch is released in the activated state, the image processing unit is deactivated. When the button switch is pressed and the button release and the press and release are continuously executed within a predetermined time period, the double click message is generated. do.

Furthermore, the optical system is characterized in that it is a single convex lens.

Further, the image processor calculates the displacement amount of the feature points by finding a predetermined number or more of feature points in the previous image by using a corner detector algorithm, calculating the optical flow between the front image and the rear image, and then calculating the displacements of the feature points. And taking the average to calculate the displacement vector between the front image and the back image.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

1A and 1B are schematic views for explaining a ring-shaped pointing device according to the present invention, FIG. 2A is a schematic view for explaining an image pickup device and an optical system in the ring-shaped pointing device according to FIG. 1A, and FIG. 2B is a view of FIG. 1A. 2C is a flowchart illustrating a processing algorithm of an image processing unit in the ring-shaped pointing device according to FIG. 1A, and FIGS. 3A and 3B are diagrams for describing the operation of the sensing unit in the ring-shaped pointing device according to FIG. Real pictures of the ring pointing device according to 1a.

1A and 1B, a ring-shaped pointing device according to the present invention includes a ring-shaped body 100, an image pickup device 200, and an optical system 300 for forming an image of a front background on an image pickup device. , A button switch 400, a sensing unit 500, an image processing unit 600, a communication unit 700, and a power supply unit 800 are provided. Each component, that is, an imaging device 200, an optical sheath The system 300, the button switch 400, the sensing unit 500, the image processing unit 600, the communication unit 700, and the power supply unit 800 are installed in the ring-shaped body 100.

The small camera composed of the image pickup device 200 and the optical system 300 is installed to face the front, and continuously acquires the front image.

Referring to FIG. 2A, the optical system 300 can sufficiently meet the purpose of use of the present invention even with a single convex lens because the algorithm of the image processing unit is not sensitive to image distortion. Since it is preferable to take an image of a far object, the distance between the convex lens 300 and the image pickup device 200 becomes the focal length of the lens 200. Therefore, the focal length of the convex lens becomes a factor for determining the size of the camera unit including the optical system and the image pickup device. If the focal length is larger than the size of the image pickup device, a small area of the image is taken and vice versa. Through the implementation of the present invention, it has been found that taking a wide range of images is more compatible with an image processing algorithm. Therefore, considering both the size of the system and the image processing algorithm, it is desirable to use a lens with the shortest focal length.

Referring back to FIGS. 1A and 1B, the button switch 400 is up-down by a user's manipulation.

The detector 500 activates or deactivates the image processor 600 according to the state of the button switch 400, and detects a click or double click event of the button switch 400 to generate a corresponding message. For example, the detection unit 500 generates a message for a click event when the user releases the pressing within a predetermined time while pressing the button switch 400. When the button switch 400 is pressed for a predetermined time or more, the image processing unit 600 is activated, and when the button switch 400 is released while the image processing unit 600 is activated, the image processing unit 600 is deactivated. And while pressing the button switch 400 while pressing and releasing and pressing and releasing the press continuously within a predetermined time to generate a message for a double click event. The operation of the sensing unit 500 will be described in more detail with reference to FIG. 2B.

Referring to FIG. 2B in conjunction with FIGS. 1A and 1B, the sensing unit 500 initially resets the timer when the button switch 400 is pressed while being at 'STATE 0' and transitions to 'STATE 1'. do. When the button switch 400 is released from the 'STATE 1' within a predetermined time T, the sensing unit 500 resets the timer, generates a click event message, and transitions to 'STATE 2'. At this time, if the button switch 400 is not released for a predetermined time (T) or more in the 'STATE 1' state, the detection unit 500 activates (on) the image processing unit (Tracker, 600) and the 'STATE T'. Transition to. When the button switch 400 is pressed down while in the 'STATE T', the detection unit 500 deactivates the image processing unit 600 and returns to 'STATE 0'. On the other hand, if the button switch 400 is pressed again within a certain time in the 'STATE 2' state (Button down), the detection unit 500 resets the timer and transitions to 'STATE 3', and again at 'STATE 3' When the button switch 400 is pressed down (Button up) within the detection unit 500 generates a double-click message and returns to 'STATE 0'. That is, the image processing unit 600 is activated when the button is held down for more than a predetermined time after the button switch 400 is pressed, and the image processing unit 600 is released when the button switch 400 is released. Is disabled (Tracker off). On the other hand, when the button is released after pressing the button switch 400 within a predetermined time, a click event message is generated, and if a continuous pressing and releasing operation is executed within a predetermined time, a double click message is generated. An important point here is that the image processor 600 does not operate when a click or double click is detected. This is to solve the problem that in the ring-shaped pointing device according to the present invention, when the user clicks or double-clicks the button switch 400, the target on the GUI currently being missed may be missed by the shaking of the hand. When the target on the GUI is clicked using the ring-shaped pointing device according to the present invention, the operation of moving the cursor to the target, the operation of clicking or double-clicking on the target is not preferable to be performed at the same time and should be performed sequentially.

Referring again to FIGS. 1A and 1B, the image processor 600 receives an image from the imaging device 200 and calculates a displacement between successive images to generate a displacement vector. The detailed algorithm of the image processing unit 600 will be described with reference to FIG. 2C.

Referring to FIG. 2C in connection with FIGS. 1A and 1B, the flag set by the detection unit 500 is checked in the first judgment box to perform a flow downward only when the tracker is in an active state. In this case, set the Old_Frame variable to NULL and return to the previous step of the judgment box.

In the activated state, a new image is taken from the image pickup device 200 and assigned to the New_Frame variable. If the Old_Frame variable is NULL, it indicates that there is no previously acquired image. Therefore, the displacement calculation step is skipped and the New_Frame variable is directly assigned to the Old_Frame variable. Return to the beginning of the flowchart. From the second execution of the processing algorithm on the image processing unit 600, since the Old_Frame variable is not NULL, the newly acquired image can be compared with the previous image and the displacement vector can be calculated.

Algorithms for displacement calculation between two images have been variously developed according to the purpose, but the algorithm used in the embodiment of the present invention is basically as follows. Using the corner detector algorithm (GoodFeaturesToTrack function of OpenCV) in the previous picture, find a certain number of feature points and calculate the optical flow between the previous picture and the next picture (OpenCV's CalcOpticalFlowPyrLK function) Is calculated and then the displacement vector (dx, dy) between the preceding image and the later image is calculated by taking the average of these displacement amounts. In order to improve the efficiency of the algorithm, a method of searching for the first image once when the image processing unit is activated without finding the feature points every time may be used to continuously track these feature points. However, in this case, the number of feature points being tracked may be reduced because the selected feature points are out of the range of the image. For this reason, if the number of feature points being tracked decreases below a certain level, it is necessary to reconfigure the feature points using the corner detector. Nevertheless, the amount of computation is considerably reduced than iterating over each feature point.

Referring again to FIG. 1A, the communicator 700 is a wireless communicator and transmits the displacement vector generated by the conversion processor 600, the click event message generated by the detector 500, and the double click event message generated by the detector 500 to the GUI subsystem of the wearable computer. To pass.

The power supply unit 800 supplies power to each component, that is, the imaging device 200, the optical system 300, the button switch 400, the detection unit 500, the image processing unit 600, and the communication unit 700. . It can be configured with a small battery and a DC / DC converter.

The GUI subsystem of the wearable computer receives the click message or the double click message and the displacement vector through the communication unit 700 and processes them in the same manner as a message from a conventional mouse.

3A and 3B, the button switch 400 is installed at an easy operation position with the thumb on the ring-shaped body, and the camera unit including the imaging device 200 and the optical system 300 is ring It is installed to be located in front of the mold body. The microprocessor for event detection and image processing, that is, the sensing unit and the image processing unit is installed to be located at the bottom of the ring body, and the wireless transmission module for wireless transmission of data is installed to be located at the top of the ring body. The power supply is installed to be located at the rear of the ring body. At this time, the position of the camera unit and the button switch in the components of the ring-shaped pointing device according to the present invention should be located on the front and the top surface of the ring-shaped body, of course, other components can be arranged differently depending on the situation Do. However, the position of the radio transmitter should be determined in consideration of the interference between the antenna and the hand, and it is desirable to be located at the rear of the ring body in consideration of the fact that the power supply requires a certain thickness at the current technology level.

According to the ring-shaped pointing device according to the present invention, after a short time of pressing a button switch and a short time of generally less than 1 second, the displacement of the front image is calculated and a continuous displacement vector is calculated from the results. Is sent to move the cursor. At this time, since the displacement signal is generated by a movement that can cause displacement of the image taken by the camera, whether the wrist or the arm is moved, it can be manipulated in various ways according to the purpose. For example, small displacements may be generated by wrist movements, and large displacements may be generated by using arm movements.

As described above, according to the present invention, by using a ring shape, the wearing comfort is good, the hand can be clicked or double-clicked instead of being held by a hand, and unlike a general mouse, the wrist can be used in any posture without a plane for operation. Alternatively, the GUI cursor can be manipulated with only the movement of the arm, making it very easy to use.

Furthermore, it provides a small but large operating radius, allowing both fast input of large displacements and fine input of small displacements.

The present invention is not limited only to the above embodiments, and it is apparent that many modifications are possible by those skilled in the art within the technical spirit of the present invention.

Claims (4)

An imaging device; An optical system for forming an image of a front background on the image pickup device; An image processor which receives the image from the image pickup device and calculates displacement between successive images to generate a displacement vector; A button switch up-down by a user's operation; A detector for activating or deactivating the image processor or detecting a click or double click event of the button switch and generating a corresponding message according to the state of the button switch; A communication unit for communicating the displacement vector, the click message and the double click message to a GUI subsystem of a computer; A power supply unit supplying power to each of the components; And a ring-shaped body having the image pickup device, the optical system, and an image processing unit, a button switch, a sensing unit, a communication unit, and a power supply unit, respectively. The method of claim 1, wherein the detection unit, When the user releases the pressing within a predetermined time while pressing the button switch, the click message is generated, and when the button switch is pressed for the predetermined time or more, the image processing unit is activated, and the button is activated while the image processing unit is activated. And depressing the switch deactivates the image processing unit, and when the button is pressed and the depressing and the depressing and depressing are executed continuously within a predetermined time, the double pointing message is generated. 8. The ring pointing device of claim 1 wherein the optical system is a single convex lens. The method of claim 1, wherein the image processing unit, Using a corner detector algorithm, find a predetermined number of feature points in the previous image, calculate an optical flow between the front image and the rear image, calculate the displacement amount of the feature points, and then average the displacement amounts to obtain the previous image. And the displacement vector is calculated between the image and the back image.

Images