KR20140060604A - Method for controlling electronic devices by using virtural surface adjacent to display in virtual touch apparatus without pointer - Google Patents

Abstract

The present invention provides a method of controlling an electronic device using a virtual plane around a display surface in a virtual touch device without a pointer. The virtual touch device includes: an image acquisition unit which comprises two or more image sensors placed at different positions from each other and takes images of the body of a user at the front of a display surface; a spatial coordinate calculation unit which calculates three-dimensional coordinate data of the body of the user using the images received from the image acquisition unit; a touch position calculation unit which calculates coordinate data of a contact point between a line connecting first spatial coordinates and second spatial coordinates received from the spatial coordinate calculation unit and a virtual plane around the display surface; and a virtual touch processing unit which generates a command code for performing a designated action corresponding to the data of the contact point received from the touch position calculation unit and inputs the code to a main control unit of an electronic device. The method of controlling an electronic device includes: a step (A) of detecting a virtual plane around a display surface, the end point of a finger, a contact point of an eye by processing the three-dimensional coordinate data (X1, Y1, Z1) of the end of the finger and the three-dimensional coordinate data (X2, Y2, Z2) of the eye; a step (B) of calculating at least one among the depth change, trace change, and speed change of the detected end point of the finger; and a step (C) of operating the electronic device based on at least one among the calculated depth change, trace change, and speed change of the end point of the finger and selecting a corresponding area as if a certain part of a touch panel is touched.

Description

TECHNICAL FIELD The present invention relates to a method for controlling an electronic device using a virtual plane around a display display surface in a virtual touch device that does not use a pointer,

The present invention relates to a method for remotely operating an electronic device by a user, and more particularly, to a method and apparatus for remotely operating an electronic device by setting a virtual plane around a display surface of an electronic device so as not to display a pointer on the display surface, And more particularly, to a method of controlling an electronic device capable of accurately controlling an electronic device remotely without displaying the electronic device.

Korean Patent No. 10-1151692 of the present applicant discloses a virtual touch device and method that does not use a pointer, and a virtual touch device and method using such a pointer do not use a pointer on a display, Thereby enabling quick operation. Accordingly, it is an object of the present invention to provide an apparatus and a method for applying the advantages of a touch panel of an electronic device in the field of an electronic device remote control device. Generally, an electronic device such as a computer, a digital TV, and the like can operate an electronic device by performing a specific additional operation after generating a pointer in a corresponding area by a control method. Up to now, a myriad of technologies, such as a method for setting the position of the display pointer at a high speed, a method for selecting the speed of the pointer in the display, a method for using one or more pointers, and a method for controlling the pointer by the remote control, Technology has been developed only.

In addition, the virtual touch apparatus and method which do not use the pointers of the above-mentioned Japanese Patent No. 10-1151692 can precisely point a specific area of the display surface of the electronic apparatus. In other words, to precisely point the display surface of the electronic device, the principle of 'pointing the object precisely by using only the fingertip and one eye' is adopted, so that it is possible to precisely You can point to the menu.

However, when a specific image is displayed on the display screen of the electronic device in the virtual touch device and method which does not use the pointer, there is a problem that when the operation screen of the electronic device is displayed on the display screen, .

In addition, when the operation area is limited only to the display surface of the electronic device, the display area becomes relatively smaller as the user moves away from the electronic device, and the operation area becomes smaller.

Accordingly, it is an object of the present invention to provide a user interface that allows a user to remotely operate an electronic device, such as touching the touch panel surface, And to provide an apparatus and method capable of operating an electronic device without displaying it.

In addition, the present invention is intended to enable the operation of the electronic device even when the user moves away from the display, by extending the periphery of the display surface of the electronic device to the operation screen. In other words, the "expansion of the touch panel area" is implemented around the display display face.

According to an aspect of the present invention, there is provided a method of controlling an electronic device using a virtual plane around a display screen of a display device, the method including two or more image sensors disposed at different positions, A space coordinate calculator for calculating three-dimensional coordinate data of the user's body using the image received from the image acquiring unit; A touch position calculation unit for calculating contact point coordinate data in which a straight line connecting the first spatial coordinate and the second spatial coordinate meets an imaginary plane around the display surface using the contact coordinate data received from the touch position calculation unit, And inputting the command code to the main control unit of the electronic apparatus, Through the virtual touch device constituted by the touch processing section,

(A) processing three-dimensional coordinate data (X1, Y1, Z1) of a fingertip and three-dimensional coordinate data (X2, Y2, Z2) of a center point of one eye to generate a virtual plane around the display screen, Detecting respective contacts of one eye,

(B) calculating at least one of a depth change of the detected finger end point, a change of a locus, a holding time, and a change speed;

(C) The electronic device can be operated based on at least one of the depth change of the calculated finger end point, the change of the locus, the holding time and the speed of change, and the user can select the corresponding area Wherein a virtual plane around a display screen in a virtual touch device without using a pointer is used to control the electronic device.

In one embodiment of the present invention, a three-dimensional coordinate calculation device for extracting three-dimensional coordinate data of a user's body and a three-dimensional coordinate calculation device for calculating three- A touch position calculation unit for calculating contact coordinate data in which a straight line connecting a coordinate and a second spatial coordinate meet the display surface and a command code for performing an operation set to correspond to the contact coordinate data received from the touch position calculation unit And a control unit including a virtual touch processing unit for generating and inputting the virtual touch signal to the main control unit of the electronic device,

(A) processing three-dimensional coordinate data (X1, Y1, Z1) of a fingertip and three-dimensional coordinate data (X2, Y2, Z2) of a center point of one eye to generate a virtual plane around the display screen, Detecting respective contacts of one eye,

(B) calculating at least one of a depth change of the detected finger end point, a change of a locus, a holding time, and a change speed;

(C) The electronic device can be operated based on at least one of the depth change of the calculated finger end point, the change of the locus, the holding time and the speed of change, and the user can select the corresponding area Wherein a virtual plane around a display screen in a virtual touch device without using a pointer is used to control the electronic device.

In another embodiment of the present invention, the virtual plane is divided into four selection areas of the upper, lower, left, and right sides of the electronic device display. In the virtual touch device without a pointer, Wherein the virtual plane of the electronic device is controlled.

In another embodiment of the present invention, the virtual plane is divided into eight selection areas on the top, bottom, left, and right sides of the electronic device display. In the virtual touch device without the pointer, A method for controlling an electronic device using a virtual plane around.

The method of using the virtual plane around the display screen of the electronic device according to the present invention as described above does not display the operation screen on the display screen while remotely operating the electronic device, .

Further, since the virtual plane around the electronic device display can be used as the operation screen, the operation is facilitated even if the electronic device is moved away from the display of the electronic device. That is, it has "expansion effect of touch panel ".

1 is a block diagram of a virtual touch device according to a first preferred embodiment of the present invention
2 is a diagram illustrating a screen for selecting a screen menu displayed on the display
Fig. 3 is a state diagram showing that a selection area is set by dividing a virtual plane around an electronic device display into 8 equal parts. Fig.
4 is a block diagram of a virtual touch device according to a second preferred embodiment of the present invention
5 is a view illustrating a screen for selecting a screen menu displayed on the display
6 is a state diagram showing that a virtual plane on the periphery of an electronic device display is divided into 8 equal parts and a selection area is set.

Other objects, features and advantages of the present invention will become apparent from the detailed description of the embodiments with reference to the accompanying drawings.

A preferred embodiment of a method for controlling an electronic device using a virtual plane around a display display surface in a virtual touch device without using a pointer according to the present invention will be described with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. It is provided to let you know. Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention and are not intended to represent all of the technical ideas of the present invention. Therefore, various equivalents There may be water and variations.

1st Example

1 is a block diagram showing a virtual touch device according to a first embodiment of the present invention.

Referring to FIG. 1, the virtual touch device 1 includes an image acquisition unit 10, a space coordinate calculation unit 20, a touch position calculation unit 30, and a virtual touch calculation unit 40.

The image acquisition unit 10 may include two or more image sensors 11 and 12 such as CCD or CMOS for detecting an image with a camera module of a kind and converting the image into an electrical image signal.

The space coordinate calculation unit 20 calculates the three-dimensional coordinate data of the user's body using the image received from the image acquisition unit 10. [ According to the present invention, the image sensors 11 and 12 constituting the image obtaining unit 20 photograph the user's body at different angles, and the spatial coordinate calculator 20 calculates the three-dimensional coordinates of the user's body using passive optical trigonometry The coordinate data can be calculated.

Generally, the optical three-dimensional coordinate calculation method can be classified into the active method and the manual method according to the sensing method. The active method is a method of calculating three-dimensional coordinate data of an object by projecting a predetermined pattern or sound wave onto an object and then measuring the amount of change through control of sensor parameters such as energy or focus, Method. On the other hand, the manual method uses intensity, parallax, etc. of the image taken without artificially projecting energy to the object.

In the present invention, a passive method that does not project energy to an object is adopted. This method is advantageous in that it is simple in equipment and can acquire a texture directly from an input image even though accuracy may be somewhat lower than that in an active method.

In the manual method, three-dimensional information can be obtained by applying trigonometry to feature points corresponding to the captured images. Camera self-calibration, Harris's corner extraction, SIFT, RANSAC, and Tsai techniques are commonly used as various related techniques for extracting three-dimensional coordinates by applying trigonometry. Particularly, a stereoscopic camera method may be used as a method of calculating three-dimensional coordinate data of a user's body. The stereoscopic camera method is a method of observing the same point on the surface of an object at two different points in the same manner as the structure of a binocular stereoscopic system in which a human is seen with two eyes and obtains a displacement, and the distance from the expected angle is obtained. The above-mentioned various three-dimensional coordinate calculation techniques can be easily understood by those skilled in the art and can be implemented. Therefore, a description thereof will be omitted. On the other hand, as patent documents related to a method of calculating three-dimensional coordinate data using a two-dimensional image, Korean Patent Laid-Open Nos. 10-0021803, 10-2004-0004135, 10-2007-0066382, 10-2007 -0117877 and so on.

The touch position calculating unit 30 calculates a touch point by using a first spatial coordinate and a second spatial coordinate received from the spatial coordinate calculating unit 20 and determines a straight line connecting the first spatial coordinate and the second spatial coordinate, And calculates the contact point coordinate data to be contacted.

Generally speaking, the fingers are the only part of the human body that allows precise and delicate manipulation. Particularly, when using either the thumb or the index finger among the fingers or using the two fingers together, it is possible to perform precise pointing. Accordingly, it is very effective to use the tip of the thumb and / or index finger as the first spatial coordinate in the present invention.

Also, in the same context, it is possible to use a pointed pointer (e.g., a pen tip) holding a finger with the finger instead of the end of the finger serving as the first spatial coordinate. The advantage of using such a pointer is that the portion that covers the display from the user's view is smaller than the fingertip, and the pointed end of the pointer can accurately point to a very small portion of the display.

In the present invention, only the center point of one eye of the user is used. For example, if the user places his / her index finger in front of his / her eyes and looks at the index finger, the index finger will appear as two. This occurs because the shapes of the index fingers viewed by the user's eyes are different from each other (due to the angular difference of the eyes). But if you look at your fingers only with one eye, your forefinger will look clearly. Even if you do not detect one eye, you can consciously see your forefinger clearly even if you look only with one eye. It is the same principle to close and close one eye for sports events that require a high degree of accuracy in aiming, such as shooting and archery.

The present invention adopts the principle that the shape of the fingertip can be clearly grasped when the fingertip (first spatial coordinate) is viewed with only one eye. In this way, the user must be able to correctly view the first spatial coordinates, so that a specific area of the display coinciding with the first spatial coordinates can be pointed.

In the present invention, when one user uses one of the fingers, the first spatial coordinate is any three-dimensional coordinates of the end of one of the fingers of the user and the end of the pointer held by the user, The second spatial coordinate will be the three-dimensional coordinates of the center point of either eye of the user.

If one user uses two or more of the fingers, the first spatial coordinates are three-dimensional coordinates of the ends of two or more fingers of the user's fingers, and the second spatial coordinates are the coordinates of the two or more users Dimensional coordinates of the center point of either eye.

In addition, when two or more users are used, the first spatial coordinates are three-dimensional coordinates of the ends of one or more fingers presented by the user of two or more users, and the second spatial coordinates are coordinates of one of the two or more users Dimensional coordinates of the center point of the center point.

The virtual touch processing unit 40 determines whether or not there is a change in the contact coordinate data for a set time or more from the calculated time of the initial contact coordinate data and sets the contact coordinate data to correspond to the contact coordinate data A command code for performing an operation may be generated and input to the main control unit 91 of the electronic apparatus. The virtual touch processing unit 40 can also process two users of one user or two or more users in the same manner.

The virtual touch processing unit 40 determines whether or not there is a change in the contact point coordinate data for a set time or more from the calculated time of the initial contact point coordinate data. If there is no change in the contact point coordinate data for the set time, And generates a command code for performing an operation set to correspond to the contact point coordinate data when the distance change is greater than or equal to the set distance, and transmits the command code to the main control unit 91 of the electronic apparatus As shown in FIG. The virtual touch processing unit 40 can also process two users of one user or two or more users in the same manner.

On the other hand, if it is determined that the variation of the contact coordinate data is within the setting range of the display 90, the contact coordinate data can be regarded as being not changed. That is, it is very difficult for the user to keep the contact coordinates as it is because the body or the finger naturally has slight movement or trembling when the user points the display to the end of the finger or the pointer. Therefore, when the contact coordinate data value is within the setting range of the display, it is regarded that there is no change in the contact coordinate data, and a command code for performing the set operation is generated and input to the main control unit 91 of the electronic device .

A representative example of the electronic apparatus to be subjected to the remote control according to the present invention is a digital television. Generally, the digital television receiver is provided with a broadcast signal receiving unit, a video signal processing unit, a system control unit, and the like. However, these components are obvious to those skilled in the art and will not be described in detail here. Other electronic devices to be subjected to the remote control according to the present invention include home networking such as home appliances as well as lighting devices, gas appliances, heating appliances, and method appliances, and various electronic appliances may be remotely controlled by the present invention .

The virtual touch device 1 according to the present invention may be inserted into the top of an electronic device frame to be remotely controlled and installed separately from the electronic device.

2 is a diagram illustrating a user selecting a particular icon on the display 90 of the electronic device according to an embodiment of the present invention. The user selects the 'music' icon on the display 90 while viewing the fingertip with one eye. The space coordinate calculation unit 20 generates three-dimensional spatial coordinates of the user's body, and the touch position calculation unit 30 calculates three-dimensional coordinate values of the three-dimensional coordinates (X1, Y1, Z1) The data (X2, Y2, Z2) are processed to calculate the contact coordinates (X, Y, Z) with the display screen. Then, the virtual touch processing unit 40 generates a command code for performing an operation set to correspond to the contact coordinate data (X, Y, Z) and inputs the command code to the main control unit 91 of the electronic apparatus. The main control unit 91 controls the display 90 to display the result of executing the command code. In FIG. 2, the 'music' icon menu is selected.

The user can select a specific song on the screen displayed on the display by sequentially selecting the submenu for the song name displayed after selecting the 'music' icon menu in FIG. 2 and the lower part thereof.

On the other hand, the touch position calculating section 30 calculates the touch position calculating section 30 based on the condition that the three-dimensional coordinate data X1, Y1, Z1 of the fingertip and the three-dimensional coordinate data X2, Y2, Z2 of the center point of one eye are constant (X, Y, Z) with the display display surface, and inputs the generated command code to the main control unit 91 of the electronic device.

That is, the touch position calculating section 30 determines whether or not there is a change in the contact point coordinate data for a set time or more from the time at which the initial contact point coordinate data is calculated. Only when the contact point coordinate data does not change for more than the set time, And inputs the generated command code to the main control unit 91 of the electronic device.

The virtual touch processing unit 40 determines whether or not there is a change in the contact point coordinate data over the set time from the time at which the initial contact point coordinate data is calculated. If there is no variation (X and Y coordinate value) of the contact point coordinate data over the set time , Determining whether or not a distance change has occurred at a distance equal to or greater than the set distance between the first and second spatial coordinates, and performing an operation that is set to correspond to the contact coordinate data when a distance change (change in Z coordinate value) And inputs the generated command code to the main control unit 91 of the electronic apparatus.

One user may have two contact point coordinates (Xa, Ya, Za) and (Xb, Yb, Zb) on the electronic device display surface using two fingertips. An example of a user having two contact points on the display surface and controlling the operation of an electronic device may be commonly found in games and the like. It will also be very useful when the user manipulates (move, rotate, zoom, zoom, etc.) the image on the display screen when using two fingertips.

Furthermore, two users may have two contact point coordinates (Xa, Ya, Za) and (Xb, Yb, Zb) on the electronic device display surface using one finger tip. An example in which two users have two contact points on the display surface and control the operation of electronic devices, respectively, will be common in games and the like.

FIG. 3 is a view showing that the periphery of the electronic device display screen is set as a virtual plane according to the present invention, and the selected area is divided into 8 equal parts. The user has selected the first area. In this manner, the virtual plane around the display can be divided into four, six, eight, and so on. Preferably, it is more effective for the user's memory or the like to divide into 8 equal parts. Of course, more compartments are possible for a variety of operations.

Further, by setting the periphery of the display surface of the electronic device to be a virtual plane, the selection area for operating the electronic device is widened, so that the operation of the electronic device can be speeded up.

Such an electronic device operating method of the present invention has the same operational feeling as the touch panel, enlargement of the operation area by using the periphery of the display screen, and technical effect of the operation screen light exposure on the display.

Second Example

4 is a block diagram showing a virtual touch device according to a second embodiment of the present invention.

Referring to FIG. 4, the virtual touch device 1 includes a three-dimensional coordinate calculation device 10 for extracting three-dimensional coordinate data of a user's body and a control unit 20.

The three-dimensional coordinate calculation device 10 can calculate three-dimensional coordinates of the user's body by various known three-dimensional coordinate extraction methods. Representative methods of such three-dimensional coordinate extraction can be divided into optical triangulation and time delay measurement. The active three-dimensional information acquisition technique using structured light, which is one of the optical triangulation techniques, projects a pattern image coded continuously using a projector and acquires an image of a scene where the structure light is projected through a camera, This is a method of estimating the position.

In addition, the time delay measurement method is a technique of acquiring 3D information by converting the time difference of the ultrasonic wave, which is reflected by the object, to the receiver, by dividing the time difference of the ultrasonic wave by the traveling speed of the ultrasonic wave. In addition, there are various methods of calculating three-dimensional coordinates using a time-of-flight measurement method, and those skilled in the art can readily implement the present invention, so that a detailed description thereof will be omitted.

The three-dimensional coordinate calculation apparatus 10 according to the present invention may be configured to include an illumination assembly 11, an image acquisition unit 12, and a spatial coordinate calculation unit 13. The illumination assembly 12 includes a light source 111 and a diffuser 112 and projects a speckle pattern onto the user's body. The image acquisition unit 12 includes an image sensor 121 and a lens 122 to capture a speckle pattern on the user projected by the illumination assembly. The image sensor 121 may be generally a CCD or CMOS based image sensor. In addition, the spatial coordinate calculation unit 13 processes the acquired image by the image acquisition unit 12 and calculates the three-dimensional coordinate data of the user's body.

The controller 20 includes a touch position calculator 21 and a virtual touch calculator 22.

At this time, the touch position calculating unit 21 calculates the touch position using the first spatial coordinate and the second spatial coordinate received from the three-dimensional coordinate calculating device 10, and determines that the straight line connecting the first spatial coordinate and the second spatial coordinate And calculates the contact point coordinate data that meets the display surface.

Generally speaking, the fingers are the only part of the human body that allows precise and delicate manipulation. Particularly, when using either the thumb or the index finger among the fingers or using the two fingers together, it is possible to perform precise pointing. Accordingly, it is very effective to use the tip of the thumb and / or index finger as the first spatial coordinate in the present invention.

Also, in the same context, it is possible to use a pointed pointer (e.g., a pen tip) holding a finger with the finger instead of the end of the finger serving as the first spatial coordinate. The advantage of using such a pointer is that the portion that covers the display from the user's view is smaller than the fingertip, and the pointed end of the pointer can accurately point to a very small portion of the display.

In the present invention, only the center point of one eye of the user is used. For example, if the user places his / her index finger in front of his / her eyes and looks at the index finger, the index finger will appear as two. This occurs because the shape of the index fingers viewed by the user is different from each other (due to the angle difference between the eyes). But if you look at your fingers only with one eye, your forefinger will look clearly. Even if you do not detect one eye, your forefinger will look conspicuous when you look at it with only one eye. It is the same principle to close and close one eye for sports events that require a high degree of accuracy in aiming, such as shooting and archery.

The present invention adopts the principle that the shape of the fingertip can be clearly grasped when the fingertip (first spatial coordinate) is viewed with only one eye. In this way, the user must be able to correctly view the first spatial coordinates, so that a specific area of the display coinciding with the first spatial coordinates can be pointed.

In the present invention, when one user uses one of the fingers, the first spatial coordinate is any three-dimensional coordinates of the end of one of the fingers of the user and the end of the pointer held by the user, The second spatial coordinate will be the three-dimensional coordinates of the center point of either eye of the user.

If one user uses two or more of the fingers, the first spatial coordinates are three-dimensional coordinates of the ends of one or more fingers presented by two or more users, and the second spatial coordinates are the second Dimensional coordinates of a center point of one eye of the users who are at least three-dimensional.

In addition, when two or more users are used, the first spatial coordinates are three-dimensional coordinates of the ends of one or more fingers presented by the user of two or more users, and the second spatial coordinates are coordinates of one of the two or more users Dimensional coordinates of the center point of the center point.

In the present invention, the virtual touch processing unit 22 determines whether or not there is a change in the contact point coordinate data over the set time from the time when the initial contact point coordinate data is calculated. If there is no change in the contact point coordinate data over the set time, And to input the generated command code to the main controller 31 of the electronic device. The virtual touch processing unit 22 can be processed in the same manner by two users of one user or two or more users.

In the present invention, the virtual touch processing unit 22 determines whether there is a change in the contact point coordinate data over a set time from the time at which the initial contact point coordinate data is calculated. If there is no change in the contact point coordinate data over the set time, And a controller for generating a command code for performing an operation set to correspond to the contact coordinate data when the distance change is greater than or equal to the set distance, And may be formed to be input to the control unit 31. The virtual touch processing unit 22 can be processed in the same manner by two users of one user or two or more users.

On the other hand, if it is determined that the variation of the contact coordinate data is within the setting range of the display 30, the contact coordinate data can be regarded as being not changed. That is, it is very difficult for the user to keep the contact coordinates as it is because the body or the finger naturally has slight movement or trembling when the user points the display to the end of the finger or the pointer. Accordingly, when the contact point coordinate data value is within the set range of the display, the instruction code is regarded as having no change in the contact point coordinate data, and the command code is generated and input to the main control unit 31 of the electronic device .

A representative example of the electronic apparatus to be subjected to the remote control according to the present invention is a digital television. Generally, the digital television receiver is provided with a broadcast signal receiving unit, a video signal processing unit, a system control unit, and the like. However, these components are obvious to those skilled in the art and will not be described in detail here. Other electronic devices to be subjected to the remote control according to the present invention include home networking such as home appliances as well as lighting devices, gas appliances, heating appliances, and method appliances, and various electronic appliances may be remotely controlled by the present invention .

The virtual touch device 1 according to the present invention may be inserted into the top of an electronic device frame to be remotely controlled and installed separately from the electronic device.

5 is a diagram illustrating a user selecting a particular icon on the electronic device display 30 in accordance with an embodiment of the present invention. The user selects the 'music' icon on the display 30 while viewing the fingertip with one eye. The three-dimensional coordinate calculation device 10 generates three-dimensional space coordinates of the user's body, and the touch position calculation unit 21 of the control unit 20 calculates three-dimensional coordinates of the fingertip based on three-dimensional coordinate data (X1, Y1, Z1) (X2, Y2, Z2) of the center point of the display screen (X, Y, Z) to calculate the contact coordinates (X, Y, Z) Then, the virtual touch processing unit 22 generates an instruction code for performing an operation set to correspond to the contact coordinate data (X, Y, Z) and inputs it to the main control unit 31 of the electronic apparatus. The main control unit 31 controls the display 30 to display the result of executing the command code. In FIG. 5, the 'music' icon menu is selected.

Now, the user sequentially selects a specific music from the sub menu and the music sub menu for the song name appearing after selecting the 'music' icon menu in FIG. 5.

Here, the touch position calculating section 21 of the control section 20 calculates the touch position calculating section 21 of the control section 20 based on the condition that the three-dimensional coordinate data (X1, Y1, Z1) of the fingertip and the three-dimensional coordinate data (X2, Y2, Z2) (X, Y, Z) corresponding to the contact point coordinates (X, Y, Z) with the display display surface only when it corresponds to the change of the Z coordinate value), and inputs the generated command code to the main control unit 31 of the electronic device .

The touch position calculating unit 21 determines whether or not there is a change in the contact point coordinate data from the time at which the initial contact point coordinate data is calculated. If the contact point coordinate data does not change over the set time, Generates an instruction code for performing an operation set to be corresponded, and inputs the generated instruction code to the main control unit 31 of the electronic apparatus.

Then, the virtual touch processing unit 22 judges whether or not there is a change in the contact point coordinate data over the set time from the time at which the initial contact point coordinate data is calculated. If there is no change in the contact point coordinate data (X and Y coordinate values) Determining whether or not a distance change has occurred more than a predetermined distance between the first spatial coordinate and the second spatial coordinate and, if a distance change (change in Z coordinate value) more than the set distance occurs, executing an operation set to correspond to the contact coordinate data And inputs the code to the main control unit 31 of the electronic device.

There are also cases where one user has two contact point coordinates (Xa, Ya, Za) and (Xb, Yb, Zb) on the display surface of the electronic device using two fingertips. An example of a user having two contact points on the display surface and controlling the operation of an electronic device may be commonly found in games and the like. It will also be very useful when the user manipulates (move, rotate, zoom, zoom, etc.) the image on the display screen when using two fingertips.

Furthermore, two users may have two contact point coordinates (Xa, Ya, Za) and (Xb, Yb, Zb) on the electronic device display surface using one finger tip. An example in which two users have two contact points on the display surface and control the operation of electronic devices, respectively, will be common in games and the like.

FIG. 6 is a view showing that an electronic device display display surface is set as a virtual plane according to the present invention, and the selected area is divided into 8 equal parts. The user has selected the first area. In this manner, the virtual plane around the display can be divided into four, six, eight, and so on. Preferably, it is more effective for storing the position of the user and the like. Of course, more compartments are possible for a variety of operations.

Further, by setting the periphery of the display surface of the electronic device to be a virtual plane, the selection area for operating the electronic device is widened, so that the operation of the electronic device can be speeded up.

Such an electronic device operating method of the present invention has the same operational feeling as the touch panel, enlargement of the operation area by using the periphery of the display screen, and technical effect of the operation screen light exposure on the display.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. It will be apparent to those skilled in the art that various modifications may be made without departing from the scope of the present invention. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

Claims (4)

Dimensional coordinate data of the user's body using the image received from the image acquiring unit; and an image acquiring unit configured to acquire three-dimensional coordinate data of the user's body using two or more image sensors disposed at different positions, A straight line connecting the first spatial coordinate and the second spatial coordinate is obtained by using a first spatial coordinate and a second spatial coordinate received from the spatial coordinate calculator, A touch position calculating unit for calculating contact point coordinate data to be contacted and a command code for performing an operation set to correspond to the contact point coordinate data received from the touch position calculating unit and inputting the command code to the main control unit of the electronic apparatus Through the virtual touch device being configured,
(A) processing three-dimensional coordinate data (X1, Y1, Z1) of a fingertip and three-dimensional coordinate data (X2, Y2, Z2) of a center point of one eye to generate a virtual plane around the display screen, Detecting respective contacts of one eye,
(B) calculating at least one of a depth change of the detected finger end point, a change of a locus, a holding time, and a change speed;
(C) The electronic device can be operated based on at least one of the depth change of the calculated finger end point, the change of the locus, the holding time and the speed of change, and the user can select the corresponding area Wherein a virtual plane around a display screen in a virtual touch device without using a pointer is used to control the electronic device. Dimensional coordinates of the user's body and a first spatial coordinate and a second spatial coordinate received from the three-dimensional coordinate calculation device, the first spatial coordinate and the second spatial coordinate being connected And a command code for performing an operation set to correspond to the contact point coordinate data received from the touch position calculating unit, and outputs the command code to the main control unit Through a virtual touch device composed of a control unit including a virtual touch processing unit for inputting a virtual touch signal,
(A) processing three-dimensional coordinate data (X1, Y1, Z1) of a fingertip and three-dimensional coordinate data (X2, Y2, Z2) of a center point of one eye to generate a virtual plane around the display screen, Detecting respective contacts of one eye,
(B) calculating at least one of a depth change of the detected finger end point, a change of a locus, a holding time, and a change speed;
(C) The electronic device can be operated based on at least one of the depth change of the calculated finger end point, the change of the locus, the holding time and the speed of change, and the user can select the corresponding area Wherein a virtual plane around a display screen in a virtual touch device without using a pointer is used to control the electronic device. 3. The method according to claim 1 or 2,
Wherein the virtual plane is divided into four selection areas of the top, bottom, left, and right sides of the electronic device display. / RTI > 3. The method according to claim 1 or 2,
Wherein the virtual plane is divided into eight selection areas on the upper, lower, left, and right sides of the electronic device display. / RTI >

Images