KR101809892B1 - Smart device including computer and operating method thereof - Google Patents

Abstract

The present invention relates to a smart device including a computer and a method of operating the same, the input device comprising: a gel layer composed of a gel uniformly configured to a predetermined thickness; A surface portion surrounding the gel layer and composed of a resilient material; A pattern forming part formed below the gel layer surrounded by the surface part; And an ultra-small camera including an optical image sensor for photographing the pattern forming unit while being separated from the pattern forming unit by a predetermined distance. The input device of the present invention realizes a touch feeling of a user through a soft gel layer and an internal pattern, thereby providing a touch feeling of soft touch unlike the conventional input device, Shaped input device, which has the advantage that the usability is maximized.

Description

TECHNICAL FIELD [0001] The present invention relates to a smart device including a computer and a method of operating the smart device.

The present invention relates to a smart device including a computer and an operation method thereof, and in order to solve problems due to environmental constraints and touch feeling of a conventional computer input device, input position estimation and operation are performed through pattern change detection of an input device To a computer smart device and method of operation.

A mouse used as a conventional computer input device moves on a opaque and flat surface (for example, a mouse pad) as shown on a desk, and a cursor displayed on the screen moves accordingly. And the like. However, the conventional mouse has a restriction on the surrounding environment because it can be used on opaque surfaces where the bottom surface of the mouse is smooth and light is not transmitted.

Likewise, a touch pad used in a portable device such as a notebook computer or a tablet PC is also implemented using a hard material, which has a problem that touch feeling is reduced and a human body (e.g., a wrist) is burdened for a long time.

Accordingly, in order to improve the environmental constraints and the sense of touch of such a mouse or a touch pad, various input devices to replace them have been developed. For example, Korean Patent Laid-Open Publication No. 10-2011-0023654, "Pinker Mouse ", which is a prior art document, is implemented to detect the movement of the user's hand or finger on the keypad and perform the role of a mouse. However, this prior art uses a switch from the same space as the keypad to the function of the keypad or the mouse, so that it is difficult for the user to operate the device, and the touch is performed even when the user unconsciously moves the finger, .

Korean Patent Laid-Open No. 10-2011-0023654 (published on Mar. 8, 2011)

As described above, the present invention has been made to solve the problems caused by environmental constraints and touch sensation of existing computer input devices, and aims at providing space efficiency and increasing touch sensitivity.

According to a first aspect of the present invention, there is provided a computer input apparatus using an optical image sensor, comprising: a gel layer uniformly configured with a predetermined thickness and made of a gel; A surface portion surrounding the gel layer and composed of a resilient material; A pattern forming part formed below the gel layer surrounded by the surface part; And an ultra-small camera including an optical image sensor for photographing the pattern forming unit while being separated from the pattern forming unit by a predetermined distance.

According to a second aspect of the present invention, there is provided a smart device input method using an optical image sensor, comprising the steps of: capturing a pattern displayed on a pattern forming unit using an optical image sensor to collect photographed image data; Measuring an interval between unit elements constituting the pattern; Recognizing a position where a change in the interval is detected as a result of the measurement; And calculating an input position corresponding to the detected position, and executing an input operation.

The input device of the present invention realizes a touch of the user through a soft gel layer and an internal pattern, thereby providing a touch feeling of a soft touch unlike the conventional input device, Can be realized as an input device of the present invention, thereby maximizing usability. Further, the input device of the present invention has an advantage that a user can arbitrarily input an input without using a fixed input button, and can perform more various input functions than a conventional button-type input device.

Further, the input device of the present invention can analyze inputs and execute touch (input) based on the learning data, so that it is possible to implement a user-customized input device by utilizing learning data through pattern analysis of the user Is advantageous. The user can execute various inputs by using the operation familiar to him / her when using the customized input device (i.e., the learning input device), thereby maximizing the convenience of use.

Figure 1 illustrates an external configuration of a smart device, in accordance with an embodiment of the present invention.
Figure 2 illustrates the internal configuration of a smart device, in accordance with one embodiment of the present invention.
3 is a diagram for explaining a method of operating a smart device according to an embodiment of the present invention.
4 illustrates an exemplary operation of a smart device according to an embodiment of the present invention.
5 is a flowchart illustrating a method of operating a smart device according to an embodiment of the present invention.
6 is a flowchart illustrating an operation method of a smart device according to another embodiment of the present invention.
Like reference numbers in the several drawings indicate like elements.

The following detailed description of the invention refers to the accompanying drawings, which illustrate, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. It is also to be understood that the position or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is to be limited only by the appended claims, along with the full scope of equivalents to which such claims are entitled, if properly explained. In the drawings, like reference numerals refer to the same or similar functions throughout the several views.

Figure 1 illustrates an external configuration of a smart device 100, in accordance with an embodiment of the present invention. The input device 100 of the present invention includes a surface portion 110 made of a flexible and elastic material, a gel layer 120 disposed therein, a pattern forming portion 122 having a dot pattern formed thereon, .

The input device 100 of the present invention may be implemented in a square shape as shown, but it is not limited thereto and may be implemented in various shapes. When the input device 100 is implemented as an input device corresponding to an output screen of a personal computer (PC), it may be implemented in the same shape and ratio as a monitor (e.g., a 4: 3 aspect ratio of a rectangle). As described above, when the input device 100 is implemented by reducing the same form and ratio as the monitor, it is advantageous in that it is easy to measure and calculate the position recognized by the touch. As another example, it can be implemented in the form of a circle, square, or other design element.

As an embodiment, the input device 100 may be made flat, but may be formed entirely or partially in a bend or pattern (engraved or embossed). For example, the center of the input device 100 may include a predetermined shape (e.g., a concave or convex pattern that can be detected with a finger) to indicate a center point.

The surface portion 110 of the input device 100 is formed of a flexible material. For example, the surface portion 110 may be formed of a Lycra material. Lycra material is a kind of artificial elastic fiber, and it has an advantage of excellent resilience and restoring ability. Since the surface portion 110 of the present invention surrounds the gel layer 120 having a predetermined thickness and requires elasticity and flexibility when the user presses the gel layer 120 with his or her fingers and requires rapid restoration of the original surface, Is preferably used. However, the present invention is not limited thereto, and the surface portion 110 may be formed of various materials having flexibility and restoring force suitable for realizing the touch and input technology of the present invention.

The gel layer 120 is formed by using a gel material of a solid or semi-solid form so as to feel a soft touch when the user presses it, that is, when the pressure is applied, . For example, the thickness of the gel layer 120 may be comprised between 1 cm and 1.5 cm. The pattern forming portion 122 may be formed on the lower surface of the gel layer 120.

The pattern forming unit 122 is a component for recognizing a position according to a user's touch in the input device 110 of the present invention, and may have various patterns. The surface forming part 122 may be formed of the same material as that of the surface part 110 as the surface part 110 covers the entire gel layer 120. [ An example of the most basic pattern is to form circular dots into a plurality of rows and columns at regular intervals (e.g., 5 mm). The touch position can be read by sensing the change of the basic value in accordance with the touch of the user after storing the interval between the dots implemented as a plurality of rows and columns as the initially measured basic value.

And a spatial region 130 is formed in the lower portion so that an optical image sensor (or a miniature camera including the optical image sensor) for detecting the change of the pattern is installed.

Such an input device is an input device that can replace both a mouse and a mouse pad, which are conventional input devices, and is implemented in a form having a predetermined area such as a mouse pad, but a smooth touch feeling can be provided using a gel layer, There is an advantage that it is not restricted by the above.

2, the internal configuration of the smart device 100 will be described in more detail. The input device 200 may include a pattern forming portion 220 at the lower portion of the surface portion and the gel layer and may include the illumination device 240 and the micro camera 230 in the internal space 130.

As shown in the figure, the pattern forming unit 220 may be formed in a dot pattern pattern having a predetermined interval under the gel layer 210. But the present invention is not limited thereto and can be implemented in various patterns of various shapes. For example, the dotted pattern may be formed as a lattice pattern having a constant width w and width l as shown, but it is also possible to select other atypical patterns (for example, a Mickey mouse pattern) Do. Furthermore, pattern shapes such as dots may be represented by embossing or embossing, or may be represented by one or more hues in a plane.

As an embodiment, the input device 200 may further include a lighting device 240 in the inner space 130 in case the light from the outside is blocked. The input device 200 can be implemented such that the light is totally interrupted or partially interrupted within a range that does not affect the interval measurement of the pattern and the light device 240 having the light intensity optimized for the dot measurement, The accuracy of photographing can be improved. The illumination device 240 may be an LED illumination, for example.

The micro camera 230 is an element provided for measuring an interval between patterns, and includes an optical image sensor. As an embodiment, the micro camera 230 may be implemented to photograph the entire surface of the pattern forming unit 220 in real time or at regular intervals. As another example, the micro camera 230 may be used to photograph a predetermined area in a scanning manner. In addition, the micro camera 230 may be equipped with two or more cameras, and may be configured to photograph the divided areas to collect information.

In another embodiment, the present invention may further comprise a temperature sensor and / or a biometric information sensor. For example, the touch position can be sensed according to the body temperature movement of the user using the temperature sensor, and the touch position can be measured by sensing the user's bio-signal using the bio-information sensor.

As an embodiment, the input device 200 may further include a communication unit for communication with a computer (e.g., a PC), and a controller (not shown) for collecting measurement information. The controller can transmit the raw data of the collected information to the computer or calculate the touch position according to the measurement information and transmit the touch position and touch execution information to the computer.

Furthermore, the controller may include a macro function to implement an artificial intelligent input device. For example, it may be implemented to send a command to learn a frequently used input method (e.g., double-clicking with two fingers). The controller can learn the touch area, touch strength, drag distance, etc. frequently used by the user, and can further improve the reaction speed and accuracy by sending out the corresponding command. In one embodiment, the controller can implement a learning function through communication with the outside. The controller can upgrade the macro function by receiving the learning result through the big data analysis of the external server (for example, the service server).

The communication unit may implement wireless or wired communication and the input device 200 of the present invention may be implemented as a wireless (e.g., Bluetooth, WiFi communication) or wired (e.g., USB port) input device according to a communication method of the communication unit .

As described above, the input devices 100 and 200 devised by the present invention are realized to recognize the touch of the user through the gel layer and the pattern formation, thereby providing a touch feeling of soft touch unlike the conventional input device, It is possible to realize various types of input devices in various spaces without any limitation of the present invention, thereby maximizing usability.

3 is a diagram for explaining a method of operating a smart device according to an embodiment of the present invention. FIG. 3 (a) shows the internal configuration before touching the input device of the present invention, and FIG. 3 (b) shows changes in the internal configuration of the input device during the touch operation.

First, as described with reference to FIGS. 1 and 2, the input device includes a pattern forming portion 320 including a surface portion 310 surrounding the gel layer and having a dot pattern formed at a predetermined interval at the lower end of the gel layer . In this figure, a pattern is shown around a partial area A of the pattern forming unit 320, but the pattern may be formed over the entire area to which the touch sensing is applied. The input device also includes an illumination device 340 for capturing a pattern change when light is intercepted, and includes a miniature camera 330 for monitoring and capturing changes in the pattern.

The micro camera 330 includes an optical image sensor and can photograph the entire pattern forming unit 320 in real time or periodically. As another embodiment, the unit area may be specified, and one camera may be provided per unit area, so that one or more cameras may be installed. As another embodiment, a method of scanning while moving the camera photographing area may be adopted. As described above, the micro camera 330 photographs the pattern generated in the pattern forming unit 320 in real time or at a predetermined cycle, and detects a change in the pattern interval.

As shown in FIG. 3B, when the user touches (presses) a specific area, as the gel layer 350 is pressed, the dot spacing of the region A of the pattern forming portion 320 corresponding thereto . For example, the interval between the dots increases in the central portion of the pattern forming portion 350, and the interval between the dots located on the left and the right decreases. The micro camera 330 senses such a change in the dot-to-dot spacing and can transmit the shooting data to the controller.

Considering that the distance between the dots is the widest in the touch area (the pressed area), a point at which the interval between the dots is the maximum is detected as the center point, and a form in which the interval between the other dots is sensed is detected Location can be calculated. For example, it is possible to obtain an optimum touch position by centering a maximum interval between dots, taking a central point of a circular shape of the pattern, and correcting a previously calculated center point.

As an embodiment, a controller (not shown) may be included in the input device, and the controller can calculate the increased dot spacing using data received from the micro camera 330. The controller can transmit the input position by transmitting the calculated position to the computer using wired or wireless communication.

As another embodiment, the input device may include a communication unit in addition to the controller, and may be implemented to calculate the input position in the computer by transmitting it to the computer (PC) without calculating the input position from the data photographed by the micro camera 330 .

The method using the micro camera 330 described with reference to FIG. 3 is a method of detecting a position using the interval between dots. In this case, an optical image sensor built in the micro camera 330 is used. However, the present invention is not limited thereto, and the present invention can be implemented using a temperature sensor and / or a biometric information sensor. For example, when a temperature sensor is used, the temperature sensor can sense the touch position according to the body temperature movement of the user, and can be used in combination with the optical image sensor. The input device can sense the touch position by averaging the touch positions sensed by the two sensors and correct the touch position sensed by one sensor using the sensing results of the other sensors. Furthermore, the bio-information sensor may be used to detect the user's bio-signal and measure the touch position.

As described above, the present invention devises a new smart device which has not been existed by detecting that the gel layer is pressed according to the pressure applied by the user's finger touch and the dot pattern interval displayed on the gel layer is changed. The input device of the present invention has the advantage that there is no spatial restriction, unlike a mouse that only operates in a smooth and opaque plane (e.g., a mouse pad). In addition, there is no restriction on the appearance, so it is easy to add design elements. The user is provided with a soft touch feeling and has an effect that it can be used for a long time without imposing on the human body.

FIG. 4 illustrates an operation example of a smart device according to an embodiment of the present invention. FIG. 4A illustrates a general touch, and FIG. 4B illustrates a multi-touch function. As described above, the pattern interval under the gel layer 310 is changed due to the pressure applied by the user's touch, and the optical image sensor can recognize the user's input by sensing the pattern change.

Since the input device of the present invention does not have a separate button, input can be performed using one finger or two or more fingers depending on the convenience of the user. As shown in FIG. 4A, when the user performs a touch input with one finger, the interval of the pattern 370 changes. For example, the spacing of the pressed central pattern may be widened and the spacing of the peripheral patterns may be narrowed. Further, when the user performs the drag input, the input direction can be determined along with the change of the pattern. At this time, the controller can sense the input size and direction from the pattern changes recognized through the image sensor, and can extract the data such as the motion vector 380. A motion vector (MV) is a vector (representing a direction and a magnitude) of a motion image, and can reduce the amount of data using coordinates. For example, the motion vector value can be calculated by calculating the difference between the coordinates of the starting point at which the pattern change is recognized and the coordinates of the end point at which the input ends.

As another embodiment, it is possible to calculate the motion vector value by multi-touch, as shown in Fig. 4 (b). First, a rule for discriminating between a single touch and a multi-touch to perform another command can be specified in advance. A multi-touch is an attempt to input data to an input device using two or more fingers, and can be recognized as multi-touch when input is performed at two or more positions at the same time. For example, when dragging is performed by multi-touch, the controller senses the first input position 360 of the finger and the second input position 390 of the other finger, detects the input start point and the input end point of the two positions The motion vector value can be obtained. The controller can execute an input command using the calculated motion vector values 365 and 395, respectively, or calculate an average value of the calculated motion vector values 365 and 395 to execute the input.

Also, as an embodiment, an input command can be executed using only the measured motion vector value. As an embodiment, it is also possible to determine the type of input using past learning data and to execute an input command. When such learning data is used, a user-customized input function can be realized.

As described above, the input device of the present invention has an advantage that a user can arbitrarily input an input without using a fixed input button, thereby performing more various input functions than the conventional button input device.

5 is a flowchart illustrating an outline of a basic operation method of a smart device according to an embodiment of the present invention. The present embodiment describes a method of recognizing a touch position by measuring the distance between dots formed in a pattern forming unit using a micro camera having an optical image sensor built therein.

First, at least one optical image sensor built in the input device captures the pattern forming unit and collects captured image data (step 410). At this time, the optical image sensor can take an image in real time, or can take an image every predetermined period (for example, in seconds) to obtain a captured image. As another example, it is possible to collect captured image data of a unit area photographed by a plurality of optical image sensors, and as another example, a single optical image sensor scans the entire captured image data It can also be collected.

When the captured image data is collected, the interval between the dots displayed on the collected image is measured (operation 420). When there is no change in the interval between the dots or within the error range of the default value of the dot interval, the touch (input) is not recognized as an execution state and no additional process such as data transfer is performed.

If a change in the distance between the dots is sensed (step 430), the optimum touch position is recognized (calculated) by performing steps such as the position where the dot interval change is the maximum, and the dot interval variation around the maximum position is detected step). It is also possible to determine the input direction according to the change in the dot interval. As described above, the process of measuring the dot spacing change from the captured image data and estimating the position may be performed in a controller built in the input device, or may be performed through a processor of the main computer.

Next, after the touch position is calculated, the input is executed at the corresponding position on the display (operation 450). For example, a cursor on the monitor is executed by clicking a specific position.

6 is a flowchart illustrating an operation method of a smart device according to another embodiment of the present invention. Compared with FIG. 5, the embodiment of FIG. 6 implements a method of operating a customized input device through learning using an artificial intelligence controller.

As in the embodiment of FIG. 5, the optical image sensor collects the sensed image data through imaging, and the controller can measure the dot spacing (steps 510, 520). If there is a change in the interval of the captured image data collected at this time, the interval variation pattern can be analyzed and learned (steps 530 and 540).

When measuring the change in the interval of the captured image data, the degree of change in the interval can be collected to calculate the touch position and strength of the user, and the touch pattern such as the pressing strength and the pressing interval of the user can be analyzed have. The information of the touch pattern means the usage tendency and the characteristic of the user and includes a number of fingers, a certain degree of strength, and a difference in strength between dragging and clicking. For example, it is possible to analyze a pattern in which a user performs a touch with one finger, a click intensity is 5, and a drag has a strength of about 3, and the press interval time at the time of double- , Can be used as a criterion to distinguish between double click and one click.

This analysis data is formed of learning data and can be stored as customized data according to the user (step 550). As an embodiment, when two or more users of a computer are used, it is possible to provide an option to select an actual user from the user list when using a computer, execute the stored learning data for the selected user, and to perform comparison in the operation of the input device.

As an embodiment, the learning data may include setting information of the user. That is, the user can specify the input device usage pattern in advance. For example, the user can arbitrarily designate the input method and the execution operation as learning data, such as executing to move to the lowermost position when two fingers are dragged and to perform window closing when touching for five seconds or more.

The input device can analyze the input to be executed and perform touch (input) based on the learning data (step 560). The present invention has an advantage in that it is possible to implement a user-customized input device by utilizing learning data through pattern analysis of a user. The user can perform the input using the operation that is familiar to him / her when using the customized input device (i.e., the learning input device), thereby maximizing the convenience of use.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments.

In addition, although specific terms are used herein, they are used for the purpose of describing the present invention only and are not used to limit the scope of the present invention described in the claims or the claims. Therefore, the scope of the present invention should not be limited to the above-described embodiments, but should be determined by the equivalents of the claims of the present invention as well as the claims of the following.

100, 200: input device
110: surface portion
120: Gel layer
122 and 220:
130: inner space
240: Lighting device
230: Ultra-compact camera

Claims (17)

A smart device comprising a computer,
A gel layer composed of gels uniformly formed to a predetermined thickness;
A surface portion surrounding the gel layer and composed of a resilient material;
A pattern forming part formed below the gel layer surrounded by the surface part;
An ultra-small camera including an optical image sensor for photographing the pattern formation portion while being spaced apart from the pattern formation portion by a predetermined distance; And
A temperature sensor for recognizing an input position of the user through body temperature sensing or a biometric information sensor for sensing other biometric information,
And corrects the input position and the input operation using the detection result of the temperature sensor or the biometric information detection sensor. The method according to claim 1,
Wherein the pattern formed in the pattern forming section is a set of one or more unit elements represented by one or more rows and columns. 3. The method of claim 2,
Wherein the micro camera measures an interval of the one or more unit elements constituting the pattern formation unit. 3. The method of claim 2,
The unit elements are displayed in a dot shape,
Wherein a spacing between the unit elements is constant. The method of claim 3,
Wherein the optical image sensor of the micro camera collects photographed image data for measuring an interval of the unit elements of the photographed pattern forming unit and transmits the photographed image data to the controller. 6. The method of claim 5,
Wherein the controller senses a change in the interval of the unit elements from the captured image data and calculates an input position corresponding to the change. The method according to claim 6,
Wherein the controller calculates an input position and calculates a motion vector value including a magnitude and direction corresponding to the input. The method according to claim 1,
And a lighting device having a brightness for photographing the pattern forming portion. delete As a smart device using an optical image sensor,
A gel layer composed of gels uniformly formed to a predetermined thickness;
A surface portion surrounding the gel layer and composed of a resilient material;
A pattern forming part formed below the gel layer surrounded by the surface part; And
And a micro camera including an optical image sensor for photographing the pattern formation section while being separated from the pattern formation section by a predetermined distance,
A learning data generating unit that generates learning data by analyzing a touch pattern and strength of a user by sensing a change in a pattern in the pattern forming unit from captured image data collected through the optical image sensor, ≪ / RTI > location and input operation of the smart device. 11. The method of claim 10,
Wherein the learning data is updated every predetermined period by collecting continuous input information either by itself or through external communication. CLAIMS 1. A method of operating a smart device including a computer,
Capturing photographed image data by photographing a pattern displayed on a pattern forming portion formed under the gel layer using an optical image sensor;
Measuring an interval between unit elements constituting the pattern;
Recognizing a position where a change in the interval is detected as a result of the measurement; And
Calculating an input position corresponding to the sensed position, and executing an input operation,
Wherein the smart device further comprises at least one sensor of a temperature sensor and a biometric information sensor,
Further comprising the step of calibrating the input location and the input operation using the sensing results of the at least one sensor. 13. The method of claim 12,
Wherein the pattern formed on the pattern forming unit is a set of one or more unit elements represented by one or more rows and columns. 13. The method of claim 12,
Wherein the step of recognizing the position recognizes a position where the variation of the interval is maximum as the input position. 13. The method of claim 12,
Further comprising the step of transmitting the photographed image data photographed by the optical image sensor to a controller,
Wherein said controller measures and senses a pattern change to execute said input operation. 13. The method of claim 12,
Generating training data by analyzing a touch pattern and intensity of a user when measuring an interval change from the shot image data;
Analyzing the learning data and the real-time shot image data to execute an input position and an input operation; And
Further comprising collecting photographed image data generated every predetermined period or in real time, and updating the learning data. delete

Images