KR20130082031A - Web-browser pointing method using camera module of smart-phone and marker on finger, and computer-readable recording medium with web-browser pointing program using camera module of smart-phone and marker on finger - Google Patents

Web-browser pointing method using camera module of smart-phone and marker on finger, and computer-readable recording medium with web-browser pointing program using camera module of smart-phone and marker on finger Download PDF

Info

Publication number
KR20130082031A
KR20130082031A KR1020120003213A KR20120003213A KR20130082031A KR 20130082031 A KR20130082031 A KR 20130082031A KR 1020120003213 A KR1020120003213 A KR 1020120003213A KR 20120003213 A KR20120003213 A KR 20120003213A KR 20130082031 A KR20130082031 A KR 20130082031A
Authority
KR
South Korea
Prior art keywords
marker
camera
screen
finger
smartphone
Prior art date
Application number
KR1020120003213A
Other languages
Korean (ko)
Inventor
김병찬
Original Assignee
주식회사 인프라웨어
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 주식회사 인프라웨어 filed Critical 주식회사 인프라웨어
Priority to KR1020120003213A priority Critical patent/KR20130082031A/en
Publication of KR20130082031A publication Critical patent/KR20130082031A/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/0412—Digitisers structurally integrated in a display
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04812—Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Position Input By Displaying (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

PURPOSE: A web browser pointing method using the camera of a smartphone and the marker of a finger and a computer-readable recording medium recording a web browser pointing program using the camera and the marker are provided to perform a click operation on a user interface (UI) screen and allow the smartphone to offer an interface which is similar to the mouse of a wired terminal, thereby improving user convenience. CONSTITUTION: A marker is formed in the part of the body of a user and is located in the recognition range of the camera of a smartphone (S11). A mouse pointer, which is controlled by the marker, is implemented on a touch screen (S12). When the marker is moved in the recognition range, the mouse pointer is synchronized according to the movement (S13). A UI screen, which the mouse pointer is implemented, is outputted to the touch screen (S14). The mouse pointer moves according to the distance information and the directional information which are generated by measuring the moving distance and the moving direction of the marker. [Reference numerals] (AA) Start; (BB) End; (S11) Marker is formed in the part of the body of a user and is located in the recognition range of the camera of a smartphone; (S12) Mouse pointer, which is controlled by the marker, is implemented on a touch screen; (S13) When the marker is moved in the recognition range, the mouse pointer is synchronized according to the movement; (S14) UI screen, which the mouse pointer is implemented, is outputted to the touch screen

Description

Web-browser pointing method using a camera and finger marker of a smartphone, and a computer-readable recording medium recording a web browser pointing program using a marker of a smartphone and a finger of a smartphone phone and marker on finger, and computer-readable recording medium with web-browser pointing program using camera module of smart-phone and marker on finger}

The present invention relates to a web browser pointing technology using a camera of a smartphone and a marker of a finger, and more particularly, after attaching or coloring various types of markers on a user's finger, the camera module of the smartphone displays the marker in a captured image. By providing a configuration that recognizes and adjusts the pointer accordingly, click operations on the UI screen can be finely made and various gesture events can be provided, especially on a smartphone to provide an interface similar to using a mouse on a PC. The present invention relates to a web browser pointing method using a smart phone camera and a finger marker, and a computer readable recording medium recording a web browser pointing program using a smart phone camera and a finger marker.

The touch screen is a device that attaches an image output device such as LCD, FED, PDP or CRT to the screen to directly touch the screen with a hand or pen to perform a desired task. It is an ideal device that is easy to use in a device having a large size and can be intuitively performed in a graphical user interface (GUI), and is suitable for a portable device (portable cell phone, PDA, smart phone, etc.).

In this way, the touch screen contacts a person's hand or an object on a character or a specific location on the screen without using a keyboard or a mouse and recognizes a click according to whether the contact is made. Whether or not to accurately distinguish between clicks and un-clicks is essential.

On the other hand, most smartphones adopt a touch screen and use the touch screen in the smartphone environment. When there are many objects to be clicked like a web browser and the sizes of the objects are very small, they need to be enlarged or reduced. It is a hassle to do. In particular, since there is a thickness of the finger itself, there is a problem that frequently clicks the wrong position.

Accordingly, in the technical field, technology development for designating and selecting a precise object such as a mouse of a wired terminal in a small smartphone environment is required.

[Related Technical Literature]

1. Click event detection method and touch screen display device having the same (Patent Application No. 10-2004-0020454)

2. Touch control click structure (Patent Application No. 10-2008-0053493)

An object of the present invention is to provide a web browser pointing technology using the camera and the marker of the finger of the smartphone. More specifically, the present invention provides a configuration in which the camera module of the smartphone recognizes this marker in the captured image and adjusts the pointer accordingly after attaching or coloring various types of markers on the user's finger. A web browser pointing method using a smartphone's camera and a finger marker that can provide fine click manipulation and provide various gesture events, and in particular, provides a similar interface to using a mouse on a PC. An object of the present invention is to provide a computer-readable recording medium recording a web browser pointing program using a smartphone camera and a finger marker.

Web browser pointing method using the camera of the smartphone and the marker of the finger according to the present invention for achieving this problem, by forming a marker (M) on any body part of the user of the camera 12 mounted on the smartphone Recognizing the marker M according to the recognition range; Implementing a mouse pointer C to be controlled using the marker M on the touch screen 11; Synchronizing the mouse pointer C according to the movement when the user moves the marker M within the recognition range; And outputting the UI screen on which the mouse pointer C is moved according to the distance information and the direction information generated by measuring the moved distance and the direction of the marker M to the touch screen 11. .

In a web browser pointing method using a camera and a finger marker of a smartphone according to the present invention, after the step of outputting the UI screen on which the moved mouse pointer C is implemented as a touch screen, the mouse pointer C is touch screen. In the UI screen implemented on (11) located in the area including the target to be selected, if a double input by the user is input so that the UI screen corresponding to the target is implemented on the touch screen (11); It is configured to include. In this case, the double input is preferably recognized by the camera 12 that two touch inputs to the touch screen 11 or the marker M are moved twice in a predetermined direction.

A computer-readable recording medium that records a web browser pointing program using a camera and a finger marker of a smartphone according to the present invention for achieving the above object, forms a marker (M) on any body part of the user smartphone Marker-recognition means (14a) for recognizing the marker (M) as positioned in the recognition range of the camera (12) mounted on the; After implementing the mouse pointer C to be controlled using the marker M on the touch screen 11, if the user moves the marker M within the recognition range, the mouse pointer C is moved according to the movement. After synchronizing, the pointer-implementation method outputs a UI screen on which the mouse pointer C is implemented according to the distance information and the direction information generated by surveying the moved distance and direction of the marker M to the touch screen 11. Means (14b).

A computer-readable recording medium recording a web browser pointing program using a camera of a smartphone and a marker of a finger according to the present invention is a target for which a mouse pointer C is to be selected on a UI screen implemented on the touch screen 11. And a selection-processing means 14c which is located in an area including a second screen so that a UI screen corresponding to a target is implemented on the touch screen 11 when a double input is input by a user. In this case, the double input is preferably recognized by the camera 12 that two touch inputs to the touch screen 11 or the marker M are moved twice in a predetermined direction.

According to the web browser pointing technology using the camera and the marker of the present invention, a user can easily perform a click operation on a UI screen and increase user convenience by providing an interface similar to using a mouse in a wired terminal (PC) in a smartphone. In particular, it can be used more conveniently in selecting a specific object in a web browser.

In addition, according to the web browser pointing technology using the camera and the marker of the present invention, a click on an object according to the minute selection on the web browser is more accurate than the conventional method, and various gesture events (e.g., impossible due to panning in the conventional touch method) To draw a circle).

1 is a view showing the configuration of a smartphone implemented a web browser pointing method using a marker of the camera and the finger of the smartphone according to the present invention.
2 and 3 are diagrams for explaining a synchronization relationship between a marker and a mouse pointer in a web browser pointing method using a camera of a smartphone and a marker of a finger.
4 is a flowchart illustrating a web browser pointing method using a camera and a finger marker of a smartphone according to the present invention.

Hereinafter, the present invention will be described in detail with reference to the drawings.

1 is a diagram showing the configuration of a smartphone 10 in which a web browser pointing method using a camera and a finger marker of a smartphone according to an embodiment of the present invention is implemented. 2 and 3 are diagrams for explaining the synchronization relationship between the marker M and the mouse pointer C in a web browser pointing method using a camera of a smartphone and a marker of a finger. Meanwhile, the present invention will be described based on a web browser, but is not limited thereto. The present invention may be applied to various applications on the smartphone 10.

First, referring to FIG. 1, the smartphone 10 includes a touch screen 11, a camera 12, a controller 13, and a pointing module 14 so that a mouse pointer when a user uses a web browser. This improves the usability of the user's web browser by enabling similar operations.

The touch screen 11 is an input means in which an output unit is formed in addition to a touch module as an input unit. In another embodiment, the touch screen 11 may be formed separately from an input unit including a plurality of keys and an output unit such as a CRT, an LCD, an LED, a PDP, or the like.

Camera 12 is formed on the front or rear portion of the outside of the housing of the smart phone 10, has an angle of view (α) of the camera 12 shown in Figure 2 as a recognition range, panning (panning) In addition, the zoom operation function may be additionally performed.

The controller 13 generally controls the pointing module 14 formed on the touch screen 11, the camera 12, and the system memory, and performs a computing function of the mobile terminal 10 by interworking with each other.

The pointing module 14 comprises a marker-recognition means 14a, a pointer-implementation means 14b, and a selection-processing means 14c.

Marker-recognition means (14a) is a marker (M) formed on any body part of the user is located in the recognition range of the camera 12 mounted on the smart phone 10, the marker (M) through the camera 12 Upon recognition, the mouse pointer C, which is the object of control using the marker M on the touch screen 11, is implemented at a preset point on the touch screen 11. The preset point may be set as a center point on the touch screen 11 as a zero-point and may be changed according to a user's correction.

The marker M may be formed by being attached or painted in the shape of a dot of a specific color to a finger, the back of a hand, or another object of the user of the smartphone 10. Here, as shown in FIG. 2, the marker M is formed at a relatively large point on the back of the hand, but when formed on the finger, the marker M may be formed at a relatively small point.

When the user moves the marker M within the recognition range, the pointer-implementing means 14b synchronizes the mouse pointer C according to the movement, and then measures the distance and the direction of the movement to move the distance according to the distance information and the direction information. The touch screen 11 is controlled to output a UI screen on which the mouse pointer C is implemented.

That is, as shown in FIG. 3, when the user rotates the marker M in a semicircle having a predetermined curvature, the mouse pointer C is also synchronized with the same curvature and moved on the UI screen.

The selection-processing means 14c is located on the area including the object that the user desires to select on the UI screen implemented on the touch screen 11, and the double input by the user is inputted. A selection signal for a desired object is generated and transmitted to the control unit 13.

Accordingly, the controller 13 allows the UI screen corresponding to the selected object to be implemented on the touch screen 11.

Here, the user may select a button or banner advertisement. In addition, the double input by the user may be recognized by the camera 12 that two touch inputs to the touch screen 11 or the marker M is moved twice in a predetermined direction.

4 is a flowchart illustrating a web browser pointing method using a camera of a smartphone and a marker of a finger according to an embodiment of the present invention.

1 to 4, after forming a marker (M) on any body part of the user, it is placed in the recognition range of the camera 12 mounted on the smart phone (10) (S11) .

Herein, the recognition range means that the camera is positioned within the range of the angle of view α of the camera 12 shown in FIG. 2.

After the step S11, the pointing module 14 implements the mouse pointer C to be controlled by the marker M on the touch screen 11 (S12).

After the step S12, the pointing module 14 synchronizes the mouse pointer C according to the movement when the user moves the marker M within the recognition range, and then measures the distance and the direction of the movement so that the distance information and the direction are measured. The UI screen on which the mouse pointer C moved according to the information is implemented is output to the touch screen 11 (S13).

After the step S13, the pointing module 14 is positioned in an area including a target to which a mouse pointer C is to be selected on the UI screen implemented on the touch screen 11 so that a double input by a user is input. By generating a selection signal for the target and transmitting it to the control unit 13, the UI screen corresponding to the selected target is implemented on the touch screen 11 (S14).

The present invention can also be embodied in the form of computer readable code on a computer readable recording medium. At this time, the computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored.

Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disks, optical data storage devices, and the like, including those implemented in the form of carrier waves (eg, transmission over the Internet). . The computer-readable recording medium can also be stored and executed by a computer-readable code in a distributed manner on a networked computer system. And functional programs, codes, and code segments for implementing the present invention can be easily deduced by programmers skilled in the art to which the present invention belongs.

As described above, the embodiments of the present invention have been disclosed in the present specification and drawings, and although specific terms have been used, they have been used only in a general sense to easily describe the technical contents of the present invention and to facilitate understanding of the invention. And is not intended to limit the scope of the invention. It will be apparent to those skilled in the art that other modifications based on the technical idea of the present invention are possible in addition to the embodiments disclosed herein.

10: smartphone
11: Touch screen
12: camera
13:
14: pointing module
14a: marker-recognition means
14b: pointer-implementation
14c: selection-processing means

Claims (6)

A first step of recognizing the marker (M) by forming a marker (M) on any body part of the user and placing it in the recognition range of the camera (12) mounted on the smartphone;
A second step of implementing a mouse pointer (C) to be controlled using the marker (M) on a touch screen (11);
A third step of synchronizing the mouse pointer C as the user moves the marker M within the recognition range;
A fourth step of outputting, to the touch screen, a UI screen on which the mouse pointer C is moved according to the distance information and the direction information generated by measuring the moved distance and the direction of the marker M;
Web browser pointing method using a marker of the camera and the finger of the smartphone is configured to include.
The method according to claim 1,
When the mouse pointer C is located in an area including a target to be selected on the UI screen implemented on the touch screen 11 and a double input is input by the user, the UI screen corresponding to the target is touched. A fifth step of being implemented on the screen 11;
Web browser pointing method using a marker of the camera and the finger of the smartphone is configured to include more.
The method according to claim 2,
The second input of the fifth step is smart, characterized in that the camera 12 recognizes that two touch inputs to the touch screen 11 or the marker M are moved twice in a predetermined direction. A web browser pointing method using a camera of a phone and a marker of a finger.
Marker-recognition means (14a) for recognizing the marker (M) as the marker (M) formed on the body of the user is located in the recognition range of the camera 12 mounted on the smartphone;
After implementing the mouse pointer (C) to be controlled by the marker (M) on the touch screen 11, if the user moves the marker (M) within the recognition range according to the movement of the mouse pointer After synchronizing (C), the touch screen 11 displays a UI screen on which the mouse pointer C is implemented according to distance information and direction information generated by surveying the distance and direction of the marker M. FIG. Pointer-implementing means (14b) for outputting;
A computer-readable recording medium having recorded a web browser pointing program using a camera of a smartphone and a marker of a finger.
The method of claim 4,
When the mouse pointer C is located in an area including a target to be selected on the UI screen implemented on the touch screen 11 and a double input is input by the user, the UI screen corresponding to the target is touched. Selection-processing means 14c to be implemented on screen 11;
A computer-readable recording medium recording a web browser pointing program using a camera of a smartphone and a marker of a finger further comprising.
The method according to claim 5,
The double input is a camera of a smartphone, characterized in that the camera 12 recognizes that two touch inputs to the touch screen 11 or the marker M is moved twice in a predetermined direction. A computer-readable recording medium recording a web browser pointing program using a finger marker.
KR1020120003213A 2012-01-10 2012-01-10 Web-browser pointing method using camera module of smart-phone and marker on finger, and computer-readable recording medium with web-browser pointing program using camera module of smart-phone and marker on finger KR20130082031A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020120003213A KR20130082031A (en) 2012-01-10 2012-01-10 Web-browser pointing method using camera module of smart-phone and marker on finger, and computer-readable recording medium with web-browser pointing program using camera module of smart-phone and marker on finger

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020120003213A KR20130082031A (en) 2012-01-10 2012-01-10 Web-browser pointing method using camera module of smart-phone and marker on finger, and computer-readable recording medium with web-browser pointing program using camera module of smart-phone and marker on finger

Publications (1)

Publication Number Publication Date
KR20130082031A true KR20130082031A (en) 2013-07-18

Family

ID=48993507

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020120003213A KR20130082031A (en) 2012-01-10 2012-01-10 Web-browser pointing method using camera module of smart-phone and marker on finger, and computer-readable recording medium with web-browser pointing program using camera module of smart-phone and marker on finger

Country Status (1)

Country Link
KR (1) KR20130082031A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101465896B1 (en) * 2013-09-26 2014-11-26 성균관대학교산학협력단 Mobile terminal for generating control commands using front side camera and rear side camera

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101465896B1 (en) * 2013-09-26 2014-11-26 성균관대학교산학협력단 Mobile terminal for generating control commands using front side camera and rear side camera

Similar Documents

Publication Publication Date Title
KR102120930B1 (en) User input method of portable device and the portable device enabling the method
KR102097452B1 (en) Electro device comprising projector and method for controlling thereof
US20130215018A1 (en) Touch position locating method, text selecting method, device, and electronic equipment
US9459704B2 (en) Method and apparatus for providing one-handed user interface in mobile device having touch screen
US20140210756A1 (en) Mobile terminal and method for controlling haptic feedback
KR20140115731A (en) Method for converting object in portable terminal and device thereof
US20140059428A1 (en) Portable device and guide information provision method thereof
US8952906B2 (en) Apparatus and method for inputting writing information according to writing pattern
EP2770423A2 (en) Method and apparatus for operating object in user device
CN109582212B (en) User interface display method and device thereof
CN108027663B (en) Combining mobile devices with person tracking for large display interaction
JP5925957B2 (en) Electronic device and handwritten data processing method
KR102165448B1 (en) Overlapped transparent display and method for controlling the same
US9117125B2 (en) Electronic device and handwritten document processing method
US20120179963A1 (en) Multi-touch electronic device, graphic display interface thereof and object selection method of multi-touch display
KR20150020383A (en) Electronic Device And Method For Searching And Displaying Of The Same
US8948514B2 (en) Electronic device and method for processing handwritten document
US20150009136A1 (en) Operation input device and input operation processing method
KR20120061169A (en) Object control system using the mobile with touch screen
WO2014122794A1 (en) Electronic apparatus and handwritten-document processing method
CN106155614B (en) Operation information transmission method and electronic equipment
JP6411067B2 (en) Information processing apparatus and input method
Ruan et al. Touchinteract: An interaction technique with large displays using touchscreen-phone
US20150062038A1 (en) Electronic device, control method, and computer program product
KR20130082031A (en) Web-browser pointing method using camera module of smart-phone and marker on finger, and computer-readable recording medium with web-browser pointing program using camera module of smart-phone and marker on finger

Legal Events

Date Code Title Description
WITN Withdrawal due to no request for examination