AU2021355988A1 - Remote control system - Google Patents
Remote control system Download PDFInfo
- Publication number
- AU2021355988A1 AU2021355988A1 AU2021355988A AU2021355988A AU2021355988A1 AU 2021355988 A1 AU2021355988 A1 AU 2021355988A1 AU 2021355988 A AU2021355988 A AU 2021355988A AU 2021355988 A AU2021355988 A AU 2021355988A AU 2021355988 A1 AU2021355988 A1 AU 2021355988A1
- Authority
- AU
- Australia
- Prior art keywords
- gesture
- unit
- input
- designated
- output
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 claims description 17
- 230000008569 process Effects 0.000 claims description 7
- 238000013507 mapping Methods 0.000 claims description 5
- 235000010627 Phaseolus vulgaris Nutrition 0.000 claims description 2
- 244000046052 Phaseolus vulgaris Species 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 8
- 238000012790 confirmation Methods 0.000 description 7
- 230000004043 responsiveness Effects 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 238000011161 development Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000007423 decrease Effects 0.000 description 2
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/04817—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/0482—Interaction with lists of selectable items, e.g. menus
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04842—Selection of displayed objects or displayed text elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04847—Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/0485—Scrolling or panning
- G06F3/04855—Interaction with scrollbars
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
-
- G—PHYSICS
- G08—SIGNALLING
- G08C—TRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS
- G08C17/00—Arrangements for transmitting signals characterised by the use of a wireless electrical link
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04Q—SELECTING
- H04Q9/00—Arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom
-
- G—PHYSICS
- G08—SIGNALLING
- G08C—TRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS
- G08C2201/00—Transmission systems of control signals via wireless link
- G08C2201/30—User interface
- G08C2201/32—Remote control based on movements, attitude of remote control device
Abstract
The present invention relates to a remote control system and, more specifically, to a remote control system capable of executing or controlling operations or functions by using gestures. The remote control system operated by gestures, according to an embodiment of the present invention, comprises: an input unit and an output unit; a storage unit in which gesture information to match an input gesture is stored; and a control unit electrically connected to at least one of the input unit, the output unit, and the storage unit to recognize the gesture and perform control, wherein the control unit recognizes a gesture input by the input unit, matches the recognized gesture to the gesture information stored in the storage unit, executes an operation corresponding to the matched gesture information, and outputs to the output unit.
Description
Field of the Invention
The present invention relates to a remote control system, and
more particularly, to a remote control system capable of
executing or controlling an operation or function through a
gesture.
Description of the Related Art Unless otherwise indicated herein, the material described in
this section is not prior art to the claims of this application,
and inclusion in this section is not an admission that it is
prior art.
With the recent development of technology, a great number of
electronic devices with various functions are emerging in
various fields that are difficult to enumerate such as smart
TVs, smartphones, AR and VR devices, interactive game consoles,
smart home appliances, smart home automation, and smartization
and electrification of automobiles. In addition, it provides
a very diverse and numerous functions.
However, the remote control method to conveniently operate
these functions from a distance has not developed significantly
compared to the number of various electronic device and their
functions are greatly increased.
The remote control method that is still most used is a method in which the remote controller has a button corresponding to each function, and when the button of the desired function is pressed, the electronic device receives the corresponding signal and the control unit recognizes it and executes the corresponding function.
The method of using this button-type remote controller provides
the best convenience when the remote controller has as many
buttons as the number of functions provided and the user easily
selects and presses the button of the corresponding function.
However, it was possible a long time ago when it had only a small
number of functions, but in the time of technological
development at the same speed as now, the method of using a
button-type remote controller has to be equipped with many
buttons, so it becomes difficult to carry as it becomes large
and heavy, in addition, it becomes difficult for the user to
find and execute a desired function from among the numerous
buttons, and convenience is lost.
Judging from the current situation, the remote controller of
the karaoke machine seems to be the remote controller with the
largest number of buttons that the user understands and can use
commercially.
So, in order to overcome this, what was introduced from the
method of using a button-type remote controller is to introduce
four-way buttons and an OK button, and to show the functions
to be executed on the screen, and to select and execute them.
For example, in the case ofa current smart TV, there is no button
that directly corresponds to selecting and running a specific
app from among multiple apps on the screen, playing or stopping
a specific video, or making a purchase on TV home shopping, Use
the button of to find the function displayed on the screen,
select it, and press the confirmation button to execute it.
For example, in the case of a current smart TV, when a user
selects and executes a specific app from among multiple apps
on the screen, plays or stops a specific video, or makes a
purchase in TV home shopping, there is no button directly
corresponding thereto. Therefore, it is to find and select the
function displayed on the screen with the buttons of the four
directions, and press the confirmation button to execute it.
However, the user's convenience is already lowered by
continuously pressing the four-way button a lot and finally
selecting the confirmation button. In addition, as more
functions are added in the future, the user will press the
four-way button and the confirmation button more and more to
find the function on the screen, and thus the user convenience
will be further lowered.
Also, recently, instead of the button-type remote controller,
a method such as voice recognition or a 3D space mouse has been
tried. However, in the case of voice recognition, the
recognition rate and quick responsiveness are not yet as desired,
and furthermore, it is annoying to the user to command and execute it by voice. And, in the case of the 3D space mouse, it has not been widely commercialized because it does not exhibit the responsiveness of selecting and executing quickly and accurately like a PC mouse.
Therefore, there is a need for a new remote control system with
increased user convenience with quick responsiveness as well
as convenient and easy remote control of many functions that
can be made through technological development in the future as
well as the present.
In this regard, Korean Patent Publication No. 10-2010-0131213
discloses a gesture-based remote control system. The
gesture-based remote control system includes a camera module
300, an image recognition module 100, a wireless transmitter
400, and a main control electronic device 200. The image
recognition module 100 is electrically connected to the camera
module 300. The wireless transmitter 400 is electrically
connected to the image recognition module 100. The main control
electronic device 200 is detachably connected to the image
recognition module 100. The main control electronic device 200
includes a monitor 202. The motion control command is obtained
by recognizing the user's image in the camera module 300 having
the image recognition module 100. The key control command is
obtained from the motion control command and key code
informationby the image recognitionmodule 100. The key control
command is transmitted to the wireless transmitter 400 by the image recognition module 100. The key control command is transmitted to a control target electronic device by the wireless transmitter to control the control target electronic device.
Prior Art
Patent Document
Korean Patent Publication No. 10-2010-0131213 (2010.12.15.)
An object of the present invention is to provide a new remote
control system with increased user convenience with quick
responsiveness while conveniently and easily executing
numerous functions of electronic devices from a distance using
gestures. In addition, the present invention is not limited to
the technical problems as described above, and it is obvious
that another technicalproblemmaybe derived fromthe following
description.
A remote control system operated by a gesture according to an
embodiment of the present invention, comprising: an input unit
and an output unit; a storage unit for storing gesture
information to be matched with an input gesture; and a control
unit electrically connected to at least one of the input unit,
the output unit, and the storage unit to recognize the gesture
and control the system, wherein the control unit recognizes a
gesture input by the input unit, matches the recognized gesture
with the gesture information stored in the storage unit, executes an operation corresponding to the matched gesture information, and outputs to the output unit.
According to a preferred feature of the present invention, the
control unit outputs information about gestures to be input by
the user to the output unit.
According to a preferred feature of the present invention, the
control unit designates each gesture graphic for
execution-purpose-graphic objects output through the output
unit, and stores the information on the designated graphic
objects and gesture graphics as gesture information to be
matched with the gesture input by the input unit to the storage
unit; wherein the graphic objects includes each menu item, an
app execution icon, a specific thing or person in a streaming
video, a character string, apart of a slide bar, and a specific
thing or person in a picture or photo output through the output
unit, and targets all objects output through the output unit,
and with respect to the gesture graphic, one gesture graphic
consists of only one or more gestures or a combination of one
or more numbers and one or more gestures.
According to a preferred feature of the present invention, the
control unit designates each gesture graphic for execution
functions that are not designated using the
execution-purpose-graphic objects output through the output
unit, and stores the information on the designated execution
functions and gesture graphics as gesture information to be matched with the gesture input by the input unit to the storage unit.
According to a preferred feature of the present invention, when
a specific gesture is input, the control unit outputs all of
the designated gesture graphics for the
execution-purpose-graphic objects on the output unit to the
output unit.
According to a preferred feature of the present invention,
wherein when a specific gesture is input, among the designated
gesture graphics for the execution functions that are not
designated using the graphic objects, the control unit outputs
a mapping list for currently available execution functions and
the designated gesture graphics therefor to the output unit.
According to a preferred feature of the present invention, the
control unit outputs all of the designated gesture graphics for
the execution-purpose-graphic objects on the output unit
together with the graphic objects to the output unit.
According to a preferred feature of the present invention, the
control unit installs, deletes, modifies, and updates the
gesture information on the execution-purpose-graphic objects
output through the output unit and the designated gesture
graphics therefor, and the gesture information on the execution
functions not designated using the graphic objects and the
designated gesture graphics therefor in real time through a
server and a network, and stores them in the storage unit.
According to a preferred feature of the present invention, the
control unit outputs a trajectory of each input gesture
operation to the output unit.
According to a preferred feature of the present invention, the
gesture information stored in the storage unit includes an app
gesture library that each app has and which is used to match
recognized gestures when an app function is executed, and a
system gesture library that the system has and which is used
tomatchrecognizedgestures when a systemfunctionis executed,
and wherein each of the gesture library includes a visible
gesture library unit which stores the information on the
execution-purpose-graphic objects output through the output
unit and the designated gesture graphics therefor as the gesture
information, and a hidden gesture library unit which stores the
information on the execution functions that are not designated
using the execution-purpose-graphic objects output through the
output unit and the designated gesture graphics therefor as the
gesture information.
According to a preferred feature of the present invention, when
an app is installed, the control unit registers the app
execution icon output through the output unit and designated
gesture graphic therefor to the systemgesture library, wherein
the designated gesture graphic for the app execution icon uses
one designated by the app producer, one recommended by the
control unit, or one desired by the user.
According to a preferred feature of the present invention, the
control unit uses selected app gesture library to match
recognized gestures when an app is currently selected among one
or more running apps, and uses the system gesture library to
match recognized gestures when none of the apps are running,
or none of the apps are selected even though one or more apps
are running.
According to a preferred feature of the present invention, the
control unit installs, deletes, modifies, updates the app
gesture library and the system gesture library in real time
through a server and a network, and stores them in the storage
unit.
According to a preferred feature of the present invention, when
a specific gesture is input to the input unit and the gesture
is determined to be a multiple gesture input signal, the control
unit outputs a signal indicating for the multiple gesture input
state anda signalwaiting for gesturesinput to the outputunit,
and then repeats a process of receiving input gestures one by
one and storing them in the storage unit continuously, wherein
when the input gesture is determined to bean termination signal
of the multiple gesture input, the control unit matches a
plurality of gestures stored so far in the storage unit with
the stored gesture information, and executes an operation.
According to a preferred feature of the present invention, when
a gesture graphic is designated with multiple gestures which has two or more times of repeated specific gesture, when the specific gesture is output to the output unit, the control unit repeatedly outputs as many gestures as the number of the gesture to the output unit, or outputs only one gesture and outputs the number of the repeated gesture as a subscript.
According to a preferred feature of the present invention,
comprising a transmitter located in the input unit and
transmitting On and Off signals to the control unit, wherein
the control unit outputs all of the designated gesture graphics
for the execution-purpose-graphic objects on the output unit
to the output unit when the transmitter of the input unit
transmits an On signal without a gesture, and maintains the
state in which the gesture graphics are output to the output
unit when the transmitter continuously maintains an On signal
without a gesture, and makes the output gesture graphics
disappear from the output unit when the transmitter transmits
an Off signal.
According to a preferred feature of the present invention,
comprising a transmitter located in the input unit and
transmitting On and Off signals to the control unit, wherein
when the transmitter of the input unit transmits an On and Off
signal twice within a predetermined time without a gesture, the
control unit outputs a signal indicating for the multiple
gesture input state and a signal waiting for gestures input to
the output unit, and then repeats a process of receiving input gestures one by one and storing them in the storage unit continuously, wherein when the transmitter of the input unit transmits an On and Off signal twice again within a predetermined time without a gesture, the control unit matches the plurality of gestures stored so far in the storage unit with the stored gesture information, and executes an operation.
According to a preferred feature of the present invention,
comprising a transmitter located in the input unit and
transmitting On and Off signals to the control unit, wherein
when a gesture graphic is designated with a Multi-On Off gesture
which has a specific gesture transmitted together with On and
Off signals two or more times within a predetermined time, the
controlunitoutputs the numberoftimes ofthe OnandOffsignals
input within a predetermined time in front of the specific
gesture as a number when output to the output unit.
According to a preferred feature of the present invention,
comprising a transmitter located in the input unit and
transmitting On and Off signals to the control unit, wherein
when a gesture is input to the input unit with On and Off signals
two or more times within a predetermined time, the control unit
matches the gesture and the number of times of the On and Off
signals input within a predetermined time with the stored
gesture information, and executes an operation.
According to an embodiment of the present invention, unlike in the conventional remote control system, in order to enable intuitive, easy and quick selection and execution of a desired function by a user, the present invention designates respective gestures to graphic objects such as app execution icons for executing a function on the screen, play buttons of a video player, and purchase button of a home shopping broadcast and the like. And, when the user takes a specific gesture, the gestures designated for the graphic objects as exemplified above are displayed on the screen. Thereby, the user can know the gesture designated to the graphic object on the screen to be executed. And, when the user takes one gesture among them, the function designated for the corresponding graphic object is executed. Through this, the present invention has the advantage ofbeing able to select and execute a desired function on the screen more intuitively and easily, and quickly than a
PC mouse.
In addition, since the installation, deletion, addition, update,
etc. of the gestures designated through the server and network
by the app producer, content provider, system producer, etc.
is possible in real time, the present invention has the
advantage that a new gesture can be designated to a new function
at any time, unlike the existing remote control system where
functions are fixed once manufactured and new functions cannot
be added.
In addition, the present invention has an advantage in that the user's convenience is not lost even with the minimum number of buttons because numerous gestures can be flexibly used in each situation when a transmitter is configured with buttons.
The effects ofthe presentinvention are not limited to the above
effects, and it should be understood to include all effects that
can be inferred from the configuration of the invention
described in the detailed description or claims of the present
invention.
FIG. 1 is a schematic configuration diagram of a remote control
system according to an embodiment of the present invention.
FIG. 2 is a schematic configuration diagram showing that the
input unit is composed of a transmitter and a receiver in the
remote control system according to an embodiment of the present
invention.
FIG. 3 is a diagram showing information about a gesture to be
input by a user is displayed on a screen in the remote control
system according to an embodiment of the present invention.
FIG. 4 is a diagram showing that gesture graphics are designated
and displayed on graphic objects displayed on a screen in the
remote control system according to an embodiment of the present
invention.
FIG. 5 is a mapping list of gesture graphics designated for
execution functions that are not designated using graphic
objects on the screen in the remote control system according to an embodiment of the present invention.
FIG. 6 is a schematic configuration diagram for installing,
deleting, modifying, and updating gesture information through
a server and a network in a remote control system according to
an embodiment of the present invention.
FIG. 7 is a schematic configuration diagram showing that the
app and the system have each gesture library, and each gesture
library is composed of a visible library and a hidden library
in a remote control system according to an embodiment of the
present invention.
FIG. 8 is a schematic configuration diagram showing the
situation in which each app gesture library and system gesture
library are installed, deleted, modified, and updated through
each server and network in a remote control system according
to an embodiment of the present invention.
Hereinafter, the configuration, operation, and effect of the
remote control system according to a preferred embodiment will
be described with reference to the accompanying drawings. For
reference, in the following drawings, each component is omitted
or schematically illustrated for convenience and clarity, and
the size of each component does not reflect the actual size.
In addition, the same reference numerals refer to the same
components throughout the specification, and reference
numerals for the same components in individual drawings will be omitted.
FIG. 1is a conceptual configuration diagram ofa remote control
system 100 operated by a gesture according to an embodiment of
the present invention.
The remote control system includes an input unit 110 and an
output unit 120; a storage unit 130 for storing gesture
information to be matched with an input gesture; and a control
unit 140 electrically connected to at least one of the input
unit 110, the output unit 120, and the storage unit 130 to
recognize the gesture and control the system, wherein the
control unit 140 recognizes a gesture input by the input unit
110, matches the recognized gesture with the gesture
information stored in the storage unit 130, executes an
operation corresponding to thematchedgestureinformation, and
outputs to the output unit 120.
The input unit 110 may be configured as a camera for receiving
a gesture operation from the camera-based remote control system
100 according to an embodiment of the present invention. Here,
when a person takes gestures using a hand gesture, the camera
captures the hand gesture and receives the gesture input.
In the remote controlsystem100 according to anotherembodiment,
as shown in FIG. 2, the input unit 110 may include a transmitter
111 for transmitting a gesture and a receiver 112 for receiving
a transmitted signal.
The output unit 120 is a device for outputting after executing an operation corresponding to the matched gesture information to be described later, and any configuration is possible as long as itincludes a device capable ofoutputting animage or avideo.
The storage unit 130 is a device for storing gesture information
to be matched with an input gesture.
And, the control unit 140 serves to control the overall process
which recognizes a gesture input to the input unit 110, matches
the recognized gesture with the gesture information stored in
the storage unit 130, executes the corresponding operation, and
outputs to the output unit 120.
According to a preferred feature of the present invention, the
control unit 140 outputs information about gestures to be input
by the user to the output unit 120.
The biggest problem with the existing gesture-based remote
control system is that the user must memorize all the gestures
to execute the desired functions. In the existing gesture-based
remote control system, as the number of functions increases,
the number of gestures to be remembered is inevitably increased.
Although the use of gestures is for the convenience of the user,
the convenience of the user decreases as the number of functions
increases. Therefore, the existing gesture-based remote
control system has been used only to the extent that a gesture
can be designated and used only for a small number of specific
functions in an auxiliary manner.
In response to this problem, the present invention proposes a solution by displaying information on the screen on gestures for executing desired functions without the user having to memorize the gestures.Referring to the example ofFIG. 3, there are many apps on the screen, and 1, 2, 3, 7, t, E, A, B, and
C are separately designated and displayed for each app. These
1, 2, 3, 7, t-, E, A, B, and C are examples of different
gestures designated to run each app.
The system of the present invention provides gestures for
executing a function desired by the user on the screen, and when
the user takes one of the gestures, the function corresponding
to the gesture is executed.
For example, in order to execute app2 310 in the figure of FIG.
3, if the user takes the gesture2 320, app2 310 is executed.
According to a preferred feature of the present invention, the
control unit 140 designates each gesture graphic for
execution-purpose-graphic objects output through the output
unit 120, and stores the information on the designated graphic
objects and gesture graphics as gesture information to be
matched with the gesture input by the input unit 110 to the
storage unit130; wherein the graphicobjects includes eachmenu
item, an app execution icon, a specific thing or person in a
streaming video, a character string, a part of a slide bar, and
a specific thing or person in a picture or photo output through
the output unit 120, and targets all objects output through the
output unit 120, and with respect to the gesture graphic, one gesture graphic consists of only one or more gestures or a combination of one or more numbers and one or more gestures.
As in the example of FIG. 4, numerous objects made of graphics
such as pictures, app execution icons, texts, slide bars,
directories, and people or things in a video are displayed on
the screen. And, execution functions corresponding to specific
graphic objects among these graphic objects are determined. In
the present invention, system producers, app producers, or
content providers may designate a desired gesture to each of
the graphic objects executing these functions.
In the FIG. 4, numbers, Korean letters, English alphabets,
symbols, or a combination thereof are displayed as examples of
respective gestures next to each graphic object. The user may
execute a function corresponding to the corresponding graphic
object by taking the corresponding gesture.
Another big problem in the existing remote control system is
that it is not easy to select and execute a desired function
from among numerous functions displayed on the screen as in the
example of FIG. 4. As mentioned above, the button-type remote
control system, which is the most used, is slow and inconvenient
for users because it is necessary to find and execute the desired
function using the 4-way button and the confirmation button to
respond to many functions. In addition, it is difficult to name
and call each of the numerous functions on the screen in the
first place in the voice recognition method, and the accuracy and responsiveness have not yet reached a satisfactory level, and it is cumbersome to speak. In addition, executing a function on the screen by moving the pointer on the screen in the same way as a 3D space mouse does not yet reach the user's desired responsiveness.
In this regard, in the present invention, as described above,
when a user intuitively sees a gesture designated to a graphic
object displayed on the screen and takes the gesture, the
function is immediately executed. This advantage is faster and
more convenient than the method of moving the mouse pointer to
the desired graphic object in the mouse system on the PC and
clicking the double button to execute it. Even if the user uses
it frequently, the gesture of a specific graphic object is
automatically remembered.
In this case, since the user executes the function by taking
the memorized gesture without having to look at the screen to
findthe gesture designatedto the graphicobject on the screen,
the desired function can be executed more quickly and
conveniently, and the user's convenience can be greatly
increased.
On the otherhand, here, asin the example ofFIG.4, the gestures
designated to each graphic object are not simply designated to
one gesture 410 and 420 per graphic object. Multiple gestures
430 and 450 may be designated as necessary. And, in the case
of having a transmitter that transmits On and Off signals to the control unit 140 to the input unit 110, which will be described later, a Multi-On Off gesture 440 in which a gesture and more than twice of On and Off signal within a predetermined time are input together may be designated.
If there are a lot of the execution-purpose-graphic objects on
the screen, a large number of gestures will be required
accordingly. As a result, there will be a limit to the number
of simple and easy gestures, so complex gestures will be
required a lot. However, the gesture-based remote control
system is intended to allow a user to easily and conveniently
execute a function using a gesture. Due to this complex gesture,
it may be difficult for the user to take a gesture or a situation
in which the gesture recognition rate of the control unit 140
decreases may happen.
In this regard, the present invention provides a combination
of a plurality of simple and easy gestures or a combination of
Multi-On Off signals and gesture by using the multiple gestures
and the Multi-On Off gesture as described above, so that many
functions can be covered with a small number of gestures.
And, as shown in FIG. 4, the gesture graphic for displaying a
designated gesture graphically on the screen is displayed as
only one or more gestures 410, 420 and 430 or a combination of
one or more numbers and one or more gestures 440 and 450, so
that the multiple gestures and the Multi-On Off gestures can
be displayed.
According to a preferred feature of the present invention, the
control unit 140 designates each gesture graphic for execution
functions that are not designated using the
execution-purpose-graphic objects output through the output
unit120, and stores the information on the designatedexecution
functions and gesture graphics as gesture information to be
matched with the gesture input by the input unit to the storage
unit 130.
As described above, it is possible to run an app, select a menu,
and move the slide bar up or down through the designated gesture
graphic in the graphic object displayed on the screen. However,
in order to operate the system using only the graphic objects
displayed on the screen and the designated gesture graphic for
all execution functions, it may be complicated by making more
graphic objects than necessary on the screen, and rather the
user's convenience may be reduced.
In this regard, the present invention designates each gesture
graphic for necessary execution functions, which are not using
the graphic objects on the screen, and stores information on
the designated execution functions and gesture graphics as
gesture information in the storage unit 130.
As in the example of FIG. 5, the present invention can directly
designate the gesture graphics 530 and 540 without a graphic
object for the function 510 for raising the soundor the function
520 for raising the TV channel.
According to a preferred feature of the present invention, when
a specific gesture is input, the control unit 140 outputs all
of the designated gesture graphics for the
execution-purpose-graphicobjects on the output unit120 to the
output unit 120.
The user cannot memorize all the gesture graphics that are
designated to all the execution-purpose-graphic objects output
on the screen. However, if the gesture graphics are displayed
continuously on the screen just because the user can't remember,
the screen will be full of the gesture graphics and the graphics
objects, so that it looks complicated. Eventually, the user's
concentration and convenience will be reduced.
In this regard, the present invention shows all the gesture
graphics designated for the execution-purpose-graphic objects
on the current screen onlywhen the usermakes a specificgesture.
That is, the user's convenience is enhanced by allowing the user
to see the gesture graphics designated for the graphic objects
on the screen only when the user wants them.
According to a preferred feature of the present invention, when
a specific gesture is input, among the designated gesture
graphics for the execution functions that are not designated
using the graphic objects, the control unit outputs a mapping
list for currently available execution functions and the
designated gesture graphics therefor to the output unit 120.
Even if the user searches for graphic objects for executing the function currently displayed on the screen, the desired function may not be found among them. For example, the sound controlfunctionis anecessary function, but for this function, the app producer may not want to put a graphic object on the screen and designate the gesture graphic to execute it. In this case, when the user takes a specific gesture, as in the example of FIG. 5, the control unit 140 displays on the screen a mapping list for all the currently available execution functions and the gesture graphics, among all the gesture graphics designated for the execution functions, which are not designated using the graphicobjects. For example, the usermay see the mappinglist, check the desired function and gesture, and then take the rightward gesture 530 to increase the sound.
According to a preferred feature of the present invention, the
control unit 140 outputs all of the designated gesture graphics
for the execution-purpose-graphic objects on the output unit
120 together with the graphic objects to the output unit 120.
As display technology develops, many electronic devices are
manufactured using a large screen as much as possible. In this
situation, even if all graphic objects are displayed on a
relatively large screen in a specific device, which doesn't have
many execution functions, there may be still abundant space
available on the screen. In this case, it does not look
complicated even if all of the graphic objects for executing
functions and the gesture graphics designated therein are displayed together on the screen. This function shows the gesture graphics on the screen even if the user does not take a gesture.
According to a preferred feature of the present invention, the
control unit 140 installs, deletes, modifies, and updates the
gesture information on the execution-purpose-graphic objects
output through the output unit 120 and the designated gesture
graphics therefor, and the gesture information on the execution
functions not designated using the graphic objects and the
designated gesture graphics therefor in real time through a
server and a network, and stores them in the storage unit 130.
One of the major drawbacks of the existing remote controlsystem
is that products that have already been produced cannot be
modified or upgraded. Therefore, when the existing remote
control system is produced and released at the factory, its
function must be used as it is. For example, in the case of a
button-type remote controller, since the button is already
physically fixed and produced, it is of course impossible to
modify it. However, in case of the smart TV, it can be used
continuously. The reason is, the smart TV displays execution
functions on the screen. And, the user can find and select a
desired function using the four direction buttons, and executes
the function by pressing the confirmation button, so the user
can continue to use it even if the functions are increased.
However, in this method, in order to find a desired function on the screen, the user's convenience is greatly reducedbecause the user has to press the confirmation button again even after finding the desired function by pressing the four direction buttons a lot.
In this regard, the present invention enables installation,
deletion, modification, andupdatingofthe gesture information
in real time through the server 150 and the network 160 as shown
in the example of FIG. 6, so that if a new function is added
to the system, the present invention designates a new gesture
graphic accordingly, and stores the information about it in the
storage unit 130.
According to a preferred feature of the present invention, the
control unit 140 outputs a trajectory of each input gesture
operation to the output unit 120.
When a user initially inputs a gesture to the input unit 110,
the user does not know which sensitivity level of speed or
magnitude of the gesture is required. In this regard, as a
preferred example, the control unit 140 may display the
trajectory display of the gesture operation input on the input
unit 110 to the output unit 120 as an overlay. The users can
see this trajectory display and adjust the gesture by estimating
the sensitivity of the gesture operation.
According to a preferred feature of the present invention, the
gesture information stored in the storage unit 130 includes an
app gesture library 710 that each app has and which is used to match recognized gestures when an app function is executed, and a system gesture library 720 that the system has and which is used to match recognized gestures when a system function is executed, and wherein each of the gesture library 710 and 720 includes a visible gesture library 711 and 721unit which stores the information on the execution-purpose-graphic objects output through the output unit 120 and the designated gesture graphics therefor as the gesture information, and a hidden gesture library unit 712 and 722 which stores the information on the execution functions that are not designated using the execution-purpose-graphic objects output through the output unit 120 and the designated gesture graphics therefor as the gesture information.
In order for app producers and system producers to store and
manage the designated gesture graphics as gesture information
in their own ecosystem, as shown in the example of FIG. 7, the
gesture information includes an app gesture library 710 that
each app has and a system gesture library 720 that the system
has. Each gesture library 710 and 720includes avisible gesture
library units 711 and 721 based on the graphic objects shown
on the screen and hidden gesture library units 712 and 722 based
on the execution functions that do not use the graphic objects
shown on the screen.
According to a preferred feature of the present invention, an
app is installed, the control unit 140 registers the app execution icon output through the output unit and designated gesture graphic therefor to the system gesture library 720, wherein the designated gesture graphic for the app execution icon uses one designated by the app producer, one recommended by the control unit 140, or one desired by the user.
The installed app itself is an app on the system, and when the
app is installed, the app execution icon displayed on the system
screen is only an execution-purpose-graphic object on the
systemscreen. Therefore, when the appisinstalled, the graphic
object and the gesture graphic accompanying it are registered
in the system gesture library 720. If the app producer is a
well-known brand producer or broadcaster, the designated
gesture graphic may want to use the app's own gestures. For
example, CNN, a famous broadcaster, can make a news app and make
its app's gesture graphics look similar to English CNN with its
own multiple gestures. However, in a specific case, multiple
gestures having a shape similar to the CNN may already be
registered in the user's system gesture library 720. In this
way, when a new gesture graphic is installed and designation
becomes impossible because there is a predefined gesture
graphic, the control unit 140 may recommend another gesture
graphic. In addition, the user can designate and use the desired
gesture graphic.
According to a preferred feature of the present invention, the
control unit 140 uses selected app gesture library 710 to match recognized gestures when an app is currently selected among one or more running apps, and uses the system gesture library 720 to match recognized gestures when none of the apps are running, or none of the apps are selected even though one or more apps are running.
As described above, the present invention includes an app
gesture library 710 produced, executed, and managed by each app
producer, and a system gesture library 720 produced, executed,
and managed by a system producer for gesture information
management. The present invention increases user convenience
by showing and selecting many gestures on the screen so that
functions can be executed quickly, easily, and conveniently.
But, the biggest problem that comes from using so many gestures
is when a gesture conflict occurs.
Each of the app gesture library 710 and the system gesture
library 720 will be managed when the occurrence of gesture
conflicts during the respective app producer and system
producer developingtheir own SWprograms. Ifa gesture conflict
occurs inside the respective gesture libraries 710 and 720, it
is discovered through the same steps as compiling during SW
development, and since this is a bug, it can be corrected.
However, if a gesture exists in the app gesture library 710 and
the system gesture library 720 at the same time, this may be
a problem. In addition, since convenient and easy gestures are
frequently used in any gesture library 710 and 720, the probability of occurrence of a gesture conflict willbe higher.
In order to solve this problem, the present invention divides
a case in which each app is executed and a case in which the
system is executed. When matching input gestures, the gestures
conflict is prevented by using its own gesture library 710 and
720 only when its own function is executed, whatever an app or
the system is executed.
According to a preferred feature of the present invention, the
control unit 140 installs, deletes, modifies, updates the app
gesture library 710 and the system gesture library 720 in real
time through a server 150 and a network 160, and stores them
in the storage unit 130.
Asin the example ofFIG.8, eachappproducerandsystemproducer
install, delete, modify, andupdate their gesture libraries 710
and 720 in real time using each server 150 and network 160,
respectively, and store them in the storage unit 130. In the
existing remote control system, when an app is executed, only
the functions within the range set by the system producer can
be used. However, according to the present invention, an app
producer can independently operate an app with their own
functions using their own gesture library 710.
For example, a home shopping company that is an app producer
may designate and send gesture graphics by updating the app
gesture library 710 in real time that there is a surprise special
discount on a product displayed on the screen during real-time broadcasting.
In this case, if the user simply takes a specific gesture as
described above while watching the real-time broadcast, then
all currently available gesture graphics are displayed on the
screen of the broadcast. If the user makes a gesture of the
designated gesture graphic for the product displayed on the
screen among the displayed gesture graphics, the information
provided by the home shopping company can be viewed.
In addition, when a news company conducts a real-time survey
during broadcastingin real time, the present invention updates
the app gesture library 710 in real time to display various
options that can be selected for real-time survey on the screen
and gesture graphics designated for the options. Therefore, the
user can select a desired option by selecting a gesture graphic
on the screen. In addition, even in a situation of exchanging
with user in real time, easy and convenient interactive
broadcasts with user is possible by using various gesture
graphics.
In addition to this, if the respective producers can install,
delete, modify, update the app and the system gesture library
710 and 720 in real time and store it in the storage unit 130,
the fields that can be applied by using it are limitless.
According to a preferred feature of the present invention, when
a specificgesture is input to the input unit110 and the gesture
is determined to be a multiple gesture input signal, the control unit 140 outputs a signal indicating for the multiple gesture input state and a signalwaiting for gestures input to the output unit 120, and then repeats a process of receiving input gestures one by one and storing them in the storage unit continuously, wherein when theinput gesture is determined tobean termination signal of the multiple gesture input, the control unit 140 matches a plurality of gestures stored so far in the storage unit 130 with the stored gesture information, and executes an operation.
As described above, unlike the previous gesture-based remote
control system, the present invention can receive not only a
single gesture, but also multiple gestures. When a specific
gesture graphicis composed ofmultiple gestures and designated
to a specific graphic object, the user must input multiple
gestures to the input unit 110 in order to execute a function
corresponding to the graphic object, and the control unit 140
must be able to receive and recognize multiple gestures, and
to determine mutual matching with stored gesture information.
In order to receive multiple gestures, the controlunit 140 does
not immediately execute a gesture upon receiving one gesture
as in a normal situation, but must wait for more incoming
gestures. When the user makes a specific gesture, the control
unit 140 can receive a multiple gesture inputs, display them
step by step on the screen, and wait until input of each of the
multiple gestures desired by the user is finished. When the user takes a specific gesture and notifies that there is no additional gesture input, the control unit 140 sums up all gestures stored so far to match the stored gesture information.
According to a preferred feature of the present invention, when
a gesture graphic is designated with multiple gestures which
has two or more times of repeated specific gesture, when the
specific gesture is output to the output unit 120, the control
unit repeatedly outputs as many gestures as the number of the
gesture to the output unit 120, or outputs only one gesture and
outputs the number of the repeated gesture as a subscript.
When the gesture graphic is displayed on the screen, if all
gestures are displayed one by one in order to display the
multiple gestures, the gesture graphic display becomes long and
may appear overlapping with a nearby graphic object. In this
regard, the present invention, as in the example 450 shown in
FIG. 4, when a specific gesture is repeated two or more times
and displayed on the screen, if the gesture is marked with a
subscript like the exponential notation in mathematics, the
gesture graphic display may be displayed shortly, and it may
be an interesting point.
According to a preferred feature of the present invention,
comprising: a transmitter located in the input unit 110 and
transmitting On and Off signals to the control unit 140, wherein
the control unit 140 outputs all of the designated gesture
graphics for the execution-purpose-graphic objects on the output unit 120 to the output unit when the transmitter of the input unit 110 transmits an On signal without a gesture, and maintains the state in which the gesture graphics are output to the output unit 120 when the transmitter continuously maintains an On signal without a gesture, and makes the output gesture graphics disappear from the output unit 120 when the transmitter transmits an Off signal.
Any device capable of generating On and Off signals, such as
a button, a switch, an optical type button, a magnetic field
type button, a touch button, a touch pad, a joystick, a touch
panel, and a touch screen can be used as the transmitter located
in the input unit 110 and transmitting On and Off signals to
the control unit 140.
As a preferred embodiment, as in the example of FIG. 2, the input
unit 110 may be configured of transmitter 111including a button
and receiver 112 for receiving the transmitted content. For
users who are bothered by gestures, when button is pressed once
(On) without a gesture, all gesture graphics designated to all
execution-purpose-graphic objects in the current screen
situation are displayed on the screen. And, while the button
is kept pressed (On), all the gesture graphics are continuously
displayed on the screen, and when the button is released (Off),
all the gesture graphics on the screen disappear.
From the user's point of view, the display function of the
graphic object on the screen and the designated gesture graphic for it will be a very frequently used function. Therefore, the user's convenience can be maximized by designating the single buttonpress, whichis the simplestoperation, to this function.
This will be more convenient and faster than voice recognition
using speech.
According to a preferred feature of the present invention,
comprising a transmitter located in the input unit 110 and
transmitting On and Off signals to the control unit 140, wherein
when the transmitter of the input unit 110 transmits an On and
Off signal twice within a predetermined time without a gesture,
the control unit 140 outputs a signal indicating for the
multiple gesture input state and a signal waiting for gestures
input to the output unit 120, and then repeats a process of
receiving input gestures one by one and storing them in the
storage unit 130 continuously, wherein when the transmitter of
the input unit 110 transmits an On and Off signal twice again
within a predetermined time without a gesture, the control unit
140 matches the plurality of gestures stored so far in the
storage unit 130 with the stored gesture information, and
executes an operation.
As a preferred embodiment, as in the example of FIG. 2, the input
unit 110 may be composed of transmitter 111 including a button
and receiver 112 for receiving the transmitted content. From
the user's point of view, gestures can also be cumbersome. The
start and end of the multiple gesture inputs are notified to the control unit 140 by double-clicking. In reality, if the multiple gesture designation is a lot to the gesture graphics, it will be a very convenient function for users.
According to a preferred feature of the present invention,
comprising a transmitter located in the input unit 110 and
transmitting On and Off signals to the control unit 140, wherein
when a gesture graphic is designated with a Multi-On Off gesture
which has a specific gesture transmitted together with On and
Off signals two or more times within a predetermined time, the
control unit 140 outputs the number of times of the On and Off
signals input within a predetermined time in front of the
specificgesture as a number when output to the output unit120.
As a preferred embodiment, as in the illustration of the present
invention in FIG. 4, when a Multi-On Off gesture using two or
more On and Off signals within a predetermined time is displayed
on the screen, the control unit 140 display (440) a number in
front of the gesture.
According to a preferred feature of the present invention,
comprising a transmitter located in the input unit 110 and
transmitting On and Off signals to the control unit 140, wherein
when a gesture is input to the input unit 110 with On and Off
signals two or more times within a predetermined time, the
control unit 140 matches the gesture and the number of times
of the On and Off signals input within a predetermined time with
the stored gesture information, and executes an operation.
When a specific gesture graphic is designated as a Multi-On Off
gesture, the user must input the Multi-On Off gesture to the
input unit 110. In this regard, the control unit 140 should be
able to receive and recognize the Multi-On Off gesture, and to
match mutually and determine with the stored gesture
information.The controlunitmatches the gesture and the number
of times of the On andOff signals input within a predetermined
time with the stored gesture information, and executes an
operation.
Although preferred embodiments of the present invention have
been described with reference to the accompanying drawings, the
embodiments described in the present specification and the
configurations shown in the drawings are only the most preferred
embodiments of the present invention and do not represent all
of the technical ideas of the present invention. Therefore, it
shouldbe understood that the presentinventionmayhave various
equivalents and modifications that can be substituted for them
at the time of filing the present application.
Therefore, the embodiments described above are to be understood
as illustrative and not restrictive in all respects, and the
scope of the present invention is indicated by the following
claims rather than the detailed description, and the meaning
and scope of the claims and their all changes or modifications
derived from the concept of equivalents should be construed as
being included in the scope of the present invention.
[Explanation of symbols]
100: remote control system
110: input unit
111: transmitter
112: receiver
120: output unit
130: storage unit
140: control unit
150: server
160: network
310: app execution icon
320: gesture of user
410, 420: single gesture
430: multiple gestures
440: Multi-On Off gesture
450: multiple gestures
510: execution function (volume up) not designated using
graphic object
520: execution function (channel up) not designated using
graphic object
530: gesture graphic designated to volume up
540: gesture graphic designated to channel up
710: app gesture library
711: visible gesture library of app gesture library
712: hidden gesture library of app gesture library
720: system gesture library
721: visible gesture library of system gesture library
722: hidden gesture library of system gesture library
Claims (19)
1. A remote control system operated by gestures, comprising:
an input unit and an output unit; a storage unit for storing
gesture information to be matched with an input gesture; and
a control unit electrically connected to at least one of the
input unit, the output unit, and the storage unit to recognize
the gesture and control the system, wherein the control unit
recognizes a gesture input by the input unit, matches the
recognized gesture with the gesture information stored in the
storage unit, executes an operation corresponding to the
matched gesture information, and outputs to the output unit.
2. The remote control system of claim 1, wherein the control
unit outputs information about gestures to be input by the user
to the output unit.
3. The remote control system of claim 2, wherein the control
unit designates each gesture graphic for
execution-purpose-graphic objects output through the output
unit, and stores the information on the designated graphic
objects and gesture graphics as gesture information to be
matched with the gesture input by the input unit to the storage
unit; wherein the graphic objects includes each menu item, an
app execution icon, a specific thing or person in a streaming
video, a character string, apart of a slide bar, and a specific
thing or person in a picture or photo output through the output
unit, and targets all objects output through the output unit, and with respect to the gesture graphic, one gesture graphic consists of only one or more gestures or a combination of one or more numbers and one or more gestures.
4. The remote control system of claim 3, wherein the control
unit designates each gesture graphic for execution functions
that are not designated using the execution-purpose-graphic
objects output through the output unit, and stores the
information on the designated execution functions and gesture
graphics as gesture information to be matched with the gesture
input by the input unit to the storage unit.
5. The remote control system of claim 3, wherein when a specific
gesture is input, the controlunit outputs allof the designated
gesture graphics for the execution-purpose-graphic objects on
the output unit to the output unit.
6. The remote control system of claim 4, wherein when a specific
gesture is input, among the designated gesture graphics for the
execution functions that are not designated using the graphic
objects, the control unit outputs a mapping list for currently
available execution functions and the designated gesture
graphics therefor to the output unit.
7. The remote control system of claim 3, wherein the control
unit outputs all of the designated gesture graphics for the
execution-purpose-graphic objects output on the output unit
together with the graphic objects to the output unit.
8. The remote controlsystemofclaims 3 or 4, wherein the control unit installs, deletes, modifies, and updates the gesture information on the execution-purpose-graphic objects output through the output unit and the designated gesture graphics therefor, and the gesture information on the execution functions not designated using the graphic objects and the designated gesture graphics therefor in real time through a server and a network, and stores them in the storage unit.
9. The remote controlsystemofclaims 3 or 4, wherein the control
unit outputs a trajectory of each input gesture operation to
the output unit.
10. The remote control system of claims 3 or 4, wherein the
gesture information stored in the storage unit includes an app
gesture library that each app has and which is used to match
recognized gestures when an app function is executed, and a
system gesture library that the system has and which is used
tomatchrecognizedgestures when a systemfunctionis executed,
and wherein each of the gesture library includes a visible
gesture library unit which stores the information on the
execution-purpose-graphic objects output through the output
unit and the designated gesture graphics therefor as the gesture
information, and a hidden gesture library unit which stores the
information on the execution functions that are not designated
using the execution-purpose-graphic objects output through the
output unit and the designated gesture graphics therefor as the
gesture information.
11. The remote control system of claim 10, wherein when an app
is installed, the control unit registers the app execution icon
output through the output unit and designated gesture graphic
therefor to the system gesture library, wherein the designated
gesture graphic for the app execution icon uses one designated
by the app producer, one recommended by the control unit, or
one desired by the user.
12. The remote control system of claim 10, wherein the control
unit uses selected app gesture library to match recognized
gestures when an app is currently selected among one or more
running apps, and uses the system gesture library to match
recognized gestures when none of the apps are running, or none
ofthe apps are selectedeven thoughone ormore apps are running.
13. The remote control system of claim 10, wherein the control
unit installs, deletes, modifies, updates the app gesture
library and the system gesture library in real time through a
server and a network, and stores them in the storage unit.
14. The remote control system of claims 3 or 4, wherein when
a specific gesture is input to the input unit and the gesture
is determined to be a multiple gesture input signal, the control
unit outputs a signal indicating for the multiple gesture input
state anda signalwaiting for gesturesinput to the outputunit,
and then repeats a process of receiving input gestures one by
one and storing them in the storage unit continuously, wherein
when the input gesture is determined to bean termination signal of the multiple gesture input, the control unit matches a plurality of gestures stored so far in the storage unit with the stored gesture information, and executes an operation.
15. The remote control system of claims 3 or 4, wherein when
a gesture graphic is designated with multiple gestures which
has two or more times of repeated specific gesture, when the
specific gesture is output to the output unit, the control unit
repeatedly outputs as many gestures as the number of the gesture
to the output unit, or outputs only one gesture and outputs the
number of the repeated gesture as a subscript.
16. The remote control system of claims 3 or 4, comprising a
transmitter located in the input unit and transmitting On and
Off signals to the control unit, wherein the control unit
outputs all of the designated gesture graphics for the
execution-purpose-graphic objects on the output unit to the
output unit when the transmitter of the input unit transmits
an On signal without a gesture, and maintains the state in which
the gesture graphics are output to the output unit when the
transmitter continuously maintains an On signal without a
gesture, and makes the output gesture graphics disappear from
the output unit when the transmitter transmits an Off signal.
17. The remote control system of claims 3 or 4, comprising a
transmitter located in the input unit and transmitting On and
Off signals to the control unit, wherein when the transmitter
of the input unit transmits an On and Off signal twice within apredetermined time without a gesture, the controlunit outputs a signal indicating for the multiple gesture input state and a signal waiting for gestures input to the output unit, and then repeats a process of receiving input gestures one by one and storing them in the storage unit continuously, wherein when the transmitter of the input unit transmits an On and Off signal twice again within a predetermined time without a gesture, the control unit matches the plurality of gestures stored so far in the storage unit with the stored gesture information, and executes an operation.
18. The remote control system of claims 3 or 4, comprising a
transmitter located in the input unit and transmitting On and
Off signals to the control unit, wherein when a gesture graphic
is designated with a Multi-On Off gesture which has a specific
gesture transmitted together with On and Off signals two or more
times within a predetermined time, the control unit outputs the
number of times of the On and Off signals input within a
predetermined time in front of the specific gesture as a number
when output to the output unit.
19. The remote control system of claims 3 or 4, comprising a
transmitter located in the input unit and transmitting On and
Off signals to the control unit, wherein when a gesture is input
to theinputunitwithOn andOffsignals twoormore times within
a predetermined time, the control unit matches the gesture and
the number of times of the On and Off signals input within a predetermined time with the stored gesture information, and executes an operation.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR1020200129696A KR102203951B1 (en) | 2020-10-07 | 2020-10-07 | Remote control system |
| KR10-2020-0129696 | 2020-10-07 | ||
| PCT/KR2021/013106 WO2022075644A1 (en) | 2020-10-07 | 2021-09-27 | Remote control system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| AU2021355988A1 true AU2021355988A1 (en) | 2023-02-09 |
Family
ID=74236764
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| AU2021355988A Pending AU2021355988A1 (en) | 2020-10-07 | 2021-09-27 | Remote control system |
Country Status (9)
| Country | Link |
|---|---|
| EP (1) | EP4227772A1 (en) |
| JP (1) | JP7457834B2 (en) |
| KR (1) | KR102203951B1 (en) |
| CN (1) | CN115398378A (en) |
| AU (1) | AU2021355988A1 (en) |
| CA (1) | CA3184971A1 (en) |
| MX (1) | MX2022016200A (en) |
| WO (1) | WO2022075644A1 (en) |
| ZA (1) | ZA202300203B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR102203951B1 (en) * | 2020-10-07 | 2021-01-18 | 김병수 | Remote control system |
Family Cites Families (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100790818B1 (en) * | 2006-12-06 | 2008-01-03 | 한국과학기술연구원 | Apparatus and method for controlling electronic appliances based on hand gesture recognition |
| KR20100131213A (en) | 2009-06-05 | 2010-12-15 | 탑시드 테크놀로지 코포레이션 | Gesture-based remote control system |
| JP5625506B2 (en) * | 2010-06-04 | 2014-11-19 | ソニー株式会社 | Operation terminal device, electronic device, and electronic device system |
| KR101720498B1 (en) * | 2010-10-29 | 2017-03-28 | 엘지전자 주식회사 | Mobile terminal and operation control method thereof |
| JP2014116768A (en) | 2012-12-10 | 2014-06-26 | Panasonic Corp | Operation device, electronic apparatus, and electronic apparatus system |
| KR20140085061A (en) | 2012-12-27 | 2014-07-07 | 삼성전자주식회사 | Display apparatus and Method for controlling display apparatus thereof |
| KR20150032101A (en) * | 2013-09-17 | 2015-03-25 | 삼성전자주식회사 | Apparatus and Method for Display Images |
| KR102145736B1 (en) * | 2013-11-27 | 2020-08-19 | 엘지전자 주식회사 | Digital device and method of processing a service thereof |
| KR20160089619A (en) * | 2015-01-20 | 2016-07-28 | 현대자동차주식회사 | Input apparatus and vehicle comprising the same |
| JP2016164725A (en) | 2015-03-06 | 2016-09-08 | 株式会社リコー | Information processing device, information processing system, and information processing program |
| KR20170014353A (en) * | 2015-07-29 | 2017-02-08 | 삼성전자주식회사 | Apparatus and method for screen navigation based on voice |
| KR101718070B1 (en) * | 2015-09-17 | 2017-03-20 | 주식회사 한컴플렉슬 | Touchscreen device for executing an event based on a combination of gestures and operating method thereof |
| KR102203951B1 (en) * | 2020-10-07 | 2021-01-18 | 김병수 | Remote control system |
-
2020
- 2020-10-07 KR KR1020200129696A patent/KR102203951B1/en active IP Right Grant
-
2021
- 2021-09-27 WO PCT/KR2021/013106 patent/WO2022075644A1/en active Application Filing
- 2021-09-27 CA CA3184971A patent/CA3184971A1/en active Pending
- 2021-09-27 MX MX2022016200A patent/MX2022016200A/en unknown
- 2021-09-27 AU AU2021355988A patent/AU2021355988A1/en active Pending
- 2021-09-27 EP EP21877896.7A patent/EP4227772A1/en active Pending
- 2021-09-27 CN CN202180028762.0A patent/CN115398378A/en active Pending
- 2021-09-27 JP JP2022562133A patent/JP7457834B2/en active Active
-
2023
- 2023-01-03 ZA ZA2023/00203A patent/ZA202300203B/en unknown
Also Published As
| Publication number | Publication date |
|---|---|
| MX2022016200A (en) | 2023-02-01 |
| CN115398378A (en) | 2022-11-25 |
| WO2022075644A1 (en) | 2022-04-14 |
| ZA202300203B (en) | 2023-10-25 |
| JP2023529529A (en) | 2023-07-11 |
| KR102203951B1 (en) | 2021-01-18 |
| EP4227772A1 (en) | 2023-08-16 |
| CA3184971A1 (en) | 2022-04-14 |
| JP7457834B2 (en) | 2024-03-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109618206B (en) | Method and display device for presenting user interface | |
| CN111698557B (en) | User interface display method and display equipment | |
| US9513802B2 (en) | Methods for displaying a user interface on a remote control device and a remote control device applying the same | |
| CN113805738B (en) | Custom setting method and starting method for control keys and display equipment | |
| CN112969084A (en) | User interface display method, storage medium and display device | |
| CN111625169B (en) | Method for browsing webpage by remote controller and display equipment | |
| CN112463269B (en) | User interface display method and display equipment | |
| CN112399212A (en) | Display device, file sharing method and server | |
| CN112380420A (en) | Searching method and display device | |
| US20040222971A1 (en) | Remote control device with button for notifying pointer movement mode | |
| EP4227772A1 (en) | Remote control system | |
| EP3247123A1 (en) | Image processing terminal for performing a different operation according to a force input and upgrade of software and method for upgrading the software | |
| CN111045557A (en) | Moving method of focus object and display device | |
| CN111586463B (en) | Display device | |
| CN112004126A (en) | Search result display method and display device | |
| CN113542899A (en) | Information display method, display device and server | |
| WO2021197078A1 (en) | Display method and display device | |
| CN112449227B (en) | Interaction method and device for touch screen application compatible with remote controller operation and smart television | |
| CN113542900B (en) | Media information display method and display equipment | |
| US20170237929A1 (en) | Remote controller for providing a force input in a media system and method for operating the same | |
| US8615722B2 (en) | Apparatus and method for providing preview of menu object | |
| CN113573115B (en) | Method for determining search characters and display device | |
| WO2021218111A1 (en) | Method for determining search character and display device | |
| JP2002033937A (en) | Apparatus and method for control of television, controller and recording medium | |
| JP2000188792A (en) | Method for sharing key and method for sharing the same function |