KR20160129399A - System and method for controlling robot in cloud environment - Google Patents
System and method for controlling robot in cloud environment Download PDFInfo
- Publication number
- KR20160129399A KR20160129399A KR1020150061500A KR20150061500A KR20160129399A KR 20160129399 A KR20160129399 A KR 20160129399A KR 1020150061500 A KR1020150061500 A KR 1020150061500A KR 20150061500 A KR20150061500 A KR 20150061500A KR 20160129399 A KR20160129399 A KR 20160129399A
- Authority
- KR
- South Korea
- Prior art keywords
- conditional expression
- communication unit
- conditional
- robot
- terminal
- Prior art date
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
- B25J9/1689—Teleoperation
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
- B25J19/02—Sensing devices
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
- B25J9/1682—Dual arm manipulator; Coordination of several manipulators
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/04—Programme control other than numerical control, i.e. in sequence controllers or logic controllers
- G05B19/05—Programmable logic controllers, e.g. simulating logic interconnections of signals according to ladder diagrams or function charts
- G05B19/056—Programming the PLC
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
Abstract
The present invention relates to a robot control system. A robot control system according to an embodiment of the present invention is a robot control system including a cloud, a terminal, and a robot, wherein the cloud includes: a first DB storing a plurality of first conditional expressions that are conditions under which the robot operates; A second DB storing a plurality of condition variables used for a condition in which the robot operates; And a first communication unit, wherein the terminal comprises: a second communication unit for receiving the one or more first conditional expressions stored in the first DB and the one or more conditional variables stored in the second DB from the first communication unit; A display unit for displaying the one or more first conditional expressions and the one or more conditional variables received by the second communication unit; A first input means for allowing a user to input a second conditional expression that is a condition that the robot operates using the one or more first conditional expressions and the one or more conditional variables displayed on the display unit; And a third DB storing one or more code blocks capable of generating an operation execution signal of the robot according to the second conditional expression, wherein the display unit displays the one or more code blocks, Further comprising third input means for allowing a user to input an operation execution signal of the robot according to the second conditional expression using the one or more code blocks displayed on the display unit, An actuating part driven based on a signal; And a third communication unit that receives the second conditional expression and the operation execution signal from the second communication unit and transmits the second conditional expression to the first communication unit to request the second conditional expression to determine whether the second conditional expression is satisfied, Wherein the cloud further comprises a control unit for determining whether the second conditional expression received by the first communication unit is satisfied, wherein the first communication unit transmits a notification signal to the third communication unit when the second conditional expression is satisfied, When the third communication unit receives the notification signal, the actuating unit may be driven based on the operation execution signal according to the second conditional expression.
Description
The present invention relates to a robot control system, and more particularly, to a system capable of controlling a robot more efficiently through a cloud environment.
As a robot that moves the power source by power, for example, there are those that operate according to an operation program made up of program codes or operate according to a user's operation through remote control, etc., and a robot (hereinafter referred to as " It is common for the user to program the operation program.
On the other hand, in the case of a typical programming robot, when a user wishes to program a robot program to execute a desired robot operation, a program controller and a personal computer built in the robot are connected (usually, wired connection) It is common to program this desired robot operation.
Therefore, programming a conventional programming robot requires a programming environment equipped with a computer, and there is a limitation that a programming expert must be able to program a robot operation using a computer. The usability and applicability of the programming robot is deteriorated, that is, it is accompanied by the restriction on the use environment.
2. Description of the Related Art [0002] In recent years, education using a programming robot (for example, maze searching using an educational robot) has been widely spread. In the conventional method of programming a robot operation using a computer, It will result in a general failure to do so, resulting in the attainment of educational goals.
Particularly, in the conventional system, there is a problem that the operation of the programmed robot or the condition formula for operating the robot is not shared among the users, which is very inefficient in programming.
In order for the programming robot to operate, in order to determine the conditions for the operation (for example, whether there is an obstacle in front of the robot), the programming robot itself must perform a twist or a device connected to the robot in order to determine whether the condition is satisfied Therefore, there is a problem that the calculation apparatus is very burdensome.
Therefore, in this specification, the programmed conditional expression is shared by the cloud, and a system in which the satisfaction of the condition for the robot to operate is calculated by the cloud is proposed.
Disclosure of Invention Technical Problem [8] Accordingly, the present invention has been made keeping in mind the above problems occurring in the prior art, and it is an object of the present invention to provide a robot control programming system capable of easily and efficiently generating new condition expressions by receiving conditional expressions from clouds, There is a purpose.
In addition, the present invention has an object of allowing a conditional expression generated in a terminal to be stored in a cloud so that a conditional expression can be shared.
In addition, since distributed information is collected and processed in the cloud, unnecessary operations are reduced.
In addition, the cloud is able to judge the conditional expression faster by computing the conditional expression, thereby reducing the computational burden of the robot.
The object of the present invention is to make it easy to perform programming by operating an intuitive tool without directly inputting an execution code in the form of text when a program for robot control is produced.
The object of the present invention is to provide an apparatus and a method for efficiently searching for a conditional expression stored in a database of a cloud by using meta data.
It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory and are not intended to limit the invention to the precise form disclosed. It can be understood.
A robot control system according to an embodiment of the present invention for realizing the above-described problems is a robot control system including a cloud, a terminal, and a robot, wherein the cloud includes a plurality of first conditional expressions 1 DB; A second DB storing a plurality of condition variables used for a condition in which the robot operates; And a first communication unit, wherein the terminal comprises: a second communication unit for receiving the one or more first conditional expressions stored in the first DB and the one or more conditional variables stored in the second DB from the first communication unit; A display unit for displaying the one or more first conditional expressions and the one or more conditional variables received by the second communication unit; A first input means for allowing a user to input a second conditional expression that is a condition that the robot operates using the one or more first conditional expressions and the one or more conditional variables displayed on the display unit; And a third DB storing one or more code blocks capable of generating an operation execution signal of the robot according to the second conditional expression, wherein the display unit displays the one or more code blocks, Further comprising third input means for allowing a user to input an operation execution signal of the robot according to the second conditional expression using the one or more code blocks displayed on the display unit, An actuating part driven based on a signal; And a third communication unit that receives the second conditional expression and the operation execution signal from the second communication unit and transmits the second conditional expression to the first communication unit to request the second conditional expression to determine whether the second conditional expression is satisfied, Wherein the cloud further comprises a control unit for determining whether the second conditional expression received by the first communication unit is satisfied, wherein the first communication unit transmits a notification signal to the third communication unit when the second conditional expression is satisfied, When the third communication unit receives the notification signal, the actuating unit may be driven based on the operation execution signal according to the second conditional expression.
The robot further includes a sensor unit for measuring a measurement value, and the third communication unit transmits the measurement value to the first communication unit, and the control unit transmits the measured value to the second communication unit, It is possible to judge whether or not the conditional expression is satisfied.
In addition, the third communication unit may periodically transmit the measured values.
The first communication unit may receive the input second conditional expression through the second communication unit, and the first DB may store the second conditional expression received by the first communication unit as the first conditional expression.
In addition, each of the plurality of first conditional expressions may include first metadata that is attribute information of each of the plurality of first conditional expressions, and the second conditional expression may include second metadata that is attribute information of the second conditional expression have.
The terminal may further include second input means for inputting a search word related to the first meta data, wherein the first communication unit receives the search word from the second communication unit, And searching means for searching the first metadata related to a search word received by the first communication unit among the plurality of first metadata and deriving a first conditional expression of the searched first metadata, 2 communication unit receives the first conditional expression derived from the first communication unit, and the display unit can display the derived first conditional expression received by the second communication unit.
Each of the plurality of first conditional expressions has a first conditional expression name of each of the plurality of first conditional expressions and the second communication unit receives the plurality of first conditional expression names stored in the first DB from the first communication unit Wherein the display unit displays the plurality of first conditional expression names and the terminal further comprises fourth input means for allowing the user to select at least one of the plurality of first conditional expression names displayed on the display unit Wherein the first communication unit receives the first conditional expression name input by the user from the second communication unit and receives the first conditional expression name corresponding to the first conditional expression name input by the user stored in the first DB And transmit the conditional expression to the second communication unit.
In order to achieve the above object, a robot control system according to an embodiment of the present invention is a robot control system including a cloud, a plurality of terminals, and a plurality of robots respectively connected to the plurality of terminals, A first DB storing a plurality of first conditional expressions that are conditions under which the robot operates; A second DB storing a plurality of condition variables used for a condition in which the robot operates; And a second communication unit, wherein each of the plurality of terminals receives the one or more first conditional expressions stored in the first DB and the one or more conditional variables stored in the second DB from the first communication unit, A communication unit; A display unit for displaying the one or more first conditional expressions and the one or more conditional variables received by the second communication unit; A first input means for allowing a user to input a second conditional expression that is a condition that the robot operates using the one or more first conditional expressions and the one or more conditional variables displayed on the display unit; And a third DB storing one or more code blocks capable of generating an operation execution signal of the robot according to the second conditional expression, wherein the display unit displays the one or more code blocks, Further comprising third input means for allowing the user to input an operation execution signal of the robot according to the second conditional expression using the one or more code blocks displayed on the display unit, An actuating unit driven based on the operation execution signal; And a third communication unit for receiving the second conditional expression and the operation execution signal from the second communication unit of each connected terminal, wherein the third communication unit of the first robot of the plurality of robots, Further comprising a controller configured to transmit a second conditional expression to request the determination of whether the second conditional expression is satisfied and to determine whether the second conditional expression received by the first communication unit is satisfied, When the third communication unit of the first robot receives the notification signal, the actuating unit of the first robot transmits the notification signal to the second communication unit of the first robot when the second conditional expression is satisfied, And may be driven based on the operation execution signal according to the conditional expression.
Each of the plurality of robots further includes a sensor unit for measuring a measurement value, and the third communication unit transmits the measurement value to the first communication unit, and the controller uses the received measurement value It is possible to determine whether or not the second conditional expression is satisfied.
In addition, the third communication unit may periodically transmit the measured values.
In addition, the controller may request transmission of the measurement value to at least one robot among the plurality of robots in order to determine whether the second conditional expression is satisfied.
The first communication unit may receive the input second conditional expression through the second communication unit, and the first DB may store the second conditional expression received by the first communication unit as the first conditional expression.
The at least one first conditional expression received by the second communication unit of the first terminal among the plurality of terminals may include a second conditional expression transmitted by the second communication unit of the second terminal to the first communication unit.
In addition, each of the plurality of first conditional expressions may include first metadata that is attribute information of each of the plurality of first conditional expressions, and the second conditional expression may include second metadata that is attribute information of the second conditional expression have.
Each of the plurality of terminals further includes second input means for inputting a search word related to the first meta data, wherein the second input means of the first terminal among the plurality of terminals inputs the search term The first communication unit receives the search word from the second communication unit of the first terminal, and the cloud searches the first metadata related to the search word received by the first communication unit among the plurality of first metadata Wherein the first communication unit of the first terminal receives the first conditional expression derived from the first communication unit, and the second conditional expression of the first meta data is transmitted to the second communication unit of the first terminal, The display unit of the first terminal may display the derived first conditional expression received by the second communication unit of the first terminal.
Each of the plurality of first conditional expressions has a first conditional expression name of each of the plurality of first conditional expressions and the second communication unit receives the plurality of first conditional expression names stored in the first DB from the first communication unit Wherein the display unit displays the plurality of first conditional expression names and each of the plurality of terminals includes a fourth input means for selecting at least one of the plurality of first conditional expression names displayed on the display unit, Wherein the first communication unit receives the first conditional expression name input by the user from the second communication unit of the first terminal among the plurality of terminals and inputs the first conditional expression name input by the user stored in the first DB The first conditional expression corresponding to the first conditional expression name may be transmitted to the second communication unit of the first terminal.
The robot control method according to an embodiment of the present invention for realizing the above-mentioned problem includes a first DB storing a plurality of first conditional expressions, which are conditions under which the robot operates, and a plurality of conditional variables used in a condition that the robot operates A robot control method using a plurality of robots including a cloud including a stored second DB, a plurality of terminals, and an actuating unit connected to each of the plurality of terminals and driven based on an operation execution signal, At least one first terminal among the plurality of terminals receiving the one or more first conditional expressions and the one or more conditional variables; A second step of the first terminal displaying the received one or more first conditional expressions and the one or more conditional variables; And a third step of the first terminal receiving a second conditional expression that is a condition that the robot operates using the one or more first conditional expressions and the one or more conditional variables indicated by the first terminal, A third DB storing one or more code blocks capable of generating an operation execution signal of the robot according to a second conditional expression, and after the third step, the first terminal includes a fourth DB for displaying the one or more code blocks, step; And a fifth step of the first terminal receiving an operation execution signal of the robot according to the second conditional expression using the one or more code blocks displayed; A sixth step of the first robot connected to the first terminal receiving the second conditional expression and the operation execution signal from the first terminal; A seventh step of the first robot transmitting the second conditional expression to the cloud and requesting the second conditional expression to determine whether the second conditional expression is satisfied; An eighth step of determining whether the cloud satisfies the second conditional expression; (E) when the cloud satisfies the second conditional expression, the cloud transmits an alert signal to the first robot; And when the first robot receives the notification signal, the actuation unit of the first robot is driven based on the operation execution signal according to the second conditional expression.
Each of the plurality of robots further includes a sensor unit for measuring a measured value and transmits the measured value to the cloud, and the cloud uses the measured value to determine whether the second conditional expression is satisfied .
In addition, each of the plurality of robots may periodically transmit the measured values.
Further, after the seventh step, the cloud may request the plurality of robots to transmit the measured values.
Transmitting, by the first terminal, the input second conditional expression to the cloud after the third step; And storing the received second conditional expression in the first DB as the first conditional expression by the cloud.
The at least one first conditional expression received by the first terminal may include a second conditional expression transmitted by the second terminal to the cloud.
In addition, each of the plurality of first conditional expressions may include first metadata that is attribute information of each of the plurality of first conditional expressions, and the second conditional expression may include second metadata that is attribute information of the second conditional expression have.
The method may further include receiving a search term associated with the first metadata before the first step; The first terminal transmitting the search query to the cloud; And deriving a first conditional expression of the searched first metadata by searching the first meta data related to the received search term among the plurality of first meta data by the cloud, In the step and in the second step, the first conditional equation may be the derived first conditional formula.
Each of the plurality of first conditional expressions has a first conditional expression name of each of the plurality of first conditional expressions, and the first terminal receives the plurality of first conditional expression names before the first step. Displaying the first plurality of conditional expression names received by the first terminal; Selecting and inputting at least one of the displayed plurality of first conditional expression names in the first terminal; And transmitting, by the first terminal, the input at least one first conditional name to the cloud, wherein in the first step, the first conditional expression includes at least one first conditional name transmitted to the cloud, It may be the first conditional expression corresponding to the conditional expression name.
A recording medium related to an embodiment of the present invention for realizing the above-mentioned problems includes a first DB storing a plurality of first conditional expressions, which are conditions under which a robot operates, and a plurality of conditional variables used for conditions under which the robot operates A plurality of terminals, and a plurality of robots including a plurality of robots each having an actuating unit connected to the plurality of terminals and driven based on an operation execution signal, Wherein the robot control method further comprises a step of controlling at least one first terminal among the plurality of terminals from the cloud by the first terminal, A first step of receiving the one or more first conditional expressions and the one or more conditional variables; A second step of the first terminal displaying the received one or more first conditional expressions and the one or more conditional variables; And a third step of the first terminal receiving a second conditional expression that is a condition that the robot operates using the one or more first conditional expressions and the one or more conditional variables indicated by the first terminal, A third DB storing one or more code blocks capable of generating an operation execution signal of the robot according to a second conditional expression, and after the third step, the first terminal includes a fourth DB for displaying the one or more code blocks, step; And a fifth step of the first terminal receiving an operation execution signal of the robot according to the second conditional expression using the one or more code blocks displayed; A sixth step of the first robot connected to the first terminal receiving the second conditional expression and the operation execution signal from the first terminal; A seventh step of the first robot transmitting the second conditional expression to the cloud and requesting the second conditional expression to determine whether the second conditional expression is satisfied; An eighth step of determining whether the cloud satisfies the second conditional expression; (E) when the cloud satisfies the second conditional expression, the cloud transmits an alert signal to the first robot; And when the first robot receives the notification signal, the actuation unit of the first robot is driven based on the operation execution signal according to the second conditional expression.
Meanwhile, a computer program related to an embodiment of the present invention for realizing the above-mentioned problems stores a first DB storing a plurality of first conditional expressions, which are conditions under which a robot operates, and a plurality of conditional variables used for a condition in which the robot operates A robot including a first DB, a second DB, a plurality of terminals, and a plurality of robots each having an actuating unit connected to the plurality of terminals and driven based on an operation execution signal, Wherein the robot control method comprises: a first step in which at least one first terminal among the plurality of terminals from the cloud receives the one or more first conditional expressions and the one or more conditional variables; A second step of the first terminal displaying the received one or more first conditional expressions and the one or more conditional variables; And a third step of the first terminal receiving a second conditional expression that is a condition that the robot operates using the one or more first conditional expressions and the one or more conditional variables indicated by the first terminal, A third DB storing one or more code blocks capable of generating an operation execution signal of the robot according to a second conditional expression, and after the third step, the first terminal includes a fourth DB for displaying the one or more code blocks, step; And a fifth step of the first terminal receiving an operation execution signal of the robot according to the second conditional expression using the one or more code blocks displayed; A sixth step of the first robot connected to the first terminal receiving the second conditional expression and the operation execution signal from the first terminal; A seventh step of the first robot transmitting the second conditional expression to the cloud and requesting the second conditional expression to determine whether the second conditional expression is satisfied; An eighth step of determining whether the cloud satisfies the second conditional expression; (E) when the cloud satisfies the second conditional expression, the cloud transmits an alert signal to the first robot; And when the first robot receives the notification signal, the actuation unit of the first robot is driven based on the operation execution signal according to the second conditional expression.
In the present invention, the conditional expression generated in the terminal may be stored in the cloud so that the conditional expression is shared.
In addition, since distributed information is collected and processed in the cloud, unnecessary operations can be reduced.
In addition, the cloud can calculate the conditional expression faster by performing the calculation of the conditional expression, and it is possible to reduce the calculation burden of the robot.
In addition, since the conditional expression is received from the cloud and the received conditional expression is used, a new conditional expression can be generated more easily and efficiently.
Also, when programming a robot control program, it is easy to program by operating the intuitive tool without directly inputting the execution code of the text form.
In addition, the conditional expression includes meta data, and the conditional expression stored in the DB of the cloud can be efficiently retrieved by using the meta data.
It should be understood, however, that the effects obtained by the present invention are not limited to the above-mentioned effects, and other effects not mentioned may be clearly understood by those skilled in the art to which the present invention belongs It will be possible.
BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate a preferred embodiment of the invention and, together with the description, serve to provide a further understanding of the technical idea of the invention, It should not be construed as limited.
1 is a block diagram of a robot control programming system according to an embodiment of the present invention.
2 is a block diagram of a robot control system according to an embodiment of the present invention.
FIG. 3A is a flowchart illustrating a robot control programming method according to an embodiment of the present invention, and FIG. 3B is a block diagram illustrating a signal flow between a first terminal and a cloud in a robot control programming method according to an embodiment of the present invention.
FIG. 4A is a flowchart illustrating a method of deriving a first conditional expression related to a search term according to an embodiment of the present invention, FIG. 4B is a flowchart illustrating a method of deriving a first conditional expression according to an embodiment of the present invention, As shown in FIG.
FIG. 5A is a flowchart illustrating a method of transmitting a first conditional expression corresponding to an input first conditional name according to an embodiment of the present invention. FIG. 5B is a flowchart illustrating a method of transmitting a first conditional expression name according to an embodiment of the present invention, FIG. 2 is a block diagram illustrating a signal flow between a first terminal and a cloud in a method of transmitting a corresponding first conditional expression; FIG.
FIG. 6A is a flowchart illustrating a robot control method according to an embodiment of the present invention. FIG. 6B is a block diagram illustrating a signal flow between a first terminal, a first robot, and a cloud in a robot control method according to an embodiment of the present invention. to be.
FIG. 7A illustrates a plurality of first conditional name names displayed on the display unit of the first terminal according to an embodiment of the present invention, FIG. 7B illustrates a step of inputting a search word according to an embodiment of the present invention, Displayed on the display unit.
FIG. 8 shows a plurality of condition variables according to an embodiment of the present invention displayed on the display unit of the first terminal.
FIG. 9 shows that the second conditional expression is displayed on the display unit according to an embodiment of the present invention.
FIG. 10 shows a plurality of code blocks capable of generating an operation execution signal of the robot on the display unit of the first terminal according to an embodiment of the present invention.
11 shows that both the second conditional expression and the operation execution signal of the first robot are inputted according to an embodiment of the present invention.
Hereinafter, a preferred embodiment of the present invention will be described with reference to the drawings. In addition, the embodiment described below does not unduly limit the contents of the present invention described in the claims, and the entire configuration described in this embodiment is not necessarily essential as the solution means of the present invention.
1 is a block diagram of a robot control programming system according to an embodiment of the present invention.
1, a robot control programming system according to an embodiment of the present invention may include a
The
First, the
The first conditional expression stored in the
This first conditional expression means a situation (event) for the
The first conditional expression may include first metadata which is attribute information of the first conditional expression. The first meta data may include any information related to the first conditional expression such as the name of the first conditional expression, the use range, and the like . Each of the first conditional expressions may have a name of each first conditional expression.
Next, the
The condition variable can be used in the first conditional expression, for example, any variable used in the conditions such as GPS information, position, height, luminous intensity, humidity, etc. may be applicable.
Next, the
In the case of wireless communication, the
The mobile communication module transmits and receives radio signals to and from at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data depending on a voice call signal, a video call signal or a text / multimedia message transmission / reception. The mobile communication technology may be a long term evolution (LTE), a code division multiple access (CDMA), a frequency division multiple access (FDMA), a time division multiple access (TDMA), an orthogonal frequency division multiple access (OFDMA) single carrier frequency division multiple access), or the like may be used.
In addition, the wireless Internet module refers to a module for wireless Internet access, and may be embedded in the
The short-range communication module is a module for short-range communication. Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and the like can be used as the short range communication technology.
Next, the search means 14 searches the first meta data of the first conditional expression related to the search term using the search word input by the second input means 24 of the terminal 20, And derives the first conditional expression.
On the other hand, the terminal 20 may be provided in plurality as a configuration for communicating with the
The terminal 20 includes a
The
The
That is, when there are a plurality of
Thereby, the second conditional expression generated by the
The
Next, the display unit 22 displays information processed by the terminal 20, and displays data received from the
The display unit 22 may be a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED) display, and a 3D display.
Next, the first input means 23, the second input means 24, the third input means 25 and the fourth input means 26 are configured to perform specific inputs.
The first input means 23, the second input means 24, the third input means 25 and the fourth input means 26 are sensors for sensing the touch operation and have a mutual layer structure with the display portion 22 A touch screen may be formed.
The first input means 23 is a means for inputting a second conditional expression which is a condition for operating the
Next, the
Examples of a code block capable of generating an operation execution signal of the
In order to execute each operation of the
2 is a block diagram of a robot control system according to an embodiment of the present invention.
The robot control system of the present invention shown in FIG. 2 is based on the robot control programming system described above, and further includes a
In the present invention, the
First, the
The
Next, the
The
The actuating
The
Various measured values measured by the
Hereinafter, a robot control programming method will be described in detail based on the above-described configurations.
FIG. 3A is a flowchart illustrating a robot control programming method according to an embodiment of the present invention, and FIG. 3B is a block diagram illustrating a signal flow between a first terminal and a cloud in a robot control programming method according to an embodiment of the present invention.
First, the
That is, the first conditional expression stored in the
Next, the
That is, the display unit 22 of the
Next, the
The second conditional expression is a condition for operating the
As described above, if the display unit 22 is configured as a touch screen, the second conditional expression can be received by receiving the touch input to the first conditional expression and the conditional variable displayed on the touch screen.
Next, the
That is, the
Next, the
The
Thereby, when the step S110 is performed again, the effect that the second conditional expression generated in the
Next, the
The code block generates an operation execution signal of the
Next, the
That is, the third input means 25 receives the first conditional expression displayed on the display unit 22 and the second conditional expression using the conditional variable.
As described above, if the display unit 22 is configured as a touch screen, a touch input may be input to a code block displayed on the touch screen to receive an operation execution signal of the
Meanwhile, in order to efficiently transmit the first conditional expression from the
FIG. 4A is a flowchart illustrating a method of deriving a first conditional expression related to a search term according to an embodiment of the present invention, FIG. 4B is a flowchart illustrating a method of deriving a first conditional expression according to an embodiment of the present invention, As shown in FIG.
First, the
The first metadata is attribute information of the first conditional expression as described above, and is stored in the
In step S10-1, the second input means 24 of the
Next, the
That is, the
Next, the first meta data related to the search word received by the
The search means 14 of the
The derived first conditional expression is a first conditional expression related to a search word input by the second input means 24 of the
Since the first conditional expression (S110) is performed and transmitted from the
Meanwhile, in order to efficiently transmit the first conditional expression from the
FIG. 5A is a flowchart illustrating a method of transmitting a first conditional expression corresponding to an input first conditional name according to an embodiment of the present invention. FIG. 5B is a flowchart illustrating a method of transmitting a first conditional expression name according to an embodiment of the present invention, FIG. 2 is a block diagram illustrating a signal flow between a first terminal and a cloud in a method of transmitting a corresponding first conditional expression; FIG.
The
The plurality of first conditional expressions stored in the
And displays a plurality of first conditional expression names received by the first terminal 20 (S20-2).
That is, the display unit 22 of the
At least one of a plurality of first conditional expression names displayed on the
That is, at least one first conditional expression name among a plurality of first conditional expression names displayed on the display unit 22 by the fourth input means 26 is selected and input.
As described above, if the display unit 22 is configured as a touch screen, the first conditional expression name displayed on the touch screen can be input to receive the first conditional expression name. The first conditional expression name selected at this time may be plural.
The
That is, the
The first conditional expression, which is performed in step S110 and is transmitted from the
The above is a programming method for controlling the robot.
Hereinafter, a method of controlling a robot based on the robot control programming method described above based on the above-described configuration will be described.
FIG. 6A is a flowchart illustrating a robot control method according to an embodiment of the present invention. FIG. 6B is a block diagram illustrating a signal flow between a first terminal, a first robot, and a cloud in a robot control method according to an embodiment of the present invention. to be.
First, the
The second conditional expression received by the
When there are a plurality of second conditional expressions and operation execution signals input to the
Next, the
The
In general, the
Next, the
That is, the
Next, when the
When the
Next, when the
The
For example, if the second conditional expression is' whether there is an obstacle in front of the first robot 30 'and the operation is' light emission', (i) the first robot 30 ' (Ii) the
Hereinafter, an embodiment of the robot control method will be described with reference to the drawings.
FIG. 7A illustrates a plurality of first conditional name names displayed on the display unit of the first terminal according to an embodiment of the present invention, FIG. 7B illustrates a step of inputting a search word according to an embodiment of the present invention, Displayed on the display unit.
7A shows a plurality of first conditional expression names displayed on the display unit 22 of the
'Near' shown in FIG. 7A is a conditional name of whether a distance between any two is smaller than a specific value, 'shorter' is a conditional name of whether or not the height of some is less than a specific value, and 'dark' 'Foggy' is the conditional name of whether the fog is lower than a certain value. However, examples of the display method of the condition expression name and the example of the condition expression name are not limited to those shown in Fig. 7A.
At least one of the displayed first conditional expression names is selected and input, and the selected first conditional expression name is transmitted to the
As shown in FIG. 7A, the first conditional expression may be selected as the first conditional name as shown in FIG. 7A, but the search term associated with the first metadata may be input to the
The
If the step S10-1 has been performed, the
The
8 illustrates a plurality of condition variables according to an embodiment of the present invention displayed on the display unit of the first terminal. Referring to FIG. 8, a plurality of condition variables' location ',' height ',' illumination ', and' humidity '. That is, the second conditional expression is used to indicate a conditional variable that can be used for input.
Next, FIG. 9 shows that the second conditional expression is displayed on the display unit according to an embodiment of the present invention.
Distance (a, b) <10 in FIG. 9 means whether the distance between a and b is less than 10, and height (b) <1.5 means whether the height of b is less than 1.5.
That is, the second conditional expression input in FIG. 9 is satisfied when the distance between a and b is less than 10 and the height of b is less than 1.5.
In order to input the second conditional expression shown in FIG. 9, 'near' and 'shorter' are used in the first conditional expression received previously, which is a more complex second conditional expression using the received first conditional expression.
The second conditional expression thus generated is transmitted to the
Next, the display unit 22 of the
The third input means 25 of the
In this embodiment, 'light emission' is inputted among a plurality of code blocks, and FIG. 11 shows that both the second conditional expression and the operation execution signal of the first robot are inputted according to the embodiment of the present invention.
A plurality of sets of input second conditional expressions and operation execution signals may be input and transmitted from the
The
The
That is, in the present embodiment, if a is the
When the
The present invention can also be embodied as computer-readable codes on a computer-readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may be implemented in the form of a carrier wave (for example, transmission over the Internet) . In addition, the computer-readable recording medium may be distributed over network-connected computer systems so that computer readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers of the technical field to which the present invention belongs.
The above-described robot control programming method, system and robot control method and system are not limited to the configuration and method of the above-described embodiments, but the embodiments may be modified so that various modifications may be made. All or some of them may be selectively combined.
In the present invention applying the above-described configuration, conditional expressions generated in the terminal may be stored in the cloud so that the conditional expressions are shared.
In addition, since distributed information is collected and processed in the cloud, unnecessary operations can be reduced.
In addition, the cloud can calculate the conditional expression faster by performing the calculation of the conditional expression, and it is possible to reduce the calculation burden of the robot.
In addition, since the conditional expression is received from the cloud and the received conditional expression is used, a new conditional expression can be generated more easily and efficiently.
Also, when programming a robot control program, it is easy to program by operating the intuitive tool without directly inputting the execution code of the text form.
In addition, the conditional expression includes meta data, and the conditional expression stored in the DB of the cloud can be efficiently retrieved by using the meta data.
10: Cloud
11: First DB
12: Second DB
13: First communication section
14: Search means
15:
20: terminal
21: second communication section
22:
23: first input means
24: second input means
25: third input means
26: fourth input means
27: Third DB
30: Robot
31: Third communication section
32: Actuating part
33:
Claims (27)
In the cloud,
A first DB storing a plurality of first conditional expressions which are conditions under which the robot operates;
A second DB storing a plurality of condition variables used for a condition in which the robot operates; And
And a first communication unit,
The terminal,
A second communication unit receiving the one or more first conditional expressions stored in the first DB and the one or more conditional variables stored in the second DB from the first communication unit;
A display unit for displaying the one or more first conditional expressions and the one or more conditional variables received by the second communication unit;
A first input means for allowing a user to input a second conditional expression that is a condition that the robot operates using the one or more first conditional expressions and the one or more conditional variables displayed on the display unit; And
And a third DB storing one or more code blocks capable of generating an operation execution signal of the robot according to the second conditional expression,
Wherein the display unit displays the one or more code blocks,
The terminal further comprises third input means for allowing the user to input an operation execution signal of the robot according to the second conditional expression using the one or more code blocks displayed on the display unit,
The robot includes:
An actuating part driven based on the operation execution signal; And
And a third communication unit that receives the second conditional expression and the operation execution signal from the second communication unit and transmits the second conditional expression to the first communication unit to request the second conditional expression to determine whether the second conditional expression is satisfied,
In the cloud,
And a controller for determining whether the second conditional expression received by the first communication unit is satisfied,
Wherein the first communication unit transmits a notification signal to the third communication unit when the second conditional expression is satisfied,
When the third communication unit receives the notification signal,
Wherein the actuating unit is driven based on the operation execution signal according to the second conditional expression.
The robot includes:
And a sensor unit for measuring a measured value,
The third communication unit transmits the measured value to the first communication unit,
Wherein the control unit determines whether the second conditional expression is satisfied using the received measured value.
And the third communication unit periodically transmits the measurement value.
Wherein the first communication unit receives the input second conditional expression through the second communication unit,
Wherein the first DB stores the second conditional expression received by the first communication unit as the first conditional expression.
Wherein each of the plurality of first conditional expressions includes first metadata that is attribute information of each of the plurality of first conditional expressions,
Wherein the second conditional expression includes second metadata that is attribute information of the second conditional expression.
The terminal,
And second input means for inputting a search word related to the first metadata,
Wherein the first communication unit receives the search word from the second communication unit,
In the cloud,
And searching means for searching the first metadata related to a search word received by the first communication unit among the plurality of first metadata and deriving a first conditional expression of the first metadata,
Wherein the second communication unit receives the first conditional expression derived from the first communication unit,
Wherein the display unit displays the derived first conditional expression received by the second communication unit.
Wherein each of the plurality of first conditional expressions has a first conditional expression name of each of the plurality of first conditional expressions,
The second communication unit receives the plurality of first conditional expression names stored in the first DB from the first communication unit,
Wherein the display unit displays the plurality of first conditional expression names,
The terminal,
Fourth input means for allowing the user to select at least one of the plurality of first conditional expression names displayed on the display unit,
Wherein the first communication unit receives the first conditional expression name input by the user from the second communication unit and outputs a first conditional expression corresponding to the first conditional expression name input by the user stored in the first DB, To the second communication unit.
In the cloud,
A first DB storing a plurality of first conditional expressions which are conditions under which the robot operates;
A second DB storing a plurality of condition variables used for a condition in which the robot operates; And
And a first communication unit,
Wherein each of the plurality of terminals comprises:
A second communication unit receiving the one or more first conditional expressions stored in the first DB and the one or more conditional variables stored in the second DB from the first communication unit;
A display unit for displaying the one or more first conditional expressions and the one or more conditional variables received by the second communication unit;
A first input means for allowing a user to input a second conditional expression that is a condition that the robot operates using the one or more first conditional expressions and the one or more conditional variables displayed on the display unit; And
And a third DB storing one or more code blocks capable of generating an operation execution signal of the robot according to the second conditional expression,
Wherein the display unit displays the one or more code blocks,
Each of the plurality of terminals further includes third input means for allowing the user to input an operation execution signal of the robot according to the second conditional expression using the one or more code blocks displayed on the display unit ,
Wherein each of the plurality of robots comprises:
An actuating part driven based on the operation execution signal; And
And a third communication unit for receiving the second conditional expression and the operation execution signal from a second communication unit of each of the connected terminals,
The third communication unit of the first robot among the plurality of robots transmits the second conditional equation to the first communication unit to request the second conditional unit to determine whether the second conditional equation is satisfied,
In the cloud,
And a controller for determining whether the second conditional expression received by the first communication unit is satisfied,
Wherein the first communication unit transmits a notification signal to the third communication unit of the first robot when the second conditional expression is satisfied,
When the third communication unit of the first robot receives the notification signal,
Wherein the actuating part of the first robot is driven based on the operation execution signal according to the second conditional expression.
Wherein each of the plurality of robots comprises:
And a sensor unit for measuring a measured value,
The third communication unit transmits the measured value to the first communication unit,
Wherein the control unit determines whether the second conditional expression is satisfied using the received measured value.
And the third communication unit periodically transmits the measurement value.
Wherein the control unit requests the transfer of the measurement value to at least one robot among the plurality of robots in order to determine whether the second conditional expression is satisfied.
Wherein the first communication unit receives the input second conditional expression through the second communication unit,
Wherein the first DB stores the second conditional expression received by the first communication unit as the first conditional expression.
Wherein the at least one first conditional expression received by the second communication unit of the first terminal among the plurality of terminals includes a second conditional expression transmitted by the second communication unit of the second terminal to the first communication unit.
Wherein each of the plurality of first conditional expressions includes first metadata that is attribute information of each of the plurality of first conditional expressions,
Wherein the second conditional expression includes second metadata that is attribute information of the second conditional expression.
Wherein each of the plurality of terminals comprises:
And second input means for inputting a search word related to the first metadata,
When the second input means of the first terminal among the plurality of terminals inputs the search word, the first communication unit receives the search word from the second communication unit of the first terminal,
In the cloud,
And searching means for searching the first metadata related to a search word received by the first communication unit among the plurality of first metadata and deriving a first conditional expression of the first metadata,
Wherein the second communication unit of the first terminal receives the first conditional expression derived from the first communication unit,
Wherein the display unit of the first terminal displays the derived first conditional expression received by the second communication unit of the first terminal.
Wherein each of the plurality of first conditional expressions has a first conditional expression name of each of the plurality of first conditional expressions,
The second communication unit receives the plurality of first conditional expression names stored in the first DB from the first communication unit,
Wherein the display unit displays the plurality of first conditional expression names,
Wherein each of the plurality of terminals comprises:
Fourth input means for allowing the user to select at least one of the plurality of first conditional expression names displayed on the display unit,
Wherein the first communication unit receives the first conditional expression name input by the user from the second communication unit of the first terminal among the plurality of terminals and receives the first conditional expression name input by the user stored in the first DB To the second communication unit of the first terminal.
At least one first terminal among the plurality of terminals receives the at least one first conditional expression and the at least one condition variable from the cloud;
A second step of the first terminal displaying the received one or more first conditional expressions and the one or more conditional variables; And
And a third step of the first terminal receiving a second conditional expression that is a condition that the robot operates using the one or more first conditional expressions and the one or more conditional variables indicated by the first terminal,
Wherein the first terminal includes a third DB storing one or more code blocks capable of generating an operation execution signal of the robot according to the second conditional expression,
After the third step,
A fourth step of the first terminal displaying the one or more code blocks; And
A fifth step of the first terminal receiving an operation execution signal of the robot according to the second conditional expression using the indicated one or more code blocks;
A sixth step of the first robot connected to the first terminal receiving the second conditional expression and the operation execution signal from the first terminal;
A seventh step of the first robot transmitting the second conditional expression to the cloud and requesting the second conditional expression to determine whether the second conditional expression is satisfied;
An eighth step of determining whether the cloud satisfies the second conditional expression;
(E) when the cloud satisfies the second conditional expression, the cloud transmits an alert signal to the first robot;
And when the first robot receives the notification signal, the actuating unit of the first robot is driven based on the operation execution signal according to the second conditional expression. Way.
Wherein each of the plurality of robots comprises:
And a sensor unit for measuring a measured value,
Transmitting the measured value to the cloud,
In the cloud,
Determining whether the second conditional expression is satisfied by using the measured value.
Wherein each of the plurality of robots transmits the measured values periodically.
After the seventh step,
Wherein the cloud requests transmission of the measurement value to the plurality of robots.
After the third step,
The first terminal transmitting the input second conditional expression to the cloud; And
And storing the received second conditional expression in the first DB as the first conditional expression by the cloud.
Wherein the at least one first conditional expression received by the first terminal includes a second conditional expression transmitted by the second terminal to the cloud.
Wherein each of the plurality of first conditional expressions includes first metadata that is attribute information of each of the plurality of first conditional expressions,
Wherein the second conditional expression includes second metadata that is attribute information of the second conditional expression.
Before the first step,
Receiving a search term associated with the first metadata by the first terminal;
The first terminal transmitting the search query to the cloud; And
Further comprising the step of the cloud retrieving a first meta data associated with the received search term among the plurality of first meta data to derive a first conditional expression of the searched first meta data,
In the first step and the second step,
Wherein the first conditional expression is the derived first conditional expression.
Wherein each of the plurality of first conditional expressions has a first conditional expression name of each of the plurality of first conditional expressions,
Before the first step,
The first terminal receiving the plurality of first conditional expression names;
Displaying the first plurality of conditional expression names received by the first terminal;
Selecting and inputting at least one of the displayed plurality of first conditional expression names in the first terminal; And
The first terminal sending the input at least one first conditional expression name to the cloud,
In the first step,
Wherein the first conditional expression is a first conditional expression corresponding to at least one first conditional expression name transmitted to the cloud.
The robot control method includes:
At least one first terminal among the plurality of terminals receives the at least one first conditional expression and the at least one condition variable from the cloud;
A second step of the first terminal displaying the received one or more first conditional expressions and the one or more conditional variables; And
And a third step of the first terminal receiving a second conditional expression that is a condition that the robot operates using the one or more first conditional expressions and the one or more conditional variables indicated by the first terminal,
Wherein the first terminal includes a third DB storing one or more code blocks capable of generating an operation execution signal of the robot according to the second conditional expression,
After the third step,
A fourth step of the first terminal displaying the one or more code blocks; And
A fifth step of the first terminal receiving an operation execution signal of the robot according to the second conditional expression using the indicated one or more code blocks;
A sixth step of the first robot connected to the first terminal receiving the second conditional expression and the operation execution signal from the first terminal;
A seventh step of the first robot transmitting the second conditional expression to the cloud and requesting the second conditional expression to determine whether the second conditional expression is satisfied;
An eighth step of determining whether the cloud satisfies the second conditional expression;
(E) when the cloud satisfies the second conditional expression, the cloud transmits an alert signal to the first robot;
And when the first robot receives the notification signal, the actuation unit of the first robot is driven based on the operation execution signal according to the second conditional expression. .
The robot control method includes:
At least one first terminal among the plurality of terminals receives the at least one first conditional expression and the at least one condition variable from the cloud;
A second step of the first terminal displaying the received one or more first conditional expressions and the one or more conditional variables; And
And a third step of the first terminal receiving a second conditional expression that is a condition that the robot operates using the one or more first conditional expressions and the one or more conditional variables indicated by the first terminal,
Wherein the first terminal includes a third DB storing one or more code blocks capable of generating an operation execution signal of the robot according to the second conditional expression,
After the third step,
A fourth step of the first terminal displaying the one or more code blocks; And
A fifth step of the first terminal receiving an operation execution signal of the robot according to the second conditional expression using the indicated one or more code blocks;
A sixth step of the first robot connected to the first terminal receiving the second conditional expression and the operation execution signal from the first terminal;
A seventh step of the first robot transmitting the second conditional expression to the cloud and requesting the second conditional expression to determine whether the second conditional expression is satisfied;
An eighth step of determining whether the cloud satisfies the second conditional expression;
(E) when the cloud satisfies the second conditional expression, the cloud transmits an alert signal to the first robot;
And when the first robot receives the notification signal, the actuation unit of the first robot is driven based on the operation execution signal according to the second conditional expression. .
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR1020150061500A KR101678781B1 (en) | 2015-04-30 | 2015-04-30 | System and method for controlling robot in cloud environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR1020150061500A KR101678781B1 (en) | 2015-04-30 | 2015-04-30 | System and method for controlling robot in cloud environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| KR20160129399A true KR20160129399A (en) | 2016-11-09 |
| KR101678781B1 KR101678781B1 (en) | 2016-11-22 |
Family
ID=57529504
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| KR1020150061500A KR101678781B1 (en) | 2015-04-30 | 2015-04-30 | System and method for controlling robot in cloud environment |
Country Status (1)
| Country | Link |
|---|---|
| KR (1) | KR101678781B1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20210091923A (en) * | 2020-01-15 | 2021-07-23 | 주식회사 코딩플러스 | Bluetooth-based coding education multi-robot system with information exchange protocol between robots |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20140126539A (en) * | 2013-04-23 | 2014-10-31 | 삼성전자주식회사 | Moving robot, user terminal apparatus, and control method thereof |
-
2015
- 2015-04-30 KR KR1020150061500A patent/KR101678781B1/en active IP Right Grant
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20140126539A (en) * | 2013-04-23 | 2014-10-31 | 삼성전자주식회사 | Moving robot, user terminal apparatus, and control method thereof |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20210091923A (en) * | 2020-01-15 | 2021-07-23 | 주식회사 코딩플러스 | Bluetooth-based coding education multi-robot system with information exchange protocol between robots |
Also Published As
| Publication number | Publication date |
|---|---|
| KR101678781B1 (en) | 2016-11-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10380194B2 (en) | Displaying sort results on a mobile computing device | |
| EP3338238A1 (en) | Displaying advertisements in application launcher | |
| US9317417B2 (en) | Smart digital message archival | |
| CN111125269B (en) | Data management method, blood relationship display method and related device | |
| US9852375B2 (en) | Techniques for mobile prediction | |
| EP2963567A1 (en) | Method, device, server and terminal for visiting webpage | |
| US11782910B2 (en) | System and method for dynamic inference collaboration | |
| US20170371925A1 (en) | Query data structure representation | |
| US11381457B2 (en) | Split decision trees on client and server | |
| CN111143543A (en) | Object recommendation method, device, equipment and medium | |
| US20180203909A1 (en) | Triggering method for instant search | |
| TW201525922A (en) | Computer-implemented method for recommending visiting booths | |
| KR101678781B1 (en) | System and method for controlling robot in cloud environment | |
| KR101690873B1 (en) | Programming system and method for controlling robot in cloud environment | |
| KR102450094B1 (en) | Method and system for providing video or image contents based on location information | |
| WO2015096660A1 (en) | Methods and devices for displaying a webpage | |
| US9664401B2 (en) | Method and apparatus for controlling air conditioner, and air conditioner | |
| US9734253B2 (en) | Method of providing a webpage using home device web browser and home device therefor | |
| CN112328349B (en) | Parameter setting method and related equipment | |
| CN112131482B (en) | Aging determining method and related device | |
| US10176457B2 (en) | System and method automatically learning and optimizing sequence order | |
| US9977767B2 (en) | Systems and methods for disabling or expiring hyperlinks | |
| US8898135B2 (en) | Information processing device, information processing method, information processing method, information processing program and recording medium | |
| CN114077366A (en) | Application control method, intelligent terminal and storage medium | |
| US10222865B2 (en) | System and method for selecting gesture controls based on a location of a device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| E902 | Notification of reason for refusal | ||
| E701 | Decision to grant or registration of patent right | ||
| GRNT | Written decision to grant | ||
| FPAY | Annual fee payment |
Payment date: 20190911 Year of fee payment: 4 |