WO2018133593A1

WO2018133593A1 - Control method and device for intelligent terminal

Info

Publication number: WO2018133593A1
Application number: PCT/CN2017/115961
Authority: WO
Inventors: 胡华智
Original assignee: 亿航智能设备（广州）有限公司
Priority date: 2017-01-17
Filing date: 2017-12-13
Publication date: 2018-07-26
Also published as: CN106708412B; CN106708412A

Abstract

A control method and device for an intelligent terminal, applied to a mobile terminal. The method comprises: sensing a hand gesture operation, the hand gesture operation being made by a user touching a screen of the mobile terminal (S110); recognizing the hand gesture operation and retrieving a hand gesture order from a preset database in the mobile terminal, the database comprising a plurality of the hand gesture orders, the hand gesture order corresponding to the hand gesture operation (S120); and sending the hand gesture order to the intelligent terminal that is worn on a head of a user and has a screen for displaying an image, and the intelligent terminal controlling, on the basis of the hand gesture order, the image displayed on the screen of the intelligent terminal to change (S130). The control of the intelligent terminal is achieved by using hand gestures. When a user is wearing the intelligent terminal, the user can still accurately perceive the location of the mobile terminal and thereby can perform accurate hand gesture operations on the mobile terminal, so as to correspondingly complete, on the basis of the hand gesture operations on the mobile terminal, the control of the image displayed on the screen of the intelligent terminal.

Description

Invention name: Control method and device for intelligent terminal

Technical field

[0001] The present invention relates to the field of UAV technologies, and in particular, to a control method and apparatus for an intelligent terminal.

Background technique

[0002] At present, the application of the drone is gradually popularized, and it is more common to perform aerial photography by the camera device carried by the drone, thereby conveniently collecting images for the user.

[0003] Most of the existing drone control methods are: Using a remote controller with physical buttons as the front control end of the drone, the user controls the speed, height and direction of the drone through physical buttons. However, when the user wears the virtual reality glasses to view the images collected by the drone, the user's field of vision is blocked by the virtual reality glasses, making it difficult for the user to control the virtual reality glasses. Therefore, a new technical solution is needed to accurately control. Virtual reality glasses.

technical problem

In view of the above, an object of the present invention is to provide a control method and apparatus for an intelligent terminal to accurately control an intelligent terminal when a user wears a smart terminal.

Problem solution

Technical solution

[0005] The technical solution adopted by the present invention to solve the above technical problems is as follows:

According to an aspect of the present invention, a method for controlling a smart terminal is applied to a mobile terminal, the method comprising: sensing a gesture operation, the gesture operation being formed by a user touching on a screen of the mobile terminal Recognizing the gesture operation, the gesture instruction is retrieved from a database preset in the mobile terminal, where the database includes a plurality of gesture commands, and the gesture instruction corresponds to the gesture operation; The gesture command is sent to the smart terminal, the smart terminal is used to be worn on the head of the user and has a screen for displaying an image, and the smart terminal controls the image displayed on the screen of the smart terminal according to the gesture instruction. Make changes.

[0007] Optionally, the foregoing method, the smart terminal controls a screen of the smart terminal according to the gesture instruction The image displayed in the screen is changed, and the method includes: displaying, by the screen of the smart terminal, a view image transmitted by the external device, and controlling the view image to move in a corresponding direction according to a direction corresponding to the gesture instruction.

[0008] Optionally, the foregoing method, the gesture instruction includes a first gesture instruction and a second gesture instruction that are lower than a preset threshold, and the smart terminal controls the smart terminal according to the gesture instruction. The image displayed in the screen is changed, and the method includes: displaying, by the screen of the smart terminal, a field of view image transmitted by the external device, and receiving the first gesture command, drawing a plurality of function icons on the view image; The smart terminal selects at least one of the plurality of function icons according to the second finger instruction, and executes a function corresponding to the selected function icon.

[0009] Optionally, in the foregoing method, the function corresponding to the multiple function icons comprises: sending a flight instruction to a drone that communicates with the smart terminal to control flight of the drone.

[0010] Optionally, the foregoing method further includes: the smart terminal drawing a gesture operation prompt on the view image to prompt the user to complete the gesture operation on a screen of the mobile terminal.

[0011] According to another aspect of the present invention, a control device for an intelligent terminal is provided, which is applied to a mobile terminal, the device includes: a sensing module that senses a gesture operation, and the gesture operation is performed by a user at the mobile terminal An on-screen touch is formed; the recognition module, the gesture operation is recognized, and a gesture instruction is retrieved from a database preset in the mobile terminal, where the database includes a plurality of gesture commands, the gesture instruction and the gesture The sending module sends the gesture instruction to the smart terminal, where the smart terminal is used to be worn on the head of the user and has a screen for displaying an image, and the smart terminal controls according to the gesture instruction. The image displayed on the screen of the smart terminal is changed.

[0012] Optionally, in the foregoing device, the screen of the smart terminal displays a field of view image transmitted by the external device, and controls the view image to move in a corresponding direction according to a direction corresponding to the gesture instruction.

[0013] Optionally, in the foregoing device, the gesture instruction includes a first gesture instruction and a second gesture instruction that the inter-turn interval is lower than a preset threshold; the screen of the smart terminal displays a view image transmitted by the external device and Receiving the first gesture command, drawing a plurality of function icons on the view image; the smart terminal selecting at least one of the plurality of function icons according to the second finger instruction, and executing the selected function The function corresponding to the icon.

[0014] Optionally, in the foregoing device, the function corresponding to the multiple function icons includes: The communicating drone sends a flight command to control the flight of the drone.

[0015] Optionally, in the foregoing device, the smart terminal further draws a gesture operation prompt on the view image to prompt the user to complete the gesture operation on a screen of the mobile terminal.

Advantageous effects of the invention

Beneficial effect

[0016] According to the above technical solution, a control method and apparatus for an intelligent terminal of the present invention have at least the following advantages:

[0017] The control of the smart terminal is implemented by the gesture action, and the user can still accurately perceive the location of the mobile terminal when the smart terminal is worn, so that an accurate gesture operation on the mobile terminal can be implemented, thereby correspondingly based on the mobile The gesture operation of the terminal completes the control of the image displayed on the screen of the smart terminal.

Brief description of the drawing

DRAWINGS

1 is a flowchart of a method for controlling an intelligent terminal according to an embodiment of the present invention;

2 is a schematic diagram of a gesture operation of a method for controlling an intelligent terminal according to an embodiment of the present invention;

3 is a flowchart of a method for controlling an intelligent terminal according to an embodiment of the present invention;

4A is a schematic diagram of a method for controlling an intelligent terminal according to an embodiment of the present invention;

4B is a schematic diagram of a method for controlling an intelligent terminal according to an embodiment of the present invention;

FIG. 5 is a block diagram of a control apparatus of an intelligent terminal according to an embodiment of the present invention.

[0024] The implementation, functional features, and advantages of the present invention will be further described with reference to the accompanying drawings.

Embodiments of the invention

[0025] The present invention will be further described in detail in conjunction with the drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

As shown in FIG. 1 , an embodiment of the present invention provides a method for controlling an intelligent terminal, which is applied to a mobile terminal, where the mobile terminal includes but is not limited to a mobile phone, a tablet computer, etc., and the method includes: [0027] Step S110, sensing a gesture operation, where the gesture operation is formed by a user touching on a screen of the mobile terminal. In this embodiment, the smart terminal includes, but is not limited to, virtual reality glasses for the user to wear; the gesture operation of the user may be completed based on a touch screen or an attitude sensor installed on the mobile terminal, for example, the user may click on the touch screen of the mobile phone. , double-click, slide, and other touch operations.

[0028] Step S120: Identify a gesture operation, and retrieve a gesture instruction from a database preset in the mobile terminal, where the database includes multiple gesture commands, and the gesture instruction corresponds to the gesture operation. In this embodiment, the operation gestures used by the user are all based on the daily life recognition operation, and the common operation gestures are as shown in FIG. 2, and the various operation gestures displayed therein are applicable to the technical solution of the embodiment, When the user's memory burden is increased, corresponding gesture commands are formed to complete various tasks required by the user. Gesture operations and corresponding gesture commands are stored in the mobile terminal database for later retrieval.

[0029] Step S130: Send a gesture instruction to the smart terminal, the smart terminal is used to be worn on the head of the user and has a screen for displaying an image, and the smart terminal controls the image displayed on the screen of the smart terminal to change according to the gesture instruction.

[0030] According to the technical solution of the embodiment, the control of the smart terminal is implemented by the gesture action, and the user can still accurately perceive the location of the mobile terminal when the smart terminal is worn, so that an accurate gesture to the mobile terminal can be implemented. Operation, thereby controlling the display of the image displayed on the screen of the smart terminal based on the gesture operation on the mobile terminal accordingly.

As shown in FIG. 3, an embodiment of the present invention provides a method for controlling an intelligent terminal, including: [0032] Step S310, sensing a gesture operation, where a gesture operation is performed by a user touching a screen of the mobile terminal Form

[0033] Step S320: Identify a gesture operation, and retrieve a gesture instruction from a database preset in the mobile terminal, where the database includes multiple gesture commands, and the gesture instruction corresponds to the gesture operation.

[0034] Step S330: Send the gesture instruction to the smart terminal, and the screen of the smart terminal displays the visual field image transmitted by the external device, and controls the visual field image to move in the corresponding direction according to the direction corresponding to the gesture instruction.

[0035] Step S340, the screen of the smart terminal displays the field of view image transmitted by the external device and receives the first gesture command 吋, and draws a plurality of function icons on the view image; the smart terminal selects from the plurality of function icons according to the second finger instruction. At least one, and perform the function corresponding to the selected function icon. The gesture instruction includes a first gesture instruction and a second gesture instruction that the inter-turn interval is lower than a preset threshold. [0036] In this embodiment, the gesture instruction is used to control the smart terminal, as follows:

[]

[0037] In this embodiment, the functions corresponding to the plurality of function icons include: transmitting a flight instruction to the unmanned aircraft communicating with the smart terminal, and controlling the flight of the drone.

[0038] In a specific embodiment, the first part is that when the aircraft has not yet taken off, the user can adjust the parameters of the smart terminal, watch the video, and the like. When the user slides on the screen of the mobile phone, the user's interface in the smart terminal is mapped to the touch screen of the mobile phone. When the finger slides from the top to the bottom of the screen of the mobile phone, corresponding to moving the screen display interface of the smart terminal from top to bottom, the field of view in the screen display of the smart terminal moves up. When the finger slides from the bottom up on the screen of the mobile phone, the screen display interface corresponding to the smart terminal moves from bottom to top, and then the field of view area in the screen display of the smart terminal moves down. When the finger slides from left to right on the screen of the mobile phone, the field of view in the screen display of the smart terminal moves to the left. When the finger slides from right to left on the screen of the phone, the field of view moves to the right. Due to the division of the option hierarchy, the field of view can only be switched at the same level. Based on the up, down, left and right movement interface, other gestures provide more functional actions, including tapping the screen to confirm the adjustment options, swiping left and right in the video to achieve fast rewind and fast forward.

[0039] In another specific embodiment, the second part is that after the system enters the flight mode, the user can touch the mobile phone interface to form the gesture operation without removing the smart terminal, and the mobile phone recognizes After the gesture operation, the command is sent to the aircraft to realize functions such as take-off, aerial shooting, position movement, return flight, and landing of the aircraft. Further, after the user touches the operation on the mobile phone interface, the visual field in the screen display of the smart terminal is moved to the "enter flight mode", and the user touches the take-off option by pressing and holding the mobile phone screen for three seconds, as shown in FIG. 4A, and then the smart The field of view inside the terminal's screen display becomes a camera The picture is transmitted, the user slides up on the mobile phone interface, the mobile phone recognizes the upward sliding gesture operation, and sends a flight instruction to the aircraft to take off the aircraft.

[0040] In the technical solution of the embodiment, the virtual reality glasses are used to control the flight of the drone, and after the user performs the gesture operation on the mobile terminal, the mobile terminal sends the instruction corresponding to the gesture operation to the virtual reality glasses, the virtual reality glasses. A plurality of operation options are provided, and an option is selected according to the instruction of the mobile terminal, and the flight instruction corresponding to the option is sent to the drone for the drone to fly according to the flight instruction, as shown in FIG. 4B.

[0041] Step S350: The smart terminal draws a gesture operation prompt on the view image to prompt the user to complete the gesture operation on the screen of the mobile terminal. In this embodiment, after the user performs a gesture operation on the mobile terminal, since the user cannot see the mobile terminal, the gesture operation performed by the user is displayed on the screen of the smart terminal. According to the technical solution of the embodiment, the mobile terminal-based control process completed by the user is as follows:

[0042] 1. The user takes the mobile phone in hand, slides from the top to the bottom, and the field of view moves to the "parameter adjustment" function option;

[0043] 2. The user slides from right to left, and the field of view moves to the "parameter adjustment 2" option;

[0044] 3. The user slides the touch screen of the mobile phone up and down to select a desired parameter option;

[0045] 4. The user slides from left to right, and returns "parameter adjustment";

[0046] 5. The user slides from bottom to top, and the field of view moves to "flight mode";

[0047] 6. The user slides from right to left and jumps to "enter flight mode";

[0048] 7. The user presses the screen for a long time by pressing the prompt for three seconds, and the vision of the glasses jumps to the image of the aircraft map;

[0049] 8. The picture transmission picture has parameters, maps, prompt pop-ups;

[0050] 9. The user slides from right to left into the screen to enter the flight parameter adjustment menu;

[0051] 10. The user slides the screen up and down, and selects the required parameters by way of a dial wheel;

[0052] 11. The user slides out of the screen from left to right, and the parameter adjustment menu is collapsed;

[0053] 12. The user clicks on the screen and a shortcut action menu appears on the screen.

[0054] 13. The user places a finger on the screen, moves without raising the finger, and selects the desired shortcut action.

[0055] As shown in FIG. 5, an embodiment of the present invention provides a control device for an intelligent terminal, which is applied to a mobile terminal. The mobile terminal includes but is not limited to a mobile phone, a tablet computer, etc., and the method includes:

[0056] The sensing module 510 senses a gesture operation, and the gesture operation is formed by a user touching on a screen of the mobile terminal. In this embodiment, the smart terminal includes but is not limited to virtual reality glasses for the user to wear. The user's gesture operation can be completed based on a touch screen or an attitude sensor installed on the mobile terminal. For example, the user can perform a touch operation such as clicking, double-clicking, sliding, etc. on the touch screen of the mobile phone.

[0057] The identification module 520 identifies a gesture operation, and retrieves a gesture instruction from a database preset in the mobile terminal, where the database includes multiple gesture commands, and the gesture instruction corresponds to the gesture operation. In this embodiment, the operation gestures used by the user are all based on the daily life recognition operation, and the common operation gestures are as shown in FIG. 2, and the various operation gestures shown in the embodiment are applicable to the technical solution of the embodiment, When the user's memory burden is increased, corresponding gesture commands are formed to complete various tasks required by the user. The gestures and corresponding gesture commands are stored in the mobile terminal database for later retrieval.

[0058] The sending module 530 sends a gesture instruction to the smart terminal, where the smart terminal is used to be worn on the head of the user and has a screen for displaying an image, and the smart terminal controls the image displayed on the screen of the smart terminal to change according to the gesture instruction. .

[0059] According to the technical solution of the embodiment, the control of the smart terminal is implemented by the gesture action, and the user can still accurately perceive the location of the mobile terminal when the smart terminal is worn, so that an accurate gesture to the mobile terminal can be implemented. Operation, thereby controlling the display of the image displayed on the screen of the smart terminal based on the gesture operation on the mobile terminal accordingly.

[0060] As shown in FIG. 5, an embodiment of the present invention provides a control device for an intelligent terminal, including: [0061] a sensing module 510, sensing a gesture operation, and the gesture operation is performed by a user on a screen of the mobile terminal Touch formation.

[0062] The identification module 520 identifies a gesture operation, and retrieves a gesture instruction from a database preset in the mobile terminal, where the database includes multiple gesture commands, and the gesture instruction corresponds to the gesture operation.

[0063] The sending module 530 sends the gesture command to the smart terminal, and the screen of the smart terminal displays the visual field image transmitted by the external device, and controls the visual field image to move in the corresponding direction according to the direction corresponding to the gesture instruction.

[0064] The screen of the smart terminal displays the field of view image transmitted by the external device and receives the first gesture command 吋, and draws a plurality of function icons on the view image; the smart terminal selects at least one of the plurality of function icons according to the second finger instruction, And execute the function corresponding to the selected function icon. The gesture command includes a first gesture instruction and a second gesture instruction that the inter-turn interval is below a predetermined threshold.

[0065] In this embodiment, the gesture instruction is used to control the smart terminal, as follows:

[0066] In this embodiment, the functions corresponding to the plurality of function icons include: transmitting a flight instruction to the unmanned aircraft communicating with the smart terminal, and controlling the flight of the drone.

[0067] In a specific embodiment, the first part is that when the aircraft has not yet taken off, the user can adjust the parameters of the smart terminal, watch the video, and the like. When the user slides on the screen of the mobile phone, the user's interface in the smart terminal is mapped to the touch screen of the mobile phone. When the finger slides from the top to the bottom of the screen of the mobile phone, corresponding to moving the screen display interface of the smart terminal from top to bottom, the field of view in the screen display of the smart terminal moves up. When the finger slides from the bottom up on the screen of the mobile phone, the screen display interface corresponding to the smart terminal moves from bottom to top, and then the field of view area in the screen display of the smart terminal moves down. When the finger slides from left to right on the screen of the mobile phone, the field of view in the screen display of the smart terminal moves to the left. When the finger slides from right to left on the screen of the phone, the field of view moves to the right. Due to the division of the option hierarchy, the field of view can only be switched at the same level. Based on the up, down, left and right movement interface, other gestures provide more functional actions, including tapping the screen to confirm the adjustment options, swiping left and right in the video to achieve fast rewind and fast forward.

[0068] In another specific embodiment, the second part is that after the system enters the flight mode, the user can touch the mobile phone interface to form the gesture operation without removing the smart terminal, and the mobile phone recognizes After the gesture operation, the command is sent to the aircraft to realize functions such as take-off, aerial shooting, position movement, return flight, and landing of the aircraft. Further, after the user touches the operation on the mobile phone interface, the visual field in the screen display of the smart terminal is moved to the "enter flight mode", and the user touches the take-off option by pressing and holding the mobile phone screen for three seconds, as shown in FIG. 4A, and then smart. The field of view in the screen display of the terminal becomes a camera image transmission screen, the user slides up on the mobile phone interface, and the mobile phone recognizes the upward sliding gesture operation, and sends a Flight instructions to the aircraft take off the aircraft.

[0069] In the technical solution of the embodiment, the virtual reality glasses are used to control the flight of the drone, and after the user performs the gesture operation on the mobile terminal, the mobile terminal sends the instruction corresponding to the gesture operation to the virtual reality glasses, the virtual reality glasses. A plurality of operation options are provided, and an option is selected according to the instruction of the mobile terminal, and the flight instruction corresponding to the option is sent to the drone for the drone to fly according to the flight instruction, as shown in FIG. 4B.

[0070] The smart terminal draws a gesture operation prompt on the view image to prompt the user to complete the gesture operation on the screen of the mobile terminal. In this embodiment, after the user performs a gesture operation on the mobile terminal, since the user cannot see the mobile terminal, the gesture operation performed by the user is displayed on the screen of the smart terminal. According to the technical solution of the embodiment, the mobile terminal-based control process completed by the user is as follows:

[0071] 1. The user takes the mobile phone in his hand, slides from the top to the bottom, and the field of view moves to the "parameter adjustment" function option;

[0072] 2. The user slides from right to left, and the field of view moves to the "parameter adjustment 2" option;

[0073] 3. The user slides the touch screen of the mobile phone up and down to select a desired parameter option;

[0074] 4. The user slides from left to right, and returns "parameter adjustment";

[0075] 5. The user slides from bottom to top, and the field of view moves to "flight mode";

[0076] 6. The user slides from right to left and jumps to "enter flight mode";

[0077] 7. The user presses the screen for a long time by pressing the prompt for three seconds, and the vision of the glasses jumps to the image of the aircraft map;

[0078] 8. The picture transmission picture has parameters, maps, prompt pop-ups;

[0079] 9. The user slides from right to left into the screen to enter the flight parameter adjustment menu;

[0080] 10. The user slides the screen up and down, and selects the required parameters by way of a dial wheel;

[0081] 11. The user slides out of the screen from left to right, and the parameter adjustment menu is collapsed;

[0082] 12. The user clicks on the screen and a shortcut action menu appears on the screen.

[0083] 13. The user places a finger on the screen, moves without raising the finger, and selects the desired shortcut action.

The preferred embodiments of the present invention have been described above with reference to the drawings, and are not intended to limit the scope of the invention.

A person skilled in the art can implement the invention in various variants without departing from the scope and spirit of the invention. For example, the features of one embodiment can be used in another embodiment to obtain a further embodiment. Any modifications, equivalent substitutions and improvements made within the technical concept of the invention are intended to be included within the scope of the invention. Industrial applicability

The control of the smart terminal is implemented by the gesture action, and the user can still accurately perceive the location of the mobile terminal when the smart terminal is worn, so that an accurate gesture operation on the mobile terminal can be implemented, and accordingly, based on the gesture to the mobile terminal Operation, complete control of the image displayed on the screen of the smart terminal. Therefore, it has industrial applicability.

Claims

Claim

[Claim 1] A method for controlling an intelligent terminal, which is applied to a mobile terminal, the method comprising:

Inducing a gesture operation, the gesture operation being formed by a user touching on a screen of the mobile terminal;

Identifying the gesture operation, and capturing a gesture instruction from a database preset in the mobile terminal

The database includes a plurality of gesture commands, where the gesture command corresponds to the gesture operation; the gesture command is sent to the smart terminal, and the smart terminal is used to be worn on the user's head and There is a screen for displaying an image, and the smart terminal controls the image displayed in the screen of the smart terminal to change according to the gesture instruction.

[Claim 2] The control method of the smart terminal according to claim 1, wherein the smart terminal controls the image displayed on the screen of the smart terminal to change according to the gesture instruction, and specifically includes:

The screen of the smart terminal displays a field of view image transmitted by the external device, and controls the field of view image to move in a corresponding direction according to a direction corresponding to the gesture command.

[Claim 3] The control method of the smart terminal according to claim 2, wherein the gesture instruction includes a first gesture instruction and a second gesture instruction that the inter-turn interval is lower than a preset threshold, the smart terminal according to the The gesture command controls the image displayed on the screen of the smart terminal to be changed, and specifically includes:

The screen of the smart terminal displays a field of view image transmitted by the external device and receives the first gesture command, and draws a plurality of function icons on the view image;

The smart terminal selects at least one of the plurality of function icons according to the second finger instruction, and executes a function corresponding to the selected function icon.

[Claim 4] The control method of the smart terminal according to claim 3, wherein the function corresponding to the plurality of function icons comprises: transmitting a flight instruction to a drone communicating with the smart terminal to control the Man-machine flight.

[Claim 5] The control method of the smart terminal according to claim 2, further comprising:

The smart terminal draws a gesture operation prompt on the view image to prompt the user to The gesture operation is completed on the screen of the mobile terminal.

A control device for an intelligent terminal is applied to a mobile terminal, the device includes: a sensing module, sensing a gesture operation, wherein the gesture operation is formed by a user touching on a screen of the mobile terminal;

An identification module, which is configured to retrieve a gesture instruction from a database preset in the mobile terminal, where the database includes a plurality of gesture commands, and the gesture instruction corresponds to the gesture operation;

The sending module sends the gesture instruction to the smart terminal, where the smart terminal is used to be worn on the head of the user and has a screen for displaying an image, and the smart terminal controls the smart terminal according to the gesture instruction The image displayed in the screen changes.

A control device for a smart terminal according to claim 6, wherein

The control device of the smart terminal according to claim 7, wherein the gesture instruction comprises a first gesture instruction and a second gesture instruction that the inter-turn interval is lower than a preset threshold; the screen of the smart terminal displays the transmission of the external device And receiving the first gesture command 吋, drawing a plurality of function icons on the view image; the smart terminal selecting at least one of the plurality of function icons according to the second finger instruction, and executing The function corresponding to the selected function icon.

The control device of the smart terminal according to claim 8, wherein the function corresponding to the plurality of function icons comprises: transmitting a flight instruction to the drone that communicates with the smart terminal to control flight of the drone.

A control device for a smart terminal according to claim 7, wherein

The smart terminal also draws a gesture operation prompt on the view image to prompt the user to complete the gesture operation on the screen of the mobile terminal.