CN109213307A

CN109213307A - A kind of gesture identification method and device, system

Info

Publication number: CN109213307A
Application number: CN201710523769.5A
Authority: CN
Inventors: 刘海军
Original assignee: ZTE Corp
Current assignee: ZTE Corp
Priority date: 2017-06-30
Filing date: 2017-06-30
Publication date: 2019-01-15

Abstract

The invention discloses a kind of gesture identification method and devices, system, which comprises the first equipment obtains the gesture list that the second equipment is supported；First equipment receives the Virtual Reality data that second equipment is sent, and shows to the VR data；First equipment acquires and identifies gesture, and the gesture list supported based on second equipment got generates gesture message, and the gesture message carries at least one of following information: gesture information, control information；The gesture message is sent to second equipment by first equipment.

Description

A kind of gesture identification method and device, system

Technical field

The present invention relates to command, control, communications, and information field more particularly to a kind of hands of virtual reality (VR, Virtual Reality) Gesture recognition methods and device, system.

Background technique

The development of digital device and network adds many convenience to people's daily life.Such as it is led in digital multimedia Domain, current technology allow people that can watch high definition television at home, and the source of TV programme may be from optical digital disk, wired TV, internet etc. can experience stereo, 5.1 sound channels, 7.1 sound channels or even sound effect more true to nature, and people are also Flat electronic equipment (PAD), mobile phone are able to use to realize these experience.In addition, people can be by network in distinct device Between shift the broadcasting of digital content, and by the broadcasting of one remote controler, gesture control multimedia equipment, such as control is cut Change a channel program, succeeding channel program etc..Some air-conditionings, refrigerator also provide network control function.

Gesture control is a kind of current relatively new equipment control mode: the camera monitoring gesture in an equipment is dynamic Make and carry out analysis identification, is finally converted into the control to the equipment；Or user use wearable device, by hand, On arm and equipment that body worn is similar similar to finger ring, wrist-watch, vest etc., identifies user action, also can be realized Control to equipment.Using gesture control, some products can allow people by gesture come controlling equipment, for example, in electricity Depending on increasing a camera on machine, for acquiring, identifying the gesture of people, then according to gesture predetermined and manipulation command pair It should be related to, achieve the effect that gesture manipulates television set, the manipulation having been carried out includes replacement channel, change volume etc..

In Gesture Recognition, a kind of mode is that the equipment being steered is required to have a camera, for calculating Machine visual identity.In certain environments (such as in home environment), there may be equipment not have camera, such as due to equipment ruler Very little reason, cost reason, but these equipment wish also be operated by user by gesture, it is desirable to camera handle in addition Gesture information is communicated to this equipment, to reach user using gesture to control this equipment.

For this purpose, transmitting control information technology between different devices based on network, branch is issued in a network by controlled plant The inventory for the gesture held, and/or the inventory of the corresponding instruction of the gesture of support；Gesture acquisition device obtains controlled set by network The inventory of the corresponding instruction of gesture of the inventory, and/or support of the standby gesture supported.Gesture acquisition device acquires gesture, by institute The gesture of acquisition is sent to controlled plant.Above scheme needs user in face of controlled plant, otherwise user can not see when implementing It is when use and inconvenient to the control operating result of equipment.

Summary of the invention

In order to solve the above technical problems, the embodiment of the invention provides a kind of gesture identification method and devices, system.

Gesture identification method provided in an embodiment of the present invention, comprising:

First equipment obtains the gesture list that the second equipment is supported；

First equipment receives the VR data that second equipment is sent, and shows to the VR data；

First equipment acquires and identifies gesture, and the gesture list supported based on second equipment got is generated Gesture message, the gesture message carry at least one of following information: gesture information, control information；

The gesture message is sent to second equipment by first equipment.

In the embodiment of the present invention, first equipment obtains the gesture list that the second equipment is supported, comprising:

First equipment receives the subscription gesture message that the second equipment is sent, in the subscription gesture message described in carrying The gesture list that second equipment is supported.

In the embodiment of the present invention, before first equipment receives the subscription gesture message that the second equipment is sent, the side Method further include:

First equipment issues inquiry broadcast or multicast message, to carry out equipment discovery to the second equipment；Alternatively, the first equipment After the inquiry broadcast or multicast message for receiving the sending of the second equipment, the inquiry broadcast or multicast message is responded；

After the first equipment and the second equipment complete equipment discovery, first equipment and second equipment establish network Connection.

First equipment is established by the network address of the second equipment and second equipment and is connected to the network.

In the embodiment of the present invention, the gesture list includes at least one of: the gesture that second equipment is supported is compiled Number, gesture numbers corresponding gesture feature data, gesture numbers corresponding gesture control instruction.

In the embodiment of the present invention, the gesture information includes at least one of: gesture number, gesture parameter, acquisition hand The gesture video flowing, predefined of position mapping parameters on present image of present image parameter, gesture when gesture, continuous acquisition Instruction encoding.

In the embodiment of the present invention, be stored at least one of following information in second equipment: second electronics is set The gesture list of standby VR data, second equipment support, wherein the VR data are VR image data or VR video counts According to；

The corresponding network address of the VR data or the network port are also carried in the subscription gesture message；Described first sets The standby VR data sent according to the corresponding network address of the VR data or the network port, reception second equipment.

Another embodiment of the present invention provides gesture identification method, comprising:

Second equipment sends the gesture list that the second equipment is supported to the first equipment；

Second equipment sends VR data to first equipment, so that first equipment shows the VR data；

Second equipment receives the gesture message that first equipment is sent, and the gesture message carries following information At least one: gesture information, control information；Believe that message executes corresponding control instruction according to the gesture.

In the embodiment of the present invention, second equipment sends the gesture list that second equipment is supported to the first equipment, Include:

Second equipment is sent to the first equipment subscribes to gesture message, carries described second in the subscription gesture message The gesture list that equipment is supported.

In the embodiment of the present invention, second equipment is sent before subscribing to gesture message to the first equipment, and the method is also Include:

Second equipment issues inquiry broadcast or multicast message, to carry out equipment discovery to the first equipment；Alternatively, the second equipment After the inquiry broadcast or multicast message for receiving the sending of the first equipment, the inquiry broadcast or multicast message is responded；

After the second equipment and the first equipment complete equipment discovery, second equipment and first equipment establish network Connection.

Second equipment is established by the network address of the first equipment and first equipment and is connected to the network.

The corresponding network address of the VR data or the network port are also carried in the subscription gesture message.

Gesture identifying device provided in an embodiment of the present invention is applied to the first equipment, and described device includes:

Receiving unit, for obtaining the gesture list of the second equipment support；Receive the VR data that second equipment is sent；

Display unit, for being shown to the VR data；

Recognition unit is acquired, for acquiring and identifying gesture, the gesture supported based on second equipment got is arranged Table generates gesture message, and the gesture message carries at least one of following information: gesture information, control information；

Transmission unit, for the gesture message to be sent to second equipment.

In the embodiment of the present invention, the receiving unit is also used to receive the subscription gesture message of the second equipment transmission, described It subscribes to and carries the gesture list that second equipment is supported in gesture message.

In the embodiment of the present invention, described device further include:

Connection unit, for going out to inquire broadcast or multicast message, to carry out equipment discovery to the second equipment；Alternatively, first Equipment responds the inquiry broadcast or multicast message after the inquiry broadcast or multicast message for receiving the sending of the second equipment；? After one equipment and the second equipment complete equipment discovery, first equipment and second equipment are established and are connected to the network.

In the embodiment of the present invention, described device further include:

Connection unit, for the network address and second equipment foundation network connection by the second equipment.

Another embodiment of the present invention provides gesture identifying device be applied to the second equipment, described device includes:

Transmission unit, for sending the gesture list that the second equipment is supported to the first equipment；It is sent to first equipment VR data, so that first equipment shows the VR data；

Receiving unit, the gesture message sent for receiving first equipment, the gesture message carry following information At least one of: gesture information, control information；

Execution unit, for executing corresponding control instruction according to the gesture message.

In the embodiment of the present invention, the transmission unit is also used to send to the first equipment and subscribes to gesture message, the subscription The gesture list that second equipment is supported is carried in gesture message.

In the embodiment of the present invention, described device further include:

Connection unit, for issuing inquiry broadcast or multicast message, to carry out equipment discovery to the first equipment；Alternatively, the Two equipment respond the inquiry broadcast or multicast message after the inquiry broadcast or multicast message for receiving the sending of the first equipment；? After second equipment and the first equipment complete equipment discovery, second equipment and first equipment are established and are connected to the network.

In the embodiment of the present invention, described device further include:

Connection unit, for the network address and first equipment foundation network connection by the first equipment.

Gesture recognition system provided in an embodiment of the present invention includes: the first equipment, the second equipment；Wherein,

First equipment, for obtaining the gesture list of the second equipment support；Receive the VR that second equipment is sent Data, and the VR data are shown；Acquire and identify gesture, the gesture supported based on second equipment got List generates gesture message, and the gesture message carries at least one of following information: gesture information, control information；It will be described Gesture message is sent to second equipment；

Second equipment, for sending the gesture list that second equipment is supported to the first equipment；To described first Equipment sends VR data, so that first equipment shows the VR data；The gesture message that first equipment is sent is received, The gesture message carries at least one of following information: gesture information, control information；Believe that message executes phase according to the gesture The control instruction answered.

In the technical solution of the embodiment of the present invention, the first equipment obtains the gesture list that the second equipment is supported；Described first Equipment receives the VR data that second equipment is sent, and shows to the VR data；First equipment acquires and knows Other gesture, the gesture list supported based on second equipment got generate gesture message, and the gesture message carries such as At least one of lower information: gesture information, control information；The gesture message is sent to described second and set by first equipment It is standby.Using technical solution of the present invention, user need not face the target device operated, so that it may know so that not having gesture The equipment of other component also can be more automation in operation, more humane by gesture control, and people easily use gesture to manipulate quilt The equipment of manipulation enjoys people's lives more easily, more, and grasps this function, also holds using when this kind of equipment It is readily understood, be easy to operate, also reduce the cost of each side in industrial chain.

Detailed description of the invention

Attached drawing generally shows each embodiment discussed herein by way of example and not limitation.

Fig. 1 is the schematic diagram of a scenario of the embodiment of the present invention；

Fig. 2 is the schematic diagram of VR aobvious equipment of the embodiment of the present invention；

Fig. 3 is the schematic diagram of VR aobvious equipment and controlled plant of the embodiment of the present invention；

Fig. 4 is the flow diagram one of the gesture identification method of the embodiment of the present invention；

Fig. 5 is the flow diagram two of the gesture identification method of the embodiment of the present invention；

Fig. 6 is the interacting message schematic diagram of the embodiment of the present invention；

Fig. 7 is the event trigger action schematic diagram of VR aobvious equipment of the embodiment of the present invention；

Fig. 8 is the flow diagram of VR aobvious equipment of the embodiment of the present invention；

Fig. 9 is the flow diagram of the controlled plant of the embodiment of the present invention；

Figure 10 is the structure composition schematic diagram one of the gesture identifying device of the embodiment of the present invention；

Figure 11 is the structure composition schematic diagram two of the gesture identifying device of the embodiment of the present invention；

Figure 12 is the structure composition schematic diagram of the gesture recognition system of the embodiment of the present invention.

Specific embodiment

The characteristics of in order to more fully hereinafter understand the embodiment of the present invention and technology contents, with reference to the accompanying drawing to this hair The realization of bright embodiment is described in detail, appended attached drawing purposes of discussion only for reference, is not used to limit the embodiment of the present invention.

Fig. 1 is the schematic diagram of a scenario of the embodiment of the present invention, as shown in Figure 1, dividing the present embodiments relate to 2 equipment It is not: the first equipment, the second equipment.Wherein, the first equipment refer specifically to VR show with gesture identification equipment (it is subsequent referred to as VR head Aobvious equipment)；Second equipment refers specifically to controlled plant, such as television set.

In scene shown in Fig. 1, television set supports gesture control function, but not for acquiring the camera of gesture； Geographically, television set aobvious can not be set within the scope of the optically-captured of VR aobvious equipment for example, user has on VR The standby television set being placed in bedroom in parlor, preparation control.In addition, in scene shown in Fig. 1, only depict one it is controlled Equipment, when practical application, VR aobvious equipment can control more than one controlled plant, and without controlled plant is located at VR and shows and set Within the scope of standby vision capture, this manipulates controlled plant for user and provides more convenience.

In scene shown in Fig. 1, there is network interface in 2 equipment, such as VR aobvious equipment, controlled plant are supported IEEE 802.11b/g/n or IEEE 802.3, so as to be connected to IP network.It include a control in each equipment Module, can control this equipment and other equipment send message, receive message and handle manipulation instruction or forwarding manipulation instruction.

Mutual discovery, connection, transmission message and the reception message capability of VR aobvious equipment and controlled plant on network, can To use universal plug and play (UPnP, Universal Plug and Play) technology to realize, multicast domain name system also can be used Unite (mDNS, multicast Domain Name System) and service discovery (DNS-SD, Domain based on domain name system Name System-Service Discovery) technology realization.Above-mentioned technology is used in an ip network, with unicast, multicast inquiry side Formula according to message format predetermined response inquiry, provides funcall.It shows and sets for example, UPnP technology defines media How standby (such as TV), server (such as DVD player, family's storage server) respond inquiry, which calling function are provided.This Outside, IP address, the domain name etc. that purpose equipment can also be inputted in local device directly search a certain equipment, such as aobvious at VR The IP address that television set is inputted on the keyboard of equipment connection, to establish the network connection of VR aobvious equipment and television set.

In one embodiment, television set stores what this equipment was supported to support gesture control in television set Gesture inventory, comprising: the gesture of support is numbered, such as 1 indicates that the five fingers open, and 2 indicate two fingers, and 10 indicate fist, 20 tables Show and shakes arm etc.；It can also include: the corresponding control meaning of gesture, such as 1 indicates booting, 10 indicate booting etc.；Also It may include: gesture parameter, such as gesture motion duration etc..When television set receives these gestures, execute corresponding Feature operation.

Fig. 2 is the schematic diagram of VR aobvious equipment of the embodiment of the present invention.As shown in Fig. 2, VR aobvious equipment is in mode of appearance Upper aobvious similar to common VR head, include: (1) image display can show VR figure with liquid crystal display or projection pattern Picture/video；(2) with image/video acquisition capacity camera and infrared distance measurement module, acquired image/video and Position data can be used in carrying out gesture identification；When practical application, more than one camera can be assembled, such as further includes identification The camera of user's eyeball movement；(3) network module provides the network interconnection function with other equipment, such as sets with controlled Standby network connection；(4) gyroscope, for measuring user's head movement data, these data are used to judge the head of user Portion's movement, such as rotary head, bow, come back etc..In addition, VR aobvious equipment further includes some structure members, convenient for users to wearing It wears.

In the present embodiment, VR aobvious equipment can identify that the gesture in its visual range, implementation method are cameras, survey Image is acquired in its three-dimensional space away from module, is analyzed in the image of acquisition, identifies gesture.

As another embodiment, VR aobvious equipment is also possible to connect a kind of wearable device to acquire and identify hand Gesture, such as the watch style equipment for being worn over finger ring type equipment on hand, being worn over gloves equipment on hand, being worn on arm, it is this can Wearable device can recognize that the finger of user, arm action, these movements are transferred to VR aobvious equipment as gesture.

It should be noted that the embodiment of the present invention understands only for facilitating, as the present invention in a manner of VR aobvious equipment Embodiment.In fact, head movement tracking module, eye tracking module, content display module, gesture recognition module, network Module, all kinds of construction modules etc. completely can be by other combinations, to support the function of the embodiment of the present invention.For example, Independent camera tracks and identifies head movement, gesture motion and need not be worn on user head, and content display module is in addition It is installed on diameter of Spherical Volume etc..

Fig. 3 is the schematic diagram of VR aobvious equipment and controlled plant of the embodiment of the present invention.It is controlled set that Fig. 3, which is respectively illustrated, Standby and VR aobvious equipment functional module.Wherein:

There is network unit in controlled plant and VR aobvious equipment, network unit for example supports IEEE802.11b/g/n, or Person supports IEEE 802.3, so as to connect devices to IP network.In the embodiment of the present invention, two equipment pass through network list Member is connected with each other.Include control module in each network unit, can control local device and other equipment send message, receive Message.

Network unit provides mutual discovery on network for equipment, connection, sends message and receive message capability, can be with It is realized using existing UPnP technology, the realization of mDNS and DNS-SD technology also can be used, and this kind of technologies are used in an ip network, In a manner of unicast, multicast inquiry, according to message format predetermined response inquiry, funcall is provided.For example, UPnP technology Define media display devices (such as TV), how server (such as DVD player, family's storage server) responds inquiry, provides Which calling function.

Controlled plant includes storage unit, and the gesture inventory that controlled plant is supported is stored in storage unit, is also stored VR image.VR image can be 360 ° of appearance images of panorama of controlled plant, can be an image file, is also possible to more A file.VR image will be transmitted to VR aobvious equipment from controlled plant by Network Element Function.

Control execution unit in controlled plant executes feature operation predetermined after receiving gesture message, such as Replace channel, adjustment volume, adjustment air-conditioner temperature etc..

Vision data acquisition unit in VR aobvious equipment is used to capture images of gestures (including the piece of video for acquiring user It is disconnected)；Visual identity unit stores algorithm routine, and gesture can be identified from images of gestures, including identify gesture shape, Hand gesture location information, gesture motion direction etc..Image-display units in VR aobvious equipment are used for after receiving VR image, VR image is displayed on the screen or is projected in user at the moment, provides the visual reference of gesture operation for user, such as show The front description of one width television set makes gesture convenient for users to being directed toward the relevant range of television set, expression manipulation television set It is intended to.The head of position detection unit detection user in VR aobvious equipment, eye movement, identify user's viewpoint, for Image-display units adjustment VR image is shown, and a part as gesture information passes to controlled plant.VR aobvious equipment Interior storage unit can store gesture inventory that the controlled plant received is subscribed to, and caching VR image etc..

Below based on the concrete function of above each equipment, the gesture identification method of the embodiment of the present invention is proposed.

Fig. 4 is the flow diagram one of the gesture identification method of the embodiment of the present invention, as shown in figure 4, the method includes Following steps:

Step 401: the first equipment obtains the gesture list that the second equipment is supported.

Specifically, first equipment receives the subscription gesture message that the second equipment is sent, in the subscription gesture message Carry the gesture list that second equipment is supported.

In one embodiment, before first equipment receives the subscription gesture message that the second equipment is sent, the method Further include:

In another embodiment, before first equipment receives the subscription gesture message that the second equipment is sent, the side Method further include:

Step 402: first equipment receives the VR data that second equipment is sent, and shows to the VR data Show.

Step 403: first equipment simultaneously identifies gesture, the gesture list supported based on second equipment got Gesture message is generated, the gesture message carries at least one of following information: gesture information, control information.

In one embodiment, the gesture information includes at least one of: gesture number, gesture parameter, acquisition gesture When position mapping parameters on present image of present image parameter, gesture, continuous acquisition gesture video flowing, predefined Instruction encoding.

In the embodiment of the present invention, according to acquisition images of gestures moment corresponding VR image parameter, images of gestures in VR image In position mapping parameters, the second equipment can calculate the corresponding virtual image of gesture according to these information, and be updated to VR It sends the first equipment in image to be shown, enhances the operating experience of user.

Step 404: the gesture message is sent to second equipment by first equipment.

The corresponding network address of the VR data or the network port are also carried in the subscription gesture message；Described first sets The standby VR data sent according to the corresponding network address of the VR data or the network port, reception second equipment.Wherein, VR The corresponding network address of data or network port network address corresponding from subscription gesture message or the network port can be different.

Fig. 5 is the flow diagram two of the gesture identification method of the embodiment of the present invention, as shown in figure 5, the method includes Following steps:

Step 501: the second equipment sends the gesture list that the second equipment is supported to the first equipment.

Specifically, second equipment is sent to the first equipment subscribes to gesture message, carries in the subscription gesture message The gesture list that second equipment is supported.

In one embodiment, second equipment is sent before subscribing to gesture message to the first equipment, and the method is also wrapped It includes:

In another embodiment, second equipment is sent before subscribing to gesture message to the first equipment, and the method is also Include:

Step 502: second equipment sends VR data to first equipment, so that described in first equipment shows VR data.

At least one of following information is stored in second equipment: the VR data of second electronic equipment, described The gesture list that second equipment is supported, wherein the VR data are VR image data or VR video data；

Step 503: second equipment receives the gesture message that first equipment is sent, and the gesture message carries such as At least one of lower information: gesture information, control information；Believe that message executes corresponding control instruction according to the gesture.

It is described in further detail below with reference to gesture identification method of the concrete application to the embodiment of the present invention.

Fig. 6 is the interacting message schematic diagram of the embodiment of the present invention；In the present embodiment, controlled plant is sent out to VR aobvious equipment The gesture inventory for sending controlled plant to support.As shown in fig. 6, this exemplary process flow includes:

1) controlled plant finds and connect with VR aobvious equipment completion equipment.

The discovery and connection of equipment can be specifically realized by modes such as UPnP, mDNS, manually input IP lookups.Finally, Controlled plant is established with VR aobvious equipment and is connected to the network.

2) controlled plant sends to VR aobvious equipment and subscribes to message, to subscribe to the gesture information of VR aobvious equipment.

Here, subscribing to message is internet message, and message content is gesture capability list.The inventory embodiment of gesture ability, Such as: the gesture number of support, such as the 1 expression the five fingers open, and 2 indicate two fingers, and 10 indicate fists, and 20 indicate shake arm Etc.；It can also include: the corresponding control meaning of gesture, such as 1 indicates booting, 10 indicate booting etc.；It can also include: hand Potential parameter, such as gesture motion duration etc..When controlled plant receives these gestures, corresponding function behaviour is executed Make.

In one example, subscribe to further includes sending the information such as address and the port of VR image in message.

In one example, VR aobvious equipment responds controlled plant, confirms the gesture of support.VR aobvious equipment response is controlled to be set Subscription request standby and that controlled plant is responded to message of the controlled plant transmission including a gesture capability list.

In one example, including video flowing, i.e., controlled plant can also be specified to the gesture information that VR aobvious equipment is subscribed to It is required that VR aobvious equipment acquires associated video stream.

3) controlled plant sends VR image to VR aobvious equipment.

After VR aobvious equipment receives VR image, VR aobvious equipment shows VR content to user.User is in viewing VR figure As during, gesture can be made, VR aobvious equipment captures and identifies the gesture of user.To which user need not face Controlled plant physical entity, also can image planes gesture is equally made to physical entity.

VR aobvious equipment prepares to receive the information such as address and the port of VR image after completing step 3).By Equipment is controlled then after the subscription response message for receiving VR aobvious equipment, i.e., sends VR image to VR aobvious equipment.For efficiency with The considerations of bandwidth, transmission subscribe to message and use different network address or different ports from VR image is sent.

In this embodiment, VR image technique includes: Integral Imaging, tracking user's head movement technology, makes The image display technology of user's eyeball viewpoint.

4) VR aobvious equipment identifies user's gesture.

The camera or wearable device of VR aobvious equipment utilization equipment, capture the gesture of user and are identified as calculating Machine indicates.It after identifying successfully, is compared with the gesture inventory of subscription, if the gesture of identification is the hand in the gesture inventory subscribed to Gesture then turns in next step, otherwise persistently captures gesture and identifies gesture.

5) VR aobvious equipment sends gesture information to controlled plant.

Wherein, gesture information includes following content:

The gesture of support is numbered, such as 1 indicates that the five fingers open, and 2 indicate two fingers, and 10 indicate fist, and 20 indicate to shake Arm etc.；

The corresponding control meaning of gesture, such as 1 expression booting, 10 indicate booting etc.；

Gesture parameter, such as gesture motion duration etc..

Present image parameter: when capture gesture, when the preceding image location data shown to user, such as current frame image Coordinate position of the top left corner pixel point in entire panoramic picture, image size data.

It optionally, can also include position mapping parameters of the gesture on present image.

The gesture information of transmission can also include the video flowing of continuous acquisition, carry out gesture analysis, gesture by controlled plant Other analyses such as position analysis, body gesture analysis.

In addition to above-mentioned gesture information is sent between controlled plant and VR aobvious equipment, as another embodiment, VR aobvious Equipment can predefine gesture, and be mapped as instruction encoding, and this mapping can also be reassigned by user；Such as Mapping a thumb indicates " determination ", and the palm for erecting all fingers indicates " cancellation ", and an index finger indicates " 1 ", etc..It is controlled Equipment can directly subscribe to these mappings, and then improve manipulation experience.

If 6) user persistently makes gesture, VR aobvious equipment continues to send gesture information.

If 7) controlled plant determines to terminate gesture control process, sending to VR aobvious equipment terminates to subscribe to gesture letter Breath disconnects the two connection to terminate gesture control, further includes the display for terminating VR image.To whole process normal terminations.

Fig. 7 is that the event of VR aobvious equipment of the embodiment of the present invention touches action schematic diagram.It is as shown in Figure 7: VR aobvious equipment After starting, by event trigger function, the function of providing in the embodiment of the present invention is executed.Here, event includes: to receive VR Image (stream) detects gesture, detection to the end/eyeball (eye pupil) movement.After detecting event, referring to Fig. 6, VR aobvious equipment touching It sends out program and executes following function: display/play VR image；It identifies gesture, calculates gesture information, send gesture information；Adjustment is aobvious Show/play VR image etc..

Fig. 8 is the flow diagram of VR aobvious equipment of the embodiment of the present invention.Present embodiment describes VR aobvious equipment works The flow chart of work.As shown in figure 8, this exemplary process includes:

1) equipment finds and connect.

VR aobvious equipment can issue inquiry broadcast or multicast message first；Can also in receiving network controlled plant After inquiring broadcast or multicast message, broadcast or multicast message is responded.After completing discovery feature, can actively or passively it establish With the connection of controlled plant.

After establishing connection, as one embodiment, VR aobvious equipment can be used as a HTTP client, with RESTful Mode sends HTTP request to the controlled plant as HTTP server.It certainly, it will be appreciated by those skilled in the art that can also be with Controlled plant is set as HTTP client, VR aobvious equipment is as HTTP server.

2) it receives and subscribes to gesture information.

As one embodiment, after establishing HTTP connection, VR aobvious equipment sends GET request message to controlled plant, Inquire the gesture inventory list of controlled plant.Controlled plant returning response message, VR aobvious equipment are obtained by receiving response message Gesture information must be subscribed to.

Here subscription gesture information includes one group of gesture description information, referring to previous embodiment.

In this step, controlled plant also sends the subsequent address and end for sending virtual reality image to VR aobvious equipment Mouthful.VR aobvious equipment receives and saves these information.VR aobvious equipment establishes network to the address and port for sending VR reality image Connection.It receives VR image and the different network ports can be used from subscription gesture information is received.

3) VR image is received.

As an embodiment, the agreement for transmitting VR image can be streaming media transmission protocol such as HLS, RSTP, DASH, or Person downloads file by HTTP.

4) VR image is shown.

VR aobvious equipment is presented to user by display screen or projection part, the VR image received, or handle needs The picture material to be updated is presented to user.

5) if detection is to the end/pupil movement, it goes to step 4).

VR aobvious equipment calculates user's pose, the image that determination should be shown to user according to detection data.For example, with It is upward using new line/eyeball, then the content of the upper part of image is shown to user.

6) it acquires and identifies gesture information.

VR aobvious equipment captures user by camera or wearable device or other computer vision acquisition components Gesture, and identification gesture is analyzed, to obtain gesture information.

7) gesture information is sent.

Wherein, gesture information includes:

Gesture parameter, such as gesture motion duration etc..

Optionally, the position mapping parameters including gesture on present image.

As one embodiment, VR aobvious equipment can be used HTTP PUT or HTTP POST message and send out to controlled plant Send gesture information.

If 8) do not receive the instruction message of controlled plant, go to step 5).

After VR aobvious equipment receives the response message of controlled plant, if analysis finds not indicate information further, Then continue to go to step 5 and continues to test user's behavior, identification gesture.

If 9) receive the instruction information of controlled plant, if the instruction information instruction of controlled plant will terminate gesture identification, Terminate process, otherwise goes to step 3).

Fig. 9 is the flow diagram of the controlled plant of the embodiment of the present invention.Present embodiment describes the work of controlled plant Flow chart.As shown in figure 9, this exemplary process includes:

1) equipment finds and connect.

Controlled plant can issue inquiry broadcast or multicast message first；Can also in receiving network VR aobvious equipment After inquiring broadcast or multicast message, broadcast or multicast message is responded.After completing discovery feature, can actively or passively it establish With the connection of VR aobvious equipment.

After establishing connection, as one embodiment, controlled plant can be used as a HTTP server, with the side RESTful Formula responds the HTTP request of VR aobvious equipment.Certainly, it will be appreciated by those skilled in the art that controlled plant conduct also can be set HTTP client, VR aobvious equipment is as HTTP server end.Also any other network protocol can be used, such as be based on Special communication packet format is defined in the network protocol of TCP/IP.

2) it sends and subscribes to gesture information.

As one embodiment, after establishing HTTP connection, VR aobvious equipment sends GET request message to controlled plant, Inquire the gesture information list of controlled plant.Controlled plant returning response message is sent by response message to VR aobvious equipment Subscribe to gesture information.

In this step, controlled plant also sends the subsequent address and end for sending virtual reality image to VR aobvious equipment Mouthful.VR aobvious equipment receives and saves these information.Controlled plant shows with VR and sets behind the address and port for sending VR image It is standby to establish network connection.It sends VR image and the different network ports can be used from hair subscription gesture information.

3) VR image is sent.

VR aobvious equipment is established to the address and port for sending VR image and is connected to the network, and here, is received VR image and is received Subscribing to gesture information can be used the different network ports.As one embodiment, the agreement for transmitting VR image can be stream matchmaker Body transport protocol such as HLS, RSTP, DASH, or file is downloaded by HTTP.

It can be the panorama of controlled plant or the outdoor scene photo of multiple angles in VR picture material, can also be controlled and set Standby inside schematic picture, the view image changed by user's manipulation etc..

4) gesture information is received.

The VR waiting aobvious equipment message such as the HTTP server on controlled plant.After VR aobvious equipment identifies gesture, The gesture information identified will be sent to controlled plant, such as is set by HTTP PUT message or HTTP POST message to controlled Preparation send message, and controlled plant receives these gesture informations.

5) according to the gesture information judgement received, virtual reality image is if desired updated, then is turned 3).

As one embodiment, after controlled plant parses the gesture information received, discovery needs to update to user's It shows information, such as the equipment picture more renewed, just goes to step 3, continue to send virtual reality image.

6) and continue analysis and receive gesture information, execute gesture control function.

If after the gesture information that controlled plant parsing receives, determining and needing itself to need to be implemented certain functions, such as electricity Volume is adjusted depending on machine, air-conditioning increase/decrease temperature opens/closes/lightening/and dims light etc., i.e. it is corresponding to execute gesture for starting Control function.

7) if controlled plant determines to terminate gesture control process, sending a message to VR aobvious device request terminates process, Otherwise it goes to step 4).

As one embodiment, final controlled plant closing is connect with the address of VR aobvious equipment, port.

In above-described embodiment, television set, player, storage server are by as controlled plant, and the present invention does not limit It is the such equipment mentioned in embodiment in controlled plant, computer, sound equipment, speaker, projector, set-top box etc. all can serve as Controlled plant, or even industrial other equipment such as automobile, lathe, steamer etc. can be manipulated by VR aobvious equipment.

In above-described embodiment, the camera of VR aobvious equipment can be various specifications, such as can be fixed focal length or change Focal length, revolution space can be all angles up and down, or only support left and right angle, it is only necessary to a camera shooting of configuration Head has various abilities described in embodiment.Laser infrared line ranging can be used in distance measuring unit, can also use other wave bands Light measurement, can be used 3 cameras calculate ranging, also can be used more multi-cam weighting adjustment the methods of calculate survey Away from.

Above-mentioned process is to realize the embodiment of the present invention, is not to limit to be realized with embodiment, nor being used to The method that detailed process executes is limited, the present invention can also realize with similar mode, for example indicates the name of unit, various disappears The type etc. of breath, that is only that name form, specific message content etc. are different.

Above-described embodiment, network is relevant, be applicable to based on IEEE 802.3, IEEE 802.11b/g/n, POWELINE, CABLE, PSTN, third generation partner program (3GPP, 3rd Generation Partnership Project) IP network that the communication networks such as network, 3GPP2 network are supported, the operating system of each device are applicable to UNIX class Operating system, WINDOWS type operating system, ANDROID type operating system, IOS operating system, are applicable to consumer interface JAVA language interface etc..

Figure 10 is the structure composition schematic diagram one of the gesture identifying device of the embodiment of the present invention, is applied to the first equipment, such as Shown in Figure 10, described device includes:

Receiving unit 1000, for obtaining the gesture list for taking the second equipment to support；Receive what second equipment was sent VR data；

Display unit 1001, for being shown to the VR image；

Recognition unit 1002 is acquired, for acquiring and identifying gesture, the hand based on second equipment support got Gesture list generates gesture message, and the gesture message carries at least one of following information: gesture information, control information；

Transmission unit 1003, for the gesture message to be sent to second equipment.

In the embodiment of the present invention, the receiving unit 1000 is also used to receive the subscription gesture message of the second equipment transmission, The gesture list that second equipment is supported is carried in the subscription gesture message.

In the embodiment of the present invention, described device further include:

Connection unit 1004, for going out to inquire broadcast or multicast message, to carry out equipment discovery to the second equipment；Alternatively, First equipment responds the inquiry broadcast or multicast message after the inquiry broadcast or multicast message for receiving the sending of the second equipment； After the first equipment and the second equipment complete equipment discovery, first equipment and second equipment are established and are connected to the network.

In the embodiment of the present invention, connection unit 1004 is also used to set by the network address of the second equipment with described second It is standby to establish network connection.

It will be appreciated by those skilled in the art that the realization function of each unit in gesture identifying device shown in Fig. 10 can join According to aforementioned gesture identification method associated description and understand.

The technical solution of the embodiment of the present invention allows user not need that also hand can be carried out to equipment directly facing controlled plant Gesture control, and it is equally simple and convenient when the more equipment of manipulation, and so that the entertainment of people more easily, is more enjoyed, people are not required to Want the more application methods of learning and mastering, additionally it is possible to reduce enterprise's production cost and customer consumption cost.

Figure 11 is the structure composition schematic diagram two of the gesture identifying device of the embodiment of the present invention, is applied to the second equipment, such as Shown in Figure 11, described device includes:

Transmission unit 1101, for sending the gesture list that the second equipment is supported to the first equipment；To first equipment VR data are sent, so that first equipment shows the VR data；

Receiving unit 1102, the gesture message sent for receiving first equipment, the gesture message carries as follows At least one of information: gesture information, control information；

Execution unit 1103, for executing corresponding control instruction according to the gesture message.

In the embodiment of the present invention, the transmission unit 1101 is also used to send to the first equipment and subscribes to gesture message, described It subscribes to and carries the gesture list that second equipment is supported in gesture message.

In the embodiment of the present invention, described device further include:

Connection unit 1104, for issuing inquiry broadcast or multicast message, to carry out equipment discovery to the first equipment；Or Person, the second equipment respond the inquiry broadcast or multicast and disappear after the inquiry broadcast or multicast message for receiving the sending of the first equipment Breath；After the second equipment and the first equipment complete equipment discovery, second equipment and first equipment are established and are connected to the network.

In the embodiment of the present invention, connection unit 1104 is also used to set by the network address of the first equipment with described first It is standby to establish network connection.

It will be appreciated by those skilled in the art that the realization function of each unit in gesture identifying device shown in Figure 11 can join According to aforementioned gesture identification method associated description and understand.

Figure 12 is the structure composition schematic diagram of the gesture recognition system of the embodiment of the present invention, the system comprises: first sets Standby 1201, second equipment 1202；Wherein,

First equipment 1201, for obtaining the gesture list of the second equipment 1202 support；Receive second equipment The 1202 VR data sent, and the VR data are shown；It acquires and identifies gesture, set based on described second got The standby 1202 gesture lists generation gesture messages supported, at least one of following information of the gesture message carrying: gesture information, Control information；The gesture message is sent to second equipment 1202；

Second equipment 1202, for sending the gesture column that second equipment 1202 is supported to the first equipment 1201 Table；VR data are sent to first equipment 1201, so that first equipment 1201 shows the VR data；Receive described The gesture message that one equipment 1201 is sent, the gesture message carry at least one of following information: gesture information, control letter Breath；Believe that message executes corresponding control instruction according to the gesture.

In the embodiment of the present invention, the gesture list includes at least one of: the hand that second equipment 1202 is supported Gesture number, gesture number corresponding gesture feature data, gesture numbers corresponding gesture control instruction.

The technical solution of the embodiment of the present invention (such as is not equipped with camera to the equipment for not having gesture identification component Equipment), the equipment is faced without user, the equipment can also be controlled by gesture (such as selection function, configuration are joined Number etc.), and cost is relatively low for equipment increase itself.

It should be understood by those skilled in the art that, the embodiment of the present invention can provide as method, system or computer program Product.Therefore, the shape of hardware embodiment, software implementation or embodiment combining software and hardware aspects can be used in the present invention Formula.Moreover, the present invention, which can be used, can use storage in the computer that one or more wherein includes computer usable program code The form for the computer program product implemented on medium (including but not limited to magnetic disk storage and optical memory etc.).

The present invention be referring to according to the method for the embodiment of the present invention, the process of equipment (system) and computer program product Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates, Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or The function of being specified in multiple boxes.

These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one The step of function of being specified in a box or multiple boxes.

The foregoing is only a preferred embodiment of the present invention, is not intended to limit the scope of the present invention.

Claims

1. a kind of gesture identification method, which is characterized in that the described method includes:

First equipment receives the Virtual Reality data that second equipment is sent, and shows to the VR data；

First equipment acquires and identifies gesture, and the gesture list supported based on second equipment got generates gesture Message, the gesture message carry at least one of following information: gesture information, control information；

The gesture message is sent to second equipment by first equipment.

2. gesture identification method according to claim 1, which is characterized in that first equipment obtains the second equipment and supports Gesture list, comprising:

First equipment receives the subscription gesture message that the second equipment is sent, and carries described second in the subscription gesture message The gesture list that equipment is supported.

3. gesture identification method according to claim 2, which is characterized in that first equipment receives the second equipment and sends Subscription gesture message before, the method also includes:

First equipment issues inquiry broadcast or multicast message, to carry out equipment discovery to the second equipment；Alternatively, the first equipment is being received After the inquiry broadcast or multicast message issued to the second equipment, the inquiry broadcast or multicast message is responded；

After the first equipment and the second equipment complete equipment discovery, first equipment and second equipment establish network company It connects.

4. gesture identification method according to claim 2, which is characterized in that first equipment receives the second equipment and sends Subscription gesture message before, the method also includes:

5. gesture identification method according to claim 1, which is characterized in that the gesture list include it is following at least it One: the gesture that second equipment is supported is numbered, gesture numbers corresponding gesture feature data, gesture numbers corresponding gesture control System instruction.

6. gesture identification method according to claim 1, which is characterized in that the gesture information include it is following at least it One: position mapping parameters on present image of present image parameter, gesture when gesture number, gesture parameter, acquisition gesture, Gesture video flowing, the predefined instruction encoding of continuous acquisition.

7. gesture identification method according to claim 1, which is characterized in that be stored with following information in second equipment At least one of: the gesture list that the VR data of second electronic equipment, second equipment are supported, wherein the VR number According to for VR image data or VR video data；

The corresponding network address of the VR data or the network port are also carried in the subscription gesture message；The first equipment root According to the corresponding network address of the VR data or the network port, the VR data that second equipment is sent are received.

8. a kind of gesture identification method, which is characterized in that the described method includes:

Second equipment receives the gesture message that first equipment is sent, and the gesture message carries following information at least One of: gesture information, control information；Believe that message executes corresponding control instruction according to the gesture.

9. gesture identification method according to claim 8, which is characterized in that second equipment sends institute to the first equipment State the gesture list of the second equipment support, comprising:

Second equipment is sent to the first equipment subscribes to gesture message, carries second equipment in the subscription gesture message The gesture list of support.

10. gesture identification method according to claim 9, which is characterized in that second equipment is sent to the first equipment Before subscribing to gesture message, the method also includes:

Second equipment issues inquiry broadcast or multicast message, to carry out equipment discovery to the first equipment；Alternatively, the second equipment is being received After the inquiry broadcast or multicast message issued to the first equipment, the inquiry broadcast or multicast message is responded；

After the second equipment and the first equipment complete equipment discovery, second equipment and first equipment establish network company It connects.

11. gesture identification method according to claim 9, which is characterized in that second equipment is sent to the first equipment Before subscribing to gesture message, the method also includes:

12. gesture identification method according to claim 9, which is characterized in that the gesture list include it is following at least it One: the gesture that second equipment is supported is numbered, gesture numbers corresponding gesture feature data, gesture numbers corresponding gesture control System instruction.

13. gesture identification method according to claim 9, which is characterized in that the gesture information include it is following at least it One: position mapping parameters on present image of present image parameter, gesture when gesture number, gesture parameter, acquisition gesture, Gesture video flowing, the predefined instruction encoding of continuous acquisition.

14. gesture identification method according to claim 9, which is characterized in that be stored with following letter in second equipment At least one of breath: the gesture list that the VR data of second electronic equipment, second equipment are supported, wherein the VR Data are VR image data or VR video data；

15. a kind of gesture identifying device, which is characterized in that be applied to the first equipment, described device includes:

Display unit, for being shown to the VR data；

Recognition unit is acquired, for acquiring and identifying gesture, the gesture list supported based on second equipment got is raw At gesture message, the gesture message carries at least one of following information: gesture information, control information；

Transmission unit, for the gesture message to be sent to second equipment.

16. gesture identifying device according to claim 15, which is characterized in that the receiving unit is also used to receive The subscription gesture message that two equipment are sent, described subscribe to carry the gesture list that second equipment is supported in gesture message.

17. gesture identifying device according to claim 16, which is characterized in that described device further include:

Connection unit, for going out to inquire broadcast or multicast message, to carry out equipment discovery to the second equipment；Alternatively, the first equipment After the inquiry broadcast or multicast message for receiving the sending of the second equipment, the inquiry broadcast or multicast message is responded；It is set first After the standby completion equipment discovery with the second equipment, first equipment and second equipment are established and are connected to the network.

18. gesture identifying device according to claim 16, which is characterized in that described device further include:

19. gesture identifying device according to claim 15, which is characterized in that the gesture list include it is following at least it One: the gesture that second equipment is supported is numbered, gesture numbers corresponding gesture feature data, gesture numbers corresponding gesture control System instruction.

20. gesture identifying device according to claim 15, which is characterized in that the gesture information include it is following at least it One: position mapping parameters on present image of present image parameter, gesture when gesture number, gesture parameter, acquisition gesture, Gesture video flowing, the predefined instruction encoding of continuous acquisition.

21. gesture identifying device according to claim 15, which is characterized in that be stored with following letter in second equipment At least one of breath: the gesture list that the VR data of second electronic equipment, second equipment are supported, wherein the VR Data are VR image data or VR video data；

22. a kind of gesture identifying device, which is characterized in that be applied to the second equipment, described device includes:

Transmission unit, for sending the gesture list that the second equipment is supported to the first equipment；VR number is sent to first equipment According to so that first equipment shows the VR data；

Receiving unit, the gesture message sent for receiving first equipment, the gesture message carry following information extremely It is one of few: gesture information, control information；

23. gesture identifying device according to claim 22, which is characterized in that the transmission unit is also used to first Equipment, which is sent, subscribes to gesture message, and described subscribe to carries the gesture list that second equipment is supported in gesture message.

24. gesture identifying device according to claim 23, which is characterized in that described device further include:

Connection unit, for issuing inquiry broadcast or multicast message, to carry out equipment discovery to the first equipment；Alternatively, second sets For after the inquiry broadcast or multicast message for receiving the sending of the first equipment, the inquiry broadcast or multicast message is responded；Second After equipment and the first equipment complete equipment discovery, second equipment and first equipment are established and are connected to the network.

25. gesture identifying device according to claim 23, which is characterized in that described device further include:

26. gesture identifying device according to claim 22, which is characterized in that the gesture list include it is following at least it One: the gesture that second equipment is supported is numbered, gesture numbers corresponding gesture feature data, gesture numbers corresponding gesture control System instruction.

27. gesture identifying device according to claim 22, which is characterized in that the gesture information include it is following at least it One: position mapping parameters on present image of present image parameter, gesture when gesture number, gesture parameter, acquisition gesture, Gesture video flowing, the predefined instruction encoding of continuous acquisition.

28. gesture identifying device according to claim 22, which is characterized in that be stored with following letter in second equipment At least one of breath: the gesture list that the VR data of second electronic equipment, second equipment are supported, wherein the VR Data are VR image data or VR video data；

29. a kind of gesture recognition system, which is characterized in that the system comprises: the first equipment, the second equipment；Wherein,

First equipment, for obtaining the gesture list of the second equipment support；The VR data that second equipment is sent are received, And the VR data are shown；Acquire and identify gesture, the gesture list supported based on second equipment got Gesture message is generated, the gesture message carries at least one of following information: gesture information, control information；By the gesture Message is sent to second equipment；

Second equipment, for sending the gesture list that second equipment is supported to the first equipment；To first equipment VR data are sent, so that first equipment shows the VR data；The gesture message that first equipment is sent is received, it is described Gesture message carries at least one of following information: gesture information, control information；It is corresponding to believe that message executes according to the gesture Control instruction.

30. gesture recognition system according to claim 29, which is characterized in that the gesture list include it is following at least it One: the gesture that second equipment is supported is numbered, gesture numbers corresponding gesture feature data, gesture numbers corresponding gesture control System instruction.

31. gesture recognition system according to claim 29, which is characterized in that the gesture information include it is following at least it One: position mapping parameters on present image of present image parameter, gesture when gesture number, gesture parameter, acquisition gesture, Gesture video flowing, the predefined instruction encoding of continuous acquisition.