WO2017107445A1 - 在虚拟现实系统中获得沉浸感的方法、系统及智能手套 - Google Patents

在虚拟现实系统中获得沉浸感的方法、系统及智能手套 Download PDF

Info

Publication number
WO2017107445A1
WO2017107445A1 PCT/CN2016/088679 CN2016088679W WO2017107445A1 WO 2017107445 A1 WO2017107445 A1 WO 2017107445A1 CN 2016088679 W CN2016088679 W CN 2016088679W WO 2017107445 A1 WO2017107445 A1 WO 2017107445A1
Authority
WO
WIPO (PCT)
Prior art keywords
gesture
finger
virtual reality
identification information
touch
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2016/088679
Other languages
English (en)
French (fr)
Inventor
张超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Le Holdings Beijing Co Ltd
Leshi Zhixin Electronic Technology Tianjin Co Ltd
Original Assignee
Le Holdings Beijing Co Ltd
Leshi Zhixin Electronic Technology Tianjin Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Le Holdings Beijing Co Ltd, Leshi Zhixin Electronic Technology Tianjin Co Ltd filed Critical Le Holdings Beijing Co Ltd
Priority to US15/249,530 priority Critical patent/US20170185142A1/en
Publication of WO2017107445A1 publication Critical patent/WO2017107445A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/01—Indexing scheme relating to G06F3/01
    • G06F2203/012—Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment

Definitions

  • the invention belongs to the field of virtual reality technology, and in particular relates to a method, a system and a smart glove for obtaining immersion in a virtual reality system.
  • VR virtual reality
  • VR Virtual
  • Reality technology is the use of computers or other intelligent computing devices as the core, combined with photoelectric sensing technology to generate a virtual environment within a specific range of realistic viewing, listening and touch integration.
  • the VR system mainly includes an input device and an output device.
  • Head mounted display (HMD, Head Mount Display) is a virtual reality display output device, shaped like glasses. By sensing human eye activity, accepting instructions, presenting images, and providing various data directly to the user directly on the glasses, it interacts with the corresponding input device. , to enable users to create a separate closed immersive interactive experience.
  • the VR system can implement gesture interaction based on 2D or 3D cameras, but in general, the VR system can display the captured image in real time on the HMD to give the user a sense of immersion, but there is no touch, and the immersion is not Strong, reducing the appeal of products to users.
  • the invention provides a method, a system and a smart glove for obtaining immersion in a virtual reality system, and the touch feeling information is applied to the human hand through the smart hand wearing device, thereby increasing the user's touch feeling, thereby improving the user's body feeling.
  • Interactive immersion and improved product stickiness is provided.
  • the present invention provides a method for obtaining immersion in a virtual reality system, the method comprising:
  • the virtual reality client acquires the gesture of the finger
  • the smart hand wear device converts the received touch feeling demand information into a response action, and applies the response action to a finger corresponding to the identification information, so that the finger generates the response The touch corresponding to the action.
  • the present invention provides a system for obtaining immersion in a virtual reality system, the system comprising:
  • the mobile terminal is provided with an imaging device and a virtual reality client;
  • the mobile terminal is configured to acquire a gesture of the finger by using the virtual reality client when the camera captures a gesture of the user, and determine, by the virtual reality client, whether the finger exists
  • the target preset preset gesture is sent to the smart hand wear device if the target preset gesture is present, and the touch sense demand information corresponding to the target preset gesture and the identification information of the finger are sent to the smart wearable device;
  • the smart hand wear device is configured to convert the received touch feeling demand information into a response action, and apply the response action to a finger corresponding to the identification information, so that the finger generates and The response corresponds to a tactile sensation.
  • the present invention provides a smart glove for implementing the above method for obtaining immersion in a virtual reality system, the smart glove comprising:
  • An information receiving device and a response action generating device An information receiving device and a response action generating device
  • the information receiving device is configured to receive the touch sensing demand information and the identification information of the finger sent by the virtual reality client;
  • the response action generating means is configured to convert the received touch sense demand information into a response action, and apply the response action to a finger corresponding to the identification information, so that the finger generates and The tactile sensation corresponding to the response action.
  • the image capturing device moves the image of the user's finger movement
  • the virtual reality client determines the gesture made by the finger according to the image, and sends the touch sensing demand information corresponding to the gesture to the smart hand.
  • the smart hand wear device converts the touch feeling demand information into a response action, and applies the response action to the finger, so that the finger generates a corresponding feeling to increase the touch feeling in the user's sense of body , thereby improving the user's sense of physical interaction and immersion, and improving product stickiness.
  • 1 is a structural block diagram of a mobile terminal
  • FIG. 2 is a schematic structural diagram of a system for obtaining immersion in a virtual reality system according to a first embodiment of the present invention
  • FIG. 3 is a schematic flowchart of a method for obtaining a immersive feeling in a virtual reality system according to a second embodiment of the present invention
  • FIG. 4 is a schematic flowchart of a method for obtaining a immersive feeling in a virtual reality system according to a third embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of a smart glove according to a fourth embodiment of the present invention.
  • FIG. 1 shows a structural block diagram of a mobile terminal.
  • the method for obtaining the immersion in the virtual reality system provided by the embodiment of the present invention can be applied to the mobile terminal 10 as shown in FIG. 1.
  • the mobile terminal 10 can include, but is not limited to, a smart phone, a notebook, a tablet, and the like.
  • the mobile terminal 10 includes a memory 101, a memory controller 102, one or more (only one shown) processor 103, a peripheral interface 104, a radio frequency module 105, a button module 106, and an audio module 107. And a touch screen 108. These components communicate with one another via one or more communication bus/signal lines 109.
  • FIG. 1 is merely illustrative and does not limit the structure of the mobile terminal.
  • the mobile terminal 10 may also include more or fewer components than those shown in FIG. 1, or have a different configuration than that shown in FIG.
  • the components shown in Figure 1 can be implemented in hardware, software, or a combination thereof.
  • the memory 101 can be used to store software programs and modules, such as program instructions/modules corresponding to the method for obtaining immersion in the virtual reality system in the embodiment of the present invention, and the processor 103 runs the software programs and modules stored in the memory 101. Thereby, various functional applications and data processing are performed, that is, the above-described method in which the mobile terminal 10 obtains immersion in the virtual reality system is realized.
  • Memory 101 can include high speed random access memory and can also include non-volatile memory, such as one or more magnetic storage mobile terminals, flash memory, or other non-volatile solid state memory.
  • memory 101 can further include memory remotely located relative to processor 103, which can be connected to mobile terminal 10 over a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof. Access to the memory 101 by the processor 103 and other possible components can be performed under the control of the memory controller 102.
  • Peripheral interface 104 couples various input/input devices to the CPU and memory 101.
  • the processor 103 runs various software, instructions within the memory 101 to perform various functions of the mobile terminal 10 and perform data processing.
  • peripheral interface 104, processor 103, and memory controller 102 can be implemented in a single chip. In other instances, they can be implemented by separate chips.
  • the radio frequency module 105 is configured to receive and transmit electromagnetic waves, and realize mutual conversion between electromagnetic waves and electric signals, thereby communicating with a communication network or other devices.
  • the radio frequency module 105 can include various existing circuit components for performing these functions, such as an antenna, a radio frequency transceiver, a digital signal processor, an encryption/decryption chip, a Subscriber Identity Module (SIM) card, a memory, and the like.
  • the radio frequency module 105 can communicate with various networks such as the Internet, an intranet, a preset type of wireless network, or communicate with other devices through a preset type of wireless network.
  • the above-mentioned preset type of wireless network may include a cellular telephone network, a wireless local area network, or a metropolitan area network.
  • the above-mentioned preset type of wireless network can use various communication standards, protocols and technologies, including but not limited to global mobile communication systems (Global System for Mobile Communication, GSM), Enhanced Mobile GSM (Enhanced Data GSM) Environment, EDGE), Wideband Code Division Multiple Access (Wideband Code Division Multiple) Access, W-CDMA), Code Division Access (CDMA), Time Division Multiple Access (Time Division Multiple Access) Access, TDMA), Bluetooth, Wireless-Fidelity (WiFi) (eg American Institute of Electrical and Electronics Engineers Standard IEEE) 802.11a, IEEE 802.11b, IEEE 802.11g, and/or IEEE 802.11n), VoIP (Voice over Internet) Protocal, VoIP), Worldwide Interoperability for Microwave Access, Wi-Max), other protocols for mail, instant messaging and short messages, and any other suitable communication protocol.
  • GSM Global System for Mobile Communication
  • EDGE Enhanced Mobile GSM
  • W-CDMA Wideband Code Division Multiple Access
  • CDMA Code Division Access
  • Time Division Multiple Access Time Division Multiple Access
  • the button module 106 provides an interface for the user to make input to the mobile terminal, and the user can cause the mobile terminal 10 to perform different functions by pressing different buttons.
  • the audio module 107 provides an audio interface to the user, which may include one or more microphones, one or more speakers, and audio circuitry.
  • the audio circuit receives sound data from the peripheral interface 104, converts the sound data into electrical information, and transmits the electrical information to the speaker.
  • the speaker converts the electrical information into sound waves that the human ear can hear.
  • the audio circuit also receives electrical information from the microphone, converts the electrical signal to sound data, and transmits the sound data to peripheral interface 104 for further processing. Audio data may be obtained from memory 101 or through radio frequency module 105.
  • the audio data may also be stored in the memory 101 or transmitted through the radio frequency module 105.
  • the audio module 107 can also include a headphone jack for providing an audio interface to the headset or other device.
  • the touch screen 108 provides an output and input interface simultaneously between the mobile terminal and the user.
  • the touch screen 108 displays video output to the user, and the content of the video output may include text, graphics, video, and any combination thereof. Some output results correspond to some user interface objects.
  • the touch screen 108 also receives user input, such as user gestures such as clicks, swipes, etc., so that the user interface objects respond to the user's input.
  • the technique of detecting user input can be based on resistive, capacitive or any other possible touch detection technique.
  • Specific examples of the touch screen 108 display unit include, but are not limited to, a liquid crystal display or a light emitting polymer display.
  • FIG. 2 is a schematic structural diagram of a system for obtaining immersion in a virtual reality system according to a first embodiment of the present invention.
  • the system includes:
  • the mobile terminal 10 and the smart hand wear device 20 that is wirelessly connected to the mobile terminal 10.
  • An imaging device 11 and a virtual reality client 12 are provided in the mobile terminal 10.
  • the virtual reality client 11 is installed and runs in the mobile terminal 10, and the virtual reality client 11 can be a VR. App(application), data interaction with the smart hand wear device 20.
  • the imaging device 12 may be a camera of the mobile terminal 10 or an imaging device that is separately set up independently of other devices.
  • FIG. 2 is an example in which the imaging device is a camera of the mobile terminal 10.
  • the smart hand wear device can be a smart glove, a smart bracelet, or other smart wearable device that can make a human hand feel a touch.
  • Wireless connection methods include WIFI, Bluetooth, infrared and other connection methods.
  • the mobile terminal 10 is configured to acquire a gesture of the finger by the virtual reality client 12 when the camera 11 captures a gesture of the user, where the camera may capture an image of the user's finger and the image may be It is a continuous number of still picture images, or it can be a dynamic video image.
  • the virtual reality client can obtain a gesture of the finger from the image. It can be gestured by one hand or two hands, and the hand-worn device is worn on the hand that makes the gesture. It can be a smart glove or a smart bracelet. In contrast, a smart glove can control five The feeling of the finger is more capable of generating a complex touch than the smart bracelet, and the smart glove is provided with a more complex response action generating device.
  • the smart wristband is worn on the wrist, and it can be configured to be similar to the smart glove by configuring an auxiliary responsive action generating device.
  • the auxiliary responsive action device can be an electronic finger ring that can be placed on each finger, and the electronic finger ring is wireless or The wired way is connected with the smart bracelet, and the smart bracelet can perform a response action through the electronic finger ring, and can also reflect the response action to each finger.
  • the mobile terminal 10 determines, by the virtual reality client 12, whether there is a target preset gesture that is the same as the gesture of the finger, and the mobile terminal is locally provided with a gesture database in which information of multiple gestures is stored, which may be an image of a gesture, a video, or the like. information. If the target preset gesture exists, the touch sensing demand information corresponding to the target preset gesture and the identification information of the finger are sent to the smart hand wearing device 20.
  • the mobile terminal 10 is further configured to query an image of the plurality of preset gestures saved in the gesture database through the virtual reality client 11 and compare the gesture image of the finger with the image of the plurality of preset gestures.
  • the static picture image of the gesture of the plurality of consecutive fingers may be respectively compared with the plurality of consecutive still picture images of the plurality of preset gestures, or may be the dynamic video images of the gesture of the finger respectively The dynamic video images of the plurality of preset gestures are compared.
  • the degree of matching refers to the degree of similarity. If the matching degree of the two gestures is within the preset range, it means that the two gestures have a high degree of similarity, and can be regarded as two identical gestures. Determining whether there is one gesture or a plurality of gestures in the plurality of preset gestures, and the degree of matching with the gesture of the finger is within a preset range, and if there is such one or more gestures, it is confirmed as a target preset gesture. Setting the range of matching can increase the flexibility of determining the gesture of the finger.
  • the mobile terminal 10 is further configured to: if the plurality of target preset gestures exist, acquire a virtual screen that is viewed by the user when the gesture of the finger is made, and determine that the gesture of the finger is pointed in the virtual screen. a virtual item, and combining the physical properties of the virtual item and the gesture of the finger, determining touch feeling demand information corresponding to the target preset gesture, and transmitting the touch feeling demand information and the identification information of the finger to the smart hand
  • the wearer device 20 is worn.
  • a relatively simple preset gesture and touch feeling requirement information may be saved in the gesture database, and the preset gesture may include only a gesture with obvious features and relatively simple gestures, and the corresponding touch sense requirement information is also simple, for example, “ The gesture of gripping the hand into a fist to the front, the corresponding touch feeling demand information is relatively simple, only the vibration of the whole hand.
  • a target preset gesture matching the gesture of the finger can be determined in the plurality of preset gestures by the above simple comparison. If the target preset gesture exists, searching for the touch sense demand information corresponding to the target preset gesture, and sending the found touch sense demand information together with the identification information of the finger to the smart hand wear device, The smart hand wearing device makes it possible to know which finger the touch demand information corresponds to.
  • the gesture database relatively complicated preset gestures and touch feeling demand information can also be saved, that is, there can be multiple preset gestures with small difference in features, and each preset gesture corresponds to more touches.
  • the demand information is related not only to the preset gesture but also to the virtual item touched by the finger.
  • there are multiple "finger click" gestures in the gesture database and each "finger click" gesture corresponds to different touch sensing needs information, and some are simply a sense of vibration, sharpness, and some may be touched.
  • the virtual items are different or have a hot or cold touch.
  • the touch sense demand information corresponding to the gesture of the finger cannot be determined according to the unique target preset gesture, but the touch sense corresponding to the gesture made by the finger of the user needs to be further determined.
  • demand information Specifically, the virtual screen can be obtained by the user when the gesture of the finger is made, and the virtual screen is viewed by the user through an HMD in the VR system or other glasses device having a function of displaying a virtual screen. Further, the human body image of the user is captured by the imaging device, and the height of the user's finger relative to the human body is confirmed. In combination with the virtual image, the virtual object pointed by the gesture of the finger in the virtual screen may be determined, or the user may be acquired.
  • the focus position of the eye when making the gesture of the finger can also determine the virtual item pointed by the gesture of the finger in the virtual screen. For example, in the manner described above, it can be determined that when the user makes a gesture of stroking with five fingers, the virtual item pointed by the gesture in the virtual screen that the user simultaneously sees is an ice sculpture.
  • the accuracy of determining the touch sensing demand information may be improved, and the touch sensing demand information and the finger are
  • the identification information is sent to the smart wearable device.
  • an ice sculpture has a cold physical property, and the user strokes the ice sculpture with five fingers, and it can be determined that the user's five fingers need to be cooled. At this time, the information that the five fingers need to be cooled and the identification information of the five fingers are sent to the smart hand wearing device.
  • the mobile terminal 10 is further configured to send, by using a wireless propagation mode, the correspondence between the touch sensing demand information corresponding to the gesture and the identification information to the smart hand wearing device 20.
  • the wireless transmission mode can be WIFI, Bluetooth, infrared, near field communication and the like.
  • the smart hand wear device 20 is configured to convert the received touch feeling demand information into a response action, and apply the response action to the finger corresponding to the identification information, so that the finger generates a response corresponding to the response action. Touch.
  • the responsive action includes one or more actions of vibration, tightening, heating, and cooling
  • the smart hand wearing device acts on the identification information by one or more actions of vibration, tightening, heating, and cooling. Corresponding on the finger.
  • the smart glove receives the touch feeling demand information, and the touch feeling demand information is that the finger needs to have a tightened feeling, and then converts into a corresponding response action, that is, tightens, and tightens the finger corresponding to the identification information. So that the finger produces a feeling of tightening.
  • the method for obtaining the immersive feeling in the virtual reality system provided by the embodiment, the image of the user's finger motion is captured by the image capturing device, and the virtual reality client determines the gesture made by the finger according to the image, and the touch sensing demand corresponding to the gesture is The information is sent to the smart hand wear device, and the smart hand wear device converts the touch feeling demand information into a response action, and applies the response action to the finger, so that the finger generates a corresponding feeling to increase the user.
  • the feeling of touch in the sense of body thereby improving the user's sense of body interaction and immersion, and improving product stickiness.
  • FIG. 3 is a schematic diagram of an implementation process of a method for obtaining immersion in a virtual reality system according to a second embodiment of the present invention.
  • the method for obtaining immersion in a virtual reality system provided by this embodiment mainly includes the following steps:
  • the virtual reality client acquires the gesture of the finger.
  • the imaging device may be a camera in a mobile terminal, or may be an independently arranged imaging device independent of other devices such as a mobile terminal.
  • the camera device transmits the captured image to the virtual reality client for processing.
  • the virtual reality client acquires the gesture of the finger.
  • the camera device may capture the image.
  • An image of the user's finger moving, the virtual reality client can obtain a gesture of the finger from the image.
  • the image may be a continuous plurality of still picture images or a dynamic video image.
  • the hand-worn device can be worn on the hand that makes the gesture. It can be a smart glove or a smart bracelet. In contrast, a smart glove can control five The feeling of the finger is more capable of generating a complex touch than the smart bracelet, and the smart glove is provided with a more complex response action generating device.
  • the smart wristband is worn on the wrist, and it can be configured to be similar to the smart glove by configuring an auxiliary responsive action generating device.
  • the auxiliary responsive action device can be an electronic finger ring that can be placed on each finger, and the electronic finger ring is wireless or The wired way is connected to the smart bracelet, and the smart bracelet can perform a response action through the electronic ring.
  • the virtual reality client determines whether there is a target preset gesture that is the same as the gesture of the finger.
  • a gesture database is locally disposed in the mobile terminal, where information of a plurality of gestures is stored, which may be information such as images, videos, and the like of the gesture.
  • the virtual reality client can determine, according to the gesture of the finger, whether there is a target preset gesture that is the same as the gesture of the finger in the gesture database.
  • the mobile terminal connects to the smart hand wear device and performs data transmission by using a wireless connection manner, and specifically performs data exchange with the smart hand wear device through a virtual reality client running in the mobile terminal.
  • the touch sense demand information is used to indicate that when the user makes the target preset gesture
  • the touched touch request information and the identification information of the finger are sent to the smart hand wear device, so that the smart hand wear device can know which finger corresponds to the touch demand information, which may be A finger can also be multiple fingers or even the entire hand.
  • the identification information of the finger is unique and is used to identify the identity attribute of the finger.
  • the identification information may be a unique number of the finger.
  • the smart hand wear device converts the received touch feeling demand information into a response action, and applies the response action to the finger corresponding to the identification information, so that the finger generates a touch corresponding to the response action. .
  • the smart glove receives the touch feeling demand information, and the touch sense demand information is that the finger needs to vibrate, and then converts into a corresponding response action, that is, vibration, and applies the vibration to the finger corresponding to the identification information, which can correspond to A finger can also correspond to multiple fingers, making the finger feel vibrating.
  • the method for obtaining the immersive feeling in the virtual reality system provided by the embodiment, the image of the user's finger motion is captured by the image capturing device, and the virtual reality client determines the gesture made by the finger according to the image, and the touch sensing demand corresponding to the gesture is The information is sent to the smart hand wear device, and the smart hand wear device converts the touch feeling demand information into a response action, and applies the response action to the finger, so that the finger generates a corresponding feeling to increase the user.
  • the feeling of touch in the sense of body thereby improving the user's sense of body interaction and immersion, and improving product stickiness.
  • FIG. 4 is a schematic diagram of an implementation process of a method for obtaining immersion in a virtual reality system according to a third embodiment of the present invention.
  • the method for obtaining immersion in a virtual reality system provided by this embodiment mainly includes the following steps:
  • the virtual reality client acquires the gesture of the finger.
  • the camera device transmits the captured image to the virtual reality client for processing, and when the camera device captures the finger of the user to make a gesture, the virtual reality client acquires the gesture of the finger.
  • the imaging device may capture an image of the user's gesture, and the virtual reality client may acquire a gesture of the finger from the image.
  • the image may be a continuous plurality of still picture images or a dynamic video image.
  • the number of the fingers is not limited, and may be a gesture made by one finger, or may be a gesture made by a plurality of fingers together. For example, a finger click, or a five-finger fist hit.
  • the virtual reality client queries an image of the plurality of preset gestures saved in the gesture database, and compares the gesture image of the finger with the image of the plurality of preset gestures.
  • An image of a plurality of preset gestures may be saved in the gesture database, and the image of each preset gesture may be a continuous plurality of still image images or a dynamic video image.
  • S403. Determine whether there is a target preset gesture in which the matching degree of the gesture of the finger is within a preset range among the plurality of preset gestures.
  • the degree of matching refers to the degree of similarity. If the matching degree of the two gestures is within the preset range, it means that the two gestures have a high degree of similarity, and can be regarded as two identical gestures.
  • the flexibility of judging the gesture of the finger can be improved.
  • a relatively simple preset gesture and touch feeling requirement information may be saved, and the preset gesture may include only a gesture with obvious features and relatively simple gestures, and the corresponding touch sense requirement information is also simple, for example, The gesture of "putting the hand into a fist and hitting the front", the corresponding touch feeling demand information is relatively simple, only the vibration of the whole hand, and, for example, the gesture of "twisting the door lock", the corresponding The touch feeling demand information may also be the vibration feeling of the whole hand, but the vibration feeling is weaker than the shock feeling when the above hit.
  • a target preset gesture can be determined among the plurality of preset gestures through the above steps S403-S404. If the target preset gesture exists, searching for the touch sense demand information corresponding to the target preset gesture, and sending the found touch sense demand information together with the identification information of the finger to the smart hand wear device, The smart hand wearing device makes it possible to know which finger the touch demand information corresponds to.
  • the gesture database relatively complicated preset gestures and touch feeling demand information can also be saved, that is, there can be multiple preset gestures with small difference in features, and each preset gesture corresponds to more touch feelings.
  • the demand information, the touch feeling demand information is related not only to the preset gesture but also to the virtual item touched by the finger.
  • there are multiple "finger click" gestures in the gesture database and each "finger click" gesture corresponds to different touch sensing needs information, and some are simply a sense of vibration, sharpness, and some may be touched.
  • the virtual items are different or have a hot or cold touch.
  • the touch sense demand information corresponding to the gesture of the finger cannot be determined according to the unique target preset gesture, but the touch sense corresponding to the gesture made by the finger of the user needs to be further determined.
  • demand information Specifically, the virtual screen viewed by the user when making the gesture of the finger is obtained, and the virtual screen is viewed by the user through an HMD in the VR system or other glasses device having a function of displaying a virtual screen. Further, the human body image of the user is captured by the imaging device, and the height of the user's finger relative to the human body is confirmed. In combination with the virtual image, the virtual object pointed by the gesture of the finger in the virtual screen may be determined, or the user may be acquired.
  • the focus position of the eye when making the gesture of the finger can also determine the virtual item pointed by the gesture of the finger in the virtual screen. For example, in the manner described above, it can be determined that when the user makes a five-finger gesture, the virtual item pointed to by the gesture in the virtual screen that the user simultaneously sees is a fireplace.
  • the accuracy of determining the touch sensing demand information may be improved, and the touch sensing demand information and the finger are
  • the identification information is sent to the smart wearable device.
  • the fireplace has a physical property of heat, and the user opens his fingers to determine that the user's five fingers need to be heated. At this time, the information that the five fingers need to be heated and the identification information of the five fingers are sent to the smart hand wearing device.
  • the smart hand wear device converts the received touch feeling demand information into a response action, and applies the response action to the finger corresponding to the identification information, so that the finger generates a touch corresponding to the response action. .
  • the smart hand wear device converts the received touch feeling demand information into a response action, and the response action may include one or several actions of vibration, tightening, heating, and cooling, and the converted response action
  • the finger corresponding to the identification information includes, for example, one or more actions of shaking, tightening, heating, and cooling, and acting on the finger corresponding to the identification information.
  • the smart glove receives the touch feeling demand information, and the touch feeling demand information is that the finger needs to have a tightened feeling, and then converts into a corresponding response action, that is, tightens, and tightens the finger corresponding to the identification information. So that the finger produces a feeling of tightening.
  • the method for obtaining the immersive feeling in the virtual reality system provided by the embodiment, the image of the user's finger motion is captured by the image capturing device, and the virtual reality client determines the gesture made by the finger according to the image, and the touch sensing demand corresponding to the gesture is The information is sent to the smart hand wear device, and the smart hand wear device converts the touch feeling demand information into a response action, and applies the response action to the finger, so that the finger generates a corresponding feeling to increase the user.
  • the feeling of touch in the sense of body thereby improving the user's sense of body interaction and immersion, and improving product stickiness.
  • FIG. 5 is a schematic structural diagram of a smart glove according to a fourth embodiment of the present invention.
  • the smart glove 50 illustrated in FIG. 5 is used to implement the method for obtaining the immersion in the virtual reality system provided by the foregoing embodiment shown in FIG. 3 to FIG. 4 .
  • the smart glove 50 illustrated in FIG. 5 is applied to the VR system and wirelessly connected with other related devices in the VR system for data exchange.
  • the smart glove 50 and the mobile terminal are connected through WIFI.
  • the smart glove 50 may include an information receiving device 51 and a response action generating device 52.
  • the information receiving device 51 is configured to receive the touch sensing demand information and the finger identification information sent by the virtual reality client, and the response action generating device 52 is configured to convert the received touch sensing demand information into a response action, and A response action is applied to the finger corresponding to the identification information to cause the finger to generate a tactile sensation corresponding to the response action.
  • Embodiments of the present invention provide an apparatus for obtaining immersion in a virtual reality system, the apparatus comprising: one or more processors;
  • One or more programs the one or more programs being stored in the memory, when executed by the one or more processors:
  • the virtual reality client acquires a gesture of the finger, and the virtual reality client determines whether there is a target preset gesture that is the same as the gesture of the finger, and if the target preset gesture exists, Sending the touch sensing demand information corresponding to the target preset gesture and the identification information of the finger to the smart hand wearing device, and the smart hand wearing device converts the received touch sensing demand information into a response action, and The response action is applied to the finger corresponding to the identification information, so that the finger generates a tactile sensation corresponding to the response action.
  • the devices in the various embodiments of the present invention may be integrated into one processing device, or each device may exist physically separately, or two or more devices may be integrated into one device.
  • the above integrated device can be implemented in the form of hardware or in the form of a software function device.
  • the integrated device if implemented in the form of a software functional device and sold or used as a standalone product, may be stored in a computer readable storage medium.
  • the technical solution of the present invention which is essential or contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product stored in a storage medium.
  • a number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read only memory (ROM, Read-Only) Memory, random access memory (RAM), disk or optical disk, and other media that can store program code.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

一种在虚拟现实系统中获得沉浸感的方法、系统及智能手套，该方法包括：当摄像设备拍摄到用户的手指作出手势时，虚拟现实客户端获取该手指的手势（S301）；判断是否存在与该手指的手势相同的目标预置手势（S302）；若存在，则将该目标预置手势对应的触碰感需求信息和该手指的标识信息发送给智能手部穿戴设备（S303）；该智能手部穿戴设备将接收到的该触碰感需求信息转换成响应动作并作用在对应的手指上，以使手指产生与响应动作对应的触感（S304）。所述方法、系统及智能手套通过智能手部穿戴设备将触碰感信息作用在人体手部，增加用户的触碰感，从而提高用户的体感交互沉浸感。

Description

在虚拟现实系统中获得沉浸感的方法、系统及智能手套
本申请要求在2015年12月25日提交中国专利局、申请号201510995889.6、发明名称为“在虚拟现实系统中获得沉浸感的方法、系统及智能手套”的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本发明属于虚拟现实技术领域，尤其涉及一种在虚拟现实系统中获得沉浸感的方法、系统及智能手套。
 背景技术
发明人在实现本发明的过程中发现虚拟现实（VR，Virtual Reality）技术是利用电脑或其他智能计算设备为核心，结合光电传感技术生成逼真的视、听、触一体化的特定范围内虚拟的环境。在VR系统中主要包括输入设备和输出设备。而头戴式显示器（HMD，Head Mount Display）是一种虚拟现实显示输出设备，形如眼镜，通过感应人的眼部活动，接受指令，呈现图像，可直接在眼镜上现实各种数据提供给用户，它配合对应的输入设备的交互，能够让用户产生独立封闭的沉浸式交互体验。
在现有技术中，VR系统中基于2D或者3D摄像头，均能实现手势交互，但是通常，VR系统可以将采集的图像实时显示在HMD上给予用户沉浸感，但是没有触碰感，沉浸感不强，降低产品对用户的吸引力。
 技术问题
本发明提供一种在虚拟现实系统中获得沉浸感的方法、系统及智能手套，通过智能手部穿戴设备将触碰感信息作用在人体手部，增加用户的触碰感，从而提高用户的体感交互沉浸感，提高产品粘性。
技术解决方案
本发明提供一种在虚拟现实系统中获得沉浸感的方法，所述方法包括：
当摄像设备拍摄到用户的手指作出手势时，虚拟现实客户端获取所述手指的手势；
所述虚拟现实客户端判断是否存在与所述手指的手势相同的目标预置手势；
若存在所述目标预置手势，则将所述目标预置手势对应的触碰感需求信息和所述手指的标识信息发送给智能手部穿戴设备；
所述智能手部穿戴设备将接收到的所述触碰感需求信息转换成响应动作，并将所述响应动作作用在所述标识信息对应的手指上，以使所述手指产生与所述响应动作对应的触感。
本发明提供一种在虚拟现实系统中获得沉浸感的系统，所述系统包括：
移动终端和智能手部穿戴设备；
其中，所述移动终端内设置有摄像设备和虚拟现实客户端；
所述移动终端，用于当摄像设备拍摄到用户的手指作出手势时，通过所述虚拟现实客户端获取所述手指的手势，以及，通过所述虚拟现实客户端判断是否存在与所述手指的手势相同的目标预置手势，若存在所述目标预置手势，则将所述目标预置手势对应的触碰感需求信息和所述手指的标识信息发送给所述智能手部穿戴设备；
所述智能手部穿戴设备，用于将接收到的所述触碰感需求信息转换成响应动作，并将所述响应动作作用在所述标识信息对应的手指上，以使所述手指产生与所述响应动作对应的触感。
本发明提供一种智能手套，所述智能手套用于实现上述在虚拟现实系统中获得沉浸感的方法，所述智能手套包括：
信息接收装置和响应动作发生装置；
所述信息接收装置，用于接收所述虚拟现实客户端发送的所述触碰感需求信息和所述手指的标识信息；
所述响应动作发生装置，用于将接收到的所述触碰感需求信息转换成响应动作，并将所述响应动作作用在所述标识信息对应的手指上，以使所述手指产生与所述响应动作对应的触感。
 有益效果
从上述本发明实施例可知，本发明通过摄像设备拍摄用户手指运动的图像，虚拟现实客户端根据该图像确定该手指所作的手势，并将该手势对应的触碰感需求信息发送给智能手部穿戴设备，该智能手部穿戴设备将该触碰感需求信息转换成响应动作，并将该响应动作作用在该手指上，以使该手指产生对应的感觉，以增加用户体感中的触碰感，从而提高用户的体感交互沉浸感，提高产品粘性。
 附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为一种移动终端的结构框图；
图2是本发明第一实施例提供的在虚拟现实系统中获得沉浸感的系统的结构示意图；
图3是本发明第二实施例提供的在虚拟现实系统中获得沉浸感的方法实现流程示意图；
图4是本发明第三实施例提供的在虚拟现实系统中获得沉浸感的方法实现流程示意图；
图5是本发明第四实施例提供的智能手套的结构示意图。
 本发明的实施方式
为使得本发明的发明目的、特征、优点能够更加的明显和易懂，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而非全部实施例。基于本发明中的实施例，本领域技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
图1示出了一种移动终端的结构框图。本发明实施例提供的在虚拟现实系统中获得沉浸感的方法可应用于如图1所示的移动终端10中，移动终端10可以但不限于包括：智能手机、笔记本、平板电脑等。
如图1所示，移动终端10包括存储器101、存储控制器102，一个或多个（图中仅示出一个）处理器103、外设接口104、射频模块105、按键模块106、音频模块107以及触控屏幕108。这些组件通过一条或多条通讯总线/信号线109相互通讯。
可以理解，图1所示的结构仅为示意，其并不对移动终端的结构造成限定。移动终端10还可包括比图1所示更多或者更少的组件，或者具有与图1所示不同的配置。图1所示的各组件可以采用硬件、软件或其组合实现。
存储器101可用于存储软件程序以及模块，如本发明实施例中的在虚拟现实系统中获得沉浸感的方法对应的程序指令/模块，处理器103通过运行存储在存储器101内的软件程序以及模块，从而执行各种功能应用以及数据处理，即实现上述的移动终端10在虚拟现实系统中获得沉浸感的方法。
存储器101可包括高速随机存储器，还可包括非易失性存储器，如一个或者多个磁性存储移动终端、闪存、或者其他非易失性固态存储器。在一些实例中，存储器101可进一步包括相对于处理器103远程设置的存储器，这些远程存储器可以通过网络连接至移动终端10。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。处理器103以及其他可能的组件对存储器101的访问可在存储控制器102的控制下进行。
外设接口104将各种输入/输入装置耦合至CPU以及存储器101。处理器103运行存储器101内的各种软件、指令以执行移动终端10的各种功能以及进行数据处理。
在一些实施例中，外设接口104，处理器103以及存储控制器102可以在单个芯片中实现。在其他一些实例中，他们可以分别由独立的芯片实现。
射频模块105用于接收以及发送电磁波，实现电磁波与电信号的相互转换，从而与通讯网络或者其他设备进行通讯。射频模块105可包括各种现有的用于执行这些功能的电路元件，例如，天线、射频收发器、数字信号处理器、加密/解密芯片、用户身份模块（SIM）卡、存储器等等。射频模块105可与各种网络如互联网、企业内部网、预置类型的无线网络进行通讯或者通过预置类型的无线网络与其他设备进行通讯。上述的预置类型的无线网络可包括蜂窝式电话网、无线局域网或者城域网。上述的预置类型的无线网络可以使用各种通信标准、协议及技术，包括但并不限于全球移动通信系统（Global System for Mobile Communication，GSM），增强型移动通信技术（Enhanced Data GSM Environment，EDGE)，宽带码分多址技术（Wideband Code Division Multiple Access，W-CDMA），码分多址技术（Code Division Access，CDMA），时分多址技术（Time Division Multiple Access，TDMA），蓝牙，无线保真技术（Wireless-Fidelity，WiFi）（如美国电气和电子工程师协会标准 IEEE 802.11a、IEEE 802.11b、IEEE802.11g 和/或 IEEE 802.11n），网络电话（Voice over Internet Protocal, VoIP），全球微波互联接入（Worldwide Interoperability for Microwave Access，Wi-Max），其他用于邮件、即时通讯及短消息的协议，以及任何其他合适的通讯协议。
按键模块106提供用户向移动终端进行输入的接口，用户可以通过按下不同的按键以使移动终端10执行不同的功能。
音频模块107向用户提供音频接口，其可包括一个或多个麦克风、一个或者多个扬声器以及音频电路。音频电路从外设接口104处接收声音数据，将声音数据转换为电信息，将电信息传输至扬声器。扬声器将电信息转换为人耳能听到的声波。音频电路还从麦克风处接收电信息，将电信号转换为声音数据，并将声音数据传输至外设接口104中以进行进一步的处理。音频数据可以从存储器101处或者通过射频模块105获取。此外，音频数据也可以存储至存储器101中或者通过射频模块105进行发送。在一些实例中，音频模块107还可包括一个耳机播孔，用于向耳机或者其他设备提供音频接口。
触控屏幕108在移动终端与用户之间同时提供一个输出及输入界面。具体地，触控屏幕108向用户显示视频输出，这些视频输出的内容可包括文字、图形、视频、及其任意组合。一些输出结果是对应于一些用户界面对象。触控屏幕108还接收用户的输入，例如用户的点击、滑动等手势操作，以便用户界面对象对这些用户的输入做出响应。检测用户输入的技术可以是基于电阻式、电容式或者其他任意可能的触控检测技术。触控屏幕108显示单元的具体实例包括但并不限于液晶显示器或发光聚合物显示器。
请参阅图2，图2为本发明第一实施例提供的在虚拟现实系统中获得沉浸感的系统的结构示意图。该系统包括：
移动终端10和与移动终端10通过无线方式连接的智能手部穿戴设备20。移动终端10内设置有摄像设备11和虚拟现实客户端12。
其中，虚拟现实客户端11安装并运行于移动终端10中，虚拟现实客户端11可以为VR app（application），与智能手部穿戴设备20进行数据交互。
摄像设备12可为移动终端10的摄像头，也可以是独立于其他设备而单独设立的摄像设备，图2以摄像设备为移动终端10的摄像头为例。
该智能手部穿戴设备可以是智能手套、智能手环，或者其他可以使得人体手部产生触碰感的智能穿戴设备。
无线连接方式包括WIFI、蓝牙、红外等连接方式。
移动终端10，用于当摄像设备11拍摄到用户的手指作出手势时，通过虚拟现实客户端12获取该手指的手势，具体可以是该摄像设备拍摄到用户的手指进行运动的图像，该图像可以是连续多张的静态图片图像，也可以是动态视频图像。
该虚拟现实客户端可从该图像中获取将该手指的手势。可以是一只手或两只手作出手势，作出手势的手上穿戴有该智能手部穿戴设备，它可以是智能手套，也可以是智能手环，相对的，一个智能手套因可控制五个手指的感觉，比智能手环更能够产生复杂的触碰感，智能手套设置有功能更为复杂的响应动作发生装置。而智能手环佩戴在手腕上，它可以通过配置辅助的响应动作发生装置达到和智能手套相近的功能，该辅助的响应动作装置可以是可套在各手指的电子指环，这些电子指环通过无线或有线的方式与智能手环连接，智能手环可通过电子指环执行响应动作，也可以将响应动作具体化到各个手指。
移动终端10通过虚拟现实客户端12判断是否存在与该手指的手势相同的目标预置手势，在移动终端本地设置有手势数据库，其中存储有多个手势的信息，可以是手势的图像、视频等信息。若存在该目标预置手势，则将该目标预置手势对应的触碰感需求信息和该手指的标识信息发送给智能手部穿戴设备20。
具体地，移动终端10，还用于通过虚拟现实客户端11查询手势数据库中保存的多个预置手势的图像，并对比该手指的手势图像与该多个预置手势的图像。可以是将多张连续的该手指的手势的静态图片图像，分别与该多个预置手势的多张连续的静态图片图像进行对比，也可以是将该手指的手势的动态视频图像，分别与该多个预置手势的动态视频图像进行对比。
进一步地，判断该多个预置手势中是否存在与该手指的手势的匹配度在预置范围内的目标预置手势。匹配度，是指相似的程度。若两个手势的匹配度在预置范围内，则表示该两个手势相似程度较高，可以认定是两个相同的手势。判断该多个预置手势中是否存在一个手势或多个手势，与该手指的手势的匹配度在预置范围内，若存在这样的一个或多个手势，即将它或它们确认为目标预置手势。设置匹配度的范围可以提高判断该手指的手势的灵活性。
进一步地，移动终端10，还用于若存在该多个目标预置手势，则获取该用户在作出该手指的手势时观看到的虚拟画面，判断该手指的手势在该虚拟画面中所指向的虚拟物品，以及，结合该虚拟物品的物理性质以及该手指的手势，确定该目标预置手势对应的触碰感需求信息，并将该触碰感需求信息和该手指的标识信息发送给智能手部穿戴设备20。
具体地，在手势数据库中可以保存相对简单的预置手势和触碰感需求信息，预置手势可以只包括特征明显、构成相对简单的手势，对应的触碰感需求信息也简单，例如，“将手握成拳向前方击打”的手势，其对应的触碰感需求信息也相对简单，只有整只手的震动感。
在这种预置手势相对简单的情况下，可通过上述简单的对比即可在该多个预置手势中判断出一个和该手指的手势匹配的目标预置手势。若存在该目标预置手势，则查找与该目标预置手势对应的触碰感需求信息，并将查找到的该触碰感需求信息和该手指的标识信息一起发送给智能手部穿戴设备，使得该智能手部穿戴设备可知该触碰需求信息是与哪个手指对应的。
另一方面，在手势数据库中也可以保存相对繁杂的预置手势和触碰感需求信息，即，可以有多个特征差别很小的预置手势，每个预置手势对应更多的触碰感需求信息，该触碰感需求信息不仅和预置手势相关，也和手指触碰的虚拟物品相关。例如，手势数据库中存在多个“手指点击”的手势，每个“手指点击”的手势对应的触碰感需求信息不同，有的只是简单的震动感、尖锐感，有的可因被触碰的虚拟物品不同而产生的或热或冷的触摸感。
当存在多个目标预置手势时，无法根据唯一的目标预置手势确定与该手指的手势对应的触碰感需求信息，而是需要进一步确定与该用户的手指所作的手势对应的触碰感需求信息。具体可通过以下方式实现，获取该用户在作出该手指的手势时观看到的虚拟画面，该虚拟画面是该用户通过VR系统中的HMD或其他具备显示虚拟画面功能的眼镜设备观看到的。进一步地，通过摄像装置拍摄该用户的人体图像，确认该用户手指相对于人体的高度，结合该虚拟画面，可判断该手指的手势在该虚拟画面中所指向的虚拟物品，或者，获取该用户在作出该手指的手势时眼睛的焦点位置，也可判断该手指的手势在该虚拟画面中所指向的虚拟物品。例如，通过前面所述的方式，可以判断出该用户在作出用五个手指抚摸的手势时，该手势在该用户同时看到的虚拟画面中所指向的虚拟物品是一个冰雕。
结合该虚拟物品的物理性质以及该手指的手势，确定目标预置手势对应的触碰感需求信息，可以提高确定该触碰感需求信息的准确性，并将该触碰感需求信息和该手指的标识信息发送给该智能手部穿戴设备。例如，冰雕具有寒凉的物理属性，该用户用五个手指抚摸该冰雕，可以确定该用户的五个手指需要冷却。此时，将五个手指需要冷却的信息和该五个手指的标识信息发送给该智能手部穿戴设备。
移动终端10，还用于通过无线传播方式，将该手势对应的触碰感需求信息和该标识信息的对应关系发送给智能手部穿戴设备20。
该无线传播方式可以是WIFI、蓝牙、红外、近场通信等方式。
智能手部穿戴设备20，用于将接收到的该触碰感需求信息转换成响应动作，并将该响应动作作用在该标识信息对应的手指上，以使该手指产生与该响应动作对应的触感。
该响应动作包括震动、收紧、加热、冷却中的一种或几种动作，该智能手部穿戴设备将震动、收紧、加热、冷却中的一种或几种动作，作用在该标识信息对应的手指上。
例如，智能手套接收该触碰感需求信息，该触碰感需求信息是该手指需要有收紧的感触，则转换成对应的响应动作，即收紧，并收紧该标识信息对应的手指上，使得该手指产生收紧的感觉。
本实施例提供的在虚拟现实系统中获得沉浸感的方法，通过摄像设备拍摄用户手指运动的图像，虚拟现实客户端根据该图像确定该手指所作的手势，并将该手势对应的触碰感需求信息发送给智能手部穿戴设备，该智能手部穿戴设备将该触碰感需求信息转换成响应动作，并将该响应动作作用在该手指上，以使该手指产生对应的感觉，以增加用户体感中的触碰感，从而提高用户的体感交互沉浸感，提高产品粘性。
请参阅图3，图3为本发明第二实施例提供的在虚拟现实系统中获得沉浸感的方法的实现流程示意图。本实施例提供的在虚拟现实系统中获得沉浸感的方法主要包括以下步骤：
S301、 当摄像设备拍摄到用户手指作出手势时，虚拟现实客户端获取该手指的手势。
该摄像设备可以是移动终端中的摄像头，也可以使独立于移动终端等其他设备的独立设置的摄像设备。该摄像设备将拍摄到的图像传给虚拟现实客户端进行处理，当该摄像设备拍摄到用户的手指作出手势时，虚拟现实客户端获取该手指的手势，具体地，可以是该摄像设备拍摄到用户的手指进行运动的图像，该虚拟现实客户端可从该图像中获取将该手指的手势。该图像可以是连续多张的静态图片图像，也可以是动态视频图像。
可以是一只手或两只手作出手势，作出手势的手上穿戴有该智能手部穿戴设备，它可以是智能手套，也可以是智能手环，相对的，一个智能手套因可控制五个手指的感觉，比智能手环更能够产生复杂的触碰感，智能手套设置有功能更为复杂的响应动作发生装置。而智能手环佩戴在手腕上，它可以通过配置辅助的响应动作发生装置达到和智能手套相近的功能，该辅助的响应动作装置可以是可套在各手指的电子指环，这些电子指环通过无线或有线的方式与智能手环连接，智能手环可通过电子指环执行响应动作。
S302、该虚拟现实客户端判断是否存在与该手指的手势相同的目标预置手势。
在移动终端本地设置有手势数据库，其中存储有多个手势的信息，可以是手势的图像、视频等信息。
该虚拟现实客户端可根据该手指的手势，判断在手势数据库中是否存在与该手指的手势相同的目标预置手势。
S303、若存在该目标预置手势，则将该目标预置手势对应的触碰感需求信息和该手指的标识信息发送给智能手部穿戴设备。
需要说明的是，移动终端以无线连接方式与智能手部穿戴设备进行连接并进行数据传输，具体是通过运行在该移动终端中的虚拟现实客户端与该智能手部穿戴设备进行数据交换。
若存在与该手指的手势相同的目标预置手势，则查找与该目标预置手势对应的触碰感需求信息，该触碰感需求信息用于表示当用户做出该目标预置手势时可能获得的触碰感的种类。例如，该触碰感需求信息是需要震动。
进一步地，将查找到的该触碰感需求信息和该手指的标识信息一起发送给智能手部穿戴设备，使得该智能手部穿戴设备可知该触碰需求信息是与哪个手指对应的，可以是一个手指，也可以是多个手指，甚至整个手部。该手指的标识信息具有唯一性，用于标识该手指的身份属性，例如，该标识信息可以是该手指的唯一编号。
S304、该智能手部穿戴设备将接收到的该触碰感需求信息转换成响应动作，并将该响应动作作用在该标识信息对应的手指上，以使该手指产生与该响应动作对应的触感。
例如，智能手套接收该触碰感需求信息，该触碰感需求信息是该手指需要震动，则转换成对应的响应动作，即震动，并将震动作用在该标识信息对应的手指上，可以对应一个手指，也可以对应多个手指，使得手指产生震动的感觉。
本实施例提供的在虚拟现实系统中获得沉浸感的方法，通过摄像设备拍摄用户手指运动的图像，虚拟现实客户端根据该图像确定该手指所作的手势，并将该手势对应的触碰感需求信息发送给智能手部穿戴设备，该智能手部穿戴设备将该触碰感需求信息转换成响应动作，并将该响应动作作用在该手指上，以使该手指产生对应的感觉，以增加用户体感中的触碰感，从而提高用户的体感交互沉浸感，提高产品粘性。
请参阅图4，图4为本发明第三实施例提供的在虚拟现实系统中获得沉浸感的方法的实现流程示意图。本实施例提供的在虚拟现实系统中获得沉浸感的方法主要包括以下步骤：
S401、当摄像设备拍摄到用户手指作出手势时，虚拟现实客户端获取该手指的手势。
该摄像设备将拍摄到的图像传给虚拟现实客户端进行处理，当该摄像设备拍摄到用户的手指作出手势时，虚拟现实客户端获取该手指的手势。具体可以是该摄像设备拍摄到用户的作出手势的图像，该虚拟现实客户端可从该图像中获取将该手指的手势。该图像可以是连续多张的静态图片图像，也可以是动态视频图像。该手指的个数不限，可以是一个手指的作出的手势，也可以是多个手指一起作出的手势。例如，一只手指的点击，或者，五只手指握拳的击打。
S402、该虚拟现实客户端查询手势数据库中保存的多个预置手势的图像，并对比该手指的手势图像与该多个预置手势的图像。
在手势数据库中可保存多个预置手势的图像，每个预置手势的图像可以是连续多张的静态图片图像，也可以是动态视频图像。
将该手指的手势图像与该多个预置手势的图像进行对比，例如，将多张连续的该手指的手势的静态图片图像，分别与该多个预置手势的多张连续的静态图片图像进行对比，或者，将该手指的手势的动态视频图像，分别与该多个预置手势的动态视频图像进行对比。
S403、判断该多个预置手势中是否存在与该手指的手势的匹配度在预置范围内的目标预置手势。
匹配度，是指相似的程度。若两个手势的匹配度在预置范围内，则表示该两个手势相似程度较高，可以认定是两个相同的手势。
判断该多个预置手势中是否存在一个手势或多个手势，是与该手指的手势的匹配度在预置范围内的，若存在，则这样的一个或多个手势即为目标预置手势。为确认手势之间是否匹配而设置一个匹配度的范围，可以提高判断该手指的手势的灵活性。
S404、若存在该目标预置手势，则将该目标预置手势对应的触碰感需求信息和该手指的标识信息发送给智能手部穿戴设备。
需要说明的是，在手势数据库中可以保存相对简单的预置手势和触碰感需求信息，预置手势可以只包括特征明显、构成相对简单的手势，对应的触碰感需求信息也简单，例如，“将手握成拳向前方击打”的手势，其对应的触碰感需求信息也相对简单，只有整只手的震动感，再如，“扭房门锁”的手势，其对应的触碰感需求信息也可以是整只手的震动感，只是震动感弱于前述击打时的震动感。
在这种预置手势相对简单的情况下，通过上述步骤S403~S404可在该多个预置手势中判断出一个目标预置手势。若存在该目标预置手势，则查找与该目标预置手势对应的触碰感需求信息，并将查找到的该触碰感需求信息和该手指的标识信息一起发送给智能手部穿戴设备，使得该智能手部穿戴设备可知该触碰需求信息是与哪个手指对应的。
相对的，在手势数据库中也可以保存相对繁杂的预置手势和触碰感需求信息，即，可以有多个特征差别很小的预置手势，每个预置手势对应更多的触碰感需求信息，该触碰感需求信息不仅和预置手势相关，也和手指触碰的虚拟物品相关。例如，手势数据库中存在多个“手指点击”的手势，每个“手指点击”的手势对应的触碰感需求信息不同，有的只是简单的震动感、尖锐感，有的可因被触碰的虚拟物品不同而产生的或热或冷的触摸感。
当存在多个目标预置手势时，无法根据唯一的目标预置手势确定与该手指的手势对应的触碰感需求信息，而是需要进一步确定与该用户的手指所作的手势对应的触碰感需求信息。具体地，获取该用户在作出该手指的手势时观看到的虚拟画面，该虚拟画面是该用户通过VR系统中的HMD或其他具备显示虚拟画面功能的眼镜设备观看到的。进一步地，通过摄像装置拍摄该用户的人体图像，确认该用户手指相对于人体的高度，结合该虚拟画面，可判断该手指的手势在该虚拟画面中所指向的虚拟物品，或者，获取该用户在作出该手指的手势时眼睛的焦点位置，也可判断该手指的手势在该虚拟画面中所指向的虚拟物品。例如，通过前面所述的方式，可以判断出该用户在作出五指张开的手势时，该手势在该用户同时看到的虚拟画面中所指向的虚拟物品是一个壁炉。
结合该虚拟物品的物理性质以及该手指的手势，确定目标预置手势对应的触碰感需求信息，可以提高确定该触碰感需求信息的准确性，并将该触碰感需求信息和该手指的标识信息发送给该智能手部穿戴设备。例如，壁炉具有发热的物理属性，该用户五指张开，可以确定该用户的五个手指需要加热。此时，将五个手指需要加热的信息和该五个手指的标识信息发送给该智能手部穿戴设备。
S405、该智能手部穿戴设备将接收到的该触碰感需求信息转换成响应动作，并将该响应动作作用在该标识信息对应的手指上，以使该手指产生与该响应动作对应的触感。
该智能手部穿戴设备将接收到的该触碰感需求信息转换成响应动作，该响应动作可以包括震动、收紧、加热、冷却中的一种或几种动作，将转换后的响应动作作用在该标识信息对应的手指上包括，即，将震动、收紧、加热、冷却中的一种或几种动作，作用在该标识信息对应的手指上。
例如，智能手套接收该触碰感需求信息，该触碰感需求信息是该手指需要有收紧的感触，则转换成对应的响应动作，即收紧，并收紧该标识信息对应的手指上，使得该手指产生收紧的感觉。
本实施例中其他细节，请参见前述图2至图4所示实施例的描述。
本实施例提供的在虚拟现实系统中获得沉浸感的方法，通过摄像设备拍摄用户手指运动的图像，虚拟现实客户端根据该图像确定该手指所作的手势，并将该手势对应的触碰感需求信息发送给智能手部穿戴设备，该智能手部穿戴设备将该触碰感需求信息转换成响应动作，并将该响应动作作用在该手指上，以使该手指产生对应的感觉，以增加用户体感中的触碰感，从而提高用户的体感交互沉浸感，提高产品粘性。
请参阅图5，图5是本发明第四实施例提供的智能手套的结构示意图，为了便于说明，仅示出了与本发明实施例相关的部分。图5示例的智能手套50用于实现前述图3至图4所示实施例提供的在虚拟现实系统中获得沉浸感的方法，具体请参见前述图2至图4所示实施例的描述，此处不再赘述。图5示例的智能手套50，应用在VR系统中，通过无线方式与VR系统中其他相关设备相连接并进行数据交换，例如，智能手套50和移动终端通过WIFI连接。
智能手套50可包括：信息接收装置51和响应动作发生装置52。
其中，信息接收装置51，用于接收虚拟现实客户端发送的触碰感需求信息和手指的标识信息，响应动作发生装置52，用于将接收到的触碰感需求信息转换成响应动作，并将响应动作作用在该标识信息对应的手指上，以使该手指产生与该响应动作对应的触感。
本发明实施例提供一种在虚拟现实系统中获得沉浸感的装置，该装置包括：一个或者多个处理器；
存储器；
一个或者多个程序，所述一个或者多个程序存储在所述存储器中，当被所述一个或者多个处理器执行时：
当摄像设备拍摄到用户的手指作出手势时，虚拟现实客户端获取该手指的手势，该虚拟现实客户端判断是否存在与该手指的手势相同的目标预置手势，若存在该目标预置手势，则将该目标预置手势对应的触碰感需求信息和该手指的标识信息发送给智能手部穿戴设备，该智能手部穿戴设备将接收到的该触碰感需求信息转换成响应动作，并将该响应动作作用在该标识信息对应的手指上，以使该手指产生与该响应动作对应的触感。
本发明实施例的未尽描述细节，请参见前述各实施例的描述。
在本发明各个实施例中的装置可以集成在一个处理装置中，也可以是各个装置单独物理存在，也可以两个或两个以上装置集成在一个装置中。上述集成的装置既可以采用硬件的形式实现，也可以采用软件功能装置的形式实现。
所述集成的装置如果以软件功能装置的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备（可以是个人计算机，服务器，或者网络设备等）执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器（ROM，Read-Only Memory）、随机存取存储器（RAM，Random Access Memory）、磁碟或者光盘等各种可以存储程序代码的介质。
需要说明的是，对于前述的各方法实施例，为了简便描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本发明并不受所描述的动作顺序的限制，因为依据本发明，某些步骤可以采用其它顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于优选实施例，所涉及的动作、设备和装置并不一定都是本发明所必须的。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见其它实施例的相关描述。
以上为对本发明所提供的一种在虚拟现实系统中获得沉浸感的方法、系统及智能手套的描述，对于本领域的技术人员，依据本发明实施例的思想，在具体实施方式及应用范围上均会有改变之处，综上，本说明书内容不应理解为对本发明的限制。

Claims (12)

  1. 一种在虚拟现实系统中获得沉浸感的方法，其特征在于，所述方法包括：
    当摄像设备拍摄到用户的手指作出手势时，虚拟现实客户端获取所述手指的手势；
    所述虚拟现实客户端判断是否存在与所述手指的手势相同的目标预置手势；
    若存在所述目标预置手势，则将所述目标预置手势对应的触碰感需求信息和所述手指的标识信息发送给智能手部穿戴设备；
    所述智能手部穿戴设备将接收到的所述触碰感需求信息转换成响应动作，并将所述响应动作作用在所述标识信息对应的手指上，以使所述手指产生与所述响应动作对应的触感。
  2. 根据权利要求1所述的方法，其特征在于，所述响应动作包括震动、收紧、加热、冷却中的一种或几种动作，则所述将所述响应动作作用在所述标识信息对应的手指上包括：
    将震动、收紧、加热、冷却中的一种或几种动作，作用在所述标识信息对应的手指上。
  3. 根据权利要求1或2所述的方法，其特征在于，所述虚拟现实客户端判断是否存在与所述手指的手势相同的预置的手势包括：
    所述虚拟现实客户端查询手势数据库中保存的多个预置手势的图像，并对比所述手指的手势图像与所述多个预置手势的图像；
    判断所述多个预置手势中是否存在与所述手指的手势的匹配度在预置范围内的目标预置手势。
  4. 根据权利要求3所述的方法，其特征在于，所述若存在所述目标预置手势，则将所述目标预置手势对应的触碰感需求信息和所述手指的标识信息发送给智能手部穿戴设备包括：
    若存在多个所述目标预置手势，则获取所述用户在作出所述手指的手势时观看到的虚拟画面；
    判断所述手指的手势在所述虚拟画面中所指向的虚拟物品；
    结合所述虚拟物品的物理性质以及所述手指的手势，确定所述目标预置手势对应的所述触碰感需求信息，并将所述触碰感需求信息和所述手指的标识信息发送给所述智能手部穿戴设备。
  5. 根据权利要求1所述的方法，其特征在于，所述将所述手势对应的触碰感需求信息和所述标识信息的对应关系发送给所述智能手部穿戴设备包括：
    通过无线传播方式，将所述手势对应的触碰感需求信息和所述标识信息的对应关系发送给所述智能手部穿戴设备。
  6. 一种在虚拟现实系统中获得沉浸感的系统，其特征在于，所述系统包括：
    移动终端和智能手部穿戴设备；
    其中，所述移动终端内设置有摄像设备和虚拟现实客户端；
    所述移动终端，用于当摄像设备拍摄到用户的手指作出手势时，通过所述虚拟现实客户端获取所述手指的手势，以及，通过所述虚拟现实客户端判断是否存在与所述手指的手势相同的目标预置手势，若存在所述目标预置手势，则将所述目标预置手势对应的触碰感需求信息和所述手指的标识信息发送给所述智能手部穿戴设备；
    所述智能手部穿戴设备，用于将接收到的所述触碰感需求信息转换成响应动作，并将所述响应动作作用在所述标识信息对应的手指上，以使所述手指产生与所述响应动作对应的触感。
  7. 根据权利要求6所述的系统，其特征在于，所述响应动作包括震动、收紧、加热、冷却中的一种或几种动作，则所述智能手部穿戴设备，还用于将震动、收紧、加热、冷却中的一种或几种动作，作用在所述标识信息对应的手指上。
  8. 根据权利要求6或7所述的系统，其特征在于，
    所述移动终端，还用于通过所述虚拟现实客户端查询手势数据库中保存的多个预置手势的图像，并对比所述手指的手势图像与所述多个预置手势的图像，以及，判断所述多个预置手势中是否存在与所述手指的手势的匹配度在预置范围内的目标预置手势。
  9. 根据权利要求8所述的系统，其特征在于，
    所述移动终端，还用于若存在多个所述目标预置手势，则获取所述用户在作出所述手指的手势时观看到的虚拟画面，判断所述手指的手势在所述虚拟画面中所指向的虚拟物品，以及，结合所述虚拟物品的物理性质以及所述手指的手势，确定所述目标预置手势对应的所述触碰感需求信息，并将所述触碰感需求信息和所述手指的标识信息发送给所述智能手部穿戴设备。
  10. 根据权利要求9所述的系统，其特征在于，
    所述移动终端，还用于通过无线传播方式，将所述手势对应的触碰感需求信息和所述标识信息的对应关系发送给所述智能手部穿戴设备。
  11. 一种智能手套，其特征在于，所述智能手套用于实现如权利要求1至5任一项所述的在虚拟现实系统中获得沉浸感的方法，所述智能手套包括：
    信息接收装置和响应动作发生装置；
    所述信息接收装置，用于接收所述虚拟现实客户端发送的所述触碰感需求信息和所述手指的标识信息；
    所述响应动作发生装置，用于将接收到的所述触碰感需求信息转换成响应动作，并将所述响应动作作用在所述标识信息对应的手指上，以使所述手指产生与所述响应动作对应的触感。
  12. 一种在虚拟现实系统中获得沉浸感的装置，其特征在于，所述装置包括：一个或者多个处理器；
    存储器；
    一个或者多个程序，所述一个或者多个程序存储在所述存储器中，当被所述一个或者多个处理器执行时：
    当摄像设备拍摄到用户的手指作出手势时，虚拟现实客户端获取所述手指的手势；
    所述虚拟现实客户端判断是否存在与所述手指的手势相同的目标预置手势；
    若存在所述目标预置手势，则将所述目标预置手势对应的触碰感需求信息和所述手指的标识信息发送给智能手部穿戴设备；
    所述智能手部穿戴设备将接收到的所述触碰感需求信息转换成响应动作，并将所述响应动作作用在所述标识信息对应的手指上，以使所述手指产生与所述响应动作对应的触感。
PCT/CN2016/088679 2015-12-25 2016-07-05 在虚拟现实系统中获得沉浸感的方法、系统及智能手套 Ceased WO2017107445A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US15/249,530 US20170185142A1 (en) 2015-12-25 2016-08-29 Method, system and smart glove for obtaining immersion in virtual reality system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510995889.6 2015-12-25
CN201510995889.6A CN105653029A (zh) 2015-12-25 2015-12-25 在虚拟现实系统中获得沉浸感的方法、系统及智能手套

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/249,530 Continuation US20170185142A1 (en) 2015-12-25 2016-08-29 Method, system and smart glove for obtaining immersion in virtual reality system

Publications (1)

Publication Number Publication Date
WO2017107445A1 true WO2017107445A1 (zh) 2017-06-29

Family

ID=56477902

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/088679 Ceased WO2017107445A1 (zh) 2015-12-25 2016-07-05 在虚拟现实系统中获得沉浸感的方法、系统及智能手套

Country Status (2)

Country Link
CN (1) CN105653029A (zh)
WO (1) WO2017107445A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107491173A (zh) * 2017-08-16 2017-12-19 歌尔科技有限公司 一种体感模拟控制方法及设备
CN111752380A (zh) * 2019-04-08 2020-10-09 广东小天才科技有限公司 一种基于腕式穿戴设备的交互方法及腕式穿戴设备

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105653029A (zh) * 2015-12-25 2016-06-08 乐视致新电子科技（天津）有限公司 在虚拟现实系统中获得沉浸感的方法、系统及智能手套
CN106054627B (zh) * 2016-06-12 2020-02-11 珠海格力电器股份有限公司 基于手势识别的控制方法、装置及空调
CN106200923A (zh) * 2016-06-30 2016-12-07 乐视控股（北京）有限公司 一种虚拟现实系统的控制方法和装置
CN106873775A (zh) * 2017-01-16 2017-06-20 深圳中科呼图电子商务有限公司 一种虚拟现实交互的实现方法、系统及mr手套
WO2018150831A1 (ja) * 2017-02-16 2018-08-23 ソニー株式会社 情報処理装置、情報処理方法及び記録媒体
CN107092353B (zh) * 2017-03-31 2020-04-24 东南大学 一种手部触感参数的采集及模拟还原系统和方法
CN110850962B (zh) * 2019-09-30 2024-05-14 深圳岱仕科技有限公司 控制方法及装置、手部外骨骼设备、计算机可读存储介质
CN111443831A (zh) * 2020-03-30 2020-07-24 北京嘉楠捷思信息技术有限公司 一种手势识别方法及装置
CN116051789A (zh) * 2022-12-21 2023-05-02 北京字跳网络技术有限公司 用于增强现实的方法、装置、设备和存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1785608A (zh) * 2005-11-10 2006-06-14 上海大学 多指机械灵巧手闭环实时动作控制平台
US20130063560A1 (en) * 2011-09-12 2013-03-14 Palo Alto Research Center Incorporated Combined stereo camera and stereo display interaction
CN104298348A (zh) * 2014-09-23 2015-01-21 广东欧珀移动通信有限公司 一种红外手势识别器、摄像方法、装置及摄像机
CN105068658A (zh) * 2015-08-19 2015-11-18 天津先驱领域科技有限公司 一种可虚拟环境控制的人机互动装置
CN105653029A (zh) * 2015-12-25 2016-06-08 乐视致新电子科技（天津）有限公司 在虚拟现实系统中获得沉浸感的方法、系统及智能手套

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8996173B2 (en) * 2010-09-21 2015-03-31 Intuitive Surgical Operations, Inc. Method and apparatus for hand gesture control in a minimally invasive surgical system
EP2585895A1 (en) * 2010-06-28 2013-05-01 TP Vision Holding B.V. Enhancing content viewing experience
JP6664069B2 (ja) * 2013-12-31 2020-03-13 イマージョン コーポレーションＩｍｍｅｒｓｉｏｎ Ｃｏｒｐｏｒａｔｉｏｎ 触覚コンテンツを伴う視点動画を記録及び再生するシステム並びに方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1785608A (zh) * 2005-11-10 2006-06-14 上海大学 多指机械灵巧手闭环实时动作控制平台
US20130063560A1 (en) * 2011-09-12 2013-03-14 Palo Alto Research Center Incorporated Combined stereo camera and stereo display interaction
CN104298348A (zh) * 2014-09-23 2015-01-21 广东欧珀移动通信有限公司 一种红外手势识别器、摄像方法、装置及摄像机
CN105068658A (zh) * 2015-08-19 2015-11-18 天津先驱领域科技有限公司 一种可虚拟环境控制的人机互动装置
CN105653029A (zh) * 2015-12-25 2016-06-08 乐视致新电子科技（天津）有限公司 在虚拟现实系统中获得沉浸感的方法、系统及智能手套

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107491173A (zh) * 2017-08-16 2017-12-19 歌尔科技有限公司 一种体感模拟控制方法及设备
CN111752380A (zh) * 2019-04-08 2020-10-09 广东小天才科技有限公司 一种基于腕式穿戴设备的交互方法及腕式穿戴设备
CN111752380B (zh) * 2019-04-08 2024-03-19 广东小天才科技有限公司 一种基于腕式穿戴设备的交互方法及腕式穿戴设备

Also Published As

Publication number Publication date
CN105653029A (zh) 2016-06-08

Similar Documents

Publication Publication Date Title
WO2017107445A1 (zh) 在虚拟现实系统中获得沉浸感的方法、系统及智能手套
WO2015183033A1 (en) Data processing method and electronic device thereof
WO2015167160A1 (en) Command displaying method and command displaying device
US20170185142A1 (en) Method, system and smart glove for obtaining immersion in virtual reality system
US20190356897A1 (en) Correlation of video stream frame timestamps based on a system clock
WO2018093005A1 (en) Mobile terminal and method for controlling the same
WO2014051367A1 (en) User terminal apparatus, electronic device, and method for controlling the same
WO2016200102A1 (ko) 카메라의 초점을 변경하는 방법 및 장치
WO2020011077A1 (zh) 通知消息显示方法及终端设备
WO2015152628A1 (en) Input device, method, and system for electronic device
WO2016027932A1 (en) Glass-type mobile terminal and control method thereof
WO2018048130A1 (ko) 컨텐츠 재생 방법 및 이를 지원하는 전자 장치
CN108322523A (zh) 一种应用推荐方法、服务器及移动终端
WO2023224433A1 (en) Information generation method and device
WO2012053875A2 (ko) 지문 정보를 통한 데이터 송신 수신 장치 및 시스템
EP3314478A1 (en) Electronic device and method for controlling display of panorama image
WO2015126097A1 (en) Interactive server and method for controlling the server
WO2018079999A1 (en) Electronic device and method for operating the same
CN108958623A (zh) 一种应用程序启动方法及终端设备
CN109005337B (zh) 一种拍照方法及终端
WO2020050432A1 (ko) 이동 단말기
WO2013058551A1 (ko) 디스플레이기기 및 디스플레이기기의 카메라 제어 방법
WO2019216498A1 (en) Electronic device and method for controlling the same
WO2019177437A1 (ko) 화면 제어 방법 및 이를 지원하는 전자 장치
WO2019137423A1 (zh) 通信方法及相关设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16877257

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16877257

Country of ref document: EP

Kind code of ref document: A1