WO2017095254A1

WO2017095254A1 - Portable tactile sensing device and system for the implementation thereof

Info

Publication number: WO2017095254A1
Application number: PCT/RU2015/000842
Authority: WO
Inventors: Владислав Анатольевич ЛУКАШЕВИЧ
Original assignee: Общество С Ограниченной Ответственностью "Интеллект Моушн"
Priority date: 2015-12-01
Filing date: 2015-12-01
Publication date: 2017-06-08

Abstract

The invention relates to the field of data processing, and more particularly to a multifunctional manipulator which enables the recognition of objects in real and virtual environments. In a preferred embodiment, the device is configured in the form of a portable device for interacting with objects in a virtual reality and/or for detecting objects in a surrounding environment, and comprises: a body which can be fastened to the user's hand and has an inner and an outer surface; a scanning unit disposed on the outer surface and containing a means for detecting objects in a real surrounding environment; a spatial positioning unit disposed on the outer surface and containing optical emitters for determining the spatial position of the portable device in a virtual environment; a computing unit containing a processor for processing data received from the scanning unit and/or the spatial positioning unit and a tactile feedback unit; and a tactile feedback unit disposed on the inner surface, said tactile feedback unit containing a vibration sensor, connected to an electrode, and being capable of responding to information received from the scanning unit and/or a sensory unit and processed by the computing unit in order to activate the vibration sensor, generating tactile feedback to the user's hand.

Description

WEARABLE TACTICAL PERCEPTION DEVICE and

IMPLEMENTING ITS SYSTEM

TECHNICAL FIELD The claimed solution relates to the field of data processing, in particular, to a wearable multifunction device, a manipulator that provides recognition of both objects of the real environment and objects of the virtual environment. This solution can be widely used in the medical field, education, computer games, computer modeling, etc.

BACKGROUND

A known system for orientation in space of people with visual impairment according to the patent of the Russian Federation Ν "2299052, published on 08.06.2007. This system for orientation in space of people with visual impairment contains at least one source and receiver of signals connected through a switch to an information processing device connected via a modulator to a human exposure device. Also, a magnetometer, a pedometer, and a GPS system are additionally connected through the switch to the information processing device.

A known method of orientation, navigation and information in space of people with visual impairment according to patent JP 8280733, published on 10/29/1996, which consists in the fact that at least one source of radio signals located in the location of the signal transmits a radio signal, and a receiver located in a person The radio signal receives these radio signals and transmits them to a device for influencing a person, signaling the location of the orientation point. Famous A device for implementing this method contains at least one radio signal source made in the form of one long antenna laid below the surface of pedestrian walkways and a radio signal receiver located at the end of the cane and connected to a human exposure device.

The disadvantages of these solutions are the very high complexity of implementation, the complexity of the complex, unreliability of work, excessive information content, complexity of management, which makes it difficult for people with visual impairment to use it and can lead to complications of movement on the street and indoors.

Known controller for US patent N ° 8,430,753, published April 30, 2013. This controller is a well-known controller of the Nintendo Wii game console and allows you to manipulate objects in a virtual environment by recognizing the location of the controller in space due to optical emitters positioning the location of the controller. This controller also contains a vibrating element for generating tactile interaction.

A known controller according to the application WO 2013/104681, published July 18, 2013. This controller is a rubber dressing worn on the user's palm. On the surface of the controller there are control buttons that interact with objects of the virtual environment. The controller also contains an element of tactile interaction, in particular a vibration sensor, which is activated when determining the fact of interaction with objects of the virtual environment.

A disadvantage of the known controllers is that they do not allow to obtain a full tactile sensation from interactions with objects of virtual space depending on the shape, density and size of such objects.

SUMMARY OF THE INVENTION

The claimed invention allows to provide a sense of full tactile contact with objects of real and / or virtual spaces by determining the distance to them, their density and spatial forms, which improves the quality of information received about objects in the surrounding space, and also expands the possibilities of interaction with objects of a virtual environment or virtual reality .

In a preferred embodiment, the claimed invention is made in the form of a wearable device for interacting with virtual reality objects and / or determining objects of the surrounding space, comprising: a compact housing with the possibility of fixing on the user's hand, containing internal and external surfaces;

- located on the outer surface of the scanning unit containing at least one means of determining objects of surrounding reality;

- a spatial positioning unit located on the outer surface, comprising at least three optical emitters for determining the spatial position of the wearable device in a virtual environment;

- a computing unit comprising at least one processor for processing data received from a scanning unit and / or spatial positioning unit and a tactile interaction unit; - a tactile interaction unit located on the inner surface, comprising at least one vibration sensor connected to at least one electrode and configured to, in response to receiving information processed by the computing unit from the scanning unit and / or sensor unit, activate said at least one vibration sensor generating tactile interaction with a user's hand.

In another preferred embodiment, the claimed invention is a system for interacting with virtual reality objects, comprising:

- at least one wearable device mentioned above;

- at least one virtual environment display device containing one or more virtual objects;

- one or more cameras providing an optical communication channel between the display device and the wearable device;

- at least one processor;

- at least one memory that stores

machine-readable instructions that, when executed by at least one processor, provide:

- generating a software pointer of a wearable device in a virtual environment;

- recognition of the position of said software pointer in said virtual environment;

- recognition of the interaction of the said program pointer in a virtual environment with one or more objects

virtual environment;

- transmitting signals to a wearable device in response to the interaction of said software pointer with one or more virtual objects; - generation of tactile interaction wearable

the device in response to the received signals of the interaction of the program pointer with the objects of the virtual environment.

In one of the private options for implementation, the housing of the device 5 is fixed on the brush.

In another particular embodiment, the means for determining objects of surrounding reality is an ultrasonic transceiver.

In another particular embodiment, the optical emitters ω are light emitting diodes (LEDs).

In another particular embodiment, the vibration sensor and the electrode connected to it are mounted on a solenoid.

In another particular embodiment, the spatial positioning unit transmits spatial coordinates to a data display device 15 reproducing a virtual environment.

In another particular embodiment, the transmission of coordinates is via a wireless data channel.

In another private embodiment, the implementation of the wireless data transmission channels includes at least an optical channel.

20 In another particular embodiment, the optical emitters are located in different parts of the outer surface of the housing.

In another particular embodiment, the scanning unit determines the parameters of the distance to environmental objects and / or their acoustic density and / or their spatial forms and transfers 25 the received information to the computing unit, which processes the data and transmits them to the tactile interaction unit.

In another particular embodiment, the computing unit determines the interaction parameters with the objects of the virtual environment, characterizing the shape and density of the object, based on data from the position of the device in the virtual environment received from spatial positioning, and transfers the received information to the tactile interaction unit.

In another particular embodiment, on the basis of the information received, the tactile interaction unit moves at least one vibration sensor with an electrode towards the surface of the user's palm.

In another private embodiment, the electrode of the vibration sensor determines the resistance indicators in contact with the skin of the palm,

In another particular embodiment, the resistance indicators are transmitted to a computing unit that processes the received data and, based on them, determines the tactile interaction strength parameter, which is transmitted to the tactile interaction unit.

In another particular embodiment, the tactile interaction strength parameter is transmitted to the tactile interaction unit, which in response to the received data generates a series of vibrational patterns.

In another particular embodiment, virtual reality objects contain points in their computer model that characterize the shape and / or density of the said object. In another particular embodiment, said points are represented by patterns characterizing the distribution of said points on the model of the corresponding object.

In another particular embodiment, signals are generated for the wearable device based on the patterns. In another particular embodiment, the signals transmitted to the wearable device generate tactile interaction in the form of vibration patterns.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 illustrates a general view of a claimed wearable device.

FIG. 2A illustrates a general view of the external side of the claimed device.

FIG. 2B illustrates a general view of the inside of the claimed device.

FIG. ZA-ZV illustrate the principle of recognition of objects in the surrounding space.

FIG. 4 illustrates a tactile interaction sensor circuit.

FIG. 5 illustrates the principle of operation of tactile interaction sensors.

FIG. 6 illustrates the claimed system for providing interaction with objects of a virtual environment.

FIG. 7A-7B illustrate the principle of interaction with objects of a virtual environment. FIG. 8A-8G illustrate vibration point distribution patterns characterizing various types of objects in a virtual environment.

DETAILED DESCRIPTION OF THE INVENTION

According to FIG. 1, the claimed wearable device 100 is a manipulator (controller), mounted on the hand of the user 10, mainly on the hand. The manipulator 100 is made in the form of a compact case, on the outer side 101 (Fig. 2 A) of which there is a spatial positioning unit, consisting of at least three optical emitters 1 10 and a scanning unit 120 containing means for scanning the surrounding space. The optical emitters of the software are light emitting diodes (LEDs). The device 100 is attached to the user's hand 10 using an elastic retainer (not shown) fixed on the sides of the case in specially made armholes 150. The retainer can be made in the form of a single tape of synthetic elastic material or belt, and can also be made in the form of two elements fastened between each other, for example, using fastex or contact tape.

In the upper part of the housing of the device 100 contains controls in the form of buttons 130, which can be programmed to perform certain functions when interacting with the manipulator. According to FIG. 2A, tactile interaction elements 140, for example, performing the tactile interaction function in the form of vibrations, may further be contained in the lower part of the housing. The emitters 110 of the spatial positioning unit are located on the outer surface 101 of the device 100 at a distance from each other and are designed to determine the position of the manipulator 100 in a virtual environment (1010). This will be described in more detail below.

In FIG. 2B is a view of the inner surface 102 of the housing of the device 100. On the inner surface 102, the main elements of tactile interaction are vibration sensors 160. The number and arrangement of sensors 160 shown in FIG. 2B. It is not a limiting embodiment of the claimed device 100 and depends on its particular hardware implementation.

Sensors 160 are designed to transmit tactile perception to the user's hand 10. The signals initiating the activation of the sensors 160 are generated when the scanning unit and / or spatial positioning unit are operated in conjunction with a computing unit (processor). With reference to FIG. ZA and ZV we consider an example of the operation of the device 100 in determining the objects of the environment. Scanning of the surrounding space is carried out using an ultrasonic transceiver (USPP) 120 (sonar). The user moves with his hand 10, with a device 100 fixed on it, which continuously scans the space using the radiation working area 200 generated by the USPP 120.

When object 20 is detected, radiation 200 falls onto the surface of object 20, thereby determining the shape of the radiation propagation area 210. The USPP 120, using data processing by the computing unit, also determines the distance D to object 20. Depending on the radiation scattering area 200, a tactile response is generated with using sensors 160.

For example, when the radiation 200 of the ultrasonic protection device hits the wall, the sensors 160 will interact evenly with the palm of the user 10, while when they get on a narrower area, for example, the edge of the table (Fig. 3B), in the case when a person is in real contact with such an object senses the object only with a part of the palm of the hand, the device 100 generates a similar tactile response when it only affects the necessary parts of the palm of the hand 10 using the appropriate sensors 160.

The distance to the object 20 when it is processed by the computing unit of the device 100 is used to determine the tactile interaction force parameter that will be transmitted to one or more sensors 160. In FIG. 4 shows the design of each of the sensors 160 of the tactile interaction unit. Each sensor 160 comprises, within the area of contact with the user's hand 10, an electrode 301 mounted on a solenoid 303 or other suitable type of pushing mechanism. The sensor 106 also includes a vibration element 302 located between the surface of the solenoid 303 and the electrode 301. The solenoid 303 is driven into movement using an electromechanical engine 304, which through the gear 306 transmits the movement to the corresponding gear 307 of the worm gear 305. The drive mechanism is located on the base 308, under which is the pressure sensor pusher 309, located on the base of the sensor unit.

By analyzing the data on the remoteness of the object 20 from the device 100, the computing unit calculates the interaction force parameter that regulates how far the displacement (pushing) of the surface 311 of the sensor 160 will take place, with the electrode 301 towards the palm of the carrier 10.

In FIG. 5 shows the principle of the effect of sensors 160 on the surface of the palm of a user 10. As mentioned above, upon receipt of information determining the type of shape of the surface of an object 20 falling into the radiation zone 200 of the ultrasonic testing device 120, the computing unit transmits signals directly to those sensors 160 using the software embedded in it which are subject to movement.

FIG. 6 shows a general diagram of a system 400 that facilitates the interaction of a wearable device 100 with objects 1001-1003 of a virtual environment 1010. A device 100 for performing the functions of a controller (manipulator) in a virtual environment 1010 communicates with a computing device 410.

The device 410 may be, for example, a personal computer, laptop, game console, tablet, smartphone, or the like. In general, the device 410 contains such necessary basic components as the processor 411, and for specialists it should be clear that one or more processors, memory 412 (RAM, ROM, flash memory, or combinations thereof) and input interfaces can be understood I / O 413. The type of memory 412 and the input / output interfaces can be presented in various variations, depending on the type of device 410. The device 410 is connected via a wired 414 or wireless channel 415 data transfer device display virtual environment 420, which may be, for example, a monitor, TV, display or projector.

Wired connection 414 can be a standard wired connection using VGA, DVI, HDMI, or the like. Wireless communication 415 is implemented using, for example, Bluetooth, Wi-Fi, IrDa. Connected to a computing device 410 is a device 430 providing an optical communication channel 416, for example, a webcam, a Kinect device for an X-Voh console, or a Sony Eye module for a Sony Playstation console. The optical communication channel 416 generated by the device 430 is used to determine the position of the device 100 in space using software LED emitters. A device 416 providing an optical communication channel is connected to the device 410 using a wired (USB) 414 or wireless (Bluetooth, Wi-Fi) 415 data channel.

In turn, the device 100 is also connected to the computing device 410 via a wired 414 (USB) or wireless (Bluetooth) 415 data channel.

When you pair the device 100 using the device 430 with the computing device 410 in a virtual environment 1010, for example, a computer game, a program pointer 1000 is generated for interacting with objects 1001-1003 using the device 100.

In FIG. 7A-7B show an example of pointing a program pointer 1000 to a three-dimensional virtual object 1003, which is cubic-shaped figure. When determining the interaction of the pointer 1000 with the edge of the object 1003 (Fig. 7A), the shape of the object 1003 is recognized by determining the vibration points describing the type of surface, shape and density of the object 1003. In FIG. 8A-8G show patterns of distribution of vibration points depending on the type of virtual object.

The nature of the distribution and relative position of the vibration points can determine various objects, in FIG. 8A, the distribution of points characterizes objects and environments with an unstable shape (water, smoke, steam, etc.), while the relative position of the vibration points will be chaotic.

In FIG. 8B-8G presents the types of virtual object (object) with a stable shape. Consider hard and soft objects, which will differ among themselves in the distribution density of vibration points. The higher the density, the higher the density of the virtual object. Such objects can be objects with flat shapes, for example, a wall. In FIG. 8B, the principle of distribution of points for a solid object of a flat shape (wall, surface of a table, door, etc.) is presented.

In FIG. 8C shows a type of soft object of a flat shape, for example, a sheet of paper, a curtain.

In FIG. 8D shows an example of the distribution of vibration points for a solid object having a face, for example, a corner of a table, cabinet, slope, etc.

In FIG. 8E shows an example of the distribution of vibration points for a soft object having a face, for example, a pillow, bag.

In FIG. 8F shows an example of the distribution of vibration points for a solid object whose plane has a convex shape, for example, a column, ball, or the like. In FIG. 8G shows an example of the distribution of vibration points for a soft object whose plane has a convex shape, leg, arm, head.

Depending on the type and shape, various objects will transmit signals characterizing the degree of tactile interaction to the tactile interaction unit of the device 100, which will activate the corresponding sensors 160.

Returning to FIG. 7A and 7B, it is seen that when pointing and interacting in a virtual environment 1010 with an object 1003, in particular when a virtual touch of its face or flat surface occurs, various sensors 160 are activated that simulate the real impact on the surface of the palm of the user 10, as this would happen when contact with objects of the real environment.

In addition to contacting with the surface of the sensor housing 160, it also generates a vibrational response, which is a series of vibrational patterns.

The signals responsible for the correct generation of a series of vibrational patterns are processed by the computing unit of the device 100 upon receipt of them from the computing device 410.

The buttons 130 located on the housing of the device 100 can be programmed to perform various types of functions, for example, capturing a virtual object, the “click” function when working in a web browser or operating system, or you can assign specific actions when using the device 100 as a controller for games .

Additionally, the device 100 may also contain a GPS transceiver, in particular this is intended to control the movement of people with disabilities.

The controls 130 can also be used to perform various useful functions for users who use the device 100 to recognize objects in the environment. space. In one embodiment, the controls 130 may be programmed to transmit a help signal, for example, an audio signal or automatically send an electronic message (SMS) to another person’s cell phone. The transmission of this kind of electronic message can be performed through a specialized resource containing a database of numbers and a module for generating SMS messages. Each device 100 has its own ID in the corresponding database and is associated with one or more cell phone numbers to which SMS messages will be transmitted in response to a signal from device 100.

The information on the implementation of the claimed invention set forth in these materials of the application should not be interpreted as information limiting other, private embodiments of the claimed invention that do not go beyond the disclosure of information of the application, and which should be obvious to a person skilled in the art having the usual qualifications, which calculated the claimed technical solution.

Claims

Formula

1. A wearable device for interacting with virtual reality objects and / or determining objects of the surrounding space, comprising:

5 - a compact housing with the possibility of fixing on the user's hand, containing internal and external surfaces;

y - located on the outer surface of the spatial positioning unit, containing at least three optical emitters for determining the spatial position of the wearable device in a virtual environment;

- a computing unit comprising at least one processor for processing data received from a scanning unit and / or spatial positioning unit and a tactile interaction unit; a tactile interaction unit located on the inner surface, comprising at least one vibration sensor connected to at least one electrode and configured to 20 in response to receiving information processed by the computing unit from the scanning unit and / or sensor unit, activate said at least one vibration sensor generating tactile interaction with a user's hand.

2. The device according to p. 1, characterized in that the housing of the device 25 is fixed on the brush.

3. The device according to p. 1, characterized in that the means for determining objects of surrounding reality is an ultrasonic transceiver.

4. The device according to claim 1, characterized in that the optical emitters zo represents light-emitting diodes (LEDs).

5. The device according to claim 1, characterized in that the vibration sensor and the electrode connected to it are mounted on a solenoid.

6. The device according to claim 1, characterized in that the spatial positioning unit transmits spatial coordinates to the device

5 data display, reproducing the virtual environment.

7. The device according to p. 6, characterized in that the transfer of coordinates is carried out through a wireless data channel.

8. The device according to p. 7, characterized in that the wireless data transmission channels includes at least an optical channel.

9. The device according to claim 1, characterized in that the optical emitters are located in different parts of the outer surface of the housing.

10. The device according to claim 1, characterized in that the scanning unit determines the parameters of the distance to environmental objects and / or their acoustic density and / or their spatial forms and transmits

15 received information to the computing unit, which processes the data and transmits them to the tactile interaction unit.

11. The device according to claim 1, characterized in that the computing unit determines the interaction parameters with the objects of the virtual environment, characterizing the shape and density of the object, based on the data

20 positions of the device in a virtual environment, received from the spatial positioning unit, and transmits the received information to the tactile interaction unit.

12. The device according to any one of paragraphs. 10 or 1 1, characterized in that, on the basis of the information received, the tactile interaction unit

25 moves at least one vibration sensor with an electrode toward the surface of a user's palm.

13. The device according to p. 12, characterized in that the electrode of the vibration sensor determines the resistance indicators in contact with the skin of the palm.

14. The device according to p. 13, characterized in that the resistance indicators are transmitted to the computing unit, which processes the received data and on their basis determines the parameter of the tactile interaction force, which is transmitted to the tactile unit

5 interactions.

15. The device according to claim 14, characterized in that the tactile interaction force parameter is transmitted to the tactile interaction unit, which, in response to the received data, generates a series of vibrational patterns.

Yu

16. A system for interacting with virtual reality objects, comprising:

- at least one wearable device according to claim 1;

15 is one or more cameras providing an optical communication channel between a display device and a wearable device;

- at least one processor;

- at least one memory that stores machine-readable instructions that, when executed, by at least one

20 processors, provide:

- generating a software pointer of a wearable device in a virtual environment;

25 - recognition of the interaction of the aforementioned program pointer in a virtual environment with one or more objects of the virtual environment;

- transmitting signals to a wearable device in response to the interaction of said software pointer with one or more virtual objects; - generation of tactile interaction by the wearable device in response to the received signals of the interaction of the program pointer with the objects of the virtual environment.

17. The system according to claim 16, characterized in that the objects of virtual 5 reality contain points in their computer model that characterize the shape and / or density of the said object.

18. The system of claim 17, characterized in that said points are represented by patterns characterizing the distribution of said points on the model of the corresponding object. Yu

19. The system of claim 18, characterized in that, on the basis of said patterns, signal generation for the wearable device is performed.

20. The system according to p. 19, characterized in that the signals transmitted to the wearable device generate tactile interaction in the form of vibration patterns.

fifteen