DE29719995U1 - System for converting visually recorded room data into tangible and tactile information - Google Patents

Description

t« *t

Description: System for converting visually recorded data into tangible and palpable information

The system described above is used to convert visually recorded data into information that can be felt and touched by a human user. When using the system, a human user is able to orient themselves in a room without having to rely on their own visual information. One potential area of application is therefore primarily in the field of rehabilitation technology: with the system presented, a blind person who no longer has access to their own visual data can orient themselves in their environment by wearing the sensors and actuators required for the system and by connecting the processing electronics. The optical sensors, preferably cameras, capture the image of the room. One camera is sufficient to capture textures and colors. If the relief of the room, the three-dimensional structure, is to be captured, a second camera must be added that captures the room from a different position. The latter capture can also be carried out using sensors for distance measurement.

The data recorded in this way is first converted by the processing computer or electronic system into a three-dimensional structure in the form of data. The computer then displays this structure for the user by converting it into signals that the person can feel through certain contacts that he or she wears on a part of his or her body. If these contacts are located in the fingertips of a glove, for example, the user can put on and move the glove. If he or she moves his or her hand in a virtual cube in front of him or her with an edge length of one meter, the computer transfers the three-dimensional data of the corresponding point in the recorded space to the user's glove. The user can therefore feel when he or she touches a part of the reduced, virtual space (which the virtual cube corresponds to).

solid object that is in an equivalent position in real space. If, for example, there is an open door opposite the user, the computer will recognize a "hole" in a solid object, namely the wall, at this point. If the user now moves his glove one meter forward (relative to his chest), he will travel through the entire real space lengthways with this movement; he will feel all of the solid objects or edges that are present. If he reaches the end of the virtual cube, he will feel the solid wall; by moving to the side, perpendicular to the previous direction of movement of his arm or hand, he will feel that there is a hole in this wall; he will be able to identify this as a door based on its shape. If there are contacts in the glove that can exert pressure on the user's fingertips (e.g. through contraction), a solid object corresponds to increased pressure, while free space corresponds to no pressure.

Applications are also conceivable in the field of virtual reality and movement in virtual spaces - the illusion of a real world is enhanced by an additional sensory moment.

Compared to other methods of processing visual data and transforming it into another sensory representation, such as orientation in a room using sound signals, the above system has a number of advantages. Many parts of the body where a person has very sensitive sensory perception are a starting point for the "feeling" instrumentation. This means that many different pieces of information, including color and texture information, can be made "tangible". In the version that is specially designed for the blind, the user can use the system as an electronic replacement for a cane. In contrast to this, the system offers far more expansion options - switching between color and texture sensations as well as relief sensations is also conceivable. In addition, the blind person can perceive a room from one location.

There are several possible ways of determining the time at which the optical sensors capture the room: if the processing computer system has sufficient computing power, capture can occur several times per second. Alternatively, capture can be performed on the user's command; this occurs when the user enters a new room or a new environment. In the current version, the user wears two small cameras (4) on their head, both of which film straight ahead. The computer creates a three-dimensional image from the two images.

Edge image; image processing software extracts edges and textures and uses their positions in both images to determine their position in space (1). The current version of the system is shown in a simplified graphical form using the drawings. Figure 1 shows a view from above. Figure 2 shows a view from the side. Figure 3 shows a highly simplified model of the glove that the user is wearing. The user (3) is wearing the glove described above ((6); also Figure 3). Instruments are attached to the fingertips of the glove that can exert pressure on the user's fingertips; these instruments are thimble-like, but they can also be contracted by applying a small voltage to create a feeling of pressure (7). Ultrasound transmitters are also attached to the glove (8). By picking up their pulsed signals using microphones that the user wears on his body, the positions of the ultrasound transmitters and thus of the prominent points on the glove in space can be determined. The processing computer system consists of the board of a Pentium computer on which the processing software runs. The computer digitizes the images from both cameras, extracts a three-dimensional image and then controls the instruments in the glove: after it has detected the position of the glove, it checks whether the user is touching an object (2) with his fingertips in the reduced "virtual" space (5) or is located in an object. If this is the case, the computer applies appropriate pressure to the fingertips by applying a low voltage to the corresponding instruments in the glove ((7), data lines to the computer idealized at (9)). The position of the glove is checked and the instruments are controlled several times per second; the space is detected on command from the user, who wears a button connected to the computer's mouse port for this purpose.

In the form of the system shown graphically below, it is ideally assumed, for the sake of simplicity and clarity, that the user stretches out his hand and puts his fingers together when using the system correctly. This is necessary because, with the design chosen in the drawing, the processing computer has no way of obtaining information about the position of the individual fingertips, depending on the position of the glove. In practice, this problem is avoided with strips that extend from the fingertips to the ball of the hand; the tension of these strips, which increases when the fingers are bent, is in turn transmitted to the

· ··· CDI

data processing computer as additional parameters. This allows the fingers of the hand to move independently of each other without the system transmitting incorrect data to the user.

The system can certainly be placed commercially in the rehabilitation sector. In virtual reality, too, systems that can provide as many sensory impressions as possible to simulate an artificial world are very popular.

Claims (11)

Application for utility model protection System for converting visually recorded spatial data into tangible and palpable information Protection claims 1. System according to one or more of the above claims, characterized in that the visual data is recorded by suitable optical sensors, preferably by two cameras. 2. System according to one or more of the above claims, characterized in that the acquisition of the visual data by optical sensors is supported by the use of rangefinders, which additionally provide information about the distances of objects. 3. System according to one or more of the above claims, characterized in that the information about a room recorded by the sensor system for recording the visual data is processed by a suitable system and converted into data that can be passed on to the human user as tactile and palpable information using suitable instruments. 4. System according to one or more of the above claims, characterized in that the system processing the visual data is a computer or electronics with software suitable for converting the information. 5. System according to one or more of the above claims, characterized in that tt * ·· ft that the instruments in contact with the user are integrated into a glove and arranged in such a way that they can generate signals or stimuli on the user's hands or fingertips that can be recognized through the user's skin or nerves. 6. System according to one or more of the above claims, characterized in that that <ÜQ signals generated by the above instruments are pressure, heat or other signals perceptible by the user. 7. System according to one or more of the above claims, characterized in that that the positions of the above-mentioned instruments relative to the sensors recording the visual information are known to the above-mentioned data processing system. 8. System according to one or more of the above claims, characterized in that that the positions of the above-mentioned instruments are recorded using suitable sensors. 9. System according to one or more of the above claims, characterized in that that the sensors required to record the positions of the above-mentioned instruments can determine the positions using inertial sensors, positioning sensors based on signal propagation times or other suitable sensors. 10. System according to one or more of the above claims, characterized in that that the above-mentioned system processing the visual data controls and switches the instruments used to generate the information that can be felt and touched by the user, depending on the visual data recorded and the positions of the instruments. 11. System according to one or more of the above claims, characterized in that that the information that the user can feel and touch can be color, texture or relief data of the space around him, which are transmitted to his body by different signals via the appropriate instruments mentioned above.