PT103625A - INTEGRATED SYSTEM OF ACCESS EXPANDED TO VIEW INFORMATION ABOUT REAL IMAGE INTEGRATING SPECIAL ACCESSIBILITY NEEDS TO INFORMATION - Google Patents

Abstract

The present invention relates generally to display systems in which real image is overlapped by computer generated image (augmented reality). SPECIFICALLY, IT REFERS TO A SYSTEM WHICH MEETS SPECIAL NEEDS FOR ACCESSIBILITY TO NON-VISUAL AND NON-SOUND INFORMATION, AND INTEGRATES ENVIRONMENTAL CONDITIONS, WHICH ARE NON-CONTROLLED. With the present invention, any user, with a special need for information or unavailability of the information, can obtain information about the space involved through the device's positioning (preferred embodiment of the invention), EG NAMES OF BUILDINGS, RECOMMENDED TRANSPORTATION, HISTORICAL INFORMATION, CULTURAL PROGRAMMING AND LOCATION OF INFRASTRUCTURES (HOTELS, RESTAURANTS, CINEMAS). IT MAY ALSO INTERACT WITH PURE VIRTUAL MODELS EMBEDDED IN THE REAL IMAGE, EXPLORING MULTIMEDIA CONTAINERS ACCESSIBLE ASSOCIATED.

Description

1

DESCRIPTION

SYSTEM. INTEGRATED ACCESS EXPANDED TO VIEW INFORMATION ABOUT REAL IMAGE INTEGRATING SPECIAL ACCESSIBILITY NEEDS TO INFORMATION

TECHNICAL FIELD OF THE INVENTION The present invention relates generally to display systems in which the actual image is superimposed computer-generated image (augmented reality).

Specifically, it refers to a system that integrates special non-visual and non-noise information accessibility needs, as well as integrates diverse and uncontrolled environmental conditions.

By assuming itself as a public equipment, the present invention provides the augmented reality experience to the general public in common environments. 2

BACKGROUND OF THE INVENTION

Telescopes for viewing landscapes and similar devices are well known; also the application of the overlay technique to the actual image of computer generated image applied to a vision device as described in the present patent does not constitute its object.

There are also a set of solutions and tools specifically designed to support the navigation and exploration of interior spaces by people with special accessibility needs, which is no longer the case for outdoor spaces. The system of the present invention being a clearly oriented and designed tool for the exploration of exterior spaces, its integration of special needs of accessibility to information is a technological solution that intends to actively promote augmented reality as a way of transmitting information.

With respect to traditional stage structures, the device, in the preferred embodiment, has the advantage of displaying the image on a screen, while traditional solutions used the frame of the bezel to allow visualization.

As a system, the device is a solution that allows to comprehensively extend the usability of the concept of augmented reality to the general public in uncontrolled environments, such application and technical solution being new and not contained in the state of the art to follow presents itself. U.S. Patent 5,004,018,827 to Oram Abram and Hewlett-Packard patents US6037914 and GB2376397 describe and claim portable real image overlay devices with computer generated images for use in augmented reality glasses which are not at all used and much less are the object of the present invention.

US2004 / 0080547, US2003 / 0179218 and US2002 / 0036649 present methods and devices for integrating real and virtual images for applications in specific environments. This is not the scope of the present invention, which is independent of the environment, and whose object is the integration of special accessibility needs to the information. PCT / PT2006 / 000012, from YDreams - Informática, SA presents an augmented reality apparatus similar to that used by the present invention, but is not its object, which is the special needs of information accessibility, which are not integrated in the patent PCT / PT2006 / 000012.

The following articles and publications are referred to as prior art in the sense that they are reference texts in the various areas of knowledge necessary to understand the present patent: 1. PEDebevec, Rendering Synthetic Objects Into Real Scenes: Traditional Bridging and Image-Based Graphics with Global Illumination and High Dynamic Range Photography, Proc. Siggraph 98, ACM Press, New York, 1998.

This article deals with virtual image accommodation in light conditions; the object of the present invention is the special needs of information accessibility in the context of augmented reality, not the technical sophistications of the image per se. 4 2. L. Rosenblum, Virtual and Augmented Reality 2020, IEEE Computer Graphics and Applications, Feb. 2000, pp. 38-39. This article deals in a general way with some of the general issues and where it is expected to evolve in augmented reality in 2000, not covering, minimum or indirectly, the special information accessibility needs that are the object of the present invention.

3. F.P. Brooks, What's Real About Virtual Reality, IEEE

Computer Graphics and Applications, Nov. 1999, pp. 16-27. This article deals with augmented reality in general. The present invention, while relating to and utilizing augmented reality, focuses rather on extending accessibility to augmented reality. Additionally, augmented reality is described in this article as " barely working ", i.e., incipient implementation, which is not the case in the present invention, which is a robust solution. 4. R.T. Azuma, A survey of Augmented Reality 2000, Presence-Teleoperators and Virtual Environments, Aug. 1997, pp. 355-385.

This article sets out to describe the state of the art of augmented reality at the time of its elaboration. It focuses primarily on the limitations to date, many of which have been alleviated by the elements included in the preferred implementation of the present invention, but in any case the object of the present invention, which is the integration of special accessibility needs to the information. 5. R. Azuma, Y. Bailot, R. Behringer, S. Feiner, S. Julier, B. MacIntyre, in " IEEE Computer & Graphics 21 (6) ", pp. 34-47, 2001. 5

Like the previous article, it also deals with augmented reality in general, and is in an analogous way different from the object of the present invention. 6. M.O. Berger et. al., Mixing synthetic and video images of an outdoor urban environment, Machine Vision and Applications, Mar. 1999, pp. 145-159.

This article highlights a specific augmented reality technique for urban environments where there are plenty of geometric patterns. Although a predominantly urban application of the present invention may be commercially dependent, this is not due to the technical nature of the present invention, nor is it its object. The abundance or scarcity of geometric patterns does not affect the preferred implementation and is far from the object of the present invention. 7. K.N. Kutulakos et. al., Calibration-free augmented reality, IEEE Transactions on Visualization and Computer Graphics, Jan-Mar. 1998, pp. 1-20.

This article describes an image acquisition infrastructure (camera and optical device), by means of markers and calibration thereof using geometric projections, elements that are not present in the preferred implementation of the present invention and are not related to its object. 8. P. Rokita, Compositing Computer graphics and real world video sequences, Computer Networks and ISDN Systems, Nov. 1998.

This article deals with the essential technical question in augmented reality of the conjugation of the two originally distinct types of images: real and virtual. The present invention, although in the field of augmented reality, has as its object the special accessibility of information accessibility, not sophistication in image processing. 9. N.M. Thalmann et. al., Animating virtual actors in real environments, Multimedia Systems, Mar. 1997, pp. 113-125.

This article, like the previous one, describes a specific and further topical technique of image processing, using different levels of processing in the image overlay and virtual object modeling method, and not entering the subject area of the present invention, which is the integration of special needs of accessibility to information. 10. M.A. Livingston et. al. , Magnetic Tracker calibration for improved augmented reality registration, Presence-Teleoperators and Virtual Environments, Oct. 1997, pp. 532-546.

This article describes methods of compensating errors in image overlay in augmented reality. The present invention is not concerned with improving the overlapping of the image, but rather with integrating special accessibility needs with the information, including, for example, stereo sound. 11. Tracking in Unprepared Environments for Augmented Reality Systems R. Azuma, J. Lee, B. Jiang, J. Park, S. You, U. Neumann, in " Journal of Computers & Graphics, Vol. 23 ", 1999.

This article deals with the general problem of the overlap of the virtual image with the real image, and within this of the very specific problem of object validation and its translation. Although in the context of augmented reality, the present invention focuses rather on the special accessibilities of information accessibility, and not on particular aspects of the nuclear implementation of augmented reality. 12. An Architecture for Outdoor Wearable Computers to support Augmented Reality and Multimedia Application; W. Piekarski, B. Thomas, D. Hepworth, B. Gunther, V. Demczuk in "3rd International Conference on Knowledge-Based Intelligent Information Engineering Systems", Adelaide, Australia, August 1999.

This article deals with an experimental application of augmented reality to portable devices, which in no way conflicts with the present invention except in that they are also an augmented reality application. 13. Augmented Realities Integrating User and Physical Models, T. Starner, B. Schiele, B. Rhodes, T. Jebara, N. Oliver, J. Weaver, A. Pentland in "First IEEE International Workshop on Augmented Reality (IWAR ' 98) ", San Francisco CA (USA), November 1998;

This article describes improvements to the correct overlap of the image and partial integration of the user's own body into augmented reality, both of which have no relation to the present invention, which focuses on the special accessibility needs in the context of augmented reality. 14. Superior Augmented Reality Registration by Integrating Landmark Tracking and Magnetic Tracking, A. State, G. Hirota, D. Chen, W. Garrett, M. Livingston, in "Proceedings SIGGRAPH'96", New Orleans (USA), August 1996. 8

This article deals with the problem of overlapping the image, not including any mention of the problematic of the special needs of accessibility to the information that is the object of the present invention.

Patents and references referred to as prior art of the present technology are the starting point. However, for the present invention presented, a new system is developed which has a characteristic flexibility and due to its specific implementation that allows solving some of the problems present in the previous systems, in the special needs of accessibility to information and the implementation in environments not controlled. 9

SUMMARY OF THE INVENTION The present invention involves an expansion of the implementation of augmented reality, providing implementation in outdoor spaces and to a broad base of users, specifically also those who have special non-visual or non-sound information accessibility needs. With the present invention, any user, with a special need for information accessibility or not, can obtain information about surrounding space by pointing the device, eg building names, recommended transport, historical information, cultural programming and location of infrastructures ( hotels, restaurants, cinemas). You can also interact with purely virtual models embedded in the real image, exploring associated accessible multimedia content. At the interface level, sound and sign language, as well as braille relief (9), allow an expansion of the usability of the device, which, in addition to its locational flexibility, defined both in terms of resistance to conditions environmental and in terms of adaptations to them, define the device as a publicly available equipment, thus being far removed from previous augmented reality implementations that take place under controlled conditions and for restricted purposes. 10

BRIEF DESCRIPTION OF THE DRAWINGS

Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure

Figure 1. Schematic of the visual interface of the device 2. Detail of the buttons of the device 3. Exploded device - all parts and their relative arrangement 4. Mechanism of movement of the handle 5. Handle displacement piston 6. Base and main vertical axis 7 Computer and camera 8. Detachable body 9. Braille relief 10. Detailed interior view of the coin door 11. Suggestion of using the device by a user with special need for accessibility to non-visual information 12. Suggested use of the device device by a user with special need for accessibility to non-sound information 11

DETAILED DESCRIPTION OF THE PREFERRED IMPLEMENTATION OF THE INVENTION Figure 1 illustrates the operation of the present invention. The system interface uses simultaneous communication modes (sound and visual), the function selection process being carried out using braille buttons (Part 20 in figure 3, detail in figure 2) placed on the handle (Part 5 in figure 3), or through the touch screen (Part 33 in figure 3), these alternative interface methods allow integrating special accessibility needs to the information.

As for the structure, it is characterized by being fluid and robust. The vertical axis (part 4 in figure 7) has two conical bearings (part 42 in figure 6) in opposition, with possibility to regulate the speed of movement through a friction disc (part 31 in figure 6). In order to configure the rotation of the system, a rotating connector (part 29 in figure 6) is also included, allowing, if desired, freely rotating the structure, as also there are removable stops (part 24 in figure 5 and part 25 in figure 6 ), which allow the configuration of the range of motion. At the level of the horizontal axis there is a solid stop system (Figure 5) and a simple counterbalance system (Part 43 in figure 5) that allows to withstand stresses. The adjustment of the speed of movement of the handle (part 17 in figure 4) is also regulated by a friction disc (part 30 in figure 5). A reduction system (Part 17 in figure 4) placed between the transmission belt from the handle movement to the horizontal axis regulates the sensitivity of the movement. Finally, at the interface level, the inclusion of stereo sound through a sound amplifier (Part 22, right, in figure 3) and of two two speakers (Parts 22, left, in figure 3) guide the user to intuitively point the structure to the point of interest. The system, by means of the precision sensors (Parts 10 in Figure 8 and part 21 in Figure 6) of both axes, of the buttons (Part 20 in Figure 3) placed on the handle (Part 5 in Figure 3), or from the screen, detects the presence of the user as soon as someone touches the device. After your presence is detected, the initial welcome message is issued with a quick explanation of how the system works.

For the non-visual communication, the user, when approaching the device, through Braille inserts (Part 37 in figure 9) on the apparatus, namely at the level of the coin dispenser (where applicable) (Part 6 in figure 3), and instructions has an immediate notion of the function of the system and what it is about. By placing your hands on the handle (Part 5 in figure 3), you intuitively identify the selection buttons (Part 20 in figure 3, detail in figure 2) also with braille relief. Although augmented reality consists of the image that this special need for information accessibility does not include, the information continues to be transmitted through sound.

For non-sound communication, all sound messages are always accompanied by sign language, and there is always the possibility of control through the touch screen, so as not to divide the user's attention.

As the structure moves, sound messages are automatically triggered as the points of interest configured in the system enter the field of view of the device. By selecting one of the buttons or the screen, it is always possible to obtain detailed information about the most recent point of interest that the device has been pointed at. The device has the possibility to search for points of interest through a system of selection and search of remarkable points with the use of stereo sound messages.

For example: By selecting the left button (Part 20 in figure 3) of the handle (Part 5 in figure 3), the search system is activated in the list of points of interest in the database; the right button (Part 20 in figure 3) allows you to navigate in the same list (each new element selected corresponds to a characteristic sound message); when the desired element is selected, the system enters search mode, constantly guiding the user about the direction in which the viewpoint (right, left, up, down) should point until reaching the point of interest; as soon as the point of interest is found, a sound message with information about the point of interest is triggered. The basic structure of the device consists of: a base, which contains the vertical axis (part 4 in figure 7) and support bases (parts 18 and 19 in figure 3) that support the entire system of the device. The Rear Protection (Part 1 in figure 3), the Main Protection (Part 2 in figure 3) and the Front Protection (Part 3 in figure 3) are attached to the vertical axis (part 4 in figure 7). In the main protection (Part 2 in figure 3) is also attached the Handle (Part 5 in figure 3) to move the Virtual Viewpoint. 14

This structure eliminates the problem of the fragility of the traditional scenic structures, which typically expose the rotational gears; simultaneously protects the

Virtual Viewpoint of the weather and, together with the anti-vandal glass, vandalism. The counterbalance mechanism (Part 43 in Figure 5) of the handle (Part 5 in Figure 3) is essentially constituted by a stop (Part 24 in Figure 5) and a plunger (Part 34 in Figure 5) with an internal spring that ensures position of the handle. In this case it is still possible to adjust the friction through the clamping screws (Part 30 in figure 5) of the friction disk. The Display (Part 13 in Figure 3), attached to the Front Cover (Part 3 in figure 3), is High Brightness (1600 cd / m2). Between the screen and the anti-vandal and anti-glare glass of 6mm (Part 45 in figure 3) there is a touch screen (Part 33 in figure 3) of projected capacitance technology with the ability to function through a glass. Under the Screen, also attached to the Front Cover (Part 3 in figure 3), there are two Stereo Speakers (Parts 22, left, in figure 3) and a Sound Amplifier (Part 22, right, in figure 3 ) to guide the user with special needs of non-visual accessibility to information during their interaction with the invention. The Camera (Part 11 in Figure 7) for capturing the video images is placed inside the Rear Protection (Part 1 in figure 3) protected by the Protective Glass of the camera (Part 7 in figure 3). Inside the guards, the chamber is fixed on a platform (part 15 in figure 7), which platform is integral with the horizontal axis 15 (part 16 in figure 8). All of this mechanism is located behind the Display (Part 13 in figure 3). The rotational movement of the horizontal Axis is imposed by an internal Transmission Mechanism (Figure 4) from the handle (Part 5 in figure 3). This mechanism was demultiplexed through a reducing system (Part 17 in Figure 4) to improve sensitivity in the interaction. A rotating sensor (part 10 in figure 8) was attached to the horizontal axis (part 16 in figure 8) in order to obtain the angle of inclination of the chamber (part 11 in figure 7). For control of the optical system of the camera there is also a Controller (Part 12 in figure 7) located laterally and attached to the Rear Protection. This controls the iris aperture, focus, and zoom of the camera lens.

The Fans (Part 14 in figure 3) located laterally in the Rear Protection (Part 1 in figure 3) and the lower part of the rear cover (Part 1 in figure 3) as well as the Heating Elements (Part 41 in figure 6) associated with a thermostat (Part 44 in figure 8) ensures the proper temperature and humidity level inside the structure, performing active compensation to the atmospheric conditions.

On the Handle (part 5 in figure 3) two push buttons (part 20 in figure 3) are attached as an alternative to the touch screen interface. The remote location of the alternative control means helps circumvent limitations of the physical nature of the user and allows the user to focus the attention on a single area of the device, facilitating its use. The coin system (part 6 in figure 3) is located inside a compartment (part 28 in figure 3) to which access is through the coin door (part 39 in figure 3). This system has as a function to control the start of the application and the time of operation, it is anti-vandalic and has a built-in safe.

There is also a power plug (Part 36 in figure 10), a push button to turn the computer on (part 26 in figure 10), a USB plug (Part 35 in figure 10), a button to simulate coin insertion (Part 27 in figure 10), and a circuit breaker (Part 32 in figure 10) for electrical protection of the system.

Behind the Coin System (Part 6 in Figure 3), secured to the main shield (Part 2 in Figure 3), there is a High Precision Rotation Sensor (Part 21 in Figure 6) which measures the angle of rotation of the device structure .

Placed next to the rotation sensor, there is a rotating connector (Part 29 in figure 6) that allows the device to rotate unlimitedly without twisting power cables.

Being an equipment that can be used indoors, it has its value in exteriors, it requires an industrial computer (Part 8 in figure 7) of great reliability for data processing, generation and composition of images and sound. This computer contains a card to acquire data from the two rotary sensors, the coin system, the buttons and the touch screen. There is also a modem (Part 38 in Figure 8) and its antenna (Part 23 in Figure 3) which, using mobile or wireless network technologies (eg GPRS, 3G, Wi-Fi), will allow connection to a remote server enabling the remote collection of information on maintenance, updating and collection of multimedia data and 17 still consulting statistical data. These data can be utilization rates, failure reports, quantity of coins, etc. Access to this information is done through a remote management application specially developed for the device. The database of points of interest can be located in the device or in remote location, in this case the availability of the information being dependent on the communication by the antenna.

For some applications requiring mobility, a compass and a geo-referencing system (e.g. a GPS) may still be incorporated. Incorporating a UPS (Part 9 in Figure 8) into the system ensures not only the correct shutdown of the system in case of power failure, but also the filtering of the supply voltage of the main components of the device.

Finally, at the structure level, there is also a set of timed relays (Part 40 in figure 8) which guarantee automatic starting when the electrical power is restored.

The position and orientation parameters of the device can both be established by position relative to the points of interest, which is referenced at the installation site; such as by image processing, which can be previously programmed for specific images; the hybrid mode of obtaining the parameters can also coexist. From these parameters, the system automatically determines how the image is overlaid. The system also includes automatic image compensation for ambient and temperature conditions; for example, when there is too much light, the system automatically 18 increases the brightness of the image to allow readability. The foregoing description represents the preferred, non-exclusive implementation of the invention. The present invention also encompasses implementations which, differing from the one set forth above in one or more components or processes, are equivalent.

Caparica, December 22, 2006

Claims (4)

INTEGRATED ACCESS SYSTEM EXPANDED TO VIEW INFORMATION ABOUT REAL IMAGE INTEGRATING SPECIFIC INFORMATION ACCESSIBILITY NEEDS in the field of public landscape display devices for uncontrolled environments with augmented reality technology characterized by comprising a set of components for in particular: - Braille relief knobs (20) present on the control handle (5) of the display system; - Braille relief along the handle (5) and the coin pocket (37), the user's initial contact locations; - a sound device comprising a sound amplifier (22) and at least two speakers (22a) which guides the user to the points of interest in the landscape and provides information on those points of interest; a touchscreen (33) for controlling the device, with a display of a sign language representation that conveys the information about the points of interest in the landscape; so that the device can simultaneously have the tactile, sound and visual communication modalities with the user. A system according to claim 1, characterized in that the structure of the device is: - robust, derived from being made of metal, and the outside glass (7, 45) is anti-vandal, protecting all parts therein except those of physical and visual interaction with the user; - being resistant to different climatic conditions, being impermeable to rain, and integrating thermostat therewith (44) which drives fans (14) and heating resistors (41). A system according to claims 1, 2, 3, characterized in that it comprises an image capture chamber (11) fixed on a platform fixed to the horizontal axis which is reinforced with a friction disk (31) for controlling the speed of movement of the visualization system; all said mechanism is located behind the screen (33); the rotational movement of the horizontal and vertical axis is imposed by an internal transmission system originating from the handle 5; to the horizontal axis is coupled a rotary sensor in order to obtain the angle of inclination of the chamber; the control of the camera's optical system is done by a laterally located controller attached to the rear guard (1) which controls the iris aperture, Focusing and Zooming of the camera lens. A system according to claims 1, 2, 3, characterized in that it comprises a modem (38) and a respective antenna (23) which, using alternative mobile or wireless communication technologies, enables connection to a remote server enabling the remote collection of information related to the maintenance, updating and collection of multimedia data, as well as the consultation of statistical data. Caparica, December 22, 2006