ES2233127A1 - Intelligent interactive information system i.e. Precision Dynamics Corporation, has vending system providing different information services and telematics services, and perceptual module performing visual input signals - Google Patents

Abstract

The system has a vending system providing different information services and telematics services. A perceptual module performs visual input signals. A cognitive module analyses sensory data and generates effector actions. An effector module manages the effector actions that are generated by the cognitive module.

Description

Intelligent Interactive Information System. (Smart Kiosk).

Technical sector

IT, interface multimodal man-machine, autonomous systems smart, computer vision, interactive multimedia.

Background of the invention

The man-machine interaction is currently based on the use as communication tools of certain devices that are not natural to human beings and their evolution. Access to these media man-machine demands training and, in general, its use is not close and can cause rejection, although its development has been marked by a clear intention to facilitate precisely the interaction. The paradox of which machines is presented made to facilitate the relationship do not adapt to the human being, and it is he who must adapt to obtain his advantages, originating a variety of the so-called functional illiteracy.

During the historical evolution of the development of man-machine interfaces have been used different metaphors:

- 70s, Metaphor: Typewriter. Implementation: Command line interfaces.

- 80s, Metaphor: Desktop, Implementation: GUI / WIMP.

- Years 2000 -... Metaphor: Natural Communication, Implementation: PUI-input multimodal output.

However, the current interfaces of computers, or in general those related to the technologies of the information and its applications in general are basically more functional than social or natural, developed in principle for productivity related applications in processing in offices and conceived at the time (for reasons purely technological) with the idea of overcoming the semantic abyss that separates people from machines.

The last two decades have not seen fundamental changes as far as Human-Machine Interaction is concerned. Most users are still limited to interacting with computers through keyboards, monitors and mice. The use of interface systems that enable interaction with the user according to these traditional paradigms has been the subject of patent, including the following, for its relationship with the invention that
proposes:

to)

Architecture Multimedia for Interactive Advertising: This is a system interactive multimedia that has a video server for provide a primary video program interspersed with secondary multimedia programming, and a database for store information about the viewer and the content of the primary video program Multimedia programming secondary has a variable content determined by the data of the database, and as such, can be changed based on a demography of the individual viewer in conjunction with the topic presented by the primary video program. Preferably, it access the system through the Internet, where the subscriber's house is connected to an Internet network head or node through a cable modem

b)

Instant System Information: which includes a touch screen on which it is displayed a selection menu with graphic support and a simulated keyboard; a microcomputer not accessible from outside that may include specific communications cards; a service keyboard inaccessible by the user; a printer that provides the Requested documentation and a piece of furniture that includes all previous elements. The instant information system is particularly suitable as a conventional information point, self-service point of printed matter and self-service point of formalities.

C)

System of Interactive Advertising for Online Terminals: This is a system to provide warning information in a system interactive that has a plurality of terminals remotely located. Each terminal includes a unit of representation of video, logic and a video storage medium such as a hard drive in addition to normal system functions interactive of which is an example a lectern. A computer commercial remote, centrally located, capable of transmitting digitized signals representing commercial offers and comprise digitized video signals to represent on the Video unit communicates with each terminal remotely located. The compressed digitized video signals are stored on the hard drive of each terminal to represent over the video unit at certain times by using of the terminal. The user selects the services or goods offered by the terminal in the conventional manner. During the timeout inherent in terminal operation a message High quality video warning is represented on the unit of Video from the hard drive. The message will last less than 15 seconds and the user selects function to continue at the conclusion of the message. The warning message is changed, updated and varies directly from the central computer.

d)

Post Advertising-Informational: characterized by present a fixed support structure in which a computer equipment composed of a computer to which they are connected several monitors for the cyclic presentation of a series of advertising spots and an external control panel with some buttons that allow the user to select within the information available on the device, the one you wish to view on a certain monitor; the monitors being distributed on the lateral surface of the post, so that at least one of they can be visualized from any point peripheral.

These teams have similarities with the invention that is proposed, but does not provide the substantive difference which is the integration of modules and a process architecture that manages images and sounds of the environment, allowing localization and identification of user gestures combined with sources sound and produce, through a cognitive core, as explained in the description section, motor actions, generation of gestures about a synthetic character and the issuance of messages sonorous

Recently, the development of post-WIMP interface and interaction techniques, including among them, both desktop solutions that support 3D graphics, such as multimodal interfaces, the virtual reality or augmented reality. All of them need the support of powerful, flexible natural interaction techniques, Efficient and expressive among which this invention is situated. He interfaces paradigm that brings together technological approaches that explore the approach to interaction is precisely that of Perceptual User Interfaces (PUI), that is, those that takes natural communication skills into consideration as well as our tendency to interact with technology so social, to model the interaction man-machine

The use of vision systems for identification, gestural and / or positional interpretation of perception systems auditory with voice and / or sound recognition, for location or location or for the reception of orders, commands and information verbal nature from the environment, as well as systems of voice generation to provide auditory or real gesture generation from physical or graphic mechanisms synthetic

Explanation

The "Smart Kiosk" is about a interactive intelligent system that physically resembles a Small autonomous kiosk, and that can be used to provide information or other services, as well as integrate with a system of vending

It is based on the application of technologies Artificial Intelligence in the interaction man-machine

The product provides, compared to traditional communication interfaces with a computer, the possibility of Natural, simple and direct communication with the user / client. Thanks to Artificial Intelligence technologies, it joins the system "human" qualities as a friendly attitude and nice in the interaction using a character synthetic or software capable of gesturing. It is precisely this ability to have human appearance and reactions what makes a difference in competitiveness by being more accepted by the Username.

The significant and differentiating of this kiosk smart with other similar products is that it allows the natural communication, that is through voice and image, with the user / client that can be done in several languages in addition to allow autonomous screen adaptation movements informative to users of different heights.

The system as a whole constitutes the framework for natural communication based on the agents paradigm smart perceptors, capable of the acquisition of multimodal information (sound, image, lighting and presence) of the environment and multimodal production of actions (on the synthetic character, the parameters of the cameras, performance engines, devices communications and touch screen content). The control of activity will be executed by a cognitive module that will combine adequately reactive functions with those of character deliberative and will provide "intelligence" to the system.

Description of the drawings

Figure 1: Simplified block diagram of the system.

This figure schematically shows the modular organization that allows the realization of the system and It enables interaction with the user from images and sounds. In the system architecture, three basic blocks are distinguished: General perceptual modules, cognitive system or module and General effector modules.

Figure 2: Scheme of the cognitive system.

This figure schematically illustrates the internal organization of the cognitive module (1) that allows the incorporation of intelligent capabilities to the system.

Figure 3: Representative hardware scheme used.

This figure shows schematically the Minimum hardware architecture for system realization.

Figure 4: Prototype with the scheme of devices that integrate it

This figure shows a possible embodiment device physics functioning as an information point intelligent in a classic base and support structure.

Embodiment A) Development of general perceptual modules (1) (Figure 1)

Its inputs are signals from sensors (images (6), sound (7), or signals from other types of sensors) and perform tasks corresponding to modalities perceptual:

-

Visuals: Includes the development of an adaptation and encapsulation of the monitoring module visual of people or objects (2) and the development of the module Gesture Recognition and Extraction (3), which includes detection, recognition and gesture tracking.

-

Auditory: What it consists of the adaptation and encapsulation of two modules, one of Sound Source Location Recognition (4) and other Voice Recognition (5).

-

Others: include information from other types of sensors (detectors of presence and lighting).

These modules are integrated into the system through the development of software templates that are generic for the Sensory Modules.

B) Development of the cognitive module or system (8)

Performs the analysis of sensory data and generates effector actions with an internal architecture that allows you to display credible character behaviors for Natural Interaction Three elements are distinguished in it constructive:

1. Development of the multimodal integration module (9)

Integration oriented for disambiguation and the strengthening of afferent information by merger at the level semantic. This development includes:

- The Development of integration techniques multimodal that allow simultaneous conjunction of image and sound from the visual and auditory modules.

- The performance test of the module, where the goodness of it is checked with an example case in which the detection of moving faces is combined with the location of two sound sources. Passed this test must proceed to the realization of the following element.

2. Development of the cognitive core (10) (figure 2)

Allows high level care processes controlled by the behavior management mechanisms of a synthetic character at every moment, modulating, inhibiting or encouraging behaviors based on the evolution of the interaction through homeostatic and emotional mechanisms. In this core defines the agenda and the main sequence activations Its internal process structure is synthesized with the following developments:

-

Development of the support infrastructure the Cognitive core consisting of the definition and implementation of data structures and communications between the subsystems.

-

Development of the communications infrastructure with the integration module Multimodal input and action decoder module of exit.

-

Development of Behavioral Subsystem (SSC) software: corresponds to the network of system behaviors.

-

Development of Stock Selection (SA) software that determines the following action to take at every moment.

-

Development of Motivation Subsystem (SSM) software that corresponds to the behavioral mechanisms of synthetic character behavior at every moment, modulating, inhibiting or encouraging behaviors depending on the evolution of the interaction to through homeostatic and emotional mechanisms. In this subsystem the agenda (SSP) and the main sequence are defined activations

-

Development of Subsystem Planner software. (SSP) that allows the selection of plans from the agenda.

-

Development of Perception Management (SPG) and Share (SGA) software: act as a core input and output link.

-

Development of Attention Subsystem (SSA) software responsible for activating initially the high level care mechanisms by the fixing the look on the user each time this is It is in front of the system.

3. Development of the action decoding module (eleven)

Whose object is the translation of the actions selected by the action selector in action sets concrete on the different effectors (Emission of sounds, platform movement or gestures of the synthetic character animated). For example, a cheerful greeting action with a face synthetic implies the vocal generation of the greeting in addition to the generation of the corresponding gesture in the face. For your realization of this module should be performed the following points:

-

Develop the data structures corresponding to decoding tables and decoding procedures.

-

Development of the decoder module interface template with modules effectors so that each related action is transformed into a script on the different modules effectors

C) Development of effector modules (12) (figure one)

Similarly to those corresponding to the Modules General perceptual, these modules manage the action on the effectors, as generated by the Module tables Stock Decoder (11). These can be physical, such as interaction with the support motor control card (13), others may be action on other third source software, such as the generation of voice and sound messages (14) and others on resources, such as an animated graphic action with the manager of the synthetic character (15).

The complete system behavior is will control through a Cognitive Core, (10), which will manage the analysis and integration of perceptual activity and synthesis of effector activity.

The system also includes the architecture of software processes and the management mechanisms of the various behavior patterns to follow in the relational events with the medium, trying at all times to provide the most next possible to a "humanized" behavior. Are included here the preparation of dialogue messages and the design and programming of a multimodal software interface that will allow interpretation and gestural emulation for the exchange of information (verbal and gestural) between people and the system.

Its realization therefore includes:

-

Development of templates and capsules of the effector modules.

-

Development of Animated Synthetic Character control module with integration of the vocal movement actions of CSLU Toolkit.

-

Adaptation and Encapsulated Voice Synthesis Module.

-

Partial tests of integration with synchronization of animated actions and voice.

With respect to the minimum hardware architecture necessary for the realization of the system (figure 3), the invention is articulated, on the integration of a PC-type Computer (16) of two cards, one for acquisition and image processing with a video multiplexer (17) and another motor control with its respective elements (18) which allows to control the inclination of a platform (22) with a degree of freedom. In addition to the optics of the cameras (21) can be controlled from a plate of four serial ports (20). The hardware is completed with the integration of a sound card (19) that allows to use a stereo pair of microphones (24), a presence sensor (23) connected to the board Serial ports (20) and a modem to facilitate communications remote and its interconnection with other equipment. The set is located in a rigid support structure of base type design (see Figure 4) at whose base the PC and the accesses for the Communications and food.

Finally, as depicted in the figure 4, a prototype is presented with the scheme of the devices that they integrate it, where a possible physical realization of the device functioning as an intelligent information point in a Classic base and support structure. The product has a pair stereo cameras (27), a pair of microphones and a pair of speakers (28) located in a folding and mobile multimedia support (25), which subject to a platform (29), in addition to other minor sensors such as they are a light level detector or a presence detector. He system will have a touch screen (26) where a synthetic character that mimics the gestures of a person's face and an area will be reserved to confirm certain orders by the Username. This screen will be located on a motorized stand. (25), with a degree of freedom to allow the variation of their inclination and adaptation to variations in external brightness (to facilitate vision to the observer) or at the point of vision from the observer (if he moves away or has a small height). The height adaptation is set automatically based on internal decisions that are established based on the information sensory provided by the cameras.

The system may be used industrially from as follows:

a) Mode 1: " Standalone " Can be used in isolation as an individual information point.

b) Mode 2: " Interconnected " Can be used as an input or output interface for voice and image connected to a local or remote device. For hotel reservation applications, telematic services, distance shopping, etc. In addition, the option of interconnecting several "Smart Kiosks" is supported, which opens up an important range of possible applications

c) Mode 3: " Embebed " As an interface, "embedded" can be used with a vending system (for example), where intelligent communication technology provides the "friendly" and "humanized" interaction elements with the vending device. It can be coupled and integrated with other devices (card readers, printers, vending machines, etc.).

For applications of:

a) Intelligent Information Points , with the possibility of communicating with other systems and with the ability to remotely view images online.

b) Product promotions . These can be static or dynamic, allowing the online reservation or purchase of products and services.

c) Education and Entertainment . The "Smart Kiosk" is an interface that can be used in the development of demonstrations for training courses and seminars.

d) Vending . Coupled or embedded in a traditional vending system, it provides the humanized interface that simplifies the use of a more attractive way to the potential consumer.

e) Telematic services . The interconnection possibilities of the device make it an intelligent agent that can provide various telematic services.

Claims (2)

1. An Intelligent Interactive Information System, characterized by the integration of modules and a process architecture that manages images and sounds of the environment, allowing the location and identification of user gestures combined with sound sources and produce, through a cognitive core , motor actions, generation of gestures on a synthetic character and the emission of sound messages. 2. An intelligent interactive information system according to the preceding claims characterized by an architecture that integrates two types of perceptual modules: a.1) Location and identification of faces and gestures from images, which captures from a pair of stereo cameras located on an oscillating mobile platform with a degree of freedom, b.1) Identification and location of sound sources and three types of action modules: a.2) Animated synthetic character management b.2) Motor control of support, c.2) Generation of sound messages.