RU2370829C2 - Method for authorisation of voice commands used in interactive video presentation system - Google Patents

Abstract

FIELD: information technologies.

SUBSTANCE: invention is related to functioning of video presentation systems, namely to methods for organisation of interaction between user and presentation system by means of radiation device and device that performs recording and analysis of speech. This result is achieved due to processing of obtained voice command parallel with processing of confirmation light signal. At the same time light signal used is projection of light image to presentation screen from pre-established set of reference images, for every of which an association is pre-established with one of voice commands included into pre-established set of commands. In case when confirmation light signal is recognised by control device as matching with reference image, voice command is converted into command for computer and is performed.

EFFECT: higher accuracy in system performance of maximum number of user-performed operations with demonstrated video images.

2 dwg

Description

The invention relates to the operation of video presentation systems, and more particularly, to methods for organizing interaction between a user and a presentation system by means of a radiating device, such as a laser pointer, a device that records and analyzes speech.

Currently, almost every computer presentation system is controlled using active computer input devices, such as mouse devices, a light pen, trackball, or keyboard. However, the use of such devices during the presentation is not very convenient, since the speaker must switch his attention from the audience to a computer display, control device or resort to the help of an assistant who manages the presentation. Presentation remote control devices transmitting control signals via infrared or radio are also known. However, this type of device also requires increased speaker attention and interaction with a projector or computer using a remote control device.

Also known are interactive presentation systems controlled by laser pointer devices. The essence of these systems is as follows: a laser beam is projected onto a presentation screen, forming a light signal in a specific area of the screen. The position of the projected laser spot and its other characteristics (color, nature of movement, etc.) are recorded by the video camera and processed in the control module to recognize the characteristics of the projected laser spot; These characteristics are used to change the position of the computer cursor or execute certain commands, for example, moving to the next slide.

Presentation control systems using voice commands are known that are recognized in the control device and converted into corresponding commands. However, another person or the speaker himself may accidentally or intentionally utter a voice command that should not be responded to. Therefore, it is necessary to provide a natural and convenient way to control a computer presentation, which would combine convenient control of the computer cursor and authorized performance of voice commands that control the presentation.

US Pat. Nos. 5,138,304 [1] and 5,181,015 [2] describe an interactive video system that is controlled by a light pen. In this system, the light reflected from the projected image is compared with the original video signal to determine the location of the light pen. The location of the light pen is used to control the computer cursor. However, the light pens used in these patents allow the cursor to be controlled only from a short distance and are wired to the computer.

In US Pat. No. 5,115,230 [3], a conventional light-emitting non-emitting light pen is replaced with an emitting hand-held laser pointer. The video camera is located behind the projection screen for recording video, which is processed in the analysis module to determine the position of the laser spot. Information about the position is transmitted to the computer and used to display the computer cursor.

The combination of a laser pointer and infrared communication for transmitting commands emulating the operation of a mouse-type manipulator is described in US Pat. Nos. 6,910,778 [4] and No.6,331,848 [5]. Executing commands to control the presentation, such as zooming in, the next slide, selecting objects, can be difficult.

The execution of such commands can be ensured in the following way. In US Pat. No. 6,331,848 [6], soft buttons project onto the screen. By pointing and holding the laser spot on the corresponding button for some time, you can achieve the execution of the command associated with this button. However, this approach may not always be convenient, since it requires the placement of additional elements on the screen, which reduces the quality of perception of the presentation.

The closest in technical features to this invention is US patent 7,091,949 [7], which describes a presentation system that can execute commands depending on the color / shape / size and other properties of the projected laser spot. The main difference of this invention is that the projected laser spot is not used independently, but for authorization and voice command execution. In other words, presentation management is based on the analysis of the speaker’s voice command, and authorization is performed using a laser pointer.

A common drawback inherent in most of the existing methods for controlling the presentation is that they do not allow the user to control the computer cursor naturally and execute presentation commands. Indeed, the most natural way to control the presentation can be considered as a voice command, while the presentation system should distinguish the voice command of the speaker (speaker) from ambient acoustic noise, for example, replicas and audience questions. Since devices for voice identification are complex and expensive, other technical means are more often used, which, in combination with a voice command, ensure that the interactive video presentation system performs the required actions. This is most clearly represented in the prototype [7].

The task to which the claimed invention is directed is to develop such a method of authorizing voice commands that would allow the speaker with minimal distraction to authorize the execution of the maximum number of operations with the displayed video images. It is understood that the new method will also require the development of a new system - a set of specific devices - for the optimal implementation of this method.

The problem is solved by developing an improved method for authorizing voice commands used in an interactive video presentation system, which includes processing the received voice command in parallel with processing the confirming light signal, characterized in that the projection onto the presentation screen of the light image from a predefined set images for each of which preset an association with one of the voice commands included in the pre an established set of commands.

This method is proposed to be implemented using an interactive video presentation system, including:

a projection screen, a presentation computer with a video projector and a microphone that records voice commands, as well as a radiating device that generates a light signal on the projection screen, while the radiating device is configured to project onto the projection screen any of the images selected by the user from a radiating device of a set of images, while the following elements connected to a presentation computer are additionally included in the system: at least Leray, one video camera mounted with the ability to capture images formed on the presentation screen; a control device configured to recognize images projected by the radiating device and voice commands captured by the microphone; a module for generating computer commands, configured to generate and transmit commands to a presentation computer to modify the projected image.

In other words, to prevent unauthorized execution of a voice command, it is proposed to use the image projected on a projection screen using a laser pointer or other emitting device. This image is recognized by the control device and compared with the standard. When they match, the voice command is converted into a command for the computer and executed. On the other hand, it is required to provide control of the computer cursor to perform operations of selecting text and objects, etc. The most natural way of this user interaction with the presentation system is to use a laser pointer to project a laser spot / image onto the presentation screen, the location of which is determined by the control system and used to move / modify the computer cursor and the projected image.

As for the implementation of the proposed method, an interactive presentation system is proposed for this, with which the user interacts using a radiating device (hand-held laser pointer) and voice commands. The laser pointer is capable of projecting a laser spot (image) of various shapes onto the presentation screen. The projected image is selected by pressing the appropriate buttons located on the body of the emitting device. The number of buttons, their location and the projected images depend on the specific implementation of the presentation system. Concretization of the design of the corresponding radiating device capable of projecting images of various shapes is beyond the scope of the current invention, however, today the electronic market offers a very wide selection of multifunctional light pointers.

The image and presentation image projected onto the presentation screen are recorded using one or more video recorders. Video recorders are placed either in front of or behind the screen. The recorded video sequence is transmitted to the control device, where it is processed in real time. Also, digital audio is transmitted to the control device, which contains a presentation speech and voice commands of the speaker. Recording audio information is performed using a microphone, conversion to digital form is performed using a digital-to-analog converter (DAC).

The process of the proposed presentation system can be described as follows. During the presentation, presentation images are generated using a computer and projected onto the presentation screen using a projector. At the same moment, the presenter can project a light image onto the same screen using a laser pointer. The camcorder captures the image formed on the presentation screen, which consists of the presentation image and the projected laser image. Using the control system, the video sequence received by the camera is analyzed to find the light image and its location is calculated in the coordinates of the presentation image. Each light image is associated with a presentation team, such as, for example, pressing a mouse button, highlighting text, drawing on a presentation image, the next slide, etc. The recognition of the light image is carried out using well-known methods of pattern recognition and image processing. The recognized image is converted into a command, which is transmitted and executed on the presentation computer. As a result, the presentation image is changed at the request of the speaker.

The movement of the light image is also tracked in the control system and is used to change the location of the computer cursor. Depending on the selected mode, the cursor can be used to highlight objects or draw graphic primitives on a presentation image.

An important presentation team is the voice command authorization team, which is recorded using a microphone and analyzed in the control device. The speaker makes a voice command, which is recognized in the control device and converted into a command to be executed on the computer. Note that the absence of speech or sounds can also be considered as a “voice command”.

However, to execute the corresponding computer command, it is necessary to obtain additional confirmation in the form of the corresponding laser image projected onto the presentation screen. This confirmation can be received immediately before, during, and immediately after a voice command.

Also, a voice command or a speaker pause can be used to authorize a computer command generated by the projected image.

The proposed method can significantly reduce the likelihood of unauthorized management of the presentation system. The likelihood of an erroneous reaction to keywords contained in voice commands is also reduced. This invention allows the speaker to concentrate on the presentation and the audience and provides a natural way of interacting and managing the presentation system.

Figure 1 is a diagram of the components of a presentation system according to the invention.

Figure 2 displays a set of light images projected by a laser pointer and recognized by the control device.

Figure 1 shows the components of an interactive presentation system consisting of: a radiating device 2 (for example, a laser pointer), configured to project light images of different shapes; presentation computer 9, generating presentation images; a projector 7 connected to the presentation computer 9 and projecting images onto the presentation screen 14; digital video camera 8, microphone 19 and the control system 10 with encoded images (Figure 2) and voice commands for subsequent recognition.

During the presentation, the presentation computer 9 generates a video stream 15.1, consisting of presentation images. This video stream is converted into a light stream 16, which is projected onto a presentation screen 14, on which a presentation image 5 is formed.

The speaker 1 interactively controls the presentation system using the emitting device 2 by projecting light images onto the presentation screen 14 (FIG. 2). The radiating device 2 has function keys 4. By pressing one of these keys, the corresponding image is selected, which is projected onto the presentation screen 14 using a laser beam 3.

Each projected image is associated with a specific team. Therefore, speaker 1 must press the corresponding key 4 to authorize the team assigned to it. The association of the team with the image is either fixed and programmed rigidly, or assigned by the user during installation and configuration of the presentation system.

The projected image 6 and the presentation image 5 form a common image 17, which is recorded by a digital video camera 8. The sequence of recorded images 13 is transmitted to the control system 10. An initial presentation image for projection 15.2 is also input to the control system 10. The control system 10 also contains encoded images 12, which are identical to the images projected using the emitting device 2. The control system includes an image and speech recognition module 11, which compares the projected images 15.2 with the images captured by the camera 13, searches for projected images and their location . Then, the recognized image is converted into a command or voice command authorization using the command generation module 18, according to the image and command association. The generated command is transmitted to the presentation computer to modify the projected image. To further confirm the execution of the generated command, you can use the corresponding voice command or the speaker pause.

The pattern and speech recognition module 11 is also responsible for recognizing voice commands recorded using the microphone 19. To execute the recognized speech command, you need to receive confirmation in the form of a specific image projected onto the presentation screen.

A presentation system using the claimed method of interaction may find application for noise-immune control of computer presentations and electronic boards. The proposed method and its implementing system can be used instead of active devices (mouse, keyboard, etc.) to control the computer cursor, to select and execute computer commands, and also to enter text information.

Claims (1)

A method of authorizing voice commands used in an interactive video presentation system, including processing the received voice command in parallel with processing the confirming light signal, characterized in that the projection onto the presentation screen of the light image from a predefined set of image standards is used as a light signal, for each of which preset the association with one of the voice commands included in the predefined set of commands, while the voice command converts It is entered into the command for the computer and is executed only if the confirming light image is recognized by the control device as coinciding with the reference image.

Images