SK289006B6 - Information device with simultaneous feedback gathering and method for presentation of the information - Google Patents

Abstract

At least one camera (5, 6) captures the person before the display unit (1 ), a reaction of the person is analyzed from the collected image and the settings of the presentation of the information are altered according to the reaction. The information device has a display unit (1 ) for display of the visually presented information, at least one speaker (2) and a computer (3) with a memory for the presented information. The device includes at least two cameras (5, 6). The stable camera (6) is connected to the module (7) for the recognition of the position of the person and for the determination of the position of the face; the output is connected to the control of the direction of the movable camera (5). The output from the movable camera (5) is connected to the computer (3) in which there is a facial expressions recognition unit (8). On the basis of the image data, the bodily position of the limbs and/or the facial expression and/or the change in the heartbeat and/or the sex and/or the approximate age of the person before the display unit (1 ), is determined, and the mode and/or content of the presentation is chosen on the basis of these data.

Description

The field of technology

The invention relates to an information device that provides information, activates perceptions, creates stimuli for a person in the vicinity of the device, while the device also records and possibly evaluates the feedback of the person receiving the information, perceptions, stimuli, and can also create or influence a suitable atmosphere of the environment. The obtained feedback can not only be recorded for further statistical processing, but can also be used to immediately influence the way information is presented, including changing some properties of the surrounding environment.

Current state of the art

In many sectors of the economy, as well as in various social situations, it is necessary to obtain feedback from persons to whom information, perceptions and stimuli are provided. Questionnaires that respondents fill out are commonly used for this, but this is a very lengthy procedure that can be burdened with many system errors. In reality, questionnaires obtain a subjectivized image of the conscious projection in relation to the relevant question, but such an image often does not correspond to the actual subconscious reaction of the relevant person. A more modern substitute for classic questionnaires are various electronic devices where a person can choose his answer, for example by touching the corresponding icon. Such devices, for example as per FR 2998077 A1, provide only very limited response data without exact synchronization with the information presented. The collected results are evaluated only later and the presentation of information for other people can be adjusted according to them.

Publication AU 2014101196 A4 describes a solution for obtaining feedback using a web-based application on a mobile phone, which, however, requires a high degree of cooperation from the person concerned. According to US 2011218930 A1, feedback is obtained using single-purpose electronic devices that must first be distributed to customers, which is complicated and narrows the group of evaluators.

There is an interest in creating a device that will present the required information of various kinds and at the same time receive an immediate immediate reaction, feedback from the person involved, or will also be able to continuously adapt the way of presentation according to the current situation and according to the reaction of the person involved.

The essence of the invention

The mentioned shortcomings are substantially eliminated by the information device with the simultaneous acquisition of feedback, where the device includes a display unit for displaying visually presented information, at least one speaker and a computer with a memory for the presented information, where the display unit is connected to a computer according to the present invention, the essence of which is in that the device includes two cameras, one camera is directionally positionable, the other camera is stable, both cameras are oriented to the space in front of the display unit. The stable camera is connected to the module for recognizing the position of the person and determining the position of the face, the output from this module is connected to the direction control of the positionable camera. The output from the positionable camera is connected to a computer in which there is a facial expression evaluation unit.

In this basic arrangement, visual and audio information is presented to a person or persons near the device. If a person is in the field of view of a stable camera, their position is recognized in the module and the current position of the person's face is evaluated. A positionable camera is directed to this position, which captures the person's face in the image. From this shot, image data is now obtained for the evaluation of facial expressions, these results are time-synchronized with the presented information. Known algorithms can be used to evaluate facial expressions, facial expressions and micro-facial expressions.

A positionable camera can have a variable focal length that changes according to the recognized size of the face in the image from the stable camera. This size depends primarily on the actual distance of the person from the display unit.

The information presented can be of any nature and content, most often it will be video recordings with sound. They can have an educational, presentational, promotional or other role.

The module for recognizing the position of a person, which is connected to a stable camera, can also be adapted to evaluate a person's gestures and movements of his limbs. The person standing in front of the display unit also reacts to the received information by changing the position of his body. Changes in body position also reveal a person's reaction to the information being received.

It is advantageous if both cameras are placed on the front surface of the device, where the display unit is, while they are placed above the display unit. The device can have an outer casing in the form of a separate panel, which is designed to stand alone on the floor or to be hung on the wall.

From the point of view of simple assembly of the device, it will be convenient if a commercially available device used in the game industry, which is capable of continuously creating a vector model of a character from an image, is used as a stable camera with a module for recognizing the position of a person. The body of a person with the relevant position of the limbs, possibly also with the position of the palms and fingers is recognized and described in a three-dimensional system. Some devices are also able to analyze a person's pulse, gender, approximate age and basic micro-emotions. The data from the still camera is not only used to direct the moving camera, but is also sent to the computer as feedback evaluation data.

According to the feedback, the content of the presented information or the form of its presentation, volume, brightness, image zoom, etc. can change.

With the help of neuroscience, optically and remotely sensed functions of the human body can be evaluated when forming an attitude towards the presented topic, for example towards a specific product or service. Examining the emotions of those to whom information is presented is increasingly important. According to neuroscientific theses, people's decision-making is not very rational and is mainly based on subconscious processes in the mind. The obtained data is interpreted and related to the judgement, emotion and decision-making of the person.

In an advantageous arrangement, an aromatizer will be part of the device, which releases the aroma into the surroundings of the device. Several aromatizers can also be used, i.e. several dispensers of different aromas. It is also convenient if the device includes a sensor for the presence of a person in the vicinity of the device. The sensor is connected to the computer. Starting the presentation on the display unit, as well as starting the operation of the entire device, can be controlled by just such a presence sensor. It can be a motion sensor that detects the entry of a person near the device, but it can also be a sensor based on another type of detection, such as a noise sensor, a thermal imaging sensor or a pressure sensor on a pad, e.g. under the carpet in front of the device. Analysis of image data from the camera can also be used to detect the presence of a person in the vicinity of the device, the image of a person's figure and their movement in the field of view can be identified in the data.

In addition to the display unit for displaying multimedia content, the device may also include an alphanumeric display element, which will mainly be used to display the time, date, or ambient temperature.

The color of the environment affects human perception, the device can have RGB LED lighting with adjustable color, which is controlled according to the content of the multimedia presentation, possibly also according to the received feedback. The lighting can also be adjusted according to the surrounding climate, or microclimate, when the air pressure in the immediate vicinity or the weather conditions in the location of the device are taken into account, for example. Weather data can be obtained from own sensors or from a third-party source, for example from data available on the Internet. RGB LED lighting will usually be located on the sides of the device.

The device may also include a wake-up indicator. The wake-up indicator can graphically accompany the sound of the device, for example, it will be rhythmically linked to the musical background of the presentation so that the light points of the wake-up indicator will light up in the rhythm of the accompanying music. The wake-up indicator can signal the degree of interaction of the person with the device according to the evaluation of the feedback. In this case, the wake-up indicator will signal, for example, the degree of the device's ability to recognize facial expressions.

In order to take into account the external effects of the weather when setting the operation of individual elements and evaluating the feedback of a person, it is advantageous if the device has a corresponding sensor or several sensors. It can be a barometric pressure sensor, an external temperature sensor, an external humidity sensor and an external lighting sensor. Information about the current nature of the weather is obtained from their data. This data can also be obtained from the Internet for the given location of the device, for which the appropriate Internet connection of the device will be required.

Aromatizer, presence sensor, RGB LED lighting with adjustable color, wake-up indicator, environmental sensors around the device or exterior sensors can be connected and controlled from a computer directly or through one or more microcontrollers. It will be advantageous if one or two microcontrollers are used to control a group of related peripheral elements.

The shortcomings mentioned in the state of the art are largely eliminated by the method of presenting information using the display unit and the speaker according to the present invention, the essence of which is that at least one camera captures a person in front of the display unit, the person's reaction is analyzed from the obtained image, and according to the reaction change information presentation settings.

Analyzing a person's reaction may include changes in body position of limbs, changes in facial expression, change in heart rate, and the like. In doing so, the gender and approximate age of the person in front of the display unit can also be determined.

Changing the setting of the presentation of information may include choosing the accompanying sound, changing the volume, creating a light atmosphere around the device, releasing a scent or even changing the content of the presentation. When influencing the presentation, the influence of the surrounding environment, such as barometric pressure, outside weather, and the like, can also be taken into account.

The device according to the present invention makes it possible to effectively detect and evaluate the consumer's attitudes, to influence him with the help of a presentation that is adapted according to his feedback. For example, if the person in front of the device reacts positively to the accompanying music, which is manifested by rhythmic changes in the person's body, the device can increase the volume of the accompanying music or select a related song in the further course of the presentation.

The device has a multispectral use in various types of organizations with research as well as practical focus (hospitals, businesses, shops, service sector, public institutions).

The facility provides a platform for combining the latest knowledge in the field of consumer neuroscience (neuromarketing) and communication and information technologies. The device is able to identify the representation of the audience through image analysis (gender, approximate age) and monitor their interest through a directionally adjustable camera. Based on the micro-emotions detected from the facial expression, it can adapt the atmosphere of the environment through various colors, scents and sound, while fully taking into account the current state of the weather outside and the temperature of the environment inside. It provides the public with information about the current time, temperature and date. The device simultaneously communicates the activities and activities of the organization (presentational video recording) towards the audience. The device provides researchers with sufficient information about the visual appeal of the presentation and the reactions of the audience due to the activated factors influencing the perception of the environment (different colors, music, smell). By combining the communication, information and research areas into one functional unit with a modern design, it is possible to create a favorable interior atmosphere and psychological well-being in the workplace. This interconnectedness of functional units can be verified in detail, evaluated and subsequently adapted to the satisfaction of the audience. Motion sensors allow you to activate colored lighting, smooth start of the sound system and the spread of fragrant essence by walking near the panel. The control unit of the panel will fully take into account the times of occurrences (breaks) of the potential audience within the maximum reduction of energy demand.

The device can present activities, projects, advertisements and activities taking place in a given location, for example at the workplace, on specified days and times. This communication activity will be supplemented by various factors affecting the general perception of the environment (colors, smells, sounds) and psychological well-being (micro-emotions) of the public. In case of movement around the device, it will be able to use the motion sensor, or presence sensor to detect the presence of people and immediately adapt the environment to the influence of colors, sounds and smells of this fact, taking into account the weather, temperature of the environment, but also the gender and approximate age of passers-by. The advantage of the device is that it simultaneously records the reaction of the audience (especially based on their facial expressions) to the presentation and the flexibility of the device. The received feedback will make it possible to analyze the reactions and subsequently make changes that will contribute to the improvement of the presentation itself, or communication of the facility and the creation of a favorable and engaging environment. Since the device has the appropriate peripherals to affect the presentation atmosphere, the device is not only able to collect data to further evaluate the impact of the presentation, but is also able to instantly adjust the current presentation conditions.

The device and method according to the present invention will be very effectively used in marketing departments of commercial companies, in all spheres of business, in survey-marketing agencies, in advertising agencies, in research and scientific centers, in non-profit organizations and in public institutions (offices, stations, hospitals , universities, schools). A device with a connection to a communication network can be fully controlled remotely, the presented content as well as the presentation criteria can be changed remotely. At the same time, the collected data and information can be retrieved from the device using remote access. This will allow multiple devices to be managed from a central point.

The advantage of the invention is the automated process of obtaining feedback and the possibility to influence the conditions of the presentation. There are a wide variety of different presentation devices, all of which basically work on the one-sided principle of transmitting pre-stored or remotely received information. The present invention breaks this stereotype, provides technical means for a much more flexible transmission of information, when the presentation can be adapted to immediate interaction with the audience and the device can estimate the attitude of the audience with a certain probability and interactively "live" with it. This makes the participants more interested in the content of the presentation. The device and method according to the present invention is characterized by a certain degree of empathic capabilities, which is subconsciously close and human to the audience.

Overview of images on drawings

The invention is explained in more detail with the help of figures 1 to 3. The displayed shape of the panel, as well as the exact position of the individual elements within the device, are only illustrative, they are not meant to be explanatory or to narrow the scope of protection.

Figure 1 shows a view of a stand-alone device that has more complex equipment with more peripheral elements.

Figure 2 shows the wiring diagram of the most basic assembly of information equipment with simultaneous feedback acquisition.

Figure 3 shows the wiring diagram of the device with a more complex set of elements.

Examples of implementation of the invention

Example 1

In this example, according to Figure 2, the information device has a display unit 1 in the form of a flat large-screen screen, which is connected to a miniature PC computer 3. The display unit 1 also includes speakers 2. The computer 3 has a memory 4 in which, among other things, a set of various multimedia presentations.

A stable camera 6 is connected to the computer 3, which has two optical elements with a lateral distance from each other. The image output is analyzed in the own module 7 for recognition of the person's position. From this module 7, the computer 3 receives information about the current position of the person in front of the display unit 1.

There is a control program in the computer 3, which, based on the detected position of the person, directs the directionally positionable camera 5. It records the image of the person's face, the image data is analyzed in the expression evaluation unit 8, the result is recorded in the memory 4 together with the exact time in relation to the presented multimedia content with different information. According to the face image, the age and gender of the person in front of the display unit 1 is also estimated in the unit 8.

The results from the evaluation of facial expressions are not only stored in the memory, but the presentation of the multimedia files is also changed according to them. The feedback is taken into account according to the set criteria, for example, according to the estimated age and gender, the music that accompanies the presentation of information changes. If the person moves to the rhythm of the music, the device evaluates this state as consent to the choice of music and increases its volume.

Example 2

The information device according to Figures 1 and 3 has the external form of a free-standing panel 9. Individual elements of the device are stored in it. In the front wall there is a display unit 1, on it there is a pair of stable cameras 6, a directionally positionable camera 5, an alphanumeric display element 12 and a wake-up indicator 14. Below the display unit 1 is a motion sensor 11 presence. On the sides of the panel 9 are strips with RGB LED lighting 13, the output from the aromatizer 10 and side speakers 2.

A display unit 1, directionally positionable camera 5, communication element 19, two speakers 2, a pair of stable cameras 6 with a module 7 recognizing the person's position (Kinect system) and a wake-up indicator 14 are connected to the minicomputer 3, which is stored in the cabinet of the panel 9. The other elements of the device are connected to the minicomputer 3 indirectly through two microcontrollers 17. One microcontroller 17 operates the alphanumeric display element 12 and the temperature sensor 16. RGB LED lighting 13, presence motion sensor 11, barometric pressure sensor 15 and aromatizer 10 are connected to the second microcontroller 17.

The wake-up indicator 14 in this example works in two basic modes. In the first mode, it works as a lighting accompaniment to the sound system, when it dynamically lights up according to the rhythm and change of tone of the reproduced music. In the second mode, arousal is shown according to the device's ability to recognize a facial expression or other evaluated feature. For example, five LEDs lit green indicate excellent contact, i.e. excellent feedback. If three LEDs light up orange, it means good contact, if only one red LED lights up, it means minimal contact with the person in front of the device. In another example, the specific light manifestation can be different, with a different number and color of lights, the essential thing is that the activation indicator 14 represents an externally observable dynamic manifestation of some function of the device.

In the standby state, the presence motion sensor 11 is active. When a person approaches the device, the presence sensor 11 activates the launch of a multimedia presentation. During the initial part of the presentation, the approximate age and gender of the person in front of the display unit 1 is detected using the stable camera 6 and the module 7 for recognizing the person's position. According to this information, the next content of the presentation is selected. For example, for children, information is presented in an appropriately simpler way.

Information about the person's position, limb and face positions is continuously sent from the person position recognition module 7 to the minicomputer 3. A directionally adjustable camera 5 is directed at the position of the face, and the person's attitude and emotions are analyzed from its image. By evaluating image data from cameras 5, 6, data on the person's reaction to the presented information is obtained. These responses are stored in memory 4 along with time synchronization. At the same time, these reactions are used to continuously influence the way of presentation. According to the programmed criteria, the color and intensity of the RGB LED lighting 13 changes, the aroma is released using the aromatizer 10, and the accompanying sound and its intensity change. These changes also take into account the temperature data from the temperature sensor 16 and the barometric pressure data, which roughly determines the nature of the current weather. The weather affects a person's mood and, depending on the setting, can also change the content of the information presented.

In this example, a communication element 19 is also connected to the minicomputer 3, which enables wireless transmission in both directions, in one direction for uploading multimedia files and for uploading presentation criteria, and in the other direction for the transmission of obtained raw or processed feedback data. The communication element 19 can take the form of a network card by which the device is connected to the Internet. The Internet connection can be used not only for two-way communication with the device operator, but also, for example, for finding local weather data from relevant websites. All measured data and recorded video can be transmitted wirelessly using the communication element 19. With the help of an Internet connection, it can be changed remotely, or complement the content of the presentation, and the software that controls the device and the interrelationships of individual elements and functions can also be updated.

The device also has free ports for connecting additional peripherals, such as an external humidity and temperature sensor and a lightning sensor.

In this example, the directionally positionable camera 5 also has a remote control 18, which can be controlled from around the device. The directionally adjustable camera 5 can also be filmed manually.

Industrial applicability

The industrial applicability is obvious. According to this invention, it is possible to industrially and repeatedly assemble and use an information device that simultaneously receives feedback and changes the surrounding perceptions and the method of multimedia presentation of information.

List of relationship tags

- display unit

- loudspeaker

- computer

- memory

- directionally adjustable camera

- stable camera

- the person's position recognition module

- facial expression evaluation unit

- panel

- aromatizer

- presence sensor

- alphanumeric display element

- RGB LED lighting

- wake-up indicator

- barometric pressure sensor

- Temperature Sensor

- microcontroller

- remote control of the camera

- communication element

PC - personal computer

Claims (24)

PATENT CLAIMS 1. An information device with simultaneous acquisition of feedback, where the device includes a display unit (1) for displaying visually presented information, at least one speaker (2) and a computer (3) with a memory (4) for the presented information, where the display unit (1) is connected to a computer (3), further includes two cameras (5, 6), one camera (5) is directionally positionable, the other camera (6) is stable, both cameras (5, 6) are oriented to the space in front of the display unit ( 1), the stable camera (6) is connected to the module (7) for recognizing the position of the person and for determining the position of the face, the output from this module (7) is connected to the direction control of the positionable camera (5), characterized in that the output from positionable camera (5) is connected to a computer (3), in which there is a unit (8) for evaluating facial expressions, it has an outer casing in the form of a separate panel (9), while both cameras (5, 6) are located on the front surface of the device, where is the display unit (1). 2. Information device with simultaneous acquisition of feedback according to claim 1, characterized in that both cameras (5, 6) are located above the display unit (1). 3. Information device with simultaneous acquisition of feedback according to claims 1 or 2, characterized in that the panel is intended for independent standing on the floor or for hanging on the wall. 4. Information device with simultaneous acquisition of feedback according to any one of claims 1 to 3, characterized in that it has an aromatizer (10). 5. Information device with simultaneous acquisition of feedback according to any one of claims 1 to 4, characterized in that it has a presence sensor (11) connected to a computer (3), preferably the presence sensor (11) is a motion sensor or a sound sensor, or a thermal imaging sensor or pressure sensor on the pad in front of the device. 6. Information device with simultaneous acquisition of feedback according to any one of claims 1 to 5, characterized in that it has an alphanumeric display element (12). 7. Information device with simultaneous acquisition of feedback according to any one of claims 1 to 6, characterized in that it has RGB LED lighting (13) with adjustable color, preferably the RGB LED lighting (13) is located on the sides of the panel (9). 8. An information device with simultaneous feedback acquisition according to any one of claims 1 to 7, characterized in that it has a wake-up indicator (14). 9. Information device with simultaneous acquisition of feedback according to claim 8, characterized in that the activation indicator (14) is intended for a light accompaniment with sound, preferably according to the rhythm or tone change of the sound. 10. Information device with simultaneous acquisition of feedback according to claims 8 or 9, characterized in that the activation indicator (14) is intended to display the degree of ability to recognize facial expressions. 11. Information device with simultaneous acquisition of feedback according to any one of claims 1 to 10, characterized in that it has a sensor (15) of barometric pressure. 12. Information device with simultaneous acquisition of feedback according to any one of claims 1 to 11, characterized in that it has a temperature sensor (16) and/or a humidity sensor. 13. Information device with simultaneous acquisition of feedback according to any one of claims 1 to 12, characterized in that it has at least one microcontroller (17) for connecting the peripheral elements to the computer (3). 14. Information device with simultaneous acquisition of feedback according to any one of claims 1 to 13, characterized in that it has a communication element (19) connected to a computer (3). 15. Information device with simultaneous acquisition of feedback according to claim 14, characterized in that the communication element (19) is connected to the network for remote control of the presented content and/or for remote collection of data obtained by the device. 16. Information device with simultaneous acquisition of feedback according to any one of claims 1 to 15, characterized in that it has a remote control (18) of a directionally positionable camera (5). 17. A method of presenting information using an information device according to any one of claims 1 to 16, characterized in that a person in front of the display unit (1) is captured by at least one camera (5, 6) located on the front surface of the device, and the reaction is analyzed from the obtained image person and the information presentation settings change according to the reaction. 18. A method of presenting information using a display unit (1) and a speaker (2) according to claim 17, characterized in that the body position of the limbs and/or facial expression, and/or the change in heart rate are detected by the camera (5, 6). 19. A method of presenting information using a display unit (1) and a speaker (2) according to claims 17 or 18, characterized in that the gender and approximate age of the person in front of the display unit (1) is determined from the image data, and the method is selected according to this data and/or presentation content. 20. A method of presenting information using a display unit (1) and a speaker (2) according to any one of claims 17 to 19, characterized in that the change of the presentation method includes the selection of the accompanying sound and/or the change of the volume, and/or the creation of a light atmosphere in the surroundings devices, and/or the release of odors. 21. A method of presenting information using a display unit (1) and a speaker (2) according to any one of claims 17 to 20, characterized in that the current weather is detected by measuring the barometric pressure and the information presentation settings are changed accordingly. 22. A method of presenting information using a display unit (1) and a speaker (2) according to any one of claims 17 to 21, characterized in that the weather in the location of the device is detected using the Internet connection, and according to the weather data, change information presentation settings. 23. A method of presenting information using a display unit (1) and a speaker (2) according to any one of claims 17 to 22, characterized in that, according to instructions from a remote center, the presented content and/or presentation conditions are changed using an Internet connection. 24. A method of presenting information using a display unit (1) and a speaker (2) according to any one of claims 17 to 23, characterized in that the data collected by the device is transmitted to a remote center using an Internet connection, preferably to one remote centers transmit data from several devices.