WO2015119507A1

WO2015119507A1 - Device, system and method for improved visually impaired spectator experience

Info

Publication number: WO2015119507A1
Application number: PCT/NO2015/050023
Authority: WO
Inventors: Svenn MARTINSEN
Original assignee: Parcels In Sport As
Priority date: 2014-02-04
Filing date: 2015-02-03
Publication date: 2015-08-13
Also published as: EP3102997A4; NO339354B1; NO20140132A1; EP3102997A1

Abstract

Device, system and method for improved visually impaired spectator experiencecomprising recognizing the position of an event of a game on a game field on a tactile map where the tactile map comprising parcels or group of parcels, Fig. 5b

Description

Device, system and method for improved visually impaired spectator experience

Description:

The subject matter disclosed herein is related to a tactile map, and specifically to a tactile map and a system adapted for improved visually impaired spectator experience of a sports game, such as a game of football or other ball and/or team games and/or racing events and/or other games.

State of the art:

Being a spectator, either at a game field or arena, or in front of a TV set is a happening comprising a plurality of information sources. This can be the combination of the mood of the supporters, the weather, the light, the visual information of the game itself, and the information from the commentators. In the case where the spectator is visually impaired there exists a string of challenges to facilitate for an authentic transmission of the game in a manner that give the visually impaired spectator to get a good understanding of the actual events of the game. The training of specialized commentators specializing to commentate in a manner that is better understandable by a visually impaired person has been developed. The challenge is that in games such as football and other team oriented games, there are too many events in the game and too short time for commenting each event and to be able to describe all the events in a fully understandable manner. It is a well known problem that the commentator of a football game may not have enough time to describe where for example the ball is at all times and where an event happens on the game field. Within seconds of a first event a second event develops requiring attention from the commentator. This follows by the fact that visually impaired persons rarely makes out the complete layout of an event of the game.

Another problem area for visually impaired persons is that they do not have a common platform for communicating with non impaired persons. When visiting a game field or gathering in front of a TV the visually impaired person cannot take part in the activity on the same level as the non impaired person.

Yet another problem arises since a visually impaired person very often put extra strain on the companions joining the visually impaired person to a game. The companions will very often have to dedicate a lot of their focus on explaining details from the game. This can be seen as a burden on both parties and an obstacle for the possibility for good spectator experiences for the visually impaired person. It is an object of the present invention to provide an improved system and method for improvement of the experience a visually impaired person has of a game when either being present at a game field as a spectator or watching/listening to a real time transmission of a game on a transmission device such as a TV set, radio, or any other type of device able to output audio and/or video signals.

It is an object of the invention to provide means for aiding the visually impaired person to identify the location of an event of the game commented in the real time transmission of the game without being able to see the actual game. The invention comprises means for the commentator to quickly identify the location of an event of the game and refer to this by a predefined game field parcel or lot, and for the spectator to quickly identify the parcel in question by locating this on his own personal location map.

It is further an object of the invention to provide means, system and method for the commentator such that the commentator can provide location information of the event of the game with only spending a very short time or no time at all describing the location.

It is also an object of the invention to improve the visually impaired person's ability to observe, to experience and to communicate with non impaired persons the events of a game.

When the phrase "visually impaired" is used in this document it should be understood that this means, but are not limited by, all kind of visually impaired persons, also blind persons.

When the phrase "game" is used in this document it shall be understood that this might be any type of game sports event where the game is played on a field or track.

When the phrase "haptic sensor pad" is used in this document it should be understood that this shall also comprise any form of haptic sensor device that are able to communicate a relative position of the indication intended by the operator of the pad.

Figures:

Fig. 1A and IB - Prior art audio description

Fig. 2A and 2B - Field map example of football field

Fig. 3A and 3B - Commentator work environment

Fig. 4 - Tactile map

Fig. 5A and 5B - System for improved visually impaired spectator experience Fig. 6 - Commentator work environment for dynamic event tracking

Fig. 7 - Flow chart describing the elements of commentating for the visually impaired spectator. Fig. 8 - Flow chart describing the elements of combined commentating and dynamic representation of the event of the game.

Detailed description of the invention:

Being a spectator, either at a game field or arena, or in front of a TV set is a happening comprising a plurality of impressions from a variety of information sources. This can be the combination of the mood of the supporters, the weather, the light, the visual information of the event of the game itself, and the information from the commentators. In the case where the spectator is visually impaired there exists a string of challenges to facilitate for an authentic transmission of the game in a manner that give the visually impaired spectator to get a good understanding of the actual events of the game. The challenge is that in games such as football and other team oriented games, there are too many events in the game and too short time for commenting each event to be able to give a good picture of the actual game. It is a movement within the big arenas and some of the TV stations to offer special commentator services for visually impaired persons. This comprises the training of specialized commentators specializing to commentate in a manner that is better understandable by a visually impaired person. Fig. la shows a typical scenario of available technology where a visually impaired viewer listen to an audio file in addition to following the game on the TV at home. . Fig. lb shows a person using the arena specific services for visually impaired persons, where a headset is provided that receives signals from a dedicated commentator commenting the game in a manner that gives the visually impaired person a marginally better understanding of the actual game.

The present invention comprises a system for a commentator to use the parcel definition of the game field as described in NO20130432 to use a predefined identification, such as a number, of a parcel in which the event of the game is happening. By using a defined parcel grid of the game field where each parcel or group of parcels have a unique identification, reporting the origin of an event of the game can quickly and easily be identified. The parcel identification can even be automatically identified when processing the video transmission of the game. For example can the ball in a football match be recognized by the video system and continuously feed the actual position of the ball to the display system used by the commentator. The commentator can in turn only refer to the information provided by the display system when explaining to the listener where the ball is at any time. Using short description of the game field position of an event of the game is a vital element of the improved experience for the spectator since the spectator will have a tactile device or map of the present invention at hand that defines the layout of the game field in a readable manner.

The tactile map of the game field will be divided in the number of parcels that corresponds to the parcel grid used by the commentator. Each parcel is numbered in a readable manner for the visually impaired person, that is either with letters, Braille print, texture, color or other.

The visually impaired person will when using the tactile map quickly understand the location of where the reported event of the game is happening. It is important that the commentator at any given time can easily get access to location information of the reported event of the game. This can be achieved by for example using an automatic augmented grid field over the video signals from the game, or it can be tracking system of the ball or other known shapes such as a specific player, referee, or other. Such tracking system can be designed to identify the parcel by updating a text frame superimposed on the video stream of the actual game. The text frame removes the need to continuously superimpose the parcel pattern on the video stream in order for the commentator to be able to report correct parcel of interest. The parcel pattern could then be displayed on demand only.

When reporting the actual position event of the game the position of the event can quickly and easily be identified and reported in short format. For the spectator using his/her map frequently will learn the layout by memory, and will only need a tactile map if a different filed type is used, or it is a long time since the memorization of the game field was done.

The tactile map in the above described embodiment of the invention is a static tactile map. It is further an aspect of the present invention to provide for an optional feature of the tactile map, a dynamic feature giving a better sensation of the dynamics of the game. The dynamic feature may comprise a separate input device used by the commentator or someone working together with the commentator to identify where an event of the game is happening on the field. If automatic tracking of an element, such as a ball, is used, this may provide position input for the dynamic system as well. The position input device/mechanism provides for a separate signal that is transmitted to the spectator/listeners tactile dynamic map. Continuous update of the tactile dynamic map may give haptic output signals such as dynamic relief output, dynamic positioned vibrations, various sound signals, or other. Using the automatic pattern recognition of the ball on the input side, may result in a single point relief output on the dynamic tactile map that moves over the dynamic tactile map correspondingly to the ball flight across the game field of the real game. The automatic pattern recognition of the ball might be replaced by a position sensor system able to pinpoint the position of a game entity, such as a ball or a specific player, based on transmitters and sensors able to locate the position of the transmitter. The selection of what sensor(s) to follow may be controlled by the commentator, or other person. It is also an option, depending on supporting technology and SW, to let the viewer select what object to follow. This can be offered in an interactive environment where the viewer makes a selection of offered objects to follow

continuously or select an object for getting current status of position on the game field.

When tracking an object of the game, either by automatic pattern recognition or by any other input method as described in this document, the manner in which the object position is transmitted and outputted to the viewer may take many forms. One can be to play a predefined unique audio file, unique for each parcel or group of parcels, or it can be identified on a haptic output device, where the haptic output device may be integrated in the tactile map already used by the visually impaired person. When using a separate input device for the dynamic position of an event of the game, the input device may be a haptic sensor pad able to sense the indicated position on the surface by pressure sensors, light sensors or other. The information from such haptic sensor pad is converted to the identified position of the corresponding position of the parcel grid valid for the game field in question. Such haptic sensor pad are state of the art, but must be customized for use in present invention. For optimal result the commentator or a person working together with the commentator must be trained to synchronize the actual audio commentating task and the identification of the position of the event currently commentated. When this is synchronized, the experience of the visually impaired person is best.

It is possible to use a combinational input where for example shape recognition system is used on the video signal of the game to track an item of the game such as a ball position, and a haptic sensor pad is used by the commentator to input specific position of other events of the game. In such a described combination of input sources it is possible to prioritize input modes. One example of prioritization is that if the haptic sensor pad identifies a position input, such as a finger pressed down on the haptic sensor pad, the signal of the haptic sensor pad overrides the position information from the automatic shape tracking system following for example the ball position of a football match. It is also an intention of the present invention to facilitate for optional usage of the invention. One such optional usage is to use the parcel position system when training and guiding visually impaired persons themselves in the exercise of a game. One example is to use a signal to voice generator as output device used by the visually impaired contestant and an automatic shape recognition system for input of for example the ball position. The contestant will then continuously be able to get ball position transmitted into recognized voice signals that can be heard for example via a radio receiver and earplugs. A simpler implementation of another use of the parcel system is in an area where the transfer of information is demanding. This is during the game, when the coach or support members need to communicate location specific information to the players. Very often the amounts of information possible to communicate are limited, and a common understanding of the location information will greatly improve the value of the short messages possible to communicate.

Some aspects of the present invention may comprise:

A device comprising a tactile map for use by a visually impaired person, the map defining the parcels of the corresponding game field being used as reference by a commentator commenting an ongoing game or match, such as a football match. Optionally the map also comprises dynamic haptic tactile output for identification of specific parcel or group of parcels in the map corresponding to input governed by the real time events of the game. The optional dynamic haptic tactile output might be originated from an input device in the form of a haptic sensor pad, such as a touch screen, which can receive position information by that an operator touches the screen in the correct parcel on the haptic sensor pad corresponding to a mapping of the real time event of the game.

A system comprising a commentator audio transmission system for commentating an event of the game relative to a map equivalent to a tactile map held by the visually impaired person, an optional commentator haptic sensor pad corresponding to the game field, the commentator haptic sensor pad connected to a communication link, spectator dynamic haptic tactile output map, transmission system transmitting input from commentator haptic sensor pad to spectator dynamic haptic tactile output map, and optionally an automatic pattern identification applied to a video signal and optionally a tracking system of a game entity. A method comprising: commentating an event of the game by identifying the location of the event relative to a predefined map representing the game field, transmitting an audio commentator transmission to a visually impaired person, the visually impaired person identifying the location of the event on a personal tactile map with corresponding layout of the predefined map used by the commentator, commentating an event of the game by identifying the location of the event relative to a predefined map representing the game field, identifying on an optional haptic sensor pad the location of the event, transmitting the sensor surface input to a personal dynamic haptic tactile map with corresponding layout held by a visually impaired person, transmitting the audio signal of the commentator to a visually impaired person audio output device, the visually impaired person sensing the output of the dynamic haptic tactile map in combination with the audio transmission of the commentator.

The following is a detailed description of some features and beneficial aspects of the invention with reference to the associated figures:

It is an aim for the present invention to utilize the mechanism of partitioning a game field into unique parcels or groups of parcels in a pattern typical for the game field in question. There should not be any constraints to form, pattern and number of parcels for any given game field, but it is preferable to somehow standardize on a fixed number and form in order to make the use of the invention easier to introduce. A typical pattern for a football game field as described in figure 2a and 2B is to divide it into 16 parts on the short side, and 26 parts on the long side of the rectangular shaped game field. The number of parcels in this example is then 416 parcels. The mechanism of partitioning a game field is well defined in the priority document NO20130432, and specifically fig. 1 of this document.

A visually impaired person cannot observe as many visual details of the surroundings as a non impaired person. It is the aim of the present invention to give a visually impaired person a much better experience of a game, such as a game of football. The present invention provide for a method where a commentator 310 can quickly and easily identify a position 301 of an event of the ongoing game (marked in the figure with a ring), and immediately communicate the position to a visually impaired person following the game either as a spectator at the arena or in front of a video- or audio- transmission of the game. The commentators work environment is shown in fig. 3, showing the commentator 310 watching the ongoing game on a monitor 302 where the parcel map 303 is augmented onto the video signal of the game in Fig. 3. The commentator will thereby very quickly identify the positional 301 information of any event of the game. The key feature of the invention is that the commentator uses the same positional details as the visually impaired viewer 410 have access to on a personal tactile map 401 in the form of a physical map as described in fig. 4. The tactile map 401 is a representation of the game field where each parcel or group of parcels 201, 202, 203, 204, 205, 206 are marked in Braille 402 or equivalent system for visually impaired persons. When commentator 310 now commentates on an event happening in the middle of right side of the home team field, the location can precisely be defined by for example saying "Cole is receiving the ball in parcel 34 and advances towards the field center and passes to Solskjaer in 325". The visually impaired person 410 who is following the game as a spectator on the arena as described in Fig. 4A or is following the game on a screen 404 or any other communication device, such as radio,

Smartphone, PC and the like, can now very quickly, with the tactile map 401, find the exact position of the events commentated.

In Fig. 6 it is shown a working environment where a commentator 310 or a commentator assistant is using a haptic sensor pad 502 (not further described), where the commentator may input a position on the haptic sensor pad 502 by pressing a finger or pencil or the like 501 on a position on the haptic sensor pad corresponding to the sports field location as observed on the monitor 302. The position of the input is then received by the personal tactile map 401 of the listener 410. The personal tactile map may then by incorporating dynamic/haptic tactile output features, such as a vibration or electric impulse or movement or other, be able to activate the corresponding position on the tactile map. Using all the features of audio signals from the commentators commenting the game, the tactile map that clearly identifies the parcel layout of the game field and optionally also the dynamic/haptic output together will greatly improve the experience of "watching" the game for a visually impaired person. Fig. 7 and Fig. 8 outline two embodiments using the above described devices and methods.

In Fig.7 two images are merged (e.g. by augmentation), one of the video signal of the game 702 and one of the pattern of parcels customized for the specific current field. The combined image shows the game field with the augmented parcel pattern to the game field to the commentator on the commentator monitor 703. The commentator can now comment 704 the events of the game related to the position as identified by the augmented parcel pattern. The radio signal of the commentator will then be transmitted 705 via normal communication channels such as a television transmission, radio transmission, internet transmission, via download of complete sports arrangement transmissions and other. When a listener/spectator receives the transmission 706, the audio signal will be output through loudspeakers (403), and the listener/spectator "decodes" the commentator audio description of the game by when listening to the transmission fell the tactile map of the current game field 707, and feel for the correct parcel identified by the commentator 708, and thereby having a much better understanding of the game. Another embodiment is described in the flow chart in Fig. 8. In this scenario the video signal of the game 802 is modified by augmentation of the parcel pattern 801 of the game field. The combined signal is showed in the monitor 803. When the commentator comments the game 804, a haptic sensor pad for input of position data 805 is used to indicate the corresponding real life position of an event of special interest in the game that is commentated. The audio signal of the commentator is transmitted 806 to the spectator/viewer together with the signal from the haptic sensor pad, either in a combined signal over a common communication channel, or separate via the same or different communication channels. When the signals are received by the spectator/viewer the signals are decoded 807. The separate signals, audio and haptic position data is sent to audio output equipment 808 for outputting the commentators commenting, and the signal corresponding to the input from the haptic sensor pad is output on a haptic/tactile device 810 used by the spectator/viewer. The spectator will when using these two output sources both hear the commentator's description of the game, and feel the positions of the selected events of the game 809.

It is in the embodiment discussed above possible to replace or add to the haptic sensor pad another technology for input of tracing or pinpointing location of specific events of the game. Such tracing input may arise from using a tracker feature of specific objects of the game, such as a ball, puck, player, referee or other using off the shelf or customized tracker equipment. In an advanced implementation of using automatic or semiautomatic tracing it would be possible to implement interactive aspects where for example the spectator through a customized user interface may control the selection of what to track on the game field.

It is expected that the visually impaired person will quickly learn the overall layout of the tactile map, and thereby not use unnecessary time when searching for the exact parcel of the commented event. It is even expected that visually impaired persons spending much time as spectator or listener to games commented from the same sport and game field will learn by heart the exact layout of the parcels of the game field, and at one point do not need to have the physical version of the tactile map, but use only the imaginary memorized tactile map to understand fully the commentator.

The same concept is used as a learning aid and communication aid between coach and player to easily define more exact positional information when communicating during the game. Specifically is this advantageous if the player is a visually impaired person that depend on exact positional information, both on own position and other players positions. In this environment it is

advantageous if the player and the coach memorize the parcel map. The tactile map can also use colors to mark groups 201, 202, 203, 204, 205, 206 of parcels as seen in fig. 2. Using color code for groups of parcels may be used for corresponding larger fields of interest of the game, such as one color for the inside of the 16 meter markings 201 in front of the goal of a football field. One example of such color coding can be: red for the parcels in the area inside the 16 meter goal line 201, orange for the 2 rows of parcels in front of thel6 meter area 203, yellow on the column of parcels shouldering the red and orange area 205, blue in the vertical goal area 206.

Parcel numbering may follow a number of different schemes, and is of minor importance, as long as the numbering pattern is known, and that the commentator 301uses the same number configuration as the visually impaired person 401 following the commentating of the game.

In practice this may lead to a numbering scheme where the home football team commentator for example always starts parcel numbering at the left corner flag of their own game field half, and increments the number of the parcels left to right, leading to that the second parcel out of the left corner flag in the direction towards the competitors goal is numbered 1 higher that the parcel at the right corner flag of the home team field half. Looking from home goal towards competitor goal in one pattern example will number the parcels 1 - 16 in the first line counting from left to right, then 17 - 32 in the second row counting from left to right, then 33 - 48 and so on until the last row at the base line of the competitor goal line is numbered 401 - 416. While the various embodiments have been described it is to be understood that other similar embodiments may be used or modifications and additions may be made to the described embodiments for performing the same function without deviating from it. Still further, one or more aspects of the above described embodiments may be implemented in or across the various displaying units described, and the present invention should not be limited to any single

embodiment, but rather should be construed in breadth and scope in accordance with the appended claims.

Claims

Claims:

1.

Tactile map c h a r a c t e r i z e d b y that the tactile map (201) comprising a map defined by uniquely marked (402) parcels or group of parcels in a predetermined pattern corresponding to a predefined parcel layout of a game field (200).

2.

Tactile map (201) according to claim 1, where the tactile map further comprising means for dynamic haptic output controlled by a signal representing corresponding events of a game.

3.

Tactile map (201) according to claim 1 or 2, where the unique marking (201, 202, 203, 204, 205, 206) is defined by color coding of groups of the parcels.

4.

Tactile map (201) according to any if the preceding claims, where the unique marking is defined by a unique number for each parcel.

5.

Tactile map (201) according to any if the preceding claims, where the unique marking is marked by using a Braille representation of the marking.

6.

System for recognizing position of game event, the system is characterized by comprising:

tactile map (201) in accordance with any of the proceeding claims,

audio recording equipment, and audio transmission means for transmission of audio input of a commentator (310) description of a game on a field (200),

audio receiver equipment (403) for outputting the commentator description,

a tactile map where the tactile map comprises parcels corresponding to a grid of parcels defined for the game field of the ongoing game.

7.

System for recognizing position of game event according to claim 6, the tactile map (201) further comprises a haptic output for identification of an event in a parcel on the tactile map corresponding to the event of the game in the corresponding parcel as that of the game field and a communication link for transmission of the position information to the tactile map.

8.

System for recognizing position of game event according to claim 7, further comprising an input device for use by the commentator, the input device comprise features for receiving input signal representing the position information of the event of the game.

9.

System for recognizing position of game event according to claim 8, where the feature for receiving input signal representing the position information of an event of the game is a haptic sensor pad.

10.

System for recognizing position of game event according to claim 7, further comprising an input device for receiving input signal representing the position information of an event of the game, and the input device is an automatic tracking sensor tracking the position of an object of the game.

11.

System for recognizing position of game event according to claim 9 and 10, where the input signal is comprised by both a haptic sensor pad, and an automatic tracking sensor, and the input signal is selected from one of the two, and the selection is made by either the commentator or an assistant commentator, or the visually impaired person.

12.

System for recognizing position of game event according to claim 6, further comprising an input device where the feature for receiving input signal representing the position information of an event of the game, and the input device is an automatic tracking sensor system tracking the position of an object of the game, and the output of the automatic sensor is comprised by playing a predefined audio file on the audio receiver equipment (403) which describes the corresponding position of the event of the game.

13.

Method for recognizing the position of an event of a game on a game field on a tactile map where the tactile map comprising parcels or group of parcels, the method comprising:

augmenting a display of a video signal of the game with a predefined pattern of parcels covering the game field,

commentating the event of the game by referring to the position of the event identified by the corresponding parcel or group of parcels of the augmented pattern of parcels,

transmitting the audio signal of the commentator to an audio output device,

a listener to the audio output device recognizing the position of the event of the game by finding the corresponding parcel of the reported event of the game on the tactile map, the tactile map comprising a corresponding pattern as the parcel pattern of the game field.

14.

Method for recognizing the position information of an event of a game on a game field according to claim 13, further comprising automatic tracking of a position of an object of the game, then transmitting a signal representing the position of the object to an output device, the output device when being a audio output device outputting a predefined audio file corresponding to the position of the object, or when output device being a haptic output device outputting a haptic signal corresponding to the position of the object.

15.

Method for recognizing the position information of an event of a game on a game field according to claim 13, the automatic tracking system of the position of an object being able to recognize different types of predefined objects of the game, and selection of what object to track is selected from a menu.