WO2011115572A1 - Appareil pour permettre une commande d'un contenu sur un dispositif d'affichage à l'aide d'au moins un geste, procédés résultants autorisés par l'appareil et applications de l'appareil - Google Patents

Appareil pour permettre une commande d'un contenu sur un dispositif d'affichage à l'aide d'au moins un geste, procédés résultants autorisés par l'appareil et applications de l'appareil Download PDF

Info

Publication number
WO2011115572A1
WO2011115572A1 PCT/SG2010/000108 SG2010000108W WO2011115572A1 WO 2011115572 A1 WO2011115572 A1 WO 2011115572A1 SG 2010000108 W SG2010000108 W SG 2010000108W WO 2011115572 A1 WO2011115572 A1 WO 2011115572A1
Authority
WO
WIPO (PCT)
Prior art keywords
display device
gestures
data
display
processing unit
Prior art date
Application number
PCT/SG2010/000108
Other languages
English (en)
Inventor
Ho Chee Yue
Original Assignee
Xyz Wave Pte Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xyz Wave Pte Ltd filed Critical Xyz Wave Pte Ltd
Priority to PCT/SG2010/000108 priority Critical patent/WO2011115572A1/fr
Priority to SG2012070074A priority patent/SG184183A1/en
Publication of WO2011115572A1 publication Critical patent/WO2011115572A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures

Definitions

  • the present invention relates to an add-on apparatus which enables gesture recognition and applications of the add-on apparatus.
  • the present invention relates to methods of gesture database consolidation which is enabled by the add-on apparatus.
  • the forms of the user- computer interaction interface has correspondingly evolved in a continual manner in order to meet expectations of users at large.
  • the user-computer interaction interface has typically centred around usage of an alphanumeric keyboard, with additional peripheral devices such as, for example, mice, joysticks, trackballs, scroll pads, and so forth being used when applications on the computer enabled usage of any of the aforementioned peripheral devices.
  • non-physical interaction interfaces Developments in the field of the user-computer interaction interface has led to non-physical interaction interfaces being introduced to the market, such as, for example, voice control and gesture control.
  • a significant reason for the development of non-physical interaction interfaces is to address the issue of strains experienced by users at the various joints of their arms resulting from continuous pounding at the physical interaction interface.
  • Another reason for the development of non-physical interaction interfaces is for hygiene reasons to prevent transmission of contagious viruses like SARS and avian flu. .
  • Gesture control interaction interfaces are increasing in popularity although such interfaces are considered to be still in their early stages of infancy in relation to their path of development. Gesture control interaction interfaces are still at a phase where a novelty factor of such interfaces is high, and that invariably also drives the popularity of such interfaces. However, it will only be a matter of time before gesture control interaction interfaces will be commonplace as a user-computer interaction interface.
  • Such an apparatus may also be used as a bridging device for a user adapting from a physical user-computer interaction interface to a non-physical user-computer interaction interface.
  • an apparatus for coupling with a display device for enabling control of content shown on the display device using at least one gesture includes a casing for encasing components of the apparatus; a data processing unit for controlling the apparatus; an image capture apparatus coupled to the data processing unit; a data storage unit coupled to the data processing unit; a data connector coupled to the data processing unit, the data connector being for allowing a functional connection of the display device to the apparatus; and at least one mirror for reflecting an image onto the image capture apparatus.
  • the image capture apparatus may advantageously be fully integrated within the casing such that incidence of damage to the image capture apparatus is minimised.
  • the data storage unit may store gestures captured by the image capture apparatus. It is advantageous that a number of gestures stored in the data storage unit increases over time.
  • the at least one gesture may be selected from, for example, hand gestures, facial gestures, leg movements, body movements and so forth.
  • the data processing unit may associate controls for the content shown on the display device with respective gestures stored on the data storage unit.
  • the associated controls for the content shown on the display device with respective gestures may also be stored on the data storage unit.
  • the data processing unit may include a counter which counts an incidence of the associated controls.
  • the counter data may also be stored on the data storage unit.
  • the apparatus may also include a network connection unit, the network connection unit being coupled to the data processing unit.
  • the network connection unit may allow data stored on the data storage unit to be transmitted to a central data server. It is advantageous that the data is shared with any of the apparatus connected to the central data server.
  • the apparatus may also include a mounting mechanism on the casing, with the mounting mechanism being for attaching the apparatus to the display device in a secure manner.
  • the display device may be either a flat panel display or a CRT display.
  • the display device may be used for displaying digital content.
  • the digital content may be used for advertising purposes.
  • the image capture apparatus being employed may be dependent on the type of images being captured.
  • the aforementioned apparatus may be used with a product vending machine to enable purchase of at least one product using at least one gesture.
  • the aforementioned apparatus may also be used with a jackpot machine to enable a single play using at least one gesture.
  • a method for counting an incidence of actions from a plurality of display devices includes counting an incidence of each action controlling content shown on the display device occurring at each display device; sending the count for each action at each display device to a central data server; and consolidating the count for each action at the central data server.
  • the consolidated count for each action may indicate a popularity of each action.
  • the display device may be either a flat panel display or a CRT display with the display device being used for displaying digital content.
  • the digital content may be used for advertising purposes.
  • a method for consolidating gestures used for controlling content shown on a plurality of display devices which are connected on a network is provided.
  • the method includes consolidating gestures used at each display device; and sending the consolidated gestures at each display device to a central data server. It is advantageous that the consolidated gestures may be usable at each of the plurality of display devices.
  • the display device may be either a flat panel display or a CRT display with the display device being used for displaying digital content.
  • the digital content may be used for advertising purposes.
  • the gestures may be selected from, for example, hand gestures, facial gestures, leg movements, body movements and so forth.
  • Figure 1 shows a schematic view of an apparatus of the present invention.
  • Figure 2 shows a perspective view of the apparatus of the present invention during use.
  • Figure 3 shows a process flow of a first method of the present invention.
  • Figure 4 shows a process flow of a second method of the present invention.
  • Figure 5 shows an overview of a system where a plurality of the apparatus of Figure 1 is employed.
  • Figure 6 shows a look-up table used to store gestures in the apparatus of the present invention.
  • Figure 7 shows a perspective view of the apparatus of the present invention during use in a vending machine application.
  • Figure 8 shows a perspective view of the apparatus of the present invention during use in a jackpot machine application.
  • a first aspect of the present invention relates to an apparatus 20 for coupling with a display device for enabling control of content shown on the display device using at least one gesture.
  • Figure 1 shows a schematic view of the apparatus 20 of the present invention.
  • Figure 2 shows a perspective view of the apparatus 20 during use. Reference will be made to both Figures 1 and 2 when describing the first aspect of the present invention.
  • the display device may be either a flat panel display (50 in Figure 2) or a CRT display.
  • the display device 50 may be used for displaying digital content, such as, for example, content 52 relating to food as shown in Figure 2.
  • a further example of the digital content may be in relation to images used to promote movies. It should be appreciated that the digital content may relate to content used for advertising purposes.
  • the apparatus 20 includes a casing 22 for encasing components of the apparatus 20.
  • the casing 22 may be made from materials such as, for example, metals, plastics, composites, and so forth.
  • the casing 22 may be of a form which prevents tampering and theft of components of the apparatus 20.
  • the casing 22 may also provide all weather protection for components of the apparatus 20.
  • the aforementioned purposes of the casing 22 are essential because the apparatus 20 may be used either in indoor or outdoor deployments without any supervision of the apparatus 20 during durations of deployment.
  • the mounting mechanism 34 may be lockable such that the apparatus 20 is not removable from the display device 50 without unlocking the mounting mechanism 34. It should be appreciated that the apparatus 20 need not be attached to the display device 50 if the apparatus 20 is able to be positioned at a location which is able to effectively capture the gestures initiated by a user 90 in front of the display device 50.
  • Figure 2 shows the apparatus 20 being mounted onto a top edge 54 of the display device 50 so as to capture gestures initiated by the user 90.
  • the data processing unit 24 may be considered to be a brain for the apparatus 20 which controls all functions and processes of the apparatus 20.
  • the data processing unit 24 includes a counter 26 which may be used for making counts of, for example, instances of invoked functions, instances of invoked processes, and so forth.
  • An image capture apparatus 28 may be coupled to the data processing unit 24.
  • the image capture apparatus 28 may be a typical lens-light sensor digital imaging device. It should be appreciated that the image capture apparatus 28 may be varied to allow for variations in relation to the type of images that are captured by the image capture apparatus 28.
  • At least one mirror 30 may be used for reflecting an image onto the image capture apparatus 28.
  • the at least one mirror 30 may be adjustable to ensure that an optimal image is captured by the image capture apparatus 28.
  • a possible arrangement of mirrors may be a "periscope" configuration where an even number of mirrors are used to correct laterally inverted images and where the image capture apparatus 28 is fully integrated within the casing 22 such that incidence of damage to the image capture apparatus 28 is minimised.
  • a practical embodiment of the apparatus 20 only includes a slot 56 for the capture of images, with the at least one mirror 30 and the image capture apparatus 28 being encased within the casing 22 of the apparatus 20.
  • the apparatus 20 also includes a data storage unit 32 coupled to the data processing unit 26.
  • the data storage unit 32 may preferable be a solid state memory module for durability purposes.
  • the data storage unit 32 may store gestures captured by the image capture apparatus 28 in a look-up table 200 form as shown in Figure 6.
  • a first column 202 shows associated controls for specific hand gestures.
  • a second column 204 shows the specific hand gestures in question.
  • a first row 206 shows a hand gesture for "flipping an object upwards”.
  • a second row 208 shows a hand gesture for "flipping an object downwards”.
  • a third row 210 shows a hand gesture for "moving an object to top left” while a fourth row 212 shows a hand gesture for "moving an object to bottom right”.
  • Figure 6 denotes a use of hand gestures, the term “gestures” should not be limited to only hand gestures. Gestures may also include, for example, facial gestures, leg movements, body movements, and so forth.
  • a number of gestures stored in the data storage unit 32 increases over time. This also means that the multiple look-up tables 200 may be stored or a number of entries into the look-up table 200 continually increases.
  • the data processing unit 26 may carry out processes to associate controls for the content shown on the display device with respective gestures (as shown in look-up table 200) stored on the data storage unit 32.
  • the associated controls for the content shown on the display device with respective gestures may also be stored on the data storage unit 32 in a form other than the look-up table 200 form.
  • the data storage unit 32 may also store data from the counter 26 of the data processing unit 24. Data from the counter 26 may include counts for an incidence of invoking the associated controls.
  • the data connector 36 may also be a data connector 36 coupled to the data processing unit 24 of the apparatus 20, where the data connector 36 is for allowing a functional connection of the display device 50 to the apparatus 20.
  • the data connector 36 may be a conduit for instructions to be sent from the apparatus 20 to the display device 50 such that the content 52 on the display device 50 appears to be controlled by gestures of the user 90 (as shown in Figure 2).
  • a cable 58 of the data connector 36 may be seen in Figure 2.
  • a network connection unit 38 which is coupled to the data processing unit 24 may also be included in the apparatus 20.
  • the network connection unit 38 may enable connection of the apparatus 20 via either a direct wired connection or a wireless connection.
  • the network connection unit 38 may allow data stored on the data storage unit 32 to be transmitted to a central data server.
  • the central data server 252 is shown in Figure 5, and may comprise more than a single data server. A more detailed description for Figure 5 will be provided in subsequent portions of the description. It should be appreciated that the data on the central data server 252 may be shared with any of the apparatus 20 connected to the central data server 252 either directly or via a network. Transmission of data from the apparatus 20 to the central data server 252 may bring forth advantages which will be described in subsequent portions of the description.
  • Figure 2 shows the apparatus 20 being used in a display device 50 used for advertising purposes
  • the apparatus 20 may be used for other purposes such as, for example, control of televisions, control input into computers for work/pleasure, and so forth.
  • An advantage brought forth by the use of the apparatus 20 may at least include at least minimising strains experienced by users at the various joints of their arms resulting from not having to continuously pound at a physical interaction interface.
  • An additional advantage relates to how the apparatus 20 enables non-contact interaction with the display device 50 which is more hygienic for the users. Further examples of the use of the apparatus 20 is shown in Figures 7 and 8.
  • Figure 7 shows the apparatus 20 used in a vending machine application
  • Figure 8 shows the apparatus 20 used in a jackpot machine application.
  • the vending machine 60 includes a display panel 62.
  • the display panel 62 may be used to show graphics which interact with the user 90 while the user 90 is deciding which products to purchase and may also be used to show products being sold by the vending machine 60.
  • the user 90 may use gestures such as, for example, hand gestures, facial gestures, leg movements, body movements, and so forth to both interact with the vending machine 60 and also to select products for purchase at the vending machine 60.
  • the interaction between the user 90 and the vending machine 60 may depend on the graphics shown on the display panel 62.
  • the graphics may be in a form of a concierge character guiding the user 90, a game which ends with a purchase of a product, a basic product display interface, and so forth.
  • the vending machine 60 would also have a dispenser compartment 64 for the products purchased at the vending machine 60.
  • the jackpot machine 180 also includes a display screen 182.
  • the display screen 182 may be used to show graphics for every play the user 90 executes.
  • the user 90 may use gestures such as, for example, hand gestures, facial gestures, leg movements, body movements, and so forth to both interact with the jackpot machine 80 and also to manage playing credits for use at the jackpot machine 180.
  • the graphics shown on the display screen 182 may depend on the gestures used by the user. For example, virtual cylinders shown on the display screen 182 may rotate at different rates depending on the gestures used by the user 90. In this manner, the user 90 may believe that an outcome for each play is dependent on the user's 90 gestures and not merely determined by probabilities dictated by a computer program.
  • the jackpot machine 180 would also have a dispenser compartment 184 for winnings at the jackpot machine 180.
  • FIG. 5 shows an overview of a system 250 where a plurality of the apparatus 20 is employed.
  • each of the plurality of the apparatus 20 is connected to the internet 254 using the network connection unit 38 of each of the apparatus 20.
  • the central data server 252 which is also connected to the internet 254. Understanding components of the system 250 will aid in understanding further aspects of the present invention in subsequent portions of the description.
  • Figure 5 shows the apparatus 20 being used in a display device 50 used for advertising purposes, the apparatus 20 may be used for other purposes such as, for example, control of televisions, control input into computers for work/pleasure, and so forth.
  • Figure 3 shows a process flow for a method 70 for counting an incidence of actions from a plurality of display devices 50.
  • the display devices 50 may be either a flat panel display or a CRT display as described earlier.
  • the method 70 may be used for counting an incidence of controls invoked by gestures at each of the display devices 50.
  • gestures should not be limited to only hand gestures. Gestures may also include, for example, facial gestures, leg movements, body movements, and so forth.
  • the method 70 includes counting an incidence of each action (72) controlling the content 52 shown on the display device 50 occurring at each display device 50. The counting of the incidence may be carried out using the counter 26 included in the data processing unit 24 of the apparatus 20.
  • the method 70 then includes sending the count (74) for each action at each display device 50 to the central data server 252 using the network connection unit 38 of each of the apparatus 20. Finally, the count for each action for each display device 50 is consolidated (76) at the central data server 252, where it can be ascertained that the consolidated count for each action indicates a popularity of each action.
  • Figure 4 shows a process flow for a method 80 for consolidating gestures used for controlling content shown on a plurality of display devices 50 which are connected on a network.
  • the display device 50 may be either a flat panel display or a CRT display as described earlier.
  • the method 80 may be used for dynamically building up a database of a variety of gestures used by different users at each of the display devices 50 for invoking a variety of actions for controlling the content 52 shown on the display device 50.
  • gestures should not be limited to only hand gestures. Gestures may also include, for example, facial gestures, leg movements, body movements, and so forth.
  • the method 80 includes consolidating gestures (possibly a number of the look-up tables 200 of Figure 6) used at each display device 50 (82) using the data processing unit 24 of each of the apparatus 20. Subsequently, the method 80 includes sending the consolidated gestures (84) at each display device 50 to the central data server 252 using the network connection unit 38 of each of the apparatus 20.
  • the consolidated gestures from each of the display devices 50 are stored on the central data server 252
  • the consolidated gestures at the central data server 252 may be used as a reference database for all the display devices 50, such that all gestures which have been used at the various display devices 50 may be recognized and usable at each of the plurality of display devices 50 (86).

Abstract

L'invention porte sur un appareil pour permettre une commande d'un contenu sur un dispositif d'affichage à l'aide d'au moins un geste. L'utilisation de l'appareil autorise également à la fois un procédé pour le comptage d'une incidence d'actions et un procédé de consolidation de gestes utilisées pour la commande du contenu sur le dispositif d'affichage. L'invention porte également sur certaines applications de l'appareil.
PCT/SG2010/000108 2010-03-19 2010-03-19 Appareil pour permettre une commande d'un contenu sur un dispositif d'affichage à l'aide d'au moins un geste, procédés résultants autorisés par l'appareil et applications de l'appareil WO2011115572A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/SG2010/000108 WO2011115572A1 (fr) 2010-03-19 2010-03-19 Appareil pour permettre une commande d'un contenu sur un dispositif d'affichage à l'aide d'au moins un geste, procédés résultants autorisés par l'appareil et applications de l'appareil
SG2012070074A SG184183A1 (en) 2010-03-19 2010-03-19 An apparatus for enabling control of content on a display device using at least one gesture, consequent methods enabled by the apparatus and applications of the apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/SG2010/000108 WO2011115572A1 (fr) 2010-03-19 2010-03-19 Appareil pour permettre une commande d'un contenu sur un dispositif d'affichage à l'aide d'au moins un geste, procédés résultants autorisés par l'appareil et applications de l'appareil

Publications (1)

Publication Number Publication Date
WO2011115572A1 true WO2011115572A1 (fr) 2011-09-22

Family

ID=44649470

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/SG2010/000108 WO2011115572A1 (fr) 2010-03-19 2010-03-19 Appareil pour permettre une commande d'un contenu sur un dispositif d'affichage à l'aide d'au moins un geste, procédés résultants autorisés par l'appareil et applications de l'appareil

Country Status (2)

Country Link
SG (1) SG184183A1 (fr)
WO (1) WO2011115572A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106328050A (zh) * 2016-10-27 2017-01-11 欧量(上海)照明有限公司 Led智能展示显示系统
CN115862144A (zh) * 2022-12-23 2023-03-28 杭州晨安科技股份有限公司 一种摄像机手势识别方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020071277A1 (en) * 2000-08-12 2002-06-13 Starner Thad E. System and method for capturing an image
US20040215689A1 (en) * 2003-01-09 2004-10-28 Dooley Michael J. Computer and vision-based augmented interaction in the use of printed media
US20070013718A1 (en) * 2000-10-06 2007-01-18 Sony Computer Entertainment Inc. Image processor, image processing method, recording medium, computer program and semiconductor device
US20070259717A1 (en) * 2004-06-18 2007-11-08 Igt Gesture controlled casino gaming system
WO2009059065A1 (fr) * 2007-10-30 2009-05-07 Hewlett-Packard Development Company, L.P. Système d'affichage interactif avec détection de gestuelle de groupe
US20100045705A1 (en) * 2006-03-30 2010-02-25 Roel Vertegaal Interaction techniques for flexible displays

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020071277A1 (en) * 2000-08-12 2002-06-13 Starner Thad E. System and method for capturing an image
US20070013718A1 (en) * 2000-10-06 2007-01-18 Sony Computer Entertainment Inc. Image processor, image processing method, recording medium, computer program and semiconductor device
US20040215689A1 (en) * 2003-01-09 2004-10-28 Dooley Michael J. Computer and vision-based augmented interaction in the use of printed media
US20070259717A1 (en) * 2004-06-18 2007-11-08 Igt Gesture controlled casino gaming system
US20100045705A1 (en) * 2006-03-30 2010-02-25 Roel Vertegaal Interaction techniques for flexible displays
WO2009059065A1 (fr) * 2007-10-30 2009-05-07 Hewlett-Packard Development Company, L.P. Système d'affichage interactif avec détection de gestuelle de groupe

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106328050A (zh) * 2016-10-27 2017-01-11 欧量(上海)照明有限公司 Led智能展示显示系统
CN115862144A (zh) * 2022-12-23 2023-03-28 杭州晨安科技股份有限公司 一种摄像机手势识别方法
CN115862144B (zh) * 2022-12-23 2023-06-23 杭州晨安科技股份有限公司 一种摄像机手势识别方法

Also Published As

Publication number Publication date
SG184183A1 (en) 2012-10-30

Similar Documents

Publication Publication Date Title
US11099637B2 (en) Dynamic adjustment of user interface
US20200099960A1 (en) Video Stream Based Live Stream Interaction Method And Corresponding Device
JP6002424B2 (ja) ユーザーの視線、及びジェスチャによるフィードバック提供システム、及び方法
US10341642B2 (en) Display device, control method, and control program for stereoscopically displaying objects
EP3077867B1 (fr) Affichage optique monté sur la tête, module de portail de télévision et procédés permettant de commander une interface utilisateur graphique
WO2016103115A1 (fr) Appariement sélectif d'une application présentée dans un espace virtuel avec un dispositif d'affichage physique
CN102893293A (zh) 位置捕获输入设备、系统及其方法
CN108027707B (zh) 用户终端设备、电子设备以及控制用户终端设备和电子设备的方法
WO2019216249A1 (fr) Procédé de fourniture d'un espace virtuel ayant un contenu prescrit
WO2015123084A1 (fr) Affichage transparent virtuel
US20130080976A1 (en) Motion controlled list scrolling
US10216357B2 (en) Apparatus and method for controlling the apparatus
Chen et al. A case study of security and privacy threats from augmented reality (ar)
CN108012195A (zh) 一种直播方法、装置及其电子设备
WO2011115572A1 (fr) Appareil pour permettre une commande d'un contenu sur un dispositif d'affichage à l'aide d'au moins un geste, procédés résultants autorisés par l'appareil et applications de l'appareil
CN104704449A (zh) 用户界面设备和用户界面方法
WO2016102948A1 (fr) Interaction cohérente sans contact avec des images 3d stéréoscopiques
EP3382505B1 (fr) Procédé et système améliorés d'interaction rv
TW201913298A (zh) 可顯示實體輸入裝置即時影像之虛擬實境系統及其控制方法
CN103686389B (zh) 一种触屏遥控器、视频播放系统及触屏遥控方法
CN115543084A (zh) 具有分布式渲染功能的虚拟现实系统及分布式渲染方法
CN112987928A (zh) 虚拟现实交互系统与方法、计算机设备、可读存储介质及虚拟现实交互装置
CN117716392A (zh) 信息处理装置、信息处理方法和程序
CN106125940A (zh) 虚拟现实交互界面管理方法和装置
CN108419037A (zh) 一种无线视频传输器

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 10848082

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 050213)

122 Ep: pct application non-entry in european phase

Ref document number: 10848082

Country of ref document: EP

Kind code of ref document: A1