US20140210799A1 - Interactive Display System and Method - Google Patents
Interactive Display System and Method Download PDFInfo
- Publication number
- US20140210799A1 US20140210799A1 US14/227,605 US201414227605A US2014210799A1 US 20140210799 A1 US20140210799 A1 US 20140210799A1 US 201414227605 A US201414227605 A US 201414227605A US 2014210799 A1 US2014210799 A1 US 2014210799A1
- Authority
- US
- United States
- Prior art keywords
- interactive display
- display surface
- computer
- electronic pen
- control operation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 230000002452 interceptive effect Effects 0.000 title claims abstract description 91
- 238000000034 method Methods 0.000 title claims abstract description 14
- 230000000007 visual effect Effects 0.000 claims abstract description 11
- 230000006870 function Effects 0.000 claims description 17
- 238000004891 communication Methods 0.000 claims description 9
- 230000003213 activating effect Effects 0.000 claims description 3
- 239000000853 adhesive Substances 0.000 claims description 3
- 230000001070 adhesive effect Effects 0.000 claims 2
- 230000003667 anti-reflective effect Effects 0.000 claims 1
- 238000005516 engineering process Methods 0.000 description 12
- 239000010410 layer Substances 0.000 description 6
- 230000009471 action Effects 0.000 description 5
- 230000003993 interaction Effects 0.000 description 5
- 239000000758 substrate Substances 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 238000004883 computer application Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000033001 locomotion Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 239000003086 colorant Substances 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000007639 printing Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000013518 transcription Methods 0.000 description 2
- 230000035897 transcription Effects 0.000 description 2
- 230000001131 transforming effect Effects 0.000 description 2
- 241001422033 Thestylus Species 0.000 description 1
- 239000002390 adhesive tape Substances 0.000 description 1
- 210000003484 anatomy Anatomy 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 239000003292 glue Substances 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 239000003973 paint Substances 0.000 description 1
- 238000010422 painting Methods 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 239000011241 protective layer Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000002604 ultrasonography Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/0304—Detection arrangements using opto-electronic means
- G06F3/0317—Detection arrangements using opto-electronic means in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface
- G06F3/0321—Detection arrangements using opto-electronic means in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface by optically sensing the absolute position with respect to a regularly patterned surface forming a passive digitiser, e.g. pen optically detecting position indicative tags printed on a paper sheet
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/0304—Detection arrangements using opto-electronic means
- G06F3/0317—Detection arrangements using opto-electronic means in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0354—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
- G06F3/03545—Pens or stylus
Definitions
- This disclosure relates to the field of micro-processor based interactive display systems and to such systems which utilize position coded patterns and position sensing devices in association with projection based display apparatus to facilitate user interaction with computer based applications and programs.
- Whiteboards have become commonplace visual communication aids which may be found in school classrooms, presentation theaters and work places. Standard whiteboards may comprise a rigid surface which is readily available for writing on with erasable marker pens termed ‘dry-wipe’ markers.
- interactive whiteboards are the natural successors to standard whiteboards and can provide a powerful communication tool which can serve as a gateway for any environment, including the workplace and also for learning ecosystems worldwide known as e-learning in which digital mediums are heavily employed for enhanced learning and communication.
- FIG. 1 illustrates an example interactive whiteboard whereby a projection screen combined with spatial interaction becomes an interactive whiteboard when operating with a projector.
- interactive whiteboard display systems make use of a specially manufactured surface which acts as the means for detecting the position of special interactive whiteboard pens, erasers, stylus, pointers and associated visual presentation aid tools (collectively referred to hereinafter as ‘interactive whiteboard pens’), which may be used in association with an interactive whiteboard for instructing, presenting, teaching, writing, pointing, selecting, activating and drawing.
- interactive whiteboard pens special interactive whiteboard pens, erasers, stylus, pointers and associated visual presentation aid tools
- interactive whiteboard pen position sensing technology employed by known interactive whiteboard systems, they may be referred to generally in the art as ‘touch based,’ and may include a display or screen arrangement having a touch sensitive surface on which contacts are made using an interactive whiteboard pen.
- a projector arrangement may be connected to a host personal computer (‘PC’) and used to project an image taken from a program running on the host PC, thereafter the interaction between the inputs and movements made using the interactive whiteboard pen and the PC may be conducted via connection between the interactive whiteboard itself and the PC.
- PC personal computer
- Such connections may include wired and/or wireless communication arrangements.
- the special construction and surface of the interactive whiteboard may ultimately provide the link between the relative movements of an interactive whiteboard pen and the associated functions which are to be controlled or activated on a host PC.
- Pointer contact with such a touch surface may be detected and used to generate corresponding output depending on areas of the touch surface where the contacts are made. It is this surface based coordinate-detecting technology that characterizes the functionality of interactive whiteboard systems and inherently dictates their physical and functional limitations.
- Such interactive whiteboard surfaces may utilize analog resistive, electromagnetic, capacitive, acoustic or machine vision properties to identify the relative position of contact of the interactive whiteboard pen on the surface of the whiteboard, and thereafter the required positional information may be transmitted to the host PC thereby providing the required association between an image projected from the host PC on to the surface of the interactive whiteboard and the function(s) associated with the interactive whiteboard pen (i.e., color, erasing capacity, selection of particular icon based functions, etc).
- the function(s) associated with the interactive whiteboard pen i.e., color, erasing capacity, selection of particular icon based functions, etc.
- Hand writing may be digitally stored in a computer via special pens that transcribe hand motion, usually called automatic transcription, and the resulting image transferred into the computer for handwriting recognition and archiving applications.
- This functionality may be achieved using enabling technologies comprised in a holding, writing and/or pointing pen arrangement (hereinafter referred to as ‘electronic pens’), in which displacement of the electronic pen tip is detected and measured accordingly.
- enabling technologies utilized in known ‘electronic pens’ for detecting and measuring pen movement include, but are not limited to, mechanical accelerometers, gyroscopes and optical pattern recognition apparatus and ultrasound transducers, all configured to detect displacement of the electronic pen, and typically the tip of said pen.
- Technology employed for the automatic recognition and computer based storage of handwriting may utilize a special pen equipped with a vision based sensor, and paper typically known in the art as ‘digital paper,’ the surface of which is covered with a position coded pattern (hereinafter referred to as a ‘PCP’), that can provide unique respective (x, y) positions on the paper.
- a processor may be utilized to decode the captured image of the pattern on the paper, and thereafter the associated system can determine the resulting coordinates, translate them into a specific location in an electronic document via the transmission of data to a computer via a wired and/or wireless connection.
- the present disclosure describes a novel interactive whiteboard system utilizing an electronic reading pen to interact with a host PC via an image from the host PC projected onto any surface, the surface may include a position coded pattern to provide specific location information and an associated host PC for providing information relating to the position of the electronic reading pen relative to the projected image.
- the present disclosure describes a method of controlling an interactive display system utilizing a processor and an electronic reading pen in conjunction with position coded pattern surface technology.
- the system described herein enables a user to view the image of a host computer screen projected onto any surface comprising a position coded pattern (PCP), and thereafter to interact with the various functions and programs available and on the host computer via the use of an ‘electronic reading pen’ by using the pen in the manner of a touch screen device on the PCP surface onto which said image is projected.
- PCP position coded pattern
- a desktop icon projected from a host PC onto a PCP surface can thereafter be selected by using an ‘electronic reading pen,’ and the particular functions or programs associated with the icon on the computer screen can be selected and ultimately activated.
- an ‘electronic reading pen,’ By utilizing this novel functionality, a multitude of computer applications can be opened and manipulated via the interaction of an electronic reading pen with any surface comprising a PCP.
- the application software in the personal computer may be designed to interpret and perform operations in accordance with inputs received from the electronic pen and accordingly the action of pointing and/or touching the electronic reading pen on the PCP display surface, the host personal computer can be controlled to perform all of its operational functions, including but not limited to opening and display menus, activating programs, dragging icons, writing, executing, operating and interacting with software.
- the system may utilize an electronic reading pen, and a PCP (position coded pattern) projection surface which can be fixedly or releasably attached onto a surface such as a wall, board, whiteboard or other suitable screen arrangement.
- the electronic reading pen may comprise a position decoding device in association with other functional elements and embodiments, a camera arrangement such a CCD device or any other visual recognition arrangement capable of recognizing the PCP employed, and an embedded processor capable of interpreting the PCP beneath the pen head when the projection surface is at a finite distance from the engaging head.
- the PCP surface may be comprised of a suitably durable projection material capable of conveying information about a unique position encoded in the printed pattern, of which the recognition, interpretation, processing and subsequent transmission to a host PC may be carried out by the systems onboard the electronic reading pen.
- the actual tip position of an electronic pen on, or over, a PCP display surface can be established by an electronic recording pen in accordance with the invention.
- the relative (x, y) coordinates of the electronic pen tip may thereby be established with reference to the display image on the host computer, and thereafter the electronic reading pen can be used to interact with projected images provided by the host computer.
- An electronic recording pen may be equipped with a real-time processor to decode the information contained in the PCP surface pattern to establish sets of predetermined (x, y) coordinates embedded within the pattern itself.
- a real-time processor may work simultaneously to decode the (x, y) coordinates which are sent via wireless communication or the like to a host PC which may be attached to a projector.
- a user may be required to point to and/or touch the electronic recording pen on to predetermined points on the PCP display surface. These actions may provide for the calibration of the relative coordinates of the image projected on the PCP display surface with that of the image on the host PC to ensure that the dimensions and scale of projected image are in proportion to the image on the associated host PC screen and are thereby understood by the electronic recording pen to host PC interface program. Any suitable position dependent automatic transcription technology may be utilized by the system.
- the electronic recording pen-to-PCP system operation may indicate the exact position as the electronic recording pen-tip position on the projected view on the surface to be the same as that of the mouse cursor on the host PC display screen.
- an interactive display system that can be produced by printing a position-coded pattern on a suitable substrate.
- An object of the present disclosure is to provide an interactive display system for transforming PCP projection surface locations into interactive areas with an electronic reading pen.
- Another object is to provide an interactive display system that provides an interactive display surface to work in conjunction with a projector.
- Another object is to provide an interactive display system that has a display surface that is easily manufactured with regular wide-format printers or other printing facilities.
- Another object is to provide an interactive display system that utilizes the streaming of real-time wireless location data via an electronic reading pen.
- Another object is to provide an interactive display system that is capable of transforming various sizes, styles and types of surfaces/display screens into interactive facades.
- Another object is the provision of an interactive display system that employs physical shortcuts that can be used to access computer applications via the use of PCP media separate from the PCP display arrangement for which functions on the host PC have been pre-assigned.
- Another object is to provide an interactive display system that is lightweight, portable and compact.
- Another object is to provide an interactive display system that does not interfere with rear projection light due to the possibility to utilize a transparent PCP substrate.
- Another object is to provide an interactive touch screen that utilizes a transparent PCP surface and an invisible pattern like UV or other invisible ink type.
- Another object is to provide a wireless mouse pad comprised of a regular size mouse pad having PCP on its surface and functions with an electronic reading pen.
- Another object is to provide an interactive display system that may be utilized as a sticker roll or as a rigid projection surface and may be provided with a releasable contact medium such as adhesive tape or glue, magnets, fastening surfaces such as VelcroTM and the like.
- a releasable contact medium such as adhesive tape or glue, magnets, fastening surfaces such as VelcroTM and the like.
- FIG. 1 is an explanatory diagram of the concept of interactive whiteboards.
- FIG. 2 is a workflow diagram illustrating the operation of interactive whiteboards.
- FIG. 3A is a side view of an example classroom operation setup in front projection mode.
- FIG. 3B is a front view of an example classroom operation setup using front projection.
- FIG. 4A is a side view of a rear projection example.
- FIG. 4B is a front view of a rear projection example.
- FIG. 5A is a bisection and front perspective view of a projection layer of an example PCP surface using rear projection.
- FIG. 5B is a bisection and front perspective view of a projection layer of an example PCP surface using front projection.
- FIG. 6 is a sticker roll version of an example PCP interactive whiteboard.
- FIG. 7 is an example packaging configuration.
- FIG. 8 is a set view of various example pattern types.
- FIG. 9A is a diagram illustrating the main components of an example electronic reading pen.
- FIG. 9B is an exploded view of the internal anatomy of an example electronic reading pen.
- FIG. 10A is an illustration of a digital pen evaluation kit.
- FIG. 10B is a snapshot of a computer desktop showing an example software driver.
- FIG. 11A illustrates an example set of physical shortcuts.
- FIG. 11B shows an example pen utilized in conjunction with a physical shortcut.
- FIG. 11C is an example painting and drawing palette physical shortcut.
- FIG. 11D illustrates an example portable PCP mouse pad.
- FIG. 11E illustrates an example set of physical shortcuts.
- FIG. 12 is an example PCP printed keyboard physical shortcut.
- FIG. 13A illustrates an example pen where the stylus head has been removed from the pen nozzle.
- FIG. 13B illustrates a stainless stylus pen head 1 and an ink cartridge ball-point pen head 2 .
- FIG. 14A illustrates an example PCP surface.
- FIG. 14B illustrates a user writing on an example PCP surface with an example electronic pen.
- an interactive display system comprising a PCP projection surface and an electronic reading pen capable of reading the PCP from the display screen.
- the projection surface may be comprised of a suitable projection material capable of displaying the projector images when a projector is placed at suitable distance away from the screen.
- the electronic reading pen may comprise a CCD infrared camera that can capture details of the PCP (up to 70 frames/second for example). Thereafter, the electronic reading pen may be capable of passing image shots to a decoding processor inside the pen that decodes the pattern image information and outputs absolute (x, y) position data, that is without a (0, 0) reference relevant to the image the pen camera is observing from the pattern.
- FIGS. 3A and 3B When using a front projection interactive whiteboard display system in accordance with the invention, an example setup is shown in FIGS. 3A and 3B .
- a setup may include a host PC for controlling the display process and operating applications and a front projector for projecting images from the host PC to the PCP encoded display screen with which the user will interact using an electronic reading pen.
- the projector when adopting a rear projection setup as shown in FIGS. 4A and 4B , the projector may be located behind a semi-transparent PCP encoded printed surface.
- a rear projection based display setup it is known in the art of rear projection film projectors and the like project the image via the use of reflecting mirrors. Previously such a system has not been possible when utilizing known interactive display systems due to the requirement for the interaction between the electronic pen and the whiteboard to be controlled via touch based sensing technology which is inherently opaque.
- FIG. 5B An example of the construction of a PCP surface is shown in FIG. 5B .
- This diagram illustrates the composition constructed of three layers, the base layer providing a supporting substrate, the middle layer carrying the PCP surface and the top or front layer providing a protective layer for the assembly.
- FIG. 5A A similar arrangement for a rear PCP surface is shown in FIG. 5A , with the distinction being that the supporting substrate may be a perimeter frame or transparent rigid surface with the PCP surface printed onto a semitransparent surface material.
- the writing surface may be in the form, but not limited to, a contact paper-like substrate which can be provided with a sticker-like, self-adhesive, or magnetic property on its front or back surface for simple adherence to a surface of choice.
- the electronic reading pen tip (e.g., a sensor), can be made pressure sensitive to confirm that the pen tip is actually in touch with the surface and to therefore initiate the reading, storage and processing of respective (x, y) coordinates from the PCP display surface.
- a moving electronic reading pen tip in touch with the display surface will generate a sequence of (x, y) coordinates indicative of the instantaneous pen tip location on the PCP surface.
- the read coordinates are thereafter passed in real-time to the host PC, in effect the pen acting as a typical interactive computer mouse.
- the electronic reading pen can be configured to send control signals such as ‘double clicking,’ ‘right and left clicks,’ and such commands can be initiated either from buttons positioned upon the pen body, or alternatively by double click sensing signals being received by and processed by appropriate sensors integrated into the pen tip.
- control signals such as ‘double clicking,’ ‘right and left clicks,’ and such commands can be initiated either from buttons positioned upon the pen body, or alternatively by double click sensing signals being received by and processed by appropriate sensors integrated into the pen tip.
- Such commands can therefore be used to control computer applications on the host PC in a manner substantially identical to that associated with the use of a standard computer mouse.
- the electronic reading pen may be designed to capture audio inputs and transmit them via wired and/or wireless communication to the host PC wherein the audio inputs may be processed and recognized by the host PC as inputs or commands in relation to a computer program such as a word processing application or the like. Accordingly, a user can utilize the electronic reading pen to interact with applications running on a host PC to input words, letters or commands such that the words, letters or commands may to be displayed on the screen, display, writing surface and/or other surface. Further, the captured audio inputs may be processed by processor(s) within or connected to the electronic reading pen and thereafter transmitted to said host PC.
- the electronic reading pen may be equipped with a microphone and wireless Internet adapter (e.g., 802.11a/b or Wi-Fi), which would enable the pen, with some particular software installed on it, to transfer spoken voice in real time using for example, Voice over Internet Protocol (VoIP) technology, without the need for storing the voice in the pen's memory.
- a microphone and wireless Internet adapter e.g., 802.11a/b or Wi-Fi
- VoIP Voice over Internet Protocol
- Matching calibration of the position coded pattern (PCP) display screen with the actual computer screen scale may be undertaken so that inputs received from the electronic reading pen when the pen's tip is pointing at a specific location on the PCP surface (e.g., double clicking on a save button), may be interpreted as a command that matches an action initiated due to pointing on the region of interest on the computer application screen.
- PCP position coded pattern
- Calibration commands may take place at the intended respective location of the pen tip superimposed on the application being projected on the PCP screen to enable command actions to be executed.
- FIG. 8 illustrates example position coding patterns to be printed on the projection surface and FIGS. 9A and 9B illustrate example electronic pens.
- PCP any suitable PCP may be utilized as illustrated in FIG. 8.1 .
- the PCP pattern may be comprised of small dots placed in a pattern that follows a certain algorithmic theme which, when decoded, reveals unique information about absolute position in (x, y) coordinates as shown in FIG. 8.2 .
- PCP patterns capable of being printed or deposited onto any display surface may be employed, including, but not limited to the Xerox data glyph pattern shown at FIGS. 8.3 and 8 . 4 , and other patterns, such as those produced by Microsoft® shown at FIGS. 8.5 and 8 . 6 .
- Such position coding pattern families may utilize an algorithm that can generate and distribute a pattern in a way that decodes position information over the surface on which the pattern is to be applied or printed.
- FIG. 9A shows an example of a reading pen produced by Anoto®.
- This electronic reading pen has the function of decoding the encoded position information embedded in an Anoto® PCP pattern.
- the pen comprises two functional principles which in a sense function as the eyes and brain of the pen. The two functions are accomplished through the use of an infrared CCD camera to read the dot pattern of the PCP, and a processor to decode the images being read by this camera.
- the decoded (x, y) position information may be sent in real-time (i.e. simultaneously whilst being decoded), to a computer via wireless transmission.
- FIG. 9A shows the streaming ‘data sending’ light flickering on an example Maxell® DP-201 R4.1 electronic reading pen which indicates that the pen is in action while operating on a physical shortcut defined by a user.
- FIG. 9B shows a semi exploded view of an example Anoto® compatible electronic reading pen, namely the DP-201 R 4.1 streaming pen.
- FIG. 10A illustrates another electronic reading pen, such as those produced by Maxell® or Logitech®, which supports additional functional components, including a pressure sensitive pen tip, storing memory chip, a vibrator to provide physical shaking feedback to a user, and a Bluetooth® antenna for wireless data transmission and reception.
- additional functional components including a pressure sensitive pen tip, storing memory chip, a vibrator to provide physical shaking feedback to a user, and a Bluetooth® antenna for wireless data transmission and reception.
- the driver software associated with the Maxell® DP-201 R 4.1 is shown.
- the example driver software contains a reading port for SPP Bluetooth® service to enable connection to the electronic reading pen via Bluetooth® ( FIG. 10B ).
- the driver contains a task bar for performing functions such as calibration, onscreen reading and mode switching ( FIG. 10B ).
- FIGS. 11A-11E Another embodiment provides at least one physical shortcut virtual button shown in FIGS. 11A-11E . These may be provided by utilizing small areas of PCP printed on paper, or any appropriate surface) with a desired thumbnail picture printed thereon.
- the driver software may be configured to recognize specific addresses allocated by programming for these physical shortcuts to linking them to specific functions inside applications or to run whole applications when selected (i.e. by clicking) using the electronic reading pen.
- FIG. 11D shows a mouse pad that can be used to remotely control a PC screen projected on a wide PCP surface or any other surface.
- a color palette may be utilized as a set of physical shortcuts to brush colors in host PC based programs and applications such as Microsoft Paint®.
- FIGS. 14A and 14B an embodiment is shown affixed to a wall with the words ‘position coded pattern’ written with the electronic reading pen on the PCP surface using the program SmartTM Notebook running on the host PC.
Abstract
The present disclosure describes an interactive display system, method and computer readable medium. In one embodiment, an interactive display surface having one or more computer generated patterns printed thereon may be utilized in conjunction with an electronic pen in order to identify the absolute position of the pen relative to the interactive display surface. In one embodiment, the absolute position of the electronic pen may be transmitted to a computer system and utilized to control one or more programs running on the computer system. In one embodiment, the visual output of the computer system may be superimposed upon the interactive display surface using one or more projectors. The computer generated patterns may have a predefined association with a control operation of the computer system such that placing the electronic pen in proximity to the associated generated pattern causes the computer system to execute the associated control operation.
Description
- This application claims priority upon and is a continuation of co-pending U.S. Patent Application No. 2009/0213070, Ser. No. 12/314,707, filed on Dec. 15, 2008; which is a continuation-in-part of PCT Application Number PCT/1B2007/052299, filed on Jun. 15, 2007; which claims priority upon U.S. Provisional Patent Application No. 60/804,969, filed on Jun. 16, 2006. The entire contents of the above applications are incorporated by reference herein.
- This disclosure relates to the field of micro-processor based interactive display systems and to such systems which utilize position coded patterns and position sensing devices in association with projection based display apparatus to facilitate user interaction with computer based applications and programs.
- ‘Whiteboards’ have become commonplace visual communication aids which may be found in school classrooms, presentation theaters and work places. Standard whiteboards may comprise a rigid surface which is readily available for writing on with erasable marker pens termed ‘dry-wipe’ markers.
- In recent years, the introduction of so called ‘interactive whiteboards’ has provided a link between the visual based presentation of information using the standard whiteboard and the functionality of the ubiquitous personal computer. Consequently, interactive whiteboards are the natural successors to standard whiteboards and can provide a powerful communication tool which can serve as a gateway for any environment, including the workplace and also for learning ecosystems worldwide known as e-learning in which digital mediums are heavily employed for enhanced learning and communication.
-
FIG. 1 illustrates an example interactive whiteboard whereby a projection screen combined with spatial interaction becomes an interactive whiteboard when operating with a projector. - Known interactive whiteboard display systems make use of a specially manufactured surface which acts as the means for detecting the position of special interactive whiteboard pens, erasers, stylus, pointers and associated visual presentation aid tools (collectively referred to hereinafter as ‘interactive whiteboard pens’), which may be used in association with an interactive whiteboard for instructing, presenting, teaching, writing, pointing, selecting, activating and drawing.
- Due to the ‘interactive whiteboard pen’ position sensing technology employed by known interactive whiteboard systems, they may be referred to generally in the art as ‘touch based,’ and may include a display or screen arrangement having a touch sensitive surface on which contacts are made using an interactive whiteboard pen.
- A projector arrangement may be connected to a host personal computer (‘PC’) and used to project an image taken from a program running on the host PC, thereafter the interaction between the inputs and movements made using the interactive whiteboard pen and the PC may be conducted via connection between the interactive whiteboard itself and the PC. Such connections may include wired and/or wireless communication arrangements.
- The special construction and surface of the interactive whiteboard may ultimately provide the link between the relative movements of an interactive whiteboard pen and the associated functions which are to be controlled or activated on a host PC.
- Pointer contact with such a touch surface may be detected and used to generate corresponding output depending on areas of the touch surface where the contacts are made. It is this surface based coordinate-detecting technology that characterizes the functionality of interactive whiteboard systems and inherently dictates their physical and functional limitations.
- Such interactive whiteboard surfaces may utilize analog resistive, electromagnetic, capacitive, acoustic or machine vision properties to identify the relative position of contact of the interactive whiteboard pen on the surface of the whiteboard, and thereafter the required positional information may be transmitted to the host PC thereby providing the required association between an image projected from the host PC on to the surface of the interactive whiteboard and the function(s) associated with the interactive whiteboard pen (i.e., color, erasing capacity, selection of particular icon based functions, etc).
- Consequently, due to the available technology employed by known systems, the associated physical dimensions of interactive whiteboards such as that offered under the trademark Smartboard™, can reach circa 30 mm in thickness, and in addition require dedicated interactive whiteboard pens to provide the functionality required to represent different colors when writing ‘electronically’ on the interactive whiteboard surface.
- The physical limitations of known interactive whiteboard systems relate to the specific requirement to use a dedicated interactive whiteboard display surface that is fixed to a solid wall or other surface (i.e., presentation theater, etc.) or alternately mounted on a moveable frame system for transportation between classrooms, offices, etc.
- Consequently, by their very nature, such systems are not readily moveable or transportable due to the weight and physical size associated with the pointer detection technology comprised therein.
- Other limitations of known interactive whiteboard systems relate to the requirement to use front illumination based methods to project images onto the surface of the whiteboard due to the thickness and non-transparent nature of the surface based sensing technology comprising the underlying pointer detection apparatus. This limitation thereby introduces the possibility of shadows being cast across the projected image on the interactive whiteboard surface by those using the system.
- Hand writing may be digitally stored in a computer via special pens that transcribe hand motion, usually called automatic transcription, and the resulting image transferred into the computer for handwriting recognition and archiving applications.
- This functionality may be achieved using enabling technologies comprised in a holding, writing and/or pointing pen arrangement (hereinafter referred to as ‘electronic pens’), in which displacement of the electronic pen tip is detected and measured accordingly. The enabling technologies utilized in known ‘electronic pens’ for detecting and measuring pen movement include, but are not limited to, mechanical accelerometers, gyroscopes and optical pattern recognition apparatus and ultrasound transducers, all configured to detect displacement of the electronic pen, and typically the tip of said pen.
- Technology employed for the automatic recognition and computer based storage of handwriting may utilize a special pen equipped with a vision based sensor, and paper typically known in the art as ‘digital paper,’ the surface of which is covered with a position coded pattern (hereinafter referred to as a ‘PCP’), that can provide unique respective (x, y) positions on the paper. When an electronic pen equipped with such a vision based sensor and a processor passes above such paper, a processor may be utilized to decode the captured image of the pattern on the paper, and thereafter the associated system can determine the resulting coordinates, translate them into a specific location in an electronic document via the transmission of data to a computer via a wired and/or wireless connection.
- The present disclosure describes a novel interactive whiteboard system utilizing an electronic reading pen to interact with a host PC via an image from the host PC projected onto any surface, the surface may include a position coded pattern to provide specific location information and an associated host PC for providing information relating to the position of the electronic reading pen relative to the projected image.
- The present disclosure describes a method of controlling an interactive display system utilizing a processor and an electronic reading pen in conjunction with position coded pattern surface technology.
- In one embodiment, the system described herein enables a user to view the image of a host computer screen projected onto any surface comprising a position coded pattern (PCP), and thereafter to interact with the various functions and programs available and on the host computer via the use of an ‘electronic reading pen’ by using the pen in the manner of a touch screen device on the PCP surface onto which said image is projected.
- Accordingly, in one embodiment, a desktop icon projected from a host PC onto a PCP surface can thereafter be selected by using an ‘electronic reading pen,’ and the particular functions or programs associated with the icon on the computer screen can be selected and ultimately activated. By utilizing this novel functionality, a multitude of computer applications can be opened and manipulated via the interaction of an electronic reading pen with any surface comprising a PCP.
- The application software in the personal computer may be designed to interpret and perform operations in accordance with inputs received from the electronic pen and accordingly the action of pointing and/or touching the electronic reading pen on the PCP display surface, the host personal computer can be controlled to perform all of its operational functions, including but not limited to opening and display menus, activating programs, dragging icons, writing, executing, operating and interacting with software.
- In one embodiment, the system may utilize an electronic reading pen, and a PCP (position coded pattern) projection surface which can be fixedly or releasably attached onto a surface such as a wall, board, whiteboard or other suitable screen arrangement. The electronic reading pen may comprise a position decoding device in association with other functional elements and embodiments, a camera arrangement such a CCD device or any other visual recognition arrangement capable of recognizing the PCP employed, and an embedded processor capable of interpreting the PCP beneath the pen head when the projection surface is at a finite distance from the engaging head.
- The PCP surface may be comprised of a suitably durable projection material capable of conveying information about a unique position encoded in the printed pattern, of which the recognition, interpretation, processing and subsequent transmission to a host PC may be carried out by the systems onboard the electronic reading pen.
- Accordingly the actual tip position of an electronic pen on, or over, a PCP display surface can be established by an electronic recording pen in accordance with the invention. The relative (x, y) coordinates of the electronic pen tip may thereby be established with reference to the display image on the host computer, and thereafter the electronic reading pen can be used to interact with projected images provided by the host computer.
- To acquire the pen tip position from the PCP surface, it is appreciated that when the pen tip touches the surface such event enables a special sensing device such as infrared charge-coupled device camera (CCD) in the pen head to read the PCP surface for subsequent interpretation.
- An electronic recording pen may be equipped with a real-time processor to decode the information contained in the PCP surface pattern to establish sets of predetermined (x, y) coordinates embedded within the pattern itself.
- Further, a real-time processor may work simultaneously to decode the (x, y) coordinates which are sent via wireless communication or the like to a host PC which may be attached to a projector.
- To facilitate the coordination and calibration of the projected image with that of the image displayed in the host PC, a user may be required to point to and/or touch the electronic recording pen on to predetermined points on the PCP display surface. These actions may provide for the calibration of the relative coordinates of the image projected on the PCP display surface with that of the image on the host PC to ensure that the dimensions and scale of projected image are in proportion to the image on the associated host PC screen and are thereby understood by the electronic recording pen to host PC interface program. Any suitable position dependent automatic transcription technology may be utilized by the system.
- After proper calibration the electronic recording pen-to-PCP system operation may indicate the exact position as the electronic recording pen-tip position on the projected view on the surface to be the same as that of the mouse cursor on the host PC display screen.
- According to a further embodiment, there is provided an interactive display system that can be produced by printing a position-coded pattern on a suitable substrate.
- An object of the present disclosure is to provide an interactive display system for transforming PCP projection surface locations into interactive areas with an electronic reading pen.
- Another object is to provide an interactive display system that provides an interactive display surface to work in conjunction with a projector.
- Another object is to provide an interactive display system that has a display surface that is easily manufactured with regular wide-format printers or other printing facilities.
- Another object is to provide an interactive display system that utilizes the streaming of real-time wireless location data via an electronic reading pen.
- Another object is to provide an interactive display system that is capable of transforming various sizes, styles and types of surfaces/display screens into interactive facades.
- Another object is the provision of an interactive display system that employs physical shortcuts that can be used to access computer applications via the use of PCP media separate from the PCP display arrangement for which functions on the host PC have been pre-assigned.
- Another object is to provide an interactive display system that is lightweight, portable and compact.
- Another object is to provide an interactive display system that does not interfere with rear projection light due to the possibility to utilize a transparent PCP substrate.
- Another object is to provide an interactive touch screen that utilizes a transparent PCP surface and an invisible pattern like UV or other invisible ink type.
- Another object is to provide a wireless mouse pad comprised of a regular size mouse pad having PCP on its surface and functions with an electronic reading pen.
- Another object is to provide an interactive display system that may be utilized as a sticker roll or as a rigid projection surface and may be provided with a releasable contact medium such as adhesive tape or glue, magnets, fastening surfaces such as Velcro™ and the like.
- A more complete appreciation of the invention and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings; it being understood that the drawings contained herein are not necessarily drawn to scale; wherein:
-
FIG. 1 is an explanatory diagram of the concept of interactive whiteboards. -
FIG. 2 is a workflow diagram illustrating the operation of interactive whiteboards. -
FIG. 3A is a side view of an example classroom operation setup in front projection mode. -
FIG. 3B is a front view of an example classroom operation setup using front projection. -
FIG. 4A is a side view of a rear projection example. -
FIG. 4B is a front view of a rear projection example. -
FIG. 5A is a bisection and front perspective view of a projection layer of an example PCP surface using rear projection. -
FIG. 5B is a bisection and front perspective view of a projection layer of an example PCP surface using front projection. -
FIG. 6 is a sticker roll version of an example PCP interactive whiteboard. -
FIG. 7 is an example packaging configuration. -
FIG. 8 is a set view of various example pattern types. -
FIG. 9A is a diagram illustrating the main components of an example electronic reading pen. -
FIG. 9B is an exploded view of the internal anatomy of an example electronic reading pen. -
FIG. 10A is an illustration of a digital pen evaluation kit. -
FIG. 10B is a snapshot of a computer desktop showing an example software driver. -
FIG. 11A illustrates an example set of physical shortcuts. -
FIG. 11B shows an example pen utilized in conjunction with a physical shortcut. -
FIG. 11C is an example painting and drawing palette physical shortcut. -
FIG. 11D illustrates an example portable PCP mouse pad. -
FIG. 11E illustrates an example set of physical shortcuts. -
FIG. 12 is an example PCP printed keyboard physical shortcut. -
FIG. 13A illustrates an example pen where the stylus head has been removed from the pen nozzle. -
FIG. 13B illustrates a stainlessstylus pen head 1 and an ink cartridge ball-point pen head 2. -
FIG. 14A illustrates an example PCP surface. -
FIG. 14B illustrates a user writing on an example PCP surface with an example electronic pen. - An interactive display system, an interactive display method and a computer readable medium for providing an interactive display is described herein. In one embodiment, an interactive display system is provided which comprises a PCP projection surface and an electronic reading pen capable of reading the PCP from the display screen.
- The projection surface may be comprised of a suitable projection material capable of displaying the projector images when a projector is placed at suitable distance away from the screen.
- The electronic reading pen may comprise a CCD infrared camera that can capture details of the PCP (up to 70 frames/second for example). Thereafter, the electronic reading pen may be capable of passing image shots to a decoding processor inside the pen that decodes the pattern image information and outputs absolute (x, y) position data, that is without a (0, 0) reference relevant to the image the pen camera is observing from the pattern.
- When using a front projection interactive whiteboard display system in accordance with the invention, an example setup is shown in
FIGS. 3A and 3B . Such a setup may include a host PC for controlling the display process and operating applications and a front projector for projecting images from the host PC to the PCP encoded display screen with which the user will interact using an electronic reading pen. - Similarly, when adopting a rear projection setup as shown in
FIGS. 4A and 4B , the projector may be located behind a semi-transparent PCP encoded printed surface. When utilizing a rear projection based display setup, it is known in the art of rear projection film projectors and the like project the image via the use of reflecting mirrors. Previously such a system has not been possible when utilizing known interactive display systems due to the requirement for the interaction between the electronic pen and the whiteboard to be controlled via touch based sensing technology which is inherently opaque. - The use of a rear projection setup provides a unique benefit due to the fact that the functionality of an interactive whiteboard display can be utilizing with the added benefit for user experience since the user cannot interfere with the projected incident light projecting the image onto the screen which is coming usually from a front position projector.
- An example of the construction of a PCP surface is shown in
FIG. 5B . This diagram illustrates the composition constructed of three layers, the base layer providing a supporting substrate, the middle layer carrying the PCP surface and the top or front layer providing a protective layer for the assembly. - A similar arrangement for a rear PCP surface is shown in
FIG. 5A , with the distinction being that the supporting substrate may be a perimeter frame or transparent rigid surface with the PCP surface printed onto a semitransparent surface material. - The writing surface may be in the form, but not limited to, a contact paper-like substrate which can be provided with a sticker-like, self-adhesive, or magnetic property on its front or back surface for simple adherence to a surface of choice.
- The electronic reading pen tip (e.g., a sensor), can be made pressure sensitive to confirm that the pen tip is actually in touch with the surface and to therefore initiate the reading, storage and processing of respective (x, y) coordinates from the PCP display surface.
- By decoding the position of the pen tip in substantially real time, a moving electronic reading pen tip in touch with the display surface will generate a sequence of (x, y) coordinates indicative of the instantaneous pen tip location on the PCP surface. The read coordinates are thereafter passed in real-time to the host PC, in effect the pen acting as a typical interactive computer mouse.
- As such the electronic reading pen can be configured to send control signals such as ‘double clicking,’ ‘right and left clicks,’ and such commands can be initiated either from buttons positioned upon the pen body, or alternatively by double click sensing signals being received by and processed by appropriate sensors integrated into the pen tip.
- Such commands can therefore be used to control computer applications on the host PC in a manner substantially identical to that associated with the use of a standard computer mouse.
- In a further embodiment of the invention, the electronic reading pen may be designed to capture audio inputs and transmit them via wired and/or wireless communication to the host PC wherein the audio inputs may be processed and recognized by the host PC as inputs or commands in relation to a computer program such as a word processing application or the like. Accordingly, a user can utilize the electronic reading pen to interact with applications running on a host PC to input words, letters or commands such that the words, letters or commands may to be displayed on the screen, display, writing surface and/or other surface. Further, the captured audio inputs may be processed by processor(s) within or connected to the electronic reading pen and thereafter transmitted to said host PC.
- The electronic reading pen may be equipped with a microphone and wireless Internet adapter (e.g., 802.11a/b or Wi-Fi), which would enable the pen, with some particular software installed on it, to transfer spoken voice in real time using for example, Voice over Internet Protocol (VoIP) technology, without the need for storing the voice in the pen's memory.
- Matching calibration of the position coded pattern (PCP) display screen with the actual computer screen scale may be undertaken so that inputs received from the electronic reading pen when the pen's tip is pointing at a specific location on the PCP surface (e.g., double clicking on a save button), may be interpreted as a command that matches an action initiated due to pointing on the region of interest on the computer application screen.
- Calibration commands may take place at the intended respective location of the pen tip superimposed on the application being projected on the PCP screen to enable command actions to be executed.
-
FIG. 8 illustrates example position coding patterns to be printed on the projection surface andFIGS. 9A and 9B illustrate example electronic pens. - Any suitable PCP may be utilized as illustrated in
FIG. 8.1 . The PCP pattern may be comprised of small dots placed in a pattern that follows a certain algorithmic theme which, when decoded, reveals unique information about absolute position in (x, y) coordinates as shown inFIG. 8.2 . - Other PCP patterns capable of being printed or deposited onto any display surface may be employed, including, but not limited to the Xerox data glyph pattern shown at
FIGS. 8.3 and 8.4, and other patterns, such as those produced by Microsoft® shown atFIGS. 8.5 and 8.6. - Such position coding pattern families may utilize an algorithm that can generate and distribute a pattern in a way that decodes position information over the surface on which the pattern is to be applied or printed.
- As to the processes used to decode PCP encoded information,
FIG. 9A shows an example of a reading pen produced by Anoto®. This electronic reading pen has the function of decoding the encoded position information embedded in an Anoto® PCP pattern. To perform this function, the pen comprises two functional principles which in a sense function as the eyes and brain of the pen. The two functions are accomplished through the use of an infrared CCD camera to read the dot pattern of the PCP, and a processor to decode the images being read by this camera. - The decoded (x, y) position information may be sent in real-time (i.e. simultaneously whilst being decoded), to a computer via wireless transmission.
-
FIG. 9A shows the streaming ‘data sending’ light flickering on an example Maxell® DP-201 R4.1 electronic reading pen which indicates that the pen is in action while operating on a physical shortcut defined by a user.FIG. 9B shows a semi exploded view of an example Anoto® compatible electronic reading pen, namely the DP-201 R 4.1 streaming pen. -
FIG. 10A illustrates another electronic reading pen, such as those produced by Maxell® or Logitech®, which supports additional functional components, including a pressure sensitive pen tip, storing memory chip, a vibrator to provide physical shaking feedback to a user, and a Bluetooth® antenna for wireless data transmission and reception. - In
FIG. 10B , the driver software associated with the Maxell® DP-201 R 4.1 is shown. The example driver software contains a reading port for SPP Bluetooth® service to enable connection to the electronic reading pen via Bluetooth® (FIG. 10B ). In addition, the driver contains a task bar for performing functions such as calibration, onscreen reading and mode switching (FIG. 10B ). - Another embodiment provides at least one physical shortcut virtual button shown in
FIGS. 11A-11E . These may be provided by utilizing small areas of PCP printed on paper, or any appropriate surface) with a desired thumbnail picture printed thereon. - The driver software may be configured to recognize specific addresses allocated by programming for these physical shortcuts to linking them to specific functions inside applications or to run whole applications when selected (i.e. by clicking) using the electronic reading pen.
-
FIG. 11D shows a mouse pad that can be used to remotely control a PC screen projected on a wide PCP surface or any other surface. InFIG. 11C , a color palette may be utilized as a set of physical shortcuts to brush colors in host PC based programs and applications such as Microsoft Paint®. - An additional extension to the concept of physical shortcuts is that relating a real physical entity to cyberspace items by hyper-linking the two. One example of this principle is illustrated as a printed keyboard shown in
FIG. 12 . - In
FIGS. 14A and 14B , an embodiment is shown affixed to a wall with the words ‘position coded pattern’ written with the electronic reading pen on the PCP surface using the program Smart™ Notebook running on the host PC. - Although the invention has been described with reference to specific embodiments, this description is not meant to be construed in a limited sense. Various modifications of the disclosed embodiments, as well as alternative embodiments of the invention, will become apparent to persons skilled in the art upon reference to the description of the invention. It is, therefore, contemplated that the appended claims will cover such modifications that fall within the scope of the invention.
Claims (20)
1. A computer implemented interactive display method comprising:
identifying an absolute position of an electronic pen relative to an interactive display surface having one or more computer generated patterns imprinted thereon, the electronic pen having an infrared camera for reading the computer generated patterns upon the interactive display surface and a processor for decoding the patterns read by the infrared camera upon the interactive display surface, wherein the decoded patterns indicate the absolute position of the electronic pen with respect to the interactive display surface; and
wherein the electronic pen further comprises an inkless stylus.
2. The computer implemented method of claim 1 , wherein one or more of the computer generated patterns imprinted on the interactive display surface have a predefined association with a control operation of a computer system in communication with the electronic pen such that placing the electronic pen in proximity to the associated generated pattern causes the computer system to execute the associated control operation.
3. The computer implemented method of claim 2 , further comprising:
generating a hyperlink between the computer generated pattern and the associated control operation; and
activating the hyperlink to execute the associated control operation.
4. The computer implemented method of claim 2 , further comprising superimposing a visual output of the computer system upon the interactive display surface using one or more projectors.
5. The computer implemented method of claim 1 , wherein the interactive display surface is substantially transparent.
6. The computer implemented method of claim 1 , wherein the interactive display surface is substantially flexible.
7. The computer-implemented method of claim 1 , wherein the interactive display surface further comprises a substantially transparent anti-reflective layer.
8. An interactive display system comprising:
a computer processor operative to:
identify an absolute position of an electronic pen relative to an interactive display surface having one or more computer generated patterns imprinted thereon, the electronic pen having an infrared camera for reading the computer generated patterns upon the interactive display surface and a processor for decoding the patterns read by the infrared camera upon the interactive display surface, wherein the decoded patterns indicate the absolute position of the electronic pen with respect to the interactive display surface; and
wherein the interactive display surface further comprises a magnetic adhesive.
9. The system of claim 8 , wherein one or more of the computer generated patterns imprinted on the interactive display surface have a predefined association with a control operation of a computer system in communication with the electronic pen such that placing the electronic pen in proximity to the associated generated pattern causes the computer system to execute the associated control operation.
10. The system of claim 9 , wherein the processor is operative to superimpose a visual output of the computer system upon the interactive display surface using one or more projectors.
11. The system of claim 10 , wherein the processor is operative to identify a scale of the computer system visual output and adjust the scale of the interactive display surface to substantially match the scale of the computer system visual output.
12. The system of claim 10 , wherein the processor is operative to identify dimensions of the computer system visual output and adjust the dimensions of the interactive display surface to substantially match the dimensions of the computer system visual output.
13. The system of claim 8 , wherein the electronic pen further comprises a microphone for capturing audio information.
14. The system of claim 8 , wherein the electronic pen further comprises an audio receiver for transmitting audio information.
15. A non-transitory computer readable medium for providing an interactive display comprising instructions which, when executed, cause a computing device to:
identify an absolute position of an electronic pen relative to an interactive display surface having one or more computer generated patterns imprinted thereon, the electronic pen having an infrared camera for reading the computer generated patterns upon the interactive display surface and a processor for decoding the patterns read by the infrared camera upon the interactive display surface, wherein the decoded patterns indicate the absolute position of the electronic pen with respect to the interactive display surface; and
wherein the interactive display surface further comprises a magnetic adhesive.
16. The computer readable medium of claim 15 , wherein one or more of the computer generated patterns imprinted on the interactive display surface have a predefined association with a control operation of a computer system in communication with the electronic pen such that placing the electronic pen in proximity to the associated generated pattern causes the computer system to execute the associated control operation.
17. The computer readable medium of claim 16 , wherein the interactive display surface is shaped to resemble a keyboard and wherein the control operation comprises one or more keyboard functions.
18. The computer readable medium of claim 17 , wherein the interactive display surface further comprises one or more thumbnail images, each thumbnail image resembling a keyboard button.
19. The computer readable medium of claim 16 , wherein the interactive display surface is shaped to resemble a mouse pad and wherein the control operation comprises one or more mouse functions.
20. The computer readable medium of claim 16 , wherein the interactive display surface is shaped to resemble a color palette and wherein the control operation comprises one or more color palette functions.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/227,605 US20140210799A1 (en) | 2006-06-16 | 2014-03-27 | Interactive Display System and Method |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US80496906P | 2006-06-16 | 2006-06-16 | |
PCT/IB2007/052299 WO2007144850A1 (en) | 2006-06-16 | 2007-06-15 | Interactive printed position coded pattern whiteboard |
US12/314,707 US8723791B2 (en) | 2006-06-16 | 2008-12-15 | Processor control and display system |
US14/227,605 US20140210799A1 (en) | 2006-06-16 | 2014-03-27 | Interactive Display System and Method |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/314,707 Continuation US8723791B2 (en) | 2006-06-16 | 2008-12-15 | Processor control and display system |
Publications (1)
Publication Number | Publication Date |
---|---|
US20140210799A1 true US20140210799A1 (en) | 2014-07-31 |
Family
ID=38558181
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/314,707 Expired - Fee Related US8723791B2 (en) | 2006-06-16 | 2008-12-15 | Processor control and display system |
US14/227,605 Abandoned US20140210799A1 (en) | 2006-06-16 | 2014-03-27 | Interactive Display System and Method |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/314,707 Expired - Fee Related US8723791B2 (en) | 2006-06-16 | 2008-12-15 | Processor control and display system |
Country Status (4)
Country | Link |
---|---|
US (2) | US8723791B2 (en) |
EP (1) | EP2035909A1 (en) |
JP (1) | JP2009545786A (en) |
WO (1) | WO2007144850A1 (en) |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140210785A1 (en) * | 2008-10-02 | 2014-07-31 | Wacom Co., Ltd. | Combination touch and transducer input system and method |
TWI636381B (en) * | 2017-10-19 | 2018-09-21 | 財團法人工業技術研究院 | Interactive display system and controlling method of interactive display |
US10146337B2 (en) | 2016-09-15 | 2018-12-04 | Samsung Electronics Co., Ltd. | Digital handwriting device and method of using the same |
WO2019005499A1 (en) * | 2017-06-28 | 2019-01-03 | Walmart Apollo, Llc | Systems, methods, and devices for providing a virtual reality whiteboard |
US10691207B2 (en) | 2015-09-22 | 2020-06-23 | Hewlett-Packard Development Company, L.P. | Display devices with virtual reprsentations of electronic devices |
US10698602B2 (en) | 2018-01-25 | 2020-06-30 | Acer Incorporated | Active touch pen |
US10976819B2 (en) | 2015-12-28 | 2021-04-13 | Microsoft Technology Licensing, Llc | Haptic feedback for non-touch surface interaction |
US11016581B2 (en) | 2015-04-21 | 2021-05-25 | Microsoft Technology Licensing, Llc | Base station for use with digital pens |
Families Citing this family (24)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4691136B2 (en) * | 2008-06-09 | 2011-06-01 | 株式会社エヌ・ティ・ティ・ドコモ | Method and apparatus for interaction between mobile device and screen |
US8243028B2 (en) | 2008-06-13 | 2012-08-14 | Polyvision Corporation | Eraser assemblies and methods of manufacturing same |
US8221229B2 (en) * | 2008-10-27 | 2012-07-17 | Sony Computer Entertainment Inc. | Spherical ended controller with configurable modes |
NO332210B1 (en) * | 2009-03-23 | 2012-07-30 | Cisco Systems Int Sarl | Interface unit between video conferencing codec and interactive whiteboard |
WO2010132588A2 (en) * | 2009-05-15 | 2010-11-18 | Polyvision Corporation | Electronic display systems having mobile components |
JP2013501247A (en) * | 2009-07-31 | 2013-01-10 | レモプティックス ソシエテ アノニム | Optical microprojection system and projection method |
US8462110B2 (en) * | 2009-11-03 | 2013-06-11 | Hewlett-Packard Development Company, L.P. | User input by pointing |
US20110219340A1 (en) * | 2010-03-03 | 2011-09-08 | Pathangay Vinod | System and method for point, select and transfer hand gesture based user interface |
US20120162061A1 (en) * | 2010-06-25 | 2012-06-28 | Polyvision Corporation | Activation objects for interactive systems |
US20120011465A1 (en) * | 2010-07-06 | 2012-01-12 | Marcelo Amaral Rezende | Digital whiteboard system |
CN102253791A (en) * | 2011-01-12 | 2011-11-23 | 南昊(北京)科技有限公司 | Chinese character input method of electronic whiteboard |
US8874657B2 (en) * | 2012-04-27 | 2014-10-28 | Seiko Epson Corporation | Method for securely distributing meeting data from interactive whiteboard projector |
US8641197B2 (en) * | 2011-12-23 | 2014-02-04 | Luidia Inc. | Acoustical receiver and pen transcription system adapted for rejecting virtual pen images |
KR101149814B1 (en) | 2012-02-10 | 2012-05-24 | 김길선 | Electronic board with roll-screen |
SE1200428A1 (en) * | 2012-07-09 | 2012-10-22 | Electrolux Ab | Appliance for the kitchen |
US10055030B2 (en) | 2013-05-17 | 2018-08-21 | Apple Inc. | Dynamic visual indications for input devices |
US20150002480A1 (en) * | 2013-06-27 | 2015-01-01 | Ying-Da Chen | Inductive Touch Pen Structure |
WO2015137544A1 (en) * | 2014-03-14 | 2015-09-17 | 주식회사 아이에스엘코리아 | Projection system having function of electronic whiteboard and wireless tablet function |
US9201523B1 (en) * | 2014-08-28 | 2015-12-01 | Joon Ung Hwang | Smart pen |
US9823755B2 (en) | 2015-02-26 | 2017-11-21 | Konica Minolta Laboratory U.S.A., Inc. | Method and apparatus for interactive user interface with wearable device |
US10417021B2 (en) | 2016-03-04 | 2019-09-17 | Ricoh Company, Ltd. | Interactive command assistant for an interactive whiteboard appliance |
US10409550B2 (en) * | 2016-03-04 | 2019-09-10 | Ricoh Company, Ltd. | Voice control of interactive whiteboard appliances |
CN107195208A (en) * | 2017-07-17 | 2017-09-22 | 安庆状元郎电子科技有限公司 | A kind of talking pen with communication function |
EP3750026B1 (en) * | 2018-11-26 | 2024-01-03 | Google LLC | Flexible display with electromagnetic adjustment |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2001016872A1 (en) * | 1999-08-30 | 2001-03-08 | Anoto Ab | System and devices for electronic recording of handwritten information |
US20010038383A1 (en) * | 2000-04-05 | 2001-11-08 | Petter Ericson | Method and apparatus for information management |
US6469685B1 (en) * | 1997-06-25 | 2002-10-22 | Viratec Thin Films, Inc. | Display panel filter and method of making the same |
US20050083301A1 (en) * | 2003-10-21 | 2005-04-21 | Nec Viewtechnology, Ltd. | Projector, electronic whiteboard system using projector and method of acquiring indicated point |
US20050179538A1 (en) * | 2004-02-13 | 2005-08-18 | Morita Mark M. | Hygienic input device for medical information systems |
US20060154559A1 (en) * | 2002-09-26 | 2006-07-13 | Kenji Yoshida | Information reproduction/i/o method using dot pattern, information reproduction device, mobile information i/o device, and electronic toy |
US7176896B1 (en) * | 1999-08-30 | 2007-02-13 | Anoto Ab | Position code bearing notepad employing activation icons |
Family Cites Families (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO1999050736A1 (en) * | 1998-04-01 | 1999-10-07 | Xerox Corporation | Paper indexing of recordings |
US6992655B2 (en) * | 2000-02-18 | 2006-01-31 | Anoto Ab | Input unit arrangement |
US6367183B1 (en) * | 2000-07-05 | 2002-04-09 | Robert Haber | Card assembly for card display unit |
JP2002149331A (en) * | 2000-11-15 | 2002-05-24 | Canon Inc | Coordinate plate, coordinate input device and coordinate input/output device |
US6870670B2 (en) * | 2001-04-06 | 2005-03-22 | 3M Innovative Properties Company | Screens and methods for displaying information |
US20030034961A1 (en) * | 2001-08-17 | 2003-02-20 | Chi-Lei Kao | Input system and method for coordinate and pattern |
JP2003285574A (en) * | 2002-03-27 | 2003-10-07 | Kokuyo Co Ltd | Purchase form |
CA2390506C (en) * | 2002-06-12 | 2013-04-02 | Smart Technologies Inc. | System and method for recognizing connector gestures |
US7262764B2 (en) * | 2002-10-31 | 2007-08-28 | Microsoft Corporation | Universal computing device for surface applications |
US7001023B2 (en) * | 2003-08-06 | 2006-02-21 | Mitsubishi Electric Research Laboratories, Inc. | Method and system for calibrating projectors to arbitrarily shaped surfaces with discrete optical sensors mounted at the surfaces |
US8717553B1 (en) * | 2003-09-16 | 2014-05-06 | Apple Inc. | Positioning a first surface in a pre-determined position relative to a second surface |
US20060127872A1 (en) * | 2004-03-17 | 2006-06-15 | James Marggraff | Method and device for associating a user writing with a user-writable element |
CA2532635C (en) * | 2005-01-12 | 2008-06-03 | Leapfrog Enterprises, Inc. | Interactive apparatus with recording and playback capability usable with encoded writing medium |
BE1017572A3 (en) | 2007-06-27 | 2008-12-02 | Polyvision Nv | INTERACTIVE ENAMELED VISUAL COMMUNICATION PANEL. |
US8243028B2 (en) | 2008-06-13 | 2012-08-14 | Polyvision Corporation | Eraser assemblies and methods of manufacturing same |
US20090309854A1 (en) | 2008-06-13 | 2009-12-17 | Polyvision Corporation | Input devices with multiple operating modes |
-
2007
- 2007-06-15 JP JP2009514974A patent/JP2009545786A/en active Pending
- 2007-06-15 WO PCT/IB2007/052299 patent/WO2007144850A1/en active Application Filing
- 2007-06-15 EP EP07789698A patent/EP2035909A1/en not_active Withdrawn
-
2008
- 2008-12-15 US US12/314,707 patent/US8723791B2/en not_active Expired - Fee Related
-
2014
- 2014-03-27 US US14/227,605 patent/US20140210799A1/en not_active Abandoned
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6469685B1 (en) * | 1997-06-25 | 2002-10-22 | Viratec Thin Films, Inc. | Display panel filter and method of making the same |
WO2001016872A1 (en) * | 1999-08-30 | 2001-03-08 | Anoto Ab | System and devices for electronic recording of handwritten information |
US7176896B1 (en) * | 1999-08-30 | 2007-02-13 | Anoto Ab | Position code bearing notepad employing activation icons |
US20010038383A1 (en) * | 2000-04-05 | 2001-11-08 | Petter Ericson | Method and apparatus for information management |
US20060154559A1 (en) * | 2002-09-26 | 2006-07-13 | Kenji Yoshida | Information reproduction/i/o method using dot pattern, information reproduction device, mobile information i/o device, and electronic toy |
US20050083301A1 (en) * | 2003-10-21 | 2005-04-21 | Nec Viewtechnology, Ltd. | Projector, electronic whiteboard system using projector and method of acquiring indicated point |
US20050179538A1 (en) * | 2004-02-13 | 2005-08-18 | Morita Mark M. | Hygienic input device for medical information systems |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140210785A1 (en) * | 2008-10-02 | 2014-07-31 | Wacom Co., Ltd. | Combination touch and transducer input system and method |
US9304623B2 (en) * | 2008-10-02 | 2016-04-05 | Wacom Co., Ltd. | Combination touch and transducer input system and method |
US11016581B2 (en) | 2015-04-21 | 2021-05-25 | Microsoft Technology Licensing, Llc | Base station for use with digital pens |
US10691207B2 (en) | 2015-09-22 | 2020-06-23 | Hewlett-Packard Development Company, L.P. | Display devices with virtual reprsentations of electronic devices |
US10976819B2 (en) | 2015-12-28 | 2021-04-13 | Microsoft Technology Licensing, Llc | Haptic feedback for non-touch surface interaction |
US10146337B2 (en) | 2016-09-15 | 2018-12-04 | Samsung Electronics Co., Ltd. | Digital handwriting device and method of using the same |
WO2019005499A1 (en) * | 2017-06-28 | 2019-01-03 | Walmart Apollo, Llc | Systems, methods, and devices for providing a virtual reality whiteboard |
TWI636381B (en) * | 2017-10-19 | 2018-09-21 | 財團法人工業技術研究院 | Interactive display system and controlling method of interactive display |
US10698602B2 (en) | 2018-01-25 | 2020-06-30 | Acer Incorporated | Active touch pen |
Also Published As
Publication number | Publication date |
---|---|
US8723791B2 (en) | 2014-05-13 |
US20090213070A1 (en) | 2009-08-27 |
JP2009545786A (en) | 2009-12-24 |
EP2035909A1 (en) | 2009-03-18 |
WO2007144850A1 (en) | 2007-12-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8723791B2 (en) | Processor control and display system | |
US9684411B1 (en) | Method and apparatus of position tracking and detection of user input information | |
US9250742B1 (en) | Method and apparatus of position tracking and detection of user input information | |
US7083100B2 (en) | Drawing, writing and pointing device | |
US20070188477A1 (en) | Sketch pad and optical stylus for a personal computer | |
US20010030668A1 (en) | Method and system for interacting with a display | |
JP2004164609A (en) | Universal input device | |
JP2008217782A (en) | Paper-based meeting service management tool and system | |
CN101615087A (en) | Electronic pen framework and data entry device thereof | |
JP6044198B2 (en) | Computer apparatus, program, and information processing system | |
JP7087364B2 (en) | Information processing equipment, information processing systems and programs | |
JP2006012184A (en) | Display device, information processor and its control method | |
JP2011084050A (en) | Electronic writing system | |
CN113176834B (en) | Screen writing system | |
GB2377607A (en) | Analysing and displaying motion of hand held instrument | |
AU3972399A (en) | Transcription system | |
JP2002278703A (en) | Information input device and electronic blackboard system | |
KR20170004102A (en) | Electronic device | |
CN215932586U (en) | Screen writing system | |
KR100892367B1 (en) | Apparatus and Method for digital rubbing | |
CN215932585U (en) | Screen writing device | |
WO2022034890A1 (en) | Handwriting device | |
GB2340712A (en) | Transcription system | |
JP2010238213A (en) | Tablet pc system and electronic writing sheet | |
KR20180033738A (en) | Detachable Touch Panel Device Equipped with Diplay Device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: KETAB TECHNOLOGIES LIMITED, MALAYSIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KALALDEH, KHALED A.;SAAD EDDIN, TAWFEEQ A.;REEL/FRAME:032636/0047 Effective date: 20090223 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |