WO1996029677A1 - An interactive display and input device - Google Patents

An interactive display and input device Download PDF

Info

Publication number
WO1996029677A1
WO1996029677A1 PCT/GB1996/000523 GB9600523W WO9629677A1 WO 1996029677 A1 WO1996029677 A1 WO 1996029677A1 GB 9600523 W GB9600523 W GB 9600523W WO 9629677 A1 WO9629677 A1 WO 9629677A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
input device
plane
object
image plane
Prior art date
Application number
PCT/GB1996/000523
Other languages
French (fr)
Inventor
Clive Van Heerden
Original Assignee
Central Research Laboratories
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to GB9505664.4 priority Critical
Priority to GBGB9505664.4A priority patent/GB9505664D0/en
Application filed by Central Research Laboratories filed Critical Central Research Laboratories
Publication of WO1996029677A1 publication Critical patent/WO1996029677A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/042Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
    • G06F3/0425Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/10Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
    • A63F2300/1087Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals comprising photodetecting means, e.g. a camera
    • A63F2300/1093Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals comprising photodetecting means, e.g. a camera using visible light
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS, OR APPARATUS
    • G02B5/00Optical elements other than lenses
    • G02B5/12Reflex reflectors
    • G02B5/122Reflex reflectors cube corner, trihedral or triple reflector type
    • G02B5/124Reflex reflectors cube corner, trihedral or triple reflector type plural reflecting elements forming part of a unitary plate or sheet

Abstract

An interactive display apparatus includes an input device, the device comprising a source of electromagnetic radiation, and an array of detectors (facing the image plane and lying in a different plane) arranged to receive electromagnetic radiation from the source reflected by an object, apparatus producing a real image suspended in space in the image plane, and control means arranged such that a user may interact with the image by placing an object in or adjacent the image plane. This arrangement enables a user to interact with an apparently three-dimensional suspended image in a way similar to using a touch screen but without having a physical screen to touch.

Description

AN INTERACTIVE DISPLAY AND INPUT DEVICE

This invention relates to an input device for a display having at least one image plane, the device comprising a source of electromagnetic radiation and an array of detectors being arranged to receive electromagnetic radiation from the source being reflected by an object being placed in or adjacent the image plane. The invention also relates to an interactive display apparatus.

Input devices for visual displays in the form of 'touch screens' are an increasingly popular way of interacting with computer systems. This is because of the inherent robustness of the devices and the elimination of electromechanical devices such as keyboards, mice and joy-sticks. One known system is disclosed in GB-A-2 268 263. In this device a light beam from a source is scanned across a plane by a shutter, and a photodetector in the same plane detects light reflected by a stylus which light returns back through the shutter. Such a system may be placed on the front of a cathode ray tube to allow interactive control of, for example, a computer system.

However, all touch screen systems have the disadvantage of being associated with a screen which has to be touched. Such screens can be damaged and tend to get dirty which can lead to spurious data input. Also, such displays are in a single plane, and cannot give 3 dimensional effects because of the presence of flatness cues which are recognized by the brain of a user. The known system disclosed in the above mentioned patent also uses expensive components which have to be carefully aligned. Furthermore, if two objects are present at the same time one can produce a shadow so that the other cannot be reliably detected.

It is an object of the present invention to enable the above disadvantages to be mitigated.

According to a first aspect of the invention, there is provided an input device as defined in the first paragraph above characterized in that the array of detectors faces, and does not lie in, the image plane. This enables the device to detect a plurality of objects simultaneously and substantially simplifies construction. Preferably, a plurality of substantially parallel planes are illuminated by at least one source. This gives the advantage of enabling the speed of an object passing through the planes to be measured. Very preferably the electromagnetic radiation being reflected by an object in a plane is characteristic of that plane.

According to a second aspect of the invention, there is provided an interactive visual display apparatus comprising the input device defined above and apparatus for providing a real image suspended in space in the image plane. This enables the apparatus to generate three dimensional effects and enables the touch screen to be dispensed with to provides a 'touch-space' which is a more intuitive interface.

In order that the present invention may be more readily understood, reference will now be made, by way of example, to the accompanying drawings, in which: Figure 1 is a perspective view of an apparatus for providing a real image suspended in space,

Figures 2 and 3 are cross-sectional side views of the apparatus of Figure 1,

Figures 4 and 5 both show an input device and an interactive display apparatus according to the invention Before describing the invention in detail, an apparatus for forming a real image suspended in space for use with the invention will be described. Referring to Figures 1 and 2, a beamsplitter 4 is mounted for example in an aperture in an opaque surface 2.

In this embodiment, retroreflector means comprising an array of retroreflectors, such as a sheet 6 of comer-cubes, is positioned in a path for light which has been reflected by the beamsplitter 4. In this example it is positioned at an acute angle thereto, which is about 45°, so as to form the second side of a triangle therewith. An object 8 such as a picture displayed on a VDU screen is positioned along the third side of the triangle, at approximately 90° to the beamsplitter. For example, the screen 8 may be positioned at an angle in the range from 70° to 110° to the beamsplitter 4. In one specific arrangement, means for screening the retroreflector 6 from the object 8 is provided, such as a louvre film 12 in front of the object 8. As indicated by arrows in figure 2, the beamsplitter 4 reflects light from the object 8 towards the retroreflector 6. In addition some of the light is transmitted by the beamsplitter 4 and is "lost". The beamsplitter 4 also transmits light reflected from the retroreflector 6 to form the suspended image 14 opposing the object 8 with respect to the beamsplitter 4.

In addition, some of the light is reflected by the beamsplitter back towards the original image 8.

In the apparatus shown in Figure 3, a further array of retroreflectors 16 is positioned in a path for light which has been transmitted by the beamsplitter. In this example, the further array opposes the retroreflector 6 with respect to the beamsplitter

4; that is, it forms a triangular arrangement with the beamsplitter 4 and the suspended image 14. The further array of retroreflectors 16 directs light which is transmitted by the beamsplitter 4 directly from the original image 8 back towards the beamsplitter 4 to prevent 'loss' of this light. In an alternative arrangement, the array of retroreflectors 6 in a path for light which is reflected by the beamsplitter 4 could be omitted, leaving only the array in the path of transmitted light.

In the example shown in Figure 3, the object 8 is a projection screen, and an image is formed thereon by a projector 18. The projector 18 is shown positioned in the gap 10 between the retroreflector array 6 and the object 8, but may have other positions. For example it may be positioned in an aperture in the retroreflector array 6. As indicated by plus and minus signs adjacent arrows in the Figure, the beamsplitter may be polarization selective, transmitting light having one type of circular polarization and reflecting light having the other type. For example, it may comprise layers of cholesteric liquid crystal polymers reflecting different spectral regions of circularly polarized light of one sense so as to reflect white light (i.e. the whole visible spectrum) of this sense.

Additionally, the object may be arranged to emit circularly polarized light, in which case one of the arrays 6, 16 may be omitted, since all of the light from the object could be of the type which is either reflected or is transmitted by the selective beamsplitter. This could be achieved by using a transmissive or reflective projection screen which either preserves or reverses polarization, and a projector which produces linearly polarized light, such as an LCD projector, combined with a quarter wave plate. A transmissive such screen is disclosed in GB 1509180.

An apparatus for producing a suspended image 14 in an image plane has been described above. Figures 4 and 5 show such an image together with an input device according to the present invention.

In Figure 4, the image 14 is produced using apparatus as shown in Figure 2. In addition, an array of detectors, in the present example a CCD camera 20, which is sensitive to infra-red light, is constructed and arranged to face the image plane. A source 23 of infra red light 24 is provided (recessed into the surface 2) for illuminating the plane which contains the suspended image 14 with infra red light. The CCD camera is connected to a computer 30, which may optionally also be connected to the apparatus producing the image 14. The camera points in a direction substantially perpendicular to the plane of the image.

In use, if the observer 21 just looks at the image, the CCD camera does not receive any infra red light scattered from the source 23. Only general background radiation from the observer and the surroundings will be received. If, however, a user interacts with the image by placing an object (for example a stylus or a finger 22) into the plane of the image, the sheet of infra red light will be interrupted, and some light 25 will be scattered in the direction of the CCD camera. The CCD camera records the position from which the light is scattered. This information is then inputted to the computer 30. If this computer is connected to the image producing apparatus this information may be used to change the information being displayed in response to the presence of the object. In order to enhance contrast between the infra red light being scattered from the object 22 and other ambient radiation, a filter 31 which selectively transmits wavelength emitted by the source 23 may be placed in front of the CCD camera.

Other techniques to enhance contrast may be employed alternatively or in addition to the above mentioned filter. For example, the light source and camera can be operated intermittently and in synchrony, in this way a higher power infra-red pulse can be employed without the background radiation reducing the signal to noise ratio between pulses. In another alternative a snap-shot of the radiation being scattered from a time modulated or non-modulated source can be captured using 'frame- grabbing' software. This snap-shot can then be compared with another snap-shot (for example taken when the rnodulated source is not radiating). This will make infra red light scattered by the object more distinguishable from ambient light. Software can be employed to look for large changes between successive frames, simplifying the amount of data analysis required. A lens with a large aperture and hence a restricted field of focus may be used for the CCD camera so that only objects in the vicinity of the image plane are focussed sharply.

Figure 5 shows a further embodiment of the invention. A suspended image 14 is produced from an image source 29 such as a cathode ray tube using a beam splitter 26 and a sheet of comer cube retroreflector 27. In this example two sources of infra red light 23 and 28 are provided, such that one source 28 produces a sheet of light parallel to the plane of the image 14 but spaced from the plane a few centimeters towards the position of an observer, and the other source 23 produces a sheet of light in the plane of the image. Using this embodiment, the user can initiate one form of data input by interrupting the light radiated from source 28, and another form of data input by interrupting the light radiated from source 23. In order to enable the CCD camera 20 to differentiate the first form of data from the second form of data, the sources may be constructed to provide electromagnetic radiation of different respective frequencies. Alternatively or in addition other methods, for example time sequential switching of the sources, may be employed to aid in differentiation of the two forms of data.

The apparatus shown in Figure 5 may alternatively or in addition be used to measure the speed of an object passing through the plane of the image by measuring the time between detection of one form of data and the other. If the object does not move directly towards the array of photodetectors (the CCD camera in the present example), the position of the two forms of data will be shifted laterally, so that it will be possible to compute the direction of travel as well as the speed of the object.

The CCD camera can record the size of the object placed in the image plane by measuring the size of the image produced in the camera. In addition the image produced in the camera can be processed using computer software to remove any distortions produced by having the detector array or CCD camera not normal to the image plane (e.g. keystone distortion).

In order to provide an interesting and efficient system of user interaction, it is desirable to include sensory feedback from the system to the user in response to objects being detected. This may be achieved aurally via sounds, visually by changes in the image or ambient lighting, tangibly by means of air currents, or by olfaction of released scents, or through a combination of any of these methods.

Although in the above examples, the image plane is illuminated from below, the illumination may be provided from the top or sides as an alternative or in addition.

The invention provides an interesting way of interacting with suspended images (including for example, holograms, etc.) in two or in three dimensions and as a function of time. It enables the construction of "touchless screens" for diverse applications including dispensing booths, video games, and novel musical instrument interfaces. The apparatus can measure many variables so that it can be employed as a natural, intuitive "soft" interface allowing lots of expression. As well as being used to signal the presence, and detect the position and speed of an object moveable relative to the display and/or suspended image, a detector array such as a CCD camera can identify the shape and orientation of objects such as fingers or hands, thereby increasing the flexibility of methods of interacting with the display and any associated computer.

The term 'suspended image' in the present description denotes a real image formed in real space which does not require the presence of a reflective or transmissive screen for its observation. The priority document of the present application, namely GB 9505664.4 is hereby incorporated herein by reference.

In summary, an interactive display apparatus includes an input device, the device comprising a source of electromagnetic radiation, and an array of detectors (facing the image plane and lying in a different plane) arranged to receive electromagnetic radiation from the source reflected by an object, apparatus producing a real image suspended in space in the image plane, and control means arranged such that a user may interact with the image by placing an object in or adjacent the image plane. This arrangement enables a user to interact with an apparently three dimensional suspended image in a way similar to using a touch screen but without having a physical screen to touch.

Claims

1. An input device for a display having at least one image plane, the device comprising a source of electromagnetic radiation, and an array of detectors being arranged to receive electromagnetic radiation from the source being reflected by an object being placed in or adjacent the image plane, characterized in that the array of detectors faces, and does not lie in, the image plane.
2. An input device as claimed in claim 1 in which the electromagnetic radiation is infra-red radiation and the detectors are responsive to infra-red radiation.
3. An input device as claimed in any previous claim in which the array of detectors is arranged to face in a direction approximately perpendicular to the image plane.
4. An input device as claimed in any preceding claim in which the source of electromagnetic radiation illuminates the image plane or an adjacent plane.
5. An input device as claimed in claim 4 in which a plurality of substantially parallel planes are illuminated by at least one source.
6. An input device as claimed in claim 5 in which the electromagnetic radiation being reflected by an object in a plane is characteristic of that plane.
7. An input device as claimed in claim 6 further comprising means for determining from which plane electromagnetic radiation is being reflected by a given object, and inputting this information to a further device.
8. An input device as claimed in any preceding claim in which the array of infra¬ red detectors comprises a CCD camera.
9. An interactive display apparatus comprising an input device as claimed in any preceding claim and apparatus for providing a real image suspended in space in the at least one image plane.
10. An interactive display apparatus as claimed in claim 10, further comprising control means being arranged such that a user is capable of interacting with the said real image by placing an object in or adjacent said at least one image plane.
SUBS. TTTUTE SHEET (RULE 26)
PCT/GB1996/000523 1995-03-21 1996-03-07 An interactive display and input device WO1996029677A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
GB9505664.4 1995-03-21
GBGB9505664.4A GB9505664D0 (en) 1995-03-21 1995-03-21 An interactive display and input device

Publications (1)

Publication Number Publication Date
WO1996029677A1 true WO1996029677A1 (en) 1996-09-26

Family

ID=10771559

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/GB1996/000523 WO1996029677A1 (en) 1995-03-21 1996-03-07 An interactive display and input device

Country Status (2)

Country Link
GB (1) GB9505664D0 (en)
WO (1) WO1996029677A1 (en)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0837418A2 (en) * 1996-10-18 1998-04-22 Kabushiki Kaisha Toshiba Method and apparatus for generating information input using reflected light image of target object
FR2773894A1 (en) * 1998-01-22 1999-07-23 Eastman Kodak Co interactive interface
US6204973B1 (en) * 1997-02-19 2001-03-20 Central Research Labs, Ltd. Apparatus for displaying an image suspended in space
US6734847B1 (en) 1997-10-30 2004-05-11 Dr. Baldeweg Gmbh Method and device for processing imaged objects
FR2853423A1 (en) * 2003-03-20 2004-10-08 Simag Dev Data input device for controlling electronic machine e.g. computer, has synchronization input unit including filter isolating detection signal correlated with signal received at input unit, from output signal of optical sensor
WO2006002666A1 (en) * 2004-06-30 2006-01-12 Telecom Italia S.P.A. Inputting information using holographic techniques
WO2006027423A1 (en) * 2004-08-09 2006-03-16 Simag Developpement Input device comprising an optical sensor and a filter means
WO2009039260A2 (en) * 2007-09-18 2009-03-26 Gefemer Research Acquisitions, Llc. Method and apparatus for holographic user interface communication
FR2928468A1 (en) * 2008-03-04 2009-09-11 Gwenole Bocquet Device for non-touch interaction with an image not based on any support
US8902225B2 (en) 2007-10-31 2014-12-02 Genedics Llc Method and apparatus for user interface communication with an image manipulator
US9110563B2 (en) 2007-10-31 2015-08-18 Genedics Llc Method and apparatus for user interface of input devices

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4593967A (en) * 1984-11-01 1986-06-10 Honeywell Inc. 3-D active vision sensor
JPH03217925A (en) * 1990-01-23 1991-09-25 Dainippon Printing Co Ltd On-display screen information input device using hologram display
EP0554492A1 (en) * 1992-02-07 1993-08-11 International Business Machines Corporation Method and device for optical input of commands or data
WO1994025915A1 (en) * 1993-04-28 1994-11-10 Mcpheters R Douglas Holographic operator interface

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4593967A (en) * 1984-11-01 1986-06-10 Honeywell Inc. 3-D active vision sensor
JPH03217925A (en) * 1990-01-23 1991-09-25 Dainippon Printing Co Ltd On-display screen information input device using hologram display
EP0554492A1 (en) * 1992-02-07 1993-08-11 International Business Machines Corporation Method and device for optical input of commands or data
WO1994025915A1 (en) * 1993-04-28 1994-11-10 Mcpheters R Douglas Holographic operator interface

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
ARAKI K ET AL: "HIGH SPEED AND CONTINUOUS RANGEFINDING SYSTEM", IEICE TRANSACTIONS, vol. E74, no. 10, 1 October 1991 (1991-10-01), pages 3400 - 3406, XP000279319 *
PATENT ABSTRACTS OF JAPAN vol. 015, no. 503 (P - 1290) 19 December 1991 (1991-12-19) *

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0837418A3 (en) * 1996-10-18 2006-03-29 Kabushiki Kaisha Toshiba Method and apparatus for generating information input using reflected light image of target object
EP0837418A2 (en) * 1996-10-18 1998-04-22 Kabushiki Kaisha Toshiba Method and apparatus for generating information input using reflected light image of target object
US6204973B1 (en) * 1997-02-19 2001-03-20 Central Research Labs, Ltd. Apparatus for displaying an image suspended in space
US6734847B1 (en) 1997-10-30 2004-05-11 Dr. Baldeweg Gmbh Method and device for processing imaged objects
FR2773894A1 (en) * 1998-01-22 1999-07-23 Eastman Kodak Co interactive interface
EP0932116A1 (en) * 1998-01-22 1999-07-28 Eastman Kodak Company Interactive interface
FR2853423A1 (en) * 2003-03-20 2004-10-08 Simag Dev Data input device for controlling electronic machine e.g. computer, has synchronization input unit including filter isolating detection signal correlated with signal received at input unit, from output signal of optical sensor
WO2006002666A1 (en) * 2004-06-30 2006-01-12 Telecom Italia S.P.A. Inputting information using holographic techniques
US8089456B2 (en) 2004-06-30 2012-01-03 Telecom Italia S.P.A. Inputting information using holographic techniques
WO2006027423A1 (en) * 2004-08-09 2006-03-16 Simag Developpement Input device comprising an optical sensor and a filter means
JP2010539620A (en) * 2007-09-18 2010-12-16 ゲフェマー リサーチ アクイジションズ エルエルシー Method and apparatus for holographic user interface communication
WO2009039260A2 (en) * 2007-09-18 2009-03-26 Gefemer Research Acquisitions, Llc. Method and apparatus for holographic user interface communication
WO2009039260A3 (en) * 2007-09-18 2009-07-02 Gefemer Res Acquisitions Llc Method and apparatus for holographic user interface communication
US9939987B2 (en) 2007-10-31 2018-04-10 Genedics Llc Method and apparatus for user interface of input devices
US8902225B2 (en) 2007-10-31 2014-12-02 Genedics Llc Method and apparatus for user interface communication with an image manipulator
US9110563B2 (en) 2007-10-31 2015-08-18 Genedics Llc Method and apparatus for user interface of input devices
US9335890B2 (en) 2007-10-31 2016-05-10 Genedics Llc Method and apparatus for user interface of input devices
WO2009112722A2 (en) * 2008-03-04 2009-09-17 Bocquet Gwenole Oled lcd hybrid interactive device with microstructured plates
FR2928468A1 (en) * 2008-03-04 2009-09-11 Gwenole Bocquet Device for non-touch interaction with an image not based on any support
WO2009112722A3 (en) * 2008-03-04 2009-12-23 Bocquet Gwenole Oled lcd hybrid interactive device with microstructured plates

Also Published As

Publication number Publication date
GB9505664D0 (en) 1995-05-10

Similar Documents

Publication Publication Date Title
US8094137B2 (en) System and method of detecting contact on a display
US9063617B2 (en) Interactive display system, tool for use with the system, and tool management apparatus
CN101571776B (en) The electronic device having a projector module
US7525538B2 (en) Using same optics to image, illuminate, and project
US7242388B2 (en) Data input device
EP0963563B1 (en) Method and device for detecting an object in an area radiated by waves in the invisible spectral range
US7978184B2 (en) Interactive window display
US6545670B1 (en) Methods and apparatus for man machine interfaces and related activity
US6061177A (en) Integrated computer display and graphical input apparatus and method
US5738429A (en) Portable projection display apparatus
KR100477462B1 (en) Flat-panel display
US20050024324A1 (en) Quasi-three-dimensional method and apparatus to detect and localize interaction of user-object and virtual transfer device
EP1059605A2 (en) Optical unit for detecting object and coordinate input apparatus using the same
US20170052646A1 (en) Touchscreen for Detecting Multiple Touches
CA2160245C (en) Holographic operator interface
US20120127128A1 (en) Hover detection in an interactive display device
US8289299B2 (en) Touch screen signal processing
DE19810452B4 (en) Optical digitizer
US7054045B2 (en) Holographic human-machine interfaces
US20180232048A1 (en) Holographic waveguide optical tracker
US8042949B2 (en) Projection of images onto tangible user interfaces
US9270978B2 (en) Method and device for 3-D display based on random constructive interference
US20030137494A1 (en) Human-machine interface
KR0145098B1 (en) Back lighting system for transmissive display
AU2004211738B2 (en) Touch screen signal processing

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): CA JP US

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE CH DE DK ES FI FR GB GR IE IT LU MC NL PT SE

121 Ep: the epo has been informed by wipo that ep was designated in this application
122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase in:

Ref country code: CA