WO2001071407A1

WO2001071407A1 - Hand-held stereoscopic imaging system with wireless communication capability

Info

Publication number: WO2001071407A1
Application number: PCT/US2001/008873
Authority: WO
Inventors: Michael Steinthal
Original assignee: Stereovision Imaging, Inc.
Priority date: 2000-03-20
Filing date: 2001-03-20
Publication date: 2001-09-27
Also published as: AU2001245874A1

Abstract

A wireless hand held stereoscopic imaging system (10) capable of simultaneously viewing remote real-time 3-D image while concurrently viewing a distant, magnified 3-D object. Real-time 3-D images are received from an/or transmitted remotely to, e.g., a node processor (82), which in turn is connected to remote computers (86), via a wireless transceiver (70, 44a, 44b). A stereo video multiplexer/demultiplexer circuit (62) is provided to multiplex stereo, 3-D image signals for transmission and demultiplex received 3-D image signals. Directly viewed 3-D images and received 3-D images may be, upon demand, stored in memory (66) in the hand held imaging system (10) and later played back in 3-D within the device (10) or transmitted externally to, e.g., the node processor (82).

Description

HAND-HELD STEREOSCOPIC IMAGING SYSTEM WITH WIRELESS COMMUNICATION CAPABILITY

Field of Invention

The general field of the present invention is hand-held stereoscopic imaging systems Specifically, however, the invention relates to a solid state stereoscopic imaging system housed within a traditional hand-held pair of pπsm binoculars, and which is also capable of real-time receipt and transmission of data via wireless communication with an external source or recipient, such as the Internet for example

Background of the Invention

The use of prisms to produce enlarged images of distant objects dates back centunes, beginning, according to the history books, when Galileo first held up two prisms and gazed through them. Soon, the appropriated juxtaposed prisms were incoφorated into elongated telescopes through which the viewer peered using one eye. The image presented was, of course, flat, consisting of only two dimensions. Later, the same technology was incoφorated into hand-held binoculars. The conventional binocular is basically two small refracting telescopes held together by a frame that positions the telescopes, one to each of

the viewer's eyes. Because the binocular incoφorates a separate telescope for each eye, it therefore produces a stereoscopic or three-dimensional view that adds "depth" the image as perceived in the viewer's brain

Each refracting telescope in the binocular has an optical path defined through an objective lens at the end nearest the object being viewed, a pair of prisms appropπately arranged withm the telescope's tubular body, and an eye piece that is a the end nearest the viewer's eye The diameter of the objective lens determines the light-gathering power The objective lenses (m the two adjacent telescopes) are often spaced farther apart than the eyepieces so as to enhance stereoscopic vision Functioning as a magnifier, the eyepiece forms a large virtual image that becomes the object for the eye itself and thus forms the final image on the retina Because of the spacing between the objective lenses, the object is "viewed" from a slightly different angle y each lens and therefore collects a slightly different image. Thus, the image projected onto the retina of each eye is also slightly different, and when the viewer's brain incoφorates and melds the two slightly different images received through both eyes, a unified but 3-D or stereoscopic image is perceived by the viewer.

Binoculars are now in ubiquitous usage throughout the world in many, many human endeavors from bird watching to opera-going to star-gazing. Over the years since the binocular was first introduced, many improvements have been made. Until recently, however, these improvements related mainly to refinements in the quality of the binocular' s basic component parts (such as improving the optical components to produce clearer images), making them adjustable, making them more durable, making them smaller, making them more ergonomically balanced, adding low light capability, etc

Recently, U S Patent No 5,581 ,399 disclosed an improvement to the traditional binoculars by incoφorating an image sensor, a first optical system (composing the traditional lens-pπsms-eyepiece arrangement), a second optical system (which digitized the signal and included some limited memory) and a display so that the viewer could choose either to view enlarged images through the first optical system in the traditional way, or to view electronically reproduced images that were previously stored in memory withm the second optical system and u n replayed on the display This invention, however, relied heavily on moving mechanical parts that are subject to wear and failure, and placed the transparent display component in the optical path such that the image was degraded. Therefore, although this system provided the new feature of electronic record and playback, it suffered from some significant drawbacks.

More recently still, U.S. Pat. No. 5,963,369 discloses a solid-state stereoscopic imaging system incoφorated within a pair of hand-held binoculars The device disclosed a first optical system, a second optical system, and a third optical system The first optical system allows for magnified stereo viewing of an external object in the traditional sense. The second optical system allows for recording the magnified stereo image(s) viewed through the optics of the first optical system. The third optical system allows for reproduction of the magnified stereo ιmage(s) captured by the second optical system The hand-held 3-D imaging system disclosed in this patent further includes record and playback modes that are activated by switches connected to electronic processing circuitry located within the frame of the binoculars Lastly, this patent further teaches the notion of simultaneously viewing pre-stored 3-D images while concurrently viewing the outside world. For example, someone engaged in bird w atch could be viewing a bird in the wild, and simultaneously scrolling through stored images of birds so as to correctly identify the bird in the wild The images of that bird could then be stored for later viewing, study and confirmation

er, the device disclosed in the '369 patent is limited to images that are either actually visible m the real world in real time, o^r to the images that have been previously electronically stored in the system. This patent neither disclosed nor suggest d any componentry that permitted real time receipt and viewing of digital images or data received from an external source, on the one hand, or real time external transmission of the traditional images perceived, on the other hand. Using the previously bird-watching example with the prior art device, if the bird-in-the-wild was not previously recorded in the system, the user would not be able to confirm its identification. Thus, the stereoscopic system disclosed in the '369 patent, while a significant advance, was nevertheless limited in this respect.

Accordingly, there is a need in the art for a hand held stereoscopic imaging system that would permit the simultaneous receipt and viewing, or viewing and transmission, of information from and to a remote location (such as any node attached to the Internet) The ability to overlay real-time 3-D information w hile stereoscopically viewing the outside w orld to pro\ ιde up to date statistics, graphics, location and/or other pertinent information is important value-added knowledge, and would free the user of the system from the inevitable limitation on the amount of memory than can realistically be included in a handheld binocular type device

Summary of the Invention

The present invention is generally directed to a wireless hand-held 3-D imaging system that can be used for outdoor 3-D viewing, recording viewed objects in 3-D, and internal or external playback of objects recorded in 3-D. In addition, pre-stored or remote real-time images can be overlaid while stereoscopically viewing the outside world as well as stereoscopically recorded and played back.

In a first, separate aspect of the present invention, the wireless hand-held stereoscopic imaging system looks and feels much like a traditional pair of binoculars with the addition of an integrated stereoscopic imaging system and wireless telemetry circuitry that will both transmit and receive data externally of the device. In another, separate aspect of the present invention remote mtormation can be received by the ireless hand-held stereoscopic imaging system and or overlaid w hile simultaneously v iew ing the outside world This can be useiul for scene inteφretation and/or image recognition

In still another, separate aspect of the present inv ention information can be transmitted by the device and sent over the Internet Live 3-D information can then be accessed by a mobile end user, processor node, or anyone w ith a personal computer tied to the Web

Accordingly, it is a pπmary object of the present inv ention to pro ide a low cost, wireless, web-enabled pπsm binocular pair with an integrated solid-state stereoscopic imaging system that will allow for 3-D viewing to a worldw ide audience, and with the capabilities to monitor, control, send, recei e, and update system information from anywhere

This and further objects and advantages will be apparent to those skilled in the art in connection with the drawings and the detailed descπption of the preferred embodiment set forth below

Brief Description of the Draw ings

Figure No is a perspective view of the preferred embodiment of this invention showing the externally-visible components of the traditional binocular device which incoφorates the user-accessible record and playback buttons and antenna for receiving / transmitting data

Figure No. 2 shows the binoculars of Figure 1 , but with a portion cutaway so as to display the intenor arrangement of its pπmary components such as the objective lens, beam splitter, pnsms, eyepiece, and wireless telemetry chip

Figure No 3 is a schematized block diagram of an integrated w ir less stereoscopic imaging system and wireless telemetry of this invention

Figure No. 4 is an illustration graphically showing how the present invention can be used by the viewer to receive and/or transmit information and images from and to any number of remote locations via the Internet.

Detailed Description of the Preferred Embodiment

Referring now to Figure No. 1 , the complete package of the preferred embodiment looks very similar to today's hand-held binoculars 10. A pair of binoculars is basically two small refracting telescopes 12 and 14 held together by a frame 16 that hold the two telescopes in stationary position a distance from one another. Extending from the other end of the frame 16 are a pair of tubes 18 and 20, each of which is capped with a preferrably rubberized gaskets 22 and 24 that are designed and shaped to comfortably fit against the viewer's eyes The distance between tubes 18 and 20 is designed to be that of the typical spacing between human eyes. Because that spacing can vary from person to person, the frame 16 is of two-piece construction, attached at pivot point 26 so that the distance between the tubes can be modified slightly.

Because of this spatial juxta-positioning, each telescope "sees" the image being viewed from a slightly different angle or perspective, just as occurs with the spaced-apart human eyes. It will be noted that in this embodiment the telescopes are actually spaced further apart than the eye tubes so as to produce even greater difference in perspective than is achieved with the human eye, and thereby to produce an enhanced stereoscopic or three-dimensional view.

Looking at Figure 2, it will be seen that each refracting telescope has an optical path defined through an objective lens 30, a beam splitter 31, a pair of pπsms 32 and 34, and an eyepiece 36. The diameter of the objective lens 30 determines the light-gathering power o f that telescope Functioning as a

magni fier, the eyepiece 36 forms a large v irtual image which becomes the ob|ec; for the eye itself and thus forms the final image on the retina

Taking advantage of the proven existing binocular design, the preferred embodiment integrates the invention's solid state imaging system and wireless telemetry within the traditional package so that only a very few modification to the extenor of the binocular are made

Refemng again to Figure 1 , a receive/record button 40 and transmit/playback button 42 button are vi ibir- on il-.e top view of the outside housing. The receive/record button 42 allows the user to record any image(s) seen through said binoculars 10 and/or receive wireless information. Likewise, the transmit/playback button 42 allows the user to instantly (or at a later time) playback the stored images and the sound file as a "video clip" stereoscopically— or in other words, the images are played back simultaneously to each eye through the eyepieces 36. In addition, antenna 44a and 44b are visible on the exterior of the device, and are used to receive and transmit wireless information to and from the device so that 3-D information can be transmitted to a remote site for real-time external 3-D viewing. It is well known in the art how to record viewed magnified images in stereo and playback internally and/ or externally in 3-D, and such conventional means (not shown) are contemplated for use in the prefened embodiment of this invention Similarly, it is well known in the art how to view a distant object in 3- D while simultaneously overlaying prestored 3-D images, and those conventional means (not shown) are also used in this preferred embodiment

Turning now to Figure 3, a schematized block diagram of the preferred embodiment of the integrated wireless stereoscopic imaging system with w ireless remote capabilities is shown and is hereafter descπbed

The embedded micro-controllci 50 io responsible f : --nab .ig, disabling the CMOS photo arrays 52, and the AMLCDs 54 are tπggered by the receive/record button 40, transmit playback button 42, respectively The video signal is digitized by the A/D converters 56. The information is processed by the digital signal processor (DSP) 58. In order to view the magnified stereo image captured by the device to an audience, a field-sequential signal 60 is provided. In field-sequential stereoscopic video, the left-eye image is stored in the odd video field, and the right image, in the even field (or vice-versa). In addition, since the two images are combined into one signal via the stereo video multiplexor 62, the left and the right images must always be "in sync." Thus the genlock and synchronization circuitry 64, plus the stereo video multiplexor 62, enable a composite video machine, i.e., a TN or PC monitor, to accept the two video signals simultaneously. The DSP 58 is also responsible for image compression before it is stored in video RAM 66 The DSP 58 also may have to be responsible for image stabilization if the binocular magnification pow er is high enough to cause any image distortions

The stereo multiplexor is tied to a wireless telemetry chip 70 The telemetry chip modulates and/or demodulates the field sequential signal for wireless transmission via attached integrated antenna 44a and 44b Likewise, the stereo video multiplexor 62 acts as a multiplexor or de-multiplexor depending whether the information is being transmitted or received, respectiveh The received information is then transferred and processed by the imbedded digital signal processor 58. Upon demand, the information can either be stored in the video RAM 66, or viewed within the device or both

How this device can be used via the Internet is shown in Figure 4. User 80 is shown using integrated hand-held stereoscopic imaging system 10 to look out at the outside world. Upon request, the user 80 can transmit what is currently being viewed. In addition the transmission can be sent, for example, to a node processor 82 that has Internet connectivity 84. Through any home computer 86, anywhere in the world, a computer user 88 can connect to the Web and remotely view, in real time, the transmission in 3-D. The user 88 or any number of viewers must wear special glasses or polarizers 90 to view the transmission in 3-D. The polarizers 90 separate the left image from the right image. The computer user's brain will attribute a sense of three dimensionality to the image 92 viewed because of the two similar but not identical images

Further information can be received from a remote site to the imaging device 10 if so desired This information may be o erlaid in 3-D so that the information is displayed while viewing the outside world or perhaps one channel views the outside world, while the other channel displays information for image recognition images For example, while bird watching, an exotic bird may be displayed It is conceivable the other channel can be uploading information from an remote server accessing a library of exotic birds, so that a match can be made and the bird's identity would be known m real time Or at a sporting event, while ^• 'e"'^ιrι2 αr athlete, caϊ time statistics of the athlete are being overlaid

It will be readily apparent to those skilled in the art that still further changes and modifications in the actual concepts descπbed herein can readily be made without departing from the spiπt and scope of the invention as defined by the following claims

Claims

WHAT IS CLAIMED :

1 A hand-held 3-D imaging system, composing

a pair of hand-held prism binoculars compπsing two refracting telescopes mounted on a single frame,

each of said refracting telescopes having a first optical system for viewing objects through the telescope along a first optical light path defined through an objective lens, a beam splitter, a pair of pπsms and an eyepiece,

a second optical system which converts one or more images from the split portion of the light beam into -^ electronic recoi d signal through the use of a solid-state imaging sensor system located outside of the first optical light path in a second optical light path defined through the objective lens, the beamsplitter and the solid-state imaging sensor system; and

a third optical which converts an electronic playback signal into one or more emitted images which are transmitted to the eyepiece along a segment of the first optical light path through the use of a solid-state imaging display emitter system located outside of the first optical light path in a third optical light path defined through the solid-state imaging display emitter system, the beamsplitter and the eyepiece; a record control device for activating the solid-state imaging sensor system in each of the refracting telescopes to record,

a playback control device for activ ating the solid-state imaging display emitter system to playback in each of the refracting telescopes,

electronic storage media,

electronic processing circuitry for processing the electronic record signal in each of the refracting telescopes and stonng it as electronic data in the electronic storage media,

electronic processing circuitry for retrieving electronic data stored in the electronic storage media and for processing it into the electronic playback signal which is sent to the solid-state imaging display emitter system in each of the refracting telescopes to create a stereoscopic image seen through the eyepiece of each of the two refracting telescopes; and

an intemal telemetry device for wireless receiving and transmitting of remote real-time signals to and from the device.

2. A wireless hand-held 3-D imaging system as recited in claim 1 , further enabling overlaying remote real-time stereoscopic images while stereoscopically viewing the outside world.

3. A wireless hand-held 3-D imaging system as recited in claim 1 , where remote real-time stereoscopic images can be recorded in 3-D within the device.

4. A wireless hand-held 3-D imaging system as recited in claim 1 , where remote real-time stereoscopic images overlaid with images seen through the device can

be recorded .in 3-D within the device.

5. A tireless hand-held 3-D imaging system as recited in claim 4, farther enabling 3-D internal or external playback of recorded images.

6. A wireless hand-held 3-D imaging system as recited in claim 1 , where 3-D images viewed through the device can be remotely transmitted to a node processor that has Internet connectivity.

7. A wireless hand-held 3-D imaging system as recited in claim 6, where the transmitted 3-D images can be remotely viewed in real-time and in 3-D to anyone logged on to the Web.

8. An imaging system, comprising: two refracting telescopes mounted on a single frame; an imaging sensor system that converts images received by the telescopes into digital image signals; and a transmitter coupled to the imaging sensor system that receives the digital image signals and transmits corresponding signals via wireless transmission.

9. The system of claim 8 wherein the digital image signals are digital stereo image signals.

10. The system of claim 8 wherein the images received by the telescopes are converted to digital signals and transmitted in real-time.

11. The system of claim 8 wherein the imaging sensor system comprises: a photoarray system that produces video signals corresponding to the received images; an analog-to-digital converter system coupled to the photoarray system that converts the video signals to digital video signals; and a processor coupled to the analog-to-digital converter system that receives the digital video signals and generates the digital image signals that are input to the transmitter.

12. The system of claim 11 wherein the digital image signals are digital stereo image signals and the processor generates two digital stereo image signals corresponding to the images received by each of the two telescopes, respectively, and wherein the imaging sensor system further comprises: genlock and synchronization circuitry coupled to the photoarray system and the processor; and a stereo video multiplexer coupled to the processor that combines the two digital stereo image signals into a single digital stereo image signal.

13. The system of claim 8 wherein the transmitter comprises: a telemetry device that modulates the digital image signals for wireless transmission; and an antenna coupled to the telemetry device.

14. The system of claim 8 further comprising a remote computer wherein the transmitted signals can be remotely viewed on the remote computer.

15. The system of claim 8 further comprising: a node processor; and a remote computer that can be coupled to the node processor; wherein the node processor receives the transmitted signals and sends corresponding received signals to the remote computer.

16. The system of claim 15 wherein the node processor and remote computer have Internet connectivity and the node processor sends the corresponding received signals to the remote computer via the Internet.

17. The system of claim 16 wherein the remote computer receives the corresponding received signals soon after the images are received by the telescopes.

18. The system of claim 15 wherein the digital image signals are digital stereo image signals and the remote computer comprises a display device for displaying images represented by the corresponding received signals; the system further comprising: a 3-D viewing device enabling users to view a three-dimensional representation of the displayed images.

19. The system of claim 11 further comprising: a memory device coupled to the processor for storing information corresponding to the digital image signals.

20. The system of claim 19 wherein the memory device is a video RAM.

21. The system of claim 11 wherein the digital image signals are compressed by the processor.

22. The system of claim 11 wherein the processor produces digital image signals corresponding to a stabilized representation of the received images.

23. The system of claim 11 further comprising: a microcontroller coupled to the imaging sensor system that enables and disables the imaging sensor system.

24. The system of claim 8 wherein the frame is capable of being hand held.

25. An imaging system, comprising: two refracting telescopes mounted on a single frame; each telescope comprising an objective lens and an eyepiece; an image playback system viewable through the eyepiece of each telescope; and a wireless receiver coupled to the image playback system that receives wireless signals representing images and sends corresponding signals to the image playback system.

26. The system of claim 25 wherein the received wireless signals represent stereo images.

27. The system of claim 25 wherein the image playback system comprises: a display system; and a processor coupled to the display system, the processor receiving the signals from the wireless receiver and generating display signals that cause the display system to display corresponding images.

28. The system of claim 25 wherein the display system comprises two displays; one display visible through the eyepiece of one of the two refracting telescopes and the other display visible through the eyepiece of the other of the two refracting telescopes.

29. The system of claim 28 wherein the received wireless signals represent stereo images and the processor generates two display signals, one for each display, and wherein the image playback system further comprises: a stereo video demultiplexer coupled to the processor that demultiplexes the signals received by the receiver into two signal streams each corresponding to different images.

30. The system of claim 25 wherein the wireless receiver comprises: an antenna; and a telemetry device that demodulates the wireless signals received by the antenna and sends them to the image playback system.

31. The system of claim 25 further comprising a remote computer wherein the remote computer is capable of causing wireless signals representing images to be transmitted.

32. The system of claim 25 further comprising: a node processor; and a remote computer that can be coupled to the node processor; wherein the remote computer sends signals to the node processor representing images and the node processor transmits corresponding wireless signals to be received by the wireless receiver.

33. The system of claim 32 wherein the node processor and remote computer have Internet connectivity and the remote computer sends the signals representing images to the node processor via the Internet.

34. The system of claim 27 further comprising: a memory device coupled to the processor for storing information corresponding to the signals from the wireless receiver.

35. The system of claim 34 wherein the memory device is a video RAM.

36. The system of claim 27 further comprising: a microcontroller coupled to the image playback system that enables and disables the image playback system.

37. The system of claim 25 wherein images from the image playback system and images received from the telescopes can be viewed simultaneously.

38. The system of claim 25 further comprising: an imaging sensor system that converts images received by the telescopes into digital image signals; and a transmitter coupled to the imaging sensor system that receives the digital image signals and transmits corresponding signals via wireless transmission.

39. The system of claim 25 wherein the frame is capable of being hand held.

40. A method of transmitting images comprising the steps of: receiving images in a binocular comprising two refracting telescopes mounted on a single frame; converting the received images into digital image signals; and transmitting the digital image signals via wireless transmission.

41. The method of claim 40 wherein the digital image signals are digital stereo image signals.

42. The method of claim 40 wherein the step of converting occurs soon after the step of receiving.

43. The method of claim 42 wherein the step of transmitting occurs soon after the step of converting.

44. The method of claim 41 wherein the step of converting comprises the steps of: producing video signals corresponding to the received images; generating digital video signals corresponding to the video signals produced in the previous step; and generating digital stereo image signals corresponding to the digital video signals.

45. The method of claim of claim 41 wherein the step of converting comprises: converting the received images from one of the two refracting telescopes into first digital stereo image signals; converting the received images from the other of the two refracting telescopes into second digital stereo image signals; and combining the first and second digital stereo image signals.

46. The method of claim 40 wherein the step of transmitting comprises modulating the digital image signals.

47. The method of claim 40 further comprising the step of viewing the transmitted digital image signals at a remote computer.

48. The method of claim 40 further comprising the steps of: receiving the transmitted digital image signals at a node processor; and sending signals corresponding to the received signals to a remote computer.

49. The method of 48 wherein the step of sending comprises sending the received signals to the remote computer via the Internet.

50. The method of claim 40 further comprising the step of storing information corresponding to the digital image signals in a memory device.

51. The method of claim 40 wherein the binocular is capable of being hand held.

52. A method of displaying images comprising the steps of: receiving wireless signals representing images; and displaying images corresponding to the received wireless signals on a display system within a binocular and viewable through the binocular's eyepieces.

53. The method of claim 52 wherein the wireless signals represent digital stereo images.

54. The method of claim 52 wherein the display system comprises two displays, one visible through one of the binocular's eyepieces and the other visible through the other of the binocular's eyepieces.

55. The method of claim 52 further comprising the steps of: sending signals representing images from a remote computer to a node processor; transmitting from the node processor via wireless transmission signals corresponding to the signals received from the remote computer.

56. The method of claim 55 wherein the step of sending comprises sending the signals representing images from the remote computer to the node processor via the Internet.

57. The method of claim 52 further comprising the step of: storing information corresponding to the received wireless signals in a memory device.

58. The method of claim 52 wherein the binocular comprises two refracting telescopes mounted on a single frame and further comprising the step of displaying images received by the telescopes, wherein the step of displaying images received by the telescopes occurs simultaneously with the step of displaying images corresponding to the received wireless signals.

59. The method of claim 52 wherein the binocular is capable of being hand held.