CA2393145A1

CA2393145A1 - Methods and apparatus for image manipulation

Info

Publication number: CA2393145A1
Application number: CA002393145A
Authority: CA
Inventors: Richard A. Murray; George K. Branner
Original assignee: Individual
Current assignee: Qualcomm Inc
Priority date: 1999-12-17
Filing date: 2000-12-14
Publication date: 2001-06-21
Also published as: WO2001045388A3; JP2004501522A; WO2001045388A2; EP1250800A2; AU2108801A; CN1409918A

Abstract

A digital camera is coupled to an Internet web-site for image storage and/or processing. The camera may transmit images to the web-site as they are acquired without storing acquired images in the camera itself. The transmission from the camera may be wireless. The digital camera operator ma y later retrieve acquired images from the web-site using a personal computer a nd web browser software.

Description

METHODS AND APPARATUS FOR IMAGE MANIPULATION
Background of the Invention Field of the Invention The invention relates to the acquisition, transmission, and processing of digital images.
Description of the Related Art Images have been acquired on various forms of light sensitive photographic film for well over one hundred years. Although a variety of advances in this technold~y have been made, the basic principles used to acquire and process such images has remained fundamentally unchanged for a long period of time.
More recently, electronic image acquisition devices have been developed.
Rather than relying on film, an array of pixels comprise light sensitive electronic elements. With these elements, digital data is generated which defines an acquired image in terms of one or more digital values associated with each of the pixels of the array.
Applications of these devices include scanners and digital cameras. Although these devices eliminate film and the chemical development process, they require a large amount of memory for storing image data. This increases the cost of these devices, and limits the number of images that can be acquired in a portable or hand-held device.
Summary of the Invention The invention comprises image manipluation methods and systems as well as devices for image capture and transmission such as digital cameras. In one embodiment, a method of image manipulation comprises acquiring one or more digital images with a hand-held digital camera; transmitting them to a network address substantially simultaneously with the acquiring, and storing the digital images in a database hosted by a network server associated with the network address. In another embodiment, a method of image manipulation comprises authorizing a portable image acquisition device operator to access a portion of an image database and receiving images from a portable image acquisition device operated by the portable image acquisition device operator automatically and substantially concurrently with image acquisition. The images may then be stored in the image database; and retrieved under the direction of the portable image acquisition device operator after verifying the authorization.
Image acquisition devices according to the invention may comprise image acquisition circuitry having digital image data as an output; a user actuated control for initiating image acquisition, a wireless transmitter, and logic circuitry coupled to the image acquisition circuitry, the user actuated control, and the wireless transmitter, wherein the logic circuitry is configured to route the digital image data to the wireless transmitter in response to the user actuated control. In another embodiment, a portable image acquisition device comprises image data acquisition circuitry, image data transmission circuitry; and a memory location having stored therein a network address of a network server to which image data is automatically forwarded.
Systems for image manipulation are also provided which may comprise a plurality of portable image acquisition devices, wherein each of the portable image acquisition devices comprises a wireless transmitter. This system may also comprise a computer system remote from the portable image acquisition devices and in WO 01/45388 PC'T/US00/34219 communication with the portable image acquisition devices by a communication link accessible via the wireless transmitters, wherein all of said plurality of portable image acquisition devices are configured to transmit images to the computer system substantially automatically as the images are acquired.
Brief Description of the Drawings FIG. 1 is a block diagram of an image manipulation system in accordance with one embodiment of the invention.
FIG. 2 is a flow chart of one mode of operation of an image manipulation system in one embodiment of the invention.
FIG. 3 is a block diagram of the components of an image acquisition device in accordance with one embodiment of the invention which is suitable for use in the system of FIG. 1.
FIG. 4 is a block diagram of the components of an image acquisition device in accordance with a second embodiment of the invention which is suitable for use in the system of FIG. 1.
FIG. 5 is a perspective view of the image acquisition device of FIG. 4.
Detailed Descriution of the Preferred Embodiment Embodiments of the invention will now be described with reference to the accompanying Figures, wherein like numerals refer to like elements throughout. The terminology used in the description presented herein is not intended to be interpreted in any limited or restrictive manner, simply because it is being utilized in conjunction with a detailed description of certain specific embodiments of the invention. Furthermore, embodiments of the invention may include several novel features, no single one of which is solely responsible for its desirable attributes or which is essential to practicing the inventions herein described.
Figure 1 illustrates a system for image manipulation which includes a plurality of portable image acquisition devices 12, 14, 16. The image acquisition devices 12, 14, 16 are devices that produce digital data defining an image.
The image is typically a two-dimensional digital representation of a document, thing, person, scenery, etc. or a combination of these things which is proximate to the image acquisition device.
In many embodiments, the image acquisition devices 12, 14, 16 are each independently operated by separate image acquisition device operators. As one specific example, the image acquisition devices may be separate digital cameras carried by separate vacationers or news reporters. As will be explained below, this is one especially advantageous application of the system of Figure 1. The image acquisition devices may also be digital scanners.
Generally speaking, digital cameras focus a field of view onto a two-dimensional pixel array using a lens system, operating essentially as a conventional camera, but with a light sensitive electronic pixel array instead of film. They are typically hand-held, lightweight, and convenient to carry. A scanner may also be light weight and portable, although they are usually larger and heavier than a digital camera. Scanners usually acquire an image by passing a one-dimensional array of pixels across a two dimensional transparent surface which has a document or other flat object held against the transparent surface. Both digital cameras and scanners are often owned, controlled, and operated by individuals.
.2.

In another embodiment of the invention, the image acquisition devices 12, 14, 16, comprise surveillance cameras. Surveillance cameras may also be lightweight and portable, although once installed, they are typically not moved frequently or carried by an individual on a regular basis. In this embodiment, each camera will likely not be independently operated by separate individuals, but will be part of a single monitoring installation.
The image acquisition devices 12, 14, 16 advantageously include communication circuitry for communicating to a network 18. The connection to the network 18 may in some embodiments comprise a wired connection, but in especially advantageous embodiments of the invention, and as illustrated in Figure 1, the communication circuitry comprises wireless communication circuitry for communicating to the network 18 via a wireless communication infrastructure. In this embodiment, the wireless communication infrastructure be cellular, satellite, digital PCS, or any other known or future developed wireless communication technology. In some embodiments, the image acquisition devices could comprise two separately housed components, one being hand-held, and the other stationary. In this case, the communication link to the network may comprise two parts, one link from the handheld portion to the stationary portion, and a second link from the stationary portion to the network 18. Preferably, one or both of these links is wireless, but this need not be the case in all embodiments of the invention.
The network 18 may be any type of electronically connected group of computers including, for instance, the following networks: Internet, Intranet, Local Area Networks (LAN) or Wide Area Networks (WANI. In addition, the connectivity within the network 18 may be, for example, remote modem, Ethernet (IEEE 802.31, Token Ring (IEEE
802.51, Fiber Distributed Datalink Interface (FDDIi, Asynchronous Transfer Mode (ATM), or any other communication protocol. The computing devices linked to the network 18 may be desktop, server, portable, hand-held, set-top, or any other desired type or configuration. In one advantageous embodiment, the network 18 is the global wide area computer network commonly referred to as "the Internet." This is an advantageous implementation when the image acquisition devices are independently owned and operated digital cameras or scanners. In a surveillance camera embodiment, the network 18 will usually be a private LAN.
Currently, many network communications within the global Internet are performed using the TCPIIP format.
Transmission Control Protocol (TCPI is a transport layer protocol used to provide a reliable, connection-oriented, transport layer link among computer systems. The network layer provides services to the transport layer. Using a two-way handshaking scheme, TCP provides the mechanism for establishing, maintaining, and terminating logical connections among computer systems. TCP transport layer uses IP as its network layer protocol. Internet Protocol (IPI is a network layer protocol used by many corporations, governments, and the Internet worldwide. IP network layer supports many personal, technical and business applications, such as electronic mail, electronic fund transfers, medical records processing and similar data transfers. 1P is a connectionless network layer protocol that performs addressing, routing and control functions for transmitting and receiving datagrams over a network.
Connected to the network 18 is a computer system 20 which receives image data from the image acquisition devices 12, 14, 16 via the network 18. The computer system 20 will typically be a network server which includes software for receiving image data and storing the image data in an associated image database. As such, the network server will be assigned a network address for identification during communications between elements of the network 18. Communications from other network connected devices such as the image acquisition devices 12, 14, 16, to the computer system 20 will include the network address of the computer system 20 as part of the communication so that the computer system 20 is identified as the recipient of the data. When the network 18 is the global Internet, the network address is an IP address which may, at least in part, route the image data to an e-mail account, a web-site, or other Internet tool. The computer system 20 may also host image processing software for arithmetically and logically manipulating the image data received from the plurality of image acquisition devices 12, 14, 16. This software may include pixel processing routines for equalization, color correction, or any of a wide variety of feature enhancement algorithms.
Also connected to the network 18 is another computer system 22. The computer system 22 will typically comprise a personal computer (PC), such as are available from International Business Machines, Compaq Computer Corporation, Sun Microsystems, and many others. These computer systems normally include at least one microprocessor, a display screen, a keyboard, and a storage medium such as a hard disk. A printer is also advantageously locally associated with the computer system 22. The computer system 22 includes software for communicating with the network server 20 via the network 18. When the network comprises the Internet, such software is commonly referred to as a web-browser. The e-mail account, web site, or other tool may be available to the web browser through a domain name (e.g. www.site.com) which maps to the IP
address of the network server 20.
The image acquisition devices, 12, 14, 16, the image storage and processing computer 20, and the image retrieval and printing computer 22 thus form a distributed image manipulation system which has its components connected through the network 18. This image manipulation system operates by initially transmitting image data acquired by the image acquisition devices 12, 14, 16 to the image storage and processing computer system 20 via the network 18, generally by routing the image data to a pre-defined network address stored in the image acquisition devices 12, 14, 16. After the image data is stored in the computer system 20, it may be retrieved by the image retrieval and printing computer system 22, where the images can be displayed andlor printed.
In one advantageous embodiment, the image acquisition devices transmit image data substantially as the image data is acquired, and perform only minimal data storage and image processing functions. This provides several significant advantages. One is that the image acquisition devices 12, 14, 16 are less expensive because a large amount of memory and computational power for image storage and processing need not be provided. In addition, image security is improved because the images are almost immediately stored on the image database associated with the image storage and processing computer system 20. This eliminates the risk of losing the image acquisition device or associated memory modules and thus losing a large number of acquired images at the same time. The wireless communication link connection to the network 18 is advantageous in these embodiments because the camera, scanner, etc. can be operated without being tied by wire or cable to any other computer entities. It can be carried around and used in a completely conventional manner as far as the user of the device is concerned.

Preferably, image data transfer is fast enough to be able to transmit image data at an average rate at least as fast as image data is acquired, although it will be appreciated that temporary differences in data acquisition and transmission rates can be tolerated with internal data buffering as described below. Accordingly, image data compression is generally advantageous prior to transmission to reduce the amount of data required to be sent with each image. In some embodiments, a sequence of images may be acquired by the image acquisition device in rapid succession, such as 1, 2, 10, or even 100 images per second. In these embodiments, the transmission data rate should be able to accommodate these image sequences. Image compression algorithms designed for motion pictures, such as the well known MPEG standard, may be used in these applications to provide for a high compression ratio.
Thus, in one advantageous embodiment, a central image repository is created for hundreds or thousands of individual digital camera users without any inconvenience or device handling complications to the users themselves.
Each camera is pre-programmed to download image data as images are acquired.
Acquired images may be retrieved at the user's convenience using their home computer. The users need never worry about having sufficient memory to acquire more images or worry about the whereabouts of disks, memory modules, or other types of storage devices. If desired, some cameras may be pre-programmed with image destinations other than the image storage and processing computer 20. For instance, some users may wish to program their cameras with their own e-mail account, personal web-site, or other Internet tool or location.
Referring now to Figure 2, one method of operation of a distributed image manipulation system is illustrated.
First, at block 28, an image acquisition device operator, an individual digital camera owner, for example, may be authorized to access a portion of an image database associated with the image storage and processing computer 20.
As the digital camera is used to acquire images, at block 30 the acquired images are downloaded automatically and substantially concurrently to the image storage and processing computer system 20, and at block 32 the images are stored there in the database portion accessible to the image acquisition device operator. As described above, the digital camera operator may therefore take an unlimited amount of pictures with the camera without purchasing, carrying, securing, or in any way dealing with the memory required for image storage. The risk of misplacing the images and loss by theft or the like is also eliminated.
Upon returning home, the camera operator's home personal computer may comprise the image retrieval and printing computer system 22. By entering a selected domain name to a web browser, the image storage and processing computer 20 and its associated image database are accessed. At block 34, by using a password or the like, the authorization to access the portion of the database in which the images are stored may be verified. After verification, the user may be allowed to retrieve the images previously acquired. The images may then be viewed, specific desired image processing algorithms may be performed under the direction of the operator, and the images may be printed with a printer associated with the computer system 22.
Figure 3 illustrates the components of one embodiment of an image acquisition device which may advantageously be used in the system described above and shown in Figure 1.
Referring now to this Figure, the image acquisition device 40 is controlled by a micro-controller 42 which includes control and processing logic. Suitable micro-controllers are commercially available from, for example, Motorola Corporation and a wide variety of other companies. The micro-controller 42 is coupled to image acquisition circuitry 44, a data buffer 46, a wireless transmitter 48, and a user control 50. The image acquisition circuitry will typically include a one or two-dimensional array of light sensitive elements, such as a CCD array or an array of photodiodes. In operation, the image acquisition circuitry 44 has digital image data as an output that is routed to the micro-controller 42. The micro-controller then routes the image data received from the image acquisition circuitry 44 to a wireless transmitter 48 for transmission to the network 18 of Figure 1. To reduce the amount of processing power required to be part of the image acquisition device 40, the micro-controller will generally not perform image analysis and processing such as equalization, color correction, or other logical and arithmetic pixel processing algorithms. In advantageous embodiments, the data manipulation performed by the micro-controller 42 andlor the wireless transmitter 48 is limited to data compression, data encryption, and packaging in appropriate format (such as TCPIIP) for reception and handling by the wireless link infrastructure and the network 18.
Advantageously, all of these functions may be initiated by a single simple user control which is used to initiate image acquisition. On a digital camera , for example, the shutter release button may function as a single push button control 50 which both initiates image acquisition as well as image transmission with a single actuation. In this embodiment, the beneficial aspects of the camera are made more user friendly, because the entire transmission process is automatic and transparent to the user of the device 40. As described briefly above, in this embodiment the image acquisition device is pre-programmed with the identification of the server 20 to which the data will be sent by storing the network address andlor other required data routing information in the image acquisition device. Generally this information is will be stored in EEPROM or other non-volatile program memory internal to or associated with the micro-controller 42 as part of the micro-controller programming that controls data acquisition and transmission.
Because image data may be generated and transmitted at different rates, and because image transmission may be slightly delayed, an image data buffer 46 may be provided as a low capacity temporary storage for image data. This is especially useful if the user control 50 is also used to initiate the establishment of the wireless link through local wireless communication infrastructure (e.g. cellular telephone service). In this case, acquired image data may need to be stored temporarily in the buffer while the link is established.
Even if the link is established prior to image acquisition, the rate of image data acquisition may be different from the rate of image data transmission, and so buffer storage 46 may also be needed during the process of transmission. To reduce device cost, it is preferable to minimize the capacity of this buffer, although of course buffer overflow during image acquisition and transmission should be avoided. In some embodiments, the buffer stores at most data for three images. More buffer memory may be required depending on the functionality of the image acquisition device.
The image acquisition device may also comprise a real-time clock 52 and a Global Positioning System (GPS) satellite signal receiver 54. Each of these devices 52, 54 may also be coupled to the micro-controller 42, and image acquisition time as well as the location of the device 40 when the image was acquired may be associated with the image data and transmitted to the computer system 20 as each image is transmitted. Status information may also be associated with the image data for each image transmitted from the device 40.
This information may include, for example, the ambient light level, battery charge level, andlor error conditions that might be present such as mechanical errors or buffer overflow.
In some embodiments, most notably the surveillance camera embodiment described above, the device 40 may also be coupled to a renewable energy source such as a solar panel (not shown). This allows for remote image acquisition without a wired power source or batteries which require periodic replacement.
In Figures 4 and 5, a second embodiment of an image acquisition device is illustrated. Operation of the device of Figure 4 is functionally similar to that described with reference to Figure 3, however, in this embodiment, the micro-controller 42 communicates with a removable circuit card 58 through a disengagable connector 59. In this embodiment, the wireless transmitter 48 is mounted on the removable card 58.
This is an advantageous configuration because many commercially available digital cameras include removable circuit cards on which is mounted the memory which stores acquired images. The circuit card 58 of Figure 4 may be compatible with these removable memory cards, but will transmit the acquired images, rather than store them in on-board memory.
An interface circuit 60 on the circuit card may mimic the interface of a standard memory card so that the micro-controller 42 operates in a manner identical to a conventional digital camera. On board control logic 62 which is pre-programmed with a network server destination for image data manages the interface circuit 60 and the buffer 46 to implement wireless transfer of the image data as described above.
Figure 5 illustrates a digital camera embodiment incorporating these features.
The circuit card 58, which may advantageously be in an industry standard PCMCIA format, fits into a receptacle 68 provided in the housing 70 of the camera. Electrical contacts 69 on one edge of the circuit card 58 couple to the connector in the camera housing 70. With this embodiment, conventional cameras which are normally used with removable memory cards may be used in the image manipulation system herein described.
The foregoing description details certain emb odiments of the invention. It will be appreciated, however, that no matter how detailed the foregoing appears in text, the invention can be practiced in many ways. As is also stated above, it should be noted that the use of particular terminology when describing certain features or aspects of the invention should not be taken to imply that the terminology is being re-defined herein to be restricted to including any specific characteristics of the features or aspects of the invention with which that terminology is associated. The scope of the invention should therefore be construed in accordance with the appended claims and any equivalents thereof.
7.

Claims

WHAT IS CLAIMED IS:

1. ~An image manipulation system comprising:
a plurality of portable image acquisition devices, wherein each of said portable image acquisition devices comprises a wireless transmitter; and a first computer system remote from said portable image acquisition devices and in communication with said portable image acquisition devices by a communication link accessible via said wireless transmitters, wherein all of said plurality of portable image acquisition devices are configured to transmit images to said first computer system substantially automatically as said images are acquired by said portable image acquisition devices.

2. ~The system of Claim 1, wherein said portable image acquisition devices comprise hand-held digital cameras.

3. ~The system of Claim 1, wherein said portable image acquisition devices comprise portable scanners.

4. ~The system of Claim 1, wherein said portable image acquisition devices comprise portable surveillance cameras.

5. ~The system of Claim 1, wherein said portable surveillance cameras comprise a solar panel.

6. ~The system of Claim 1, wherein said portable image acquisition devices contain an image data buffer.

7. ~The system of Claim 1, wherein said computer system is accessed by said portable image acquisition devices using a network address.

8. ~The system of Claim 7, wherein said computer system is coupled to a global Internet, and wherein said computer system is accessed via an IP address.

9. ~The system of Claim 8, wherein said portable digital cameras communicate said image data in TCP/IP format.

10. ~The system of Claim 1, additionally comprising a second computer system remote from said first computer system, wherein said second computer system comprises a printer, and wherein said first computer system and said second computer system are coupled by a communication link so that images may be transmitted to said second computer system for printing.

11. The system of Claim 10, wherein said first computer system comprises an Internet network server, and said second computer system comprises a personal computer system comprising web browser software.

12. The system of Claim 1, wherein said image acquisition device is configured to acquire and transmit image sequences taken at a rate of at least one image per second.

13. The system of Claim 12, wherein said image sequences are compressed prior to transmission.

14. ~A camera comprising:
image acquisition circuitry having digital image data as an output;
a user actuated control for initiating image acquisition;
a wireless transmitter;
logic circuitry coupled to said image acquisition circuitry, said user actuated control, and said wireless transmitter, wherein said logic circuitry is configured to route said digital image data to said wireless transmitter in response to said user actuated control.

15. ~The camera of Claim 14, additionally comprising an image buffer.

16. ~The camera of Claim 14, wherein said logic circuitry is configured to download said digital image data to a network address using said wireless transmitter.

17. The camera of Claim 16, additionally comprising a wireless receiver;
wherein said logic circuitry is configured to upload digital image data from a network address using said wireless receiver.

18. The camera of Claim 17, wherein said logic circuitry is configured to download image data to and upload image data from one or more IP addresses sharing the same domain name.

19. The camera of Claim 16, wherein said logic circuitry is configured to download said digital image data in raw, compressed, or encrypted form without performing an image processing algorithm on said image data.

20. The camera of Claim 14, wherein said wireless transmitter is mounted on a circuit card which is disconnectable from the remainder of the camera.

21. The camera of Claim 20, wherein said circuit card additionally comprises an image data buffer.

22. The camera of Claim 21, wherein said memory has a capacity that is insufficient to store more than three images.

23. The camera of Claim 16, wherein said circuit card is configured in an industry standard PCMCIA
format.

24. A method of digital image manipulation comprising:
acquiring one or more digital images with a hand-held digital camera;
transmitting said one or more digital images to a network address substantially simultaneously with said acquiring;
storing said one or more digital images in a database hosted by a network server associated with said network address.

25. The method of Claim 24, wherein said one or more digital images is transmitted to an e-mail address.

26. The method of Claim 24, wherein said one or more digital images is transmitted to an Internet web-site.

27. The method of Claim 24, additionally comprising performing image processing using software hosted by said network server.

28. The method of Claim 24, additionally comprising transmitting said one or more digital images from said network server to a computer system comprising a printer, and printing said one or more digital images.

29. The method of Claim 24, additionally comprising transmitting supplemental information regarding time, location, or status along with said one or more digital images.

30. A digital camera system comprising:
a housing;
image acquisition circuitry mounted inside said housing;
a receptacle formed in said housing;
a circuit card adapted for removable insertion into said receptacle, said circuit card comprising (1) contacts for receiving image data from said image acquisition circuitry and (2) a wireless transmitter for transmitting said image data to a second location.

31. The digital camera of Claim 30, additionally comprising:
a user actuated control for initiating image acquisition;
logic circuitry coupled to said user actuated control, wherein said logic circuitry is configured to route said image data to said wireless transmitter in response to said user actuated control.

32. A circuit card for receiving digital images from a digital camera and transmitting said digital images to a remote location, said circuit card comprising:
electrical contacts adapted to removably couple to a connector provided as part of said digital camera; and a wireless transmitter coupled to said electrical contacts, whereby image data from said digital camera can be transmitted to said wireless transmitter for transmission to said remote location.

33. The circuit card of Claim 32, wherein said circuit card is configured in a PCMCIA format.

34. A method of manipulating images comprising:
authorizing a portable image acquisition device operator to access a portion of an image database;
receiving images from a portable image acquisition device operated by said portable image acquisition device operator automatically and substantially concurrently with image acquisition by said portable image acquisition device;
storing said images in said portion of said image database; and retrieving said stored images under the direction of said portable image acquisition device operator after verifying said authorization.

35. The method of Claim 34, additionally comprising the step of processing said images under the direction of said portable image acquisition device operator after verifying said authorization.

36. The method of Claim 34, wherein said image database is associated with an Internet web-site.

37. The method of Claim 34, wherein said receiving is performed substantially simultaneously with acquiring said digital images by said portable image acquisition device.

38. The method of Claim 34, additionally comprising the step of using a wireless transmitter to transmit images from said portable image acquisition device.

39. A portable image acquisition device comprising:
image data acquisition circuitry;~
image data transmission circuitry; and a memory location having stored therein a network address of a network server to which image data is automatically forwarded.

40. The portable image acquisition device of Claim 39, wherein said image data transmission circuitry is wireless transmission circuitry.

41. The portable image acquisition device of Claim 39, wherein the memory location is non-volatile.