WO2007105164A2

WO2007105164A2 - Method and system for adding object information to captured images

Info

Publication number: WO2007105164A2
Application number: PCT/IB2007/050810
Authority: WO
Inventors: Marcin Gramza
Original assignee: Koninklijke Philips Electronics, N.V.; U.S. Philips Corporation
Priority date: 2006-03-14
Filing date: 2007-03-12
Publication date: 2007-09-20
Also published as: WO2007105164A3

Abstract

A system and method (400) permits automatic recording of information associated with a captured image. The system includes a camera (200) and a data broadcast apparatus (300) located in the vicinity of an object (50) to be photographed. The camera (200) receives an instruction to capture an image including the object (50) and, in response thereto: captures the image including the object (50) during an exposure time period, TEXP; and during the exposure time period TEXP, receives a broadcast signal in a non-visible portion of the electromagnetic spectrum from the data broadcast apparatus (300), the signal including a frame of data regarding the object (50) broadcast at a frame frequency, FFRAME, where FFRAME > 1/ TEXP. The camera (200) stores the image the frame of data regarding the object (50) such that the data regarding the object (50) is coupled to the captured image including the object (50).

Description

METHOD AND SYSTEM FOR ADDING OBJECT INFORMATION TO CAPTURED IMAGES

This invention pertains to digital photography, and more particularly to a system and an associated method of adding information about an object to a captured image of the object.

Digital photography continues to grow in popularity as digital cameras have become ubiquitous. In addition to dedicated devices, digital cameras are now also integrated into mobile telephones, personal digital assistants (PDAs), binoculars, and a number of other devices.

Meanwhile, as the capacity of digital storage media increase, users can capture and store an ever increasing number of digital images with these digital cameras. A 1 GByte flash memory card can typically store more than 500 high resolution (4 Megapixel) images. And flash memory cards are now available which can store several gigabytes of data. With so much available capacity, users are capturing more images than ever before.

For example, a tourist may take more than 500 photos during a 2 week holiday trip without ever needing to upload the images from the camera's memory device (e.g., flash memory card) to a mass storage device (e.g., computer hard disc) or print the images on a printer.

Tourists typically take photos of famous objects, such as buildings, monuments, picturesque views, works of art, museum artifacts, etc. Very often famous objects serve as a background for photographing people. When only a few objects have been photographed, it is fairly easy to recognize them afterwards. However, when the number of photos taken by a user is very large (e.g., in the hundreds or even thousands), after some time a user of a digital camera can forget or become confused about the objects that she/he has photographed.

Therefore, when a photographer takes a photograph or captures a digital image at a well know site such as a sightseeing, tourist, or business location, the photographer may need to remember information related to the site which is the subject of the photograph taken, or the image captured. Additionally, the photographer may desire to record the information at the time that the photograph is taken or that the image is captured. Storage of information on a film, or in the memory of a digital camera, is well known. However, such storage of information is done in an ad hoc and often incomplete manner. The photographer must already know, or must look up, information about the subject of the photograph or digital image from a source that may not be related to, or in proximity to the camera at the time the photograph is taken or the image is captured. For example, the photographer may look up the information from a travel information book or from a published reference document. Information about the site which is the subject of the photograph or digital image is often important for business reasons such as record keeping, brochures and flyers, or for personal reasons such as enjoyment of photograph albums or transmission of digital images to friends and family.

Rodriguez et al. U.S. Patent Publication 2002/0076217, entitled "Methods and apparatus for automatic recording of photograph information into a digital camera or handheld computing device," describes a bidirectional device that may be located in the proximity of a photographic site that may communicate data with a two-way communication device associated with a camera. The two-way communication device initiates the data exchange by transmitting a signal to the bidirectional device. In response thereto, the bidirectional device transmits data to the two-way communication device.

However, this system suffers from several disadvantages. First, it requires that the camera include a transmitting device either built-in, or physically separate, adding to cost and power consumption. Second, since the bidirectional device transmits data only in response to signals it receives, and since many users may be photographing an object at the same time, the bidirectional device must operate with a complicated communication protocol to insure that requests from users do not collide with each other or other wise interfere with each other, and to insure that the bidirectional device can rapidly provide customized data for each user on separate communication links, again without interference or confusion at the two-way communication device. This adds to the cost and complexity of the bidirectional device. Third, Rodriguez et al. do not explain how such an arrangement is supposed to distinguish between objects that only have been focused upon by a camera, and those objects whose images are actually captured. Moreover, the bidirectional communication process described by Rodriquez et al. appears to be somewhat time consuming and requiring user interaction, which is contrary to desired characteristics of simplicity and speed. Accordingly, it would be desirable to provide an improved system and method of capturing object- specific data when capturing an image of an object, and associating that data with the corresponding image. It would also be desirable to provide such a system and method which can operate automatically without requiring user intervention. It would be further desirable to provide such a system and method that is simplified and able to accommodate many an almost unlimited number of user simultaneously capturing images of the same object. The present invention is directed to addressing one or more of the preceding concerns.

In one aspect of the invention, a method of recording information associated with a photograph comprises: receiving an instruction to capture an image including an object; and in response to the instruction: capturing an image including the object during an exposure time period, T_EXP; during the exposure time period T_EXP, receiving a broadcast signal in a non- visible portion of the electromagnetic spectrum from a transmitter located in a vicinity of the object, the signal including a complete frame of data regarding the object broadcast at a frame frequency, FFRAME, where FFRAME > 1/ TEXP; storing the image; and storing the complete frame of data regarding the object such that the data regarding the object is coupled to the captured image including the object.

In another aspect of the invention, an apparatus comprises: an image capturing device; a receiver adapted to receive a signal broadcast in the non-visible electromagnetic spectrum; at least one memory device operationally connected to the image capturing device and the receiver to receive and store images from the image capturing device and to store data from the receiver; and a control device operationally connected to the image capturing device and the receiver, the control device being adapted to provide one or more control signals to the image capturing device and the receiver, in response to which the image capturing device captures an image only during an exposure period, T_EXP, and the receiver decodes and stores a frame of data received from the broadcast signal only during the exposure period, T_Eχp.

In still another aspect of the invention, an apparatus comprises: memory adapted to store a frame of data regarding an object located in a vicinity of the apparatus; a broadcast transmitter adapted to generate a signal in a non- visible portion of the electromagnetic spectrum, the signal being continuously modulated with the frame of data; and an antenna adapted to broadcast the signal in the non- visible electromagnetic spectrum. Further and other aspects will become evident from the description to follow.

FIG. 1 is a diagram illustrating aspects of a system and method for capturing object- specific information about an object while capturing an image of the object; FIG. 2 is a functional block diagram of one embodiment of an apparatus for capturing object-specific information about an object while capturing an image of the object; and

FIG. 3 is a functional block diagram of one embodiment of an apparatus for broadcasting object- specific information regarding an object located in the vicinity of the apparatus; and

FIG. 4 is a flowchart illustrating one embodiment of a method of capturing object- specific information about an object while capturing an image of the object.

The present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which preferred embodiments of the invention are shown. This invention may, however, be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided as teaching examples of the invention.

FIG. 1 is a diagram illustrating aspects of a system and method for capturing object- specific information about an object while capturing an image of the object.

As shown in FIG. 1, a camera 200 is oriented to capture an image of a landmark 50. Camera 200 may have an adjustable focal length lens or other mechanism for adjusting the field of view, defined by the diagonal lines emanating from camera landmark 50 in FIG. 1. As shown in FIG. 1, camera 50 is adjusted such that landmark 50 falls within its field of view. Although the example of FIG. 1 illustrates capturing an image of a landmark (e.g., the Eiffel Tower), in principle it may be applied to any often-photographed object (e.g. a monument, an ocean, a work of art, a museum piece, etc.).

Meanwhile, a data broadcast apparatus 300 is located in the vicinity of landmark 50. Data broadcast apparatus 300 broadcasts a signal in the non-visible portion of the electromagnetic spectrum, for example at VHF, UHF, or microwave frequencies.

Beneficially, when landmark 50 is in the field of view of camera 200, the broadcast signal may be received at the location of camera 200. The broadcast signal from data broadcast apparatus 300 is a modulated with data regarding landmark 50. The data may include: text describing landmark 50; a small, compressed, image or icon of landmark 50; a uniform resource locator of an Internet address for a server containing additional information about landmark 50; etc.

In particular, data broadcast apparatus 300 continuously broadcasts a frame of data pertaining to landmark 50 at a very high frame rate. The frame rate or frequency, F_FRAME, is selected such that the entire frame of data may be transmitted within a time required by most cameras to capture an image of landmark 50. For example, if a required exposure time, T_EXp, is 1/500 second (2 ms), then the entire frame of data is transmitted in less than 2 ms such that F_FRAME > 1/ T_EXP = 500 frames/sec. Now, if, for example, a frame of data contains 1000 bytes, then the data rate must be 4 Mbps.

Meanwhile, camera 200 includes a receiver adapted to receive the signal broadcast by data broadcast apparatus 300 and to decode the signal to recover the frame of data regarding landmark 50. Since F_FRAME > 1/ T_EXP, the entire frame of data can be received by camera 200 during a time period when it is capturing the image of landmark 50. This eliminates any ambiguity about when data is to be captured and whether or not any received data corresponds to any image that was actually captured by camera 200. It should be noted that there is no requirement that the broadcast signal from data broadcast apparatus 300 be synchronized to operation of camera 200. So, it is possible that camera 200 will begin capturing the image and therefore start receiving the data in the middle of a broadcast frame. However, so long as F_FRAME > 1/ T_EXP then camera 200 will capture the entire frame of data while capturing the image of landmark 50. So, all that is required is that the data be properly rearranged by camera 200. This may be easily accomplished by providing a special start of frame (SOF) and/or end of frame (EOF) sequence in the signal broadcast by data broadcast apparatus 300. FIG. 2 is a functional block diagram of one embodiment of an apparatus, e.g. camera 200, for capturing object- specific information about an object while capturing an image of the object.

Camera 200 includes control block 210, image capture device 220, data receiver 230, and memory 240. Control block 210 may include a user interface for receiving instructions from a user, particularly an instruction to capture an image, and provides one or more control signals to image capture device 220 and data receiver 230. Beneficially, control block 210 includes a processor executing one or more software algorithms to implement various functions of camera 200.

Image capture device 220 operates in response to one or more control signals from control block 210. Beneficially, image capture device 220 includes an adjustable focal length lens or other mechanism for adjusting the field of view of the camera 200. Image capture device 220 may include a charge couple device (CCD), or any other convenient image capture arrangement.

Receiver 230 is adapted to receive a signal in the non- visible portion of the electromagnetic spectrum, for example at VHF, UHF, or microwave frequencies. In particular, receiver 230 is adapted to receive a signal on the same frequencies on which apparatus 300 transmits. Beneficially, receiver 230 includes a directional antenna on a surface of the body of camera 200 such that receiver 230 receives a signal only from a same direction as landmark 50 whose image that is being captured by camera 200. Receiver 230 decodes the received signal to obtain the frame of data about landmark 50 that is broadcast by apparatus 300. Memory 240 may include one or more separate memory devices. Beneficially, memory 240 comprises a single removable memory device, such as a compact flash card, a memory stick, a secure digital data card, etc. Also beneficially, memory 240 may include erasable, programmable non-volatile memory, such as Flash memory.

Operationally, when a user desires to capture an image including landmark 50, she/he aims the camera 200 in the direction of landmark 50 and adjusts the camera's field of view to include landmark 50. When everything is adjusted properly, the user depresses a button or other control device in camera 200 's user interface to instruct camera 200 to capture the desired image. In accordance with either manual settings, or an automatic mode of camera 200, an exposure period T_EXP is set for image capture device 220 to capture the image.

Accordingly, in response to one or more control signals from control unit 210, image capture device 220 captures the image including landmark 50 during exposure period T_EXP. Image capture device 220 stores the image in memory 240.

Meanwhile, in response to one or more signals from control unit 210, which may include one or all of the control signals supplied to image capture device 220, during time period T_EXP receiver 230 receives the broadcast signal from apparatus 300 and decodes the signal to recover the frame of data regarding landmark 50. Receiver 230 stores the data frame received during time period T_Eχp in memory 240. Beneficially, receiver 230 only stores data that is received during the same time period, T_EXP, that image capture device 200 actually captures the image of landmark 50.

Beneficially, the data is stored in such a way that the data regarding landmark 50 is coupled to the captured image including landmark 50. When we state that the data is coupled to the image file, we mean that an arrangement has been provided to automatically associate the data with the corresponding image when the image is downloaded, printed, or otherwise extracted from camera 200. Beneficially, the image file and corresponding data file may be coupled by having a same filename prefix. For example, an image file may be named IMG000001.jpg, and the corresponding data file may be named IMG000001.txt. Alternatively, the image file and corresponding data file may be coupled by the locations where they are stored in memory 240 (e.g., consecutive addresses). In another arrangement, a software algorithm may create a database in memory 240 associating the stored image with the stored data fro that image. Many arrangements are possible.

FIG. 3 is a functional block diagram of one embodiment of an apparatus, e.g., apparatus 300, for broadcasting object- specific information regarding an object located in the vicinity of the apparatus.

Data broadcast apparatus 300 includes Data I/O 310, memory 320, transmitter 330, and antenna 340.

Data I/O 310 includes one or more ports by which data regarding a nearby object may be provided to and stored in data broadcast apparatus 300. Beneficially, operating parameters (e.g.: transmission frequencies; hours of transmission; energy level; etc.) of data broadcast apparatus 300 may be entered into data broadcast apparatus 300 through Data I/O 310. For example, Data I/O 310 may include a port into which an external PDA or laptop computer may be plugged for downloading data to data broadcast apparatus 300 and/or programming operating parameters of data broadcast apparatus 300. Data I/O 310 may include a wireless adapter so that data and/or operating parameters may be communicated to data broadcast apparatus 300 remotely over a wireless connection, including over the Internet. Optionally, Data I/O 310 may be omitted, in which case data and operating parameters may be internally stored in memory 320 when apparatus 300 is manufactured.

Memory 320 is programmable and erasable, such as Flash memory. Memory 320 may include one or more separate memory devices. Memory 320 stores a frame of data regarding an object (e.g., landmark 50) located in the immediate vanity of data broadcast apparatus 300. Such data may include: text describing landmark 50; a small, compressed, image or icon of landmark 50; a uniform resource locator of an Internet address for a server containing additional information about landmark 50; etc. In one embodiment, several frames of data are stored in memory 320 in different languages. Memory 320 may also store data for various operating parameters of apparatus 300.

Beneficially, in one alternative embodiment, memory 320 comprises a removable memory device, such as a compact flash card, a memory stick, a secure digital data card, a USB memory device, etc. In that case, data regarding landmark 50 and/or various operating parameters of data broadcast apparatus 300 may be pre-programmed into memory 320 extraneous to data broadcast apparatus 300, and then the programmed memory 320 may be inserted into an appropriate connector provided in data broadcast apparatus 300. In that case, in contrast to the embodiment of FIG. 3, Data I/O 310 may connect memory 320 to transmitter 330.

Transmitter 330 is a broadcast transmitter that generates a signal in the non- visible portion of the electromagnetic spectrum, e.g., at VHF, UHF, or microwave frequencies. Beneficially, transmitter 330 includes a frequency generator generating a frequency carrier and a modulator for modulating the data onto the frequency carrier. Beneficially, transmitter 330 may generate a plurality of different signals corresponding to a plurality of different languages in which the data regarding landmark 50 are broadcast, where each signal includes a frame of data in a different language. Such signals may be frequency multiplexed onto different frequencies, code division multiplexed through spread spectrum techniques, or perhaps even multiplexed in time on dedicated time frames for each language.

Antenna 340 broadcasts the signal(s) from transmitter 330. In one embodiment, antenna 340 is a directional antenna that steers the signal(s) in a direction toward a location from which landmark 50 is commonly photographed. This location could be identified or marked with a plaque, marker, or other indicator to inform a user that data regarding landmark 50 may be wirelessly received at that location.

Operationally, transmitter 330 retrieves data regarding landmark 50 from memory 320 and formats it into a data frame, for example by adding a SOF and/or EOF sequence. Transmitter 330 modulates the data onto a desired frequency carrier, including any error correction code or other formatting that is desired, and provides the modulated signal to antenna 340. The data frame is continuously transmitted one frame after another. This process may be replicated for several signals carrying data frames in different languages. FIG. 4 is a flowchart illustrating one embodiment of a method of capturing object- specific information about an object while capturing an image of the object. The method 400 will be explained in the context of FIG. 1.

In a first step 410, camera 200 receives an instruction from a user to capture an image including landmark 50. For example, a user depresses a button on camera 200 to take a picture. In a next step 420, camera 200 captures an image including landmark 50 during an exposure period, T_Eχp.

Meanwhile, in a step 430, camera 200 receives a broadcast signal, including a frame of data about landmark 50, during the exposure period T_EXP, where F_FRAME > 1/ T_Eχp In a step 440, camera 200 stores the captured image including landmark 50. Also, in a step 450, camera 200 stores the frame of data received during the exposure period T_Eχp. Beneficially, as explained above, camera 200 only stores data received during the exposure period T_EXP when an image is captured, thereby automatically preventing any ambiguity regarding whether the data that is stored pertains to an image that was actually captured. The system and method described above have several desirable characteristics.

First, they do not require that camera 200 include any transmitting device, thus avoiding an increase in cost and power consumption. Second, since data broadcast apparatus 300 continuously broadcasts the data regarding landmark 50, many users may be photographing an object at the same time and all of theme will receive the data regarding landmark 50 without interference or confusion. This reduces to cost and complexity of data broadcast apparatus 300. Third, the arrangement described above automatically distinguishes between objects that have only been focused upon by camera 200, and those objects whose images are actually captured. Moreover, in the process described above the data is captured rapidly and without requiring any special user interaction. While preferred embodiments are disclosed herein, many variations are possible which remain within the concept and scope of the invention. Such variations would become clear to one of ordinary skill in the art after inspection of the specification, drawings and claims herein. The invention therefore is not to be restricted except within the spirit and scope of the appended claims.

Claims

CLAIMS:

1. A method (400) of recording information associated with a captured image, comprising: receiving (410) an instruction to capture an image including an object (50); and in response to the instruction: capturing (420) an image including the object (50) during an exposure time period, T_EXP; during the exposure time period T_Eχp, receiving (430) a broadcast signal in a non-visible portion of the electromagnetic spectrum from a transmitter (300) located in a vicinity of the object (50), the signal including a frame of data regarding the object broadcast at a frame frequency, F_FRAME, where F_FRAME > 1/ T_EXP; storing (440) the image; and storing (450) the frame of data regarding the object (50) such that the data regarding the object is coupled to the captured image including the object (50).

2. The method (400) of claim 1 , wherein the frame of data comprises text.

3. The method (400) of claim 1, wherein the frame of data comprises a uniform resource locator for an Internet address of a server containing additional information about the object (50).

4. The method (400) of claim 1 , wherein the frame of data includes an image file including an image of the object (50).

5. The method (400) of claim 1 , wherein the image and the data regarding the object (50) included in the image are stored in a same memory device.

6. The method (400) of claim 1 , wherein storing (450) the frame of data regarding the object such that the data regarding the object (50) is coupled to the captured image including the object (50) comprises assigned a same filename prefix to a stored data file having the frame of data and a stored image file having the image.

7. An apparatus (200), comprising: an image capturing device (220); a receiver (230) adapted to receive a signal broadcast in the non- visible electromagnetic spectrum; at least one memory device (240) operationally connected to the image capturing device (220) and the receiver (230) to receive and store images from the image capturing device (220) and to store data from the receiver (230); and a control device (210) operationally connected to the image capturing device (220) and the receiver (230), the control device (210) being adapted to provide one or more control signals to the image capturing device (220) and the receiver (230), in response to which the image capturing device (220) captures an image only during an exposure period, T_EXp, and the receiver (230) decodes and stores a frame of data received from the broadcast signal only during the exposure period, T_EXP.

8. The apparatus (200) of claim 7, wherein the stored data comprises text.

9. The apparatus (200) of claim 7, wherein the stored data comprises a uniform resource locator of an Internet address for a server containing additional information about an object (50) included in the image.

10. The apparatus (200) of claim 7, wherein the stored data includes an image file including an image of an object (50) included in the image.

11. The apparatus (200) of claim 7, wherein the image and the data are stored in a same memory device (240).

12. The apparatus (200) of claim 7, wherein the receiver (230) is adapted to receive the signal in the non-visible electromagnetic spectrum only from a same direction as the image is being captured by the image capturing device (220).

13. The apparatus (200) of claim 7, wherein the image and the data are assigned a same filename prefix when stored in the at least one memory device (240).

14. The apparatus (200) of claim 7, wherein the image and the data are stored in different memory devices (240).

15. An apparatus (300), comprising : memory (320) adapted to store a frame of data regarding an object (50) located in a vicinity of the apparatus (300); a broadcast transmitter (330) adapted to generate a signal in a non- visible portion of the electromagnetic spectrum, the signal being continuously modulated with the frame of data; and an antenna (340) adapted to broadcast the signal in the non-visible electromagnetic spectrum.

16. The apparatus (300) of claim 15, wherein the antenna is a directional antenna.

17. The apparatus (300) of claim 15, wherein the frame of data comprises text.

18. The apparatus (300) of claim 15, wherein the frame of data comprises a uniform resource locator for an Internet address of a server containing additional information about the object (50).

19. The apparatus (300) of claim 15, wherein the frame of data includes an image file including an image of the object (50).

20. The apparatus (300) of claim 15, wherein: the memory (320) is further adapted to store a second frame of data regarding the object (50) located in a vicinity of the apparatus (300), the second frame of data including text in a different language than the first frame of data; the broadcast transmitter (330) is further adapted to generate a second signal in the non- visible portion of the electromagnetic spectrum, the second signal being continuously modulated with the second frame of data; and the antenna (340) is further adapted to broadcast the second signal in the non- visible electromagnetic spectrum.