Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/60—Control of cameras or camera modules
  • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
  • H04N23/633—Control of cameras or camera modules by using electronic viewfinders for displaying additional information relating to control or operation of the camera
  • H04N23/635—Region indicators; Field of view indicators
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/60—Control of cameras or camera modules
  • H04N23/66—Remote control of cameras or camera parts, e.g. by remote control devices
  • H04N23/661—Transmitting camera control signals through networks, e.g. control via the Internet
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/60—Control of cameras or camera modules
  • H04N23/62—Control of parameters via user interfaces
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/60—Control of cameras or camera modules
  • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
  • H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/20—Special algorithmic details
  • G06T2207/20112—Image segmentation details
  • G06T2207/20132—Image cropping

Definitions

• the present invention relates to electronic imaging systems having a display for presenting images.
• Hand held and portable display devices are becoming increasingly popular communication tools for capturing, sharing and displaying images that are in digital form. Examples of such devices include digital cameras, hybrid/film electronic cameras, personal digital assistants, digital photo albums, so called e- books, CD and DND players and the like. Because many consumers prefer display devices that are relatively small, many display devices provide video displays that are also relatively small and therefore such displays have limited image resolution capacity. For example, the image resolving capability of the displays in some of the most popular hand held devices is on the order of 320 picture elements (referred to herein as "pixels”) by 320 pixels.
• pixels picture elements
• the down sampling process can form evaluation images that mask conditions such as red eye, closed eyes, subject motion, handshake and/or other lighting conditions that will be readily apparent when the archival image is presented by display having better resolution or when the archival image is printed.
• a display device such as a camera with a camera with a display that more effectively presents an image in a way that automatically assists a user in the process of reviewing a digital image.
• the entire evaluation image is formatted and presented on a video display so that the entire evaluation image can be viewed in the display.
• a second mode only a part of the evaluation image can be viewed on the display. Because the entire display or a substantial proportion thereof is used for presenting only a portion of the image, the portion of the evaluation image being viewed has a greater effective magnification than it has when that same portion is viewed in the first mode.
• the user can manually adjust what is displayed so that particular portions of the evaluation image can be viewed with increased magnification. However, this requires that a photographer manually selects the magnification mode and manually locates an area of importance in the image in order to make determinations.
• hand held image display devices are increasingly called upon to present more than one evaluation image at a time. For example a collection of images may be available for viewing using the display device.
• a menu of so-called thumbnail images that each use only a fraction of the image forming capabilities of the display. Where this is done, the extent of the down sampling is increased further masking conditions in an image that may make the image have a less than desirable appearance. Further, in certain circumstances, it can become difficult to discern what is depicted in the thumbnail image.
• the invention is a method for forming an evaluation image for presentation on a display.
• an image is obtained having a resolution greater than a resolution of the display and an area of importance automatically determined comprising less than all of the obtained image.
• Image elements from the area of importance are used to form an area of importance image adapted for presentation on the display; and the area of importance image is presented.
• the invention is a method for using a display having a predetermined image resolution to display an image have greater image resolution than the display.
• the image is obtained and an area of importance is determined in the obtained image.
• An area of importance image is formed containing image information from the area of importance resampled for presentation at the image resolution of the display and the area of importance image is presented on the display.
• Another embodiment of the invention is a method for using a display having a predetermined image resolution to display an image have greater image resolution than the display.
• the image is obtained and an area of importance is determined in the obtained image.
• An evaluation image is formed having an appearance that corresponds to the obtained image and is resampled for presentation on the display and the evaluation image is presented.
• An area of importance image is formed containing imaging information that corresponds to the area of importance resampled for presentation at the display resolution and presenting the area of importance image on the display when an area of importance verification mode is selected.
• Another embodiment of the invention is a method for presenting an area of importance in an image.
• an archival image is obtained and an evaluation image is formed corresponding to the appearance of the archival image and resampled for presentation on the display.
• An area of importance is determined in the archival image.
• the area of importance comprises less than all of the archival image.
• An evaluation image is displayed that corresponds to the archival image and an area of importance image is displayed that contains image content that corresponds to the determined area of importance.
• the image content of the area of importance as displayed in the area of importance image has a greater effective magnification than the area of importance has as displayed in an evaluation image that corresponds to the archival image.
• the display device has a source of archival images operable to obtain archival images; a display; and a processor adapted to determine an area of importance in an obtained archival image, and to form an area of importance image that contains less than all of the archival image including image information from the area of importance and that is adapted to be presented on the display and to cause the display to present the area of importance image.
• Still another embodiment of the invention is a display device.
• the display device has a source of an archival image; a display; and a signal processor.
• the signal processor receives the archival image and is capable of forming images for presentation on the display based upon the archival image.
• a controller is provided and is operable to cause the signal processor to form an evaluation image for presentation on the display and to form an area of importance image for presentation on the display.
• the evaluation image depicts what is shown in the archival image and the area of interest image depicts only a portion of the archival image that corresponds to the area of importance.
• FIG. 2 is an illustration of a back view of the display device of Fig 1;
• Fig. 3 is a flow diagram of one embodiment of a method of the present invention;
• Fig. 4 illustrates a photographic scene separated into rangefinding regions;
• Fig. 5 illustrates an area of importance displayed as part of an evaluation image;
• Fig. 6 illustrates a displayed area of importance image;
• Fig. 7 illustrates another embodiment of a method of the invention;
• Fig. 8 illustrates another embodiment of a method of the invention;
• Figs. 9a, 9b, and 9c each show an archival image;
• Fig. 10 shows a conventional thumbnail listing of archival images;
• Fig. 11 shows a thumbnail listing of area of importance thumbnail images;
• Fig. 1 shows a block diagram of an embodiment of a display device 10.
• Fig. 2 shows a back, elevation view of the display device 10 of Fig. 1.
• display device 10 takes the form of a digital camera 12 comprising a body 20 containing an image capture system 22 having a lens system 23, an image sensor 24, a signal processor 26, an optional display driver 28 and a display 30. In operation, light from a scene is focused by lens system 23 to form an image on image sensor 24.
• Lens system 23 can have one or more elements.
• Lens system 23 can be of a fixed focus type or can be manually or automatically adjustable. In the embodiment shown in Fig. 1, lens system 23 is automatically adjusted.
• Lens system 23 can be simple, such as having a single focal length with manual focusing or a fixed focus.
• taking lens unit 22 is a motorized 6x zoom lens unit in which a mobile element or elements (not shown) are driven, relative to a stationary element or elements (not shown) by lens driver 25.
• Lens driver 25 controls both the lens focal length and the lens focus position of lens system 23 and sets a lens focal length and/or position based upon signals from signal processor 26, an optional automatic range finder system 27, and/or controller 32.
• the focal length and/or focus position of lens system 23 can be automatically selected using a variety of known strategies.
• image sensor 24 is used to provide multi-spot autofocus using what is called the "through focus” or “whole way scanning” approach.
• the scene is divided into a grid of regions or spots, and the optimum focus distance is determined for each image region.
• the optimum focus distance for each region is determined by moving lens system 23 through a range of focus distance positions, from the near focus distance to the infinity position, while capturing images.
• between four and thirty-two images may need to be captured at different focus distances.
• capturing images at eight different distances provides suitable accuracy.
• the captured image data is then analyzed to determine the optimum focus distance for each image region.
• This analysis begins by band-pass filtering the sensor signal using one or more filters, as described in commonly assigned U.S. Patent No. 5,874,994 "Filter Employing Arithmetic Operations for an Electronic Synchronized Digital Camera” filed by Xie et al, on December 11, 1995, the disclosure of which is herein incorporated by reference.
• the absolute value of the bandpass filter output for each image region is then peak detected, in order to determine a focus value for that image region, at that focus distance.
• the optimum focus distances for each image region can be determined by selecting the captured focus distance that provides the maximum focus value, or by estimating an intermediate distance value, between the two measured captured focus distances which provided the two largest focus values, using various interpolation techniques.
• the lens focus distance to be used to capture a digital image can be determined using known algorithms.
• the image regions corresponding to a target object e.g. a person being photographed
• the focus position is then set to provide the best focus for these image regions. For example, an image of a scene can be divided into a plurality of sub-divisions.
• a focus evaluation value representative of the high frequency component contained in each subdivision of the image can be determined and the focus evaluation values can be used to determine object distances as described in commonly assigned U.S. Patent No. 5,877,809 entitled “Method Of Automatic Object Detection In An Image", filed by Omata et al. on October 15, 1996, the disclosure of which is herein incorporated by reference. If the target object is moving, object tracking may be performed, as described in commonly assigned U.S. Patent No. 6,067,114 entitled “Detecting Compositional Change in Image” filed by Omata et al. on October 26, 1996, the disclosure of which is herein incorporated by reference.
• the focus values determined by "whole way scanning” are used to set a rough focus position, which is refined using a fine focus mode, as described in commonly assigned U.S. Patent No. 5,715,483, entitled “Automatic Focusing Apparatus and Method", filed by Omata et al. on October 11, 1998, the disclosure of which is herein incorporated by reference.
• bandpass filtering and other calculations used to provide auto-focus information for digital camera 12 are performed by digital signal processor 26.
• digital camera 12 uses a specially adapted image sensor 24, as is shown in commonly assigned U.S. Patent No 5,668,597 entitled “An Electronic Camera With Rapid Automatic Focus Of An Image Upon A Progressive Scan Image Sensor", filed by Parulski et al.
• digital camera 12 uses a separate optical or other type (e.g. ultrasonic) of rangefinder 27 to identify the subject of the image and to select a focus position for lens system 23 that is appropriate for the distance to the subject.
• Rangefinder 27 can operate lens driver 25, directly or as shown in Fig.
• lens driver 25 automatically adjusts the position of one or more mobile elements (not shown) relative to one or more stationary elements (not shown) of lens system 23 based upon signals from signal processor 26, an automatic range finder system 27, and/or controller 32 to provide a zoom magnification.
• Lens system 23 can be of a fixed magnification, manually adjustable and/or can employ other known arrangements for providing an adjustable zoom.
• Light from the scene that is focused by lens system 23 onto image sensor 24 is converted into image signals representing an image of the scene.
• Image sensor 24 can comprise a charge couple device (CCD), a complimentary metal oxide sensor (CMOS), or any other electronic image sensor known to those of ordinary skill in the art.
• the image signals can be in digital or analog form.
• Signal processor 26 receives image signals from image sensor 24 and transforms the image signals into an image in the form of digital data.
• the digital image can comprise one or more still images, multiple still images and/or a stream of apparently moving images such as a video segment.
• the digital image data can comprise image data stored in an interleaved or interlaced image form, a sequence of still images, and/or other forms known to those of skill in the art of digital video.
• Signal processor 26 can apply various image processing algorithms to the image signals when forming a digital image. These can include but are not limited to color and exposure balancing, interpolation and compression. Where the image signals are in the form of analog signals, signal processor 26 also converts these analog signals into a digital form. Controller 32 controls the operation the display device 10 during imaging operations, including but not limited to image capture system 22, display 30 and memory such as memory 40.
• Controller 32 causes image sensor 24, signal processor 26, display 30 and memory 40 to capture and store archival images in response to signals received from a user input system 34, data from signal processor 26 and data received from optional sensors 36.
• Controller 32 can comprise a microprocessor such as a programmable general purpose microprocessor, a dedicated micro-processor or micro-controller, a combination of discrete components or any other system that can be used to control operation of display device 10.
• Controller 32 cooperates with a user input system 34 to allow display device 10 to interact with a user.
• User input system 34 can comprise any form of transducer or other device capable of receiving an input from a user and converting this input into a form that can be used by controller 32 in operating display device 10.
• user input system 34 can comprise a touch screen input, a touch pad input, a 4-way switch, a 6-way switch, an 8-way switch, a stylus system, a trackball system, a joystick system, a voice recognition system, a gesture recognition system or other such systems.
• user input system 34 includes a shutter trigger button 60 that sends a trigger signal to controller 32 indicating a desire to capture an image.
• shutter trigger button 60 that sends a trigger signal to controller 32 indicating a desire to capture an image.
• user input system 34 also includes a wide-angle zoom button 62, and a tele-zoom button 64 that cooperate with controller 32 to control the zoom settings of lens system 23 causing lens system 23 to zoom out when wide angle zoom button 62 is depressed and to zoom in when tele-zoom button 64 is depressed.
• Wide-angle zoom lens button 62 and telephoto zoom button 64 can also be used to provide signals that cause signal processor 26 to process image signal so that the digital image formed thereby appears to have been captured at a different zoom setting than that actually provided by the optical lens system. This can be done by using a subset of the image signals from image sensor 24 and interpolating the subset of the image signals to form the digital image.
• User input system 34 can also include other buttons including joystick 66 shown in Fig.
• Sensors 36 are optional and can include light sensors and other sensors known in the art that can be used to detect conditions in the environment surrounding display device 10 and to convert this information into a form that can be used by controller 32 in governing operation of display device 10. Sensors 36 can also include biometric sensors adapted to detect characteristics of a user for security and affective imaging purposes. Controller 32 causes an image signal and corresponding digital image to be formed when a trigger condition is detected. Typically, the trigger condition occurs when a user depresses shutter trigger button 60, however, controller 32 can determine that a trigger condition exists at a particular time, or at a particular time after shutter trigger button 60 is depressed.
• controller 32 can determine that a trigger condition exists when optional sensors 36 detect certain environmental conditions. Controller 32 can also be used to generate metadata in association with each image. Metadata is data that is related to a digital image or a portion of a digital image but that is not necessarily observable in the image data itself. In this regard, controller 32 can receive signals from signal processor 26, camera user input system 34 and other sensors 36 and, optionally, generates metadata based upon such signals.
• the metadata can include but is not limited to information such as the time, date and location that the archival image was captured, the type of image sensor 24, mode setting information, integration time information, taking lens unit setting information that characterizes the process used to capture the archival image and processes, methods and algorithms used by display device 10 to form the archival image.
• the metadata can also include but is not limited to any other information determined by controller 32 or stored in any memory in display device 10 such as information that identifies display device 10, and/or instructions for rendering or otherwise processing the digital image with which the metadata is associated.
• the metadata can also comprise an instruction to incorporate a particular message into digital image when presented. Such a message can be a text message to be rendered when the digital image is presented or rendered.
• the metadata can also include audio signals.
• the metadata can further include digital image data.
• the metadata can also include any other information entered into display device 10.
• the digital images and optional metadata can be stored in a compressed form.
• the still images can be stored in a compressed form such as by using the JPEG (Joint Photographic Experts Group) ISO 10918-1 (ITU - T.81) standard.
• This JPEG compressed image data is stored using the so-called "Exif image format defined in the Exchangeable Image File Format version 2.2 published by the Japan Electronics and Information Technology Industries Association JEITA CP-3451.
• other compression systems such as the MPEG-4 (Motion Pictures Export Group) or Apple QuickTime TM standard can be used to store digital image data in a video form.
• Other image compression and storage forms can be used.
• the digital images and metadata can be stored in a memory such as memory 40.
• Memory 40 can include conventional memory devices including solid state, magnetic, optical or other data storage devices. Memory 40 can be fixed within display device 10 or it can be removable. In the embodiment of Fig. 1, display device 10 is shown having a memory card slot 46 that holds a removable memory 48 such as a removable memory card and has a removable memory interface 50 for communicating with removable memory 48. The digital images and metadata can also be stored in a remote memory system 52 that is external to display device 10 such as a personal computer, computer network or other imaging system. In the embodiment shown in Figs. 1 and 2, display device 10 has a communication module 54 for communicating with the remote memory system.
• the communication module 54 can be for example, an optical, radio frequency or other transducer that converts image and other data into a form that can be conveyed to the remote display device by way of an optical signal, radio frequency signal or other form of signal.
• Communication module 54 can also be used to receive a digital image and other information from a host computer or network (not shown).
• Controller 32 can also receive information and instructions from signals received by communication module 54 including but not limited to, signals from a remote control device (not shown) such as a remote trigger button (not shown) and can operate display device 10 in accordance with such signals.
• Signal processor 26 and/or controller 32 also use image signals or the digital images to form evaluation images which have an appearance that corresponds to archival images stored in display device 10 and are adapted for presentation on display 30.
• Display 30 can comprise, for example, a color liquid crystal display (LCD), organic light emitting display (OLED) also known as an organic electroluminescent display (OELD) or other type of video display.
• Display 30 can be external as is shown in Fig. 2, or it can be internal for example used in a viewfmder system 38.
• display device 10 can have more than one display 30 with, for example, one being external and one internal.
• Signal processor 26 and/or controller 32 can also cooperate to generate other images such as text, graphics, icons and other infonnation for presentation on display 30 that can allow interactive communication between controller 32 and a user of display device 10, with display 30 providing information to the user of display device 10 and the user of display device 10 using user input system 34 to interactively provide information to display device 10.
• Display device 10 can also have other displays such as a segmented LCD or LED display (not shown) which can also permit signal processor 26 and/or controller 32 to provide information to user 10. This capability is used for a variety of purposes such as establishing modes of operation, entering control settings, user preferences, and providing warnings and instructions to a user of display device 10.
• controller 32 causes signal processor 26 to cooperate with image sensor 24 to capture preview digital images during composition and to present a corresponding evaluation images on display 30.
• controller 32 enters the image composition process when shutter trigger button 60 is moved to a half depression position.
• Any component of user input system 34 can be used for this purpose; for example, the "mode” button 67 or the "select-it” button 68 shown in Fig. 2 can be depressed by a user of display device 10, and can be interpreted by controller 32 as an instruction to enter the composition process.
• the evaluation images presented during composition can help a user to compose the scene for the capture of an archival image.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Human Computer Interaction (AREA)
• Studio Devices (AREA)
• Controls And Circuits For Display Device (AREA)
• Indication In Cameras, And Counting Of Exposures (AREA)
• Image Processing (AREA)
• Editing Of Facsimile Originals (AREA)

Priority Applications (1)

Applications Claiming Priority (2)

Publications (1)

Family

ID=34679027

Family Applications (1)

Country Status (3)

Families Citing this family (61)

Citations (5)

Family Cites Families (65)

• 2003
  • 2003-12-23 US US10/745,006 patent/US20050134719A1/en not_active Abandoned
• 2004
  • 2004-12-16 WO PCT/US2004/042329 patent/WO2005064538A1/en not_active Ceased
  • 2004-12-16 JP JP2006547146A patent/JP2007523521A/ja active Pending

Patent Citations (5)

Also Published As

Similar Documents

Legal Events