JP2000032322A - Image capture system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To automatically correct the direction of a digital image to be captured without the intervention of a user. SOLUTION: An image capture system 11 is provided with an image capture subsystem 21 for capturing the image of real scene. The image capture system 11 is provided with a direction sensor 22 as well. The direction sensor obtains and stores the direction data of the image capture ?subsystem with respect to a prescribed reference (such as a gravity direction, for example,) in the case of capturing the image through the image capture subsystem 21. The direction data are integrated into image data and stored. The linked image data and direction data are sent to a correction processor. Corresponding to the direction data, the correction processor corrects the direction of the image and outputs it onto a display.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] The present invention relates to image processing. In particular, the present invention relates to an image capture system having a direction sensor that enables automatic correction of the orientation of an image captured by the image capture system without user intervention.

[0002]

BACKGROUND OF THE INVENTION As is known, recent developments in digital imaging have enabled the development of digital cameras. Digital cameras offer the convenience and flexibility to capture, instantly view and store images. Using a digital camera, images can be quickly captured and imported into a variety of applications such as publishing and digital photography for immediate processing. For digital photography applications, images stored on a digital camera can be transferred to a digital data processing system or to a monitor or television for display. The data processing system can be a computer system such as a personal computer. Further, the computer system can process the image before displaying it. Images can also be printed by a printer connected to the computer system. The printer can be a single color printer or a multi-color printer.

[0003] One advantage of using digital cameras to capture images is the portability of digital cameras.
Digital cameras can be made the same size as conventional cameras. This allows the user to easily operate the camera when capturing images. Images can be captured with a continuous range of angles because the camera can be easily operated.

[0004] Another advantage of digital cameras is that images captured by the digital camera can be viewed immediately. In this case, a mini-display is typically provided in the digital camera, and after the user operates the camera to capture the digital image, the user can view or review the captured image on the mini-display. it can.

[0005] The captured image is stored in a digital camera in a predetermined direction. Typically, the "up" direction for each image stored in the camera is relative to the top of the camera. This means that each stored image is displayed only in the "up" direction above this camera. However, the natural "up" direction of the image is not related to the top of the camera, but to the viewer's eyes, viewing angle, and sky and ground in the picture. Thus, as the image is captured, the "up" direction of the camera top is suitable for displaying the image, unless the top of the camera rotates with respect to a natural reference direction, such as gravity or the ground. If the camera is tilted with respect to the ground reference, the content of the captured image will be displayed tilted. For example, if the camera is tilted sideways when recording an image of a person in a standing position, the person in the image will be displayed sideways. Fortunately, people are relatively good at compensating for slight rotation or tilt in the image. They can adjust their heads when viewing tilted or rotated images.
However, this compensation can be very difficult and cumbersome for the viewer, when viewing images rotated beyond tolerance, especially 90 ° or more from the “up” direction.

[0006] Another conventional solution for compensating for a tilted view is to rotate the display so that the image can be viewed in the appropriate direction and angle. This requires that the display be partially or fully rotated to compensate for the camera orientation when the image is captured. When the display is a large or fixed display such as a computer monitor or television, rotation of the display is typically not possible. In this case, the viewer must tilt his or her head to compensate for the tilted image. As an option, the images can be manually "corrected" in their natural "up" direction before being displayed. However, this manual work can be tedious and time consuming when processing large numbers of pictures.

[0007]

One feature of the present invention is that when an image capture system is capturing an image so that the captured image can be automatically corrected without user intervention. The purpose is to obtain the direction information of the image capture system.

Another feature of the present invention is to incorporate directional information into the captured image so that the captured image can be automatically corrected without user intervention.

A further feature of the present invention is that it allows for automatic correction of captured images without user intervention.

[0010]

SUMMARY OF THE INVENTION An image capture system includes an image capture subsystem that captures an image of a physical scene. The image capture system includes a direction sensor that determines direction data of the image capture subsystem with respect to a predetermined reference when the image capture subsystem captures an image. The direction data is used to correct the image so that the image can be automatically corrected without user intervention.

[0011] The imaging system includes an image capture system for capturing an image of a real scene. The image capture system includes (1) an image capture subsystem that captures an image, and (2) a direction sensor that determines and records direction data of the image capture subsystem with respect to a predetermined reference when the image capture subsystem captures an image. And The direction data is used to correct the image so that the image can be automatically corrected without user intervention. Further, the imaging system includes an image delivery system connected to the image capture system that automatically corrects the image according to the directional data without user intervention and delivers the captured and corrected image.

Other features and advantages of the present invention will become apparent from the following detailed description, which illustrates the invention by way of example with reference to the accompanying drawings.

[0013]

FIG. 1 shows an imaging system 10 embodying one embodiment of the present invention. As described in more detail below, the imaging system 10 includes an image capture system 11 and an image delivery system 12. Image capture system 11
Is used to capture an image, an image delivery system
12 is used to send out the captured image. The image sending function of the image sending system 12 displays the captured images, stores the captured images for future use,
And / or transmitting the captured image to an external device connected to the imaging system 10.

According to one embodiment of the present invention, the image capture system 11 further includes an image capture system 11 that can automatically correct captured images according to directional data without user intervention. When capturing an image of a real scene, it includes a direction sensor 22 (shown in FIG. 2) that determines and records direction data for the image capture system 11. As mentioned above, the image orientation data indicates the orientation of the image capture system 11 with respect to a predetermined reference (eg, gravity) when the image capture system 11 is capturing an image. Thus, the direction data of the captured image indicates the direction of the captured image. The orientation data can be used to automatically rotate and correct the captured image without user intervention.
Correction means rotating the captured image according to the directional data. Correcting a digital image may require recalculating each pixel value of the image.

Further, the directional data can be incorporated into or combined with the image before using it to correct the captured image. This is because the direction data is
Enables easy transmission with image data before image correction. Further, the incorporation of directional data into the image data allows the image capture system 11 to not require the data processing functions required for the correction process. Without implementing the data processing functions within the image capture system 11, the image capture system 11 can be made relatively small in size. The imaging system 10 is described in more detail below with reference to FIGS.

In one embodiment, imaging system 10 is a digital imaging system. In this case, the image capture system
11 can include either a digital image capture system for capturing a digital image of a real scene, or a conventional image capture system. If the image capture system 11 is a conventional image capture system (e.g., a conventional camera), the imaging system 10 includes an image capture system 11 and an image delivery system 12
In between, a digital converter (eg, a scanner) for converting the captured image to a digital image can be included. If image capture system 11 is a digital image capture system, system 11 can include a digital camera. In other embodiments, imaging system 10 can be another type of imaging system.

As described above, the imaging system 10 of FIG.
Includes an image capture system 11, and an image delivery system 12 connected to the image capture system 11. Two systems
The connection between 11 and 12 may be a wire or cable connection or via a network. As a choice,
The connection can be made through a storage device that can be inserted into both systems 11 and 12. The storage device for the connection can be, for example, a floppy disk, a rewritable optical disk, a non-volatile memory disk (such as a PCMCIA flash memory card), or a zip drive disk. Optionally, the storage device may be a cassette tape.

Image capture system 11 is used to capture images of real scenes. The real scene can be a portrait or a landscape. Further, the image capture system 11 determines and records orientation data of the captured image according to one embodiment of the present invention. The orientation data can be used to automatically correct the captured image if necessary. As described above, the direction data indicates whether the image capture system 11 is tilted with respect to a predetermined reference (eg, gravity) when capturing an image. FIG. 2 shows the image capture system 11 in more detail and will be described in more detail below.

Referring again to FIG. 1, the image transmission system 12
Is used to display, store, and transmit captured images. In one embodiment, the image capture system 11 and the image delivery system 12 are integrated together into an integrated unit. As an option, two systems 11 and 12
Are two physically separate systems. FIG.
Shows the image delivery system 12 in more detail and will be described in more detail below.

Referring to FIG. 2, the image capture system 11 comprises:
It includes an image capture subsystem 21, a direction sensor 22, an encoding processor 23, and a storage device 24. The image capturing subsystem 21 is used to capture an image of an external real scene. Image capture subsystem 21 may be embodied by any known image capture device or module. In one embodiment, image capture subsystem 21 is a digital camera.

In one embodiment, components 21 to 24 are all
Integrated together in a physically integrated unit. As an option, components 21 to 24 are not integrated together. Further, the image capture system 11 can include more or less components than components 21-24. For example, the image capture system 11 may not have the storage device 24.

The direction sensor 22 includes an image capturing subsystem 21
When is capturing an image, the direction data of the image capturing subsystem 21 is determined and recorded. Thus, when an image is captured by the image capture subsystem 21, directional data of the image is obtained. Captured image is a natural “above”
So that it can be displayed or indicated by location,
This orientation data can be used to automatically correct the captured image. As can be seen, the natural "up" position of the displayed image is related to the viewer's eyes, viewing angle, and sky and ground in the image.

The direction sensor 22 can be embodied by any known mechanical and / or electrical direction sensing means. For example, the direction sensor 22 can be a gravimetric sensor that includes a non-uniformly weighted disk with various resistive or capacitive encodings around the disk. This encoding can be easily calibrated and detected using simple electronics. More sophisticated sensors can include electronic components that are integrated with components made using micromachining techniques. Further, the direction sensor 22 can acquire the direction data in another known manner.

Once the image capture subsystem 21 has generated image data (or pixel data) for the captured image and the direction sensor 22 has generated direction data for the captured image, an encoding processor 23 is used. Thus, the two data can be combined or integrated. This function is very simple, so the encoding processor 23 can be embodied such that it performs only a very simple integration or combination function. FIG. 3 illustrates the process of combining image data and direction data performed by the encoding processor 23, and is described in further detail below. As an option, the encoding processor 23 may be powerful enough to handle image correction functions. This image correction function is also described in more detail below with respect to FIG.

FIG. 3 shows a process for incorporating directional data into image data. As can be seen from FIGS. 2 and 3, the process starts at step 31. At step 32, image data is received from the image capture subsystem 21 (shown in FIG. 2) to the encoding processor 23 (shown in FIG. 2). Step 33
Then, the encoding processor 23 receives the direction data from the direction sensor 22.

At step 34, the direction data (and the user rotation selection data) are combined or embedded with the image data. This can be done using any known means. For example, the combined image data format for the captured image may include an image data field for storing pixel data of the captured image, and a direction data field for storing direction data. . These fields are related together. Step 35 is performed to transmit the combined image data to the storage device 24. The process determines whether image data for another image has been received. If yes, the process returns to step 32. If not, the process ends at step 37. If the image stored in the storage device 24 is later requested, the encoding processor
23 retrieves the combined image data from storage device 24 for transmission.

As mentioned above, the image capture system 11
The storage device 24 may not be included. In this case, the combined image data from the encoding processor 23 is displayed on the display system.
Sent directly to 12 (shown in FIG. 1).

Referring to FIG. 4, the image sending system 12 includes:
It includes a straightening processor 41 and a display 42 connected to the straightening processor 41. The correction processor 41 is used to perform the automatic image correction process shown in the flowchart of FIG. Further, in one embodiment, the correction processor 41 may include a simple correction option. In this case, the automatic image straightening process automatically rotates or straightens the image according to the simple straightening option or directional data included or embedded in the image data. The corrected image is displayed on the display 42. The automatic image correction process is described in further detail below in connection with FIG.

Referring again to FIG. 4, the simple straightening option, when it is possible, causes the straightening processor 41 to provide one of a number of predetermined rotation angles (eg, 90 °, 180 °, and 270 °). Rotate the image according to If the option is selected by the user, the correction processor 41 rotates the image to one of the predetermined rotations that is closest to the direction measured for the image. The correction processor can encode the remaining angle (ie, the measured direction minus a predetermined rotation angle) and place it in the image data. Therefore, this function is not
Allows the image to be rotated to a substantially vertical position without interpolating the image data with respect to the direction data.
In this case, the interpolation is performed later and the remaining angle data can be used to completely correct the image.

Display 42 can be any known image display device or system. As an option, the display 42 can be a device or module other than a display. For example, the display 42 is a printer,
It may be a storage device for storing the corrected image data, or an interface circuit (for example, a network adapter or a modem) for connection to an external device or a network.

Further, the automatic image correction process need not necessarily be performed by the correction processor 41. 1
In one embodiment, the process is performed by the remediation processor 41. In another embodiment, the automatic straightening function is integrated into the display 42. In this case, the display 42
Performing the process, the image delivery system 12 does not include the correction processor 41. As an option, the automatic straightening process
For example, it is executed by the encoding processor 23 of FIG.
In this case, the image sending system 12 again uses the correction processor
Does not have 41.

Referring to FIG. 5, the automatic straightening process begins at step 50. As mentioned above, the process
Can be executed by the correction processor 41. As an option, the process may be performed by the encoding processor 23 of FIG. In this case, the correction processor 41 is not required, and the processor 23 functions similarly to the correction processor. Further, when the display 42 includes some data processing capability, the process is
Can be integrated. By way of example, the process illustrated in FIG. 5 will be described with reference to the remediation processor 41.

At step 51 of the process, the straightening processor
41 receives image data from the image capture system 11.
The received image data consists of pixels of the captured image.
Includes data and direction data. As described above, the direction data is embedded or encoded into the pixel data of the captured image. Step 52 is performed and the correction processor 41 determines whether the received image data does indeed include direction data. If not, no correction is needed for the image and the step
57 will be implemented. If yes, the process proceeds to step 53
Then, it is determined whether the simple correction option is selected.

If the simple correction option is selected at step 53, then at step 55, the correction processor 41 does not correct the image according to the direction data. Instead, the image is
One of three predetermined rotations (eg 90 °, 180 ° and 270 °)
And only gives the minimum remaining angle with respect to the orientation data of the image. This means that in step 55, the correction processor 41 determines which of the predetermined angles is closest to the direction data and selects that angle for rotation. As mentioned above, this rotation does not require interpolation of the image data to generate a rotated image. Step 56 is performed,
The image is rotated based on the selected angle, and the remaining angles are encoded and included in the image data. This is done using known image rotation and processing techniques. The process,
Move to step 57. In an alternative embodiment, the process does not include steps 55 and 56.

If it is determined at step 53 that the simple correction option has not been selected by the user, the process proceeds to step 54 where the correction processor 41 rotates or corrects the image according to the directional data. This is also
This is done using known image rotation and processing techniques. Further, the orientation of the image is set to zero and this data is
It is encoded and placed in the image data as new direction data for the image. The process proceeds to step 57.

At step 57, it is determined whether additional images have been received by the correction processor 41. If so, the process returns to step 51. Otherwise, the process ends at step 58.

In the above description, the present invention has been described with reference to specific embodiments thereof. However, one of ordinary skill in the art appreciates that various modifications and changes can be made without departing from the broad spirit of the invention. Accordingly, the description and drawings are not limiting and can be considered as illustrative.

The present invention includes the following embodiments by way of example.

(1) (A) An image capturing subsystem (21) for capturing an image of an actual scene, and (B) an image capturing subsystem (21)
A directional sensor (22) for obtaining directional data of the image capture subsystem (21) with respect to predetermined criteria when capturing an image, such that the image can be automatically corrected without user intervention. A direction sensor that corrects the image using the direction data.
1).

(2) The direction sensor (22) and the image capturing subsystem (21) are connected to the image capturing system (10, 1).
The image capture system (10, 11) according to (1) above, forming an integrated unit of 1).

(3) The integrated unit of the direction sensor (22) and the image capturing subsystem (21)
The image capture system (10, 11) according to (2) above, which is a digital camera that also includes a built-in display.

(4) The image capture system (10) according to (1), further comprising a correction processor (41) for correcting the image using the direction data or one of a number of predetermined rotations. , 11).

(5) The image capturing system (10, 11) according to (4), further including a processor (23) for incorporating the direction data into the image data.

(6) The image capturing system (10, 11) according to the above (4), further comprising a display (42) for displaying an automatically corrected image.

(7) The image capturing system (10, 11) according to (1), wherein the predetermined criterion is gravity.

(8) Further, an image delivery system (12) connected to an image capture system (11), which automatically corrects the image according to the direction data without user intervention, and sends out the captured and corrected image. The image capturing system (10, 11) according to the above (1), comprising:

(9) Further, the image transmission system (12)
The imaging system (10, 11) according to (8), further comprising a correction processor (41) that corrects the image using directional data or one of a number of predetermined rotations.

(10) Further, the image transmission system (12)
The display (4
The imaging system (10, 11) of (8) above, including 2).

[0049]

According to the present invention, the direction of a captured digital image can be corrected without user intervention.

[Brief description of the drawings]

FIG. 1 is a block diagram of an imaging system including an image capture system and an image delivery system, embodying one embodiment of the present invention.

FIG. 2 is a block diagram of an image capture system that includes a direction sensor that determines direction data of the image capture system when capturing an image, according to one embodiment of the present invention.

FIG. 3 is a flowchart illustrating a process for incorporating direction data into image data of a captured image.

FIG. 4 is a block diagram of an image delivery system that includes a correction processor that performs an automatic correction process that automatically corrects an image using directional data, in accordance with one embodiment of the present invention.

FIG. 5 is a flowchart showing an automatic straightening process.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 Imaging system 11 Image capture system 12 Image transmission system 21 Direction sensor 22 Image capture subsystem 23 Encoding processor 41 Correction processor 42 Display

Claims (1)

[Claims] 1. An image capture subsystem for capturing an image of a real scene, and a direction sensor for obtaining direction data of the image capture subsystem with respect to a predetermined reference when the image capture subsystem captures an image. An image capture system that uses the directional data to correct the image so that the image can be automatically corrected without user intervention.