WO2007075071A1

WO2007075071A1 - Method and processor for compensating image tilt and record media recored program therefor

Info

Publication number: WO2007075071A1
Application number: PCT/KR2006/005881
Authority: WO
Inventors: Richard Kim
Original assignee: Mtekvision Co., Ltd.
Priority date: 2005-12-29
Filing date: 2006-12-29
Publication date: 2007-07-05
Also published as: KR100686310B1

Abstract

A method, processor, and recorded medium for correcting image tilt. The method of correcting image tilt involves receiving tilt information sensed by the a sensor; outputting to a viewfÊnder, when previewing, a guide frame indicating absolute vertical/horizontal lines based on the tilt information; and correcting the tilt by cropping areas outside the guide frame from a captured original image and rotating the cropped image. With the present invention, a user can obtain an improved image by automatically correcting the tilt of a captured image.

Description

[DESCRIPTION]

[Invention Title]

METHOD AND PROCESSOR FOR COMPENSATING IMAGE TILT AND

RECORD MEDIA RECORED PROGRAM THEREFOR

[Background Art]

The present invention relates to a method, processor, and recorded medium for

compensating image tilt, more specifically to a method, processor, and recorded

medium that can achieve an improved image by automatically compensating the tilt

while capturing the image.

[Background Art]

The analog photographing of the past has been mostly replaced today by digital

image photography, thanks to high integration of image sensors and development of

image processing technologies, such as CMOS (Complementary Metal Oxide

Semiconductor) and CCD (Charge-Coupled Device).

Moreover, today's mobile communication terminals and portable terminals are

added with the function of photographing still images and moving pictures, using an

internal/external camera. When photographing with such a camera equipped in a portable phone, it is

common that the photographing is made while in transit, without an aid of any tripod,

often resulting in tilted images undesirably.

Unless the tilt is intended, the user usually stores the image in a computer

before making an image correction through a separate image editing program.

Such a process inconveniences the user, and thus Korean Patent Publication

2005-0052155 (HORIZONTAL AND VERTICAL OF ANGLE DISPLAY METHOD

FOR MOBILE STATION) discloses a method of displaying a base angle or a base

coordinate on a liquid crystal display to allow the user check for image tilt.

This publication helps the user photograph an image without tilt by allowing

the user check the horizontal/vertical angles of the image through the base angle or base

coordinate.

With the related art such as the above publication, however, adjusting the

vertical/horizontal angles through the base angle and base coordinate is quite difficult

with a mobile communication terminal, which is commonly held with one hand for snap

photography.

Although the user tries to align the angle through the base angle and base

coordinate, external factors (e.g. moving vehicle or vessel, strong wind, or shaking by

sudden photographing or unskilled operation) make it extremely difficult to succeed in

tilt-free photography. As tilted images are caused by a variety of factors, the related art still stores

and outputs tilted images and does not present a method of automatically correcting the

tilted image.

When the base angle is simply displayed, as in the related art, it is hard to

identify where to crop although the image tilt is later compensated by use of a separate

image editing program, making an accurate compensation of tilt difficult.

[Disclosure]

[Technical Problem]

To solve the above problems, the present invention provides a method,

processor, and recorded medium recording a program for automatically correcting a

tilted image in a portable camera device such as a mobile communication terminal.

The present invention provides a method, processor, and recorded medium

recording a program that can efficiently compensate image tilt.

The present invention provides a method, processor, and recorded medium

recording a program that can easily compensate image tilt by use of tilt information.

The present invention provides a method, processor, and recorded medium

recording a program that can allow the user to preview the tilt-compensated image.

Other problems that the present invention solves will become more apparent

through the following description. [Technical Solution]

To solve the above problems, an aspect of the present invention features a

method of correcting the tilt and/or a computer-readable recorded medium recording a

program executing the method.

According to an embodiment of the present invention, the method of correcting

image tilt in an image processing apparatus including a tilt sensor and a viewfinder

displaying a preview image has the steps of receiving tilt information sensed by the tilt

sensor; outputting to the viewfinder, when previewing, a guide frame indicating

absolute vertical/horizontal lines based on the tilt information; and correcting the tilt by

cropping areas outside the guide frame from a captured original image and rotating the

cropped image.

The outputting step can be done when a user presses a predetermined shortcut

key.

The outputting step can be done when the image processing apparatus is tilted

from the absolute vertical/horizontal lines by more than a predetermined angle.

The above correcting can be made either vertically or horizontally in

accordance with a user setting.

The method can also involve writing the tilt information of the captured image.

The method can also have the steps of storing the captured original image in a memory and determining whether a shooting mode of the image processing apparatus is

terminated. The correcting can be done by extracting the original image file stored in

the memory if the shooting mode is terminated.

The outputting step can output to the viewfinder direction information for

moving the image processing apparatus in order to make an alignment with the absolute

horizontal/vertical lines.

The correcting step can also involve treating the cropped areas with black

transparency when previewing.

The method can also delete from a memory an original image corresponding to

the tilt-corrected image and enlarge the tilt-corrected image to a size of the original

image.

Another aspect of the present invention features a processor for correcting

image tilt.

According to an embodiment of the present invention, the processor for

correcting image tilt has a tilt information receiving unit, which receives tilt information

detected by a tilt sensor, a guide frame outputting unit, which outputs to a viewfinder,

when previewing, a guide frame indicating absolute vertical/horizontal lines based on

the tilt information, and a tilt correcting unit, which crops areas outside the guide frame

from a captured original image and rotating the cropped image. [Description of Drawings]

FIG. 1 shows a block diagram of an image processing apparatus in accordance

with an embodiment of the present invention;

FIG. 2 shows modules of a tilt correction application in accordance with an

embodiment of the present invention;

FIG. 3 shows a flowchart of an image capture process in accordance with an

embodiment of the present invention;

FIG. 4 shows a flowchart of a tilt correction function activation process in

accordance with an embodiment of the present invention;

FIG. 5 shows a flowchart of a tilt correction process in accordance with an

embodiment of the present invention;

FIG. 6 shows an example of a guide frame output while previewing an image

in accordance with an embodiment of the present invention;

FIG. 7 shows an example of an actual captured image and a guide frame in

accordance with an embodiment of the present invention;

FIG. 8 shows an example of a tilt-corrected image in accordance with an

embodiment of the present invention; and

FIG. 9 shows an example of an image treated with black transparency while

previewing the image in accordance with an embodiment of the present invention. [Mode for Invention]

The above objects, features and advantages will become more apparent through

the below description with reference to the accompanying drawings.

Since there can be a variety of permutations and embodiments of the present

invention, certain embodiments will be illustrated and described with reference to the

accompanying drawings. This, however, is by no means to restrict the present invention

to certain embodiments, and shall be construed as including all permutations,

equivalents and substitutes covered by the spirit and scope of the present invention.

Throughout the drawings, similar elements are given similar reference numerals.

Throughout the description of the present invention, when describing a certain

technology is determined to evade the point of the present invention, the pertinent

detailed description will be omitted.

Terms such as "first" and "second" can be used in describing various elements,

but the above elements shall not be restricted to the above terms. The above terms are

used only to distinguish one element from the other. For instance, the first element can

be named the second element, and vice versa, without departing the scope of claims of

the present invention. The term "and/or" shall include the combination of a plurality of

listed items or any of the plurality of listed items.

When one element is described as being "connected" or "accessed" to another

element, it shall be construed as being connected or accessed to the other element directly but also as possibly having another element in between. On the other hand, if

one element is described as being "directly connected" or "directly accessed" to another

element, it shall be construed that there is no other element in between.

The terms used in the description are intended to describe certain embodiments

only, and shall by no means restrict the present invention. Unless clearly used otherwise,

expressions in the singular number include a plural meaning. In the present description,

an expression such as "comprising" or "consisting of is intended to designate a

characteristic, a number, a step, an operation, an element, a part or combinations thereof,

and shall not be construed to preclude any presence or possibility of one or more other

characteristics, numbers, steps, operations, elements, parts or combinations thereof.

Unless otherwise defined, all terms, including technical terms and scientific

terms, used herein have the same meaning as how they are generally understood by

those of ordinary skill in the art to which the invention pertains. Any term that is

defined in a general dictionary shall be construed to have the same meaning in the

context of the relevant art, and, unless otherwise defined explicitly, shall not be

interpreted to have an idealistic or excessively formalistic meaning.

Hereinafter, preferred embodiments will be described in detail with reference

to the accompanying drawings. Identical or corresponding elements will be given the

same reference numerals, regardless of the figure number, and any redundant

description of the identical or corresponding elements will not be repeated. Although the present invention can be usefully applied to portable terminals,

such as portable phones and PDAs, having a camera, the invention is by no means

restricted to these devices, and it shall be evident to anyone of ordinary skill in the art

that the method of correcting tilt in accordance with an embodiment of the present

invention can be applied to standalone cameras.

FIG. 1 is a block diagram of an image processing apparatus in accordance with

an embodiment of the present invention.

As shown in FIG. 1, the image processing apparatus in accordance with an

embodiment of the present invention can comprise an image sensor 100, an image

processing unit 102, a memory 104, a codec 106, a processor 108, a display unit 110, a

tilt sensor 112, and a user input unit 114.

The image sensor 100 has a pixel array, including a plurality of unit pixels, and

a circuit, such as ADC (analog-digital converter), which converts data collected from

pixels to digital data and outputs the digital data. The image sensor 100 outputs the data

collected from each pixel to the image processing unit 102 when previewing and image

capturing.

The image sensor 100 outputs RGB Bayer data, and the image processing unit

102 interpolates and gamma-corrects this Bayer data and converts the actual RGB color

to YCC data, which consists of luminance (Y component) and chrominance (Cb and Cr

components). The image processing unit 102 can be realized by means of an ISP (image

signal processor) or DSP (digital signal processor).

The image-processed data can be displayed on the display unit 110 after being

temporarily stored in the memory 104 when being previewed, and can be stored in the

memory 104 after being encoded (compressed) through the codec 106 when being

captured.

The processor 108 controls the above-described image processing unit 102,

memory 104, and codec 106.

Referred to as "geomagnetic sensor" also, the tilt sensor 112 senses the

magnetic field of the earth and provides tilt information of how much the image

processing apparatus is tilted relative to the absolute vertical/horizontal lines. As the tilt

sensor 112 is well known, its description will not be provided here.

The user input unit 114 is where the user information is inputted for setting the

configuration for image capture, and can include key pads.

According to an embodiment of the present invention, the image processing

apparatus corrects the tilt of an image captured by the user through the use of tilt

information detected by the tilt sensor 112.

The tilt correction in accordance with an embodiment of the present invention

can be made through a software process by use of a tilt correction application.

FIG. 2 illustrates modules in a tilt correction application in accordance with an embodiment of the present invention. The tilt correction application in accordance with

the present invention can include a guide frame outputting module 200, a tilt

information writing module 202, a tilt correcting module 204, and a data

post-processing module 206.

When the user selects an image shooting mode through the user input unit 114,

a preview image is outputted on a viewfinder (e.g. the display unit 110). At this time,

the guide frame outputting module 200 outputs a guide frame on the viewfinder with

reference to configuration information.

It is preferable that the guide frame be outputted selectively when the user

desires to correct the tilt. For example, it can be outputted for every image if the user set

the configuration to make tilt correction for every captured image. Alternately, if the

user set the configuration to make tilt correction only when a certain shortcut key is

pressed, the guide frame may be outputted when the user presses the shortcut key while

shooting an image.

More preferably, the guide frame in accordance with an embodiment of the

present invention can be outputted when the tilt of the image processing apparatus

exceeds a predetermined angle from the absolute vertical/horizontal lines (e.g. 10

degrees from the vertical/horizontal lines).

The guide frame outputted on the viewfinder maintains the tilt angle of 0 (zero)

degree (absolute vertical/horizontal angle) regardless of the motion of the image processing apparatus. The guide frame can maintain the vertical and horizontal angles in

accordance with the information detected by the tilt sensor 112.

FIG. 6 is an example of the guide frame, in accordance with an embodiment of

the present invention, outputted while previewing the image. FIG. 6 shows a

photographed subject 600, a viewfinder 602, and a guide frame 604 when the image

processing apparatus is tilted by 5 degrees in the horizontal direction and by 0 degree in

the vertical direction.

When the image processing apparatus is tilted horizontally by 5 degrees and

vertically by 0 degree, as shown in FIG. 6, the viewfinder 602 and the guide frame 604

do not coincide, and the user gets to capture the photographed subject by matching the

guide frame 604 to the viewfinder 602.

In the mean time, it is possible that the guide frame outputting module 200

provides direction information (e.g. an arrow), indicating which direction to move the

image processing apparatus to bring the apparatus back to the absolute

vertical/horizontal lines, when the image processing apparatus, as well as the guide

frame, is tilted.

As the present invention corrects the tilt by cropping areas outside the guide

frame, the black transparency module 202 in accordance with an embodiment of the

present invention calculates the viewfinder area (display area) and guide frame area to

process the difference area with black transparency. FIG. 9 is an example of a screen that is treated with black transparency during

the preview, in accordance with an embodiment of the present invention. The black

transparency module 202 in accordance with an embodiment of the present invention

can treat an area 700 to be cropped from the captured image with black transparency,

through which the user can easily identify the image remaining after the tilt correction.

As described above, however, it is inevitable that the captured image is tilted

due to rushed photographing or shaking by external factors when the image is captured

with a device such as a portable phone. The tilt information writing module 204 in

accordance with an embodiment of the present invention writes the tilt information of

the image when the user captures the image.

The tilt information can be used to correct the tilt of the image captured by the

image processing apparatus, but it can be also used as reference data when the original

image file is transferred to a computer and then the image is corrected with an image

editing program installed in the computer.

The tilt correcting module 206 corrects the tilt of the captured image, using the

tilt information written as described above.

The tilt correction process in accordance with an embodiment of the present

invention is carried out by cropping the outside of the guide frame from the original

image captured by the user and rotating the copped image.

Since the guide frame indicates the absolute horizontal/vertical lines, as described above, the tilt correcting module 206 in accordance with an embodiment of

the present invention selects the data inside the guide frame.

FIG. 7 is an example of the original captured image with the guide frame in

accordance with an embodiment of the present invention. As shown in FIG. 7, the tilt

correcting module 206 selects the image inside the guide frame 604 from the original

image 602.

Since the original image itself gets tilted if the data inside the guide frame is

selected, the tilt correcting module 206 rotates the cropped image to the left/right by

referencing the tilt information.

Although the original image is tilted, the user can ultimately obtain a

tilt-corrected image, as shown in FIG. 8, through the process described above.

Although the tilt correcting module 206 corrected the original image to the

absolute vertical/horizontal lines in the above description, it would be possible that the

user sets to correct the tilt with reference to the absolute horizontal or vertical line only.

Although tilt correction can be made at the same time as image capture,

automatically correcting the tilt can delay the data processing time over the general

shooting mode, and thus it is not preferable that the tilt is corrected in real time while

consecutively photographing.

Therefore, the tilt correcting module 206 in accordance with an embodiment of

the present invention can determine whether the user has finished shooting an image to start the tilt correcting process of the previously captured original image using the tilt

information if the shooting mode has been finished.

The data post-processing module 208 post-processes the data in accordance

with the user configuration after the tilt is corrected.

For example, since the corrected image may be smaller than the original image

size, the data post-processing module 208 can enlarge the tilt-corrected image to fit the

original image size.

Moreover, it may be preferable that the pre-corrected image be deleted in a

device such as a portable phone, which has a limited memory capacity. Therefore, the

data post-processing module 208 in accordance with an embodiment of the present

invention can delete the original file from the memory 104 after the tilt correction.

Although correcting the tilt through a software process by use of a tilt

correction application has been described, the tilt correction can be also made through a

hardware process by use of a processor, which includes: a tilt information, receiving

unit being connected to the tilt sensor and viewfinder and receiving the tilt information

sensed by the tilt sensor; a guide frame outputting unit, outputting the guide frame to the

viewfinder during the preview; and a tilt correcting unit, correcting the captured image

based on the tilt information.

FIG. 3 is a flowchart of an image capture process in accordance with an

embodiment of the present invention. Referring to FIG. 3, when the image processing apparatus enters a shooting

mode, in S 300, in response to a request by the user, the tilt correction application

determines whether to activate the tilt correction function, in S302, by making reference

to the user configuration.

The activation of the tilt correction function can be determined by whether the

user has set to correct the tilt of every captured image, whether the shortcut key for tilt

correction has been pressed, and whether the tilt sensed by the tilt sensor 112 has

exceed a predetermined angle.

If the tilt correction function is activated, the tilt correction application outputs

the guide frame to the viewfinder, in S304. When the guide frame is outputted, the

arrow, for aiding to maintain the absolute horizontal/vertical lines can be outputted, and

the black transparency treatment of the area to be cropped can be carried out.

If the user captures a photographic subject in S306, the tilt correction

application writes the tilt information of the correctly captured original image, in S308.

Using the tilt information, the tilt is corrected in S310, and the tilt-corrected

image is stored, in S312.

After the tilt-corrected image is stored, the data post-process is carried out, and

the data post-process enlarges the corrected image or deletes the original image from the

memory, in S314.

If the automatic tilt correction is not activated in S302, the image processing apparatus stores the original image with the general capture mode.

FIG. 4 is a flowchart of activating the tilt correction function in accordance

with an embodiment of the present invention. The flowchart shows the tilt correction

function being activated according to the angle of the image processing apparatus.

Referring to FIG. 4, the image processing apparatus receives the tilt

information detected by the tilt sensor, in S400. With the tilt information, the image

processing apparatus determines whether the apparatus is tilted from the absolute

vertical/horizontal lines by more than a predetermined angle, in S402. If the tilt exceeds

the predetermined angle, the process moves to S304, and, if not, moves to S316.

FIG. 5 is a flowchart of a tilt correction process in accordance with an

embodiment of the present invention.

FIG. 5 shows a detailed process of the tilt correction application. Referring to

FIG. 5, the tilt correction application decodes the original image file stored in the

memory, in S500.

The decoding step is described because it is assumed that a captured image is

usually compressed. However, if the captured image remains without being encoded,

the above decoding process may be skipped.

The tilt correction application checks the tilt information of the captured

original image, in S502, and crops the area outside the guide frame from the original

image, in S504. Since the cropped image, without the area outside the guide frame, is tilted, the

tilt correction application rotates the cropped image, using the tilt information, and

generates an image that coincides with the absolute vertical/horizontal lines, in S506.

The corrected image through the above steps is encoded and stored in the

memory, in S508.

Although capturing an image in a device such as a portable phone has been

described, it shall be evident to anyone of ordinary skill in the art that the present

invention can be also applied to a standalone camera by installing a tilt sensor, such as

an electronic compass, extracting tilt information, and correcting the tilt according to

this tilt information.

The drawings and detailed description are only examples of the present

invention, serve only for describing the present invention and by no means limit or

restrict the spirit and scope of the present invention. Thus, any person of ordinary skill

in the art shall understand that a large number of permutations and other equivalent

embodiments are possible. The true scope of the present invention must be defined only

by the spirit of the appended claims.

[Industrial Applicability]

As described above, the present invention can correct a tilted image automatically in a portable camera device such as a mobile communication terminal.

The present invention can also efficiently correct the tilt because the tilt

correction application is activated according to a variety of setting information.

The present invention can also preview the area to be cropped by applying

black transparency in the viewfϊnder.

Claims

[CLAIMS]

[Claim 1 ]

A method of correcting image tilt in an image processing apparatus including a

tilt sensor and a viewfinder displaying a preview image, comprising:

receiving tilt information sensed by the tilt sensor;

outputting to the viewfinder, when previewing, a guide frame indicating

absolute vertical/horizontal lines based on the tilt information; and

correcting the tilt by cropping areas outside the guide frame from a captured

original image and rotating the cropped image.

[Claim 2]

The method of claim 1, wherein the outputting the guide frame is done when a

user presses a predetermined shortcut key.

[Claim 3]

The method of claim 1, wherein the outputting the guide frame is done when

the image processing apparatus is tilted from the absolute vertical/horizontal lines by

more than a predetermined angle.

[Claim 4]

The method of claim 1, wherein the correcting is made either vertically or

horizontally in accordance with a user setting.

[Claim 5]

The method of claim 1, further comprising writing the tilt information of the

captured image.

[Claim 6]

The method of claim 1 , further comprising:

storing the captured original image in a memory; and

determining whether a shooting mode of the image processing apparatus is

terminated,

wherein the correcting is done by extracting the original image file stored in

the memory if the shooting mode is terminated.

[Claim 7]

The method of claim 1 , wherein the outputting outputs to the viewfinder

direction information for moving the image processing apparatus in order to make an

alignment with the absolute horizontal/vertical lines.

[Claim 8]

The method of claim 1, wherein the correcting further comprises treating the

cropped areas with black transparency when previewing.

[Claim 9]

The method of claim 1 further comprising deleting from a memory an original

image corresponding to the tilt-corrected image.

[Claim 10]

The method of claim 1 further comprising enlarging the tilt-corrected image to

a size of the original image.

[Claim 11 ]

A recorded medium recording a program readable by a computer, the program

executing a method of any of claims 1-10.

[Claim 12]

A processor for correcting image tilt, comprising:

a tilt information receiving unit, receiving tilt information detected by a tilt sensor;

a guide frame outputting unit, outputting to a viewfinder, when previewing, a

guide frame indicating absolute vertical/horizontal lines based on the tilt information;

and

a tilt correcting unit, cropping areas outside the guide frame from a captured

original image and rotating the cropped image.