EP1855459B1

EP1855459B1 - Apparatus and method for photographing a business card in a portable terminal

Info

Publication number: EP1855459B1
Application number: EP07107926.3A
Authority: EP
Inventors: Sung-Kyu Yoon; Eung-Seo Shin
Original assignee: Samsung Electronics Co Ltd
Current assignee: Samsung Electronics Co Ltd
Priority date: 2006-05-11
Filing date: 2007-05-10
Publication date: 2015-03-25
Anticipated expiration: 2027-05-10
Also published as: CN101102572A; US20070262148A1; EP1855459A2; EP1855459A3; KR20070109451A; US7793833B2; KR100784332B1

Description

The present invention relates generally to a portable terminal, and in particular, to an apparatus and method for allowing a business card to be automatically photographed by detecting the boundary lines of the business card.
In general, most persons exchange their business cards presenting their simple profiles and contacts when they meet a guest for the first time. In this way, tens of business cards may be exchanged each day. In this case, a person must keep and manage the received business cards in a business card case or a purse by himself/herself, and search all the business cards in order to find the business card of a specific guest.
Accordingly, portable terminals which have recently been developed are capable of photographing a business card using a camera, and providing a business card recognition function in order to recognize the business card and manage information regarding the owner of the business card. In this case, similarly to a general photographing process, the business card is photographed based on a user's subjective judgment without limiting the size and location of the business card that is to be displayed on a display unit of the portable terminal. In other words, the user photographs the business card by adjusting the size and location of the business card that is to be displayed on the display unit of the portable terminal by manually moving the portable terminal, and selecting a key button or a menu for photographing the business card when the adjusted size and location of the business card are determined to be appropriate for photographing.
In the business card recognition function, the probability of satisfactorily recognizing the business card in order to obtain the information contained in the business card varies depending on photographing conditions, such as the size, brightness, and image quality of the photographed business card. Thus, as described above, when the user photographs the business card based on his/her subjective judgment, the business card may be photographed to be too large, small or dark, thereby lowering the probability of satisfactorily recognizing the business card. Also, as described above, when the user manually selects a key button or a menu in order to photograph the business card, shaking may occur due to the selecting of the key button or the menu, and thus, shaking noise may be present in an image of the photographed business card, thereby significantly lowering the probability of satisfactorily recognizing the business card.
US-A-2004/218916 discloses an automatic photography system comprising a photographing portion, a control portion comprising shape extracting means for repeatedly extracting shapes that represent expressions of human subjects within images, a match calculating means and a control means for controlling the photographing portion to photograph the subjects when the calculated degrees of match exceeds a predetermined threshold value. The Figures of document D1 illustrate boundary lines of parts of the human body, such as head, eyes and extremities.
US-A-2004/208372 relates to object recognition based on a plurality of objects in a database. The document teaches a user aligning a sensor of an image capture device with a object pressing a button on the device or sensor and automatically recognizing that an image is to be acquired. The image type determination is accomplished with a discriminator algorithm, which operates on the input image and determines whether the input image contains recognizable symbols, such as bar codes, matrix codes, or alphanumeric characters. Decoding barcodes and alphanumeric characters will result in numerical and/or text information.
US-A-2005/281452 discloses a system and method for performing optical inspection of objects, such as printed circuit boards comprising various electronic circuits. The document teaches a user interaction with a CAD program and to select regions of interest for inspecting the printed circuit board. Further, the document discloses recognition of object boundaries in photographs taken of objects.
US-A-2004/100572 provides a method for controlling the operation of a digital camera in order to take an identification photograph in a natural setting by displaying a frame and a reference outline of the identification photograph displaying a detection area to detect an outline of a figure around the reference outline on a photographed image, detecting the outline of the figure in the detection area and forming a background image for the identification photograph in a set background area with respect to the detected outline. According to the method disclosed by the document, a user can take a photograph by roughly matching the outline of the object figure according to the reference outline.
US-A-2004/201720 discloses a digital camera taking photographs of printed materials upon recognition of a specified text pattern. In one embodiment, the method comprises a manual two-pass approach, so that when a text pattern is recognized an audible tone is emitted prompting confirmation by the user.
US A 2005/031168 relates to road position detection. This document discloses a dynamic contour extracting means that detects, using a dynamic contour extraction method generally known as SNAKE, a contour of a recognition object included on an image by the following steps: a curve with control points is disposed in an image; an internal energy Eint indicating smoothness of the curve, an image energy Eimage, and an external energy Eext are summed up; and wherein the curve is modified so that the summed total energy becomes the minimum. Here, a template is stored for including disposition information of primary control points that are bound to be moved only in a vertical or horizontal direction based on a road shape. The dynamic contour extracting means moves the primary control points based on the template. The primary control points can be rapidly moved owing to their limited movement direction.
US A 2006/084509 relates to games with targeting features. The invention disclosed in this document is directed generally toward computer-implemented methods for targeting objects in a gaming system. One aspect of the invention disclosed in this document is directed toward a method for targeting an object in a gaming system that includes receiving a soft lock command and applying a soft lock to an object in the game. The soft lock designates an action site on a portion of the object that was proximate to an aiming reference when the soft lock was commanded. The action site is proximate to a location where a selected game action will occur if the game action is commanded. Once the soft lock is applied, the aiming reference can be moved away from the action site. Another aspect of the invention disclosed in this document is directed toward a method of targeting an object in a gaming system that includes receiving a command to hard lock a portion of an object, applying the hard lock, and displaying at least one selectable sub-target associated with the object. During the hard lock, the aiming reference remains at least proximate to the portion of the object. When selected, a sub-target includes an action site on part of the object proximate to a location where game action will occur if selected.
It is the object of the present application to improve photographing of a business card in a portable terminal with respect to an advantageous further processing of the acquired digital image of a business card.
This object is solved by the subject matter of the independent claims.
Preferred embodiments are defined by the dependent claims.
Another aspect of the present invention is to provide an apparatus and method for allowing a business card to be automatically photographed by recognizing the boundary lines of the business card in a portable terminal.
A further aspect of the present invention is to provide an apparatus and method for photographing a business card in a portable terminal by providing a user with an environment appropriate for photographing the business card.
The above and other aspects, features and advantages of the present invention will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which:

FIG. 1 is a block diagram of a portable terminal according to the present invention;
FIG. 2 is a flowchart illustrating a method of photographing a business card in a portable terminal, according to the present invention; and
FIG. 3 illustrates screen images illustrating photographing a business card in a portable terminal, according to the present invention.

Preferred embodiments of the present invention will be described herein below with reference to the accompanying drawings. In the following description, well-known functions or constructions are not described in detail when it is determined that they would obscure the invention in unnecessary detail.
The present invention provides a user with an environment appropriate for photographing a business card, and an apparatus and method for allowing a business card to be automatically photographed by recognizing the boundary lines of the business card.
FIG. 1 is a block diagram of a portable terminal according to the present invention. Referring to FIG. 1, the portable terminal includes a controller 100, a camera unit 102, an image processor 104, and a display unit 106. The controller 100 performs processing or controlling in order to establish a voice/data communications. In particular, in order to photograph a business card, the controller 100 receives brightness data from the camera unit 102, checks a brightness environment under which the business card is to be photographed, and controls whether to perform automatic photographing on the business card. Also, the controller 100 displays on the display unit 106 reference boundary lines, for presenting a size and location, of a business card, which are appropriate for recognizing the business card; and determines whether the boundary lines of the business card received from the image processor 104 coincide with the reference boundary lines. If they coincide with one another, the controller 100 controls automatic photographing to be performed.
The camera unit 102 includes a camera sensor that converts an optical signal, which is sensed when an image is photographed, into an analog image signal; and a signal processor that converts the analog image signal into a digital data. The camera unit 102 outputs the digital data into which the analog image signal has been converted, to the image processor 104. Also, the camera unit 102 converts the brightness of an image received from the camera sensor into measurable brightness data, and transmits the brightness data to the controller 100. Here, the camera sensor may be embodied as a Charge Coupled Device (CCD) sensor, and the signal processor may be embodied as a Digital Signal Processor (DSP).
The image processor 104 processes the digital data received from the camera unit 102 in units of frames, and outputs the result of processing to be appropriate for the characteristics and size of the display unit 106. Also, the image processor 104 includes a video codec that codes an image signal in a predetermined format or decodes coded frame image data to the original frame image data. When a business card is photographed, the image processor recognizes the boundary lines of an image received from the camera unit 102, and detects a location where each of the boundary lines of the business card is recognized and provides the result of detecting to the controller 100 when the boundary lines are recognized. Here, the locations of the boundary lines of the business card may be detected by extracting the contour of the business card by using an algorithm, such as Sobel, Prewitt, Robert, or Laplacian algorithms.
The display unit 106 displays status information and characters (which consist of limited numbers) generated during the operation of the portable terminal. In particular the display unit 106 outputs an image received from the image processor 104, displays the reference boundary lines for the business card under control of the controller 100, and changes and displays the color, shape or thickness of each of the reference boundary lines.
FIG. 2 is a flowchart illustrating a method of photographing a business card in a portable terminal, according to an embodiment of the present invention.
Referring to FIGS. 1 and 2, first, the portable terminal determines whether a business card recognition and photographing mode has been selected, in step201. If it is determined that the business card recognition and photographing mode has been selected, the portable terminal outputs an image received from the camera sensor to the display unit 106, and displays on the display unit 106 reference boundary lines for appropriately fixing the size and location of the business card that is to be photographed in step 203. For example, as illustrated in FIG. 3A, boundary lines for presenting a size and location of a business card, which are appropriate for recognizing the business card, are displayed as a rectangular frame in the center of the display unit 106. Here, the reference boundary lines present a location and size, of a business card, for achieving the highest probability of satisfactorily recognizing the business card; and may be obtained through an experiment.
Next, the portable terminal checks brightness data from the received image in step 205, and determines whether the current brightness of the image is appropriate for photographing the business card, that is, for recognizing the business card, based on the brightness data in step 207. Here, whether the current brightness is appropriate for photographing the business card may be determined by comparing the brightness data with a reference brightness value for recognizing the business card.
If it is determined that the current brightness is not appropriate for photographing the business card, the portable terminal displays a message indicating this fact on the display unit 106 in step 227, and then returns to operation 205. If it is determined that the current brightness is appropriate for photographing the business card, the portable terminal recognizes the boundary lines of the business card by analyzing an image received from the camera sensor in step 209. Next, the portable terminal proceeds to operation 211, and determines whether at least one of the reference boundary lines (upper, lower, left, and right boundary lines) coincides with the boundary lines of the photographed business card. Here, a predetermined margin of error is allowed when determining whether the reference boundary lines coincide with the boundary lines of the business card. For example, if the predetermined margin of error is 3 mm, the left boundary line of the business card can be determined to coincide with the left reference boundary line when the top (or bottom) of the left boundary line of the business card is 2 mm away from the top (or bottom) of the left reference boundary line in a vertical or horizontal direction.
If none of the reference boundary lines coincides with the boundary lines of the business card, the portable terminal returns to step 209. If at least one of the reference boundary lines coincides with the boundary lines of the business card, the portable terminal proceeds to step 213, and replaces the color of the coinciding reference boundary line with a predetermined color. For example, as illustrated in FIG. 3B, when one of the boundary lines of the photographed business card coincides with the left reference boundary line, the color of the left reference boundary line may be replaced with a predetermined, different color. If the boundary lines of the photographed business card coincide with the left and upper reference boundary lines, the colors of the left and upper reference boundary lines may be replaced with a predetermined, different color (FIG. 3C).
Next, the portable terminal determines whether all the upper, lower, left and right reference boundary lines coincide with the boundary lines of the business card in step 215. If all the reference boundary lines do not coincide with all the boundary lines of the business card, the portable terminal returns to step 209. If all the reference boundary lines completely coincide with the boundary lines of the business card as illustrated in FIG. 3D, the portable terminal proceeds to step 217 and displays a message informing the user of the start of automatic photographing for the business card. Here, it is possible to inform the user of the start of automatic photographing by generating a predetermined alarm sound of automatic photographing or changing the shapes, thicknesses or colors of the reference boundary lines.
Next, the portable terminal performs auto focusing in order to photograph the business card in step 219. Next, the portable terminal determines whether the reference boundary lines still coincide with the boundary lines of the business card even after performing auto focusing in step 225. If at least one of the reference boundary lines does not coincide with the boundary lines of the business card, the portable terminal changes the current color of the at least one reference boundary line, which does not coincide with the boundary lines of the business card, to a default color, i.e., the original color of the reference boundary line being initially displayed on the display unit 106 in step 221, and then returns to step 209. However, if all the reference boundary lines still coincide with all the boundary lines of the business card, the portable terminal photographs the business card in step 223, and ends the processes of the present invention.
In the current embodiment of the present invention, if the reference boundary lines coincide with the boundary lines of the business card, the colors of the coinciding reference boundary lines are replaced with a default color. However, the thicknesses or shapes of the coinciding reference boundary lines may be changed.
Alternate embodiments of the present invention can also comprise computer readable codes on a computer readable medium. The computer readable medium includes any data storage device that can store data that can be read by a computer system. Examples of a computer readable medium include magnetic storage media (such as ROM, floppy disks, and hard disks, among others), optical recording media (such as CD-ROMs or DVDs), and storage mechanisms such as carrier waves (such as transmission through the Internet). The computer readable medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. Also, functional programs, codes, and code segments for accomplishing the present invention can be construed by programmers of ordinary skill in the art to which the present invention pertains.
As described above, according to the present invention, it is possible to allow a business card to be automatically photographed in a portable terminal by recognizing the boundary lines of the business card, thereby preventing shaking when a user selects a key button or a menu in order to photograph the business card. Further, it is possible to improve the probability of satisfactorily recognizing the business card by achieving a business card image having a size, location, and brightness that are appropriate for recognizing the business card.
While the invention has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the invention as defined by the appended claims.

Claims

A method of photographing a business card in a portable terminal, the method comprising:
recognizing (209) boundary lines of the business card from an image received from a camera when a menu or key for recognition and photographing of a business card is selected;

displaying predetermined reference boundary lines and the image having the recognized boundary lines of the business card;

determining whether recognized boundary lines of the business card coincide with predetermined reference boundary lines, wherein

when at least one recognized boundary line of the business card coincides with a reference boundary line (211), replacing (213) a color or thickness of said at least one coinciding reference boundary line with a predetermined color or thickness and wherein

when all predetermined reference boundary lines coincide with recognized boundary lines of the business card (215), automatically photographing (223) the business card.
The method of claims 1, wherein when all predetermined reference boundary lines coincide with recognized boundary lines of the business card, further comprising informing (217) a user that automatic photographing of the business card is performed.
The method of claim 2, wherein the informing (217) of the automatic photographing comprises at least one of generating an alarm sound and displaying a message.
The method of claims 1 or 2, wherein automatically photographing the business card comprises:
performing (219) auto focusing when the boundary lines of the business card coincide with the reference boundary lines;

determining (225) whether the boundary lines of the business card still coincide with the reference boundary lines after performing auto focusing; and

automatically photographing (223) the business card when the boundary lines of the business card still coincide with the reference boundary lines.
The method of one of claims 1 to 4, further comprising:
when the recognition and photographing key is selected, determining (205) whether the brightness of the image received from the camera satisfies a predetermined condition; and

when it is determined (207) that the brightness of the image does not satisfy the predetermined condition, displaying (227) a message requesting brightness control.
An apparatus for photographing a business card in a portable terminal, the apparatus comprising:
a camera unit (102);

an image processor (104) for recognizing boundary lines of the business card from an image received from a camera when a menu or key for recognizing and photographing of a business card is selected;

a display unit (106) for displaying reference boundary lines and the image received from the camera; wherein the display unit replaces a color or thickness of at least one coinciding reference boundary line with a predetermined color or thickness when at least one of the boundary lines of the business card is recognized to coincide with said reference boundary line; and

a controller (100) for controlling the business card to be automatically photographed when all predetermined reference boundary lines coincide with recognized boundary lines of the business card.
The apparatus of claim 6, wherein the controller (100) performs auto focusing when all predetermined reference boundary lines coincide with recognized boundary lines of the business card.
The apparatus of one of claims 6 or 7, wherein the camera unit (102) is adapted for extracting brightness data from the image received from the camera, and providing the brightness data to the controller (100),
wherein the controller (100) compares the brightness data with a reference brightness value, and displays a message requesting brightness control on the display unit when the brightness data is less than the reference brightness value.
A computer-readable recording medium having recorded thereon a program for photographing a business card in a portable terminal, comprising code segments that when executed by a portable terminal perform the method of one of claims 1 to 5.