KR20090039404A - Image processing apparatus and controlling method thereof - Google Patents

Abstract

An image processing apparatus and a controlling method thereof are provided to improve a user convenience by recognizing a character representing a geographical region in a photo image, and to provide search results of the geographical information. An image preprocessor(130) transforms an image in order to enhance the character recognition rate to easily recognize characters in a photo image. A character recognition unit(140) extracts characters included in a photo image, and transforms the extracted characters to a text data recognizable by a CPU. A database(160) stores geographical information corresponding to the transformed text data. A digital signal processor(150) controls components of the digital image processing apparatus.

Description

Digital image processing apparatus and control method thereof

The present invention extracts local names captured on photographic images and recognizes them as readable characters, and provides the user with local information retrieved from the recognized characters in real time. The present invention relates to a digital image processing apparatus having an automatic search function for local information and a control method thereof so that the desired local information can be conveniently provided.

The services provided based on the conventional character recognition technology are mostly simple and low in efficiency, such as displaying a recognized character from image information or recording the recognized character in a specific database. For example, there is a technology that reads a character from the target image and stores it in the database, and then displays the content of the character read from the database according to a user's request, but such a conventional technology has a limited application field. In the multi-functionalization trend of digital devices, there is a problem in that the contribution to the user's product satisfaction is very limited.

An object of the present invention is to extract the local name captured on the photographic image to recognize it as a readable character, and to provide the user with local information retrieved from the recognized character in real time, so that the user does not enter the local name directly In order to conveniently provide local information desired by the user, a digital image processing apparatus having an automatic search function for local information and a control method thereof are provided.

It is another object of the present invention to provide a digital image processing apparatus and a control method thereof, which provide a variety of information related to almost all subjects within the limits of storage capacity from photographic images in which arbitrary search terms are taken out of region-based search. .

In order to achieve the above and other objects, the control method of the digital image processing apparatus of the present invention,

a) extracting character information from the photographed photographic image and recognizing it as readable information;

b) retrieving a database from the recognized information; And

c) providing the searched results to the user.

Preferably, in the step a) extracting character information from the photographed photographic image and recognizing it as readable information,

a-1) extracting a text area from the background of the photographic image; And

a-2) applying a character recognition technique to the extracted character region. In this case, in step a-2), template matching may be applied.

On the other hand, in the step c) providing the searched results to the user,

c-1) listing and displaying categories of information available to the user; And

c-2) providing information belonging to a category selected by the user. In the area-based search, the category of the information that can be provided may include navigation substitute map graphic information and tourism guide substitute travel information.

On the other hand, the image processing apparatus according to another aspect of the present invention,

An image processing apparatus which extracts and recognizes text information from a photographed photographic image and provides related information.

An image acquisition unit generating a photographic image;

An image preprocessor for converting a photographic image input from the image acquisition unit into an image for easy character recognition;

A character recognition unit for recognizing a character from the photographic image passed through the preprocessing unit and outputting the character as readable character data;

A database containing information corresponding to the character data; And

And a digital signal processor for searching the database using the text data obtained from the text recognition unit as a keyword and providing the searched results to the user.

In this case, the image processing apparatus may be a digital camera or a camera-mounted mobile device.

According to the digital image processing apparatus and control method thereof of the present invention, it extracts and recognizes a character part from a photographed image taken through image processing and automatically searches for local information on a current location. By providing the automatic search function, users can provide the desired local information through a single operation, such as taking a picture, without having to have a separate database for input or search for the current location by the user. This can be improved dramatically.

In particular, when the technical principles of the present invention are applied to digital cameras, mobile communication devices with built-in camera functions, and the like, the multi-functionalization contributes to product competitiveness, thereby improving the user's satisfaction with the product.

On the other hand, by expanding the same technical principle, it is possible to provide an automatic search function for any subject not based on the region, it is possible to provide a higher value-added service.

Hereinafter, a digital image processing apparatus and a control method thereof according to an exemplary embodiment of the present invention will be described. In the following embodiments, a digital camera is assumed as a form of a digital image processing apparatus. However, this is merely an example for the convenience of understanding. Through this specification, the "digital image processing apparatus" may be used to capture and capture an image. It is used in a broad sense to encompass all image processing devices having an image processing means for a processed image. The technical principles described below may be equally applied to mobile communication devices such as a mobile phone having a camera function in addition to a digital camera. .

1 illustrates a functional block diagram of the digital camera 100. The illustrated digital camera 100 includes a user input unit 170, an image acquisition unit 110, an image data storage unit 120, and a display panel 180. The image acquisition unit 110, together with an optical unit (OPS) consisting of a plurality of optical lenses, to process the output signal to obtain a digital data photographed image CCD (Charge Coupled Device) or CMOS (Complementary Metal- And a photoelectric conversion unit (OEC) such as an oxide-semiconductor, an A / D converter (ADC), a driving circuit, a signal processing circuit, and the like. The photographic image output from the image acquisition unit is stored in an image data storage unit 120 implemented as a temporary storage device such as a dynamic random access memory (DRAM) or a secondary storage device such as a removable flash ROM (Flash ROM). do. In this case, the image data output from the image acquisition unit 110 may be stored in the data storage unit 120 after being converted into image data of an Exif file format including Exif information recording recording information at the time of shooting in a file header. have.

The digital camera 100 further includes the following components so as to automatically search for information on the region by extracting the region-related name from the captured image. That is, the digital camera 100 is an image pre-processing unit 130 for converting the image to the character recognition easy to increase the character recognition rate, a character that can be recognized by the CPU (Central Processing Unit) by extracting the characters recorded in the image A digital signal composed of a character recognition unit 140 for converting into data, a database 160 containing local information corresponding to the converted character data, and a central processing unit (CPU) for controlling these units. A processor 150.

The image preprocessor 130 converts the input image into an image that is easy to extract characters through image filtering, color space conversion, and illumination correction to remove high frequency noise by passing the input target image through a low pass filter. Preprocessing is performed to increase the recognition rate.

The character recognition unit 140 extracts each character element from the luminance signal on the preprocessed image and applies a character recognition technique. For example, template matching may be applied to the character recognition technique. In the template matching, a character corresponding to the most similar template, that is, the template having the highest matching rate, is determined by comparing the extracted character elements with those of the normalized characters (templates) stored in the system. do. The character recognition unit 140 converts the determined input character into binary character data and provides it to the database. On the other hand, when a matching rate of a predetermined threshold or less is obtained according to a character recognition result, the character recognition unit 140 displays the corresponding character part as a wild character (for example, "*") to perform the corresponding part. The character recognition may be notified, and the database 160 displays options on the display panel 180 regarding a plurality of region-related names searched for only the recognized characters except wild characters, thereby allowing the user to display the correct region. You may ask to choose a relevant name.

The database 160 includes related information of each region corresponding to the input text data. The content included in the area information may include map information of the area indicated by the input text data and related detailed information (for example, travel information such as a local festival or history), and information about a regional area including the surrounding area. have.

The digital signal processor 150 performs an overall operation of the digital camera 100 from generation of image data according to image capturing, recognition of a region-related name captured on an image, to provision of local information according to a recognized result. Specifically, the digital signal processor 150 collectively controls the operation of each functional component and the flow of input / output data therebetween.

Meanwhile, the user input unit 170 may include a mode selection key for switching between various operation modes of the digital camera 100 such as a shooting mode, a playback mode, and a smart location mode, and a cursor activated on a displayed menu screen. A four-way function key for moving the cursor to a specific selection, a select-confirmation key for selecting a menu item indicated by the activated cursor, a shutter release button for initiating a shooting operation in relation to the shooting operation, and a wide-angle zoom It includes a plurality of operation buttons, such as a button, a telephoto zoom button.

Hereinafter, a digital image processing apparatus and a control method thereof according to an exemplary embodiment of the present invention will be described. 2 and 3 are flowcharts of procedures performed by the digital camera 100 and mainly represent algorithms performed by the digital signal processor 150. When operating power is applied to the digital camera 100, the digital signal processor 150 performs initialization. When the initialization step is performed, the digital signal processor 150 may perform a preview mode to display an external input image on the display panel 180 (S10).

Next, when an input for mode setting is detected from the user input unit 170, the digital signal processor 150 may display a menu screen as shown in FIG. 4 on the display panel 180 to receive a user input. There is (S11). On the screen shown, the "ON" and "OFF" choices for the "Smart Location" mode are displayed, and if the user input is detected regarding the "ON" selection, the digital signal processor 150 will be described below. As will be described, the steps after S14 are performed. However, if an input relating to the user's "OFF" selection is detected, for example, the digital signal processor displays another menu screen on the display panel, and according to the user's input signal, for example, selected playback mode or the like. Operations associated with the mode are performed (S13).

On the other hand, after the smart location mode is selected ("Smart Location"-> "ON"), if the input for the region / language selection from the user is detected (S14), the digital signal processor 150 is a choice for the region / language zone. In operation S15, the user may request the user to specify the region right to which the current location belongs and / or the language right to which the subject character belongs. For example, with respect to the territorial area, the digital signal processor 150 sequentially displays options ranging from large national units to more specific small regional units, thereby shaping the level of refinement to the level at which the user perceives the current location. You can request Similarly, with respect to the language zone, options for various language zones may be listed and displayed so as to specify the language zone to which the subject character to be photographed belongs. This request for the user to specify regional / language rights is made easier for the recognition process of the characters included in the subject, and in particular, to increase the accuracy of the recognition, but it is not necessary and is optional. It is.

On the other hand, after switching to the "Smart Location" mode, the digital signal processor sets and initializes a cumulative variable (N) for counting the number of times the character recognition has failed (S16). As will be described later, the digital signal processor 150 terminates the "Smart Location" mode as it is without re-taking when character recognition fails more than a predetermined number of times.

Next, when an operation of the shutter release button is detected from the user input unit 170 (S17), the digital signal processor 150 performs the photographing mode (S18). Specifically, the digital signal processor 150 generates an image file in the image data storage unit 120 and then performs image capture. That is, the digital signal processor 150 transfers the image data received through the photoelectric conversion unit (OEC) such as CCD or CMOS and the A / D converter (ADC) to the image preprocessor 130, and performs the image storage operation. In other words, the image data is compressed, the compressed image data is converted into image data in an Exif file format, and then recorded in the data storage 120. 5 shows an example of a screen of a digital camera displaying an image of a photographed subject.

On the other hand, since the recognition rate of the characters captured on the image depends on the clarity of the acquired image, it is preferable that a high level of contrast and edge emphasis processing be performed. For this reason, it is preferable that the digital signal processor 150 adjusts each unit of the image acquisition unit to maximize the sharpness of the image even by the user's input regarding the contrast and the edge emphasis or without the user's input.

Next, the image preprocessor 130 receiving the image data receives low pass filtering to remove high frequency noise components from the input image, color space conversion for converting RGB color information into YCrCb space, and character recognition rate according to changes in lighting environment. Preprocessing such as roughness compensation to offset the difference is performed (S19). The image data easily processed in the character recognition through the preprocessing step is transferred to the character recognition unit 140 under the control of the digital signal processor 150.

Next, the character recognition unit 140 having received the image data extracts the characters captured on the image (S20), and performs a character recognition process of identifying and generating the character data (S21). First, the character recognition unit 140 distinguishes between the character element and the background in the entire image based on the luminance distribution appearing on the image (S20). As a simplified example, in a pixel array of a corresponding image, a difference (Dup, Dlow, Dlef, Drig) of luminance signals with adjacent up, down, left, and right pixels is calculated for each pixel, and the difference of the luminance signal is calculated. It is determined whether the averaged value (Dup, Dlow, Dlef, Drig) / 4 is above the threshold. As a result of the determination, it is determined that the pixel whose luminance signal difference is greater than or equal to the threshold constitutes a character element.

Next, the character recognition technique is applied to the character element extracted in the previous step (S21). For example, a matching ratio for each template is calculated by comparing the extracted text elements with a plurality of text templates stored in the system at a ratio of 1: 1. Here, if the language right is specified by the user, a character template belonging to the specified language right will be selected for comparison. Then, by comparing the calculated matching rate, the character corresponding to the template having the highest matching rate is determined as the character on the image, and the determined character is converted into binary character data to be provided to the digital signal processor 150. However, when the calculated matching rate is less than or equal to a preset threshold, the character recognition unit 140 displays the character portion as a wild character (eg, "*").

Next, the digital signal processor 150 determines whether the character recognition is successful based on the character data obtained from the character recognition unit 140 (S22). For example, the digital signal processor 150 determines whether the search for the database 160 using the input text data as a keyword is possible. If most or all of the character data is composed of wild characters, the search condition is insufficient, and re-shooting of the region-related names will be required. The digital signal processor 150 reads the number N of accumulated character recognition failures (S23). For example, if N ≦ 3, the digital signal processor 150 repeats the steps of S17 or less to retake the image. On the other hand, if the accumulated number of failures (N) of the character recognition is N> 3, the digital signal processor determines that a valid region-related name cannot be recognized from the same subject and displays a warning message (S24). The procedure ends.

On the other hand, if the character recognition is successful, for example, if it is possible to recognize the entire character line, or at least a part, the digital signal processor 150 is a character data obtained from the character recognition unit 140 As a keyword, corresponding information is retrieved from the database 160 (S25). In this case, when the wildcard is included in the character data, the database 160 is searched using only the recognizable characters as keywords except the corresponding character portion. For example, when a plurality of candidate groups exist in a region-related name because the portion corresponding to the wild character is not specified (S26), the digital signal processor 150 displays the plurality of searched candidate groups on the display panel 180. In operation S27, an accurate region-related name may be specified among a plurality of candidate groups displayed according to a user input in operation S28. In this case, if the region right about the current location is specified by the user, only candidate groups belonging to the specified region right will be selected as the candidate group.

FIG. 6 shows a configuration of a screen for requesting user identification by displaying a plurality of search results including "Pont" when only part of "Pont" is recognized among characters "Vedettes du Pont Neuf" captured on a photographic image. An example is shown. For example, the user may select "Pont Neuf" from among the displayed candidate groups. On the other hand, Figure 7 is an example of the configuration of a screen showing the user the recognized results when all the "Pont Neuf" sufficient to specify the region is recognized.

Next, the digital signal processor 150 displays the search results from the database 160 on the display panel 180 to provide the user with various information related to the current region. For example, a menu screen may be displayed so that the user can select the map information / travel information, etc. (29), and a user input signal may be determined (S30), and only the selected information may be selectively provided (S31-1, S31-2). 8A and 9A show configuration examples of a screen displaying a selection of map information / travel information on a menu screen and waiting for a user's selection. The digital signal processor 150 may perform a navigation function by providing map information according to a user's selection (S31-1), or perform a function of a tour guide by providing travel information. It may also be (S31-2). Examples of map information or travel information displayed by the digital signal processor according to a user's selection are illustrated in FIGS. 8B and 9B.

Meanwhile, the subject character to be photographed in the present invention also includes handwritten characters written by hand, but a separate preprocessing process for solving size imbalance or difficulty in recognition usually associated with the handwritten character may be preceded. Further pretreatment may follow a variety of known techniques.

In the above-described embodiment, the text data is extracted by performing image processing on an image including the region-related name, and the user is provided with the region information retrieved from the database. However, in a modified embodiment to which the technical principles of the present invention are equally applied, the present invention is not limited to a region-based search, but an image obtained by capturing an arbitrary search query is input as an input, and related information about the query is provided. The field of application can be expanded. In the modified embodiment, the above-described technical configuration may be applied in the same manner. For example, a text area may be extracted from an image captured by the user's search query and a text recognition technique may be applied to the extracted area. After recognizing it as a character, the result of searching the database with this recognized character can be returned to the user in response to the search request.

However, it may be desirable for the database to hold a variety of information covering a vast range of fields in preparation for search requests on almost all subjects. In FIG. 3, a category of information provided to a user through steps S29 to S31-1 / S31-2 is simplified as map information / travel information on the premise of region-based search. It may be divided into more general types of categories corresponding to the search object, for example, dictionary information providing brief prior knowledge and specialized information providing more in-depth knowledge.

Although the present invention has been described with reference to the embodiments illustrated in the accompanying drawings, it is merely exemplary, and various modifications and equivalent other embodiments are possible from those skilled in the art to which the present invention pertains. You will understand the point. Therefore, the true scope of protection of the present invention should be defined by the appended claims.

1 is a block diagram showing a functional configuration of a digital image processing apparatus according to an exemplary embodiment of the present invention.

2 and 3 are flowcharts illustrating a control method of a digital image processing apparatus according to an exemplary embodiment of the present invention.

4 is an example of the configuration of a menu screen relating to ON / OFF setting of the "Smart Location" mode.

5 is a diagram showing an example of a screen capturing a region-related name.

FIG. 6 is a configuration example of a screen for requesting the user's specification by displaying a plurality of region-related names searched from a part of recognizable characters on a screen.

7 is a configuration example of a screen showing a region-related name specified from a recognized character.

8A and 8B illustrate screens for displaying a selection of map information / region information and displaying map information selected by a user.

9A and 9B are examples of screens displaying a selection of map information / region information and displaying local information selected by a user.

<Description of the symbols for the main parts of the drawings>

100: digital image processing apparatus 110: image acquisition unit

120: image data storage unit 130: image preprocessing unit

140: character recognition unit 150: digital signal processor

160: database 170: user input unit

180: display panel

Claims (7)

As a control method of a digital image processing apparatus, a) extracting character information from the photographed photographic image and recognizing it as readable information; b) retrieving a database from the recognized information; And c) providing the searched result to the user. The method of claim 1, In the step a) extracting text information from the photographed photographic image and recognizing it as readable information, a-1) extracting a text area from the background of the photographic image; And a-2) applying a character recognition technique to the extracted character region; and controlling the digital image processing apparatus. The method of claim 2, The control method of the digital image processing apparatus, characterized in that the template matching (templet matching) is applied in step a-2). The method of claim 2, In c) providing the searched result to the user, c-1) listing and displaying categories of information available to the user; And c-2) providing information belonging to a category selected by the user. The method of claim 4, wherein The category of the information that can be provided includes a navigation substitute map graphic information and a tour guide substitute travel information. An image processing apparatus which extracts and recognizes text information from a photographed photographic image and provides related information. An image acquisition unit generating a photographic image; An image preprocessor for converting a photographic image input from the image acquisition unit into an image for easy character recognition; A character recognition unit for recognizing a character from the photographic image passed through the preprocessing unit and outputting the character as readable character data; A database containing information corresponding to the character data; And And a digital signal processor for searching a database using the text data obtained by the text recognition unit as a keyword and providing the searched result to a user. The method of claim 6, The image processing apparatus is a digital camera or a camera-equipped mobile device, characterized in that the digital image processing apparatus.

Images