WO2018037519A1

WO2018037519A1 - Mobile terminal, image processing method, and program

Info

Publication number: WO2018037519A1
Application number: PCT/JP2016/074720
Authority: WO
Inventors: 朋也穴澤; 清人小坂
Original assignee: 株式会社Pfu
Priority date: 2016-08-24
Filing date: 2016-08-24
Publication date: 2018-03-01
Also published as: JP6613378B2; JPWO2018037519A1

Abstract

Captured image data of an image captured by an image capture unit is acquired, a source document image included in the captured image is identified, image data of partial areas in the source document image is acquired, blur in the partial areas is detected, a partial area is set as an area for identifying the orientation of the source document image, on the basis of the blur, the orientation of content in said area is identified, the orientation of the source document image is identified on the basis of the orientation of the content, and corrected image data of the source document image which has been corrected so as to be upright is acquired on the basis of the orientation of the source document image.

Description

Mobile terminal, image processing method, and program

The present invention relates to a mobile terminal, an image processing method, and a program.

Conventionally, techniques for detecting image blur have been disclosed.

Here, a technique is disclosed in which blur or blur in a subject area of a photographed image is detected and a success image in which blur blur is intentionally expressed by a user or an unsuccessful failure image is evaluated (Patent Literature). 1).

JP 2013-12906 A

However, the conventional image processing apparatus (Patent Document 1) has a problem that blur determination of an image is not used for image orientation correction.

The present invention has been made in view of the above problems, and the inclination of the device at the time of shooting is corrected by correcting the orientation of the document image using a non-blurred area in the document image shot by the user with the mobile terminal. An object of the present invention is to provide a mobile terminal, an image processing method, and a program capable of appropriately correcting the inclination of a document image caused by the above.

In order to achieve such an object, a mobile terminal according to the present invention includes an image acquisition unit that acquires captured image data of a captured image captured by a capturing unit, and a document specifying unit that specifies a document image included in the captured image. A partial area acquisition means for acquiring partial area image data of a partial area in the document image, a blur detection means for detecting blur in the partial area, and the orientation of the document image based on the blur. Target area setting means for setting as a target area for identifying the content, orientation direction means for specifying the orientation of the content in the target area, and specifying the orientation of the document image based on the orientation of the content, and the document image Orientation correcting means for acquiring post-correction image data of the original image corrected upright based on the orientation of the image.

The image processing method according to the present invention includes an image acquisition step of acquiring captured image data of a captured image captured by a capturing unit, a document specifying step of specifying a document image included in the captured image, and a document image A partial region acquisition step of acquiring partial region image data of the partial region, a blur detection step of detecting blur of the partial region, and a target region for identifying the orientation of the document image based on the blur A target area setting step that is set as: a direction of content in the target area is specified; a direction specifying step that specifies a direction of the document image based on the direction of the content; and a direction based on the direction of the document image; And a direction correction step of acquiring post-correction image data of the original image corrected upright.

The program according to the present invention includes an image acquisition step of acquiring captured image data of a captured image captured by the capturing unit, a document specifying step of specifying a document image included in the captured image, and a partial area in the document image. A partial region acquisition step for acquiring the partial region image data, a blur detection step for detecting blur of the partial region, and the partial region as a target region for identifying the orientation of the document image based on the blur A target region setting step, a direction of content in the target region is specified, a direction specifying step of specifying a direction of the original image based on the direction of the content, and an upright state based on the direction of the original image Causing the computer to execute an orientation correction step of acquiring corrected image data of the corrected original image. And butterflies.

According to the present invention, it is possible to appropriately perform orientation correction on a document image captured by a user with a mobile camera, regardless of the document type or the tilt of the device at the time of shooting.

FIG. 1 is a block diagram illustrating an example of a configuration of a mobile terminal according to the present embodiment. FIG. 2 is a flowchart illustrating an example of processing in the mobile terminal according to the present embodiment. FIG. 3 is a diagram illustrating an example of a captured image in the present embodiment. FIG. 4 is a diagram illustrating an example of the orientation specifying process in the present embodiment. FIG. 5 is a diagram illustrating an example of the orientation correction process in the present embodiment. FIG. 6 is a diagram illustrating an example of partial area acquisition processing in the present embodiment. FIG. 7 is a diagram illustrating an example of partial area acquisition processing in the present embodiment. FIG. 8 is a diagram illustrating an example of partial area acquisition processing in the present embodiment. FIG. 9 is a diagram illustrating an example of a captured image in the present embodiment. FIG. 10 is a diagram illustrating an example of a document image in the present embodiment. FIG. 11 is a diagram illustrating an example of blur determination in the present embodiment. FIG. 12 is a diagram illustrating an example of blur determination in the present embodiment. FIG. 13 is a schematic diagram illustrating an example of the orientation correction process in the present embodiment.

Hereinafter, embodiments of a mobile terminal, an image processing method, and a program according to the present invention will be described in detail based on the drawings. In addition, this invention is not limited by this embodiment.

[Configuration of this embodiment]
Hereinafter, an example of the configuration of the mobile terminal 100 according to the embodiment of the present invention will be described with reference to FIG. 1, and then the processing and the like of the present embodiment will be described in detail. FIG. 1 is a block diagram illustrating an example of a configuration of the mobile terminal 100 according to the present embodiment.

However, the embodiment described below exemplifies the mobile terminal 100 for embodying the technical idea of the present invention, and is not intended to specify the present invention to the mobile terminal 100, and claims. The present invention is equally applicable to the mobile terminal 100 of other embodiments included in the scope of the above.

In addition, the form of function distribution in the mobile terminal 100 exemplified in the present embodiment is not limited to the following, and may be configured to be functionally or physically distributed / integrated in arbitrary units within a range where similar effects and functions can be achieved. be able to.

Here, the mobile terminal 100 is a portable information processing having portability such as a tablet terminal, a mobile phone, a smartphone, a PHS, a PDA, a notebook personal computer, or a wearable computer such as a glasses type or a watch type. It may be a device.

First, as shown in FIG. 1, the mobile terminal 100 is generally configured to include a control unit 102, a storage unit 106, a photographing unit 110, an input / output unit 112, a sensor unit 114, and a communication unit 116.

Here, although omitted in FIG. 1, in this embodiment, an input / output interface unit (not shown) for connecting the input / output unit 112 and the control unit 102 may be further provided. Each unit of the mobile terminal 100 is connected to be communicable via an arbitrary communication path.

Here, the communication unit 116 is a network interface (NIC (Network Interface Controller) or the like), Bluetooth (registered trademark), or the like for transmitting and receiving IP data by wired communication and / or wireless communication (WiFi (registered trademark), etc.), or An interface that performs wireless communication by infrared communication or the like may be used.

Here, the mobile terminal 100 may be communicably connected to an external device via a network using the communication unit 116.

Also, the sensor unit 114 detects a physical quantity and converts it into a signal (digital signal) of another medium. Here, the sensor unit 114 includes a proximity sensor, a direction sensor, a magnetic field sensor, a linear acceleration sensor, a luminance sensor, a gyro sensor, a pressure sensor, a gravity sensor, an acceleration sensor, an atmospheric pressure sensor, and / or a temperature sensor. Also good.

Also, the input / output unit 112 performs data input / output (I / O). Here, the input / output unit 112 may be, for example, a key input unit, a touch panel, a control pad (for example, a touch pad and a game pad), a mouse, a keyboard, and / or a microphone.

Also, the input / output unit 112 may be a display unit that displays a display screen of an application or the like (for example, a display, a monitor, a touch panel, or the like configured by liquid crystal or organic EL).

Also, the input / output unit 112 may be an audio output unit (for example, a speaker or the like) that outputs audio information as audio. The input / output unit (touch panel) 112 may include a sensor unit 114 that detects physical contact and converts it into a signal (digital signal).

Further, the photographing unit 110 acquires continuous (moving image) image data (frames) by continuously photographing a subject (for example, a document or the like). For example, the imaging unit 110 may acquire video data. The imaging unit 110 may acquire ancillary data.

Here, the photographing unit 110 may be a camera or the like provided with an image sensor such as a CCD (Charge Coupled Device) and / or a CMOS (Complementary Metal Oxide Semiconductor).

In addition, the photographing unit 110 may acquire captured image data of a captured image that is a still image by capturing a still image of the subject. Here, the captured image data may be uncompressed image data. The captured image data may be high-resolution image data.

Here, the high resolution may be full high vision, 4K resolution, super high vision (8K resolution), or the like. Further, the photographing unit 110 may shoot moving images at 24 fps, 30 fps, or the like.

The storage unit 106 stores various databases, tables, and / or files. The storage unit 106 may store various application programs (for example, user applications).

The storage unit 106 is storage means, for example, a memory such as RAM / ROM, a fixed disk device such as a hard disk, a solid state drive (SSD), a flexible disk, and / or a tangible storage device such as an optical disk, Alternatively, a memory circuit can be used.

Further, the storage unit 106 stores a computer program and the like for giving instructions to the controller and performing various processes.

Among these components of the storage unit 106, the dictionary data file 106a stores dictionary data. Here, the dictionary data may be data relating to characters, numbers, symbols and the like of each language.

The form data file 106b stores characteristic data and layout data of a specific form. Here, the specific form may be a prescribed form having a predetermined layout such as various licenses including a driver's license, various identification cards including a passport, or a health insurance card.

The image data file 106c stores image data (such as a frame). Here, the image data file 106c may store captured image data, document image data, partial area image data, target area image data, and / or corrected image data.

Further, the image data file 106c may store position data such as a document image, a partial area, and / or a target area. The image data file 106c may store character data corresponding to the image data.

Also, the image data file 106c may store video data. The image data file 106c may store ancillary data.

In addition, the control unit 102 is a CPU that centrally controls the mobile terminal 100, a many-core CPU, a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an LSI (Large Scale Integration), and an ASIC (Application Splicing Spec.). And / or a tangible controller including a FPGA (Field-Programmable Gate Array) or the like, or a control circuit.

The control unit 102 has an internal memory for storing a control program, a program defining various processing procedures, and necessary data, and performs information processing for executing various processes based on these programs.

Here, the control unit 102 has an image acquisition unit 102a, a document specification unit 102b, a partial region acquisition unit 102c, a blur detection unit 102d, a target region setting unit 102e, a form determination unit 102f, a direction specification unit 102g, in terms of functional concept. An orientation correction unit 102h and an image display unit 102i are provided.

The image acquisition unit 102a acquires image data. Here, the image acquisition unit 102a may acquire captured image data of a captured image captured by the capturing unit 110.

Further, the image acquisition unit 102a may acquire captured image data obtained by re-shooting by the shooting unit 110 when the blur detection unit 102d does not detect a blur below a predetermined reference value. The image acquisition unit 102a may acquire non-compressed and high-resolution image data.

In addition, the image acquisition unit 102a may acquire image data (frame) corresponding to one frame by controlling continuous image shooting or moving image shooting by the shooting unit 110. The image acquisition unit 102a may acquire image data by controlling still image shooting by the shooting unit 110.

Further, the image acquisition unit 102a may acquire document image data, partial area image data, target area image data, and / or corrected image data. The image acquisition unit 102a may acquire ancillary data.

The document specifying unit 102b specifies a document image included in the photographed image. Here, the document may be a rectangular document. The document specifying unit 102b may detect the position data of the document image from the captured image data.

Here, the document specifying unit 102b may detect the corner coordinates (four points) of the document image from the captured image data. The document specifying unit 102b may detect the layout of the document image from the captured image data.

The document specifying unit 102b detects the position data of the document image from the photographed image data using the edge detection method and / or the feature point matching method, and specifies the document image based on the position data of the document image. May be.

The partial area acquisition unit 102c acquires partial area image data of a partial area in the document image. Here, the partial area acquisition unit 102c may acquire partial area image data of a partial area obtained by dividing the document image.

Further, the partial area acquisition unit 102c may acquire partial area image data of a partial area indicating characters in the original image by labeling the original image data of the original image.

The blur detection unit 102d detects the blur of the image. Here, the blur detection unit 102d may detect blur in the partial area.

The target area setting unit 102e sets a target area for identifying the orientation of the document image. Here, the target area setting unit 102e may set the partial area as a target area for identifying the orientation of the document image based on the blur.

Further, the target area setting unit 102e identifies the orientation of the original image of the partial area in which the blur equal to or less than the predetermined reference value is detected by the blur detection unit 102d. The target area may be set.

Further, the target area setting unit 102e may compare the blur detected by the blur detection unit 102d and set a partial area that is least blurred as a target area for identifying the orientation of the document image.

The form determination unit 102f determines whether the document image corresponds to the specific form based on the feature data of the specific form.

The orientation identifying unit 102g identifies the orientation of the document image. Here, the orientation specifying unit 102g may specify the orientation of the content in the target area, and may specify the orientation of the document image based on the orientation of the content.

Further, the orientation specifying unit 102g may specify the orientation of the document image based on the layout data of the specific form when the form determining unit 102f determines that the original image corresponds to the specific form.

Further, the orientation specifying unit 102g specifies a character area indicating a character in the target area by labeling the target area image data of the target area, and based on the comparison between the character area data of the character area and the dictionary data, The direction of the character may be specified, and the direction of the document image may be specified based on the direction of the character.

Further, the orientation specifying unit 102g may specify the direction of characters in the target area based on the comparison between the target area data of the target area and the dictionary data, and may specify the direction of the document image based on the direction of the characters. Good.

The orientation correction unit 102h acquires post-correction image data of the original image that has been corrected upright. Here, the orientation correction unit 102h may acquire post-correction image data of the document image that has been corrected upright based on the orientation of the document image.

The image display unit 102i displays image data. Here, the image display unit 102i may display captured image data, document image data, partial area image data, target area image data, and / or corrected image data.

Further, the image display unit 102i may display the image data on the input / output unit 112. The image display unit 102i may display character data.

[Process of this embodiment]
An example of processing executed by the mobile terminal 100 configured as described above will be described with reference to FIGS. FIG. 2 is a flowchart illustrating an example of processing in the mobile terminal 100 of the present embodiment.

As shown in FIG. 2, first, the image acquisition unit 102 a controls shooting by a shooting unit (camera) 110 using a rectangular document as a subject, and acquires shot image data of a shot image shot by the shooting unit 110 ( Step SA-1).

Then, the document specifying unit 102b detects the position data of the document image from the captured image data by using the edge detection method and / or the feature point matching method, and includes it in the captured image based on the position data of the document image. A document image to be printed is specified (step SA-2).

At this time, the image display unit 102i may display the document image data of the document image specified by the document specifying unit 102b on the input / output unit 112, thereby allowing the user to confirm the specified document image.

Then, the form determination unit 102f determines whether the document image corresponds to the specific form based on the characteristic data of the specific form stored in the form data file 106b (step SA-3).

If the form determination unit 102f determines that the document image corresponds to a specific form (step SA-3: Yes), the form determination unit 102f shifts the process to step SA-4.

Then, the orientation specifying unit 102g specifies the orientation of the document image based on the layout data of the specific form stored in the form data file 106b (Step SA-4).

Then, the orientation correction unit 102h acquires the corrected image data of the original image corrected upright based on the orientation of the original image (step SA-5), and shifts the processing to step SA-12.

Here, with reference to FIG. 3 to FIG. 5, an example of the orientation correction processing in a specific form in the present embodiment will be described. FIG. 3 is a diagram illustrating an example of a captured image in the present embodiment. FIG. 4 is a diagram illustrating an example of the orientation specifying process in the present embodiment. FIG. 5 is a diagram illustrating an example of the orientation correction process in the present embodiment.

In the present embodiment, the original image shown in FIG. 4 is specified by extracting a rectangle from the photographed image shown in FIG.

In the present embodiment, the feature A (portion surrounded by a thick frame) is extracted from the document image of the driver's license shown in FIG.

In the present embodiment, whether the feature is a specific form (driver's license) is determined by determining the consistency between the extracted feature and the feature data of the form registered in the database (form data file 106b) in advance. It is determined whether or not.

In this embodiment, when the form is a specific form, the form type information is set, and the orientation of the original image is corrected based on the layout data specific to the specific form as shown in FIG. .

Returning to FIG. 2, when the form determination unit 102f determines that the document image does not correspond to a specific form (step SA-3: No), the form shifts the process to step SA-6.

Then, the partial area acquisition unit 102c acquires partial area image data of the partial area obtained by dividing the document image (step SA-6).

The partial area acquisition unit 102c may acquire partial area image data of a partial area indicating characters in the original image by labeling the original image data of the original image.

Here, an example of the partial area acquisition process in the present embodiment will be described with reference to FIGS. 6 to 8 are diagrams illustrating an example of the partial region acquisition process in the present embodiment.

As shown in FIG. 6, in this embodiment, partial area image data of a partial area obtained by simply dividing an original image into 2 × 2 quadrants may be acquired.

Further, as shown in FIG. 7, in this embodiment, partial area image data of a partial area obtained by simply dividing a document image into 3 × 3 9 areas may be acquired.

Also, as shown in FIG. 8, in the present embodiment, by performing a labeling process on the binarized document image B document image data, the partial region C divided into character units (label units) is divided. Partial area image data may be acquired.

Returning to FIG. 2, the blur detection unit 102d detects blur in the partial area (step SA-7). Here, the blur detection unit 102d may detect blur in the partial region using a determination method based on edge strength or the like.

Here, with reference to FIG. 9 and FIG. 10, an example of blur occurrence in the present embodiment will be described. FIG. 9 is a diagram illustrating an example of a captured image in the present embodiment. FIG. 10 is a diagram illustrating an example of a document image in the present embodiment.

When a document is photographed with a mobile camera, it may be photographed from multiple directions, such as from an oblique direction.

Therefore, in the document image D included in the photographed image of FIG. 9, as shown in FIG. 10, in the region E close to the photographing position, blurring is unlikely to occur and the character resolution is unlikely to decrease, but the document image far from the photographing position. In the region F in D, blurring is likely to occur, and the character resolution is likely to decrease.

Returning to FIG. 2, the blur detection unit 102d determines whether or not there is a partial region where the detected blur is equal to or less than a predetermined reference value (step SA-8).

If the blur detection unit 102d determines that there is no partial region where the detected blur is equal to or less than the predetermined reference value (step SA-8: No), the process shifts to step SA-1.

On the other hand, if the blur detection unit 102d determines that there is a partial region where the detected blur is equal to or less than the predetermined reference value (step SA-8: Yes), the process shifts to step SA-9.

Here, an example of blur determination in the present embodiment will be described with reference to FIGS. 11 and 12. 11 and 12 are diagrams illustrating an example of blur determination in the present embodiment.

As shown in FIG. 11, in the present embodiment, the state where the blur of the document image is equal to or less than the reference value (not blurred) is a state where the document image is not blurred and the visibility of the characters is good.

On the other hand, as shown in FIG. 12, in the present embodiment, the state in which the blur of the document image is larger than the reference value (blurred) is a state in which the document image is blurred and the character visibility is poor. Retry is required.

Referring back to FIG. 2, the target area setting unit 102e sets a partial area in which blur less than a predetermined reference value is detected as a target area for identifying the orientation of the document image (step SA-9).

For example, the target area setting unit 102e may detect blur in each partial area in order, and set the partial area as the target area when the blur is confirmed to be equal to or less than a reference value (not blurred). .

Further, the target area setting unit 102e may compare the blurs of the partial areas and set the least blurred area as the target area.

Then, the orientation specifying unit 102g specifies the orientation of the content in the target area, and specifies the orientation of the document image based on the orientation of the content (step SA-10).

For example, the orientation specifying unit 102g performs a labeling process on the character area data of the character area that is the content included in the target area, and compares the character area data with the dictionary data stored in the dictionary data file 106a to determine the content direction. May be specified.

Then, the orientation correcting unit 102h acquires post-correction image data of the original image that has been corrected upright based on the orientation of the original image (step SA-11).

Then, the orientation correction unit 102h saves (stores) the corrected image data in the image data file 106c (step SA-12), and ends the process.

Further, the image display unit 102i may cause the user to confirm the orientation-corrected document image by displaying the corrected image data of the document image acquired by the orientation correction unit 102h on the input / output unit 112.

Here, with reference to FIG. 13, an example of an outline of the orientation correction processing in the present embodiment will be described. FIG. 13 is a schematic diagram illustrating an example of the orientation correction process in the present embodiment.

As shown in FIG. 13, in the present embodiment, a document is photographed (step SB-1), and a rectangle that becomes a document image is extracted from the photographed image (step SB-2).

At this point, in the present embodiment, the document image data of the clipped document image is displayed and the user is confirmed. In the present embodiment, document image data of a document image that has undergone projective conversion may be displayed.

In this embodiment, after the original image extracted in a rectangle is displayed to the user, if the photographed original is a general document, the original image is divided into 2 × 2 parts (step SB-3). .

In the present embodiment, each partial region is detected as blurred (step SB-4), and when a partial region with a certain blur or less is detected, the partial region is used to perform direction correction, thereby correcting the direction. The subsequent document image is saved (step SB-5), and the process is terminated.

Here, if the partial area is blank or there are few candidates for the number of characters, in this embodiment, the partial area may not be subjected to blur detection, and the blur determination may be performed in another partial area.

On the other hand, in this embodiment, the blur of each partial area is detected (step SB-4), and the blur of all four partial areas is larger than the reference value (blur) (step SB-6). In order to redo (retry) the photographing itself, the process proceeds to step SB-1 (step SB-7).

As described above, in the present embodiment, the original image included in the image is detected, the original image is divided, the blur of each divided area is detected, and the area with less blur is determined as the target area for orientation correction. Then, the orientation correction of the document image may be performed from the orientation correction target area.

In the present embodiment, the form is determined based on the feature amount of the original image, and in the case of a specific form (such as a driver's license or health insurance card), orientation correction processing specialized for the form type and post-processing The image data may be stored.

In recent years, with the widespread use of mobile terminals such as smartphones and tablets, the work that has been used in the past has been changed to work that uses a mobile terminal having a camera.

This is because scanning by a camera has an advantage that the degree of freedom is high because it does not matter where it is or there is no restriction on the medium.

On the other hand, the camera image has a problem that it is difficult to obtain an image equivalent to the image quality of the scanner because the surrounding light quantity, shooting direction, and environment such as motion during shooting are not stable.

Furthermore, in the conventional orientation correction process, the correct orientation of the document is determined by recognizing characters at the top of the document image or at random positions.

However, when shooting a document with a general camera, the document image may be shot from multiple directions including oblique directions, resulting in a blurred area in the document image, causing a reduction in the accuracy of orientation correction processing. .

Therefore, in the present embodiment, the image processing performed with the scanner image quality can also be applied to the mobile camera image quality.

[Other Embodiments]
The embodiments of the present invention have been described so far, but the present invention may be implemented in various different embodiments other than the above-described embodiments within the scope of the technical idea described in the claims. Is.

For example, the mobile terminal 100 may perform processing in a stand-alone form, performs processing in response to a request from a client terminal (which is a separate housing from the mobile terminal 100), and outputs the processing result to the client terminal You may make it return to.

In addition, among the processes described in the embodiment, all or a part of the processes described as being automatically performed can be manually performed, or all of the processes described as being manually performed can be performed. Alternatively, a part can be automatically performed by a known method.

In addition, the processing procedure, control procedure, specific name, information including parameters such as registration data or search conditions for each processing, screen examples, or database configuration shown in the description and drawings are specially noted. It can be changed arbitrarily except for.

Further, regarding the mobile terminal 100, each illustrated component is functionally conceptual and does not necessarily need to be physically configured as illustrated.

For example, all or some of the processing functions provided in each device of the mobile terminal 100, particularly the processing functions performed by the control unit 102, are realized by the CPU and a program interpreted and executed by the CPU. Alternatively, it may be realized as hardware by wired logic.

Note that the program is recorded on a non-transitory computer-readable recording medium including a programmed instruction for causing a computer to execute the method according to the present invention, which will be described later, and the mobile terminal 100 as necessary. Read mechanically. That is, in the storage unit 106 such as a ROM or an HDD, computer programs for performing various processes by giving instructions to the CPU in cooperation with an OS (Operating System) are recorded. This computer program is executed by being loaded into the RAM, and constitutes a control unit in cooperation with the CPU.

The computer program may be stored in an application program server connected to the mobile terminal 100 via an arbitrary network, and may be downloaded in whole or in part as necessary. .

Further, the program according to the present invention may be stored in a computer-readable recording medium, or may be configured as a program product. Here, the “recording medium” includes a memory card, USB memory, SD card, flexible disk, magneto-optical disk, ROM, EPROM, EEPROM, CD-ROM, DVD, Blu-ray (registered trademark) Disc, etc. Including any “portable physical medium”.

In addition, “program” is a data processing method described in an arbitrary language or description method, and may be in any form such as source code or binary code. Note that the “program” is not necessarily limited to a single configuration, and functions are achieved in cooperation with a separate configuration such as a plurality of modules and libraries or a separate program represented by the OS. Including things. In addition, a well-known structure and procedure can be used about the specific structure for reading a recording medium in each apparatus shown in embodiment, a reading procedure, or the installation procedure after reading.

Various databases and the like stored in the storage unit 106 are storage means such as a memory device such as a RAM or a ROM, a fixed disk device such as a hard disk, a flexible disk, and / or an optical disk. Various programs, tables, databases, and / or web page files used may be stored.

The mobile terminal 100 may be configured as an information processing apparatus such as a known personal computer, or may be configured by connecting an arbitrary peripheral device to the information processing apparatus. The mobile terminal 100 may be realized by installing software (including programs, data, and the like) that causes the information processing apparatus to implement the method of the present invention.

Furthermore, the specific form of distribution / integration of the devices is not limited to that shown in the figure, and all or a part of them may be functional or physical in arbitrary units according to various additions or according to functional loads. Can be distributed and integrated. That is, the above-described embodiments may be arbitrarily combined and may be selectively implemented.

As described above, the mobile terminal, the image processing method, and the program can be implemented in many industrial fields, particularly in the image processing field that handles images read by a camera, and are extremely useful.

DESCRIPTION OF SYMBOLS 100 Mobile terminal 102 Control part 102a Image acquisition part 102b Original specification part 102c Partial area acquisition part 102d Blur detection part 102e Target area setting part 102f Form determination part 102g Orientation specification part 102h Direction correction part 102i Image display part 106 Storage part 106a Dictionary data File 106b Form data file 106c Image data file 110 Imaging unit 112 Input / output unit 114 Sensor unit 116 Communication unit

Claims

Image acquisition means for acquiring captured image data of a captured image captured by the imaging unit;
Document specifying means for specifying a document image included in the photographed image;
Partial area acquisition means for acquiring partial area image data of a partial area in the original image;
Blur detection means for detecting blur in the partial area;
Target area setting means for setting the partial area as a target area for identifying the orientation of the document image based on the blur;
Orientation specifying means for specifying the orientation of the content in the target area and specifying the orientation of the document image based on the orientation of the content;
Orientation correcting means for acquiring post-correction image data of the original image corrected upright based on the orientation of the original image;
A mobile terminal comprising:
Form data storage means for storing characteristic data of specific forms and layout data;
A form determination means for determining whether the document image corresponds to the specific form based on the feature data;
Further comprising
The orientation specifying means includes
2. The mobile terminal according to claim 1, wherein when the document determination unit determines that the document image corresponds to the specific document, the orientation of the document image is specified based on the layout data.
Dictionary data storage means for storing dictionary data;
Further comprising
The orientation specifying means includes
A character area indicating a character in the target area is identified by a labeling process on the target area image data in the target area, and the character in the character area is compared based on a comparison between the character area data in the character area and the dictionary data. The mobile terminal according to claim 1, wherein the orientation of the document image is identified, and the orientation of the document image is identified based on the orientation of the character.
The partial area acquisition means includes
The mobile terminal according to claim 1, wherein the partial area image data of the partial area obtained by dividing the document image is acquired.
Dictionary data storage means for storing dictionary data;
Further comprising
The partial area acquisition means includes
The partial area image data of the partial area indicating characters in the original image is obtained by labeling the original image data of the original image,
The orientation specifying means includes
2. The direction of the character in the target area is specified based on a comparison between the target area data of the target area and the dictionary data, and the direction of the document image is specified based on the direction of the character. Or the mobile terminal of 2.
The target area setting means includes
When the blur detection unit detects the blur below the predetermined reference value, the partial area where the blur below the predetermined reference value is detected is set as the target region for identifying the orientation of the document image. The mobile terminal according to any one of claims 1 to 5.
The target area setting means includes
6. The blur detected by the blur detection unit is compared, and the partial area that is least blurred is set as the target area for identifying the orientation of the document image. Mobile devices.
The image acquisition means includes
The mobile terminal according to any one of claims 1 to 5, wherein when the blur detection unit detects no blur that is equal to or less than a predetermined reference value, the captured image data is acquired by the imaging unit again. .
An image acquisition step of acquiring captured image data of a captured image captured by the imaging unit;
A document specifying step for specifying a document image included in the photographed image;
A partial area acquisition step of acquiring partial area image data of a partial area in the document image;
A blur detection step for detecting blur in the partial area;
A target area setting step for setting the partial area as a target area for identifying the orientation of the document image based on the blur;
A direction specifying step of specifying a direction of content in the target area and specifying a direction of the document image based on the direction of the content;
A direction correction step for obtaining post-correction image data of the original image corrected upright based on the direction of the original image;
An image processing method comprising:
A form determination step for determining whether or not the original image corresponds to the specific form based on the stored characteristic data of the specific form;
Further including
In the orientation specifying step,
Furthermore, when it is determined in the form determination step that the document image corresponds to the specific form, the orientation of the document image is specified based on the stored layout data of the specific form. The image processing method as described.
In the orientation specifying step,
The character area indicating the character in the target area is identified by the labeling process for the target area image data of the target area, and based on the comparison between the character area data of the character area and the stored dictionary data, The image processing method according to claim 9 or 10, wherein a direction of the character is specified, and a direction of the document image is specified based on the direction of the character.
In the partial region acquisition step,
The image processing method according to claim 9, wherein the partial area image data of the partial area obtained by dividing the document image is acquired.
In the partial region acquisition step,
The partial area image data of the partial area indicating characters in the original image is obtained by labeling the original image data of the original image,
In the orientation specifying step,
The direction of the character in the target area is specified based on a comparison between the target area data of the target area and stored dictionary data, and the direction of the document image is specified based on the direction of the character. Item 11. The image processing method according to Item 9 or 10.
In the target area setting step,
The partial area where the blur below the predetermined reference value is detected as the target area for identifying the orientation of the document image when the blur below the predetermined reference value is detected in the blur detection step. The image processing method according to claim 9, wherein the image processing method is set.
In the target area setting step,
The blur detected in the blur detection step is compared, and the partial area that is least blurred is set as the target area for identifying the orientation of the document image. The image processing method as described.
In the image acquisition step,
The image according to any one of claims 9 to 13, wherein when the blur that is equal to or less than a predetermined reference value is not detected in the blur detection step, captured image data obtained by re-shooting by the shooting unit is acquired. Processing method.
An image acquisition step of acquiring captured image data of a captured image captured by the imaging unit;
A document specifying step for specifying a document image included in the photographed image;
A partial area acquisition step of acquiring partial area image data of a partial area in the document image;
A blur detection step for detecting blur in the partial area;
A target area setting step for setting the partial area as a target area for identifying the orientation of the document image based on the blur;
A direction specifying step of specifying a direction of content in the target area and specifying a direction of the document image based on the direction of the content;
A direction correction step for obtaining post-correction image data of the original image corrected upright based on the direction of the original image;
A program that causes a computer to execute.